DIAdem

cancel
Showing results for 
Search instead for 
Did you mean: 

Plugin Script For CSV file with mixed data types

Solved!
Go to solution

I need to create a data plugin for the example csv format attached. It only has one one line with a time stamp, then the next line is channel names and then data below. All the columns are float numbers except for a few that are hex (written as 0xAB). I am new to writing my own data plugins so I'm a little lost as how to sort through each row of data and specify a different data type for just the hex columns? I was going to import them as strings and then manipulate them as needed. Hex could be converted to decimal as well. I just need to get it all in the data portal. The actual data files have much larger row lengths so the Plugin Wizard doesn't quite get the job done. 

0 Kudos
Message 1 of 5
(886 Views)

If you do not need the start time you can import it using

"Textfile - Auto detect (*.csv)"

which refers to the "CSV" plugin.

0 Kudos
Message 2 of 5
(876 Views)

My Biggest problem with the auto-detect is it treats the 0xAB data as a string for the first 50 columns, but after that the auto detect treat all columns as numerical data. So I need to make my own plugin anyways. 

0 Kudos
Message 3 of 5
(873 Views)
Solution
Accepted by topic author jdferdon
09-15-2017 07:05 AM

The following skipt might be a start point. It is still reading the hex values as string by searching for it.

It tries to make sure that it is only used for this format to avoid loading of other CSV files.

 

Option Explicit

Sub ReadStore(File)
  File.Formatter.Delimiters = ","
  File.Formatter.DecimalPoint = "."
  File.Formatter.TimeFormat = "MM/DD/YYYY hh:mm"
  
  File.Formatter.LineFeeds = vbNewLine

  dim cT : set cT = file.GetNextStringValue(eTime)
  if cT is nothing then
    RaiseError ' Not my format
  end if
  if cT.Year < 1970 OR cT.Year > 5000 then
    RaiseError ' Not my format
  end if  
  Root.Properties("datetime").Value = cT
  file.SkipLine

  dim captions : captions = split(file.GetNextLine, ",")
  dim pos : pos = File.Position ' remember position of line to get block there
  dim firstLine : firstLine = split(file.GetNextLine, ",")
  File.Position = pos ' set position back
  
  if UBound(captions) <> UBound(firstLine) OR -1 = UBound(captions) then
    RaiseError ' Not my format
  end if

  Dim ChannelGroup : Set ChannelGroup = Root.ChannelGroups.Add(File.Info.FileName)
  Dim Block : Set Block = File.GetStringBlock()
  dim i : for i = 0 to UBound(captions)
    if 0 = instr(firstLine(i), "0x") then
      ChannelGroup.Channels.AddDirectAccessChannel Block.Channels.Add(captions(i), eR64)
    else
      ChannelGroup.Channels.AddDirectAccessChannel Block.Channels.Add(captions(i), eString)
    end if    
  Next
End Sub

It is used for your file.

2/3/2017 15:06,,,,,,,,,,,,,,,,,,,,
Time[sec],A,B,C,D,E,error,F,G,H,I,J,K,L,M,N,error,P,Q,R,S
1,0.18,1.01,0.18,1.01,32,0x0,0,1,0,4,65535,65535,65535,65535,0,0x0,0,1,0,0
2,0.19,1.07,0.19,1.07,32,0x0,0,1,0,4,65535,65535,65535,65535,0,0x1,0,1,0,0
3,0.23,1.3,0.23,1.3,32,0x0,0,1,0,4,65535,65535,65535,65535,0,0xF,0,1,0,0
0 Kudos
Message 4 of 5
(841 Views)
Highlighted

Used the above script as start point and as usual the problem has become more complicated, so continuing to develop new Data Plugin but its a work in progress. 

 

-J

0 Kudos
Message 5 of 5
(754 Views)