How to load CSV file with no headers, spaces in data

[steveshort]

Hi,

I have CSV files with no header information in the file.  Data is arranged in columns in the file, Time, sample 1, sample2, sample 3, ... sample N

The number of samples is variable in each line and maybe zero.  I can't get the CSV wisard to read more than the first column of data (time points).

Any suggestions?

Thx,

Steve 

[Traderhans]

Your example includes only one column. It is correct.

[AndreasK]

The point here is that you have 140 empty lines which causes the wizard

to guess that there is only one column.

If you copy line 140 to line 1 use the wizard to create a plugin you will be able to

load files of this kind that do not  have more columns than line 140.

You can create a dataplugin programatically if you know more about your format.

[steveshort]

Hi,

Thanks for that, majes sense.  The problem is I do not know in advance how many lines will have one entry, or which line has the most columns.  Isn't there a way to look ahead and see which row has the most columns and then pad the other rows with the missing data?  I could accept NaN or even "0.0" for missing values. Excel easily does this when I load it in a spreadsheet.

Steve 

[AndreasK]

Dynamic adaption would need to read the complete file which is not done by the Wizard.

YOu can prepare a file with 100 hundred values in the first line. Use the wizard to generate a

plugin and you will always have 100 channels most of them empty.

But if the file contains more channels they will miss.

It is possible to write a dataplugin doing it programatically and create a new channel if needed

but it has to be done by hand in vbs and will be slow if you file become huge.

So there is no automatic generated solution using the Wizard.

[AndreasH_Ha]

Is there a maximum number of channels which can be found in a file ? . If so, you could always try to read this maximum number. If the file contains less columns, some of the channels woul donly have Novalues. It would be easy to identify those channels in DIAdem and remove them.

[AndreasK]

Option Explicit

Sub ReadStore(File)
  'Tell the file how the string data is formed.
  File.Formatter.Delimiters = ","
  File.Formatter.DecimalPoint = "."
  File.Formatter.LineFeeds = vbNewLine
 
  Dim ChannelGroup : Set ChannelGroup = Root.ChannelGroups.Add("Group")

  dim i, val, chObj, pos
  
  pos = 1
  do
    i = 1
    while 0 <> i
      val = File.GetNextStringValue(eR64)
      if isempty(val) then
        i = 0
      else
        if ChannelGroup.Channels.Count < i then
          set chObj = ChannelGroup.Channels.Add("ch_" & i, eR64)
        else
          set chObj = ChannelGroup.Channels(i)
        end if
        chObj.Values(pos) = val
        i = i + 1
      end if
    wend
    pos = pos + 1
  loop until (NOT File.SkipLine)

End Sub

Would be the code of the dataplugin used to load.

[steveshort]

OK,

I have no more than 22 columns in any file.  I will give these ideas a test and see if I can get it to work this weekend.