Creating dataplugin to read alternating channels and data

[Greg_Gran]

I have a csv file for which I'm trying to create a dataplugin. The Wizard helpfile says that if I specify a line as data, then all subsequent lines must also be data.

The first line in my file is a header of test parameter names. I'd like these to be channel names for the actual data (the parameter values) that follow on the next line.

I then have a blank line, and then a line with my main data channel names. The subsequent lines are then the actual main data.

I'm pretty sure I need a VBScript to do this, but am having a little trouble getting something that works based upon examples I've found on the board.

I want to bring in the header data into the data portal so that I can place some of it alongside charts in my reports.

I have attached a sample data file. Also, I'd typically like to ignore some of the data columns (in this case, columns 3 thru 10)

Thanks for any suggestions!

[Greg_Gran]

I also tried to read in the header lines as group values (name and description), but was unable to figure out how to get it to read in more than one column of header data.

[DRock]

Hi Greg_Gran -

For your reference, the first two lines in your file are, in fact, property names and corresponding values - not channel names and data.

This can be easily accomplished with a custom DataPlugin, but is starting to push the boundaries of what the Wizard can do as-is.  I'd be happy to write a custom DataPlugin for the example file you gave me, given the parameters in your first post.  

If you give me a few hours here in between meetings, I'll get that knocked out for you and post back.

[DRock]

Hi Greg_Gran -

By "...I'd typically like to ignore some of the data columns (in this case, columns 3 thru 10)," do you mean you'd like to avoid loading the columns (and data) labeled:

Raw Ang [deg]

Corr X [m]

Spread Loss [dB]

...

IXmtPhs [deg]

Or, are you attempting to ignore the third through tenth properties in the first two lines?

I've currently completed your DataPlugin that loads the entire file, but if you don't want to load some of the information, I want to incorporate that before posting.

[Greg_Gran]

I don't want to throw out anything in the first two lines. Just some of the data channels...

Raw Ang [deg]

Corr X [m]

Spread Loss [dB]

...

IXmtPhs [deg]

Thanks!

[DRock · Accepted solution]

Hi Greg_Gran -

I've attached a custom DataPlugin that will load files structured according to your posted example and will ignore the third through tenth data channels.

To use the DataPlugin, simply unzip the attached *.zip folder to your hard drive.  Double-click the included *.uri file to register the DataPlugin with your computer.  Once you get the successful registration dialog, you'll be able to load your data files.  Because CSV is a generic file format with many variations, you may need to boost the priority of this particular DataPlugin for the *.CSV extension through DIAdem's NAVIGATOR » Settings » My DataFinder » Configure » File Extensions menu so that this DataPlugin is always used instead of other *.CSV DataPlugins.

I wasn't sure whether you wanted to ignore the third through tenth data columns all or some of the time, so I built a small hook into the DataPlugin that will allow you to adjust the "range" of channels ignored (e.g. 3-10, 1-5, 21-30).  To modify the range that the DataPlugin ignores when loading: 

1. Select DIAdem NAVIGATOR.
2. Select Settings » Options » Extensions » DataPlugins.
3. Right-click the Greg_Gran_CSV DataPlugin and select Edit Script.
4. In Line 5 of the DataPlugin, the "Ignore" variable is defined as a one-dimensional array of two values (3 and 10).  These are the upper and lower values defining the range of data channels to ignore.  To load all of the data, change either (or both) value to -1.
5. Save the DataPlugin.

Since I only had one file to test with, I can't guarantee that there won't be problems with your other files - let me know if you run into trouble or if you have any further questions.

[Greg_Gran]

Derrick,

Thank you very much, this does exactly what I needed. I did some VB programming about 10 years ago, but I have forgotten most of it. Studying this plugin will no doubt help me to do things on my own in the future.

I also found the NI DataPlugin.chm windows help file yesterday afternoon, and that explains all of the concepts that struggling with.

Best Regards,

Greg Granville

[DRock]

Hey Greg -

I unfortunately didn't put much effort into making the code as "readable" as possible  Therefore, please let me know if there's anything that I did in the DataPlugin code that you can't figure out by poking around the help documentation.

There are a few things I did in the DataPlugin -

1. Parse out the property label / value pairs from the first two lines.  
1. Clean up the names of the properties (e.g. spaces are not supported in proprty names) and convert the numeric properties to actual numbers.
2. Skip the third line.
3. Parse out the data channel names from the fourth line.
1. Extract the units from the channel names and store them to the "unit_string" property of each corresponding channel.
4. Define a block of raw data to read (the rest of the file), but only add the channels to DIAdem if they're not "skipped."

Let us know if we can help when you go to get your reports set up and automated.