binary file data plugin : navigating through different data types

[Phex]

Hi

I am currently trying to load a binary file into DIAdem with the following structure:

block 1

1x 8-bit ascii

12x double

block 2

1280x int16

block 3

1280x int16

block 4

12x double

block 5

1280x int16

block 6

1280x int16

block 7

12x double

block 8

1280x int16

block 9

1280x int16

...

I managed to read the first string value, but I was unsuccessful in getting any further.

Could anyone please give me a lead on how to proceed?

/Phex

[robertpf]

Hi,

You can request a custom plugin to be made for you, follow the link below for more information:

Requesting a DataPlugin

http://www.ni.com/example/14564/en/

I also add some links with general information on the current existing plugins and some additional resources for plugin development:

DataPlugins: Supported Data Formats
http://www.ni.com/product-documentation/4065/en/

Resources for Writing Your Own DataPlugin
http://www.ni.com/product-documentation/13803/en/

Best regards,

[AndreasH_Ha]

Hello Phex,

Just reading a double value is easy and would go like this

For K = 1 To 12

 FloatValue = oFile.GetNextBinaryValue(eR64)

Next

Once you read the data, you probably want to assign the data to channels. I can try to help with this but would need some more details about the logical structure

e.g. how the different chunks in the binary file should be moved/copied to a channel ?

Also, should each chunk go to a different channel or should the first of the I16 blocks always go to the same channel ?

Do you happen to know whether the byte order in the file is BigEndian or SmallEndian ? etc.

How many of the blocks you described are contained in the file. Most binary files have something like a header containing this type of information

If you want some more help, please provide at least one example file

Andreas

[Phex]

Hello Andreas and thank you for your reply.

I already tried the method you described but unfortunately I get very strange values that do not match at all with the expected ones.

I enclosed a zip-file containing an example file together with an explanation of the binary file structure and an example output file showing the first blocks of the file in ASCII-format (note that values in <> are just for informational purpose but not physically existent in the file). Also you will find a vbs-file with my poor attempts so far.

What I basically want to achieve is:

• save the software version as a root property >> 18 byte string value at beginning of file
• create a channel group for each set of (two) int16 blocks
• save the values of each header block as respective group properties >> blocks with 12 double values
• rename the channel group according to the frameNr >> 5^th value of header block
• save the two int16 blocks as channels to the group >> two blocks each 1280 int16 values

/Phex

[AndreasH_Ha · Accepted solution]

Hello Phex,

try this. Not sure whether its exactly what you are looking for, but probably close.

Andreas

[Phex]

You are awesome Andreas! Thank you so much. I need to do some further splitting and formatting on the signal channels but I guess I can take it there myself with the help of your well-documented script. Thanks again, much appreciated.

/Phex

[AndreasH_Ha]

Sure, no problem. I know that writing a Plugin for a binary file is totally non obvious when you do it the first time. Once you understand which functions to use, its typically very efficient.

If you find anything missing, just let me know

Andreas

[AndreasH_Ha]

One thing I forgot : Is this a format which is likely used by others too ? Is this output from a Latronix Laser Scanner ?

If its something publicly available we could make the plugin available from ni.com/dataplugins and R&D would take over long term maintenance.

That's just an option. Let me know if you think this would make sense

Andreas

[Phex]

The current file format has been specifically designed for a product that is until now unique, so I don't see any benefit to add it to DIAdems Plugin portfolio at this point. In case the circumstances change in the future I would gladly assist to establish a contact between Latronix and NI.

/Phex

[Phex]

Sorry for revoking this thread, but I need some further advice on two issues:

1. All values that are set to -32768 in the data mean actually NaN. Right now I am converting all those values to NaN in a post-processing script using the CTNV function. Is there a way to do this directly in the DataPlugin? My idea was to apply the NoValueSet command but the bInterfaceLocked variable returned TRUE, so not accessible. No problem if there is no other way, I can live with the CTNV function.

2. When I tested the DataPlugin with larger files (up to 1 GB) I pretty quickly ran into the maximum channel number limitation of DIAdem (available 65k, needed 400k). The only thing I could think of right now is to concatenate all data to a few channels and then extract the relevant blocks needed for the analysis during post processing. But that does not seem very efficient and will surely bring up other problems. Is there a way to work with the DataPlugin interactively, i.e. tell the plugin to read X number of blocks at file position Y? Or is it possible to save loaded data sequentially to different files within the plugin? Increase the number of available channels? Any comments or other ideas are very much welcome.

/Phex