binary file data plugin : navigating through different data types

[usac]

Hi Phex,

regarding your first question, you can set a NoValue sign either on file or channel level.

File.Formatter.NoValueSign = "-32768"
'or
Channel.Formatter.NoValueSign = "-32768"

Using one of the settings above should convert this value into a NoValue without the need of a post-processing script.

[Phex]

Hi usac and thanks for your advice.

Unfortunately the first option does not have any effect and the second option is not available for binary channels.

/Phex

[Phex]

Hi

Regarding the question about the channel number limitation I got the idea to connect the larger files as a DataStore. That actually works quite well and I am able to browse the data properties. What I could not achieve yet was to load a certain group into the DataPortal for further evaluation. The Navigator.LoadData()-command requires a loading configuration. Unfortunately the Help pages are not very helpful on this topic. Can anyone please describe how such a loading configuration should look like? 

/Phex

[Phex]

Another much simpler idea was to use the DataFileLoadSel-command in combination with the specific DataPlugin, but unfortunately it takes ages to load a single channel group into the data portal. Is there hope that it will go quicker with the Navigator.LoadData-command? Or is there another way?

/Phex

[Phex]

No one with an idea?

If not, can anyone please explain why DIAdem is loading the whole file into the working memory even if I just want to extract a single root property?

What I do is:

Set oMyDataStore = Navigator.ConnectDataStoreByParameter("rp_DataPlugin", "<filename readonly=""yes"">"& replace(sRPFile, "&", "&amp;") & "</filename>")
MyGroupCount = oMyDataStore.RootElements(1).Properties("ProfileCount").Value
Call oMyDataStore.Close()

As an example, if the size of the sRPFile is about 1.5 GB, DIAdem would load ~ 10 GB into the working memory before it actually extracts the root property. Same thing is happening when using the DataFileLoadSel()-command. So just connecting the DataStore requires memory more than six times the original binary file size which sooner or later freezes my workstation...

What is happening in the backgroud and is there a way to avoid that? 

/Phex

[Brad_Turpin]

Hi Phex,

First of all the "File.Formatter.NoValueSign" property is only for ASCII Blocks, so that explains why it's not helping you with your Binary Blocks.  To my knowledge there is no easy option to convert a certain bitset to NaN inside the DataPlugin.

The performance of Root property reads will depend strongly on how you program your DataPlugin.  If you only have DirectAccessChannels in your DataPlugin, this will be very fast.  In all cases, though, I'd recommend that you index the data file(s) in the DataFinder, then query that file and read the properties from the searrch results-- that will talk to the DataFinder database and not bother the file on disk.

Brad Turpin

DIAdem Product Support Engineer

National Instrumentts

[Phex]

Thanks for your reply, Brad.

My final solution is a DataPlugin that loads all measurement quantities into just six channels. The outcome will be saved as TDM and for evaluating the data I assigned Workers that extract the respective sections from the six channels with the help of DataFileLoadRed.

Thanks for all contributions. Thread closed.

/Phex