Some errors when calling LabVIEW VIs Interactively from DIAdem

[Roger Isaksson]

Jimmie and Brad, I still can not understand why diadem uses about 200MB for exporting 20MB data? I attach the vi for your reference.

Could you have a look at this? About 10 times the size of the exported data is quite much? The windows task manager reports the following numbers:

Before the export: ~507MB
Peak value during the export: ~700MB
After the export: ~571MB

Regards

Roger Isaksson

[Brad_Turpin]

Hi Vishal,

There is absolutely no connection between the code of a DataPlugin and the DIAdem environment-- which has to be the case when you consider that the LabVIEW Storage VIs can use a DataPlugin to load a data file on a computer which does not have DIAdem installed.  So invoking DIAdem menu methods or calling any other DIAdem functionality from within a DataPlugin is out of the question.

There is no way that I know of with LabVIEW 7.1 and DIAdem 9.1 to call LabVIEW code with a DataPlugin.  For those product versions I think you're stuck calling the LabVIEW code from a VBScript/menu.

DIAdem 10 went Beta last week, and it will be possible (though it will not be documented yet) with DIAdem 10 to register a G-based DataPlugin.  This would be the best option for you, but I don't know if your implementation time frame will accomodate a Beta option.

Regards,
Brad Turpin
DIAdem Product Support Engineer
National Instruments

[Roger Isaksson]

Brad,

Have you noticed the two posts I submitted to page 2?

Regards

Roger Isaksson, Damill AB

[Jimmie_A.]

Hello Roger and Brad!

This problem is unfortunately something I cannot provide any insight on due to the problem’s complexity.

Brad, I am not sure if you noticed Rogers post where he also posted an example VI that shows the behavior he is experiencing. Can you please look at it and see if any changes can be made? Roger showed me the memory allocation when I visited them a couple ago. Even transferring reasonably small amounts of data consumes a huge amount of memory.

Thanks in advance.

Regards,

Jimmie A.

Applications Engineer, National Instruments

[Brad_Turpin]

Hi Roger,

I'm sorry I missed your 2 most recent posts until now.  That thread got intertwined with one or two other customer requests, and I just lost track of your stuff.  My bad.

Your 2 VIs are doing 2 radically different things.  I do NOT recommend stuffing huge amounts of data through the ActiveX communication layer, which is what you're doing in "TestDiaDem-2.vi".

Your first VI, "TestDiaDem.vi", looks pretty good, and represents well what I am suggesting, with one potential caveot.  You do not necessarily need to save your data as DBLs.  For large data sets like these, if your acquisition process returns data to only 16 bit resolution or less, then you would ideally stream the I16 data as well as the scaling factors instead of DBLs.  That will reduce your file size by a factor of 4.  I'm including a large LLB below which shows how to do this with DAT files, though it is more complicated than the approach you took.

In all cases I am advocating that you save the LV data to a DAT file, then load the DAT file into DIAdem.  If you need to drive this from LV, as in your "TestDiaDem-2.vi", then I recommend using slightly different LV-DIAdem Connectivity VIs to instruct DIAdem to load the DAT file, rather than stuffing the whole array through the ActiveX communication line.

Let me know as you have further questions,
Brad Turpin
DIAdem Product Support Engineer
National Instruments

[Roger Isaksson]

Hi Brad, thanks for the help!

I think the tdmhw.llb is needed to open the tdm examples? The dat vi's are working.

Edit: The data is streamed via ethernet to a console computer that stores data into the suggested formats, the daqmx task is not available.

Regards

Roger Isaksson, Damill AB

Message Edited by Roger Isaksson on 10-14-2005 10:07 AM

[Brad_Turpin]

Right you are, Roger,

If you're interested in checking out the TDM file streaming approach, look at the following KB:

http://sine.ni.com/apps/we/niepd_web_display.display_epd4?p_guid=00AD0A77D3D5318CE0440003BA7CCD71&p_...

The KB has the DLL and example code, including the DAQmx example I sent you with the TDM streaming VIs that look almost identical to the DAT file streaming VIs for DAQmx.

Brad Turpin
DIAdem Product Support Engineer
National Instruments

[Roger Isaksson]

Brad, thanks for the help!

I have evaluated the "daqmx stream to dat file.vi" and my conclusion is the same as before. For example, I store data sampled at 50.000Hz from 16 channels and after about 2 minutes of aquisition I reach a file size of 200MB. Now, to load the 200MB of data into diadem takes about 4 minutes! Is it supposed to take that long?

Regards

Roger Isaksson, Damill AB

Edit: Typo..

Message Edited by Roger Isaksson on 10-20-2005 07:08 AM

[Brad_Turpin]

Hi Roger,

That depends on a lot of things.  If you have enough RAM in your computer to hold all the values you are loading (as DBLs) in active memory (and your operating system will give DIAdem that much RAM to play with, on top of what DIAdem needs itself to run), then it should load much faster than that.

But if for any of the above reasons you have to use virtual memory, then you're just rewriting the file from it's current location on the hard drive to a different location on your hard drive, and then accessing the values from that new location.  This is SLOW.

DIAdem has a way around this unnecessary rewriting which can be useful when you have no choice but to dip into virtual memory-- it's called registered loading.  If instead of dragging the DAT file into the NAVIGATOR you will right click on it and select "Register data", then the file will appear instantly in the Data Portal, though with greyed-out channel names and a funny arrow icon next to each channel.  Registered channels are fantastic for looking at individual values in a VIEW table, but graphing is another matter.  Registered channels usually graph about as slowly as channels loaded using virtual memory.

But, now that you have your data in a DAT format, you can make use of DIAdem data reduction and time windowing features, which only work for the DAT format-- you can find them in the online help under "paste from file".  Better yet, try out the below application which makes it easy to look into a huge data set (DAT file only) by interactively zooming into interesting portions of the data set at different magnifications (data reduction values) or looking at the whole data set with different data reduction settings.

http://zone.ni.com/devzone/conceptd.nsf/webmain/9363EB9540886FAC86257015007DDCDF

Let me know if you have additional questions,
Brad Turpin
DIAdem Product Support Engineer
National Instruments