max record NI-5124

[Naturalarch]

With our measurement system we use a NI-5124 oscilloscope card with 256MB memory.

can anybody tell me what the maximum record size is that i can put to the memory of the card for a single channel measurement?

best regards and happy new year

Joost


I made a program to save data to disk, but this already gives proplems with a record size bigger than 2,5 MS, this seems a bit low.... see attachement

Message Edited by Naturalarch on 01-03-2006 10:10 AM

[DFGray]

For a single channel, the maximum record size is going to be a bit under 128 million (2bytes/sample and there is a bit of overhead). However, your program will not handle that. I estimate you are making between 8 and 10 copies of your data once you fetch it from the 5124. In addition, the data is being fetched as scaled doubles, which increases its size a factor of 8. So you are getting a data multiplication factor of somewhere between 64 and 80. Depending on the amount of RAM your computer has, this could definitely cause problems.

Not to worry, however, you just need a few simple changes. First, fetch your data as I16 instead of double. This will give you an automatic reduction in size of a factor of eight. You will need to scale and offset the results, but that is not an issue. Do this scaling when you need to. Next, use the NI-SCOPE measurement functions to compute the amplitude and frequency of the signal. They are far more efficient than the Tone Measurements Express VI in terms of copies, and probably speed as well.

Finally, decimate your data before you display it. The graph makes at least one extra copy, and usually more. Since your display is on the order of 1000 pixels wide, 2.5 million points is a waste of RAM. You can get decimation code from the tutorial Managing Large Data Sets in LabVIEW. The tutorial will also tell you how to avoid data copies and how to find them in the first place. The techniques in the tutorial are used in the NI-SCOPE Soft Front Panel, which is written in LabVIEW.

Note that dealing with large data sets in LabVIEW is a fairly advanced topic, and your block diagrams will start to get larger. However, with a bit of care, it can be done. Good luck. Let us know if you need more help.

[Naturalarch]

I still did not thank you for you're help, so i'll do it now and right away ask an other question.

I made some of the changes you suggested, this did help a lot, i got no problems saving the data now (i use data storage). In the file I store a new burst in a different channel.

But now i got the problem on the other site, when i try to read 1 burst and perform some demodulation on it (make baseband IQ and do PSK demod) it says it runs out of memory. could you give me some tips on how to proceed in the reading of large data files and perform measurements on them?

Attached is the updated file for saving the data..

the second file tries to determine the pulse width, i am aware that this is probably not the best approach and i am searching for a better way, but it is to demonstrate the fact what sort of measurements i want to perform besides decoding the message (PSK modulated pulse)

Best regards

Joost van Heijenoort
Ursa Minor space and navigation

[DFGray]

My apologies for the slow reply.  My current project is in crunch time, so I don't have a lot of spare time.

The first VI you posted is missing some subVIs, so I can't see what you are actually doing.  The second one would not open for me, so is probably corrupt.

Despite that, I will give you some general advice.

1. For large waveform type files, NI-HWS is far more efficient than the data storage VIs, and not much harder to use.  I would recommend you use HWS instead of the storage VIs.  HWS was installed when you installed NI-SCOPE (it is the native file format of the NI-SCOPE Soft Front Panel).
2. I am very ignorant of the algorithms you mention.  However, with large data, you need to read it from the file, process it, and write it back to a (new?) file in pieces small enough to avoid memory problems.  I believe the demodulations you mention can be done this way.  HWS is fast enough that disk access should not be a huge issue.

Create a development distribution of the VIs below, zip it up, and repost it and I will try to find to some time to take a look at it.  Good luck.  Keep me posted with your progress.

[Naturalarch]

Maybe this is a stupid question. But how can i save it as a development distribution.

regards

Joost

[DFGray]

My apologies.  I should have posted that as well.  For LabVIEW 7.1 and before, select File->Save With Options... from the menu of the VI you want to save.  When the dialog comes  up, click the Development Distribution radio button and make sure that To new location - single prompt is selected in the Selection options (that is the default).  You will be asked for a location for the resulting LLB.  Select one and go.  The resulting LLB will have all your VIs.

Due to the more flexible build system included in LV8, things are a bit different (note that I am using the Pro version - things may be different in other versions).  Open your VI and select File->Save As... from its menu.  Select Duplicate hierarchy to new location from the dialog, then hit the Continue... button.  If the VI is not part of a project, you will be prompted to create one at this point.  Enter a name for the project, if necessary.  You will then get the build dialog.  Call your build specification anything you want.  Set the Packaging Option to Single Destination so that it will go into a single LLB.  You probably want to uncheck Exclude user.lib as well.  Click the Build button and let it run.  Note that it may take awhile if your hierarchy is at all complex, since the validation is more strict under LV8 than under previous versions.

[Naturalarch]

Oke thanks,

I think it works...nice feature if you want to back-up you're project (is it possible to do with the complete project, not just one file)...
 
I made 2 versions, one with user libs unchecked and one with all unchecked..

best regards

Joost

[DFGray]

Please post the analysis code that is causing you memory problems.  The save code could use some work, but it nothing that will cause you massive memory issues.  I will try to find some time to look at it.

[Naturalarch]

The Computer crashes with this memory error the moment i try to read one Burst at the time. This is still 80 MB then. And if i want to process or display that the system crashes.
If i then look at the memory use of the computer, the memory use exceeds 1,2 GB.

I am now implementing reading it in smaller peaces (with the seek function in HWS file stuff). But i am not sure whether i can use this with  demodulation stuff ....

Is this okay?

Best regards

Joost

[DFGray]

Since I don't have your code, I am guessing, but this is probably what is happening.

1. Your read 80MBytes of data for a burst (I16 format).
2. Convert to DBL format - your data is now 320MBytes
3. Pass to analysis routine - crash due to memory violation when analysis makes four copies of it
   

The major issue here is data copies.  Four copies of your data will produce over 1.2GBytes.  The theoretical maximum amount of data you can have in memory at once is 1.5GBytes for Windows (see previously referenced tutorial on large data use for why).  However, LabVIEW itself both fragments and consumes memory, so you cannot get to 1.5GBytes.  You will need to process your data in chunks from the original file.  I am pretty sure this can be done with the algorithms you need.  I will point someone a bit more knowledgeable than myself about these algorithms to this thread.