setting up windows for streaming

[Garcas]

I am looking for information on how to optimize windows for a streaming application

I am working in a DAQ application where I need to acquire 4 channels with 12 bits resolution at 10 MS/s during at least 40 seconds. I'm using a PXI system with a PXI-1044 chassis, a PXI-6115 DAQ board and a PXI-8105 controller running Windows. I am using LabVIEW 8.2

As expected, I am having some problems to maintain the acquisition going for 40 seconds consistently (sometimes I get to 40 seconds, sometimes it crashes before). Among other things I suspect Windows has something to do with this (I am also conscious that with a 80MB/s throughput I am pushing PXI bandwidth to the limit, but I cannot change this). Therefore I have stopped the antivirus and all programs that I can see in the task bar, but I would like to know if I can do anything else with Windows

Thanks and... happy new year !  😉

PS. I haven't got into detail with the acquisition technique used in the LabVIEW program in order to focus the thread in a single theme, but if anyone's interested I'd be glad to discuss it. I am using DAQmx and queues, and planning to play with software pipelining to see if I can get any advantage of the 8105's double core

 

[EdDickens]

How are you doing the acquisition and logging?

Is it a single loop with the logging and acquisition, or are you sending the data through a queue to a second loop where it's logged?

Are you logging ASCII or binary files?

LabVIEW does ship with some pretty good examples showing high speed data logging. Have you looked at them?

Any chance of posting your code or a simlified version of the DAQ portion?

Ed

[Ben]

Ditto Ed's request to see the code.

This post and thread has some notes that may help.

Ben

[Garcas]

Sorry for taking so long but I was out of the office. Here goes the complete code in the attached ZIP file.

The Main.vi is the top level as expected. There is also a version that tries to use pipelining (Main Piped) as my controller is a multicore processor, but I don't see any improvements

Here goes a brief description of the application first, then the problems I am experiencing

The goal... is to acquire four channels at 10 MS/s during 40s, processed them (the "algorithm.vi" which defines the processing is included, but this is not relevant now) and store the results to disk. The specifications oblige to process data as it is acquired, so post-processing is not permitted.

The solution... Once the DAQ card is configured and the acquisition is "triggered", five modules run in parallel: one to control the acquisition time, one to acquire the data from the PXI 6115 into memory, one to process it, one to store it in a file, and one to monitor the queues. The three modules in the middle communicate through two queues: one to pass data from the acquisition to the processing, and another to do the same between processing and storing to file. A third queue is used to stop the acquisition

The problems... I am not capable of running a 40s acquisition without an error 😮 I succeed to do it with only one channel, but the system feels already quite in the limit (actually, if I try to get it longer it crashes around 43s). Of course if I increase to 4 channels I don't even get to 10s (the problem is different though, LabVIEW says it doesn't have enough memory to complete the operation. I think this might be because the acquisition queue gets too big, as the processing throughput is much slower than the processing

thanks

[Garcas]

I forgot to add that I have been playing also with different configurations of the execution properties of the three main VIs ("Acquisition acquire", "Processing" and "File store") and I haven't got any improvements. I know I am doing something wrong (well in fact it must be a lot of things wrong) but I don't see it

[Garcas]

I have just realized that what I'm trying to do it's actually impossible

I've been testing my PXI-8105 hard drive and the maximum transfer rate for a write operation is around 30MS/s even using Win32 functions and optimizing Windows. The acquisition routine generates 80MB/s so the queue I am using to pass the data between the "Acquisition" and the "Store to File" modules is filling at 50MB/s. Of course it's not possible to mantain this situation more than a few seconds before LabVIEW crashes... because there is "not enough memory to complete the operation"

I will try to use some external RAID HDD connected via a ExpressCard/34 to my PXI-8105 to try to get a higher throughput when writing to file. I'll keep this thread updated in case someone's interested

Sorry if I made you loose your time