Data Acquisition rate is not as expected. Only half of the data were logged.

[Billy90]

Hello community, I am working on a project that can acquire voltage and current data from a three phase motor.

If you open the vi file I attached to the post, you'll find a section in the front panel where you can input "number of samples" and "Sample rate. " 

When I set the number to samples to 2000 and the sample rate to 25000 then run the program for 2 minutes. Only half of the data were logged into the txt file. (There were supposed to have 25000*120 = 3000000 data, but in reality there were only 1500000 data)

Can someone tell me where I did wrong? 

I am using NI9244 to acquire voltage data and NI9227 to acquire current data. The maximum acquisition rate of these two DAQ are 50000 so I don't think the problem is on the DAQ. 

Thanks a lot for reading and replying. 

[ZYOng]

Using the Elapsed Time Express VI to time the logging is a bad idea. It might have terminated the file saving prematurely. Saving data to a hard disk actually is a relatively slow process, at least much slower than reading from a DAQ device. 

You should use a shift register as a counter to count the number of data being saved instead.

[Billy90]

But if I use the shift register to count the data then the time I need will be twice of the time. For example, I want the sample rate to be 25000 which means that the data should be acquired at the rate of 25000 data per second. Then if I run the program for 2 minutes, there should be 3,000,000 data. However, now in reality, to achieve this acquired data number, I need to wait for 4 minutes. Do you know how to fix this? 
Thanks for replying to my post. 

[GRCK5000]

Hi Billy,

I modified your code using local variables. Can you give it a try and let me know if it works or if it doesn't work. If you don't want to let me know, that's fine, but I would appreciate to hear from you because I also want to learn from my mistake.

See attached code. 

[Bob_Schor]

The Good News is that you recognize that you should use a Producer/Consumer Design Pattern to get the data out of the Acquisition (Producer) loop as fast as it comes in so that you can acquire data at the full rate.  What you didn't do correctly is to set up the Queue(s) properly.  The other thing to consider is that Producer/Consumer should (ideally) have a single "Producer" inside the loop -- recall that the Loop runs as fast as the slowest element in the Loop.

Here's a suggestion -- try making a simple Producer/Consumer pattern.  For the Producer, make a For Loop run 10 times to simulate 10 1-second generation of 1000 random numbers generated at a rate of 1 kHz (model this inner loop as a For Loop with N = 1000, inside of which is a 1 ms Wait (to simulate the 1 kHz sample rate) and a Random number generator, brought out through an Array Indexing Loop.  Put that in your Producer/Consumer Queue.

Your Consumer Loop should gather the 10 "batches" of 1000 samples and save them to disk.  Feel free to use the same logic you used in your (flawed) example code. 

Incidentally, you should only use .zip files (not .rar, which are not supported by Windows natively), and you only need to compress VIs when you have more than 3 or 4 files to send, otherwise with a single VI, just attach it by itself.

You might want to examine the example Producer/Consumer Design Pattern that ships with LabVIEW.  You can find it by starting LabVIEW, click "File", then "New ..." (the one with three dots, that's important), then choosing 

"New VI", "From Templates", "Frameworks", "Design Patterns", "Consumer/Producer Design Patterns (Data)".

I look forward to you having your own "Ah, ha!" moment.

Bob Schor

[ZYOng]

This is because writing to a text file takes more space and more time. You should use binary-based TDMS for high speed streaming. NI TDMS File Format - What is a TDMS File?

You can refer to the attached benchmarking code. TXT logging takes 30 times longer time than TDMS logging.

[Billy90]

Dear Mr. Schor, 

First, I want to thank you for your detailed response.

Today I had a meeting with the professor, he said the exact same thing. I am going to try using the queue message handler which is really advance to me haha. But I'll do my best and again thank you so much for your help and reply. 

Hope I can find my "Ah, ha! " moment someday. Right now it's more like "HUH!? What!!?" moment haha. 

Have a great day.

Billy

[Billy90]

Dear @GREK5000,

Hello! Thanks a lot for your help. I tried it today and it has some problems. 

The data were too much. It exceeded the maximum data acquisition rate of the DAQ card I use. 

I just sent you a message. Hope you receive my message. 

About the test files, I don't think I am able to attached those files here 'cause those files are too large. Therefore, I will attach a screen shots of the txt file. 

Thank you for your help! 

Can't wait to hear from you. 

Billy

[GRCK5000 · Accepted solution]

Hi Billy,

I think too much is better, that way we can sample it to a desired amount. 

The attached code is similar to the one I attached previously, but for this one I'm using queues to transfer data. 

Also instead of 20 seconds, try to change it to a number lower and see how many data you are being able to collect. Keep changing it until you get the desired amount of data. How many data are you expecting to get in 20 seconds?

One thing I forgot to do on the first code I attached is what is shown with the red arrow. You want to make sure the arrays are empty before the test starts. It's a good habit. That's what I failed to do on the previous code. The code is updated now. So try again both codes, the one with the queues and the updated one with the local variables. Let us know what you get. 

[mcduff]

I did NOT look at the original RAR file, but there are some problems with the code presented later in thread.

1. Are both cDAQ modules in the same chassis? If so combine into a single task, that way they are synchronized. (I assume you want to measure the power factor.)
2. Check the spec sheets for both modules. Both modules are 24 bit DSA digitizers. There is a phase delay that depends on the sample rate through each module. It will probably be small (microseconds), but there are time differences if you are comparing current and voltage directly, depends on how precise you want to be.
3. Use the built in TDMS logging feature and post-process your data. You will never have a problem saving the data.
4. Clear the task after you stop it.

Here's something that could fit on the back of an envelope that assumes both cards are in the same chassis, and uses the automatic TDMS logging feature. You can modify to display line to line voltage, just the basics here. Simulated instruments have no problems at 25k, 50k rates.