Measurement Studio for .NET Languages

cancel
Showing results for 
Search instead for 
Did you mean: 

Asynchronous AI with threaded File I/O

I am using AnalogMultiChannelReader to do asynchronous AI.
The AI thread runs independently of my UI.
It uses a thread-safe queue to send acquired data to a few UI controls.

The AI code resides in my UI class.

There is a separate class for my file IO.
The fileIO simply writes acquired data to a Binary or CSV file.
An instance of the FILE io class is created in the UI class.
The file writing is executed on the AI thread.

What is the best way to get the file writing
on a thread separate from the Asynchronous AI?

another queue?


my sample rates vary depending on the job
..up to 20 or 30 Ksamples/s /ch

thanks


Philip Newman
General Dynamics
Electric Boat
0 Kudos
Message 1 of 11
(4,916 Views)
Dear Philip,
 
Start another thread for the file IO and use another queue. Have a great day!
 
Sincerely,
Marni S.
0 Kudos
Message 2 of 11
(4,900 Views)
Philip,

Even I am trying to do the same in my code but I have two cards instead of one. So I am confused if I want to use two AI threads, one for each card.

Can you help me by telling how you have coded you app? A sample code or your code snippet would be great.

Your help is greatly appreciated. Thanks.

Anirudha

0 Kudos
Message 3 of 11
(4,893 Views)
from NI sample code:
ContAcqVoltageSamples_IntClk_ToFile

  'Create a new task
    myTask = New Task()

  'Create a virtual channel
    myTask.AIChannels.CreateVoltageChannel(physicalChannelComboBox.Text, "", CType(-1, AITerminalConfiguration), Convert.ToDouble(minimumValueNumeric.Value), Convert.ToDouble(maximumValueNumeric.Value), AIVoltageUnits.Volts)

  'Configure the timing parameters
    myTask.Timing.ConfigureSampleClock("", Convert.ToDouble(rateNumeric.Value), SampleClockActiveEdge.Rising, SampleQuantityMode.ContinuousSamples, 1000)

  'Verify the Task
    myTask.Control(TaskAction.Verify)

one thread can do two cards.
the physicalchannelcombobox.text refers to any channel on any board (device)

it will become clear if you look at
ContAcqVoltageSamples_IntClk_ToFile example code

i am not sure if i will make the file writing a separate thread from the AI..

Philip Newman
General Dynamics
Electric Boat
0 Kudos
Message 4 of 11
(4,891 Views)
Dear Philip,
 
Start another thread for the file IO and use another queue. Have a great day!
 
Sincerely,
Marni S.


would you use another form timer to dequeue ?
Philip Newman
General Dynamics
Electric Boat
0 Kudos
Message 5 of 11
(4,883 Views)
Hi serpos

Before you go down the route of another thread and another queue, i would recommend you try and benchmark using file i/o synchronous writes from the UI thread and see if you notice any UI slugishness. You're probably streaming data to a disk and I'm sure the disk can keep up with the rate you are going at.

You could consider asyncrhonous I/O, but that might not give you any advantage. If you are writing small data chunks often, you might end up hogging the thread pool because of the high number of I/O requests pending, not to mention the possibility of data being written out of order. See this link for more info. So it might cause more harm than good.

I guess you could also go for synchronous file i/o in the AI thread, but the choice between using the UI and the AI thread would affect how you might handle file error handling (hard disk full, file not found, stuff like that). You might be able to better handle those conditions in the UI thread by buffering up the data until the error is dealt with (writing to the event log, backing up to a db), but thats just something to consider.

i hope this helps. Good luck with your app.
Bilal Durrani
NI
0 Kudos
Message 6 of 11
(4,882 Views)
bilalD,

I appreciate your comments. Thanks.

my streamwriter is in the AI thread

i put it in the callback method

this is what you mean by "synchronous in the AI thread" ?
--------------
my UI thread has a form timer that dequeues a queue for  UI controls (graph, gauge, etc..)

i am guessing that writes from the UI thread
would only be reasonable if my AI was not on a separate thread..

seems like the data coming in on the AI thread would
not be synched with the UI timer...dangerous or impossible?
------------
i started this post because when I press my "stop recording to file" button,
the AI thread hogs resources and my UI thread stops for a few seconds while the AI thread finishes writing.


Cheers.
Philip Newman
General Dynamics
Electric Boat
0 Kudos
Message 7 of 11
(4,865 Views)
Hi serpos

I meant synchronous file i/o in the AI thread, as opposed to async file i/o. instead of the filestream class scheduling the write in a worker thread (using BeginWrite), it does it synchronously (using Write) and doesnt return until the data is written to file.

The AI callbacks will be firing at their own rate independant of the timer. If you were doing the writes in the UI timer callbacks (and hence in the UI thread) you would be grabbing data from the queue. This is the same data that is being displayed anyway. Since you are reading from the queue is responsible for making sure that the data being returned by the AI operation is in order. Ultimately things would sync up as you empty out the queue.

But Im not sure why the AI thread is hogging resources when you hit the stop button. Im assuming that you dispose the task when you hit the "stop data to file" button. Did this happen after you added the file I/O to the AI async callback? Are you closing the file stream when you stop recording? If its happening becuase of the file I/O, you could consider flushing the filestream every so often so as to prevent a buffer build up at the end.

I hope this helps.

Bilal Durrani
NI
0 Kudos
Message 8 of 11
(4,867 Views)
"stop data to file" button closes the file stream

"stop data to file" button does NOT dispose the task


after i finish recording to disk, i want to continue
acquiring with my task seamlessly

so that's why i don't dispose the task

i have attached the error in a JPeg

i only get the error at samples rates > ~15Ksampls/s

thanks for replying

cheers
Philip Newman
General Dynamics
Electric Boat
0 Kudos
Message 9 of 11
(4,834 Views)
i have found a temporary solution.

when the "stop recording to disk" button is pressed, i do the following

1) stop task
2) file stream flush
3) file stream close

with a 2 minute recording (8ch, 30KS/s)
i have to wait 2.5 minutes for the task to restart.
due to file io buffering ? i wonder where the buffer is?
my RAM is steady throughout at 422MB

4) start task


i tried flushing after writing each row to the datafile
but that results in a steady increase in AI Scan Backlog.. at 30KS/s

but seems to work okay at 20KS/s


Philip Newman
General Dynamics
Electric Boat
0 Kudos
Message 10 of 11
(4,835 Views)