Urgent: continous data recording from NIDAQmx and other instruments using multi-thread vc++

[kgy]

I'm working on a software using vc++ to control several instruments including a NI-DAQmx6289. For example, some of the software features;

1.      Communicate with data acquisition card (DAQ card), and continuously acquire data from multiple channels at a frequency of 1 kHz.

2. Connect with function generator through RS232 port 1, send triggered bursts to function generator, real time modify ultrasound status variable (G_usstatus).

3.Connect with optical switch through RS232 port 2, send signals to switch between the two input channels (0,1) and six output channels (2,3,4,5,6,7) at user specified sequence and time interval (by default:150 ms), real time modify the channel connection variable (G_optchannel).

            4.  Real-time save the data together with the corresponding ultrasound status and optical channel connection status into a use specified file for post-analysis.

I'm using multi-threads to do the data acquisition (thread 1), control of the function generator (thread 2) and optical switch (thread 3).  I also need to record the data from daq together with the corresponding ultrasound status and optical channel connection status into a use specified file for post-analysis.I'm using the following codes to read the data continuously.

void DataCollectionWin::ConnectDAQ()
{DAQmxErrChk(DAQmxCreateTask ("", &taskHandle));
  //DAQmxErrChk(DAQmxCreateAIVoltageChan(taskHandle,"Dev1/ai0","",DAQmx_Val_Cfg_Default,-10.0,10.0,DAQmx_Val_Volts,NULL));
  DAQmxErrChk(DAQmxCreateAIVoltageChan(taskHandle,"Dev1/ai0,Dev1/ai1,Dev1/ai2,Dev1/ai3,Dev1/ai4,Dev1/ai5,Dev1/ai16,Dev1/ai17,Dev1/ai18,Dev1/ai19,Dev1/ai20,Dev1/ai21,Dev1/ai6,Dev1/ai7,Dev1/ai22","",DAQmx_Val_Cfg_Default,-10.0,10.0,DAQmx_Val_Volts,NULL));
  DAQmxErrChk(DAQmxCfgSampClkTiming(taskHandle,"",1000.0,DAQmx_Val_Rising,DAQmx_Val_ContSamps,60000));

  DAQmxErrChk (DAQmxRegisterEveryNSamplesEvent(taskHandle,DAQmx_Val_Acquired_Into_Buffer,50,0,EveryNCallback,NULL));
  DAQmxErrChk (DAQmxRegisterDoneEvent(taskHandle,0,DoneCallback,NULL));

  DAQmxErrChk (DAQmxStartTask(taskHandle));

   Error:
 if( DAQmxFailed(error) )
 {
  DAQmxGetExtendedErrorInfo(errBuff,2048);
  MessageBox(errBuff);
  DAQmxStopTask(taskHandle);
  DAQmxClearTask(taskHandle);
   return;
 }

}

int32 CVICALLBACK EveryNCallback(TaskHandle taskHandle, int32 everyNsamplesEventType, uInt32 nSamples, void *callbackData)
 {
 char l_optstatus_s[1],l_optstatus_e[1];
char l_usstatus_s[1],l_usstatus_e[1];

 /*********************************************/
 // DAQmx Read Code
 /*********************************************/
if (!m_bStopTracking)
{
 l_usstatus_s[0]=g_usstatus[0];
 l_optstatus_s[0]=g_optstatus[0]; //optical switch status before reading the 50*15 data

 DAQmxErrChk (DAQmxReadAnalogF64(taskHandle,50,10.0,DAQmx_Val_GroupByScanNumber,data,50*15,&read,NULL));
   

 SetEvent(hEvent);
l_usstatus_e[0]=g_usstatus[0];
 l_optstatus_e[0]=g_optstatus[0];////optical switch status at the end of reading the 50*15 data

 if( read>0 )  ///// save the data into exl file specified by 'datafile'.
 {

   //indicator=1;
  

 for (i=0;i<read;i++)
 {  //fprintf(datafile,"%d\t",i);
  fprintf(datafile,"%c\t",l_usstatus_s[0]);
  fprintf(datafile,"%c\t",l_usstatus_e[0]);
        fprintf(datafile,"%c\t",l_optstatus_s[0]);
  fprintf(datafile,"%c\t",l_optstatus_e[0]);
          fprintf(datafile,"%.2f\t",data[15*i]);
         fprintf(datafile,"%.2f\t",data[15*i+1]);
     fprintf(datafile,"%.2f\t",data[15*i+2]);
     fprintf(datafile,"%.2f\t",data[15*i+3]);
     fprintf(datafile,"%.2f\t",data[15*i+4]);
          fprintf(datafile,"%.2f\t",data[15*i+5]);
    fprintf(datafile,"%.2f\t",data[15*i+6]);
     fprintf(datafile,"%.2f\t",data[15*i+7]);
     fprintf(datafile,"%.2f\t",data[15*i+8]);
     fprintf(datafile,"%.2f\t",data[15*i+9]);
          fprintf(datafile,"%.2f\t",data[15*i+10]);
    fprintf(datafile,"%.2f\t",data[15*i+11]);
    fprintf(datafile,"%.2f\t",data[15*i+12]*5);
          fprintf(datafile,"%.2f\t",data[15*i+13]*5);
    fprintf(datafile,"%.2f\n",data[15*i+14]*5);
       
      }
  
    fflush(stdout);
  
  }
}
}

Now the problem is the data acquired from daq card does not match the corresponding saved optical swtich status (G_optchannel, which specified the connecting light channels). High readings expected at certain status actually showed up in other status.  It seems there is misalignment of data from the multi-threads. Because the status of optical switch changes at 150ms, so I have set DAQmxRegisterEveryNSamplesEvent to be trigued every 50samples,which means 50ms with 1khz sampling rate to avoid missing the changes.I also check whether there is any status change during DAQmxReadAnalogF64, by recording l_optstatus_s and l_optstatus_e, which were actually shown to be the same. I wonder whether it is because the data are first saved to the buffer. When the software starts reading, at that time, the optical swtich status no longer reflects the status when the data were first saved. Is there any way to solve this issue?  Thank you very much!

[Dan_K]

Hello kgy,

I am not sure I understand exactly what you are doing, but I have some suggestions for this type of task. First realize that if the EveryNSamples Event is used to trigger an acquisition from your serial device, this information should be semisynchronous with the next set of samples, not the ones that caused the event to execute.

The whole problem with this is that it can be very difficult to synchronize a serial task with a DAQ task. The only way to do this reliably would be for your serial device to either be able to supply or accept a digital signal as a trigger. If this is possible you can set up your DAQ task to trigger off of your serial acquisition or vice versa.

This would be best if you were wanting to take discrete numbers of samples for a given serial acquisition and then stop and restart the task to be ready for (or generate) another trigger. If this would not be acceptable you would possibly need to look into some other type of device (such as a Digitizer).

Of course, most of this is irrelevant if you are unable to perform hardware triggering with your serial device. If this is the case you will need to use the software triggering described in the first part of this post.

Let me know if you have any other questions.

[kgy]

Thank you very much, Dan_K. I think I don't really need to synchronize the DAQ task with other instruments, since i have two other instruments both generating some kinds of signals. What I want to do is that how to get the most recent collected data from the buffer? In this way I guess the time delay is brought down to minimum. I saw there are some property nodes can be set in labview. How to do it in VC++? For example, how to achieve the following from labview in vc? Thanks a lot!

DAQmxRead.RelativeTo=MostRecentSample
DAQmxRead.OverwriteMode=OverwiteUnreadSamples

[Mcdan]

kgy,

Any property that is settable from LabVIEW should also be accessible through the C API.  For the two functions you've mentioned:

LabVIEW Property                                                                         C API Function

DAQmxRead.RelativeTo=MostRecentSample ->                   DAQmxSetReadRelativeTo=DAQmx_Val_MostRecentSamp

DAQmxRead.OverwriteMode=OverwiteUnreadSamples ->  DAQmxSetReadOverWrite=DAQmx_Val_OverwriteUnreadSamps

You should be able to search the DAQmx C Reference Help by function name for details.

Dan

[kgy]

Thanks a lot! That's very helpful.

[kgy]

I have one more question. If I set the data acquisition frequency at 1Khz and doing continuous acquisition. For the data collected in the buffer,can I simply add the time tag with 1ms increments? Will other serial threads cause any delay?

For example, take start time as 0

            1ms   2ms   3ms................nms

buffer: data1 data2 data3 .............datan

[Mcdan]

kgy,

That depends on the method you are using to read the data.  Your hardware will sample data every millisecond.  That data will get sent to the buffer.  If the method you use to read this data ensures that you read every sample in order, then yes you can simply increment time tags by one ms.  Be advised that that probably won't yield accurate results if you allow samples in your buffer to be overwritten and only read the newest data.  The DAQmx call back function get called every time the device transfers the specified number of samples to the buffer.  However, nothing is preventing the device from transferring more data to the buffer in the time it takes the callback to get called.  So let's suppose your first callback read samples 0-49.  Once sample 99 is transferred to the buffer, DAQmx again begins the process of calling you back.  However, lets suppose your system is busy, and the call to DAQmxReadAnalogF64 gets delayed by several ms (if you are running on Windows, this will likely happen at some point).  By this time, samples 100-102 may have been transferred into your buffer.  Since you configured DAQmx to read the newest samples, it would return samples 52-102, and your timestamps would then be off by 3 ms.

One way you might be able to get around this would be to read relative to the last read sample, but to read all available data (set numSampsPerChan parameter in DAQmxReadAnalogF64 to -1), rather than specifying 50 points exactly (you'd have to provide an array to the read function that was big enough to handle some extra samples as well).  DAQmx should provide the number of sample returned via the sampsPerChanRead  parameter.  This way, you can ensure that you've always read all the samples from the buffer.  However, it'd mean your code would need to be able to handle reading a variable number of samples.

Hope that is of some help,

Dan

[kgy]

Hi, Dan,

Thank you so much for your detailed explanation. It is very very helpful. I'm changing the code to get all the available data in the buffer. But I still have one doubt, which is how to tag a time stamp to the data  samples.  For example, if I get the current system time before each DAQmxReadAnalogF64 action, can I simplely tag the time to the last sample in the acquired data, then minus 1ms to the prior data points?

Is there a simpler way to get the time stamp?

Thank you very much.

[Mcdan · Accepted solution]

kgy,

Glad to hear that you are progressing with your project.  Timestamping data is always a little bit tricky, as the process of querying a counter on the CPU is done asynchronously with the acquisition in your DAQ hardware.  However, your hardware will guarantee that the relative timing between samples is consistent (in your case data will be sampled every 1 ms).  And since you have modified your program such that you are now reading all samples acquired, you know that each sample follows the previous by 1 ms.  As such, if I were to implement this I believe that I would take an initial timestamp when I started the task, and use this timestamp to calculate the timestamp for all samples to follow (timestamp SampleN = (N * .001s. + initial timestamp) or timestamp SampleN = timestamp SampleN-1 + .001s).

I would lean towards doing things this way, rather than timestamping the end of read for the following reason.  As I mentioned before, your read callback will run when the OS schedules it.  As such, it is possible that it gets delayed or doesn't run exactly in sync with the acquisition in hardware, and you've made adjustments to your code to handle this behavior.  However, when you're timestamping there is one other thing to pay attention to.  That is the fact that your hardware has a FIFO where sampled data can accumulate before getting transferred to the buffer you read from (say if the PCI bus were busy when the sample was acquired).  Now suppose the stars align against us and the following happens:

1) The OS is busy with other things, and our read callback gets delayed a few ms.

2) At the same time, some other device ties up the PCI bus (or some part of the data path between your device and memory used for the buffer).

3) One or more samples are momentarily stranded in the device's FIFO.

If you were to read data, and timestamp at this moment in time, your timestamp would reflect the data that is in the buffer as well as the data that was currently in your device's FIFO.  However, you can only read the data that is in the buffer.  As a result, the timestamp applied to the data you just read would be a millisecond or two later than they should be.  Suppose on the next read callback, this condition has cleared itself up.  This timestamp taken here would be accurate, however, you would return a couple of additional samples (the ones that were stranded in the FIFO last time).  If you back-calculated your timestamp at this point, I think the timestamp calculated for these would overlap with timestamps calculated on the previous read.  This would not be ideal, nor would it reflect when the data was actually sampled.

The one remaining difficulty is how to accurately timestamp the start function.  For this, I would add a call to DAQmxTaskControl(taskHandle, DAQmx_Val_Task_Commit) before calling DAQmxStartTask.  This step will advance the DAQmx state model as far as possible without actually starting the task.  This will help make DAQmxStartTask as fast as possible.  Now, either before calling start or immediately afterwards, I would take my initial timestamp (perhaps timestamp both before and after and take the average).  I would then use this initial timestamp and my known sampling rate to calculate timestamps for all the rest of my samples.

That got a bit lengthy, but hopefully was helpful.

Dan

[kgy]

Hi, Dan,

Thanks a lot for detailed explaination. It helps A LOT!! I love this forum.

Thanks again.