Where are nidaqmx low level functions?

[Merlin-3]

Hi,

>> PS, if you are using the Unscaled (e.g. BinaryI16) read calls with M-Series, be sure you correctly apply the scaling coefficients for the device.  M-Series uses a polynomial scaling function, not linear like E-Series.  The polynomial scaling coefficients are a channel property, and you can read them back.


Huh??
I am using DAQmx_Val_Volts and not DAQmx_Val_FromCustomScale to set the channels in the AI task.
Thus, if the range is, let's say -1 and +1 volts, the binary numbers read by DAQmxReadBinaryI16() are a linear representation of such range, isn't it? Or things in the M series are even worst?

The only thing the documentation says is:
"Unscaled data is in the native format of the device, read directly from the device or buffer without scaling. The native format of a device can be an 8-, 16-, or 32-bit integer, signed or unsigned."
It does not say the binary numbers must be converted to volts using a polynomial function or table (if we do not use a custom scale).
The natural thing to suppose is that -32768 is min range, +32767 is max range, 0 is zero and in between the proportion is linear.
If we use U16 then the binay range just change to 0 to 65535, but still a linear proportional between min and max range.
It is not???

>> It sounds like you have written your application such that you want to call Read very often (e.g. specifically, you benchmarked that with Traditional, you can read 20000 times a second).  Can you can describe what you are doing in your application that would benefit from calling Read more often than 2500 times a second?


Everything you wrote matches what we saw in our tests and I agree about the block reading of data.
Yes, the ONLY way to use the nidaqmx is reading a block of data, instead of one by one, to achive the same purpose. In general terms, yes, it must be a better approach. But...

Let me try to describe what we do in the Traditional. Then you will understand why we use the callback to read one scan (2 channel-samples), what seems, at first glance, a bad solution.

In our equipment (a potenciostat), we run four independent aquisitions at the same time (reading 2 channels each one).
Each aquisition has its own sample frequency, set by the user.
To achive this, we run the board at a higher speed... 20Khz ...  and set FOUR callbacks-events to call us when it is time for each acquisition (with reading frequency slower or equal the max true board scan frequency) meaning "it is time to read": Obviously, the board buffer (set for some seconds of data) is filled with a lot of samples continuosly... at 20Khz ... and each callback, when fired, PICKS UP the correct data it wants in the buffer (it is just a small calculation with a pointer), copying the two channels reading (a voltage and a current, to be exact) to a circular temporary buffer wich is read by a thread (there are four temp buffers and threads) that processes the data, writes a graphic in real time and saves the data in a file (in fact there are four windows with four charts and control buttons).

Because the callback in the Traditional is very efficient and because the code to pick up the data is small (in addition, we can do an average to work as a filter, as option, because there are more samples in the board buffer than we need....), this approach works VERY well and fast.

Using the nidamx, on the other side, it is impossible to use the callbacks that way.... what is a huge pity.... because the callbacks are much slower... and the functions used to read the data are also very heavy, slow and using a lot of useless parameters.
Of course, as you wrote, the only option is to read a block of data. (and this is what I am trying to do now).
Yes, it works, I have tsted it on a test program. IF it will be faster than the Traditional approach, I do not know.... but surely it will be more complicated.
If the block reading is better (theoretically in terms of speed), it imposes also a more complicated code to pick up the data in the right places by the FOUR concurrent aquisitions we run. We can be faster in one side, but are slower in the other side. Not to mention the overhead of the callback which is really bad in the current nidaqmx.


Even if I did not finish and tested the nidaqmx approach... I think it will work. But I am not sure if it will work as well as using the Traditional. My bet is that it will be slower with the current nidaqmx.
It would be much better for the programers (true ones, not Labview users), if NI sets some low level functions in nidamx, to give the programmers more options.
Until now, the real advantage of nidaqmx was the task configurarion, very easy. On all other issues it only created problems instead of (help) solving them. If, at one side to configure the tasks was easier, on the other side I am having a hard time to create much more complicated code to do the same things I did (and still do) with the Traditional.

Thanks for your help,
Lavio

Message Edited by Merlin-3 on 10-31-2007 02:51 PM

[Jonathan_Brumley]

Hi Merlin-3,

Thanks for explaining your application.  Hopefully this will help me optimize future versions of DAQmx. 

If I understand correctly, the fact is you want to read often because you want to process samples one point at a time.  I why that would be easier for some applications.  Also, I didn't realize until now that you are saying the DAQmx event callback is also slower than Traditional DAQ.  I will do some investigations to confirm what you are saying about the callback, because I haven't yet done a comparison between DAQmx and Traditional DAQ for callback performance.

By the way, have you thought about using the Hardware-Timed Single Point (HWTSP) mode when configuring timing?  Hardware timed single point is specifically designed with low latency to support reading one sample at a time.  In other words, it's designed for control applications, not monitoring.  HWTSP doesn't use a buffer and is therefore faster than buffered reads when you measure latency.  Instead of using the Every N Samples event, you would need to use the Sample Clock event with HWTSP. 

Regarding scaling on M-Series:

You can use a linear function to scale the results returned by the BinaryI16 read.  The values will be approximately correct; however, I would recommend using the scaling coefficients, if possible, if your application requires accuracy.  The fact that the scaling coefficients are polynomial is a result of the MCal algorithm we use on M-Series.  The polynomial coefficients help compensate for small non-linearities across the range of the ADC, which when applied correctly, result in more accurate measurements.  With E-Series products, the scaling doesn't take into account the nonlinearity of the ADC, the resulting measurements are therefore more nonlinear. 

Just out of curiosity, why don't you use the scaled DAQmx reads (e.g. AnalogF64). 

[Merlin-3]

// Also, I didn't realize until now that you are saying the DAQmx event callback is also slower than Traditional DAQ.

Well... in my tests, it was slower. Much slower.
I did a test also using the counter and a callback to use as a "clock" just incrementing a variable which was monitored through a 500 ms timer interval to make things very simple. And the max frequency it was able to work was 2 - 3 times slower than the Traditional in a AMD true 2GHz machine.
It was so bad that I discarded using callbacks for fast repeated working while in the Trad. it worked very well.

// Hardware timed single point is specifically designed with low latency to support reading one sample at a time.

Latency is not our issue.
Continouous real time fast acquisition is, with progress display (and saving in disk) of results for FOUR acquisitions engines (2 channels each) at same time.

/* You can use a linear function to scale the results returned by the BinaryI16 read.  The values will be approximately correct; however, I would recommend using the scaling coefficients, if possible, if your application requires accuracy.  The fact that the scaling coefficients are polynomial is a result of the MCal algorithm we use on M-Series.  The polynomial coefficients help compensate for small non-linearities across the range of the ADC, which when applied correctly, result in more accurate measurements. */

This is new stuff for me... I did not read it in the documentation. Or, I did not find it.
I read about custom scales. But if I understood, you are talking about internal scaling coefficients not applied when we use the Binary reads and used when we use the scaled Analog reads.
We do not use the scaled Analog reads because they are slow. As you see, we are always looking for the fastest approach. The scalling calculations we can do later, through threads for processing and displaying, instead of when getting the data in the time-critical callback.

But now I need to know what are these internal scalling coeficients ! HOW I get them?

=-=-=-=-==

Another very serious stuff....
We bought recently 3 x 6023E boards.
In the boxes and CDs, there was no Tradtional anymore. Only the NiDAQMx cds.
We installed the 7.4x Traditional and.... a bad surprise: only one of the boards worked with the Traditional. The other two did NOT work.

Did NI change something in the 6023E hardware???????? to make it work only with the NiDAQMx??????

Thanks,
LMP

[Merlin-3]

Another question....

Can we change gain (Traditional) or analog input range (MX) without interrupting the acquisition (we want to be continuous)?

Thanks,
LMP

[FOM]

Hi there, Merlin-3

I don't want to hijack your thread, but I seem to be in a situation very similar to yours. I am trying to obtain continuous, pseudo-real time acquisition on 30 channels at 4000Hz with a PCIe-6259, and I find the NIDAQmx drivers unable to perform as required. I find it strange as the board is supposed to handle much higher data transfer rates.

Would you (or any NI staff around here, for that matter) have any pointer as to how I could proceed from here ? Have you found any way to directly access the buffer without going through the pains that you described in your previous messages ? I would be more than grateful if you could just point me in the right direction for faster acquisition.

Thanks in advance for your input

Regards

FOM

[Merlin-3]

> I don't want to hijack your thread

You are welcome! If more NI customers complain... maybe it would make NI rethink about the nidaqmx crap.

> I am trying to obtain continuous, pseudo-real time acquisition on 30 channels at 4000Hz with a PCIe-6259, and I find the NIDAQmx drivers unable to perform as required.

Ahhh...

Yes, they are unable. Just that.

> Would you (or any NI staff around here, for that matter) have any pointer as to how I could proceed from here ? Have you found any way to directly access the buffer without going through the pains that you described in your previous messages ?

That's the point: to directly access the buffer!!

But they do not tell us. I think they just want to impose LabView and do not give a penny to old customers using other (and better) approaches.

One option is obvious: to start looking for more reliable manufacturers, like Agilent.

Meanwhile to bypass nidaqmx limitations we are reading (by callback) continously blocks of data at highest speed possible and created a thread to pick up inside THIS buffer we manage the data we want.

The callbacks in nidaqmx have pitiful performance, but we can compensate it reading blocks of data. In our case, about 200 samples (up to 8 channels each) each callback at 100Hz.

Thus, the callback is slow and we collect data blocks and pick up the data we want inside the buffer we read the data blocks assynchrously.

Unfortunatelly, with so many threads and buffers in the path ... there is some latency. If the application needs very fast response in real time ... use the Traditional and forget the nidaqmx.

🙂

Message Edited by Merlin-3 on 01-19-2008 08:34 AM

[FBurton]

I too am disappointed by the dropping of the Traditional NI-DAQ API and the hoops it looks as if I will have to jump through with nidaqmx. I don't believe it is even possible to use Trad with the new M series boards. 😞

My application collects 256 channels (4xPCI-6225 board) at 1-5kHz per channel set, continuously over DMA. With Trad, it was easy to start acquisition going and then retrieve the data from the buffer using my own code. The reason I want to do this is because the channels need to be remapped (renumbered), and it was convenient to do this in the transfer from the DMA buffer to my data buffer. Then if I need to write the data out to disk (for example), I can just give an address in my data buffer. Now it looks like I have to do two lots of memory transfers: one from the DMA to my buffer using nidaqmx, and then a second one from my buffer to another buffer (or the same one, by shuffling). That is twice the memory accesses. Programmatically it is also more complicated because I have to manage two on-going transfer processes: one via nidaqmx callbacks, and another via my own thread.

Does this sound like a reasonable summary of my problem? Is there an obvious solution I am missing?

[Merlin-3]

>> Does this sound like a reasonable summary of my problem? Is there an obvious solution I am missing? Does this sound like a reasonable summary of my problem? Is there an obvious solution I am missing?

Yes, it resumes very well the same type of problem we have because nidaqmx does not give us the buffer address to pick up what we want, when we want and how we want.

[FBurton]

Ok, so assuming that I'm not going to run into performance bottlenecks dealing with 256 (usually, but potentially 512) channels at 5kHz, what is the best way to emulate what I had before, i.e. an accessible block of memory acting as a circular buffer with a means of obtaining the latest sample collected (previously via DAQ_Check)?

By "best way", I mean what are the appropriate routines to use? Is there example code available to do this already?

I have come across a routine called DAQmxSetReadReadAllAvailSamp but cannot find documentation for it. Would it be useful for what I am trying to do?

Finally a question about the non-linearity of the ADCs. I would very much prefer to work with 16 bit integers rather than floats - they would need to get turned back into integers anyway to satisfy file format requirements. Am I correct in assuming that if I use simple linear scaling, the non-linearity of the M-series cards is no worse than that of the E-series equivalents?

(P.S. I use Delphi as, in my experience, it provides a much better framework and facilities for developing the UI parts of applications than Visual C++.)

[Merlin-3]

>  i.e. an accessible block of memory acting as a circular buffer with a means of obtaining the latest sample collected (previously via DAQ_Check)?
 i.e. an accessible block of memory acting as a circular buffer with a means of obtaining the latest sample collected (previously via DAQ_Check)?

You have to do it reading several scans at once to fill a buffer you have control over it and then creating a thread only to read from this buffer what you want.

In other words:

1- Using a slow frequency continious callback you read some hundred scans at once each time and fill a large circular buffer. You fill this buffer by blocks of, let's say, 300 scans (I mean by scan a set of N channels reading).

2- And parallel thread keeps picking up from this circular large buffer what you want to do the same thing your older routines did directly in the board buffer because you had the buffer address - now, because you do not have the board buffer address, you need to copy the data into another buffer first inside the callback.

>  Is there example code available to do this already?

The only examples I got from National documentation were very simple routines to create a task, read some floats and stop. Very dumb.

> Finally a question about the non-linearity of the ADCs. I would very much prefer to work with 16 bit integers rather than floats - they would need to get turned back into integers anyway to satisfy file format requirements. Am I correct in assuming that if I use simple linear scaling, the non-linearity of the M-series cards is no worse than that of the E-series equivalents?

Ahhhhhhhhh... here we have another serious problem for using Nidaqmx.

There are several analog binary read functions. And only a couple of analog Float read functions. The dumb examples? They use ONLY the float readings. Never the binary readings.

Yes, I also did assume that the binary readings were linear ... but our first real board running program with the Nindaqmx is giving us a serious reading inacurracy. A problem that the E series board did not have.

We are trying to figure out how to solve this new problem caused by the nidaqmx M boards... using calibrations arrays, to keep using the binary readings instead of the very inefficient float readings. This problem is very cumbersome.

On other message I asked some Nat guy who mentioned the binary UNSCALED non linearity of the M series...how we can use the binary reading functions and get realiable readings, but I had not response.

I would like to know HOW can we use the binary analog reading functions, if they are non linear and the documentatin does not tell us how to get THE CORRECTION COEFFICIENTS. Then what they exist for? To read wrong numbers we cannot fix?

To use the float analog readings (scaled funcions) is VERY inefficient and stupid on several circunstances, specially inside callbacks and processing a lot of numbers in real time!

I would like to be wrong on something. But I do not see any good solution from the documentation and the nidaqmx functions we can use. The binary analog reading functions just do not give us the right readings and we cannot get the correction coefficients.

>  I use Delphi as, in my experience, it provides a much better framework and facilities for developing the UI parts of applications than Visual C++.)

The same we did.

🙂