USB-6008 voltage calculation from binary data?

[stilly32]

Hey George,

What are the values of the sacling coefficients returned from the 6008, and what terminal configuration (RSE, Differential) are you using? I'm trying to track down a 6008 to compare. Also, what are the binary values returned at those voltages? We seem to be close, but just off somewhere...

cheers,

Andrew S

[George_Cs]

Hi stilly32,

The coeff. values are:

coef[0] = -10.2685031890869
coef[1] = +0.0003180253475693

Terminal config is RSE. Measuring range is +/-10 V. The binary values are seen in my previous post.

Greets
- George Cs -

[stilly32]

Hey George,

You may want to check how you (or possibly my code) are doing calculations. When I do it by hand:

+1.62 V = 0x9200 = 10010010:00000000 = 37376 decimal.

37376 *0.0003180253475693 + -10.2685031890869 = 1.62

I did note that with such large numbers, the number of significant digits really matter - cutting the calc off at .000318 yields significant error.

Did you use my direct code for the calcuations or do something different?

Cheers,

Andrew S

[George_Cs]

Yes, you caught it! Here is the problem. I'm using SmallInt (int16) for the bin array, and in this case 0x9200 means -28160 (not +37376), so the calculation is false. If I change the type of bin array to Word or I complement the equation with a "reading & 0xFFFF" expression the result will be correct!

Thank you very much for your help!

Greets
- George Cs -

[George_Cs]

Though the routine works fine now, I think I should clear up this whole enigmatic story.

Plamen says, that there is 1 sign bit, but this is a mistake. The binary array contains only positive numbers, i.e. the legal range of the binary value is between 0..65535 (not -32768..32767). The offset coefficient "pulls down" the negative voltage values into the negative range (from the positive binary numbers) in the course of calculation.
With another words, the binary representations of the values are (approx):
-10 V = bin 0; +10 V = bin 65535
It's a very clear logic, but unfortunately nobody could define it to me as yet.

I don't understand why did NI determine a signed integer type for this bin array... If it's specified as Word, there is no problem. I don't know, how C compiler handles the int16 type, and how negative int16 binary number turns to positive in the above mentioned equation, but in Delphi pascal (in which I'm working) the hexa 0x9200 is always -28160.
In the equation, the "reading & 0xFFFF" (in Delphi: "reading AND $FFFF") trick says for the Delphi compiler, that this binary value must be handled as Word instead of signed int16.
It seems, C compiler does it automatically, this is the cause, why equation worked for you stilly32, and why didn't for me.

Greets

- George Cs -

[stilly32]

I'm glad you go that figured out and thanks for posting back with your resolution.

Cheers,

Andrew S