LabVIEW

cancel
Showing results for
Did you mean:

Highlighted

Data type

Can anyone please explain what does 'Approximate number of Decimal digits' indicated in 4th column of the image.??

Message 1 of 5
(568 Views)

Re: Data type

How many digits you can see if you wrote that number out in decimal, i.e. base 10.

Think significant digits.

Message 2 of 5
(562 Views)

Re: Data type

Thanks for the reply.
just for clarification, taking an example 'I8' numeric data type, we have '2' as approximate number of decimal digit written in 4th column of the image. The range is -128 to 127 , which involves three significant number..!! Ex: 123.
Am i right ?

Message 3 of 5
(551 Views)

Re: Data type

I think it has to do with the fact that all floating point numbers cannot perfectly represent decimal digits or fractions of a whole number. The chart is a guide to where you will start to see error in the number in terms of number of digits to the right of the decimal.

Example: Divide 1 by 10 and show the result in a double indicator, the chart says you get about 15 digits of precision, in this case we got 17 digits before error. Message 4 of 5
(548 Views)

Re: Data type

@vishal777 wrote:

Thanks for the reply.
just for clarification, taking an example 'I8' numeric data type, we have '2' as approximate number of decimal digit written in 4th column of the image. The range is -128 to 127 , which involves three significant number..!! Ex: 123.
Am i right ?

Yes.  But it also says "approximate".  That third digit isn't really a full digit.  It is only 0, 1, or 2 for an unsigned U8, and -1, 0, or 1 for a signed 8-bit.  So it is really 2+ significant digits.

Message 5 of 5
(542 Views)