LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Data type

Highlighted

Can anyone please explain what does 'Approximate number of Decimal digits' indicated in 4th column of the image.??

0 Kudos
Message 1 of 5
(568 Views)

How many digits you can see if you wrote that number out in decimal, i.e. base 10.

 

Think significant digits.

0 Kudos
Message 2 of 5
(562 Views)

Thanks for the reply.
just for clarification, taking an example 'I8' numeric data type, we have '2' as approximate number of decimal digit written in 4th column of the image. The range is -128 to 127 , which involves three significant number..!! Ex: 123.
Am i right ?

0 Kudos
Message 3 of 5
(551 Views)

I think it has to do with the fact that all floating point numbers cannot perfectly represent decimal digits or fractions of a whole number. The chart is a guide to where you will start to see error in the number in terms of number of digits to the right of the decimal.

 

Example: Divide 1 by 10 and show the result in a double indicator, the chart says you get about 15 digits of precision, in this case we got 17 digits before error.

 

Capture.PNG

Message 4 of 5
(548 Views)

@vishal777 wrote:

Thanks for the reply.
just for clarification, taking an example 'I8' numeric data type, we have '2' as approximate number of decimal digit written in 4th column of the image. The range is -128 to 127 , which involves three significant number..!! Ex: 123.
Am i right ?


Yes.  But it also says "approximate".  That third digit isn't really a full digit.  It is only 0, 1, or 2 for an unsigned U8, and -1, 0, or 1 for a signed 8-bit.  So it is really 2+ significant digits.

Message 5 of 5
(542 Views)