LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Using ini-files with write-key/read-key (double) and defined accurarcy

Solved!
Go to solution

OnlyOne,

 

Regardless of how you have configured the numeric control display format, the value on the wire uses the full double precision representation of what was entered. If you expand the display to show >16 digits after the decimal separator you will find that many 2-digit values are not exact. 

 

The config filres are text files so only the number of digits specified for the format are actually saved in the file. When the file is read and the values are restored to the control, it will be as though the user typed the digits in the config file into the control.  

 

Enter 1.53 and the value on the wire is 1.5300000000000000300. 1.531 => 1.5309999999999999200.  1.532 => 1.5320000000000000300.

 

Lynn

0 Kudos
Message 11 of 11
(388 Views)