LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

VI "Date Time to seconds" manipulates fractional seconds

Entering a fractional second e.g. 15.001 or 59.999 results in a manipulated fracional part if the value
is above ----,008. See attached Example.
0 Kudos
Message 1 of 2
(2,447 Views)
This is just a normal limitation of the precision of floating point numbers in general. Watch what happens if you increase the number of visible digits in your vi:



Please refer to this link for further explanations.

Best regards,

Jochen Klier
National Instruments Germany

Message Edited by Jochen on 01-17-2006 09:31 AM

0 Kudos
Message 2 of 2
(2,424 Views)