11-29-2011 01:26 PM
Hi -
I am trying to convert a string in scientific notation into a double. I first use a "string to number" conversion then a "to double" conversion. When I probe the values my string is "+050.00E-03" but the number is "50". I have yet to add the "to double" portion. I tried to find where and how the string is being written but was unable to find it so that was a bust. I think it is in the instrument drivers.
Any suggestions?
Thank you.
Solved! Go to Solution.
11-29-2011 01:31 PM
Exactly what 'string to number' function? The Fract/Exp String to Number works perfectly fine, as would a Scan From String. No need for the to DBL.
10-02-2017 11:38 AM
Hello i have the same problem, i did it with fract function, but the number has only 4 decimals, how i can do for have all decimals??? My format is "7.38389053e1" and in the output of fract function it appears 73,8389 but i need the last 3 decimals to.
Thanks for helping!
10-02-2017 12:03 PM - edited 10-02-2017 12:05 PM
@Erizosinpuas wrote:
My format is "7.38389053e1" and in the output of fract function it appears 73,8389 but i need the last 3 decimals to.
Change the display format of the indicator to show more digits. (right-click...display format).
The precision of DBL is independent of the display format