LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

String to Number Significant Digits

Group,
I am received data from a GPIB device that is in string form. When I use
the Engineering/Fractional String to Number conversion Vi I only get two
decimal places of accuracy.

eg: Input String: -15.67834 Output Number from Conversion: -15.68

How can I increase the significant digits while converting the data?

Regards,
Seth
0 Kudos
Message 1 of 2
(2,676 Views)
Seth Fuller wrote:
> Group,
> I am received data from a GPIB device that is in string form. When I use
> the Engineering/Fractional String to Number conversion Vi I only get two
> decimal places of accuracy.
>
> eg: Input String: -15.67834 Output Number from Conversion: -15.68
>
> How can I increase the significant digits while converting the data?
>
> Regards,
> Seth
>
>


This might be a display problem. The precision property of an indicator
does not change the precision of the double type. Click on the indicator
to access the 'format and precision' sub-menu and change the number of
sig. digits to 6.
0 Kudos
Message 2 of 2
(2,676 Views)