Data type in labview 6.1

[Se]

Hi,
I want to make a value with a lot of decimal poits e.g. 0.0000056 to be the default value. I tried to change the data type to extended precision but every time I right click on the object to "make value as default" it will return 0.00 back. How can I fix that?

[unDees]

Right-click on the object and select 'Format and Precision....' This will allow you to specify 7 decimal places (or however many you need). You may also have to resize the control to make it big enough for all those digits to show up.

[Se]

Thanks Ian.

[mikeporter]

Also be aware that if you are running Windows you might not get all the precision you need regardless of the datatype you use--especially if you need "lots" of decimal points.

Mike...

[Se]

Is it ok with 15 decimal points?

[mikeporter]

The last time I specifically checked this was a year or so, but I believe the limit was 7. Note that this limit doesn't exist with the Mac or UNIX versions of LV.Mike...

[unDees]

In LabVIEW 6.1, I just tried typing in pi to many decimal places, and made it to 17 before LabVIEW started truncating it. Of course, due to the way that floating-point numbers are represented internally, in general very large numbers will have even fewer available decimal places (unless you use scientific notation).

[mikeporter]

Also, I checked the number with 6.0, maybe NI fixed the problem with 6.1. Or maybe I got the "7" right but forgot the "1"... Oh well, age does that to you.

Mike...