labview precision

[todd_westley]

I am trying to a read value (39.198943) from a text file.  Labview reports the value as 39.1989.  I wrote a simple VI to explore this issue.  When I add 0.198943 to 39 the answer reported is 39.1989.  There must be some setting somewhere that I do not have correct.  But I can't figure out how to make this problem go away.

[RavensFan]

Your display format is probably set to 6 significant figures.  Try increasing that or setting it to 6 digits of precision.

[todd_westley]

What exactly do mean by "display precision?"  I changed the "representation" of the indicators with no improvement.  A screenshot is found on the third slide.

[altenbach]

right-click...display format... (nothing to do with representation).

This is a purely cosmetic issue and has nothing to do with the underlying data in the wire.

[RTSLVU · Accepted solution]

---

@todd_westley wrote:

What exactly do mean by "display precision?"  I changed the "representation" of the indicators with no improvement.  A screenshot is found on the third slide.

---

[RavensFan]

I never used the phrase "display precision".

I said display format.  You have two ways of selecting the display.  And RTSVLU shows both of them.  One is the number of significant digits.  (I used the common phrase (significant figures".)  Your was set for 6, which is why you got 39.1234    If you set it for digits of precision, then you can get 39.123456, the number of digits of precision after the decimal point.

Click on the help button in that dialog to learn more.

[BowenM]

---

@todd_westley wrote:

What exactly do mean by "display precision?"  I changed the "representation" of the indicators with no improvement.  A screenshot is found on the third slide.

---

Just FYI, please post images or vis directly, not in a PowerPoint slideshow.  For front panel images you can use the Windows snip tool, and for code you can either just post your vi or create a snippet and post that image.

Edit:  I almost forgot... or you can be fancy and use the LAVA snippet tool and do both at the same time!

[altenbach]

---

@BowenM wrote:

---

@todd_westley wrote:

What exactly do mean by "display precision?"  I changed the "representation" of the indicators with no improvement.  A screenshot is found on the third slide.

---

Just FYI, please post images or vis directly, not in a PowerPoint slideshow.  For front panel images you can use the Windows snip tool, and for code you can either just post your vi or create a snippet and post that image.

Edit:  I almost forgot... or you can be fancy and use the LAVA snippet tool and do both at the same time!

---

Or even simpler, just attach the VI (possibly after making the current values the default). 😄