Rounding Off Issue in Double to String

[JLewis · Accepted solution]

---

@Karthik_Abiram wrote:

LabVIEW should be better handling these things.

So, is it not possible to represent a number like 488.345(488.34500000000003) as 488.345 exactly?

---

I agree that the behavior you're seeing is a bug. 488.345 is not exactly representable in floating-point, but 488.375 is, and the truncation it exhibits is inconsistent with other numbers that round half to even.

Here's a very simple example involving exact storage values:
to 1 digit,  1.75 --> 1.8
to 2 digits, 1.375 --> 1.37

I'm not aware of any rounding mode that should behave that way (other than randomized!). I've filed CAR 515549 on this.

[Karthik_Abiram]

---

@JLewis wrote:

I agree that the behavior you're seeing is a bug. 488.345 is not exactly representable in floating-point, but 488.375 is, and the truncation it exhibits is inconsistent with other numbers that round half to even.

Here's a very simple example involving exact storage values:
to 1 digit,  1.75 --> 1.8
to 2 digits, 1.375 --> 1.37

I'm not aware of any rounding mode that should behave that way (other than randomized!). I've filed CAR 515549 on this.

---

I agree, and thanks for filing the CAR!