Machine Epsilon must depend on the representation and architecture

[altenbach]

In the real world, machine epsilon is a function of the binary representation of a floating point number.

The labview help describes it as:

---

Machine Epsilon

"Represents the round-off error for a floating-point number with a given precision. Use the machine epsilon constant to compare whether two floating-point numbers are equivalent."

---

From the term "given precision", we would assume that epsilon depends on the representation. In fact, we can right-click on the machine epsilon and select between SGL, DBL, and EXT.

However, if we look at the actual value, we can see that machine epsilon has the identical decimal value for SGL, DBL, and EXT. No matter what representation we chose, we get the value for DBL.

This is not right!

Suggestion: the machine epsilon must depend on the representation. Since the exact representation of EXT depends on the architecture (64, 80, 96, 128 bits total), machine epsilon for EXT needs to adapt accordingly.

Here's one possible way to calculate machine epsilon explicitly. Note the discrepancy for SGL and EXT.

[Darin.K]

I agree in principle, although the context help says:

The smallest number, in double precision, which when added to 1 gives a number different than 1.

[altenbach]

OK, this is another place where context help and LabVIEW help disagree. 😉

If only the DBL value is actually available, we should not be able to select a representation for the constant. It should be fixed at DBL and show a coercion dot when used with other types to warn the programmer that things might be not as expected.

If I e.g. do computations in SGL or EXT, the machine epsilon value of DBL is meaningless!

[Darin.K]

I agree that it should either give the correct value based on representation, or not allow the change of representation.

Add this to my list of reasons why I think hard before doing calculations in either SGL or EXT.

[JLewis]

This works in LabVIEW 2010. There is a nice article here (04/26/16 - Admin Note - Content has been removed from ni.com - Karina Barles) that makes use of it to analyze various floating-point behaviors.

[altenbach]

> This works in LabVIEW 2010

Sorry, I don't know what yo mean here in the context of this idea.

What is "this"? What is "works"?

Even in LabVIEW 2010, the machine epsilon constant does not return an appropriate value if the representation is NOT DBL.

[dthor]

Altenbach, try switching the Machine Epsilon to a different representation and back again. Then your VI works (at least in 2010). So there is some kind of bug where the ME value isn't being updated properly when changing the representation.

[altenbach]

Ah, I see! Great! So it's not a code problem but an update bug. I'll file a bug report instead.

[altenbach]

I guess this is idea is already implemented. However, old code containing non-DBL ME constants, do not correctly update to the new values.

This might be a minor flaw and actually as expected in order not to change functionality of existing code.

(I won't file a bug report after all.)

[MathGuy]

Yes. This behavior has been known since the introduction of the constant in 8.0 but the architecture used to define constants was unable to allow a different value to be associated with a representation.

Becuase of some new features internally, we were able to work around this and fix it in 2010. It was not a good candidate for automated conversion in 2010 as using the constant in a non-DBL precision may or may not have been the intent of the programmer.

You'll find information on the change in a recent DevZone article I wrote which addresses this change in context:

An Introduction to Floating-Point Behavior in LabVIEW (04/26/16 - Admin Note - Content has been removed from ni.com - Karina Barles) 

Based on this post and others you've had, you pay close attention to this. I would appreciate any feedback you have regarding the document or on this topic in general.

Darren

[altenbach]

Hi Darren,

I remember reporting this, probably in early 8.0 (or beta) and I am glad it has been fixed for 2010. I think the new help and context help is good. I guess Darin.K above quoted from an older version of the context help.

I am going to be mostly on the road today, so I'll have a look at your linked document tomorrow. Looks great at first glance. 😄