LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

When is 1.00 not equal to 1.00

Solved!
Go to solution

Ok, i tried looking around the forum for an answer, but wasn't really sure what a similar post may be stored under.  I have an application I've written that selects a PO based on the condition of the device being sorted.  If there are multiple POs for a particular item, the tool will look at the priority of the PO and if there are any % distributions across this PO.  for example PO#1 gets 50% of all volume, PO#2 gets 25%, PO#3 gets 25%.  Once one of the POs is fulfilled, it then calculates the remaining %.  So in the example if PO#1 is fulfilled, it recalcuates PO#2 and PO#3 (from above) to be 50/50 split until one or both of those POs is fulfilled.  It has worked pretty full proof so far - unless my production control does not ensure that all the POs entered total 100%.  But once they verify the data they've entered, we haven't had any issues.  

 

This morning I got a call that some devices aren't sorting to a valid location and I stepped through the vi to find what the issue may be.  As I've said, if the data entered does not = 1.00 the tool doesn't work.  But in this case I have 4 POs with an allocation of 0.8 + 0.07 +0.07+ 0.06 - which does in fact =1.  But the comparison to see if that sum is equal to 1.00 is coming up as FALSE!  I know sometime when you run sums there can be truncation of some data, but this is the first I've seen this and I'm only dealing with % that max out at 3 decimal places - and these are only at 2.  Please help! 

 

I've attached the vi and I've saved the 4 POs to the PO array.  

0 Kudos
Message 1 of 7
(3,631 Views)
Solution
Accepted by topic author dbrown78

When is 1.00 not equal to 1.00?

 

When you are using a BINARY computer to compare them.    https://en.wikipedia.org/wiki/Machine_epsilon

 

This is a super common "gotcha" and it applies to all computer languages.  Basically, you can never trust an equality test on floats using a digital computer so you need to test if the number is within a specific range instead.  You can even use the machine epsilon constant in the numeric palette if you need to get right down to the most accurate comparison possible.

LabVIEW Pro Dev & Measurement Studio Pro (VS Pro) 2019
0 Kudos
Message 2 of 7
(3,619 Views)
Solution
Accepted by topic author dbrown78
0 Kudos
Message 3 of 7
(3,611 Views)
Solution
Accepted by topic author dbrown78

If you take your display out enough digits, you will see the inaccuracy.  What you really need to do is subtract 1 from your value and check to see if the difference is less than a desired tolerance.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
0 Kudos
Message 4 of 7
(3,599 Views)

@dbrown78 wrote:

Ok, i tried looking around the forum for an answer, but wasn't really sure what a similar post may be stored under.  


The problem with comparing IEEE 754 floating point numbers comes up often on the forums but, It might be hard to search unless you've see it before.

 

One of the more detailed discusions ( with many links to great articles that delve deep into methods) started about here where I posted a library that implements one of the more rugged methods.

 

If your head isn't spinning by noon, you did not read the source referances or you skillfully wrapped your head in duct-tape prior to diving in to the reading.Smiley Very Happy


"Should be" isn't "Is" -Jay
0 Kudos
Message 6 of 7
(3,572 Views)

Thank you everyone for the feedback.  Yes, I am now seeing the issue.  I was hoping since I was only going to the hundredths place I wouldn't see that issue - guess I assumed incorrectly... doh!!! thanks for the help!

0 Kudos
Message 7 of 7
(3,531 Views)