03-07-2016 07:30 AM
Ok, i tried looking around the forum for an answer, but wasn't really sure what a similar post may be stored under. I have an application I've written that selects a PO based on the condition of the device being sorted. If there are multiple POs for a particular item, the tool will look at the priority of the PO and if there are any % distributions across this PO. for example PO#1 gets 50% of all volume, PO#2 gets 25%, PO#3 gets 25%. Once one of the POs is fulfilled, it then calculates the remaining %. So in the example if PO#1 is fulfilled, it recalcuates PO#2 and PO#3 (from above) to be 50/50 split until one or both of those POs is fulfilled. It has worked pretty full proof so far - unless my production control does not ensure that all the POs entered total 100%. But once they verify the data they've entered, we haven't had any issues.
This morning I got a call that some devices aren't sorting to a valid location and I stepped through the vi to find what the issue may be. As I've said, if the data entered does not = 1.00 the tool doesn't work. But in this case I have 4 POs with an allocation of 0.8 + 0.07 +0.07+ 0.06 - which does in fact =1. But the comparison to see if that sum is equal to 1.00 is coming up as FALSE! I know sometime when you run sums there can be truncation of some data, but this is the first I've seen this and I'm only dealing with % that max out at 3 decimal places - and these are only at 2. Please help!
I've attached the vi and I've saved the 4 POs to the PO array.
Solved! Go to Solution.
03-07-2016 07:37 AM
When is 1.00 not equal to 1.00?
When you are using a BINARY computer to compare them. https://en.wikipedia.org/wiki/Machine_epsilon
This is a super common "gotcha" and it applies to all computer languages. Basically, you can never trust an equality test on floats using a digital computer so you need to test if the number is within a specific range instead. You can even use the machine epsilon constant in the numeric palette if you need to get right down to the most accurate comparison possible.
03-07-2016 07:41 AM
Also:
http://digital.ni.com/public.nsf/allkb/B01682241DD825948625665100663F61
http://digital.ni.com/public.nsf/allkb/49D72E8658F6AEE48625758A00710BB4?OpenDocument
03-07-2016 07:52 AM - edited 03-07-2016 07:52 AM
If you take your display out enough digits, you will see the inaccuracy. What you really need to do is subtract 1 from your value and check to see if the difference is less than a desired tolerance.
03-07-2016 08:18 AM
Take a look at this thread
03-07-2016 08:19 AM - edited 03-07-2016 08:29 AM
@dbrown78 wrote:Ok, i tried looking around the forum for an answer, but wasn't really sure what a similar post may be stored under.
The problem with comparing IEEE 754 floating point numbers comes up often on the forums but, It might be hard to search unless you've see it before.
One of the more detailed discusions ( with many links to great articles that delve deep into methods) started about here where I posted a library that implements one of the more rugged methods.
If your head isn't spinning by noon, you did not read the source referances or you skillfully wrapped your head in duct-tape prior to diving in to the reading.
03-07-2016 09:43 AM
Thank you everyone for the feedback. Yes, I am now seeing the issue. I was hoping since I was only going to the hundredths place I wouldn't see that issue - guess I assumed incorrectly... doh!!! thanks for the help!