LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

How do you guys deal with floating point errors

Solved!
Go to solution

Hi guys,

I was wondering how do people deal with floating-point errors? Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue? When precision matters, after X amount of computations you inevitably end up with an answer that is off by a certain amount. For example, if I subtract 0.2 from 2.0 10 times in a row, the answer is not zero. It's close to zero but not zero and when you compare two numbers that's a big issue. If you scale numbers, that's a big issue as well. Any thoughts? 

0 Kudos
Message 1 of 13
(2,634 Views)

Hi NDT,

 


@datatechNDT wrote:

I was wondering how do people deal with floating-point errors? 


Ignore them! 😄

 


@datatechNDT wrote:

Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue?


What's the datasource?

When using DAQ data you end up with 16-24bit of information per sample: this gives you <=7.5 decimal digits. Why care about small issues in the 12th+ digit? (I guess you are talking about DBL/EXT values when calling them "floats".)

For anything else (CSV files, any kind of reports) than data storage (like TDMS files) I round the output values…

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 2 of 13
(2,628 Views)

I meant Single Precision Point when I was talking about a "float". And the error becomes an issue only when I compare two numbers or I scale a number. If I get a number thats 0.000000002980232239 (due to floating point error) and I drop the "is equal to zero", my answer is false even though it should be true. the particular case I encountered that was when i subtracted 0.2 from 2.0 10 times in a row, where you expect the answer to be zero... it wasnt zero.. took me ages to find out what was wrong :D. So I guess i need to write helper functions to compare two numbers if they are equal, where I compare the difference between the numbers and just never use the equality statement?


The particular example in which I got the error was when I did an FFT of a wave that was saved as an array of Single Precision Point, then calculated the Total Harmonic Distortion, and I knew that the answer was zero as there are no harmonics.. the answer was not zero it was 0.0145..

0 Kudos
Message 3 of 13
(2,620 Views)
Solution
Accepted by topic author datatechNDT

Hi NDT,

 


@datatechNDT wrote:

 If I get a number thats 0.000000002980232239 (due to floating point error) and I drop the "is equal to zero", my answer is false


Simple rule: NEVER compare floats for equality! REALLY!!!

 


@datatechNDT wrote:

I meant Single Precision Point when I was talking about a "float".


Why do you use SGL data?

Is there a specific reason? (The only reason I know of is LabVIEW-FPGA - and even then you can replace most of that math by using FXP data.)

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 4 of 13
(2,613 Views)

I was using SGL, as I was concerned about memory. I am processing a wave that has 10Million points in it, and I thought that a Double(64bits) might be too extreme.

0 Kudos
Message 5 of 13
(2,595 Views)

You should determine a tolerance for that will meet your accuracy requirements and use that. Don't compare for equality but rather for in range. For you subtraction example, check to see if your two values are within 0.0001 (or whatever tolerance is appropriate) of each other.



Mark Yedinak
Certified LabVIEW Architect
LabVIEW Champion

"Does anyone know where the love of God goes when the waves turn the minutes to hours?"
Wreck of the Edmund Fitzgerald - Gordon Lightfoot
Message 6 of 13
(2,582 Views)

@Mark_Yedinak wrote:

You should determine a tolerance for that will meet your accuracy requirements and use that. Don't compare for equality but rather for in range. For you subtraction example, check to see if your two values are within 0.0001 (or whatever tolerance is appropriate) of each other.


Yes, this is the way I do it.  You can raise both sides of the comparison by 10^x where x is the amount of decimal places, convert them to integers, do your comparison.  (It also works for places to the left of the decimal place as well.  To compare to the nearest ten, for example, use -1.  Useful for when you want to compare to the nearest MHz, for example.)

Bill
CLD
(Mid-Level minion.)
My support system ensures that I don't look totally incompetent.
Proud to say that I've progressed beyond knowing just enough to be dangerous. I now know enough to know that I have no clue about anything at all.
Humble author of the CLAD Nugget.
0 Kudos
Message 7 of 13
(2,571 Views)
Solution
Accepted by topic author datatechNDT

Rounding Numbers to Various Decimal Places in LabVIEW - NI Community
Works nicely

CLD; LabVIEW since 8.0, Currently have LabVIEW 2015 SP1, 2018SP1 & 2020 installed
Message 8 of 13
(2,516 Views)

@James_W wrote:

Rounding Numbers to Various Decimal Places in LabVIEW - NI Community
Works nicely


Yes, that's exactly what I described, except after rounding to integer, I also coerce to integer, even though it's probably not needed because i believe that whole numbers are always represented exactly by FP.

Bill
CLD
(Mid-Level minion.)
My support system ensures that I don't look totally incompetent.
Proud to say that I've progressed beyond knowing just enough to be dangerous. I now know enough to know that I have no clue about anything at all.
Humble author of the CLAD Nugget.
0 Kudos
Message 9 of 13
(2,486 Views)
Solution
Accepted by topic author datatechNDT

@datatechNDT wrote:

Hi guys,

I was wondering how do people deal with floating-point errors? Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue? When precision matters, after X amount of computations you inevitably end up with an answer that is off by a certain amount. For example, if I subtract 0.2 from 2.0 10 times in a row, the answer is not zero. It's close to zero but not zero and when you compare two numbers that's a big issue. If you scale numbers, that's a big issue as well. Any thoughts? 


Over here is @ message 14 is a zipped up project with polymorphic instances for floating point comparisons with a tolerance based on ULP (Units in Last Place).

 

The polymorphic instances could be improved by converting to *.vim (Malleable vis) but work very well.  The ULP method has several advantages over tolerance by %, log10 precision, or significant digit.


"Should be" isn't "Is" -Jay
0 Kudos
Message 10 of 13
(2,472 Views)