LabVIEW

cancel
Showing results forΒ 
Search instead forΒ 
Did you mean:Β 

How do you guys deal with floating point errors

Solved!
Go to solution

Hi guys,

I was wondering how do people deal with floating-point errors? Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue? When precision matters, after X amount of computations you inevitably end up with an answer that is off by a certain amount. For example, if I subtract 0.2 from 2.0 10 times in a row, the answer is not zero. It's close to zero but not zero and when you compare two numbers that's a big issue. If you scale numbers, that's a big issue as well. Any thoughts? 

0 Kudos
Message 1 of 13
(5,963 Views)

Hi NDT,

 


@datatechNDT wrote:

I was wondering how do people deal with floating-point errors? 


Ignore them! πŸ˜„

 


@datatechNDT wrote:

Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue?


What's the datasource?

When using DAQ data you end up with 16-24bit of information per sample: this gives you <=7.5 decimal digits. Why care about small issues in the 12th+ digit? (I guess you are talking about DBL/EXT values when calling them "floats".)

For anything else (CSV files, any kind of reports) than data storage (like TDMS files) I round the output values…

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 2 of 13
(5,957 Views)

I meant Single Precision Point when I was talking about a "float". And the error becomes an issue only when I compare two numbers or I scale a number. If I get a number thats 0.000000002980232239 (due to floating point error) and I drop the "is equal to zero", my answer is false even though it should be true. the particular case I encountered that was when i subtracted 0.2 from 2.0 10 times in a row, where you expect the answer to be zero... it wasnt zero.. took me ages to find out what was wrong :D. So I guess i need to write helper functions to compare two numbers if they are equal, where I compare the difference between the numbers and just never use the equality statement?


The particular example in which I got the error was when I did an FFT of a wave that was saved as an array of Single Precision Point, then calculated the Total Harmonic Distortion, and I knew that the answer was zero as there are no harmonics.. the answer was not zero it was 0.0145..

0 Kudos
Message 3 of 13
(5,949 Views)
Solution
Accepted by topic author datatechNDT

Hi NDT,

 


@datatechNDT wrote:

 If I get a number thats 0.000000002980232239 (due to floating point error) and I drop the "is equal to zero", my answer is false


Simple rule: NEVER compare floats for equality! REALLY!!!

 


@datatechNDT wrote:

I meant Single Precision Point when I was talking about a "float".


Why do you use SGL data?

Is there a specific reason? (The only reason I know of is LabVIEW-FPGA - and even then you can replace most of that math by using FXP data.)

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 4 of 13
(5,942 Views)

I was using SGL, as I was concerned about memory. I am processing a wave that has 10Million points in it, and I thought that a Double(64bits) might be too extreme.

0 Kudos
Message 5 of 13
(5,924 Views)