LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

trig wrong result

Hi. I'm sure that is a simple representation setting problem. Can anyone help me? The PI number it appear to be in other representation than the result from the arctan function. The subtract shoul be 0 theoreticaly.

Thank you.

 

 

0 Kudos
Message 1 of 5
(2,343 Views)

Hi sorinpetrila,

 

The subtract shoul be 0 theoreticaly.

Yes, you're right - in a perfect world.

 

But using floating point numbers isn't far from perfect! Please read the basics on using floats and their shortcomings - there are nice articles on Wikipedia…

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 2 of 5
(2,327 Views)

Thank you for the fast reply.  But how can I fix this? This subtraction affects other results that come after.

0 Kudos
Message 3 of 5
(2,318 Views)

Hi sorinpetrila,

 

you cannot FIX this issue - you have to deal with it!

 

There are so many threads on the topic of floating point precision (aka "rounding errors" aka "unexpected results" aka "why does adding 0.1 ten times doesn't result in 1.0" aka …), please read them! Whenever you use floating point numbers to calculate any numbers you are responisble for paying attention to imminent rounding errors!

 

In your case a simple comparison may work:

IF abs(result(subtraction)) < needed precision
  THEN result = 0
ENDIF

 

Another good description of the topic

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 4 of 5
(2,309 Views)

Thank you. I was searching for this answer.

Best regards.

Sorin

0 Kudos
Message 5 of 5
(2,283 Views)