Instrument Control (GPIB, Serial, VISA, IVI)

cancel
Showing results for 
Search instead for 
Did you mean: 

Number Precision - Close the Loop

I'm incrementing the Start Number till it reaches the Stop Number.

 

Apparently there are some number resolution that ruin my idea of closing the loop.

 

7.20000000000000155632 

 

I use 'not equal' to compare numbers and make 'false' to close the loop.

 

My loop closes at 7.2.  

 

What would be the most elegant solution for this?

 

I tried using greater or equal but that works most of the time.

 

Thanks,

 

Jovan

 

 

 

 

 

0 Kudos
Message 1 of 2
(3,033 Views)
Hi Jovan

Floating point numbers are not exact and comparing those should always be done in >= or <= instead of equal.
The difference can be (depending on the representation and value) very big or very small.
That is why they are called floating point. You don't get fixed resolution but depending on the value you get more (small value) or less (big value) numbers after the decimal point.
If you have a small number like 7.1 you will be tortured by very small differences.
(try displaying them with at least 20 digits after the floating point and see what happens)

greetings from the Netherlands
0 Kudos
Message 2 of 2
(3,020 Views)