Using greater than or equal comparison. The equal is not recognized in tenths of a value. Any reason why?

[Revilo]

I've written a routine to ramp voltage and/or current for a GPIB controlled power supply. Using a local variable I monitor the last value set and compare it with a voltage and current limit to ensure neither limit is exceeded. For this comparison I used the Greater than or equal comparison. It seems I've found an oddity I can't exlain.

When I set the voltage or current limits to a whole number value (1.00 or 2.00) the routine works as expected. When I tried incrementing and comparing values in tenths of volts or amps (0.10 the comparison doesn't see the equal portion of the comparison and continues with one more iteration where the greater than catches the di
fference. the final values are then (1.10 or 2.10). If I set the increment to 0.01 the program operates as expected.

Has anyone seen this and been able to determine what the problem is and correct it?

[Shan]

Can you post an example? At first I thought it might be a representation mismatch, but if a 0.01 increment works, it should be fine with any increment.

~Tim

[wiebe@CARYA]

Hi,

I can't tell exactly what you're doing, but it has something to do with the
floating points you can't see.

There are only 32 bits to store the data, so numbers have to be 'converted'
to there closest match. (This is standard, this is not something only
LabVIEW does.)

Try setting the number of floating points to 20. You'll see 1.10 appear as
1.10000000000000009000. But 1.1000000000000009800 also rounds to 1.10. So
they only *appear* equal, but in fact are not.

Using 'Equal' and 'Not Equal' with floats (dubbles, singles, extended in LV)
is always tricky. Try to do the comparation with the In Range And Coarce,
Min & Max, Greater?, Less? etc.

Only use 'Equal' and 'Not Equal' with floats to see if a value has changed,
or to compare with NaN, -Inf and Inf.


Regards,

Wiebe.

"Revilo" wrote in message
news:5065000000080000004B680000-1031838699000@exchange.ni.com...
> Using greater than or equal comparison. The equal is not recognized
> in tenths of a value. Any reason why?
>
> I've written a routine to ramp voltage and/or current for a GPIB
> controlled power supply. Using a local variable I monitor the last
> value set and compare it with a voltage and current limit to ensure
> neither limit is exceeded. For this comparison I used the Greater than
> or equal comparison. It seems I've found an oddity I can't exlain.
>
> When I set the voltage or current limits to a whole number value (1.00
> or 2.00) the routine works as expected. When I tried incrementing and
> comparing values in tenths of volts or amps (0.10 the comparison
> doesn't see the equal portion of the comparison and continues with one
> more iteration where the greater than catches the difference. the
> final values are then (1.10 or 2.10). If I set the incr
ement to 0.01
> the program operates as expected.
>
> Has anyone seen this and been able to determine what the problem is
> and correct it?

[Greg_McKaskle2]

> Has anyone seen this and been able to determine what the problem is
> and correct it?

The problem is that floating point numbers are inexact. This is true in
all languages, and on all processor types as this is the defined IEEE
standard. Specifically, floating point numbers are written as a binary
fraction. Just as 1/3 cannot be well represented in base ten, 1/10
cannot be well represented in base 2. The case where this usually rears
is head is when you take a value, say 0.1, and add it to something else
N times and expect to get the same as adding N/10.

10/10 != 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1

The solution is usually to be aware of the floating point roundoffs and
write the expression another way such as subtracti
ng the two values and
comparing the absolute difference to some epsilon such as 1e-8. If the
numbers are that close, they are considered to be equal. Since the
magnitude of the numbers comes into play, it is somewhat better to check
for zeroes, then divide the numbers and subtract one and compare the
result to epsilon. If either number is zero, then just compare the
other number to epsilon.

I suspect there is a knowledgebase article on this already, and if not,
I know it has been discussed several times before.

Greg McKaskle