Problem searching array

[H]

I have an array in a column that I extracted from a data file. I want
to search the array for particular values and return the index which
is easily implemented by the search 1d array subvi. However, when I
input the values to search, it returns a -1 for some values and others
it finds fine where I know all the values exist. To be more precise,
my array is about 80 values between -2 and 2 in steps of 0.05. When I
search for -2, -1.5, -1, etc., I get the correct index. When I search
for other numbers like -1.8, -1.7, -1.85, etc., I get a return value
of -1. It is really strange and I am not sure of the problem. Anyone
have any ideas? Does it have anything to do with the precision or how
I read the spreadsheet file to extract the column?

T
hanks
Heather

[Mikael_Garcia]

Without knowing details of your program I think what you see is the result of different (binary)representations.

If you use a DBL representation value and compare it with and EXT representation you will see extactly what you see now. For some values it works but for others not. It all comes down to how the numbers are stored in the binary format. To aviod this, make sure your numbers have the same representation. If you still have problems, please add your VI here and I can take a look at it. Hope this helps. /Mikael

[Dennis_Knutson]

It probably has to do with the imprecision of binary representation of floating point numbers in general. It is never a good idea to try to see if one floating point number is equal to another. Try converting to strings or integers, compare, and then convert back. If you need more information, the LabVIEW FAQ at http://www.icon-tech.com.au/ has a topic called "Why does my output show -0.998 instead of -1?" by Scott Hannahs.

[Rob Cole]

As stated by others, the problem comes in how the binary numbers are stored. If you only care about 2 digits after the decimal point, then multiply by 100 and round off. Then compare the numbers. Beyond a certain number of digits past the decimal point, most numbers become meaningless.

An example is that I worked with a program that someone had written where the accuracy of the measurement was +/- 0.5 and the program was storing 8 decimal places. Meaningless digits. Decide on the accuracy required and go with that.

You can also do this by changing to strings and wiring how many digits you need. Then compare the strings.

Rob

[Labviewguru]

Heather,

Unfortunately, this is an issue. I tried to replicate your problem by generating the array of data in LabVIEW. I got the exact same results!!! This is entirely unbelievable on the whole, because I input a starting number of -2.05 and added 0.05 to each instance 80 times. I had the exact same results, that is, until I wired the number I was looking for in from the array itself.

I am sure someone else has already answered this by now (I was interrupted for 8 hours while writing the reply), but here is the answer.

Use fixed point math. Computers are terrible with floating point math. When you input the number you are looking for, it is actually not the number that you think. If you set the precision to 16, you would see that when you, for ex
ample, enter .25, that your number is actually 0.249999164325. So, as you know that you only have two decimal places, muliply by 100, set it to an integer, do your operation, and if necessary, divide by 100 again.

Good luck.

[H]

Thanks for all your help. I ended up converting both the search array
and the search element to strings with 2 decimal point precision and
it works fine.

You guys are the best.
Heather

snshn98@att.net (H) wrote in message news:<4d6ecbad.0109120947.56af5398@posting.google.com>...
> I have an array in a column that I extracted from a data file. I want
> to search the array for particular values and return the index which
> is easily implemented by the search 1d array subvi. However, when I
> input the values to search, it returns a -1 for some values and others
> it finds fine where I know all the values exist. To be more precise,
> my array is about 80 values between -2 and 2 in steps of 0.05. When I
> search for -2, -1.5, -1, etc., I get the correct index. When I
search
> for other numbers like -1.8, -1.7, -1.85, etc., I get a return value
> of -1. It is really strange and I am not sure of the problem. Anyone
> have any ideas? Does it have anything to do with the precision or how
> I read the spreadsheet file to extract the column?
>
> Thanks
> Heather