Type cast causes bad compare

[Bullet]

Can anyone tell me why these are not equal? See attached vi.

Thanks
Chris

[Johnner]

You should use integer datatypes beginning with the Type Cast type input, and your problems will go away. This is a common, but subtle, gotcha in any programming environment--see comments about the limits of precision with floating-point data representations here and elsewhere in the forums.

As things stand with your VI, if you adjust the digits of precision of your y control upward to 10 or 15, you'll find that you don't have exactly 825, but instead a floating-point value that approaches 825 very closely (slightly above 825, in this case). Ditto with the output of the type cast, except that it happens to approach 825 closely from the negative side. So, the bits underlying the numbers are not quite identical, and the Equal? node is telling the truth.

Since the hex values in your response data are integers, you should have no problem using integers to do your comparisons.

--John

[Bullet]

THANK-----YOU!!!!!!!!!!!!!!!!!!!!

[altenbach]

Comparing the hexadecimal representation before typecasting is not a solution, because presumably your Y values to be compared are SGL and entered on the front panel. If you cast e.g. your 7.5 back to hex, you have the same problem again, not all bits are the same because of limits in the representation.

You simply need to see if the difference between the two numbers is smaller than a certain threshold (1e-4 in my example, but should be adjusted according to the exact requirements). I have attached a modified version of your VI to show one possibility (LabVIEW 7.1).

My VI also shows a much simpler way to extract all your values by casting the entire cluster at once. (You can eliminate all "index array functions", all "join numbers", and most of the typecast functions!).

Your code is in a red box while my suggested alternative is in a blue box. Have a look. Let me know if you have questions.

[Bullet]

Much eaiser, Thanks!