trim to two decimal from double datatype

[பரத்_குமார்(bharathkumar)]

Hello,

I am trying to trim the decimal places in a number. ( say 1.2345 to 1.23). I have tried the method of number to fractional string and frac/Exp string to number. Since I am working on RT timing needs more precision digits(comparison is involved for time). Increasing the digits to 17 does not result in desired value. attached VI

Thanks,

Bharath

[paul_cardinale]

The "precision" input specifies the number of decimal places to the right of the decimal point.

You could use "Number to exponential String", but converting it back to a double will result in conversion errors that leave non-zero values in the low order bits.  Maybe you just want to convert to s Single, or perhaps a Fixed Point number.

[GerdW]

Hi Bharat,

Since I am working on RT timing needs more precision digits(comparison is involved for time).

So you need more precision by reducing the number of relevant digits in the value???

What about using integer values like number of microseconds for your RT timing problem?

(Or integer number of ticks in a FPGA target?)

Increasing the digits to 17 does not result in desired value. attached VI

This will NEVER change with float values!

 Floats have a limited resolution and CANNOT store each value correctly.

When showing 17+ digits you will always notice rounding errors of floats!

[billko]

If you want to compare two FP numbers to a certain precision, then you need to multiply each number by 10 to the power of how many decimal places and then round the numbers.  Compare those two numbers instead.  (It even works for comparing to places left of the decimal point - just use a negative exponent.)

Edit:  I must've taken too many cold capsules at the same time.  It seems my response is only peripherally related to the subject.  Sorry!

[altenbach]

Many exact decimal fractions (e.g. 0.1) cannot be represented in binary unless you have an infinite number of bits ... and you don't!!! Comparing floating point numerics is always very dangerous. All computations are done in binary. "Decimal" is just a convenient cosmetic representation of the internal data that has limitations. Since it is only cosmetic, simply change the display format of the array elements accordingly.

In your VI, you have two lossy conversions. First you format the DBL into a string rounded to the nearest value with two decimal digits. The you scan it and it will get rounded to the nearest possible binary DBL value.

You could do all computations on integers with values 100x higher and scale it back at the very end and for display. Or you could compare the formatted strings, which is computationally more expensive of course.

[Minions]

Your comment about the float to number conversion not limiting to 2 digits is actually incorrect.  Either probe the array of strings or set an indicator and see.  It is actually the conversion back to a float from the string in your number array indicator that displays your issue.  It has the understandable issue if one understands the IEEE-754 conventions for floats or doubles.

Seeing how the float to string is correct (with 2 digits of precision), compare strings and NOT the numbers.

[பரத்_குமார்(bharathkumar)]

Hello,

I tried the NI solution to remove the digits after decimal. this was the method 1 suggested. I guess as the significant digits after decimal increases, it is safer to compare after converting to integers. And back to double for display purpose.

[GerdW]

Hi Bharath,

it is safer to compare after converting to integers.

It's not just "safer", it's the only safe way…