05-08-2013 12:53 PM
Hello,
I am trying to make a program to round a number to the nearest thousandths place. I am seeing extra decimal places, which I didn't add, shown in the input. The screenshot below shows what happens after I entered ".001" as the input and ran the program. This accumulation of digits at far out decimal places is causing me problems in my code.
Thanks
05-08-2013 01:10 PM
This is because of the finite representation of numbers in the computer. 0.001 is an infinitely repeaing fraction in binary. That means that you can never exactly round off a number to 3 decimal places in a binary computer.
If you really need finite representation, multiply your values by 1000 and convert to integers. However, when you divide by 1000 again you will get the same kind of extended fractions for your doubles.
This is true of any computer language unless it is doing BCD arithmetic.
Lynn
05-08-2013 03:11 PM
Hmmm so I guess for any operation I just have to multiply by a power of ten to get all my decimal points on the correct side. Then convert to an integer.
05-08-2013 06:10 PM
@tda*! wrote:
Hello,
This accumulation of digits at far out decimal places is causing me problems in my code.
Thanks
What kind of problems is it causing? As pointed out by others, what you are seeing is an inherent problem in dealing with floating point numbers in a computer's binary system. It generally isn't a problem unless you are trying to use the numbers in a way they weren't meant to be used.
Instead of trying to recreate a computer's way of representing decimal numbers, you might need to solve the problem that your implementation of those numbers is causing.
05-09-2013 09:07 AM
@RavensFan wrote:
@tda*! wrote:
Hello,
This accumulation of digits at far out decimal places is causing me problems in my code.
Thanks
What kind of problems is it causing? As pointed out by others, what you are seeing is an inherent problem in dealing with floating point numbers in a computer's binary system. It generally isn't a problem unless you are trying to use the numbers in a way they weren't meant to be used.
Instead of trying to recreate a computer's way of representing decimal numbers, you might need to solve the problem that your implementation of those numbers is causing.
Ugh, working with FP numbers is so tricky because comparing numbers is never a straightforward task. At least, it's tricky for me.
05-09-2013 10:40 AM
So here is an example or how I'm getting hung up. I round .10003 to .1. Then I subtract it from .6. I try to determine if the difference is divisible by .1 but the numbers don't come out in agreement.
05-09-2013 10:52 AM
@tda*! wrote:
So here is an example or how I'm getting hung up. I round .10003 to .1. Then I subtract it from .6. I try to determine if the difference is divisible by .1 but the numbers don't come out in agreement.
If you round something to the nearest .1 and subtract it from .6, won't it always be dvisible by .1? Or is this just an exercise to show us your problem?
05-09-2013 11:01 AM
@billko wrote:
@tda*! wrote:
So here is an example or how I'm getting hung up. I round .10003 to .1. Then I subtract it from .6. I try to determine if the difference is divisible by .1 but the numbers don't come out in agreement.
If you round something to the nearest .1 and subtract it from .6, won't it always be dvisible by .1? Or is this just an exercise to show us your problem?
Not in Floating Point. since there is no IEEE 754 value that exacly represents 0.1 Multiply by 1000, 10000 (or whaterver you want for precision) convert to integer and do the comparison with integers. You can cast back to float after the comparision
05-09-2013 11:02 AM
0.1 cannot be exactly represented in a binary format. That is the danger of floating point numbers. You will not have exact representation in the factional part unless it is a power of 1/2.
To see this, just put a numeric control down on the front panel, change the display to show something like 30 decimal places, and type in 0.1. You will notice that your 0.1 is not truely 0.1. This is a problem in all computer languages.
05-09-2013 11:11 AM
@JÞB wrote:
@billko wrote:
@tda*! wrote:
So here is an example or how I'm getting hung up. I round .10003 to .1. Then I subtract it from .6. I try to determine if the difference is divisible by .1 but the numbers don't come out in agreement.
If you round something to the nearest .1 and subtract it from .6, won't it always be dvisible by .1? Or is this just an exercise to show us your problem?
Not in Floating Point. since there is no IEEE 754 value that exacly represents 0.1 Multiply by 1000, 10000 (or whaterver you want for precision) convert to integer and do the comparison with integers. You can cast back to float after the comparision
I didn't represent myself clearly. I was wondering about the reasoning behind doing this, not the mechanics of doing it. 🙂