Scale by power of two and negative numbers?

[tartan5]

Hello,

Just need a quick sanity check.............

If I run a negative and scale by a negative power of two, my results seem to be one off?

for example, -1 * 2^-5 should be less than -1, so I was expecting it to return a zero.  Instead I get -1.

When my input hits -33, I get -2, so it looks like it is off?  I was doing this with a big FXP, and wanted to divide by 32 so I thought this would be a fast way of doing it...

Thanks for any input!

Message Edited by tartan5 on 04-18-2008 12:34 PM

[JeffOverton]

whoops, forget it

edit 2 - the help file for the scale by power of two says that if x is an int data type, the result is an arithmetic shift, so you might be losing data to right or left bit shifting

Message Edited by JeffOverton on 04-18-2008 12:46 PM

[johnsold]

From the help for the Scale by Power of 2:


"If x is an integer, this function is the equivalent of an arithmetic shift."


If you change your datatype to DBL, you get the division you were expecting.


Lynn

[tartan5]

I expected this would be an arithmetic shift, I just didn't think it through in binary......

-1 = 1111 1111b (for I8)

-2 = 1111 1110

-3 = 1111 1101

So it is a shift with the sign bit carried in, which for the cases above (if, say, we shift 1 or divide by 2) equals -1, -1, -2.......

Makes sense, thanks!