lost Digits of big Number - convert Double into Hex

[MaikThierer]

My Number is 24563383270390080

if i debug the Variable it has got a value of 2.45633832703901e+016, so i lost a little bit of the value.
if i bring ist into a sting, i can see the complete value "24563383270390080"

i have to convert the number in to a Hex-Value, the Reuslt should be "$57444456503941" an can be written into a string.

my problem is to bring the original value into a string of hex, because diadem round the last digits of my number
i tryed several convertions, but the the probmel of lost digits is still there....

maybe you´ve got an idea 🙂

*****************************************************

dim Number, NumberString, HexTest1, HexTest2, HexTarget

Number = 24563383270390080  '17 Digits
HexTarget = "$57444456503941"

NumberString = Str(Number, "d")

HexTest1  = Str((Data.Root.ChannelGroups("Neu").Channels("Vin_Data").Values(1)), "$")
HexTest2  = Str(Val(Number), "$")

'---> Result

''--> Number       = 2.45633832703901e+016
'--> NumberString = "24563383270390080"

'--> HexTarget  = "$57444456503941"  
'--> HextTest1  =  $57444456503940  !!!!! Last Postion !!!!
'--> HextTest2  =  $57444456503940

[AndreasK]

You are a little bit out of luck with your example because

dim number : Number = 24563383270390080  '17 Digits
dim txt
txt = txt & Str(Number, "d") & " ref: 24563383270390080" & VBCRLF
txt = txt & Str(Number, "$") & " ref: $57444456503940" & VBCRLF
MsgBox txt

 0 at the end is correct. (At least the MS calculator says so.) But this is by incident because of the number you picked.

VBS only supports 64 bit double values. 64 bit doubles have a mantissa of 52 bits.

24563383270390080

becomes

1010111010001000100010001010110010100000011100101000000

which has 55 bit in use. Because your last bits aren't in use the number you picked can be represented by a double without loosing precission.

24563383270390081

would show the behavior you described because it is using 55bit range. The 1 is just cut when converted to double.

But I have no solution for this because it will not work with VBS.

[Brad_Turpin]

Hi MaikThierer,

Both DIAdem and VBScript are limited to mapping hex strings to long integers-- that is, a signed 4 byte integer, which can contain up to 8 hex characters.  I recommend that you parse your incoming hex strings into 4 character segments and convert each to a long integer-- using exactly the first half of the long integer to avoid the signed bit issues.  It is possible then to multiply each resulting integer by its base value and add them together to reconstruct the overall decimal value, but DIAdem and VBScript will store this in a double, so if your hex string is too big it won't fit in the double.  In that case you just need to keep the parts separate.

Sorry for the hassle,

Brad Turpin

DIAdem Product Support Engineer

National Instruments