LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

64-bit Float to Hex

Solved!
Go to solution

I am trying to convert a 64 bit number to hex, and the normal TypeCast does not seem to be working.

 

0.000063092

 

Using TypeCast to a hex U8 array or U64 , yields a hex value of 0x3F108A0840000000

 

Using multiple websites, if I convert the number to hex, I get 0x3F108A084E7A83CC

 

It appears that the top 4 bytes are not being converted in LabVIEW for some reason.

 

 

Kenny

0 Kudos
Message 1 of 6
(573 Views)
Solution
Accepted by topic author Kenny_K

Your value is actually 0.0000630919967079535127

 

Once you show more digits on your input and enter the correct value, the results agree.

(Note that the closest you can enter is 0.0000630919999999999984).

 

That series of zeroes at your end tells me that your DBL was probably derived from a lower precision representation number. Where does it come from?

 

altenbach_0-1585070640090.png

 


LabVIEW Champion. It all comes together in GCentral GCentral
What does "Engineering Redefined" mean??
0 Kudos
Message 2 of 6
(563 Views)

Change the display format of your constant from 6 significant digits to something higher like 18 digits of precision.

 

You'll see your constant is actually something else and the rounding to 6 sig. figs. made it look like 0.000063092

0 Kudos
Message 3 of 6
(560 Views)

@altenbach wrote:

That series of zeroes at your end tells me that your DBL was probably derived from a lower precision representation number. Where does it come from?


Yes, as I suspected, your number is actually derived from a SGL, and converting it to DBL does not really give you more significant bits. Here's the proof!

 

Your original incorrect result is at the bottom.

 

altenbach_0-1585071103147.png

 


LabVIEW Champion. It all comes together in GCentral GCentral
What does "Engineering Redefined" mean??
Message 4 of 6
(542 Views)

Interesting.  I didnt realize that If I originally typed the value in a SGL, and then converted the numeric constant to a double, it would still retain the 0 afterwards.

 

I am trying to program the value 6.30920E-05 into a device.   To write the number to the device, it needs to be converted to HEX.  

 

Got it now, after I deleted the NC and started with a double first, before I entered the value manually.

Kenny

0 Kudos
Message 5 of 6
(503 Views)

Usually the SGLs 7 digit precision is plenty (you only use 6 in your post), but when looking at bit level things like this appears. 🙂

/Y

G# - Award winning reference based OOP for LV, for free! ADDQ VIPM Now on GitHub
"Only dead fish swim downstream" - "My life for Kudos!" - "Dumb people repeat old mistakes - smart ones create new ones."
Certified-LabVIEW-Developer
0 Kudos
Message 6 of 6
(445 Views)