64-bit Float to Hex

[Kenny_K]

I am trying to convert a 64 bit number to hex, and the normal TypeCast does not seem to be working.

0.000063092

Using TypeCast to a hex U8 array or U64 , yields a hex value of 0x3F108A0840000000

Using multiple websites, if I convert the number to hex, I get 0x3F108A084E7A83CC

It appears that the top 4 bytes are not being converted in LabVIEW for some reason.

[altenbach · Accepted solution]

Your value is actually 0.0000630919967079535127

Once you show more digits on your input and enter the correct value, the results agree.

(Note that the closest you can enter is 0.0000630919999999999984).

That series of zeroes at your end tells me that your DBL was probably derived from a lower precision representation number. Where does it come from?

[RavensFan]

Change the display format of your constant from 6 significant digits to something higher like 18 digits of precision.

You'll see your constant is actually something else and the rounding to 6 sig. figs. made it look like 0.000063092

[altenbach]

---

@altenbach wrote:

That series of zeroes at your end tells me that your DBL was probably derived from a lower precision representation number. Where does it come from?

---

Yes, as I suspected, your number is actually derived from a SGL, and converting it to DBL does not really give you more significant bits. Here's the proof!

Your original incorrect result is at the bottom.

[Kenny_K]

Interesting.  I didnt realize that If I originally typed the value in a SGL, and then converted the numeric constant to a double, it would still retain the 0 afterwards.

I am trying to program the value 6.30920E-05 into a device.   To write the number to the device, it needs to be converted to HEX.  

Got it now, after I deleted the NC and started with a double first, before I entered the value manually.

[Yamaeda]

Usually the SGLs 7 digit precision is plenty (you only use 6 in your post), but when looking at bit level things like this appears. 🙂

/Y