LabVIEW
U32 to DBL converstion
I am just trying to understand this conversion operation. My understanding was that the conversion to higher data type would be loss less.
Can anyone explain why "8389388" of type U32 gets converted to "8389390" DBL.
Thanks
I think because your indicator is set to show 6 significant digits. The last digit rounds up and you see 8389388 as 8389390
Change your indicator's properties to show 7 (or more significant digits and you will see 'the correct value'.
@Madan wrote:
I am just trying to understand this conversion operation. My understanding was that the conversion to higher data type would be loss less.
Can anyone explain why "8389388" of type U32 gets converted to "8389390" DBL.
Thanks
What exactly were you expecting?
A conversion converts the data from one type into another type and keeps the information at the same value.
If you are interested into a binary transer (meaning that the binary data is the same and only the datatype is converted) have a look at typecast:
Ton
LabVIEW user groep 
@Phillip Brooks wrote:
I think because your indicator is set to show 6 significant digits. The last digit rounds up and you see 8389388 as 8389390
Change your indicator's properties to show 7 (or more significant digits and you will see 'the correct value'.
To elaborate on Philip's (correct) post, there is a difference between how data is displayed and how data is stored.
When you use U32, that is the internal data type,which explicitly dictates the amount of memory LabVIEW should allocate for that object.
Displaying it is just a matter of formatting the number for the user, in this case, set to 6 significant digits.
But again, this does not mean internally LabVIEW threw out that precision, it just isn't displaying it.
