LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

WTH is an I32 precision?

I was struggling with a bin value (Control) of 0.1 that my FP was refusing to register:

 

ScreenHunter_002.jpg

 

so I looked at the display format and changed it to show 2 digits of precision, like so:

 

ScreenHunter_001.jpg

 

Strangely enough, whatever I would enter 0.1, 0.01, it would always reset to 0...

Hmm...

Then I did something really stupid, which is to look at the "Representation" of my control and..tada! it turned out to be an I32  control.

 

I am asking: why am I actually capable of adjusting the display format of an I32 to show 2 digits of precision?

I mean what is the intent here, besides playing trick with the unsuspecting developer?

The Property "Format" tab for that type is exactly the same as for a float:

 

ScreenHunter_002.jpg 

 

 

Tab and I get this:

 

 

ScreenHunter_004.jpg

 

After all, I guess an integer with 9 digits of precision is better than one with none:

 

ScreenHunter_005.jpg

 

 

 

0 Kudos
Message 1 of 23
(3,922 Views)

That is advanced preparation for the fraction integer datatype which will be introduced in LV2547.

 

Lynn

Message 2 of 23
(3,920 Views)
I want to show DBLs in hex myself.
0 Kudos
Message 3 of 23
(3,905 Views)

For the same reason that %d stands for decimal when formatting a string with a number which has no possiblity of having digits after the decimal Smiley Very Happy

 

disclaimer: I'm sure there is a valid reason for this

Message 4 of 23
(3,889 Views)

@for(imstuck) wrote:

For the same reason that %d stands for decimal when formatting a string with a number which has no possiblity of having digits after the decimal Smiley Very Happy

 

disclaimer: I'm sure there is a valid reason for this


Actually, there is a very valid reason.  Decimal simply means base 10.  Just the same way that Binary means base 2, Octal means base 8, and Hexadecimal (often shortened to Hex) is base 16.  All of these are dealing with integers.  The %d, %b, %o, and %x have been around at least since C got started.  They have nothing to do with the English name of decimal point.

 

Plus, look at your string<->Number conversions.  You will see a Decimal String to Number and Number to Decimal String.  Those are just using the %d convension.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
Message 5 of 23
(3,866 Views)

@johnsold wrote:

That is advanced preparation for the fraction integer datatype which will be introduced in LV2547.

 

Lynn


That's just too funny.

Doug

"My only wish is that I am capable of learning each and every day until my last breath."
0 Kudos
Message 6 of 23
(3,841 Views)

@Darin.K wrote:
I want to show DBLs in hex myself.

I actually took that seriously for a second and if you actually start from an I32 type and change it to DBL while in the Property Dialog window, you can get there:

 

ScreenHunter_001.jpg

 

Problem is, clicking on hexadecimal (or octal, or binary) doesn't register. The display is stuck wherever it was.

You have to close and reopen the dialog window to get what I think is natural, i.e. grayed out properties that do not make much sense for that type:

 

ScreenHunter_002.jpg

 

But I guess it makes some sense to display an integer with added precision after all ???

 

 

0 Kudos
Message 7 of 23
(3,832 Views)

All decimal digits of an integer are zero. Why shouldn't you be able to display them if you want to?

Message 8 of 23
(3,817 Views)

 

It makes a bit more sense when you use the same specifiers with Fixed Point.  Although, I never understood the limitation of not being able to display "." as a true radix point.  I've yet to see an explaiation why 0d1.5 can be displayed "0b1.1"  where "." is the binary point but not an octal point of hexadecimal point

Example_VI_FP.png

Spoiler
Almost evil of me to dangle so many cosmetic bugs in front on X isn't it?Smiley Wink

"Should be" isn't "Is" -Jay
0 Kudos
Message 9 of 23
(3,811 Views)

@altenbach wrote:

All decimal digits of an integer are zero. Why shouldn't you be able to display them if you want to?


Wait. What?

 

All digits after the decimal point are zero. 

Message 10 of 23
(3,799 Views)