LabVIEW

cancel
Showing results for
Did you mean:

WTH is an I32 precision?

I was struggling with a bin value (Control) of 0.1 that my FP was refusing to register:

so I looked at the display format and changed it to show 2 digits of precision, like so:

Strangely enough, whatever I would enter 0.1, 0.01, it would always reset to 0...

Hmm...

Then I did something really stupid, which is to look at the "Representation" of my control and..tada! it turned out to be an I32  control.

I am asking: why am I actually capable of adjusting the display format of an I32 to show 2 digits of precision?

I mean what is the intent here, besides playing trick with the unsuspecting developer?

The Property "Format" tab for that type is exactly the same as for a float:

Tab and I get this:

After all, I guess an integer with 9 digits of precision is better than one with none:

Message 1 of 23
(3,233 Views)

Re: WTH is an I32 precision?

That is advanced preparation for the fraction integer datatype which will be introduced in LV2547.

Lynn

Message 2 of 23
(3,231 Views)

Re: WTH is an I32 precision?

I want to show DBLs in hex myself.
Message 3 of 23
(3,216 Views)

Re: WTH is an I32 precision?

For the same reason that %d stands for decimal when formatting a string with a number which has no possiblity of having digits after the decimal

disclaimer: I'm sure there is a valid reason for this

Message 4 of 23
(3,200 Views)

Re: WTH is an I32 precision?

@for(imstuck) wrote:

For the same reason that %d stands for decimal when formatting a string with a number which has no possiblity of having digits after the decimal

disclaimer: I'm sure there is a valid reason for this

Actually, there is a very valid reason.  Decimal simply means base 10.  Just the same way that Binary means base 2, Octal means base 8, and Hexadecimal (often shortened to Hex) is base 16.  All of these are dealing with integers.  The %d, %b, %o, and %x have been around at least since C got started.  They have nothing to do with the English name of decimal point.

Plus, look at your string<->Number conversions.  You will see a Decimal String to Number and Number to Decimal String.  Those are just using the %d convension.

There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
Message 5 of 23
(3,177 Views)

Re: WTH is an I32 precision?

@johnsold wrote:

That is advanced preparation for the fraction integer datatype which will be introduced in LV2547.

Lynn

That's just too funny.

Doug

"My only wish is that I am capable of learning each and every day until my last breath."
Message 6 of 23
(3,152 Views)

Re: WTH is an I32 precision?

@Darin.K wrote:
I want to show DBLs in hex myself.

I actually took that seriously for a second and if you actually start from an I32 type and change it to DBL while in the Property Dialog window, you can get there:

Problem is, clicking on hexadecimal (or octal, or binary) doesn't register. The display is stuck wherever it was.

You have to close and reopen the dialog window to get what I think is natural, i.e. grayed out properties that do not make much sense for that type:

But I guess it makes some sense to display an integer with added precision after all ???

Message 7 of 23
(3,143 Views)

Re: WTH is an I32 precision?

All decimal digits of an integer are zero. Why shouldn't you be able to display them if you want to?

Message 8 of 23
(3,128 Views)

Re: WTH is an I32 precision?

It makes a bit more sense when you use the same specifiers with Fixed Point.  Although, I never understood the limitation of not being able to display "." as a true radix point.  I've yet to see an explaiation why 0d1.5 can be displayed "0b1.1"  where "." is the binary point but not an octal point of hexadecimal point

Spoiler
Almost evil of me to dangle so many cosmetic bugs in front on X isn't it?

"Should be" isn't "Is" -Jay
Message 9 of 23
(3,122 Views)

Re: WTH is an I32 precision?

@altenbach wrote:

All decimal digits of an integer are zero. Why shouldn't you be able to display them if you want to?

Wait. What?

All digits after the decimal point are zero.

Message 10 of 23
(3,110 Views)