converting decimal string to a number

[krd12]

I am trying to convert a string with a lot of digits (ex 324.41897635) into a number and want to get back the same precision in the number as in the string. I tried the "Decimal String to Number" (get back no digits after decimal point ex 324) and "Frac/Exp String to Number" (get back three decimal places rounded ex 324.419) 

[Darin.K]

For reasons obvious to somebody, a decimal number has no decimal point.  I am guessing that the display format of your numeric indicator is not set to enough digits of precision.  Right-click and select display format to increase the precision.

[Yamaeda]

Chack the properties of the Numeric indicator, by default it shows 6 significant digits. You'll want it to be 15 or so. The number is unaffected though, it's just a presentation thing.

/Y

[muks]

[JÞB]

---

krd12 wrote:
I am trying to convert a string with a lot of digits (ex 324.41897635) into a number and want to get back the same precision in the number as in the string. I tried the "Decimal String to Number" (get back no digits after decimal point ex 324) and "Frac/Exp String to Number" (get back three decimal places rounded ex 324.419) 

---

Not a trivial exercise!

Here is 1 possible solution I've limited it to non-trivial digits (ext floats are only 15 digits precise on some OS's.)  But you can see the steps necessary.  I assume the radix symbol is "." but the vi could be modified to use"System Decimal Seperator"

[Darin.K]

Not to venture too far into Rube country since I think changing the display is what the OP is after.  I like Jeff's thinking, if I were to have a little fun with this I would try the following which also adjusts for scientific notation in the string.

Message Edited by Darin.K on 03-11-2010 02:03 PM

[JÞB]

Thanks Darin.  but the link isnt there. edit-there now

On a side note-  I wanted to use Property>...Numeric.DisplayFormat.Precision but this seems to duplicate  Numeric.displayformat.format

And why not use Numeric.displayformat.format Darin Other than to set "Automatic(0)?"

Did I find a bug?

Message Edited by Jeff Bohrer on 03-11-2010 04:11 PM

[Yamaeda]

 This should be close enough and slightly simpler.

[Darin.K]

---

Jeff Bohrer wrote:

Thanks Darin.  but the link isnt there. edit-there now

On a side note-  I wanted to use Property>...Numeric.DisplayFormat.Precision but this seems to duplicate  Numeric.displayformat.format

And why not use Numeric.displayformat.format Darin Other than to set "Automatic(0)?"

Did I find a bug?

Message Edited by Jeff Bohrer on 03-11-2010 04:11 PM

---

I use the format property in case the format is set to Custom (9) which is read-only and prevents changing the precision.  I also toyed with changing to scientific notation if the string was in scientific notation.

Format and precision are different, both properties are duplicated in the numeric control properties so I am not quite understanding the question.

[JÞB]

I- cannot duplicate but, I was seeing an error when setting the Format.Precision to 9 or above.  I think the format.format WAS set to 9 when this occured.