Showing results for 
Search instead for 
Did you mean: 

frac/exp string to number - shifting decimal point, what's going on?

Go to solution

Trying to convert string to number, i.e. 1.234 string to 1.234 dbl. When I use the frac/exp string to number, sometimes the decimal point gets shifted and sometimes it does not. I pulled a small section out and uploaded it here. Can anyone figure out why it shifts the decimal in one case but not the other? I'm sure it's some simple thing I'm missing. 




0 Kudos
Message 1 of 8

It seems to work perfectly fine for me.


It would've helped if you had saved as default in the two strings the values that were and weren't giving you problems.  When I type in 1.234, it works fine.  But perhaps your issue is that you typed something in wrong, but not obvious.

0 Kudos
Message 2 of 8

I'm not seeing any shift in decimal place. Can you perhaps save some default values that exhibit the problem and upload the vi again?


Edited to add: Looks like RavensFan beat me.

0 Kudos
Message 3 of 8

It works for me.



Did you read the help for the function?

"Interprets the characters 0 through 9, plus, minus, e, E, and the decimal point" Perhaps your string is scientific representation?



EDIT: Guess I am too slow... 2 other people posted before me. LOL

0 Kudos
Message 4 of 8

wow, thanks for the quick responses! I've entered in numbers that are giving me problems and made them default. Here is an updated vi. 


thanks everyone!

0 Kudos
Message 5 of 8
Accepted by topic author mes291

These numbers work fine on mine. The numbers are 1.23456 and 9.70695970695971E-02 (which corresponds to 0.0970695970695971).


Edited to add: You can change the Display Format of the numbers to scientific if you want the numbers to always be x.xxxxxxEy.

Message 6 of 8
Accepted by topic author mes291

Make String #2 wider.  You'll see it says.  9.70695970695971E-02  That would be 0.097....


So no problems!

Message 7 of 8

how embarrassing. Two hours of my life totally wasted on an undersized box.......ugh.


thanks again everyone. 

0 Kudos
Message 8 of 8