12-28-2014 06:00 AM
hi
im having a slight issue converting ascii to decimal.
i have an ascii string which is basically an incrementing number.
when i look at it as a hex string (by setting hex code in an indicator) i get:
0001
0002
0003
and so on.
however, when i convert this ascii string to decimal i get really large numbers.
when i get 0002 the decimal number is 131072.
this is equivalent to 20000 hex.
so it appears that i have 4 trailing hex zeros in there. not sure where they come from or how to get rid of them.
any ideas anyone?
12-28-2014 06:12 AM
12-28-2014 06:20 AM
brilliant thanks
working
12-28-2014 02:45 PM - edited 12-28-2014 02:54 PM
@slugger wrote:
im having a slight issue converting ascii to decimal.
Next time also try to be more specific. 😄
Your problem also shows why type cast is relatively dangerous unless you know exactly what you are doing. There is no error output, e.g. if the string is shorter than needed for the datatype. It would be safer to use "unflatten from string" becaue it would alert you about the data mismatch at run time.
12-28-2014 03:30 PM - last edited on 12-28-2014 04:01 PM by altenbach
That was throwing me off too, especially considering 0x30-0x39 are the ASCII values for 0-9. 0x01-0x09 result in characters that can't be seen. I wouldn't expect them to show up as recognizable numbers.
12-28-2014 04:01 PM
@natasftw wrote:
That was throwing me off too, especially considering 0x30-0x39 are the ASCII values for 0-9. 0x01-0x09 result in characters that can't be seen. I wouldn't expect them to show up as recognizable numbers.
If I understood correctly, he said that the string was set to hex display.