LabVIEW
Complete String- Conversion Tutorial
Hello Community,
since this is a frequently asked topic, I would like to try here to provide an overview (as complete as possible) of this complex of topics. It is about converting strings to numbers and numbers to strings.
Most of the questions I have seen in the forum about this arise from the conversion of data, which is transmitted via a serial interface, into numeric equivalents or vice versa. Often the numbers are to be represented in hexadecimal (base 16) or binary (base 2). And here already the first misunderstandings begin... Now to start this topic, here first some definitions of terms.
Computer data is stored in bytes, which consist of 8 bits. By combining several bytes, different types of numbers with different value ranges can be stored. The value of a byte can now be represented in different ways:
Binary: Each bit of a byte is represented by 0 or 1.
Hexadecimal: each 4 bits (a so-called nibble) are represented by a hexadecimal digit 0 - F, so a byte is described by two hexadecimal digits.
Decimal: the value is described by a decimal number 0 – 255.
ASCII: In the ASCII table is defined, which character is assigned to a certain value.
The octal representation, which is not considered here, is not very common.
When combining the bytes, larger ranges of values can be stored. In the example of the figure, the number would be represented like this:
Binary: 1 0 1 0 0 0 1 1 0 0 1 1 0 1 0 0
Hex: 0xA334
Decimal: 41780 (if interpreted as U16)
Decimal: -23756 (if interpreted as I16, for more details see Two's complement)
String: '£4'
For the numerical interpretation, however, the above representation is only valid if this 16-bit value is also interpreted as a big-endian. If, on the other hand, the 16-bit value is interpreted as a little-endian, other values are produced:
Binary: 0 0 1 1 0 1 0 0 1 0 1 0 0 0 1 1
Hex: 0x34A3
Decimal: 13475 (if interpreted as U16)
Decimal: 13475 (if interpreted as I16)
When interpreted as a string, the characters are output in English always in such a way that the byte with the lowest memory address appears on the far left, and the Bytes with ascending addresses then always to the right of it (big endian).
Of course, it is also possible to combine even more bytes to represent even larger number spaces. So four bytes are interpreted as U32 / I32 / Sgl, and eight bytes are interpreted as U64 / I64 / Dbl. For the sake of clarity, we will stick to 16-bit values in our examples.
Now there are different requirements when converting these byte values into human readable form.
Input type: String
Let us first consider the case where a string is to be converted from the serial interface or from a file into another form. We have to distinguish between ASCII strings and binary strings.
ASCII- String
With an ASCII string, each byte is regarded as a character from the ASCII table. And with it numeric values can be represented in different ways. Depending on the value to be interpreted, the conversion to the numeric value must be done in different ways:
(For clarification, I have placed a copy of the input string in hexadecimal representation on the block diagram. The displayed abbreviation for the display style clarifies this).
01 / Binary ASCII String
02 / Hexadecimal ASCII- String
03 / Decimal ASCII- String
04 / Text- String
As you can see from the examples, the VIs adapt to the input type of the default value (U16, I16) and they interpret the string accordingly.
Binary string
So, if the input string is not ASCII coded, it contains the binary data representing a numeric value. In this case, a typecast must be applied to map the bytes in the string to a numeric type. As a result, the memory location that was just considered a string is now considered a numeric value. The corresponding VI is called "Unflatten from String" in the Data Manipulation palette.
(There is another VI called "TypeCast", but this offers less control over type conversion).
To convert a two-byte binary string to its numeric equivalent, the following code can be applied:
05 / Binary string to numeric value
The lower part shows how each character of the string can be converted to its 8-bit value. With the help of "String to Byte- Array" it is possible to process the binary string byte by byte in a loop.
06 / With "Unflatten from String" it is easily possible to switch the endianness:
In order to display the values in the indicators in a different format (hexadecimal, binary...), the display format properties can be changed via the properties dialog of the indicator.
Input type: Numeric value
If a calculated numeric value has to be converted into a string to send it via the serial interface or to save it to a file, again different display types are possible:
ASCII- String
Here the numeric value is converted into a string that can be read by humans. It is to be considered that the numeric value is stored at the memory location reserved by LabVIEW in the binary form
1 0 1 0 0 0 1 1 | 0 0 1 1 0 1 0 0 on the memory location of two bytes. The fact that the U16 constant displays the value 41780 is due to the fact that LabVIEW performs exactly this conversion into an ASCII string of the decimal representation for us.
07 / To Binary ASCII string:
(Since the snippet tool automatically numbers controls or indicators with the same name in ascending order, I have placed an underscore in front of the label of one of the indicators and then hidden it somewhat unprofessionally above the terminal).
08 / To Hexadecimal ASCII string:
09 / To Decimal ASCII- String
Again, it can be seen that the conversion VIs adapt to the type of the input value and perform a corresponding conversion. To further illustrate this, the last snippet changes the representation view of the constants to binary, showing that the binary representation does not differ, but is interpreted differently by the conversion VIs.
10 / The data type makes the difference
All these VIs do not provide the possibility to specify the endianness, because the output format is a string, which is always displayed Big Endian.
Binary String
When a numeric input value is to be converted to a string containing the binary equivalent of the memory location of the numeric value, the Flatten to String VI is used. This can be used to copy any value into its binary equivalent in a string.
11 / Flatten to String
12 / If the value is interpreted as Little Endian, the following result is obtained:
Interpretation of serial data
The problems with conversion of strings arise mostly when reading serial data or file streams. The cause is often an unclear terminology ("Hex- Values", "Raw Data vs. Hex Data"). In the following some terms shall be clarified.
This section would also have fitted well to the beginning of the article, since only misunderstandings lead to the selection of wrong methods, but to be able to follow here, the before described should already be read and understood.
ASCII coded strings
Basically, the data received from a serial interface are always binary strings, not hex strings. These binary strings can be interpreted in different ways, e.g. as an ASCII string (each byte value in the string corresponds to one character in the ASCII table) or as a Unicode string (two bytes correspond to one character). The string control displays the string as ASCII string in the display style "Normal Display". The display style "'\' Codes Display" ensures that mixed strings of control characters and texts remain easily readable and the "Hex Display" shows the value of all bytes in the string with their two-digit hexadecimal value.
A look at the received data in the display style "Normal Display" quickly reveals whether it is a pure ASCII coded string, if no square symbols and other special characters are shown.
If this ASCII string consists only of the ASCII characters 0-9 and A-F, then it is a HEX (hexadecimal) coded ASCII string. This can be converted into its numeric equivalent as in recipe 02. A decimal coded ASCII string (the string contains only the ASCII characters 0 - 9) can be converted according to recipe 03. If the string consists of several numbers, it can be split with string operations and then converted into the numeric equivalent. Here LabVIEW offers powerful possibilities to react to different situations. These will only be touched upon very briefly.
If the received string consists of similar numeric values, the VI "Spreadsheet String to Array" can be used:
13 / Conversion of a list of decimal ASCII coded values into an array.
14 / Convert a list of hexadecimal ASCII coded values into an array.
In this example, a different separator was simply used.
If the input string consists of different types of values, the VI "Scan from String" is useful. Here, a template can be defined and on its basis the string is interpreted.
15 / Converting a mixed list into single values
Non ASCII coded string or mixed.
If the input string looks similar to the one shown in the figure below, it can be assumed that binary data is being transmitted. In this case it is important to know the format of this data to be able to interpret it correctly.
It is now possible to use string operations (String Subset, Search/Split String, etc.) to extract individual parts at known positions from the string and then convert them as in Recipe 05 and 06. But again LabVIEW offers by far better possibilities to solve these tasks. Since the VI "Unflatten from String" has an input to which a template type can be wired, the contents of the string can be mapped to this type in this way.
As an example it is described in the documentation of a measuring device that on the command "Send Data Packet" a response is sent in the following format:
|
Name |
Datatype |
Value- Range |
|
Unit_ID |
Byte |
0 - 255 |
|
Velocity |
32-bit IEEE-754 floating point (Single) |
0.0 – 50.0 m/s |
|
Quality of Signal |
Byte |
0 – 100 % |
|
CRC |
16-bit Unsigned Integer |
- |
This packet can now be converted into a corresponding cluster and wired as a default for "Unflatten from String". In the example below, both steps are taken: first, a binary string is generated from the cluster, and this is converted back into the equivalent as a cluster in the second step.
16 / Create cluster from binary string
In this case, the contents of "data packet string" would be the string sent by the meter in response to the request. It reads:
There are two things to watch out for:
- that the defined data types match in order to let LabVIEW assign the length of the data fields correctly
- that the endianness matches.
If the input string contains fields with numeric values (these always have a defined memory allocation) as well as with strings or other variable data, the matter can become more difficult (but it does not have to).
At the beginning it should be briefly explained how strings and 1D arrays are stored in LabVIEW. Both are dynamically resizable data fields. To know the size of the used memory, LabVIEW places a U32 value at the beginning of the memory, which contains the number of elements (for strings the number of bytes, for arrays the number of elements). So an empty string still takes up 4 bytes of memory! If now a string or an array is defined in the default data type, LabVIEW ALWAYS expects 4 bytes with the size specification first.
If string fields or arrays with fixed length are transferred, where understandably the size specification is not transferred, therefore a placeholder must be defined at this point. Sub-clusters that correspond to the expected size can be used here. Now creating a cluster as a template that is to reserve storage space for a fixed size of 50 elements sounds like a lot of work at first, but it is quickly done with "Array to Cluster" from the "Cluster, Class & Variant" palette. If a string is then to be recovered from this placeholder cluster, "Flatten to String" can be used again.
17 / Unflattening with a fixed-size string
If string fields or arrays with dynamic size are transmitted and the size information is transmitted in a different way than LabVIEW expects, you can try to capture as much as possible with the type default and then extract the data from the output port "Rest of the binary string" with string operations.
The best way to handle dynamic data is to arrange with the programmer of the sending device (maybe yourself) that accordingly the length information is prepended as a U32 value, because then "Unflatten from String" does all the work in no time.
Comparison of conversion methods
Conversion to / from ASCII string
This method has its appeal because the data is human readable and therefore easy to debug. In addition, this data can be easily processed with tools like Excel.
However, the conversion into ASCII coded strings can lead to a noticeable performance loss on the sending device, such as an Arduino. In my practice it happened that an AVR-Mega 2560 driven with 16 MHz while sending a pulse sequence to control a stepper motor needed five times longer to transmit the current pulse count after conversion with itoa() or sprintf() than to simply send the binary value to the UART (about 50 µs to 250 µs in my memory). Furthermore, the amount of data to be transmitted is generally higher for ASCII encoded strings. On the PC side, however, there should be no significant difference with modern computers, unless the files to be read are many MB in size.
Conversion to / from binary string
The advantage of this method is that simply memory contents are transferred and then interpreted as binary data of certain structure. The string itself is not well human readable. The performance is better than ASCII encoded strings because there is no need to encode into ASCII space.
For data read in via a serial interface, this difference in performance is not noticeable if, for example, a conversion is already taking place in parallel with the read-in, because then the transmission speed of the interface is the limiting factor. However, if the data is from a byte stream with a large data volume, the performance difference can be significant. Here is a brief illustration of this (this applies to my (old) computer with XEON CPU, E5-1620 with 3.6 GHz and 8 GB, and no effort at programmatic optimizations):
In a test program, half a million lines with 20 Dbl- values are generated and then (1.) converted into an ASCII-encoded string or (2.) into a binary string. The string is then written to a file. The ASCII encoded file is a little bit more than 100 MB, the binary file about 80 MB (not MiB!). Then these files can be read in again and converted into a 2D array of Dbl values, each separated into (3.) ASCII encoded and (4.) binary string. In each case the time (in ms) was measured for generation of the data (step 1) and conversion (step 2). Step 1 is omitted for (3.) and (4.), therefore the value is zero.
At the last point I thought there was an error and repeated the measurement, but reading in the 80 MB - file and transferring it into a 2D array really takes only 1 ms (sometimes).
It should also be noted that the method of binary storage of data is not optimal, there are far better possibilities here e.g. with TDMS.
Conclusion:
I'm aware, there are still points not mentioned and hard to understand, but in general this should cover all aspects of getting string data from interfaces and process it to use it in your LabVIEW- programs.
Very well put together. I mostly just skimmed it (hard to give a full read when a 5-year-old wants to play with a balloon with you), but it looks like you covered the very large majority of cases. You cannot cover everything here, but I don't think I have seen anything that could not be done by combining some of these ideas.
I find the biggest hurdle in helping people with these types of issues is getting the terminology in sync. I have seen way too many people say "I have this hex string...", which makes some people think it is a binary/raw string and others will think it is a hexadecimal formatted ASCII string, what I often refer to as "ASCII Hex". If I can get the person to give an example, then we can get our terms corrected and actually figure out the issue.
