Isolate data from string

[IMSargon]

I have a string that looks like this:

= 0016.8

and I want to convert it to a number. The problem is sometimes it looks like this:

8= 0016.

Or something equally scewed. What's the most robust way to read this data, or to at least identify that 8= 0016. is in the wrong form so can redo the data read.

[JMBacker]

Are you familiar with the "Match Pattern" and "Match Regular Expression" VI's in the string pallette?  These two VI's search for a character or a set of characters, and split the string when it finds them.  If your string always contains "=" before the number, then you could simply search for "=", then use the rest of the string to convert to a number. 

[RavensFan]

Where is this string coming from?  If the data is sometimes looking one way, and other times another, it seems like you either have some sort of race condition based on how the string is built, or you are reading it from the serial port and sometimes starting your VISA read in the middle of the data coming into the port.

[stevem181]

1) Perhaps you have already done this, but it is worth mentioning just in case.

Can you prevent the bad data from occurring? In other words, if you are acquiring the data from an instrument (for example) - maybe there is an error in the way you are reading the data. It is better to prevent the problem than it is to try to fix it after the fact.

2)Assuming that you have considered the first item, then you need to make sure you have a definitive way to detect a bad string.

For example, does a good string *always* begin with an equals sign?

If that is the case, you could do something like this:

The second case structure shows what you can't see in the first one.

I hope this helps,

steve

[Darin.K]

 First I'll reiterate, your best bet is to find the source, but let's assume these errors are rare and you'd still like to be safe.  Assuming the first character is '=' in a good string, you could use ScanFromString

If the first character is not '=' you will get an error in the error cluster and offset past scan will be 0.  Otherwise, your number is waiting for you.

[IMSargon]

@ JMBacker  - I did try the "Match Pattern" VI, but even after reading the help, I wasn't quite able to get it to work for me.

@ Ravens Fan - You are correct, this is bad VISA reading. The unit I am reading continuously spouts data, so if I happen to catch it in the middle of a word, them's the breaks... Unless you know some trick?

@ Stevem181 - I like this idea. If the offset past match is 0 I know the data is bad and if the offset past match is 1 I know the data is good.

@ Darin.K - This is the method I've been using for many months. I never quite had it right, though.

@ All - So is there a way to deal with the VISA problem directly? I had assumed I was stuck with things the way they are.

[RavensFan]

Are you opening and closing the com port every iteration of your loop?  If so, don't.  Open the com port at the beginning of your program, close it at the end.  Only do reading of the com port during the loop.

Does the device put out a termination character at the end of each piece of data sent?  If so, then it is best to enable the termination character and read a larger number of bytes.  Then when you read the serial port, you will get everything up to and including the termination character.  Anything that is in the serial buffer after that character will remain until the next reading.

If the the "=" sign is at the beginning of every piece of data, then that is sort of like a termination character.  Unfortunately you won't know that the last piece of data is complete until the = sign comes in for the next piece of data.

If you know each piece of data that comes in has the same number of bytes, then what you could do is read the port 1 byte at a time until you get the equals sign, then do a VISA read for the remaining number of bytes in the data.

If on the very first reading of data you take, you don't get the = sign which would happen if you open up the serial port in the middle of a packet of data, just discard the data that comes before the equals sign.

If you use a communication/parsing scheme where you wind up reading more data than what exists in the data packet, strip out the data for that iteration, take any remaining characters and put it into a shift register for the next reading.  Then in the next iteration you can concatenate the data from the shift register with the characters/bytes that come in on the next VISA read.

There are a lot of different ways to read and parse data from a serial port.  Some ways are better than others, but in the end, it all depends on how the data is formatted, if there are a fixed number of bytes, or if there is a termination character present in the data you can use to tell when one piece of data ends and the next begins.

For the best help, please provide more information as to what the data stream looks like (how many bytes, any characters present that you can't see such as a tab, line feed, carriage return), and also how frequently the data comes in.

[IMSargon]

 Taking your advice about reading the VISA more carefully, here's what I came up with. There is no termination character, but the repeating data is always the same number of bytes with no invisible characters. It is a continuous stream of "= 0017.9= 0017.9= 0017.9= 0017.9= 0017.9= 0017.9= 0017.9= 00". I have hit a snag though, and I've seen it before so I'm sure someone else has as well. Take the example data stream above. The string I read out would be " 0017.9" but the number I get out of the "decimal string to number" VI is only "17". I'm sure this is a simple fix, but could someone point me in the right direction? 

Any other criticisms of my method would be well appreciated. Is there anything I should consider in using this as a subVI?

[RavensFan]

Decimal string only gives you the integer part of the number.  I think it is poorly named because if you want the decimal part, (i.e. everything after the decimal point), you need to use the "fraction string" conversion function.

[stevem181]

Short Answer: Try this code and post your results.

 

Long Answer: 

From looking at your code (and your posts) I see the following:

The serial instrument you are getting data from puts out a steady stream of data at 2400 Baud in a format that is “= XXXXXX\n”. Where the 6 “X” bytes are a floating point string preceded by a space character. So there are 8 total data bytes.

Now you might be surprised that I show a linefeed (\n) in the data format because you said that there is no termination character. Well you are right in a way because you will not see the termination char in your read buffer data. However, if you look at the “VISA Configure Serial Port VI” detailed help, you will notice that the default settings are to: “Enable termination Char” and “term char = \n” (inputs at the top of the vi).

 The reason this is important is for the Visa Read function. Since by default you have set your serial port to use a term char, the only way for the Visa Read function to complete is for it to receive a termination character or a timeout. Since you are not timing out, I know that you are getting a termination character and it has to be a “\n” to match the settings. The VISA read does not pass the term char on so you never see it.

In your first VISA Read you are asking for 1 byte of data. Based on the above info, your VISA Read must actually read 8 bytes + the term char before it will finish. So you are only pulling out part of the data that was read. The subsequent reads are doing the same thing. I believe this is why your data is getting jumbled.

So try this: Set your byte count to 8 and only use one read routine.

NOTE: In your while loops you do not have any delay which means you are really pounding on the read routines. Since your data is coming in at about 30 readings a second, it would be good to put in a ~25 millisecond delay because the reads can’t happen any faster than that.

steve

Message Edited by stevem181 on 11-06-2009 03:13 AM