VISA string parse problem understanding

[Simon-lee]

I’m looking for some help parsing string data.

I’m playing with a cheap gas sensor, its talking (sending commands is hit and miss but once started it streams ok).

I’m trying to graph the CO concentration and write the data to csv.

I feel I must be missing some basic knowledge on VISA somewhere, I was expecting simple csv strings of data but my last attempt ended up at a 547mb csv. I was hoping that using line feed termination meant I had blocks of 65 characters to play with but I'm struggling to understand whats really going on. 

If someone can point me in the right direction I would be most grateful.   

P.S. I save the vi in 2017 if anyone need a earlier version please just ask 

[Bob_Schor]

You fooled me with your VISA configuration.  I was going to complain that you set up Termination wrong, but I realized that you set it up "to use the defaults" (I just failed to recognize you reset the Termination character to 0xA, <LF>, the default.  Most of us leave these terminals unwired, and accept the Default Values (rather than trying to set them to the Default, and possibly make a mistake).

But then you make the Very Common (and foolish) Error and use "Bytes at Port".  No, if your device that is communicating using VISA is using a Termination Character, you ignore Bytes at Port, and instead do a VISA Read (when you expect VISA to be sending you data, such as right after you give a VISA Command) and ask for many more characters than you expect.  Common requests are for 1000 characters (I tend to specify 1024, myself).  If your Device is acting as it should, it will send "a reasonable amount of data" (say 20-40 bytes) followed by a Termination Character, so your VISA Read will wait for the entire String and then present it to you.

Depending on how your device delivers data, you can simply write the entire String to a Text File (terminating with <CR><LF>), parse the String to get out the data, arrange it as you like, then write it as a Delimited Spreadsheet File, or whatever else seems reasonable.

Do you expect a single Command (such as "*IDN?") to produce many lines (strings terminated by <LF>) of responses?  Notice that if you have not pushed the Send Command button, the code will continually read (possibly 0 bytes) over and over, sending the string to the Queue where it will pile up fairly rapidly.

So study your device.  You might want to have a sub-VI called "Initialize VISA" that sends the *IDN? command and parses the results, then sends the "Now Start Sending Me Real Data and Don't Stop Until ... " (you have to think about how to stop your Device from sending data -- read the manual).  If you get rid of Bytes at Port, then the VISA Read will put "real" data onto the Queue at the rate the Device sends it to you, which is probably at a rate of 10-100 Hz (again, you haven't told us what the Device is, so I'm only guessing).

Bob Schor

[cbutcher]

---

@Bob_Schor wrote:

You fooled me with your VISA configuration.  I was going to complain that you set up Termination wrong, but I realized that you set it up "to use the defaults" (I just failed to recognize you reset the Termination character to 0xA, <LF>, the default.  Most of us leave these terminals unwired, and accept the Default Values (rather than trying to set them to the Default, and possibly make a mistake).

---

This is right, unfortunately... The type-cast is set to produce an I32, and the input is U8. The result is that the value given to Configure Serial Port is "0x0A000000" -> "00", which presumably isn't the end character (unless, of course, it is).

When I first saw the image, I was "going to complain" that the mechanical action was one of the Switch modes, and that you were reading multiple lines. However, the FP image shows that you managed to read part of a line in the last case. Coupled with the fact that given a valid TermChar and a number of bytes to read, you'll get whichever ends first, and I'm going to guess that in the case shown, you ended with the Bytes at Port value (multiple times?), which was less than the total string because you hadn't waited long enough yet.

As Bob pointed out, we'd need to know the device to really know the solution, but if the real TermChar is LF, you can either A) remove the inputs and use the default, avoiding the mistake making opportunities or B) use a U8 for the top input to the Type Cast, and then get "0x0A" (the same as the default), or C) wire the constant "0d10" U8 value (in this case, you can also use a larger numeric width and have it be coerced, but there's no reason to do that that I can see from this BD (and perhaps no reason I can think of at all)).

[Simon-lee]

Hi Folks,

I apologise for not including the sensor data sheet (DGS-CO 968-034) please find it below.

It came from Digi-key and cost 71 dollars – using Tera Term as described in the data sheet “C” start a continuous output and a “R” stops it. The promised timing options don’t seem to exist and the data stream is on a 1 sec cycle.

The data sheet is very sparse on detail so I made assumptions on termination. (first mistake)

The *IDN?\n and bytes at port are hang-overs from the example finder simple serial.

I’m going to have another stab at it today starting from scratch but any additional input would be welcome.

[cbutcher]

Hi Simon,

As you said, the datasheet is a little lacking in clarity regarding a possible termination character.

I'd first go ahead and try with the default (10, "LF" -> U8, etc - easiest just to not wire this connection to Configure Serial Port).

Hopefully the "CSV" values are terminated with some sort of end-of-line character (since otherwise, it isn't a CSV output), and everything will work a little better.

Just as a reminder, to do this (as Bob said) you'll need to remove the Bytes at Port and just wire a sizeable constant. In your case, the constant needs to be larger than ~80 (I didn't carefully count - I recommend stetting this to perhaps 200 or 300, but 1000 as was suggested by Bob Schor is also fine and removes the chance of a silly long output messing up your acquisition) ->(SN [XXXXXXXXXXXX], PPB [0 : 999999], TEMP [-99 : 99], RH [0 : 99], RawSensor[ADCCount], TempDigital, RHDigital, Day [0 : 99], Hour [0 : 23], Minute [0 : 59], Second [0 : 59])

Let us know if it still doesn't work!

[Simon-lee]

Hi folks,

Thanks for all the advice - I have had a second stab at the problem and hopefully I have understood your advice. 

Now I played around using the hyper terminal and concluded that the sensor isn't very responsive. 

To help I have added a state machine to tickle the port until I get a bit count, then start the read loop. 

I've also tried the same trick to deactivate the sensor on exit.

Something is still not quite right - it starts and streams data about 30% of the time - the rest of the time it records 1 line (65 characters), sits there fat dumb and happy until the read loop times out.

I hope you folks can shine some light on this problem 

Many thanks

Si

[Simon-lee]

After much frustrating effort I'm begining to suspect the on-board VISA to USB converter (SILICON LABS VCP DRIVER). I read a few post where cheap RS232 converters cause problems in a Windows enviroment. I guess at the $70 price point it might not be brilliant.

If its a persistant pain I'll attempt to repalce it - any recommendations?

[cbutcher · Accepted solution]

Hi Si,

I took another look at the latest VI you uploaded. I have a couple of questions and then some minor observations. I'll number them to make it simpler to refer to specific points (not to imply importance):

1. What's a typical value for "return count"? Does it stay at 65?
2. What's a typical value for "Serial Settings:Number of Bytes at Serial Port"? Is it fairly constant or changing a lot? Is it less than "return count" systematically? (Is it zero or close to zero systematically?)
3. Have you tried just waiting repeatedly (rather than repeatedly resending the "c" command) for the first loop?
4. Can you try appending a Line Feed character to your Write commands (like below, after numbers)
5. You can create an indicator for the Enum (rather than writing a string out of each case). Alternatively (if in future you need a string) then Format to String accepts enums and will output strings (image below)
6. Does it behave the same if you remove the VISA Clear at the beginning? Or perhaps replace it with a Read command (maybe after using Stall Data Flow for a couple of seconds as you do between the loops)?

Point 4

Point 5

Regarding the possibility of a broken converter - it's certainly not impossible, but I haven't had it happen to me that much (once from maybe ~10 $10-$15 adaptors? Numbers from the top of my head, may be horribly inaccurate). I have seen it reported on the forums though... (probably the same posts you've been reading)

Following the writing of my post, I'd suggest taking a look at #4 first.

[Simon-lee]

Hi Cbutcher,

Thanks for taking the time – it is appreciated.

1, Once the read loop is running the “read count” is steady at 65 – in fact if the read loop can get past two iterations it keeps going fairly well. The shortest run time was 31 minutes and the longest was 109 minutes before error (I don’t have the error code with me but it involved the next character arriving before the last one had cleared)

2, Serial setting etc. I was expecting either zero or 65 but it’s often in the 130 region. (I’ll make an exact note tomorrow)

3, I’ve not tried it exactly – there is a ¼ second between read and cmd  and 1 full second between cmd and read again.

4, I have tried concatenating a carriage return but that made no difference – again I will try a LF tomorrow.

5, Yes I shall try that.

6, I’ve removed the Clear at the beginning (no change) The rest I will have a play.

It might be noteworthy but if I start with highlight execution on and switch it off once the read loop iterations reaches 3, if runs as expected (until it trips up) (this is my emergency plan if the boss asks to see some numbers changing on the screen)

I will report back my findings  tomorrow and again thanks for the advice

[Simon-lee]

Hi Cbutcher,

Great news, after following your suggestions we have success (well almost).

First tried the addition of LF to the C cmd – no difference, so I removed it.

I played with the timing suggestion –the tickle loop was slowed right down and configured it to listen more often. That seemed to have done the trick, playing around with some other features was fun but the timing was the key. After about 2 dozen stops and starts it started (and stopped) the sensor without issue.

The only fly in the ointment is the random error (see snip), as it occurs after thousands of cycles, I’ve look to the obvious (power saving on port etc.) but nothing stands out.

I’ve included my latest work in progress, I found time to add a DAQ to bring in pressures and temperatures and a few indicators to show what’s going on.     

I’ve still got to add the write to disk functionality but it’s looking good. Oh and the only other oddity is sometimes the consumer loop doesn’t stop but it’s the first time I’ve used channel writers so I need to revisit that.

Oh top tip on the enum indicator, I haven’t had a chance on this project but I’ve filed that away for next time.

Thanks

Si