Writing data from two while loops to an Excel sheet

[johnson64]

---

@Bob_Schor さんは書きました:

... clearly meant as a Consumer, but that's the only place the Queue wire is used!  If you are not going to Enqueue anything, you don't need a Queue.   One wonders whether you need a Notifier ...

---

Not sure whether you have seen the enqueue part in the TRUE-case?  The plan is enqueueing in the TRUE-Case while waiting for a new notifier/ stimulus intensity and then dequeuing in the FALSE-Case after the new value has arrived.

Can you please explain, why the use of the notifier seems to be nonsense as it is the trigger for the described process?

Thank you & Best regards,

Johnson

[Bob_Schor]

Gomen nasai, Johnson-san.  I was careless in reading your attached code, and failed to look at the True case of the loop holding the Queue, where you enqueued some data.  That's (in part) because I've (almost) only used Queues as a means to communicate between two parallel (asynchronous) Loops (hence the reference to the Producer/Consumer Design Pattern).

You were using the Queue as a "local storage" mechanism within a single While Loop.  This is, in fact, an interesting (and potentially valid) use of a Queue.  However, a much simpler (and much more efficient) mechanism is to simply use a Shift Register within the While Loop.  A Queue can be considered (and implemented) as an Array of whatever you want to put on the Queue (here an Array of Dbls).  An Array of (Array of Dbls) is simply a 2D Array of Dbls, initially empty.  To Enqueue, you add an Array of Dbls to the end of the "Queue" (the 2D Array of Dbls), so that "First In" to the Queue is the first element of the 2D Array.  To Dequeue, you "Delete from Array" the current First Element, i.e. "First In, First Out" (FIFO, the traditional mechanism for a Queue) (the other "storage model", LIFO, is something call a "Stack").

Here's a Snippet of using a Shift Register for a local (single-loop) "Queue" --

Shift Register as Queue

[This is for illustrations only, so I could show both the False, "Enqueue" or "Save in Shift Register", and "Dequeue", or "Return First Element", cases].

At the end of my reply #15, I said that you need to concentrate on what you want to do and not get entangled in how you want to do it.  At the time, I didn't realize your Data I/O was all through VISA (I'm used to using DAQmx, which misled me in my thinking).  Now that I know a little more, and particularly since if we are using VISA, then "Time" isn't as well controlled as with a DAQ A/D converter, I think I'd consider a slightly different scheme for saving the data (and, indeed, in designing the entire Project).

I don't know much about the instrument that is providing the VISA measurements to you, in particular if it is "streaming" (i.e. writing a data packet at some rate, say 100 Hz) or if it is responding to a specific command ("Give me 100 measurements as 100 Dbls at a rate of 100 Hz, i.e. a 1-second data stream, then stop until I ask for the next one).  For simplicity, let me assume the latter.  So here's what you (want to) do:

1. Send a VISA Command to set your stimulator to a certain level, call it S(i) (the i'th Stimulus).
2. Send a VISA Command to read a Response Array R(j)(i), the Array of j responses to the i'th Stimulus.
3. Save these data to a file (more on this shortly).
4. Go back to Step 1 with a new stimulus (i+1), or alternatively exit the loop.

So what data do you have in a single loop?  You have one Stimulus, S(i), a number of response R(j)(i) [100, in your case], and possibly some timing information (you can certainly get a Time Stamp, or what I like to do, use the Millisecond Clock to save the number of milliseconds since the program started).  My temptation would be to write a Delimited Spreadsheet (which is "human-readable" and easily parsed) with two columns -- the first holds an identifier, "T", "S", or "R", and the second holds a Dbl.  At the beginning of the loop, I write the Timer ("T") value (which is just a number if you are using "milliseconds since the Program started running), then the Stimulus ("S"), and then the Response Array ("R").  

When you read it, you will want to parse the data into an array of T values (1D Array), S values (1D Array), and R Values (2D Array), which you can then use in your analysis.  Since I don't know anything about your goniometer and how well it is "timed", I can assume that all the R values are (roughly) equally spaced in time, so I can calculate the sampling rate by dividing the number of columns of the R array (number of samples/read) by the time difference T(i+1)-T(i), i.e. samples/time.

This code is much simpler to write (no queues, no complicate "Excel" format) and reflects what you are actually doing.

Bob Schor

[johnson64]

Hello Bob_Schor,

First of all, I hope you (and everybody else reading this) could enjoy some joyful Christmas holidays with your family. 

No worries about this little mistake! I`m very thankful for your will to even look at this and provide such an extended answer to my problem including a totally new way to solve my problem. But I have to admit that until now I`m quite happy with the result I got. I get the desired data to the file and it seems like both loops are working together as intended. So I will stick to this because my time is running af. 

But of course, there is some kind of new problem. Although the stimulation amplitude is increasing as planned, when I connect the stimulator to my muscle (or even an Oscilloscope), I can also feel (see) the increase of amplitude. But then suddenly the stimulation kind of vanishes although the program is still increasing the amplitude. After a second the increased amplitude returns and then everything seems fine again until this problem occurs again. So it's like switching off the stimulation and then switching it on again without the wish to do so. Additionally, from time to time an unknown system error (–1073807360, VI_ERROR_SYSTEM_ERROR, (miscellaneous error)) is appearing and I have no clue why this is happening. 

When I use the Vi of the stimulator to manually determine the Min and Max stimulation voltage, everything is working fine even so the stimulation is also applied over a longer period of time. Therefore I think that using a loop to write the data to the stimulator is causing some kind of problem but my understanding of the underlying processes is not sufficient enough. 

As always I`m very thankful for every advice and possible solution!

Best regards,

Johnson

[GerdW]

Hi johnson,

---

@johnson64 wrote:

So it's like switching off the stimulation and then switching it on again without the wish to do so. Additionally, from time to time an unknown system error (–1073807360, VI_ERROR_SYSTEM_ERROR, (miscellaneous error)) is appearing and I have no clue why this is happening. 

---

When does that error occur?

Where (aka at which function) does that error occur?

Why do you still need to read BytesAtPort as fast as possible in a greedy loop?

Why do you still need BytesAtPort at all???

[johnson64]

Hello GerdW,

By now I can provide some more details about the stated problem.

The stimulator seems to struggle when the bytes at the port are not read in fast enough. From observation, I can tell that the stimulation interrupts when this happens. As a result, I increased the ratio and the error does not occur anymore.

But by doing so, I get a new error regarding the measurement of the goniometer. If I increase the ratio (for example to 64 bytes), the goniometer is not measuring continuously anymore as the first loop (calculating the stimulation loop) is taking longer than one second to read in the desired bytes. Therefore, the timing is not fitting anymore. I tried to delete the part (which can be seen in the first picture) because I don`t use the read in bytes at all, but the error of discontinuous stimulation still occurs. 

Is there a possibility to skip the delay due to the read-in of the bytes sent by the stimulator?

If I don`t wanna use the read-in bytes do I have to delete more than the described part (first picture)?

Or is it possible to increase the speed at which I communicate with the stimulator for example by increasing the baud rate? 

Once again, thank you for your help!

Best regards,

Johnson

[johnson64]

P.S. I have to read-in at least 48 Bytes per cycle to avoid an error due to the stimulator. The time needed is about 1020ms ... By measuring over a longer period of time this might cause an error in the assignment of the stimulator to the desired angle I guess... 

I tried to create a third while loop just for reading in the bytes. Might be a quick-and-dirty-solution. But now the timing of my calculating-while loop seems to be fine. 

[Bob_Schor]

There have been many discussions on the Forum concerning using VISA to read data from a Serial Device, and discussing not using Bytes at Port (which usually leads to all kinds of timing and interpretation problems, not unlike the ones you are having).

I don't know anything about the protocol your Device is using to send data to you over the VISA port.  Many devices, particularly those that send at a rapid Baud Rate, send character data and end with a termination character, usually <LF>, 0x0A.  When this is the case, you set up your code as follows:

• When you configure VISA, you Enable Termination Character.  You have, in fact, done this, since it is the Default Setting for VISA Configure Serial Port.
• You send whatever commands are necessary to get the VISA Device to start sending you data at some rate.  [I noticed you had no such command, so I presume that when Configured, your VISA Device just starts sending strings to you.  This seems odd, to me, but I don't know what device you are using.]
• You are now ready to read and process the data that VISA sends to you.  Configure a While Loop that will read one set of VISA data (a string terminated with the Termination Character) and (quickly) process it, perhaps by "exporting" the data to a parallel Consumer loop.
  • Start by asking VISA to read a "large number of bytes".  I ask for 1024 bytes, others like to ask for 1000.  You expect to get 10-50, depending on the nature of the data your device transmits -- when it has data to send, it will send it and add its termination character.
  • Now you have the data.  Go process it, and let the While loop return to "wait for" the next bit of data.
  • The While loop will automatically run at the speed that your device sends data through the VISA port (unless you try to do too much processing inside the While loop, which is why I said "Process quickly").

Do you know how to test your VISA Device in MAX?  If you don't know if your device uses Termination Characters, you can use MAX to "talk" with it and find out.

Bob Schor

[cbutcher]

Hi Johnson,

I note that this isn't related directly to your problem, but you're not in fact doing anything with the values read (64 or 48 bytes) from the serial port in that section.

I also suspect that the error might be instead a warning(?) and that it is related to the number of values to read?

I rewrote a possible implementation of your subVI - here you can see that since you have the same operation 6 times, you could instead greatly simplify your VI and use a For loop to make reading the code a little easier. A picture (not VI) of connecting it is below (note I gave it an icon):

I also note that you didn't have a value wired to the Channel 1 Value, was this intentional? It might be a feature (bug) related to my placing your earlier attached subVI into a later attached main VI - maybe you rearranged but I used an older subVI...

I removed the VISA Write node from the subVI, so you'd have to additionally use VISA Write in the calling VI. I did this to make sure that a reader could see all of the VISA calls in the same place (not always desirable, for example with very complicated code, but here I thought that was clearer) and to remove the need to pass this VI (which is really a message constructor) references to VISA (a separate issue - reading and writing a serial device, unrelated to the actual message being read/written). Does that make sense?

With regards to the main VI, I'd suggest you have quite a lot going on - you might be able to simplify somewhat and reduce the area you need to search for future bugs.

As has already been suggested, a State Machine might be helpful in this regard - beyond that, you have for example a Case Structure with i=0 as the selector - just place this code before the loop and you're done - no Case Structure needed!

You have a race condition with your save paths - and the two top loops are unnecessary. You can rewrite that code to avoid needing a lot of it and eliminate the possibility of invalid paths due to a race condition issue.

Here's a possible rework:

This snippet does nothing about the saving (which I discussed in a much earlier post) but instead demonstrates the use of the File Dialog express VI rather than the Prompt for Input, to simplify handling of paths. The While loops are removed and the two Case Structures are combined to avoid the local variable usage. You could re-add indicators if desired to give more debugging information to users, but the Local Variables can be easily avoided and that improves your safety here.