Visualise and log data at the same time using TDMS?

[VinnyAstro]

Hi!

I feel like it's quite an easy one, but for some reasons I can't get a hand on it, I think it's translating my lack of knowledge on data management in Labview.

So I'm receiving data (representing a rotation speed) from a device over a communication interface (NI USB-8451) and I would like to visualise this data on a chart so the user can assess the device behaviour "in live" and also to save this data using TDMS files.

I am attaching a very quick example to illustrate my situation, I know this is not a workable solution. 

I should point out that I've already done some research, especially with the Labview examples, but they were not really helpful as they are mainly using Express VIs for generating simulated data, making it really easy to look like how you'd want to .... which is exactly my problem 🙂

My issues/questions:

1. Given that I'm using the QMH template, my "Communication Loop", handling all commands to be sent and received by and to the device, cannot guarantee a stable 100ms rate for receiving the speed. (How could I do that?) How, in my graph can I send and display this information: A graph showing the current speed with a relative time in X-Axis? Should I use an XY-graph ? 
2. TDMS write is accepting minimum a 1D array of data. I've seen that when you give it a waveform you can actually enter automatically a lot of parameters as attached in the example from the Labview example "TDMS Write Time Domain Data" (waveform increment (not constant for me); wf time start etc.)  but it also creates an "Implicit" channel for the time, showing relative time, intervals etc. How could I do that?

Thank you in advance for your help.

Best,

Vinny.

[User002]

First, let me restate your questions to make sure I understand them correctly. You're asking:

1. My data will not be received at regular intervals, so I want to graph/chart your data vs. time, but I want my graph to show the time that the data was actually received. How can I do this?
2. How do the waveforms have extra information in the TDMS file? How can I enter in that information?

Assuming I understand what you're asking...

1.

"The waveform data type is used...to display and store periodic signal measurements." (found here, emphasis mine)

So, for irregularly (i.e. non-periodic) received data, the waveform data type will not work.

(If I'm wrong, I'm hoping that a more experienced user will correct me.)

Instead, since your "Communication Loop" is the only place that will know when the messages are received, you should collect the timing data there, then pass it to your consumer loop via your queue (possibly in place of the data that you label "Time" in your TDMS file).

2.

I *think* what you're asking about are the Attributes of the waveform data type.

 

Similar to Variant data Attributes, you can Set or Get these. And IIRC, when you write Waveform data to a TDMS file, these Attributes are saved as TDMS properties.

But, since it seems that the waveform data type won't help you, you can instead set properties directly using these:

 

Is that any help?

-joeorbob

[VinnyAstro]

Hi Joeorbob. Thanks for your answer.

---

@User002 wrote:

First, let me restate your questions to make sure I understand them correctly. You're asking:

1. My data will not be received at regular intervals, so I want to graph/chart your data vs. time, but I want my graph to show the time that the data was actually received. How can I do this?

Yes that's correct. I would like it to be well regulated, but for now I don't know yet how to deal with it. (Basically my current solution is using a state machine that is checking the loop time, if it greater or equal to 100, then take one measurement, otherwise go check for messages that could contain other commands like switch off the motors etc.)

---

@User002 wrote:

1. ...
2. How do the waveforms have extra information in the TDMS file? How can I enter in that information?

Correct as well.

---

@User002 wrote:

"The waveform data type is used...to display and store periodic signal measurements." (found here, emphasis mine)

So, for irregularly (i.e. non-periodic) received data, the waveform data type will not work.

(If I'm wrong, I'm hoping that a more experienced user will correct me.)

Instead, since your "Communication Loop" is the only place that will know when the messages are received, you should collect the timing data there, then pass it to your consumer loop via your queue (possibly in place of the data that you label "Time" in your TDMS file).

---

Ok but how would you actually do that? I'm looking to generate, store and display the relative time here, so would you rather do a subtract of tick count and addig the result? Would this be a "good practice" solution?

And then I'm supposed to send that together with my actual measurement so I would then just need to build and array of those two and send it to the TDMS write?

Something like this? (It's obviously a non-executable VI, it's just for imaging)

Best,

Vinny.

[rwunderl · Accepted solution]

Vinny,

There are several issues with that approach. The most egregious is the lack of data flow. LabVIEW is a data flow language, meaning that your structures and wires must explicitly enforce execution order. This is in contrast to a text language where one line of code necessarily and predictably executes before the next. All this to say that your data source, the Wait, and the Tick Count primitives have no data flow dependencies between them, and as such may execute randomly in time with respect to each other in each loop iteration.

The most simple fix would be to put these items into a Sequence Structure to force their execution order as shown below. You should also use the Wait Until Next Multiple primitive instead of Wait to account for loop overhead once your actions are sequenced. You can also split these things up into different states in your state machine. Even so, the timestamp must be collected as close to the data sample as possible. Is there any reason that you are not using a more stable clock source? Is this data acquisition or instrument communications or what?

Speaking of timestamps, you should just collect the timestamp versus using a tick count. You can do the subtraction of the timestamps to determine the time difference with as much precision as this application seems to require.

The TDMS approach you sketched out works to store the two channels, a Time and a Data channel. You just have to be careful with the format of the data. There are a bunch of different ways to do this, I illustrated just one.

I still wonder why you want to store a relative time versus the timestamp. This seems quite irregular to me. Normally, data is stored versus a relative time since the beginning of the acquisition or an absolute time (local or universal time depending on your preference). I showed the relative time just to let you know you can make a DBL out of it for calculations.

This example works if you give it a valid file path. (Don't know if you are familiar with VI snippets, but just drag the PNG file onto the block diagram and it will be converted to code.)

Hope it helps.

[VinnyAstro]

Hi Rwunderl and thanks for your answer.

I feel some clarification about my requirements/code is needed, I'll try to be short:

• I need to command (and not control) a device (literally a motor which goal is to spin a flywheel to stabilize a system). This device has an On-Board Computer that do all the control: ramping, PID, speed claculation etc. --> I interact only with some registers to:
  • Enable/disable the motor
  • Set Velocity and acceleration
  • Read current Velocity calculated by the OBC
• The commanding is done in SPI protocol via the NI USB-8451 (USB to SPI/I2C)
• My goal is to test the performances of this (these) device(s). (I need to run tests at different speed and acceleration and log the speed and current consumption over time.

Now why don't have I a consistent 100ms measurements?

I'm using the QMH mixed with P/C: My Event loop receives commands from the diffrents buttons and send an appropriante answer to my "control loop" which has a state machine (as described below), convert the speed and acceleration in a readable command to my device send it over the coms loop and check at every iteration if 100ms has passed, if yes, then send a "read speed" command to the coms loop.

So this state machine is constantly looping mainly over "reading Speed" (if the "loop" has reached 100ms minimum) and "Checking for new messages" (from the event loop).

I also attached my code (downgraded in LV17) if you're interested.

Now: 

---

@rwunderl wrote:

The most simple fix would be to put these items into a Sequence Structure to force their execution order as shown below. You should also use the Wait Until Next Multiple primitive instead of Wait to account for loop overhead once your actions are sequenced. You can also split these things up into different states in your state machine. Even so, the timestamp must be collected as close to the data sample as possible. Is there any reason that you are not using a more stable clock source? Is this data acquisition or instrument communications or what?

What's the added value of "Wait Until" compared to "Wait for"? The goal of my coms loop is to exectue as fast as possible, I haven't put any waiting solution appart from the Dequeue VI (which, by function prevent any execution until there is a message). But that would be in my control loop that I would like it to make sure that every 100ms I send the command Read Speed.

---

@rwunderl wrote:

Is there any reason that you are not using a more stable clock source? Is this data acquisition or instrument communications or what?

Is "not knowing this is not stable" a good reason? 😂 

As said above, it is both: Communication AND Acquisition.

---

@rwunderl wrote:

Speaking of timestamps, you should just collect the timestamp versus using a tick count. You can do the subtraction of the timestamps to determine the time difference with as much precision as this application seems to require.

---

Not much indeed, honnestly up to a few ms will be more than enough.

---

@rwunderl wrote:

The TDMS approach you sketched out works to store the two channels, a Time and a Data channel. You just have to be careful with the format of the data. There are a bunch of different ways to do this, I illustrated just one.

---

Ah yeah thanks for the tip! I haven't payed attention to the matrix style of arrays 🙂

---

@rwunderl wrote:

I still wonder why you want to store a relative time versus the timestamp. This seems quite irregular to me. Normally, data is stored versus a relative time since the beginning of the acquisition or an absolute time (local or universal time depending on your preference). I showed the relative time just to let you know you can make a DBL out of it for calculations.

---

Again it's for testing: at some point (not finished yet) my program will read a test file time stamped line by line that basically says:

at 0s go to 2000rpm at 200rpm/s (+other random info)

at 50s go to 1000rpm at 300rpm/s (+other random info)

etc.

So I want in my test report to see an actually relative time and not absolute. (just to spare me an additional 2 mins of work on an excel file) But also because There will be some commands like "Start recording" "stop recording" "create new test file" ... So I want that every "speed recording" starts with a 0s time 🙂

---

@rwunderl wrote:

This example works if you give it a valid file path. (Don't know if you are familiar with VI snippets, but just drag the PNG file onto the block diagram and it will be converted to code.)

Hope it helps.

---

Didn't know how it was called, gonna abuse it now, thanks.

And yes it did help.

Thanks.

Vinny.

[VinnyAstro]

---

@VinnyAstro wrote:

Hi Rwunderl and thanks for your answer.

I feel some clarification about my requirements/code is needed, I'll try to be short:

Looks like I failed.

Also forgot the code, here it is.

Vinny.

[Kevin_Price]

One small-ish piece of advice:  If you know that you typically want to query speed over SPI every 100 msec, I would change up your approach.

Put more of this timing control into the Comms loop.  Other than the relatively rare Init, Start, Stop, and Exit messages, all it *usually* does is query speed, right?   I would set it up so that the "Start" message makes it start running its own speed query timing autonomously.

One simple way to do that is to use a shift register for the Dequeue timeout in the Comms loop.  It starts out at -1 (infinite), but once you receive the "Start" message, you change it to something like 100 msec.  Then you have a timeout case that does the speed query.

If the queries take several msec, you can always recalc the timeout value to 96, 97, whatever so the next timeout is pretty much 100 msec after the previous.

Timing won't be *perfect*, but putting dedicated control of timing into the Comms loop will make it more independent of things you may add to your control loop.

-Kevin P

[VinnyAstro]

That's an interesting idea!

How would you do this timeout case though ?

This is what I have in mind, what do you think about it? (That's just a quick example)

The VI is constantly looping on "read Message", if there is nothing in the queue it's just iterating. If "Start" is put in the queue, then a TRUE is sent over to the first Case structure, reading then the time. If it's over 100ms, then a timeout is fired.

BUt then I also have to think about the Current acquisition that needs to be synchronized with the Speed acquisition, but that's another story.

Vinny.

 

[Kevin_Price]

I had something a little different in mind.  I took your snippet and ran with it a bit to illustrate what I was suggesting with actual code.   I use the actual Dequeue timeout value to pace the loop once it's time to start the regular readings.  And I show how to recalculate the timeout after completing each read query in case these queries consume a measurable amount of execution time.

-Kevin P

[rwunderl]

Vinny,

Ok, I now understand better what you are trying to do. I still do not understand how you are correlating the current measurement with the velocity response from the motor controller. I think Kevin has explained a scheme to more accurately time the velocity readings, but now on to how to get the current reading.

We can only presume that the velocity reported by the motor controller is "instantaneous" as of receiving the request, which we will also presume is sent instantaneously. It is at the moment of sending the request that you should log the current, as it is at this moment that the controller is copying the velocity value into its response message. The delays here are probably negligible since you are using SPI, but we'll try to be accurate anyway.

One concept that comes to mind as we are talking about synchronization is events. Have you ever tried User Events? You could have 3 loops in your application: the user interface loop, the motor controller communications loop, and the data acquisition loop. The user interface loop would monitor buttons and other user interface items. The motor controller loop would be much like you and Kevin have outlined, except driven by an Event Structure with User Event handlers. Then the data acquisition loop would just be an Event Structure as well.

The motor controller and data loops would each handle only 2 user events: Stop and Acquire. The main loop could send the first Acquire user event when the user starts the run. From then on, you could use a scheme like Kevin's to send all subsequent Acquire events from the motor controller loop (handled by itself and the data loop). The timing could be a simple Wait Until Next Multiple in parallel with the comms; the Wait Until Next Multiple will hold the event case until the next 100-ms multiple to avoid drifting too much over time (a simple Wait would cause the acquisition to take longer than the saved data points would suggest over time). If comms take longer than 100 ms, it's probably an error anyway. The Acquire event would be fired after the Wait in the Acquire handler case (if there's no error). This will re-trigger the Acquire event handler in both loops simultaneously. The nice thing is there is no polling; every loop is completely idle until something of interest happens.

This is just off the top of my head, maybe it's a bad idea. Things can get much more complicated, but I like Event Structures because they keep things relatively simple. There are other synchronization methods too, like Occurrences, Notifiers, and the dreaded local variable polling. I don't like state machines because of the incessant polling for no reason. Just my 2¢.