Adding timestamps to raw CAN frames

[viamotors]

Hi NI community members,

I am writing a program that collects sensor data and broadcasts it over the CAN bus in real time. For CAN broadcasting, I am using an xnet session of type signal single point. After getting a list of signals from the dbc file (each signal corresponds to an individual sensor), I am creating an array that contains all sensor data at a given instant and writes it to the xnet session. I have confirmed that this part of the vi runs successfully. 

I also want to save all collected data in the form of a tdms file. My strategy to do that has been to first get raw CAN frames from the CAN signals and then feed those into the write tdms vi. The issue however is that I am unable to add timestamps to the CAN frames. I have run the program and the data login is successful besides the absence of timestamps. 

If you have any tips on how I could add timestamps to the frames I am writing to the tdms file, please let me know. I tried adding a second channel for an array of timestamps but that doesn't seem to do the job.  

Thank you!

(PS. In the VI that I am attaching with this post, you will notice I have assigned random values to each CAN signal. This is to avoid complications related to data acquisition. Once I get this program working, I will substitute the randomly generated values with real time sensor data.)

[viamotors]

Here is the subvi that I use in the main VI. It is simply takes in a CAN database and returns all the signals and frames present in it 

[RTSLVU]

It's been a while since I worked with CAN but I have made a couple Can Bus "sniffers" to monitor a BCM and Li-Ion batteries...

I simply decoded the CAN frame and used the timestamp that was there...

If your CAN data does not have a time stamp then you can just add one when you save the data.

[viamotors]

Thank you for your input. 

However, since I am the one who is creating these CAN frames by using an xnet convert vi to get frames from signals, the frames are not being assigned any timestamps to begin with (there is probably something I need to do to first assign timestamps to frames?) 

Here is a snipper of the part of the code where I am "creating" these frames.

 

I also switched the session type from Signal to Raw Frames to Signals to CAN frames to verify if there were any timestamps being added automatically, but there aren't. Here is a snippet demonstrating that, along with indicators for signal as well as frame data being written.

 

 

I am pretty sure adding timestamps is something I will have to do, I just am not sure how, especially to raw CAN fames, since from my understanding, those are what I ultimately write to the TDMS file.

Looking forward to hearing from you,

Thank you!

[RTSLVU]

---

@viamotors wrote:

Thank you for your input. 

However, since I am the one who is creating these CAN frames by using an xnet convert vi to get frames from signals, the frames are not being assigned any timestamps to begin with (there is probably something I need to do to first assign timestamps to frames?) 
---

I am pretty sure adding timestamps is something I will have to do, I just am not sure how, especially to raw CAN fames, since from my understanding, those are what I ultimately write to the TDMS file.

---

Hmm.. I don't knw anything about creating a CAN Frame but...

It looks like all you need is a floating point representation of the current time added to each frame.

Convert timestamp to floating point

 

Convert floating point to timestamp

Personally I would not store the raw CAN frames. I would decode them and store the actual data in a human readable format.

[viamotors]

Thank you!

Would you happen to know how to add the floating point representation of current time to the frames? That's the part I am mostly struggling with. 

Ultimately I am storing the data as CAN frames that can be decoded using a dbms . The only reason I am creating raw frames first is because that is the datatype that the Write TDMS vi understand and accepts (from what I know). You will see in the code that the TDMS channel is configured as a raw frame data.

If I try to write CAN frames to it, I get an error saying - "Polymorphic terminal cannot accept this data type" 

If there is a way to write signal data/ CAN frame data directly to a tdms file, I am open to learning about it and executing it that way. However, in that case too, I will have to add timestamps to the CAN frames first which brings me back to my original problem.

Thank you!

[Hooovahh]

---

@RTSLVU wrote:

---

Personally I would not store the raw CAN frames. I would decode them and store the actual data in a human readable format.

---

This really depends on the usage of the log after it is made, but I generally just stick with logging the raw CAN data to the TDMS file in the format that NI has semi-officially created.  As mentioned in other places, there is an example in the Help >> Find Examples for logging and reading from a TDMS file that is in this format.  Look for Display TDMS Log file as CAN Frames for the read, and CAN Input Stream to TDMS Logfile for the write.  This format is also supported by DIAdem if you ever want to consume the data there.  It basically flattens the CAN frame into an array of bytes, then writes that data to a single TDMS channel.  This is also the same file format used by the XNet Bus Monitor.

As you mentioned this will result is a very not human readable format.  But in practice I've had a hard time getting useful information out of a stream of CAN data on its own no matter the format.  I mean if you did log to a more human readable format you could have one TDMS channel for the ID for instance, and then another ID for each byte of the payload.  One could then write a program for finding all payloads when the ID is some particular value.  Since this isn't stored as properties, filtering will still require reading all the data from the file, then conditionally displaying it.  It likely can be more efficient than reading a stream of bytes and then throwing out the data you don't need.  But in practice I don't think it matters much.

However based on other discussions, OP is likely wanting to just convert this data to a BLF file.  Then use other 3rd party tools to process the data as needed.  And again in this case, if you really are just logging to a TDMS file for the sole purpose of then converting it to a BLF, then logging everything in a single raw channel isn't that big of a deal.  Also it should be noted that this TDMS file will be somewhat large, since the raw CAN log will have no compression at all and CAN data is highly compressible since it often repeats.  For this reason you should expect the BLF file to be about 1/10th the size.

And to answer the original question, these raw frames will have timestamp data in them.  You can open multiple sessions at the same time to an XNet interface so you can have one input frame stream for logging, then an input signal single-point for displaying data in the engineering units at the same time.

[viamotors]

Thank you so much for your very thorough reply. 

The issue I am facing is that I do not see timestamps for the frames that I create using an xnet session on LabVIEW. I do see timestamp data for frames that are being broadcast on the same CAN interface from external nodes (in my case, a vcu) 

Since the ultimate goals of the vi are to 1) broadcast signal data on CAN bus 2) login real time acquired sensor data, 

My strategy is the following: 

1) acquire signal data

2) write that data to the CAN bus (using xnet write signal point signals data)

3) convert written CAN signals to CAN raw frames (using xnet convert)

4) write the raw frames to a TDMS file 

However, like I mentioned, converting written signal to frames does not automatically add timestamps to them. Here is a snippet demonstrating that. Also, I am attaching the vi wth this post. 

 

I tried using an alternative strategy to accomplish this, where I:

1) write the signals to the CAN bus (using xnet session)

2) confirm they are being written successfully (by tracing them on canalyzer) 

3) read the in stream frames on LabVIEW (similar to how they do in the labview example you referenced in your reply)

4) write instream raw frames to tdms file

The issue with this approach is that I cannot write and then read the same CAN data at the same time. The xnet read instream frames includes frames that are generated by external nodes (eg. the vcu) but not the ones that were written on Labview. I created a community post for that too, and it can be found (along with snippets and the vi) here: https://forums.ni.com/t5/LabVIEW/Write-then-read-the-same-CAN-signals/m-p/4169613#M1204568

I would so appreciate any assistance regarding these issues

Looking forward to hearing from you

Thank you!

[Hooovahh · Accepted solution]

There is no time associated with the signal you are writing.  You are converting the signals, to the raw bytes, then converting those raw bytes to frames, but of course those frames have no timestamp data, because the original signals have no timestamp data. 

I think you can use the XNet Convert to go from Signal to Frame CAN, then write those frames into a single point frame session.  From here you can turn on the echo since it is a frame session type.  Then you can have another read session where you use the XNet Read, getting the echo'd frames to see the actual values going out if that is what you want, and those will have the timestamps.

[viamotors]

Thank you so much! That worked.

The only change I had to make was to write Frames Out Stream instead of Single Point. For some reason, the latter wrote (and then read) only 3 of the 5 frames on to the CAN bus. I verified on CANalyzer that only the 3 frames were being transmitted to the CAN bus, so the issue was with the write session and not the read session. 

Here are the snippets showing that. 

Session type: Frame Out Single Point

 Output (3 out of 5 frames read):

 Session type: Frame Out Stream

 

Output (all frames read): 

This could be a timing thing, I am not sure.

But thank you. Converting sessions to frames first and then writing those frames with echo transmit on did exactly what I was trying to do.

Appreciate everybody's help a lot!