CAN Frame Signal Conversion

[Hooovahh]

Description

Description-Separate-1

Overview

The example demonstrates how to perform CAN Bus Frame and Signal conversion, leveraging the NI-XNET API for parsing the database.

Description

The attached zip contains an example VI demonstrating the signal to frame, and frame to signal conversion, using a class.  This code uses the XNet API for database parsing but the conversion itself is done with pure G.  The Frames to Signal conversion function is a polymorphic VI and supports 4 different modes:  Single Sample Double, which reads the latest value for the signal, XY which reads every value along with a timestamp for each value, Waveform, which reads every value into a waveform, and Single Sample String, which reads signals and displays them by converting doubles to strings, and displaying enumerated signals as their string representations.  Going in the reverse direction is also supported with the String to Frames supporting an array of doubles, or an array of strings.

Requirements

 Software

• LabVIEW 2015 Base Development System (or compatible)
• NI-XNET 14.0 (or compatible)
• OpenG Libraries (install over VIPM)

 Hardware

• No hardware is necessary to use this example VI
  (Although the NI-XNET software is used, there is no need for NI-XNET hardware)

Steps to Implement or Execute Code

There are three examples on the root with instructions in each.  For the Basic and Mux example just run the VI and look at the output.  For the Conversion Speed Test, select the conversion test to perform by changing the tab value, then run the VI.

Additional Information or References

Changelog

• Version 8. All conversion is now done with raw G code.  The XNet API is only used for reading signal information like scaling and string enumeration.  Converting to and from strings, using the signal enumeration is added as polymorphics.  The Init now supports a DBC or a database name.  New properties for reading and writing data added.  Speed of conversion greatly improved with Conversion Speed Test added to demonstrate the differences between Version 7 and 8, and the NI method.

Spoiler

• Version 7 removes the NI-XNET conversion API for frames to signals, and implements the conversion with G.  This makes the conversion process about 10 times faster.  Signal to frame still uses NI-XNET conversion. NI-XNET 16.0 added new restrictions to the characters allowed in an alias, and so Get Alias from DBC was updated to replace invalid characters with underscores.
• Version 6 added CAN and LIN Frames to Table VIs and added them to the demos. Also added NI-XNET Frame Conversion which is a polymorphic VI that converts from NI-CAN frames, neoVI CAN frames, and NI-XNET LIN frames to NI-XNET CAN frames.  Other polymorphic types can be added.
• Version 5 handles muxed signals slightly differently.  If a signal was requested to be converted which is a muxed signal, but the multiplexer wasn't specified, then when the conversion takes place the multiplexer values will not be returned from any of the Frames to Signals conversions.
• Version 4 added better support for multiple conversion sessions at once.  It does this by generating unique names in the Get Alias from DBC instead of using the same temp name.  The Close was also modified to help cleanup these temp names, which correspond to files that don't exist anymore.
• Version 3 added multiplexed signal support.  Frames can be made from muxed signals, and converting frames back to signals will return nothing if the multiplexer value is not correct.  If no signals a multiplexed the conversion works as it always has.
• Version 2 added which sets the payload size of all frames to 8 bytes on start.  This avoids a bug in the XNet conversion library. 

**The code for this example has been edited to meet the new Community Example Style Guidelines. The edited copy is marked with the text 'NIVerified'. Read here for more information about the new Example Guidelines and Community Platform.**

Description-Separate-2

[TestEngie]

Hooovahh, I think what mattyG is saying is that the library treats all values as unsigned, and ignores the Data Type from the DBC:

I added this functionality to your library for my use by modifying "Convert Payload to EU" and adding "Data Type" and "Number of Bits" to "Cluster - Read Conversion Settings.ctl", perhaps this saves you some effort if you choose to implement it yourself:

[Hooovahh]

I mean I get that my code didn't read the Data Type and use it, but I'm trying to figure out when that would matter.  At the moment my code looks at the Scaling Offset and Scaling Factor.  So if your signal is Signed, and your offset is negative, then your resulting output can be negative depending on the payload.  It will always be a double obviously but it will result in what I believe is the correct value.

Now if you have an inconsistent database, where the offset is negative, but the data type is unsigned, then the result will be a negative value which one could argue is incorrect.  But in this case I'd suggest the database is wrong for allowing two things that appear to contradict each other.  That's why I was asking for an example of a conversion that results in the wrong value. 

Here is an example where the conversion results in the correct negative number, for the signed data type.  If there is more to this issue and I'm doing a conversion incorrectly for some cases I can update the code.  For performance reasons I try to keep that conversion part as minimal as possible.  And in fact I don't like that type cast and may try to code improvements.

[TestEngie]

You are correct that with an appropriate value for offset, that you can obtain negative values from unsigned datatypes. However, this ignores the Vector DBC file format, and FIBEX format used by XNET, which allow for setting a signedness for signals. 

Practically speaking, it means that my existing DBC/FIBEX with 1000's of signals is not interpreted correctly, as it was written with the expectation that signedness would be followed. 

In regards to your example, the value of the signal is the same for a payload of 00 for both signed or unsigned. However, for a payload of F0, your code would return a signal of 240-10=230. Whereas if the signedness was respected, you would get a signal of -16-10=-26

[Hooovahh]

OMG I can't believe it took me almost a year to look at this again and realize you are right.  In all cases that I've seen Signed signals, they always just had a huge negative offset, so the value from the payload would always be positive making it less negative, then eventually positive as it grew.  So if it was a 4 bit signal, you might have an offset of -4.  Then if you had 0b0111 (0d7) you'd get a positive 0d3.  But if it was 0b1111 what should the value be?  Well it is obvious to me now that you have the -4 offset, then apply the 0b1111 which is now -1 in decimal.  Applying the offset means the value should now be -5.

Okay with that out of the way I see the need to read the data type.  But I don't fully understand the logic to handle the signed payload.  It might be right but I don't understand it.  The logic that I think of is see if the highest bit is set, and if so get the most negative number (based on the number of bits) then add to that, the positive portion of the value (so taking away the signed bit).  I actually subtracted the positive portion, then negated it just so I could stay in U64 up until the end.  I don't do much bit manipulation like this so I don't know the most efficient method, but I think I can cut some corners with some precompiled array of values of things that get indexed, since the number of bits will be 64 or less, and in practice it will be 52 or less.  Here is what I came up with.  Am I on the right track?

Also sorry there still is no CAN-FD.  I have it, it works, I've been using it, but it uses some internal libraries that I need to spend some time working on.

[godel]

Hi Hoover, 
I follow your ideas to create our own "Conversion" classes. And there are three points to discuss with you: 

1. In you code, the type of "Bit Length Mask"  is U32. The type of it should be set to U64. 

2. The way we adopted to handle the "Signed" signal (in 2's complement representation) is as the following picture for your reference: 

3. In CAN-FD, the length of payload is not constant and can be greater than 8 bytes. So we cannot use the way of bit operations with 64bit mask to get the signal value and vice versa. What is your way to get the signal value of a CAN-FD frame and vice versa?

Thanks

Godel

[Hooovahh]

Hi Godel.  I have a preliminary release of an update which handles CAN-FD, and the negative signed values in a way that I think is correct.  I've put them on my Google Drive just because I'm unsure if these will be published in this state, or if there are more changes to make.  First install the XNet Tools package, and its dependencies which it should find on the internet.  Then install the CAN Frame Signal Conversion package.  After installing it you should have the Hooovahh palette, and then CAN, then the Frame Signal Conversion library which is an updated version of what is posted here, but uses LabVIEW 2020 or newer. 

I handled the Signed conversion differently than you and I believe we should get the same result.  I used some precompiled arrays in the hopes it would make it more efficient.  I also handle the CAN-FD conversion in this release.  I did this by looking at single Quads of the payload, or 64 bit chunks.  If a signal fits within a single Quad the conversion is pretty straight forward.  If it straddles two quads then it is more complicated.  I'm fairly sure it works but there are corner cases that might need more attention, like signed values that span quads, can have issues.  After some more testing and updating I plan on posting these packages to VIPM.IO.

[enderwiggin]

Would it be possible to get the CAN Frame Signal Conversion package compatible with LabVIEW 2019?  Stuck with using an older version for the time being.

[Hooovahh]

Version 8 should work in LabVIEW 2015 and newer.  I don't have 2015 (or 2019) to test with at the moment.  If you are referring to the beta packages I posted in October then you might be out of luck.  The oldest version I intend on supporting is 2020 because it uses array VIMs, which would also need to be reverted.

[Michael_78]

Hi! I am using these great libraries, thank you! Today i found i needed a signed value in a DBC and quickly found my way to this page 🙂  However, when i install the packages posted on 10-06-2023 there is a problem opening my project. VIPM has been adding sufixes to the packages. For example LabVIEW cannot find the file Frame Signal Conversion_Conversion.lvclass or CAN Frame XNET Model_XNet.ctl.   The underscore extension was present in earlier versions of the packages but not in these. Is this an issue with the packages or my VIMP? Please could you look and see? Thanks!

[Hooovahh]

@Michael_78.  Earlier versions of the code did have an underscore with some identifier.  Really early versions of VIPM needed to have unique name space and so I got used to adding that.  In the later years VIPM has improved to where you don't need to rename files to make a package.  This makes development easier since if you want to use the source, or the installed package files, you just need to load them into memory first then it will relink to it since it is the same name.

What this means for you is that if you used the older stuff, using the older name, you'll need to replace those VIs and type def controls with the new package ones.  Sorry.  Also the shorter name helps with making builds since building on a temp location may create a file path that is too long on Windows and cause problems.

[Michael_78]

@ Brian Thank you! Im a bit stuck now because i have had to do some fixes on early versions and i wanted packages, so i re-packaged everything! I was not aware these VI's were being actively developed, my bad!  Now I could go back to using the latest packages but if then there is the issue of fixes.  I would be happy to share the details and see if you would integrate the changes.  Is the package source available anywhere or do you maintain it yourself?

One fix example is DTC reading where a defined message may already be in the CAN buffer from a failed previous read, so I fluch the RX buffer just before doing a diagnostic request.

Another issue is with Kvasser I changed the init so that you can set silent mode before going on bus, this stops the kvaser generating error frames if the baud rate is wrong for example.

The next thing i was going to do was make all drivers plug-in classes.  This is so we dont have to include all dll's to make the software work but also to make it so we can add other hardware:  maybe sbRIO CAN, FPGA CAN, arduino CAN etc..

The libraries are amazing thank you!!!  I would be keen to discuss further. I am keen to colaberate!

thank you,

Michael.

[Hooovahh]

Any suggestions or bug fixes are appreciated.  There's been several issues found here that have helped me, and others so whatever you can share would be nice.

As for driver specific versions.  Internally the CAN code I have is quite a bit better than what I've posted online.  Yes I've developed them to be a hardware abstraction layer with a class for each hardware type.  Vector, Kvaser, NI-CAN, NI-XNet, and Intrepid.  I did start down a path of Arduino CAN but couldn't get the performance I wanted. A Teensy seems way more capable but I never finished it.  Then a separate classes for UDS, XCP, A2L parsing, CDD parsing, BLF and TDMS logging.  Then I made some helper sessions that handle asynchronous reading and writing.  This is updated for CAN-FD, and 64 or 32 bit platforms when possible.  I'm quite proud of it but it does use some code that I would probably not be able to license out to be shared.  So whatever you can share I'll try to update the publicly available packages.

Ideally one day I would post these things over on VIPM.IO so that VIPM finds them and there is a good place to publish.  As for source I'm the only owner of it and it isn't online.  It is just a collection of VIs, and anyone can make changes but there isn't a repo anywhere.

[Hrachya]

Hi Hooovahh,

I think I found an issue and fixed it in my side, You can check also one more time youself.

Source: Convert Payload to EU.vi

in thsi VI the case for "Data Type" had a default value "1". when the data type is IEEE Float, Data Type value is 2, and the code entering to the default case and trying to do data convertion as Unsigned data, instead of doing conertion as Signed data type. Simpley I changed the default case to "0", and the code is working right. 

You can easley check the issue creating 1 signal for example with data length 3 bytes and data type signed with the payload of for example  FF FE 53. and by the same logic other signal with the Data type Float (data lenght automaticly becomes 4 bytes) with the payload FF FF FE 53. in both cases You should have the same dbl value at the out. but current code working wrong, by changeing the Default case to the "0", at out You will have the same number for both examples.

Hope I explained it right. open for any discussion. thanks

[Hooovahh]

I am unsure, but I think I have that fixed in a planned future release.  I have a case 2 where it converts it to a U32, then a Single data type.  This should support the IEEE Float type.

Notice that the block diagram constant is a Single, not Double data type. This is done by changing the representation by right clicking on a Double constant.

At the moment I don't handle this properly if the signal spans Quads.  I feel like that is a pretty rare case, but to be fair I should be able to handle it with some extra math, I just haven't yet.