Data Acquisition Idea Exchange

The request is for NI to prioritize development of VIs that allow integrators’ CAN data to be measured and stored in standardized formats.

While TDMS and XNET are decent tools, it’s lack of modern development and open compatibility in industry makes it somewhat difficult to deal with. NI could probably expand and service its user base if it supported XNET and MDF4 better.

Currently our process is as follows:

1. LabView XNET data comes from the cRIO FPGA (eg. NI-9853) as an array of 6 U32s
1. https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000000PASzSAO&l=en-CA
2. LabView XNET default data structure is the typdef ‘XNET Frame CAN.ctl’
1. See XNET help ‘XNET Convert (Byte Array to Frame CAN).vi’
2. There are obscure (near obsolete) VIs or the end user must write code to convert from 6 U32 to XNET Frame CAN on poor documentation.
3. As an aside, there doesn’t seem to be a clean standardized way to store either of these structures in the TDMS files
1. There is a way to configure a TDMS channel for CAN frame logging, but it is not easy for signal post processing because it is only one channel of frames.
2. We believe a much better way would be something like this if we stuck with TDMS.

However as to this feature request we have found that the industry standard MDF4 file format to be advantageous in most aspects. 

Our tests found that a 1 hour 1.1GB raw CAN frames and analog data TDMS file could be stored as a 200 MB file in the compact MDF4 file ! When the CANbus data was expanded into actual signals, the MDF4 file was 900 MB (still less than the raw TDMS). Further this MDF4 file loaded into FAMOS for review in ~1 sec !

We would prefer to output directly to MDF4 rather than having to script this conversion process.

1. One key detail is the storage of raw CAN frames such that other industry tools (eg. MATLAB, FAMOS, PYTHON, MDFvalidator, ASAMmdf GUI) can natively interpret and process with various DBC files (the standard includes tools to extract CAN signals).
2. Secondly with high channel counts of fast sample rates, the file size quickly becomes enormous. MDF4 allows excellent compactness, further compression, and usability of data (including but not limited to CAN).
3. However, NI’s DataPlugin needs work as it fails to read much of the structure, properties and data of MDF4 files (as seen from ‘Data File Viewer.vi’ utility below). To unlock the true power of MDF4, the CAN frame has to be written correctly. Currently this is impossible.
4. We were surprised that NI as an ASAM member had not fully supported the MDF4 format.
5. LV vs. free ASAMmdf GUI 
6. Vector’s free MDFvalidator tool useful for seeing the file structure                    

    

APPENDIX: here are some links and resources to summarize importance of MDF4 file format.

1. Link to the standard’s homepage ASAM MDF [asam.net]
2. Link to the MDF4 format’s documentation asammdf "7.3.14" documentation [asammdf.readthedocs.io]
3. Link to a founding company’s page Measurement Data Format MDF | Vector [vector.com]
4. Link to NI MDF4 plugin showing features https://www.ni.com/pdf/manuals/asam-ev-dataplugin-mdf4-2023-q3.html (note that #10 ByteArray is not supported).

5. Why MDF4 is Popular for Testing:

   • Efficient Storage: As a binary format, MDF4 allows for the efficient storage of large volumes of data, which is crucial in extensive testing scenarios.
   • Standardization: Being an ASAM (Association for Standardization of Automation and Measuring Systems) standard ensures interoperability between different tools and systems used in the industry.
   • Metadata Support: MDF4 allows for the inclusion of rich metadata, providing context and facilitating the interpretation of the raw measurement data. This includes information about the signals, units, and test conditions.
   • High Performance: The format is optimized for fast read and write access, which speeds up the analysis of large datasets.
   • Flexibility: MDF4 supports various data types and allows for extensions, making it adaptable to evolving testing needs.

I have a cDAQ-9185 with the 'advertised' TSN feature, yet I can't use it unless I buy LabView and the Ethernet IP Industrial Network Driver. If FlexLogger is for data acquisition, why is it not supporting the TSN features of the cDAQ-9185? I'm disappointed in the way NI advertised the TSN features of the cDAQ-9185 by the way the require all this additional software to be purchased to actually use it. If FlexLogger doesn't support TSN and OPC they should state it.

This is a continuation of another forum post, from here.

Hooovah, the capability I need is very similar to how the "ID CRC" feature currently works, I just need a little bit more flexibility in how the ID is added to the payload bytes, prior to CRC calculation.  As you described it, the 4-byte CAN ID is concatenated to the front of the CAN payload, and by your example, it is being added in "big endian" fashion.  What I require for my application, is to add that same 4-byte CAN ID to CAN payload, but I need it added in "little endian" format, such that:

if CAN ID = 0x1FFF313

and CAN payload = 0x01 0x02 0x03 0x04 0x05 0x06 0x07 0x[CRC]

then the CRC is calculated on:

0x13 0xF3 0xFF 0x01 0x01 0x02 0x03 0x04 0x05 0x06 0x07