Interpreting CANdb signal configuration

[Kamen · Accepted solution]

I may not see any (meaningful) responses in this forum, but it'd be because no one here knew the answers. Luckily, I found a few answers elsewhere that have lead me to making good progress. "Not trusting anyone" is a short way of expressing a robust scientific approach to resolving difficult problems also known as "peer review". Just because Max Planck was able to describe the quantum properties of electromagnetic radiation did not mean that Einstein, Bohr, and a ton of others didn't have to "check for themselves" the duality of waves and particles, instead of trusting his conclusions just because "he's Planck, you know". The approach is perfectly valid in engineering, as well; albeit, not as popular.

The proper answer to my question is: because of the nature of the CANdb format - proprietary, evolving and incomplete, yet widely used and "reimagined" by many - there are multiple issues with interpreting its meaning, including ambiguity and contradiction. However, to a large extent, it can be used to translate a dbc file into a set of rules that would help in reading (and writing) values of signals encoded in a CAN message, as long as due diligence is applied and the end-user is involved in the final conflict-resolution process, thus taking responsibility for the final outcome.

I have found answers to my specific questions that are not authoritative but are most-likely correct. I will present them here for the benefit of anyone finding this post with an Internet search (it already came up in some of my searches) Of course, those answers should not be trusted because that's the proper thing to do (and because I'm telling you not to), but they are a good start towards finding the answers that would work for you - it's better than starting from scratch, or not even knowing where to start from.

1) Signed integer interpretation.

This one is the most confusing, as I have found some answers, but they are quite vague and some are even contradictory. It seems that in the beginning only signed and unsigned integers were permitted and, thus, the signal definition only allows for those two (any other types are defined via extended signal-value-type attributes). The case of unsigned values is straightforward, but there seem to be multiple popular ways of representing signed integers in a bit field. What I have established is that most likely it is either two's complement (what your computing platform most likely uses) and signed magnitude (what some communications protocols use). One source claimed that other options are also in use, such as one's complement and BCD (packed or not). Which alternative representation is used in a specific case is probably defined in extended attributes, but in the absence of those, there is a default one that is used, and I can't be sure which, at this time. I will continue my research and see if I can identify a dominant answer. In the unlikely case that multiple alternatives are prevalent, my implementation will have to provide for each alternative.

As for the specific question of scaling signed numbers, as long as the number is represented as "signed" in the bit field, there is no ambiguity about how to apply the scaling factor and offset. The boundaries should not be involved in scaling (or typing; but it gets hairy in the case of floating-point numbers), especially since they are optional, but the issue of conflicting values (e.g., when the scaled value would exceed the supplied boundaries), remains unclear. Some sources suggested that boundaries, when provided, are not enforced (i.e., no signal clipping shall occur); then, what is their purpose? This is a general issue with CANdb: there is very little about validation of attribute values; e.g., if something is undefined or contradicts another, is it an error?

2) Default value of a signal

This one is much clearer: a default value can be defined for a signal in an extended attribute ("GenSigStartValue", in this case). The difficulty comes when one starts questioning the purpose of a default value for a signal. It seems to be commonly understood that if a value is not specifically written to a signal, since a new frame must include the entire message, all such signals will receive their default value. That may cause discontinuities in output physical signals, which is the case with my software, so in my case, all signals receive a "new" value (which is the old value, if there was no update for it) each time a CAN frame is written to the bus. But I have encountered much more complicated interpretations of the signal default value, having to do with the specifics of CAN networks, where some frames must be periodically sent but different signals have differing refresh frequencies, and thus, receivers are instructed not to accept updates when they have the default value. I ignore such vague explanations, but in the back of my mind I have it that there is more to it to consider.

3) Floating-point types

This one is also quite lucid. From a document that I found in multiple places, and which claims to be an official Vector "DBC File Format Documentation", complete with company logos and version (1.0 of January 2007, revised on February 9, 2007, by an editor named "Qu"):

Signals with value types 'float' and 'double' have additional entries in the signal_valtype_list section.

signal_extended_value_type_list = 'SIG_VALTYPE_' message_id signal_name signal_extended_value_type ';' ;
signal_extended_value_type = '0' | '1' | '2' | '3' ; (* 0=signed or unsigned integer, 1=32-bit IEEE-float, 2=64-bit IEEE-double *)

Of course, there is much to be desired here, as well. For one, no one tells you that, but the consensus seems to be that the signedness of floating-point types should always be "unsigned" and then my original assumptions about scaling would apply, i.e., the bit-value is taken as an unsigned integer and then it is scaled to obtain the floating-point number. But, once again, there is no clarity or assurance of any sort.

There are a lot of additional capabilities of CANdb that seem very interesting, such as support of enumerations and text-typed signals, as well as options for rich metadata, which bodes well for my software. But all that vagueness, uncertainty, ambiguity and just plain inaccuracies, make it very difficult to deal with it. Just as another example: in that "official" document cited above, there is a blatant error that is even self-contradicting - the definition of endianness is as follows:

The byte_format is 0 if the signal's byte order is Intel (little endian) or 1 if the byte order is Motorola (big endian).
 byte_order = '0' | '1' ; (* 0=little endian, 1=big endian *) 

I have seen at least one other source that listed the same definition, but this one is easy to refute - it is not a matter of trust, because it can be checked easily. At the end of the document they have a sample dbc file, in which the following signal is defined (among others):

SG_ EngSpeed : 0|16@1+ (1,0) [0|8000] "rpm" Gateway

The endianness symbol is the "1" after the "@", and its value is "1", so according to their definition it should be big endian; however, in big endian ordering, there would simply not be any room to fit a 16-bit number (or any over 8, for that matter) such that its MSB were at position "0". This is corroborated in many other sources, including by the NI-CAN/XNET tools. So, who do you trust? No one but your own good judgment and common sense; and verify, validate, and check everything, when it comes to important non-trivial matters.

I hope this will benefit someone coming here in search of some clarity with interpretation of CANdb.

Kamen