Support serialization of LabVIEW objects to interchangeable form of XML

[PaulLotz]

I suggest NI include a native feature to serialize LabVIEW objects in an interchangeable form.  (Alternatively, NI can at least provide enough access to allow a third party to develop such a framework.)

By "interchangeable" I mean in a manner that allows sharing object data between platforms (e.g., between LabVIEW and Java).  (Hence having "default data" without specifying the values of the default data is not allowed.)  Moreover, using a more common format (such as "Simple XML") is appropriate.

Of course, including the object version number is only meaningful within LabVIEW, but this is useful within LabVIEW thanks to the capability of LabVIEW objects to translate between versions.  (Note: I recognize the versioning can't avoid all possible issues, but in practice I think that is rarely a practical issue.)

I understand that for security reasons a developer may want to turn off the ability to serialize an object.  To support that, I envision a checkbox to allow serialization (default = True) in the class properties dialog.

I think XML is the best option for this for several reasons:

1) It is a common way to serialize objects in different environments.  This means that I can exchange serialized data with Java applications, for example.

2) It is readable, albeit not easily readable, by human beings.  (I actually don't want humans to read serialized data very often--and really never the operator, but it is good that they can on the rare occasion when they need to do so.)

Why I think NI should implement this:

1) This is relatively straightforward for NI to do since NI can already serialize a class to the current (noninterchangeable) LabVIEW XML format.

2) Having this capability could greatly expand the application space of LabVIEW, since it would make it orders of magnitude easier to interface with nonLabVIEW applications.  This is especially important in large systems, in which it may be advantageous to implement some system components in one environment and other components in another environment.  This is, I think, by far the most compelling reason to include this feature.

3) That there is a need for this seems obvious, given the number of lengthy discussions just on LAVA about this topic.

4) The current situation, in which each class must contain specific code for serialization, is patently inefficient and nonsensical.

5) In other major languages meaningful object serialization is a given, and LabVIEW should include (indeed, must include) this functionality to be competitive.

For the record, to serialize LabVIEW object data for communication within LabVIEW we use either the methods to flatten to string or to XML, and this works fine.  I realize it's not theoretically 100% fool-proof, because of potential issues across different object versions, but in practice we use version control, so that we build applications using the same versions of interface code (usually), and we only have one large system, so we can pretty easily control our deployed applications.   (I think that versioning an application could achieve the same.)  In practice, we've never experienced a version problem with this approach, and it avoids having to write any class-specific code (which, again, a developer should definitely not have to do) to support serialization.

[Darin.K]

2) It is readable, albeit not easily readable, by human beings.  (I actually don't want humans to read serialized data very often--and really never the operator, but it is good that they can on the rare occasion when they need to do so.)

I believe the strength of XML is not that it is readable, but rather that it is structured.  Unfortunately, the fact that it is readable means that a lot people attack it with Regexes and String Parsing instead of the well-defined DOM route utilizing for example XPath.  A simple style sheet is all that is required to make the raw XML (even for the bloated LV schema) quite easily readable. 

[PaulLotz]

Well, I agree that the structure is more important (with the caveat that I don't want to share binary strings with Java).  The biggest issue with the current native implementation in terms of serializing objects is that flattened LabVIEW objects can just say they have "default data," which is useless unless one has the class definition for the flattened version.  (I'm not sure the description of the class hierarchy is adequate for interchange, either.)

[tst]

---

@Paul.dct wrote:

Why I think NI should implement this:

...

3) That there is a need for this seems obvious, given the number of lengthy discussions just on LAVA about this topic.

---

I remember discussions about persisting objects to disk and then reloading them if the class version was bumped, but I don't remember that many people caring about moving the data to other languages (which isn't to say there isn't such a need, just that I feel that that's not the problem most people there have).

[PaulLotz]

You may be right.  I have advocated implementation of interchangeable XML for years, but the larger discussion has focused more on other issues of serialization.  I think we can solve both issues together.

Here are some related LAVA threads:

http://lavag.org/topic/14946-need-lvoop-object-vis-in-lvdata-library/

http://lavag.org/topic/13939-thoughts-on-class-data-default-values/ (where blawson makes a similar statement about default values that I do, although the application may differ)

http://lavag.org/topic/13905-serializing-objects/

http://lavag.org/topic/15195-serializing-aggregated-objects/

http://lavag.org/topic/12729-examples-messaging-with-objects-command-and-state-patterns-configuratio... (see especially referenced paper)

http://lavag.org/topic/9965-return-class-cluster-type/page__fromsearch__1 (very relevant--Apr. 2009)

http://lavag.org/topic/9865-lvoop-xml-and-java/page__fromsearch__1 (very relevant--Apr 2009)

http://lavag.org/topic/11056-writing-a-value-that-has-a-unit-to-xml/page__fromsearch__1 (adds concerns about units)

http://lavag.org/topic/10961-xml-for-data/page__fromsearch__1

http://lavag.org/topic/8476-unflatten-from-xml/page__fromsearch__1

And some more loosely related pages:

http://lavag.org/topic/12928-lv-rt-xml-support/page__fromsearch__1

http://lavag.org/topic/537-lvoop-messaging-scalibility/page__fromsearch__1 (Doesn't deal with XML directly, but with messaging with objects, and there are many, many pages about this.  We can't talk about object messaging without talking about serialization, and if we are talking about serialization I think we need to include XML in the conversation.)

[PaulLotz]

I have also pressed for this during presentations I did for NI Developer Day 2011 at NI Week 2011.

[Jim_Kring]

JKI's EasyXML provides a solution to this problem -- it's designed as an easy way to generate and parse any XML data using relatively simple mapping between the names of LabVIEW data elements and XML element names. (Or, was that what you meant by "Simple XML"?)

Disclaimer: I work at JKI and helped write EasyXML.

[PaulLotz]

Jim,

We use EasyXML.  Yes, EasyXML is quite helpful to turn the elements into XML ( thanks!), but it doesn't handle LabVIEW objects directly (and returns a message saying so).  We deal with this by writing object methods to break up the data into pieces that EasyXML can handle.  We want to be able to serialize objects directly without having to write a method for each class (which is quite tedious and error-prone).

[PaulLotz]

There is a loosely connected thread here as well:

http://lavag.org/topic/15415-xml-transformationsstylesheets/page__p__92985#entry92985

[PaulLotz]

I just created a thread on LAVA specifically on this issue:

http://lavag.org/topic/15440-object-serialization-xml-and-labview/

[LouisM]

Improvement on LabVIEW's XML serialization is badly needed. Using the Web Services capability included in LV2011, the XML output of a vi which has been configured for web services is badly broken and virtually un-usable:

<Response>
   <Terminal>
      <Name>relay error out</Name>
      <Value>
         <Name>status</Name>
         <Value>0</Value>
         <Name>code</Name>
         <Value>0</Value>
         <Name>source</Name>
         <Value/>
      </Value>
   </Terminal>
   <Terminal>
      <Name>activity change request enquired?</Name>
      <Value>1</Value>
   </Terminal>
</Response>

 The above XML has no structure and has to be read from start to finish in order to get any meaning from it. Names and values are associated by order not by structure. XML should be structured correctly so you can parse out what information you need. If I configured the same vi to output JSON, the result is a little better, a JSON parser can actually make since of this:

{"relay error out":{"status":false,"code":0,"source":""},"activity change request enquired?":true}

 Using JSON to XML online parser I can get a more structured XML:

<?XML version="1.0" encoding="UTF-8" ?>
	<relay error out>
		<status>false</status>
		<code>0</code>
		<source></source>
	</relay error out>
	<activity change request enqueued?>true</activity change request enqueued?>

 This XML actually has some structure to it. For vi's which output arrays the JSON is fairly good but is still missing some proper root elements.

Basically by not putting proper structure around you XML serialization you have crippled LabVIEW's native ability to pass data to other systems without custom parsers.