LVClass Private Data Reference in Run-Time

[PaulLotz]

I'll grant that there are many difficult aspects to a complete solution that I am sure I don't appreciate.  On the other hand, the deal-breaker of the current solution is the representation of default values as simply default.  If there were a verbose rendering of the default values, even if not in the format we want, we could work with this.  Is this particular aspect of the problem exceptionally difficult?  (It may be.  I don't know.)

In any event, the question at hand is whether a third party should be able to solve this problem.  Currently the answer is no.  Which is easier--to solve the serialization problem or to make it such that a third party can solve the problem?  (I don't know the answer to that one!!!!)

[Daklu]

Serializing objects for sharing with an application in another development environment presently requires a great deal of custom code (we do this!), and I certainly don't see why this should be the case.

I'm confused... I can see passing data objects between different execution environments (LV, Java, etc.) but passing command objects?  Do you maintain copies of all the command classes in each programming language?  I don't see how you could pass command objects without a great deal of custom code.  How do you envision it would work?

Is this particular aspect of the problem exceptionally difficult?

You mean adding verbose rendering of default values?  I suspect changing the code to reflect that behavior is fairly trivial.  The problem is with changing the behavior at all.  Undoubtedly there are some people who (perhaps unknowingly) depend on the existing behavior and their applications will break with the change.  There are valid arguments for including default values and there are valid arguments for excluding default values in an object's serialization.  Regardless of the choice AQ made there were going to be people with the opposing use case who think he did it wrong.

I've done more than my share of grumping about various design decisions implemented by NI, including serializing classes.  I really can't find much fault in the decision to exclude default values from the serialized string though.  I suppose an argument could be made that it is easier to remove default values from a serialized string than it is to add them in, but that will be little comfort to the developer who has to implement code to do that.

In any event, the question at hand is whether a third party should be able to solve this problem. Currently the answer is no. Which is easier--to solve the serialization problem or to make it such that a third party can solve the problem?

At the risk of sounding flippant, which is easier--stopping an irresistable force or making it so a third party can stop it?  Can one task be easier when both are impossible?  A universal serializer is impossible to implement because different classes require different serialization techniques.  If you accept an object's private data should remain private, any sort of external serializer (meaning classes don't need to implement methods to support serialization) is impossible to implement regardless of whether it is intended to be universal or not.

[Daklu]

AristosQueue wrote:
...and it has changed a lot since the last time I rev'd the document...

Post new version?

[AristosQueue (NI)]

Daklu wrote:

AristosQueue wrote:
...and it has changed a lot since the last time I rev'd the document...

Post new version?

Soon. It's a side project and I've had this little thing called an Actor Framework just release in LV 2012 that has taken all my side project time. 🙂

[PaulLotz]

"I'm confused... I can see passing data objects between different execution environments (LV, Java, etc.) but passing command objects? Do you maintain copies of all the command classes in each programming language? I don't see how you could pass command objects without a great deal of custom code. How do you envision it would work?"

Yes, we are talking about data objects, and yes, each development environment implements the command class hierarchy.  (Note only one side needs to implement methods.)  So we simply send a serialized command object via messaging.  We do this now successfully, but it requires the creation of serialization methods on each class.  (Note that this is not necessary in certain other development environments with which I hope LabVIEW intends to compete in the system control arena.)

"Undoubtedly there are some people who (perhaps unknowingly) depend on the existing behavior and their applications will break with the change."

I fully understand and expect that people will want existing applications to work the same way, and I am not suggesting that we break existing functionality.  I think it is quite reasonable, however, to provide a verbose version, either in separate VIs or as an option on the existing VIs.

"Can one task be easier when both are impossible?"

I don't understand this objection, since clearly what we are asking for is not impossible.  In particular, the Flatten To XML VI already provides what we need, except for the default values problem.  (I understand there is another more difficult problem to solve, but we are not asking for a solution to that problem.  By the way, we want a solution somewhere in between, i.e., with an improved XML schema, but this at a different level of discussion.)

Let me reiterate: The question at hand is whether there is a legitimate need for a third party to access this information (information about the classes).  Clearly there are valid use cases (and I see these as critical use cases for being competitive in certain markets).  How will NI address these use cases (if at all)?

[PaulLotz]

Daklu wrote: "Serialization decisions are highly specific to the class being serialized. It is a responsibility best left to the class designer, not the class user or serializer designer."

For the record, I agree with this.  The way Simple XML handles this (as I understand it) for Java is that there is a generic framework but the class developer lists which items to serialize in the class header.  I think something like that is a fine approach (of course, I would prefer a cleaner way to spell out what to serialize than a set of strings).

For our purpose we are talking about serializing data messages formulated into objects (these data messages may or may not be commands) that define communication interface between software components.  As such, we always want to serialize all the data in the objects.  Moreover, we don't have language-specific references (or any references) or the like, simply because they wouldn't make any sense in this context.

Anyway, I don't want to hijack the thread.  I am just endeavoring to point out there are valid (framework-oriented) use cases where the original poster's concerns are quite valid.

[Daklu]

"Can one task be easier when both are impossible?"

I don't understand this objection, since clearly what we are asking for is not impossible.  In particular, the Flatten To XML VI already provides what we need, except for the default values problem.

I suppose I didn't explain it very clearly.  Let me try again... "If you accept a class' private data should remain private, it is impossible."

In particular, the Flatten To XML VI already provides what we need, except for the default values problem. I understand there is another more difficult problem to solve, but we are not asking for a solution to that problem.

I agree it currently is possible to do what you want using the flatten to string or flatten to xml functions. However, my personal opinion is that it was a mistake to allow those functions to work on classes.  (No disrespect intended.)  Not only does it violate encapsulation and open the door to the kinds of problems discussed earlier, but it makes it that much harder for us to get a better LVOOP serialization framework from NI.

I understand you're not asking for a solution to the more difficult problem.  My point is (I believe) it would be a mistake for NI to release more vis or more features to do something that shouldn't be allowed in the first place.  It might simplify your use case, but it makes the overall problem worse, not better.

Let me reiterate: The question at hand is whether there is a legitimate need for a third party to access this information (information about the classes).  Clearly there are valid use cases (and I see these as critical use cases for being competitive in certain markets).  How will NI address these use cases (if at all)?

The questions of "legitimate need" and "valid use cases" depend wholly on your view of private data.  If you believe a class' private data should remain private, then there is no legitimate need or valid use case for exposing that information in the execution environment.  There may be times when having that ability would make a task easier, but imo easier coding alone does not equate to a "legitimate need" or "valid use case."

------

Truthfully, correct object serialization isn't at the top of the list of features I'm super passionate about.  I'd like to see it, but I'll work with whatever vis NI gives us.  However, as someone who has released a third party api to the community, my lack of control over object serialization is a huge hole inviting numerous problems.  It is very, very difficult to guarantee backwards compatibility without significantly limiting the kinds of updates I make to an object.  I'm not talking about obvious breaking changes like vi names and conpanes... I'm talking about things one would reasonably expect to have no effect on backwards compatibility that, in fact, can.  (Though serialization is only one of the reasons why backwards compatibility is hard.) 

[PaulLotz]

I recognize the inherent theoretical ambiguity (or even contradiction) in being able to serialize private data.  On the other hand, all class attributes in LabVIEW classes are private....  As an intermediate approach I would be in favor of being able to designate within a class which attributes (if any) should be serializable.  (Java has a Serializable interface....)

I realize not everyone will use object serialization.  On the other hand, serialization is a pretty core and oft-used technology in other languages and common architectures, and it is a key technology for our networked applications (especially for communicating with a real-time application on a cRIO within LabVIEW but a across a network, but also for sharing data across a network interface with nonLabVIEW applications).  I think it should be a very important point to consider when a team is starting a project (particularly for controlling a large system in a large, multi-language environment such as we have, but in many, many other areas as well) and debating which development environment to select.  LabVIEW has a lot of pluses in this arena.  It would great if an adequate serialization schema could be an advantage for it rather than a disadvantage.  For complex systems in multi-language environments especially I have found this to be one of the most important features LabVIEW lacks.  (It's not the only feature, of course, but it is one of my top three in this arena.)

[Daklu]

As I've read back through this thread I've noticed something that might be causing confusion.  The original poster is essentially asking for run-time reflection.  I have no problem with VI Server returning information about private data fields as long as the property node resides in one of the methods of the reflected class.  I would expect a compile error if the property node were not part of a class method, similar to what you see when trying to unbundle a class outside of a method.  This kind of reflection would be a welcome addition.  It would allow a generic serialization method to be written for a parent class that will work on all child classes regardless of their data types.

I (perhaps incorrectly) interpreted Ryan's, Michael's, and Paul's comments as a desire to use this ability in non-class member vis to create an external or universal serializer they would use instead of the flatten functions.  The external aspect is what I was specifically referring to with my original comments about it being error prone and not a good global design decision.  If the code is intended to run inside one of the reflected class' methods, I have no objections.  My apologies if I misunderstood what any of you were saying.

The way Simple XML handles this (as I understand it) for Java is that there is a generic framework but the class developer lists which items to serialize in the class header.  I think something like that is a fine approach (of course, I would prefer a cleaner way to spell out what to serialize than a set of strings)...

As an intermediate approach I would be in favor of being able to designate within a class which attributes (if any) should be serializable.  (Java has a Serializable interface....) 

I have a really hard time getting excited about this idea. While the current serialization behavior with respect to default values is where you're feeling the pain, the root cause of the pain is the lack of extensibility. The existing flatten functions adequately serialize classes for some subset of Labview users. The inability to extend the flattens renders them useless for the rest of us as we cannot customize the behavior to fit our specific needs. Adding additional vis or serialization option switches to the current flatten vis isn't a sustainable long term strategy--every time a new option is desired we have to go back to NI and hope they agree to implement it. A much better strategy is to have them give us an extensible framework and let us build the functionality we need.  More framework, less functions.

The intermediate approach you suggest only addresses one aspect of serializing a class--namely, which fields to serialize. It does not give the class developer sufficient control over the entire process. Simply selecting the private data fields to be serialized excludes the designer for doing any kind of data transformation prior to serialization. Maybe some fields can be compressed or need to be encoded. Maybe the designer wants to use a bit packing scheme. Maybe he needs to add mutation code to support different class versions. The serialization process requires flexibility only achievable by code, and code requires a method. 

If you accept that the broad spectrum of object serialization requirements requires code then the whole concept of external serializers like the flatten functions becomes moot.  The *one* way I can think of modifying the flatten functions to give control back to the class developer is by having them invisibly attempt to call a "SerializeObjectToString" method.  If the class (or any of its ancestors) contain that method then it is used.  If not it defaults to the current serialization rules.  However, I'd be disappointed to see NI go that route as I think it's unnecessarily obscure and still limits flexibility.

On the other hand, all class attributes in LabVIEW classes are private....

They are private in the context of being unable to unbundle them outside a class method.  They are not private in the context of the execution environment.  If they were the flatten functions wouldn't work and this thread would have died with crelf's response.