LabVIEW Development Best Practices Documents

cancel
Showing results for 
Search instead for 
Did you mean: 

LabVIEW Data Container

Purpose

 

The LabVIEW Container is a new way to pass data around on your block diagram. It combines the strengths of the array with the strengths of the cluster. NOTE: this is an early release! Containers are still in development.

 

Check out this short, 5 minute video.

 

Description

 

Let me first start off by giving some motivation. I created the Container to combine the strengths of the array and cluster.

 

Array Strengths

  • Dynamic size. I can change the size of the array at run time by adding or removing elements.
  • Dynamic access. I can change which element I'm accessing dynamically at run time using the "Index Array" function.

 

Array Weaknesses

  • I can't embed an array in an array (hierarchy) since LabVIEW doesn't allow jagged arrays.
  • I can't easily access my data by name. What if the index of the element I care about changes? I don't want "Index 3". I want "My
  • Thermocouple Data" regardless of where it is in the array

 

Cluster Strengths

  • I can mix data types.
  • I can get my data by name.
  • I can embed a cluster in a cluster (hierarchy).

 

Cluster Weaknesses

  • I can't change what I'm getting from the cluster at run time. "Unbundle" is hardcoded on the block diagram.
  • I can't add \ remove an element from a cluster.

 

The LabVIEW Container

  • Dynamic size
  • Dynamic access to elements
  • Mixed data types
  • Hierarchical
  • Named data

 

In summary, the LabVIEW container allows you to store named data of ANY type in hierarchical relationships. The strength of the Container is its flexibility at run time. You can change the hierarchy or change what data you're accessing programmatically, almost as if you could programmatically edit or interact with a cluster. The API below describes how you interact with Containers.

 

Introduction to the Container API

 

I've created an API so that you can easily and intuitively construct, compose, access, display and clean up your containers. Below is a brief example of the most basic usage of the API:

Capture.PNG

 

Construct

The first step is to bring a Container into existence and give it a name with the Create.vi. Think of this as dropping an empty cluster constant on the block diagram.

 

Compose

To create a hierarchy, use the "Add Child to Container.vi". This is similar to adding a cluster to a cluster, or (if it were possible) putting an array into an array.

 

Access

The Get \ Set Attribute VIs are designed to allow adding or retrieving data of ANY type from a Container. This is like the bundle and unbundle by name. But, whereas the bundle \ unbundle by name is static on the block diagram. the Get \ Set takes in the name of the attribute that you'd like to interact with similar to the Index Array array function.

 

Display

Because the container is arranged in a very predictable way, it can be presented through many different views. I've implemented a tree view that displays the Container as a tree, which mirrors the "One Parent - Many Children" model of the Container design. The tree below is a display of the "Top Level" container from the block diagram above.

Capture.PNG

 

Destroy

In order to prevent memory leaks, call Destroy.vi. This releases the Container and all of its children recursively.

 

Complete API

Here is a palette view of the complete API. This palette is installed from the VI Package attached below. The most common operations are promoted to the top. Namely, Create \ Destroy, Set \ Get Attribute, Add \ Get Child, and properties. The more advanced functions are located in the Utility folder.

 

Capture.PNG

 

Here are all of the API VIs:

 

Capture.PNG

 

API Explanation

A few of the API VIs warrant some explanation.

 

Get Attribute

The Get Attribute VI returns a value by its name. I designed this VI to be polymorphic for many of the common data types, so that you won't have to cast from a variant to your desired data type.

 

Capture.PNG

 

Get Child Container(s)

The Get Container(s) VI is a polymorphic VI allowing you to get 1, N, or All (non recursive) contained containers.

 

Capture.PNG

 

Cluster To Container

The cluster has many advantages, one of which is the ability to design your datatype on the block diagram. There is a visual representation on the block diagram of your cluster vs. the Container, which has no visual representation at edit time. So I created this VI to create a Container hierarchy from a cluster:

 

Capture.PNG

 

Here's an example of the Cluster to Container:

Capture.PNG

 


Extending the functionality of the Container

The Container is a class based implementation "under the hood". The fact that it is classed based is hidden away by the API. The Container presents no barrier to usage if you're unfamiliar with LV Object Orient Programming. However, if you are familiar with LV Object Oriented Programming you create your own type of Container based on the work I've already done. I've created two child classes that inherit from Container named "VI" and "Control" as examples of how to override the API.

 

"Back end" Implementation

I wanted to spend a few paragraphs describing the implementation of the Containers. The Container fundamentally consists of three objects of the Container Intermediate Representation (CIR) class.

 

Capture.PNG

 

The three objects are intended to house key \ value pairs of three types:

1. Public Attributes

2. Private Attributes

3. Children

 

The public attributes are those edited by using the Get \ Set Attributes API. The Private Attributes are used for information that could "muddy" the contents of the Container. For example, the "Container type" is a Private Attribute. The Children object is meant to store the references to the children contained within the parent Container.

 

I chose this implementation because I wanted to allow a Container to have an attribute that has the same name as a child Container. I also wanted an "easy" system for distinguishing between an attribute and a Container.

 

The CIR itself simply defines an interface between the Container and the organizational mechanisim that implements the key \ value lookup table. I've chosen to implement the CIR as a variant. The variant has the ability to store meta data about a value using the "Get Attribute" and "Set Attribute" primitives. That meta data just so happens to look and feel like key \ value pairs. So I've recycled the variant to act as the organizational mechanism. However, the CIR is overrideable allowing you to use other mechanisms such as XML or perhaps a data base. Using these other representations may allow easier complex querying, etc.

 

Getting the Containers Installed

You'll find the Containers VI Package attached below. The Container depends on the AQ Character Lineator (attached below) and the ni_lib_class_retrieval (attached below).

 

Lastly you'll need the free Open G toolkit which can be installed from the VI Package manager. Upon installation you'll find the Containers on the palette under "Functions". The install location is ..\vi.lib\_ApplicationTools\Containers where you will find 6 API demos.

I've also included a small test suite accessible from the Container.lvclass at ..\vi.lib\_ApplicationTools\Containers\Container\Container.lvclass. Opening the class you'll find the test suite at SubVIs \ Not on palette>>Tests>>TestSuite.vi.

I've created a custom probe which can be found at  ..\vi.lib\_ApplicationTools\Containers\Container\SubVIs\Custom Probes\ContainerTreeProbe_DVR.vi

 

Future Efforts

There is still a fair amount of work to be done to polish things up. There are many "todo" comments and VI Documentation needs, etc. So this is a work in progress. But I wanted to get it out there to start getting feedback. Please feel free to add your comments and requests and how you're using it.

 

New features

1.0.1.39

Fixed how I call into the serializer. Now serialization works!

I've added a "tree" vi to the pallette that contains all of the API vis.

 

2.0.0.4

Fixed serialization of nested containers. Moved the source from vi.lib to user.lib. Source is still compiled in LabVIEW 2014.

2.1.0-3

Including build instructions

Minor bug fixes

reorganized zip file to include instructions and dependencies

 

Known Issues

1.0.1.40 - Deserialization is functional but under some cricumstances can be very slow. I'm still characterizing the bug.

REQUIRES LATEST vi package manager

 

1.0.1.41 - Serialization \ Deserialization was broken for nested containers. I've made a fix, added a test to the test suite, and reved up the major version to 2.0. Please be aware I've moved the location of the container from vi.lib to user.lib as this is not a National Instruments product. You'll find it in user.lib\_Cirrus Logic\LabVIEW Container\Source.

 

Comments
LabBEAN
Active Participant
Active Participant
on

Nice work, Chris!  I'm considering a tree approach on an upcoming job which will involve a test executive.  While I've used both LabVIEW and TestStand extensively, I'm leaning toward toward pure G this time.  Have you noodled on this idea at all?  I could imagine each test step type being represented as a cluster.


Certified LabVIEW Architect
TestScript: Free Python/LabVIEW Connector

One global to rule them all,
One double-click to find them,
One interface to bring them all
and in the panel bind them.
scrook
Member
Member
on

Hi Chris,

This certainly looks interesting. Do you have any preliminary data on how this approach stacks up in terms of performance against something like a variant attribute based associative array? Do  you have any plans to support being able to stuff non-cluster datatypes (maybe generic LV Objects) into the containers? Another thing I am curious about is if there is planned to be any built in methods for diff or merge of two containers?

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Hey thanks LabBEAN! I've considered applications very similiar to what you're proposing. Instead of a test step type, I was considering a Measurement configuration that I would pass to a "DAQ engine". That is, I want to create a subVI that takes in a container of DAQ configurations. Those configurations could be AI, AO, DI, DO etc. The individual containers would represent all of the parmeters for each of the "tasks" that I want the engine to execute. The beauty is the reuse of a DAQ engine. I want to be able to drop my DAQ engine into any application is it "just knows" how to handle the AI container or the DI container, similar to your test executive handeling a container of test parameters.

I like the idea of an engine and I think containers could help enable such an idea.

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Thanks scrook!

So I realized I need to rework some of my wording in the document. You can already put ANYTHING into a container (including a class). It does NOT have to be a cluster. The API vi "Convert Cluster to Container" is designed to create a container based on a cluster.

An area for development would be representing the "anything" in the tree. I've handled many types... but not all. For example and N Dimension array is something I don't handle nor the contents of a class.

I had considered providing a vi that could be overridden for merge and diffing two containers. I think Diff could be "easier" although still complicated. Merge is a complicated task though. But it has crossed my mind and should time permit, those are on my radar.

I have some data comparing the container (before I abstracted it into the Container Intermediate Representation layer). I varied the number of key value pairs from 1 to 20000 for reading and writing. The results are below. I need to rerun these test after introducing the abstraction layer though.

Capture.PNG

scrook
Member
Member
on

Hi Chris,

It's great that more 'squishy storage' datatypes are making an apperance in LabVIEW.

In our company we make pretty heavy use of an associative array class in LVOOP. I put together a whitepaper on that a couple years ago (though we did switch the under the hood to variant attribute lookups for speed).

The way I handled dealing with 'anything' was an everything is a string model internal to the class but the 'getters' and 'setters' would be polymorphic and implement different types. You lose some memory (and pay some unmartialing penalty) when you're unflattening complex stuff like LV Objects from string but it's still pretty fast.

If you have an interest it is located here: https://decibel.ni.com/content/docs/DOC-28267

CharlesB64
Member
Member
on

Great job, that could find a use case for our application  A few remarks:

  • the "serialize container" looks great, but it's broken, I guess it's a work in progress?
  • it would be handful to query a nested attribute, say for example if I want to query the "bool_attr" that is part of the "cont1" child container, I'd like to pass the string "cont1::bool_attr" to Get Attributes VI, instead of having to call Get Attributes two times.
  • I don't understand your requirement of being able to have an attribute and a container of the same name, how can it be useful? 
  • Destroy should execute even if the error in cluster is true (standard behavior for destroy/close/etc in LabVIEW)
LabBEAN
Active Participant
Active Participant
on

<I don't understand your requirement of being able to have an attribute and a container of the same name, how can it be useful?>

I like this feature as it allows us to create sub-containers and attributes programmatically with no regard as to whether there will be name collisions.


Certified LabVIEW Architect
TestScript: Free Python/LabVIEW Connector

One global to rule them all,
One double-click to find them,
One interface to bring them all
and in the panel bind them.
Chris_Cilino
Active Participant Active Participant
Active Participant
on

Hey Charles,

First off thanks for checking it out man!

  • Yup, there's a bug in the AQ Character lineator that needs to get fixed. I'm going to be working with the authors in the next few days to solve that problem. I've listed it in the known issues at the bottom of the article.
  • I really like the idea of a "complex querry". I've been thinking on how to solve that problem and I agree it needs some "beefing up".
  • I wanted the container two distiguish between the two items it can contain (key value pari and container) because the are fundamentally different monsters. I wanted the "rules for one monster" to have a minimal impact on the rules for the other.
  • You're absolutely right about the execution of destroy on error. One of many things I'm sure I've overlooked. Good catch!

Seriously, thanks for the interest!

Manzolli
Active Participant
Active Participant
on

This will fill a gap that has been bugging me for a long time. I have deal with the pros and con of Arrays and Clusters and choose one. Kudos!

André Manzolli

Mechanical Engineer
Certified LabVIEW Developer - CLD
LabVIEW Champion
Curitiba - PR - Brazil
DanyAllard
Active Participant Active Participant
Active Participant
on

Hello Chris,

The VI.lvclass:AddAllControls.vi is broken.

Thank you to provide this it's realy good work

Dany

LabVIEW ChampionArchitect
Chris_Cilino
Active Participant Active Participant
Active Participant
on

Hey DanyAllard,

So that's just a tad embarassing . Problem fixed. I posted 1.0.1.34. Thanks man!

DanyAllard
Active Participant Active Participant
Active Participant
on

Hello Chris,

another one: The Delete Attribute is missing from the palette.

Dany

LabVIEW ChampionArchitect
mike_nrao
Member
Member
on

This is an invaluable tool, and the refinement going on here exemplifies the value of the Community.  Thanks Chris.  My thoughts and suggestions:

Is it possible to package separately the dependencies on AQ Character Lineator?  This dependency feels like a lot of unnessary baggage (only speaking for myself).

Don't stress about the tree view.  Objects and N Dimensional arrays would be tricky, and I can't envision the use-case for showing a tree of datatypes and values to the user anyway.

- Michael

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Great catch! I've added it back in and rearranged the palette in 1.0.1.35. Thanks again Dany!

Chris_Cilino
Active Participant Active Participant
Active Participant
on


Is it possible to package separately the dependencies on AQ Character Lineator?  This dependency feels like a lot of unnessary baggage (only speaking for myself).

I agree with you MIchael. I would like to lower the overhead for depending on AQ Character Lineator. I'll take a look into that.

Chuan
Member
Member
on

This is an innovative data storage tool that overcomes the limitations of using arrays and clusters. And more importantly, it allows us to edit data hierarchy programmatically.

Florent_A
NI Employee (retired)
on

Really nice job

Flo

Chuan
Member
Member
on

Hi,

Check the Print Container API out at https://decibel.ni.com/content/people/Chuan/blog/2014/06/27/print-container-api.

It can recursively print container hierarchy into .txt files.

Print Container.PNG

Hooovahh
Proven Zealot Proven Zealot
Proven Zealot
on

I do not find this valuable.  Sorry.

What is wrong with using the Set and Get Variant attributes?  I'm sure the performance of a single primative must be better then your library.

As for displaying it I would just use the Variant Probe.

http://lavag.org/topic/13748-cr-variant-probe/

Chris_Cilino
Active Participant Active Participant
Active Participant
on

I think that variants are great if you don't need a hierarchy. You could thing of that basically as a dynamic cluster, without the ability to embed a cluster in a cluster. But there are many reasons why it can be very beneficial to create a hierarchy of clusters. The containers allows the embed of a container in a container and the ability to navigate by name.

Hooovahh
Proven Zealot Proven Zealot
Proven Zealot
on

Why can't a variant have a attribute that is a variant?  That makes your hierarchy.  This still has navigation by name.

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Very true, but what if you were given a variant and wanted to know what variant it came from? It is easy to navigate down into the variant table but not easy to navigate up. That's one of the abilities of my containers.

Hooovahh
Proven Zealot Proven Zealot
Proven Zealot
on

If I wanted to know where a variant came from, I wouldn't be passing around the variant I would be passing around the variant's, variant.  If you got a container you also can't know where it came from, unless you pass around the container that contains the container.

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Ahhhh.... that's the trick. I embed in each container a reference to the container that contains it. It is a "two way linked list" like behavior.

Hooovahh
Proven Zealot Proven Zealot
Proven Zealot
on

Okay you got me there, that is a benefit.  I can't think of a time I would want to do that but it is something variants can't do without extra work.

JKSH
Active Participant
Active Participant
on

Hi Chris,

I posted some questions and suggestions at your forum post last month: http://forums.ni.com/t5/LabVIEW/The-LabVIEW-Container/td-p/2838928

Could you please have a look?

Certified LabVIEW Developer
jdunham
Member
Member
on

Hi Chris: I checked this out based on your comment from the LV 2014 new features page from yesterday.  Your library seems really nice but like Hooovahh, I don't really need hierarchical data. I don't even really need named data. What I do need are performant associative arrays.

Like many others, I am using variant attributes, but they would work so much better if I could make polymorphic wrappers for them.  Just think how much more useful queues and notifiers are, now that they are primitives rather than VIs which only handle strings (which was how they debuted).

What I would love to see is something that is almost exactly like the queue feature, but instead of accessing items in a specific order, they would be accessed by a key, which could be any labview data type.  A huge bonus would be to be able to access a single element with an inplace memory structure so that a read/modify/write operation could be done safely and efficiently.  Sure some of that stuff can be implemented with existing LV primitives, but now that we're in the era of "big data", it's long past time when this should have been added to the language.

JKSH
Active Participant
Active Participant
on

@jdunham: There's no reason why performant associative arrays can't coexist with hierarchical trees, right?

I, too, would like easy polymorphism that adapts to any type we throw at it. But I get the impression that such a feature would need to be implemented in the LabVIEW kernel itself (or through XNode black arts) -- something only LabVIEW R&D can touch. I'm guessing that non-R&D NI employees like Chris would be restricted to the same set of tools available to you and I.

Rather than discourage him, let's welcome all efforts to give us better tools

Certified LabVIEW Developer
BrianGShea@NGC
Member
Member
on

I love the idea, i have a similar approach, but i think i like your implementation better. Going to spend some time with this to if i can use it.

Thanks!

Brian G. Shea
Certified LabVIEW Architect
legrimpeur
Member
Member
on

Hi I have been trying this for a project in LV2014 but as soon as I put any VI on the block diagram I get a broken main VI with all sort of errors. Am i doig something wrong? Instaled the latest versions. Thanks for your help

Chris_Cilino
Active Participant Active Participant
Active Participant
on

I'm guessing that you've missed a dependency. The list of dependencies above should still work. Make sure you have them installed and give it another shot.

Intaris
Proven Zealot Proven Zealot
Proven Zealot
on

I recommend you re-evaluate the benchmarking of this code.

Looking at it myself I disabled debugging, put a single timer around the array of attribute operations as opposed to for each iteration and perhaps more importantly made sure both parts of the code were doing the same work - actually returning values.

If you actually USE the values returned when reading (The native Variant already presents the correct value for you) this means adding a Variant to Data after the Container read.  I then average the returned values to make sure LabVIEW cannot optimise anything away.  I see the read speed being factor 4-5 slower than variant with wwrite speed being around 2 times slower.

Bear in mind I have NOT disabled debugging in ALL VIs in the project, just the benchmark VIs.

I would also recommend not setting sub-vis to Time critical priority.  That's not likely to go down well in real code.  Debugging and priority don't seem to change much on my results though.

stbe
Active Participant
Active Participant
on

Hi Chris,

concerning your "Known Issues": - have a look at my latest comment here: https://decibel.ni.com/content/docs/DOC-24015#/?page=4

cheers, Benjamin

_________________________
CLA
CilinoCirrus
Member
Member
on

So I'm getting the chance to user my containers for continuous integration. The use case is i have a complicated hierarchy of VI Packages. I want to call "build" on one partiuclar VI Package and my build system be able to build a VIPs dependencies if there are unbuilt changes or update my VIPs dependencies to the latest version of a sub VIP. To do that I have to be able to construct the tree of dependencies which will involve performing searches, copies, and moves. I also want to have information at each node, like vip name, version or anything else... SOUNDS LIKE A JOB FOR CONTAINERS!

T.L.
Member
Member
on

Hi Chris,

i tryed to build a ppl using containers. Build was unsuccessfull cause of a missing file in Class Retrieval.lvlib:

Source: P:\alsmith\SolutionsEngineeringNI\AQ Character Lineator\Class Retrieval\.trunk\Name to Path Conversion\NTP Conversion.mnu

ClassRetrieval.png

i can solve this for my own, but it should be fixed in the vip.

regards

Thomas

Ethanlu2
Member
Member
on

Hello Chris,

There are some broken VIs in Container.lvclass. What am I supposed to do to fix this?

I have installed open G toolkit and all three vip packages.

mikeporter
Proven Zealot Proven Zealot
Proven Zealot
on

Is this container class part of LabVIEW, or the open g code?

...after all, He's not a tame lion...


Certified Professional Instructor
Certified LabVIEW Architect
LabVIEW Champion

"... after all, He's not a tame lion..."

For help with grief and grieving.
Chris_Cilino
Active Participant Active Participant
Active Participant
on

Which VIs are you seeing broken? I've seen that *sometimes* "something happens" that causes the VIs to break without errors being listed. To solve the problem, I usually uninstall and reinstall the containers package. Give that a shot.

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Hey Mike

The container is netiher a part of LabVIEW nor the OpenG code. At this point it lives on its own.

SAMUEL_NIF
Active Participant
Active Participant
on

Hello Chris,

First of all thanks for this nice job.

I have an issues on the container Serveral Vi broken as Property Node.I am still able to use the API but I do not understand why this VI is Broken.

Do you have an Idea?

containers.PNG

 

Here VIs Broken

containeris.PNG

 

Sam

 

 

 

Samuel G. | GEMESIS

Certified LabVIEW Architect

Certified TestStand Developer

GEMESIS.EU

Chris_Cilino
Active Participant Active Participant
Active Participant
on

Hi @ Samuel,

Thanks man! That VI is broken because, on the palette, the vi is set to "Place VI Contents". So everything should be ok. I'm able to open any of the api demo vis in the "LabVIEW Container\Source" directory.