Labview 8.2 variants are completely different! They no longer contain their size in the first 4 bytes!

[billings11]

It looks like NI made fundamental changes to the variant datatype in 8.2.  It used to be the first four bytes of the variant contained the size of the variant in bytes as an i32.  Now I don't know what is in the first 4 bytes now, but it sure isn't the size of the variant!

Check this example zip if you have 7.1 and 8.2.  They are the same VI, I created it in 7.1.  But then I opened one in 8.2 and saved it again without changing anything.  Notice the variant size is completely different!

I found this problem because the Database Toolkit is all wrong now when it comes to variants.  You are supposed to be able to insert a variant into a database, and if you look way down into the database toolkit you will find a vi called DB Tools Get Element Size.vi.  In this vi it gets the size of a variant from the first four bytes just like in my example.  This is how it knows how to parse through the data and insert the right sets of bytes into the right database fields.  Well now the first four bytes give a totally bogus number, so it tries to insert 20 bazillion bytes into one database field.  All the data gets parsed out all wrong.  I attached another example in a zip with a generic mdb file to support it.  This all worked fine in 7.1. 

Please somebody tell me the variant datatype still contains size information somewhere!  At least then I can fix the database toolkit myself and maintain my own modified version of it.

And NI, please support this quickly.  My world has been brought to a grinding halt!  None of my applications can go to 8.2 because they all save variants to a database.  As much as I am an advocate, you guys can't make fundamental changes to a core datatype and just not tell anyone!  You broke your own toolkits!

[tst]

The variants were changed in 8.0 and I assume that the data size is still found somewhere inside.

The LabVIEW memory management white paper probably documents this.

I don't use 8.x, but I believe that the DBCT should have a version compatible with 8.x. You should probably just talk to your local NI office.

[billings11]

Yeah I was just looking at what I think was the memory management whitepaper.  But it just says Variants are stored as handles to internal data structures, and that the data type is made up of 4 bytes.  I can't find any documentation anywhere about where the size of the data is stored.

By the way, I am using the newest 8.2 compatible database toolkit.  It's not just a mass compiled version from 7.1, its a new release from NI.  I just installed it with the latest service release. 

I only went to 8.2 this week even thogh I get regular service releases because this type of thing always seems to happen.  I spent the last week backing up all my old code (I've been through this before), installing a 8.2 labview's and toolkits on multiple PC's, then mass compiling all my code.  Then opening about a dozen 1200vi+ applications one by one fixing things and creating new projects in the project manager.  I really don't want to have to roll everything back now!

[billings11]

Check out the spreadsheet.  These are the byte arrays produced from the Open Me In XX vi's I attached above.  The byte array is completely different all the way down to byte number 92.  After that they are the same.  So something changed pretty significantly in Labview's internal data structure.  Somehow it is able to read previously stored 7.1 variant data back the right way in 8.2 even though the byte array is obviously different.  They probably made the variant to data function able to decode either type of variant data.  But since the size isn't in the first four bytes, you can't store variants to a database using the database toolkit vi's.

The way the toolkit vi's work is they parse out a big cluster of data for you into the individual columns of data.  You make a cluster with all your columns of data in order by column, whatever datatype they are.  It takes the whole input cluster, flattens it to string, and then turns the string into a byte array.  So now it has two arrays, one array containing the type data from the flatten to string funtion, and one array containing the actual byte array.  So as it parses through, it uses the type data to tell what datatype is next on the byte array.  If it is a double, for example, it takes the next 8 bytes on the byte array and assigns them to that database column.  If it is an I32, it takes the next 4 bytes, etc.  In this way it parses its way down the long byte array from all the data.  Well if the type is a variant, it thinks the first four bytes of the variant contain the size of the variant (the way it was in 7.1), so it looks at the next 4 bytes of the byte array and uses the cast to type funtion to turn them into an I32.  Then the value of that I32 is supposed to be the variant size in bytes.  So it takes whatever that size is and allots that many bytes from the byte array to that database column.  But in 8.X, the first 4 bytes contain something else, so it allots some bogus number of bytes and corrupts that column and every column thereafter.

If anyone knows where in the 8.X variant byte array the size is contained I can fix it myself and not worry about this anymore!  Is there any other way to get the size of the variant data from the variant byte array itself?

[NIJanell]

Hello Billings,

Just so you know, there is actually only one version of the database connectivity toolkit and it was released in 2001.  A patch was released soon after, making the "newest" version 1.0.1, which has been out since well before LabVIEW 7.1.

As far as determining the size of the variant, I have searched and searched with little to no avail.  Is there a reason why you aren't using the variant to data vi?

Good luck,

Janell R | applications engineer

[Odd_Modem]

I wonder if you can make a wrapper vi that takes a var, but uses another technique to talk to the tool kit. Maybe a polymorphic vi? I think poly was added to 'full' for 8.

[tst]

I can't look at the DBCT VIs at the moment, and I'm not sure how your use case is different from anyone else who uses the toolkit (maybe it's because you're trying to store a variant?), but I would suggest you check the type descriptor coming out of the Variant to Flattened String function and not the string itself (and the help for the function).

Also, you might wish to check out the OpenG data tools package, which did a lot of work in this area.

[tst]

Another small point is that you're using the millisecond timer to obtain a unique ID. This is a bug because the ms timer is reset when the OS is reset and every 49 days, so that values are not guranteed to be unique. You should either use a date/time string (which is not guranteed to be unique either, for example when DST goes into effect) or let the DB assign an auto ID.

[TCPlomp]

If I open the 'Open me in 8.2' VI and right click on the conversion VI and select 'Convert 7.x' data I get a length of 111 (looks OK)
If this VI is imported from 7.x into 8.x that perticular option should be set by default!

Can you confirm that?

Ton

[billings11]

First, the millisecond timer is just a bogus thing to generate a unique ID for this example.  It is not the source of the problem or part of any real application.

 Yes, other people have not had this problem because most are probably not saving the variant datatype to a database column.  The reason I use the variant datatype is that I have lots of different datatypes to save and I don't want to have dozens of database columns.  With two database columns I can store any type of data I want.  And it can be complex clusters of waveforms and XY graphs if I want... any typedef I can make in Labview can be stored to a database as a variant and read back later using variant to data.  The first column can say what type of data it is so I will know what data type to plug into the variant to data function, and the second column can contain the variant.  So then when I read the data back from the database I can use variant to data to convert the variant data from the database back to the correct data type.  That way I can add new datatypes to save and maintain the application for years without ever having to add columns to the database.  I can add twenty new data types and never redesign the database.  This saves weeks or months of labor each year for me.

Think of it like this, I might have to execute a test vi that has 10 different parameters.  One of the parameters might be a waveform, one might be an integer, several might be doubles.  I can take all of those parameters and put them into a single strict typedef cluster.  Then I can turn it to a variant and save the whole parameter set into a single column of a database.  This means one database column instead of twenty.  I am trying to be object oriented, treating test results as generic entities rather than specific type sof data.  Then the results of the test are the same way, I sometimes have XY graphs and waveforms and numbers and booleans all coming out as test results.  For each test type, I just define a strict typedef for that result type and save all the different types to the same column of the database.  Within my software I handle all parameters and results as variants until they get inside the lower-level vi's that actually need to pull out the data.  It makes everything interchangeable, and before LVOOP it was as object oriented as you can get in labview.  (I am very excited about LVOOP but I need to solve this first).  So instead of hundreds of database columns to describle all the parameters and results, I just have 2, by saving all the different types interchangably as variants.

 Saving variants to a database is supposed to be supported by NI and is clearly written into the database toolkit in version 7.1.  I have been doing that for a couple of years now.  If you look at NI's comments in the toolkit code it says the first four bytes of the variant should contain the size of the variant.  Please check the NI toolkit code regarding the type descriptors coming out of flatten to string.  I am just using it on the front panel to show the real size.  If you follow the data down into the database toolkit you will see that this type descriptor from the flatten to string function is what describes the data type as a variant and drives the whole bug.  It is NI's own code that is trying to pull the variant size out of its first fiour bytes.  The other vi's are just something I made to illustrate the problem.

 As far as the last comment about converting to a 7.1 datatype, I don't know anything about that.  What is the 'conversion vi'?  Do you mean the data to variant vi?  Because I am right clicking that and I do not see that option.  I am not sure what you are right clicking on there.  All I did was create the vi in 7.1 and open it in 8.2, and the data size was completely different with the first four bytes no longer representing the size.  I have never seen any option to convert to 7.1 data types.  This is a new installation of 8.2 over 7.1, and that option is definitely not set by default because I've never changed it or even seen it.  Just open the 7.1 vi in 8.2 and run it, you will see the data comes back as totally different sizes.  

Ideally someone from NI could tell us how the variant datatype is different, why was it changed, and how can we extract the size of the variant data from the new variant datatype?