LabVIEW Development Best Practices Blog

Community Browser
cancel
Showing results for 
Search instead for 
Did you mean: 

Re: Why does LabVIEW give all my VIs 'dirty-dots' when I only changed one VI? (and how do I prevent this)

Elijah_K
Active Participant

For those of you using source code control (I hope that's everyone reading this), you may have run into this situation before: You make a simple, seemingly trivial change to one your VIs, and then LabVIEW starts asking you to re-save a dozen other VIs you didn't touch.  Although this may befuddle you, it may go unnoticed for the majority of you reading this.  However, if you have to justify, track or even validate any and all changes to your source code, this can be extremely frustrating.  In addition to the fact that you have to check-in 'modified' VIs that you didn't touch, this may raise some red-flags in a regulated environment, especially if someone else insists on examining your changes.

So first off, why does this happen?  Unlike other programming languages, the VI file format not only contains the source code, it also contains the compiled code (also referred to as object code).  For the vast majority of LabVIEW users, this simplifies application management and avoids having to keep track of the compiled code separately.  However, one simple change may cause LabVIEW to recompile other parts of your application - this can be attributed to optimization routines and the propagation of datatypes.  When this happens, the new object code is saved into the respective VIs, which have then been modified by LabVIEW, even though the source code hasn't changed.

LabVIEW 2010 introduced a new capability to address this problem.  You can now remove the object code from VIs and store them separately on disk.  As a result, the only thing that can cause a change to a VI is the modification of source code, which is what anyone from a text-based background would expect.

The LabVIEW 2010 Help documents how to turn this on for a VI or even the contents of a Project file, but we've had several requests from users to enable this feature by default so that any and all new VIs are saved as 'source-only' VIs.  Thankfully, this can be done using an ini token: SourceOnlyEnvironment=True.  As with any 'secret' ini token, it's not supported and it's considered experimental.  One side affect to be wary of is that you cannot create a normal 'uni-file' with this setting enabled (the VI Property setting is ignored), which is required for dynamically loaded VIs.

Visit the LabVIEW 2010 Help for more information on this topic.

UPDATE

Many people have asked why this isn't just on by default in 2010.  The original plan was that it would be on by default, but benchmarks showed that the operation required to check the cache of object files 'mod date' slowed down load time to the point that we were not comfortable turning it on for everyone.  We also weighed the ramifications for people who were calling VIs directly from a built executables (not a recommended practice, by the way).  Because the RTE does not have a compiler, the executable would not be able to create the object files for source-only VIs and would not be able to run source-only VIs.  As a result, you either have to build VIs into something that includes the object files, or include them with the executable when it's built.  If this affects you, consider packaging your VIs as packed project libraries (new in 2010), DLLs, separate executables or including them in the main executable.  Before you protest, consider the fact that you would never call a .C file directly from an EXE.

Anyway, R&D has been hard at work looking for ways to address the problems we saw with load time and they've come across some very promising solutions using databases.  Without making any guarantees, I strongly encourage everyone to assume that this feature will be the only available option for future versions of LabVIEW.  The bottom line, is that if you have applications you'll know you'll be upgrading to 2011 and beyond, turn this setting on sooner, rather than later.

Elijah Kerry
NI Director, Software Community
Comments
Tomi_Maila
Active Participant

Elijah, thanks for a nice post. One issue I would like clarification. You mentioned that one cannot dynamically load the source only VIs. I am confused. Technically I see no reasons for this limitations. In LabVIEW development environment, LabVIEW does know the location of the compiled binary code and could fetch it into memory once the VI reference is opened. Also if the binary code does not yet exists, LabVIEW development environment has all the means of compiling it, or returning an error when trying to open the VI reference. In runtime environment I understand the restriction, but then again, who distributes source only VIs in runtime environment. Most of the time you build your source only VIs inside an executable or build them as "uni-file" VIs outside an executable.

I guess mostly power users would need source only VIs. And I am sure 95% of power users do use dynamically loaded VIs. So it feels strange if there was such an artificial limitation. Can you please provide more information on the issue. Have I understood it properly. Is there really such a limitation? If the limitation exists, what are the technical reasons behind it?

--
Tomi Maila
Elijah_K
Active Participant

Good question - sorry for the confusion.  The problem I referred to only arises in the run-time environment since the compiler is not present.  The run-time environment needs the compiled code in order to run VIs.  Obviously, in the development environment, the compiler can check for the existence of the cached object code or recreate it.

Elijah Kerry
NI Director, Software Community
Tomi_Maila
Active Participant

Thanks for clarification. When I am building an executable or installer and choose some source only VIs to be placed outside the executable, does LabVIEW compile these source only VIs into uni-file format and place the uni-file format VIs into the target?

--
Tomi Maila
Jim_Kring
Trusted Enthusiast

And, related to Tomi's question, is it possible to easily specify in a Source Distribution that all VIs in a destination are to be made into uni-files?  I ask, because I may want to build an application "plug-in" that's distributed as VIs (perhaps in an LLB).

danny_t
Active Participant

Ellijah,

If we chose not to use token you mentioned is there a scripting property we can use to check and or set the VI property setting.  As we run the VI analyzser on ALL our new and changed LabVIEW code I would be interested in creating my own VI Analyser test (as we can in 2010 now) to ensure the state of the "seperated source and compile code" setting is as I want it.

cheers

Dannyt

EDIT,  thanks I just found what I wanted, quite simple really "containsCompiledCode" sets and get the flag setting for the "Seperate compiled code from source file" so if I open the VI in edit mode I can make sure I have what I want and then do a "Save.Instrument" to save the file

Danny Thomson AshVire Ltd
Elijah_K
Active Participant

Absolutely.  Here's a link to the help file on this topic: http://zone.ni.com/reference/en-XX/help/371361G-01/lvprop/vi_is_unifile/

Also, here's a screenshot of setting the property.  Note that it doesn't actually require scripting - it's just a normal property.

11-15-2010 10-20-24 AM.png

Elijah Kerry
NI Director, Software Community
danny_t
Active Participant

Hi,

Just read your update, and I must say I find it really good that thinks are moving in that direction, I remember all the talk about this on the Beta forums. For myself I think the seperation by default is the right thing to do.  Thanks for confirming the property node for me

EDIT  I just had a question about something going forward, not sure if this is the best place to put this so I may cross post to LAVA.

BUt what should we be doing going forward for thirdparty addons like for example Openg s/w or JKI plugin's should these be saved with source and compile code seperated ? I cannot remember what the situation is for the actual NI supplied VI in VI.lib ect are these seperated  or do they includ compiled code ?

Danny Thomson AshVire Ltd
Elijah_K
Active Participant

Well, it won't be long before all of the VIs we ship in LabVIEW are source-only.  Likewise, it would make sense for all the OpenG add-ons to be shipped as such eventually.

Also, as a general rule, we will probably never automatically convert someone's application to source-only code when you upgrade.  It'll still be something you have to opt in to for an existing project.  Only new projects will [eventually] be set to this option by default.

Elijah Kerry
NI Director, Software Community
danny_t
Active Participant

I hope you are reading this thread Jim,  I must admit I had already thought there is the next VIPM feature as well as the mass compile option a "separate source and compiler option" for when you install a package

I am currently converting a LabVIEW 8.2.1 project to 2010  the project consists of 2100 VI's BUT as I do not use the project explorer the way you would like me to, I only have my top level VI in the actual project, so all other VI's are dependencies trying to set the default setting of existing VI is VERY VERY hard in this situation ,  when you used the project explorer property's tab the "Mark Project VI's button only find the actual top level VI and not all its dependencies. So if you have a large existing project you need to write some LabVIEW and set it one VI at a time

Dannyt

Danny Thomson AshVire Ltd
danny_t
Active Participant

Sorry another question,

I looked at the help but I could not find a list saying why the option to seperate soucre and compile code could be greyed out.

I have a few VI's this is the case for, but I cannot see what is special about them and why I cannot set the flag.

Dannyt

Danny Thomson AshVire Ltd
tst
Knight of NI Knight of NI
Knight of NI

danny_t wrote:


                       

So if you have a large existing project you need to write some LabVIEW and set it one VI at a time


                   

Either that, or create a temporary project with all the VIs in it.


___________________
Try to take over the world!
BradyDuggan
NI Employee (retired)

danny_t wrote:

I looked at the help but I could not find a list saying why the option to seperate soucre and compile code could be greyed out.

There are a few cases where we are currently unable to separate the object code and the source code, which is why the option gets greyed out. Any diagram that contains a ExpressVI or a Statechart VI cannot separate source and object code due to how we embed those subVIs inside their caller. Also a PolyVI can't be marked, because it doesn't really have a diagram (although an instance of a PolyVI can be). I'm guessing you're running into the ExpressVI limitation, but let us know if that doesn't seem to explain it.

vishots.com
Member

I posted a video tutorial of this new feature on VI Shots recently:

Separating Compiled Code From VIs (VI Shots)

lordsathish
Active Participant

Elijah,

I have a couple of questions

You have mentioned that, benchmarks show that load time were too high in certain cases, hence this was made as an option instead of making all VI's source only. Can you throw some light into this ? Are there any specific type of VI's for which these lod times are high? Can you give a simple guideline that will help us enable this option of selected VI's, so that it optimizes the load time.

My second question:

As you have mentioned, LabVIEW recompiles the sub VI's even if you save the top level VI, to optimize the sub VI based on the top level VI. So this means that, the compiled code of each sub VI is unique, and will be different for each instance of the sub VI. Is this right ? Am I understanding this correctly ?

Elijah_K
Active Participant

To the best of my understanding, the load-time hit is not related to the size or complexity of the file.  It's an initial hit that's taken when asking Windows, 'when was this file last modified?' so it should be a somewhat consistent.  As I mentioned, future versions of LabVIEW will use a more sophisticated mechnaism (based on a back-end database) to address this issue.  By the way, if load/save time is a big issue for you in LabVIEW, I strongly recommend switching to a solid-state hard drive - it'll change your life!

Regarding your second question, every VI has it's own compiled binary, and that shouldn't change depending on who calls it.  Typically causes of subVI recompilation include changing a connector pane or a shared data type.  Note that in 2010, you can also opt to in-line a subVI with it's caller, in which case the object files of callers include the object file of the SubVI, which allows further transformations and optimizations to take place.

Having written this, I'm going to ask someone from the compiler team to take a look and make sure everything I've told you is 100% accurate.

Elijah Kerry
NI Director, Software Community
mfletcher
NI Employee (retired)

Someone from the compiler/language team here. Elijah's answers to both questions are correct. The load-time hit is due to disk seeks when looking up stats about the file, and we're working on a solution to that performance issue. For your second question, we don't generate different object files for different callers of a VI. As Elijah mentions, LabVIEW only performs optimizations particular to call sites when you elect to inline the subVI.

apakettQC
Member

Your comment "Before you protest, consider the fact that you would never call a .C file directly from an EXE." is interesting.  In C you would call a DLL.  Are you suggesting that we create LabVIEW DLLs and call them from a LabVIEW EXE?  I thought that was not recommended.  What is the proper method to call a plug-in from a LabVIEW EXE where the plug-in is unknown at the time the EXE was created?

LuI
Active Participant
Active Participant

Look  here: http://vishots.com/separating-compiled-code-labview/, as MichaelAivaliotis stated before in this thread.

Basically you have to use uni-files (VIs that contain both the sorces and the object code) and load them as a plugin at runtime into your LV application (exe).

Elijah_K
Active Participant

apakettQC - this is a great question, to which their are numerous answers.  Assuming that your framework has been built into an exe and you want to call into plugins, I believe the following is an almost exhaustive list of your options:

- Store VIs on disk as uni-files, which then contains the compiled object files, thereby allowing them to be run in the RTE, but also allowing them to easily be modified (although you could password protect them)

- Include your VIs for the plugins in the framework executable, thereby eliminating the possibility to modify them without modifying the framework

- Create packed project libraries to represent each plugin.  See a recently posted example here

- Compile your VIs into separate executables, perhaps allowing parameters to be passed via the command line

- Compile your VIs into either a DLL or .NET Assembly to be called from LabVIEW

Did I miss any options?

Elijah Kerry
NI Director, Software Community
!SuperB!
Member

Elijah,

Regarding your reply ...

"- Store VIs on disk as uni-files, which then contains the compiled object files, thereby allowing them to be run in the RTE, but also allowing them to easily be modified (although you could password protect them)"

An additional note should be added that new for 2011 you need to add the token 'RTEHasObjCacheAccess=TRUE' to the application INI otherwise the RTE will not locate all the object code. We worked with NI to dig up the solution to this a few weeks ago.

Incidentally, I would emphasize that all these options require your application to be written that way so it isn't like it is a simple process of going from one approach to another.

Not applicable

Hey Elijah and everyone,

Thank you all for the information! The setup I have is LV 2010 with TestStand 4.2.1.

It seems from reading the comments that I might run into some issues when deploying the TS/LV code since only the LV RTE will be present and TS calls the VIs directly. Am I understanding this correctly? If so, what would be a good workaround for me?

On a side note, when I use the token and look at the VI's properties, the checkbox is not check. Is this the expected behavior since the token is ''secret" ? (I did restarted LabVIEW)

VIProperties.png

Gary_Wolford
Member

The TestStand Deployment toolset is your best choice when using Separate Compiled from Source option (virtually required when using source control). The deployment tool will compile and combine all of your vi's in addition to all of your other required files. It can be somewhat tedious to set up if your application is complex but worth the effort. Like everything else, don't wait until release date to setup the deployment and test it out before release!

Not applicable

Thanks Gary_Wolford for the reply!

I do use the TS Deployment tool but I guest (wrongly based on your answer) that it would have tried compiling the code and save it in the same directory in my Source Code Control. I just tried it and everything seems to be OK. It must saved the compiled VI directly in the Build Folder given in the TS Deployment tool.

Thanks again!

Not applicable

To asnwer my other question in my previous post.

It turns out that you have to actually add a sub-vi and then save you VI for the Checkbox to be check...Hence, when I created a New VI, the checkbox to seperate the compile code is not check but once you add sub-VIs and save it, it is checked.