A Tidy Project is a Happy Project

[swatts]

Howdy Doody my precious programmers

I'm often asked "How do you keep your project so shiny, clean and smelling so minty fresh?"

Well first let's have a look at a clean project

This is our General Application template and you'll notice that the main VI (GenAppMain.vi) sits atop the hierarchy and that's because he is the king and everything should be relative to him. This is how LabVIEW like things organised, it will always look below itself in the hierarchy first as standard behaviour. You'll also notice that we use auto-populating folders showing the entire hierarchy, the way we structure our projects lends itself to having no virtual folders (IMO it's an abstraction too far) Dlls, config database file, startup VIs are also at this top level. All of this stops LabVIEW going searching when you move the project around. Keeping everything at this level also helps when you build an executable. It's much easier just to search in your own directory. Like Unit Testing? well look in the TestVIs directory (we are trying to use Teststand if we need to do Unit Testing, this is quite new to us as we far prefer lots of functional testing) Next we need to make sure there are no conflicts. Click on the exclamation mark and solve the problems. What we are aiming for is a portable project. Ideally I want to dump my project directory on a new computer and the it to work. At the very least I want to be able to move the project around on the computer you are developing on. All this preparatory work is to allow us to upload the project into our repository (SVN). If the project is portable we can then download the project onto any SSDC machine or even across customers machines. We take an uncompromising view on dependencies and in truth anything outside of SCC (Source Code Control) is something that can change how your software works in unpredictable ways. See here for a discussion on this. With this in mind we can see that the Dependencies section is pretty light and we aim to keep it that way.

Project Flossing

Over time scrappy bits of old project can get caught in the gaps, and these need cleaning. This is because they are indicative that a lack of control and understanding. One common area that needs regular flossing is in the Dependencies.

In this example we can see that although there are no conflicts, there is still some ugliness in the Dependencies section.

Cleaning these out is fairly straight-forward, you just need to find the unused VI in the project that is referring to these dependencies. But if you use dynamic VIs you should employ the following trick first.

Sticking any dynamic VIs in a non-called case of the main calling VI will not effect the running of the VI (LabVIEW will compile them out), but it will allow the IDE to tell you if they are broken. It will also allow LabVIEW searches to ascertain if the search VI is in the hierarchy.

Now you are ready to floss....

Right-click on the offending Dependency and select Find>>Callers, because the main calling vi is not broken and all dynamic VIs are on the main calling VIs block diagram we can be pretty sure that there is a broken, unused vi in the project. Find it and delete it (using the SCC delete function!)

Also notice the \user.lib folder, for us this is a sure sign of something untoward in our project structure. This is because we NEVER use user.lib. Also view instr.lib with suspicion, anything that is not part of the standard LabVIEW install should be moved out and put under the project (use lvlibs to protect the namespace).

Having a tidy project makes cross-linking and unpredictable loading a rare thing and it may even prevent gum disease.

Lots of Love

Steve

[GregPayne]

Yet another informative article by Mr Watts,

You mention that you stay away from instr.lib. When you are using an instrument driver, do you have that driver in your project folder for each project? (Duplicate copies) The same applies to common reuse code, does that go in your project folder, for each project?

One process I always dislike going through is setting up a new PC and installing all the nessessary drivers into instr.lib. Having them packaged with your project will make this whole process much quicker and easier, just copy and paste a single folder.

[swatts]

Where practical we want everything under the project, so the short answer is yes. The slightly longer answer is that we strive to avoid loading toolkits, addons, packages etc. So we'd certainly commonly remove drivers from the instr.lib wrap them in an lvlib. Even stuff in the vi.lib will sometimes get the same treatment. An example is the ftp toolkit. It is missing functionality we need, so rather than change it for reuse across projects we have another copy in each project.

[Terry_ALE]

I am following your posts more and more Steve and often share them with my team as good examples or at at minimum good discussion seeds, looking forward to meet in person in Berlin this April.

When the project came out in LV8 we used auto-populating folders.  We have since gravitated away to virtual folders, auto-populating folders remain the exception.  We felt that there may be some developer offshoots and experiments in the folders that we didn't want in the project but also didn't want to remove them from the directory hierarchy.  I like the floss analogy so maybe those extra files should be flossed out and noted somewhere in the project notes as 'unused nuggets'.

I noticed your screenshot has auto-populating folders.  Are they the rule and virtual folders the exception?

[FabiolaDelaCueva]

Steve,

I find this article very interesting, specially since it is very different from the way we organize our projects (see: http://delacor.com/a-method-to-project-madness/). It seems that you embrace auto-populating folders, where we embrace virtual folders under project libraries (.lvlib).

The flossing of dependencies is handled in our case by creating an auto-populating foler of the library folder, the .lvlib take precedence to auto-populating folers, so if there are .lvlib in the auto-populating folder, the auto-populating folder will only show the VIs and controls that are not claimed by the .lvlib. This helps to clean up: either a VI has to be deleted because it is no longer used or it has to be moved to be part of the library, because it is lost and we forgot to make it part of the library. Other than cleaning up, we don't use auto-populating folders that much.

Regarding the Dynamic VIs example, why not use the "diagram disable structure" instead of the case structure with a false constant wired to it?

You and us have had the discussion on dependencies several times, we favor using VIPM packages and a vipc file with all the dependencies for the project. With VIPM pro, instrument drivers that were downloaded from ni.com/idnet can be packaged into a VIPM package and included in the vipc for the project. We rather have that than end up with copies of the same instrument in different projects and risking there being slight differences between them. We can still have different versions of the same instrument, but VIPM package has the added benefit of having release notes for each version where we can list what is the slight difference for each version. The same approach could be followed using SCC externals.

Thanks again for sharing your thoughts.

Regards,

Fab

[swatts]

Hi Terry,

I use auto populated folders only, we all tend to agree (within SSDC) that we like to know where stuff is on the hard drive. Our view is that experiments etc are still part of the project, while it is actively being developed at least. Like most things with us, it's easy and doesn't give us problems.

Sadly I won't be in Berlin as we're doing NIWeek this year .

Glad you find the articles of some worth!

[swatts]

FabiolaDelaCueva wrote:

                       

Steve,

I find this article very interesting, specially since it is very different from the way we organize our projects (see: http://www.walkingthewires.com/2015/06/07/a-method-to-project-madness/ ). It seems that you embrace auto-populating folders, where we embrace virtual folders under project libraries (.lvlib).

                   

One thing I've learnt when designing processes is that if there is extra manual process involved I won't do it (ditto my colleagues). We're just lazy I guess but it's been a fairly constant thing in many years of putting processes into factories.

So auto-population gives me the gift of not having to do it myself. So a flawed system that takes no effort is better for us than a perfect system that takes some effort. All these are methods that work for us and the most interesting thing for me is hearing what works for others, especially this kind of internal methodology. (I'll say this quietly, I find it more interesting than talking about LabVIEW).

FabiolaDelaCueva wrote:

                       

Regarding the Dynamic VIs example, why not use the "diagram disable structure" instead of the case structure with a false constant wired to it?

                   

The simple advantage is that the IDE tells me if the dynamic VI is broken, updates on a save-all and transfers when I do a Save..As and save the entire hierarchy (a forceful way to clear-out the project).

I'll re-iterate that we need to add THIS WORKS FOR US... at the end of every proclaimation. Age has shown me that there are things that are just wrong that need challenging, and there are things that are 50 shades of right and understanding these lead to improvements.

Thanks for your input everyone.

[joerg.hampel]

swatts schrieb:

                       

Sadly I won't be in Berlin as we're doing NIWeek this year .

                   

I'm sorry to hear that, I was looking forward to meeting you again. It was fun last year 😉

[swatts]

It was fun, and my daughter rates Berlin very highly. But it's not fair on the rest of SSDC if I spend all company time and resources on jollies. I have a solid business reason to go to NIWeek, with the CLA summit the justifications aren't quite as tangible so both trips on one year is hard to find money for.

[JÞB]

I'm surprised by the use of automatic pop folders over virtual folders.

Frankly I don't like them.  I have the files view and the whole right side of the project explorer window to show paths.  Since my folders are virtual I can even move files to better locations when needed. And reduce sub folders on disk  and prevent SCC  source distro, zip etc... long name errors.

[swatts]

Sorry about the late response Jeff, no notification!!!

I don't need or want the project manager in LabVIEW to manage my other project documents for me. It always seems like just another abstraction to manage to us. So our LabVIEW project sits in our project directory, the project directory is the repo that gets source code controlled.

For us the benefits of Auto-pop folders is that they don't hide anything to do with the project, I like that.

[JÞB]

We could discuss how many windows are needed to manage a project.  Having the documentation referances in the lvproj has helped me in the past when those documents did "Bad Things " while the code was under Development;)

[swatts]

I think it comes down to personal preference really. Our method has worked nicely for lot's of projects and we don't have many issues.

I'm willing to bet your method has too...

[JÞB]

For me,  auto pop folders hide things done by someone else... as a single 8-Ball Operator...I simply prefer to be paid for what I was asked to do when I did that work .  And I will be happy to charge you for the updates 

[swatts]

I've had a similar discussion with Mr Relf, and I think we ended up agreeing to disagree.

Here's my perspective....

I don't want the project to hide anything from me.

If a customer has been poking around in my project directories I want all the alarms to go off. It's the antithesis of what I want in my designs where I want superfluous information hidden from me, with projects I want everything visible.

That said I put a lot of energy into making what I see make sense,  so managing dependencies, hierarchy, naming and file layout are all fairly important.

[FabiolaDelaCueva]

Jeff, Steve,

I think it boils down to consistency. The problem is when the same project has a mix of auto-populating folders and virtual folders.

Steve and I had had this discussion several times and I even wrote our approach here https://delacor.com/a-method-to-project-madness/

In our case, we rely on project libraries and we only use the auto-populating folders during clean-up time because it is then when you can see files that are within the library directory but not in the library itself. The question then is whether the file should be deleted or added to the library. 

Both approaches work as long as the whole team is consistent and everyone understands why we do it one way or the other. 

Regards,

Fab

[JÞB]

I'm not gonna restart a fight.  Auto pop folders are needed! 

In both cases Fab mentions,  unless...you use the project explorer files view...you use lvlibs and you use templates! 🙂  

The lvproj should warn*1 when dependent lvlib members are badly in/ex cluded.  Unless the lvproj  is clean!  

And, keeping the lvproj CLEAN AND FLOSSED forces developing users to use the tools right!

OK, maybe it's flame war restart.  Let me say it out loud. 

Auto pop folders are never needed in controlled source code.  But, like goto, an environment must support them

[swatts]

We can revisit that when I'm shown some controlled source code.

Genuinely, I still fail to understand the antipathy towards them, they work splendidly for us. I certainly wouldn't place them anywhere near the goto issues of old.

Also who has declared that auto-pop folders are wrong? I seem to have missed that particular bit of LabVIEW dogma. I would counter your argument with the the theory that if auto-pop is causing you issues you may actually have problems with your project structure that going virtual masks.

Maybe it's where projects share code, all of our projects are atomic so all code always spawns from a single hierarchy. There's obviously something here we're missing in our method.

[JÞB]

They,   auto pop folders,  would work splendidly for any team that works with you.  A lot of people have not had that privilege.   

 And. The declaration that automatically populating folders are wrong...probably   that was raised by yours truly;)

Evidence of the efficiency of using APFs,? I would wager large that it Works well in SSDC after SSDC On boarding. 

How does it work when taking over a stalled project?

[swatts]

Part of my latest article tackles this, so part of our process for rescue jobs is to tidy up the project structure, and this involves removing all the virtual folders...

Rescue Job (Code Review) - Process Description

I think that we may have slightly different use-cases tho'. For us we want to get a project to a stage where we can support it on a fixed price (this is a destination, some projects and customers are on the journey with us). This is a fairly common way to work in Europe. I get the impression that in the US customers find it easier to arrange payment for general consultancy (time and materials) rather than fixed rate. Therefore the things that are really central to our process may well be a cost in that environment.

This is pure speculation tho', I have no evidence to support it.

Thanks for putting your case so robustly Jeff, it's trying to justify these positions that really make the brain work.

[Newt]

I realize this is an older article, so apologies in advance for raising the dead.

In the debate of auto-populating vs virtual, I don't think there is one; use both. There is a specific use case for which I find auto-populating folders invaluable : lvlibs.

When VIs are in an lvlib they no longer show up in the autopopulating folder. This gives a convenient way of quickly identifying any VIs that may have been either neglected or mistakenly included.

If you have a folder that keeps any trial VIs, tests, etc that shouldn't be in the library you will see them all in the autopop folder.
Any other folders where you intend to keep library members will display nothing in them.

[swatts]

Welcome Newt,

It's a good point you make, I like it!