Code Re-use

[fergusson]

Hi All,

I haven't posted on here before but thought I would give it a stab. Hoping to get some feedback on something that is somewhat of a personal battle at the moment - too much time to think with all this quarantine malarkey,

In essence, I am trying to evaluate the best way to distribute, re-use and maintain libraries in the LabVIEW code that I write. The way I see it there are three options I have and really I want to commit to one so as to avoid a headache, these are:

• VI Packages
• Sub-modules/Externals
• Copy & Paste 

I think I can safely discount copy & paste as this offers no way to maintain the code should an update to the library be required.

As a background, I work as a solo developer, so I have the freedom. However I am always cautious when developing that I will not be the only person ever that needs to look at the source code and both VI Packages and sub-modules have benefits and drawbacks (which I won't get into just yet).

Any thoughts and feedback would be greatly appreciated.

Thanks,

Matt

[James_McN]

Hi Fergusson,

Thanks for posting. I'm not sure there is a good solution to this - I've had a similar dilemma!

I've ended up going for VI packages for my reuse. The main reason is I create a vipc for each project with my reuse code and third party code which means I have a track of versions and licenses per project. However it does require a pro version of VIPM which still seems a bit pricey for this limited benefit.

I have used sub-modules where I want to split an individual project. for example when I have some common processing and a windows and RT application that share them. It has worked quite well although there are lots of warnings against them on the web! I think more complex use cases are harder.

I know there are people in this group who have been successful with all 3 though and it is really a question of preferences and priorities.

[swatts]

Hi Matt,

I take a fairly contrarian position on re-use with LabVIEW, but it comes from experience. At SSDC we tend to be pretty brutal about rejecting practices that we feel lack value and this has included formal library re-use. 

You can read about it here - https://forums.ni.com/t5/Random-Ramblings-on-LabVIEW/Controversial-thoughts-on-reuse/ba-p/3489026

You have to bear in mind that this works for our way of programming and our use cases, but in my career I've never successfully implemented and maintained a general library of re-usable components. They have all taken too much effort and then rotted away in a dusty corner.

We have successfully used templates and automated tools, re-used documentation and processes. At some point we will automate our code reviewing via VI Analyzer, I know what I want to do I just need the time.

Another reason is that I hate, hate, hate dependencies. One of the main things I see hampering LabVIEW programs is the massive libraries of dependencies. To my mind thoughtless use of libraries and toolkits cause more harm to understanding a project than good. (think about the time wasted chasing weird bugs in poorly understood transport mechanisms, when simple UDP will do as an example). I don't count libraries I can embed into my project as part of this problem as long as their source is open.

The executive summary for SSDC is that if we have a re-use library we would embed it in an example project and give it it's own repo. This is easy for us, as part of our standard procedure is to make all our code portable. https://forums.ni.com/t5/Random-Ramblings-on-LabVIEW/LabVIEW-Life-Lessons-1-Project-Portability/ba-p.... This allows us to export a repo into another project and edit accordingly. Examples of this would be Calibration stuff, bug reporting, flexible database screens.

And this is absolutely NOT what you are taught or recommended to do! But we've been doing it for years and it works fine for us. That is the key point, find a method that works for your brain / way of working. You get judged by your output, how you get there is fairly invisible to scrutiny.

Stay Healthy 

[drjdpowell]

I use VIPM packages, but with a high bar; a library has to have a lot of value and reusability to be worth the effort.  And the library needs focus; I no longer maintain a general reuse bag of tricks that I once had as a junior programmer.  Instead I am happy to cut and paste little things.

[joerg.hampel]

Hey Matt, we've experimented a lot over the last years. Disclaimer: We're on the same page as Steve regarding the usefulness of practices in daily life and regarding dependencies in or outside of projects. Taken from our DokuWiki:

---

How do we maintain dependencies in general?

We like having our dependencies copied into the project. As much as possible.

- First reason is to be able to have different versions “installed” for different projects on one machine.

- Second reason (more philosophical): We're of the firm opinion that a central repository is a bad hack from times when disk space was still expensive. There's absolutely no reason to mix or share dependencies in a single location from a project point of view, it only brings the potential of errors introduced without even being aware of it.

The downside of this strategy is that we must refrain from using VI Package Manager as it doesn't support installing into user-defined locations or installing multiple versions of packages into different locations.

How do we use HSE reuse libraries in a project?

We use our own Release Automation Tools for server-side automated execution of various actions. Reuse libraries are built into release packages: .zip archives containing the source distribution build results.

For any project, we take the release packages and extract their contents into the project directory. This way, the built code (eg VIs with version numbers in their context help) is made a part of project itself.

How do we facilitate git submodules?

We only use git submodules where we think that we want to implement modifications to reuse libraries within the project and push back those modifications to the original repo of the library.

And what about VI Package Manager?

We build VI packages for public, generic (i.e. unrelated to our hse structures) reuse libraries, because people are used to that way of working. If we need different versions, we fall back to our own HSE release packages.

---

TL;DR: For us it has boiled down to the question of "Do we want to modify re-use code (i.e. libraries, toolkits, frameworks, modules...) from within a project?" If yes, make it a git submodule. If no, mostly make it a .zip release package or a VI package. 

PS: I also put G Package Manager to the test, but couldn't make our libraries work as both source code (git submodule) and packaged code due to the way GPM dictates the location of installed packages.

[andy_glass]

Hi all 

I think that code re-use is an important element of software development. However there is a limitation to the level you take it to. 

Example 

If you are developing software for a common product with common intefaces and front ends then the code reuse can be massively powerful,. 

If you are a developer working across multiple products and architectures then the reuse will be of a differing level. 

Me personally I try to re-use as much as possible as I am using common communication platforms but the functionality of the test equipment differs. 

I think alot of the re-usae is based on a tiered system which can be defined by the architect at the beginning of a project and can then be identified into a stored library. 

ie user data entry panel, test results output sheets, sequence engine for test sequencing. 

Again it is one of those things which is down to the user, there abilities and the application of the software. 

regards 

Andy G

[fergusson]

Hi All,

Thanks for the replies and the insight into what has and hasn't worked for you in the past. As with anything when it comes to software development there is always more than one good solution.

Considerations:

Before I wanted to discuss where I may consider each use case a benefit I wanted to list the things I consider to be important:

• Dependencies - if I send someone my source code, I want them to be able to open it with ease, opening source code to only have LabVIEW prompt you for a missing dependency rubs even the best detective up the wrong way.
• Modification - modifying a library should be easy and something that can be done on the fly, having to open different projects to make the same modification elsewhere is inefficient
• Versioning - making a change to a re-use library shouldn't break another project that uses it - or pose any sort of risk, having different copies is important
• Efficiency - I'm Lazy

Takeaway:

I think based on the above and reading through your posts and the experimenting I have done I can see myself following:

Submodules - primarily used during development:

They satisfy nearly all of my considerations when it comes to development work. As much as I would like for my libraries to be perfect from the moment they are conceptualized, this is of course not true. They expand in functionality and bugs are often found along the way. Maintaining a repository for each of these code re-use modules means that you can bring them into a project as required maintaining dependencies at a project level instead of a central location.

VIPM - distribution of high value, heavily focused and well designed and tested libraries

This does not mean they start this way.

It seems to me that although slick and provide a nice palette within LabVIEW, that the cons outweigh the pros when it comes to every day development. They introduce dependencies outside of the project, reducing "project portability", on the fly modification becomes difficult and an issue tracking solution may almost be required to remember the changes you need to make and packing and re-distribution of packages requires effort (remember I'm lazy)

I never intended this to start as a VIPM bashing excersize but it seems that my concerns have only been compounded and although it has it's place I cannot see it fitting with the way that I work.

To answer the question: "Do we want to modify re-use code (i.e. libraries, toolkits, frameworks, modules...) from within a project?"

Yes, if it is my code, always.

Other thoughts:

• Admittedly I had not come across a "vipc" before - this seems like a good solution to the issues I am having above with dependencies for an individual project although I still feel like the issue of not being able to have multiple versions an issue. Cost also seems unjustified.
• Being selective of how you split up your code for re-use is important

Thanks again for your thoughts,

Matt

[drjdpowell]

There's different levels of "code reuse", and git submodules seem designed more for "common code supporting multiple subprojects of a single organisation", than the "LabVIEW libraries for use by multiple organizations" that VIPM is meant for.  You sound like you are more in the first category.  Certainly, you never mentioned supporting multiple LabVIEW versions, which seems to me to be a big flaw in git submodules.  Most of my reuse code is in LabVIEW 2013, but most projects are in 2017 or 2018.  

[joerg.hampel]

---

@fergusson wrote:

To answer the question: "Do we want to modify re-use code (i.e. libraries, toolkits, frameworks, modules...) from within a project?"

Yes, if it is my code, always.

---

It's not that obvious in our way of working. I much prefer to have properly released code (eg a source distribution with a random password applied) in a project. That way, I'm forced to apply changes at the source repo and create another version through the "official" release channels.

Only for those libraries or modules or tools that are very hard to develop outside the context of a project will we go with git submodules. Again, I prefer to avoid the pitfalls their flexibility brings.

[James_McN]

I am more and more inclined towards this - or with CI in place even PPLs. It is now pretty easy to test and build in each version of LabVIEW that I have active projects in. Although nested dependencies become a problem then (not that I have many but I do have some architecture level modules that depend on more fundamental modules)