Random Ramblings on LabVIEW Design

Community Browser
Labels
cancel
Showing results for 
Search instead for 
Did you mean: 

Re: I'm not being critical but... (Re-use) Part 2

swatts
Active Participant

Hello Hunnies

Over 4 years and a 100 articles ago I wrote "I'm not being critical but... (Re-use) Part 1" and amongst the 20 or so people that care about such things it caused a bit of a stir. I think it might be worth revisiting this and talking about something Greg Payne brought up in CSLUG the other month.

 

Here's a link to his article on the topic

 

If you've not read the aforementioned article I strongly recommend reading it and also all the comments too. But to summarise I expressed the opinion that the current and LabVIEW model for reuse sucked and explained that you can have source-code control where project code is in the repo, but if the dependencies (user lib, instr lib and even unprotected vi lib files) weren't also under source-code control a high-percentage of your code could be modified without control.

 

Various techniques for handling this issue were discussed and at SSDC we settled on making all of our projects atomic (various articles and presentations describe this). This sorted out the control but made re-use a little cumbersome, so I would mark it as about 8/10. Here's some pros and cons

 

Pros

  • Every project is self-contained in the repo.
  • Easy to manage.
  • Portable from machine to machine.
  • No cross-linking

Cons

  • Managing re-use across projects is cumbersome and manual
  • Each project repo is big

The presentation that filled me with enthusiasm at CSLUG was given by Greg Payne and he talked about was Git Submodules (SVN External Items may also work). So for now let's talk just about a repo and external links to other repos.

 

Submodules.png

A project will have links to a version of a toolkit, dependency or library (including a hash number for traceability). The VIs will from all the linked repos will be checked out as you would normally checkout a project, but there will be notifications if a toolkit can be updated. You could also push back requests and updates for a toolkit.

 

A nice user interface for this would be a palette of toolkit repos that you can include into your project and this needn't be much different than the palette view currently available. Additional info would be version data I reckon. Some search, bug reporting and registration interfaces would also be useful. 

 

And that would be perfect.

So we now have the challenge, this is all very possible IMO.

 

Thought food and homework peeps

Lots of Love

Steve

 

 

 

 

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

Comments
FabiolaDelaCueva
Active Participant

Steve,

Way to send us down memory lane! Has it already been 4 years of this discussion? Time flies!

 

The thing I like that VIPM provides that managing subrepositories or externals doesn't provide is the idea of "released code". Sure, you can set a process for your team to tag a new released version of your code in SVN or you could set a rule that you only use code from the released branch of your subrepository in Git or Hg. 

The problem comes when dealing with different dependencies from different teams in the same company.

I don't want to impose how they develop within their own teams, it is easier to say we will all use VIPM for released code or we will all use PPLs for released code. The path you use to get there is all yours!

 

I stayed away from PPLs because they were version dependent and that would mean having to compile them for each version of LabVIEW used by the consumers of the PPL. Starting with 2017 they will no longer be version dependent, so I might start using them more. However, they do have another disadvantage. We are back to PPLs not being easy to move. The teams I work with who decide to use PPLs have to come up with a standard deploying location and stick with it. For example C:\<company name>\PPLs\<year>

I can hear you cringe all the way from the UK to Austin. We are back to having to rely on a central location outside of the project tree! Building PPLs into VIPM packages is not an easy feat either, I believe VIPM 2017 has made some improvements but I have not tried that yet.

 

So, I still like vipc file because they list all the versions of my dependencies and I can click on each package and get a list of released notes. Reading back on the comments from your first post reminded me that I still have on mt to-do list to create a tool that applies the vipc file before opening the project. Also, some developers still have a hard time remembering to apply a vipc file before opening a project or updating the packages in the vipc file when they make changes on their computer. It would be nice for something to complaint when the project is saved if the versions on its associated vipc file do not match the current versions installed.

I am looking forward to what other comments you get on this post. I know I learned a lot from the post and comments on part I.

 

Thanks,

Fab

For an opportunity to learn from experienced developers / entrepeneurs (Steve, Joerg, and Brian amongst them):
Check out DSH Pragmatic Software Development Workshop!

DQMH Lead Architect * DQMH Trusted Advisor * Certified LabVIEW Architect * Certified LabVIEW Embedded Developer * Certified Professional Instructor * LabVIEW Champion * Code Janitor

Have you been nice to future you?
swatts
Active Participant

I flung the bait out in the hope of hooking some feedback, I'd like the discussion to move on to what we would like to see in the IDE for this type of functionality. I'm quite enthused about what Greg proposed.

I'm being a bit hard-nosed about the current ways we manage these issues, because I see them as work-arounds to a problem and I don't like constraints. Hopefully we can thrash out some concepts at CSLUG next month too.

 

Meantime look up Git Submodules. I need to work on scenarios but this might be the key. It would be nice then to convert the git interface to something more LabVIEW-esque or even VIPM-esque.

Suffering from too many idea and too little time. 

 

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

FabiolaDelaCueva
Active Participant

Steve,

 

I like Git Submodules, I have used them both for Hg and Git. The issue I have with them is that I would have to impose onto other teams to use the same development process our team uses. The only manifest file would be the list of submodules (which is really all the top level repository is), but there is not an easy view, at least not that I know of, to see which version I am using for all the submodules. I have to go to a tool like Atlassian SourceTree or TortoiseHg to look at the submodule and then go to its revision history to see the release notes of the latest version.

 

 

Also, we would have to convince the rest of the LabVIEW community to use this approach too. This way you could, for example, have DQMH be one of your sub repositories and you would get updates when we release a new version. 

 

I think sub repositories are great for source code control that you have complete control over. We still need to figure out how to deal with dependencies that come from third parties. 

 

I think the NI Package Manager is going on the right direction but it needs to include all the features VIPM has (like creating palettes) and it needs a way to ensure that when a project is opened LabVIEW takes care of installing all the dependencies. Also, if the developer decides to change the version of one of the dependencies on its computer, LabVIEW should inform the developer that this changes the manifest for this project and prompt the developer to update the dependencies manifest. This way when the developer commits/pushes their changes to their repository, the rest of the team when they pull will be advised by LabVIEW that they need to install the new version. 

 

The cherry on top would be for NI Package Manager to allow for installing the packages either in a central location like we do now (vi.lib, instr.lib, c:\PPLs\, etc) or offer the option to install the packages in a directory that is relative to our repository.

 

Now, my first reaction was "I would not put this directory relative to our repository in a sub-repository because if I do so, now I no longer can use any updates to the packages done via Package Manager". Now that I think about it, I could install in the sub repository, this way when I install an update of the package to the same repository, Source Code Control will clearly show what changed from one version to the next and I have control  of what I put on the commit notes "I updated from version 3.1 to 4.0 because it had the ability to implement TDD". The other developers could still go to the package release notes to learn more about what the original developer reported AND go to their own revision history to see why a developer chose to update to that new version.

 

Please let us know what the CSLUG brains come up with next time you guys meet.

 

Thanks,

Fab 

For an opportunity to learn from experienced developers / entrepeneurs (Steve, Joerg, and Brian amongst them):
Check out DSH Pragmatic Software Development Workshop!

DQMH Lead Architect * DQMH Trusted Advisor * Certified LabVIEW Architect * Certified LabVIEW Embedded Developer * Certified Professional Instructor * LabVIEW Champion * Code Janitor

Have you been nice to future you?
swatts
Active Participant

Booom!

Fab - "Also, we would have to convince the rest of the LabVIEW community to use this approach too. This way you could, for example, have DQMH be one of your sub repositories and you would get updates when we release a new version. "

 

That's what I want!

I want to park the toolkit into my project

I want to be told if there's updates

I want updates to be optional

I want complete coverage of all the code/dependencies in the project.

When I build a new project I want to be able to bring the latest version of the tool-kits

I want to know what changes have been made

I want traceability, so when I checkout my project it brings the correct versions into my project.

I want to be able to push back feedback and even update tool-kits.

I want to do this from the LabVIEW IDE.

I want it to be industry standard and not dependent on 3rd party support.

 

I want I want I want - I'm like a toddler

 

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

justin.goeres
Member

Steve,

 

 

I'll be critical :).

 

I think LabVIEW gets reuse and dependency management badly wrong. Or at least it imposes conditions on it (mostly via vi.lib and how the palettes work) that make doing it right much harder.

 

Lately I've been working in Java, which necessitates using Maven. Although I kind of knew about Maven before, I didn't really grasp it until now. This post on /r/labview gives a pretty reasonable description of how it works.

 

I'm struggling to find a concise way to say it, but basically it seems to me like Maven/Java (and I assume others) operate on a model that assumes lots of external dependencies on a per project basis. LabVIEW on the other hand assumes that dependencies are installed internally (i.e. under <labview>, in vi.lib/user.lib/etc.).

 

In essence, LabVIEW makes the dependencies more about "What does LabVIEW know about?" whereas the more important question should be "What does this project need to know about?"

 

I should note, also, that there are some pretty wonderful IDEs out there. IntelliJ, for instance, has an incredibly powerful autocomplete engine that knows what libraries you have installed, their APIs, and even the context you're typing in. So if you're in a place where you probably need a function call, it shows functions. If you need an annotation, it shows annotations, etc. In a lot of ways it's better than QuickDrop. And it works for all the external reuse libraries, as well as the core Java API (which itself is not really different from external libraries as far as the IDE/compiler is concerned).

 

It's just a better designed approach, and implemented much more elegantly.

 

swatts
Active Participant

Hey Justin, always a pleasure when you drop by!

 

"I'm struggling to find a concise way to say it, but basically it seems to me like Maven/Java (and I assume others) operate on a model that assumes lots of external dependencies on a per project basis. LabVIEW on the other hand assumes that dependencies are installed internally (i.e. under <labview>, in vi.lib/user.lib/etc.).

 

In essence, LabVIEW makes the dependencies more about "What does LabVIEW know about?" whereas the more important question should be "What does this project need to know about?"

 

I like the way you put this, it gives a different and clearer view of the point I'm trying to put across.

 

The other aspect about this that I have come to think is important is that dependency handling is a foundational aspect of an IDE. Re-use will be significantly more robust on a strong and clear foundation.

 

On a side note I think there should be some sort of forum rule about comments get more Kudos than the article!

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

James_McN
Active Participant

Well put Justin. That is exactly the issue in LabVIEW. My biggest disappointment with NIPM was I thought it would start to move us towards the model of project dependency installation, not LabVIEW dependency installation.

 

You can see this is true by the "smell" that everyone wants to use VMs for LabVIEW - because it gets around these dependency issues.

 

To Fabs point about tooling though she is right - for something to be successful we need publishers as well as users to embrace it. Inside an organisation that isn't such a problem but as a wider community it is. The ideal would be a tool that can wrap a multi-purpose or existing system.

 

I think for git submodules to work we would have to define some conventions for the repo. I believe Node Package Manager does this already by insisting on a json file that points to the relevant files and tags for versioning (and it's own dependencies). Then you can point directly at a git repo

James Mc
========
CLA and cRIO Fanatic
My writings on LabVIEW Development are at devs.wiresmithtech.com
swatts
Active Participant

Hey James (and Fab)

I've been thinking about the "for something to be successful everyone needs to buy-in argument" and I'm going to go out on a limb....I don't care about other peoples buy-in.

 

Kevin Costner said "if you build it they will come" and I think this is applicable here, if the interface is intuitive and the tool is useful, it will be used.

 

I would find immediate use for the module proposed above, but not if it involves tinkering around with git clients etc etc. From the project would be OK, from the palette would be lovely. I'm going to propose we try and rough out a UI for it within the IDE, part of this is a list of approved repositories. I'm going to set this as a topic for CSLUG in May.

 

It will be a diverting exercise if nothing else.

 

 

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

Taggart
Active Participant

"You can see this is true by the "smell" that everyone wants to use VMs for LabVIEW - because it gets around these dependency issues."

 

I agree, but fixing the dependency issues doesn't get rid of the need for VMs.  You

still, have to contend with the fact that different versions of LabVIEW don't play well together.

 

As to getting buy-in:  I think if it solves the problem and is intuitive people will quickly come around.  No one really likes the current solution, but everyone does it because there is currently no viable alternative.

Sam Taggart
CLA, CPI, CTD, LabVIEW Champion
DQMH Trusted Advisor
Read about my thoughts on Software Development at sasworkshops.com/blog
GCentral
James_McN
Active Participant

@Taggart: That is true, at least it would be one layer less of VMs though!

 

@SWatts: I think in the short term you are right. You can't expect to get everyone on board immediately and a good tool will probably be adopted. In the long run (depending on the overhead) I would prefer one good tool than two seperate ones which leads to two places to check everything etc.


@swatts

 

James Mc
========
CLA and cRIO Fanatic
My writings on LabVIEW Development are at devs.wiresmithtech.com
Taggart
Active Participant

@James_McN:  Having multiples tools is definitely a problem, especially if you are often inheriting other people's code.  You never know what you are going to get (cue Forrest Gump quote about a box of chocolates).

Sam Taggart
CLA, CPI, CTD, LabVIEW Champion
DQMH Trusted Advisor
Read about my thoughts on Software Development at sasworkshops.com/blog
GCentral
richardscheff
Member

I am used to Visual Studio that has project files with dependency information including package versions that give notices of updates.

It is up to me to decide if I want to use the update or not. 

It seems like people are asking for that type of system with LabVIEW.

My biggest pet peeve with LabVIEW is that even in projects I never know where it is going to go for a dependency.  So I have gotten tot the point of renaming folders on and off just to safe guard against version conflicts. 

My system works for me, but not for shared code. 

Code reuse is a great idea.  But I jump through hoops to make sure I work on duplicate copies and not originals.  And insert those into projects.  It is not elegant and it is nothing like toolboxes.  But it protects against developers unintentionally changing shared code ( which I have seen).  Maybe I am not using LabVIEW tools correctly.

 

 

swatts
Active Participant

Hi Richard,

I think you're describing pretty much what we all do in some fashion or other. It's very similar to what SSDC does and for pretty much the same reasons.

The first part of the article back in 2014 started off in my head as a why do we not use LabVIEW as we're supposed to. VIPM is an extremely elegant way to automate our way round the problem, but it's not actually a cure. What I want is the IDE to give us this functionality.

In CurrentGen we can modify the IDE and add functionality, I now think it's possible to have what I want. The next question is whether it's worth the effort!

So the second point of writing this is to get us thinking about an interface we would like to see, we can then put in a well thought out request for NXG.

Thanks for your input.

Steve

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

Sumedha
NI Employee (retired)

This has been an interesting thread to read. @swatts - Thanks for starting this!  (Thanks for sharing this, Fabiola!)

 

Reading through this, my mind immediately started categorizing the dependencies. Does this categorization sound sane to you all? Did I miss anything?

 

1. Extensions to the IDE: E.g., SCC plugins for LabVIEW or Resharper for Visual Studio. These seem like something that as a developer I would want to have one version for across the editor, right?

 

2. Code dependencies that are needed to compile/ run my code: This one seems more complicated. There are some dependencies that are needed only for development vs some that are needed for deployment as well. E.g. test frameworks are only needed in development but not for final shipping code.

  • This category is the one that as a developer I would want to control on a per project basis to ensure I am using the right versions. @James_McN, @justin.goeres - you mentioned wanting some notion of per project dependency management instead of IDE level. This reminded me of how C# projects can be configured to use specific nugets. (I have heard Javascript does that too but I haven't used that before). I think this desire is more applicable for this category rather than IDE extensions, right? 
  • The other question that comes to mind is - what about drivers? Would you expect to manage those per project? 

@swatts - If you don't mind sharing, I would love to see what you guys come up with at the CSLUG! 

 

Thanks,

Sumedha

Sumedha Ganjoo | LabVIEW R&D
swatts
Active Participant

Hi Sumedha,

Thanks for taking the time and welcome to my blog.

 

"1. Extensions to the IDE: E.g., SCC plugins for LabVIEW or Resharper for Visual Studio. These seem like something that as a developer I would want to have one version for across the editor, right?"

 

For my part I'm not that interested in the mechanism (svn,git,mercurial), I'm more interested in the functionality. So if it's incorporated in the IDE and uses/builds upon interfaces I'm familiar with then I'm a happy boy. 

 

Must Haves that I can think of at the mo'

Notification - I want emails and UI notification of changes and logs of those changes

Tracebility is key here - want to be able to import updates into my project

Feedback - if I want the dev to make changes I want to push back to them

Management of changes to checked out sub-repos - what happens if I make local changes to a sub-repo'd toolkit - notification/warning

 

"2. Code dependencies that are needed to compile/ run my code: This one seems more complicated. There are some dependencies that are needed only for development vs some that are needed for deployment as well. E.g. test frameworks are only needed in development but not for final shipping code.

  • This category is the one that as a developer I would want to control on a per project basis to ensure I am using the right versions. @James_McN, @justin.goeres - you mentioned wanting some notion of per project dependency management instead of IDE level. This reminded me of how C# projects can be configured to use specific nugets. (I have heard Javascript does that too but I haven't used that before). I think this desire is more applicable for this category rather than IDE extensions, right? 
  • The other question that comes to mind is - what about drivers? Would you expect to manage those per project? "

From a code perspective, because all our projects are atomic and self contained the code dependency part is handled. The sub-repo will install the code locally to the project and this could be saved to the project repo (or the hash code of the sub-repo to save space/reduce replication), this replaces the worry about dependencies with one about disk space (essentially free).

 

Drivers - our current method is to keep external drivers and INSTR lib stuff in our project repo (manually move them!). With NI Drivers as they are currently designed this is not feasible. It is desirable tho'. I want to install my repo on a vanilla machine with as little dependency installation as possible. Every dependency is a little bit of control taken away from the developer. Maybe betters links, package tracebility in the form of a dependency management list (with some sort of hash code) would be the way to go. 

 

This is definitely per project management - If it's per project it cuts out the need for per IDE I would imagine.

 

I'll probably be filming, but not streaming CSLUG this time (as James is doing NIWeek practice) - I anticipate it will be a freeform ideas dump on this subject. If technology allows I will definitely stick the vid up on youtube.

 

WARNING : these are my ideas and not fully formed, the other contributors to the blog have ideas that are usually better formed and presented. My hope is through the conversations we can thrash out something useful.

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

James_McN
Active Participant

Hi Sumedha,

 

I think your categories are exactly right. Add-ons to the IDE I would expect one install (though per-project configuration might be needed depending on the add-on).

 

The libraries I would say are exactly as you have described - the same mechanism as nuget in C# (I've only slightly used) or npm in Javascript.

 

Drivers are a unique challenge for us over other environments (well Javascript has no need for them anyway). The ideal case is that these would be managed per project but of course there are technical limitations that only one version can be installed. I think at least I would want the project to track the dependent drivers so either a) a new system can install them or b) it can warn you if they have been changed on the existing system.

 

(As @swatts suggests actually what you mean by a driver varies! instrument drivers are really just code libraries so they can be treated the same. Drivers that require an external dependencies installed to the OS are the problem).

 

Thanks,

James

James Mc
========
CLA and cRIO Fanatic
My writings on LabVIEW Development are at devs.wiresmithtech.com
joerg.hampel
Active Participant
npm in Javascript

I didn't do a lot of work in Javascript, but especially for my most recent project - a webinterface and update mechanism for an embedded device (Raspberry Pi) - it worked like a charm. Made setting up the device itself and updating the application that runs on it very easy and straight-forward.

 

at least I would want the project to track the dependent drivers

Same here. I could even live with virtual machines for those projects that actually depend on "external drivers" (we don't have a lot of those), if the IDE only had a way to inform about differences or changes.




DSH Pragmatic Software Development Workshops (Fab, Steve, Brian and me)
Release Automation Tools for LabVIEW (CI/CD integration with LabVIEW)
HSE Discord Server (Discuss our free and commercial tools and services)
DQMH® (The Future of Team-Based LabVIEW Development)


swatts
Active Participant

Re-reading this I didn't answer the question very well!

 

"1. Extensions to the IDE: E.g., SCC plugins for LabVIEW or Resharper for Visual Studio. These seem like something that as a developer I would want to have one version for across the editor, right?"

 

Right! - agree with James here

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

Sumedha
NI Employee (retired)

Thanks all for the clarifications and details. @swatts, hoping to get to see the video:)

Sumedha Ganjoo | LabVIEW R&D
swatts
Active Participant

Hi Sumedha,

You can find the discussion here

https://forums.ni.com/t5/CSLUG-Central-South-LabVIEW-User/Meeting-15th-May-2018/gpm-p/3774238

While I highly recommend the other presentations, you'll find our discussion on this subject at the 2 hour mark. I found it very interesting.

Steve

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

Sumedha
NI Employee (retired)

Great. Thank you!

Sumedha Ganjoo | LabVIEW R&D
craigc
Active Participant

Good stuff Steve,

 

For some time now I have been thinking of going about the same problem but from within the project explorer itself under the dependencies item.  Somthing along the lines of adding repository information as part of the dependencies.  The dependencies would then filter project code into the user defined dependency folder.  This is sort of mirroring what the vi.lib folder already does but with the addition of allowing us to manage dependencies a bit better.  Perhaps someone has already solved this approach and perhaps it could even be done under the project provider framework...

 

A little like this way maybe,,

Dependencies_1.pngDependencies_2.pngDependencies_3.png

 

Craig

LabVIEW 2012
swatts
Active Participant

Hey Craig how are you?

Take a look at part 3, we're very much on the same trajectory I think!

Steve


Opportunity to learn from experienced developers / entrepeneurs (Fab,Joerg and Brian amongst them):
DSH Pragmatic Software Development Workshop


Random Ramblings Index
My Profile

craigc
Active Participant

Good thanks Steve,

 

I did read Part 3 but managed to reply to part 2... Tab overload!

Dependency management really needs a bit of an overhaul.  Lots of room for wiz bang solutions.

 

Might be able to catch up at the next CSLUG, hope all is well

LabVIEW 2012
Gregory
Trusted Enthusiast

Recently MGI released the G Package Manager (gpackage.io) which handles project dependencies, and installing the right version of a toolkit for the project. I haven't played around with it yet, but it seems like it would address some of the concerns here.