Developer Center Discussions

cancel
Showing results for 
Search instead for 
Did you mean: 

Avoiding 'vip hell' with VIPM packages

Can you propose a solution? Can you define what SxS install means in your mind, specifically?

0 Kudos
Message 21 of 71
(2,009 Views)

MichaelAivaliotis wrote:

Can you propose a solution? Can you define what SxS install means in your mind, specifically?

Fair question.  SxS installation in my mind is a strategy that allows users to have incompatible versions of the "same" resource to be installed and used simultaneously, instead of only allowing one version to be installed.  MS's SxS implementation takes place at the OS level--none of the applications are aware it is even occurring.  I don't consider implementation at the OS level an inherent part of sxs installations.  (The thread title refers to dll hell, but that is just a specific case of dependency hell.)

For clarity, let me throw out a couple definitions here.  I don't know if they match up with what you use internally at JKI, but I'll use them for the purposes of this discussion:

package instance - a package instance is a single, versioned .vip file on disk.  MyPkg-1.0.0.1.vip is a different instance than MyPkg-1.0.0.2.vip.  Two files named MyPkg-1.0.0.1.vip are the same instance.

package - a package is the collection of all packages instances that VIPM considers the "same."  MyPkg-1.0.0.1.vip is part of the same package as MyPkg-1.0.0.2.vip.  YourPkg-1.0.0.1.vip is not part of the same package.

package line - a package line is a group of separate, incompatible packages that are intended to be an upgrade path.  I'd consider LDM.v1-1.0.0.1.vip and LDM.v2-2.0.0.2.vip part of the same line, even though they are not the same package.

package family - a package family is a group of separate packages that are related to each other.  The OpenG packages might all be part of the same family, as would all the LapDog packages.  (Messaging, Collections, DataStructures, etc.)

Currently VIPM doesn't support package lines or families, and only allows a single package instance from a package to be installed at any given time.  That's fine, as long as we take steps to avoid dependency hell.  The simplest solution I've found is what I described earlier--to create new packages with small variations in the package name, namespaces, and install locations.  I do that by leaving the descriptive part of the name the same and appending a version number to the package name, namespace, and install point.  (LapDog.Messaging.v1, LapDog.Messaging.v2, etc.)  Users can see the packages are part of the same line by the similar names, but they retain the flexibility to install any combination of LapDog.Messaging packages they need by virtue of them being completely independent from each other. 

Perhaps a more elegant solution would be for VIPM to group families and lines together.  A collapsable tree is the first thing that comes to mind, but I'm sure there are other good solutions as well.  When building a package I would tag it with the family/line it belongs to, and VIPM would display the package under that branch.  Multiple packages with the same tag show up under the branch.  Obviously this solution requires a lot more work on your part and I have no idea if families/lines fit in with VIPM's long term plans, so I've been looking for ways to prevent dependency hell within the existing framework.

[We might not need separate constructs for a package line and a package family.  It's possible a general family relationship would be enough to cover the cases when a line relationship would be useful.  I'd have to think about that for a while...]

The path I've taken isn't the only viable solution.  As I understand it, some linux distributions avoid dependency hell by maintaining pretty tight control over the distribution's packages--even to the point of the distributor taking over maintenance of the packages.  (I don't have much direct experience with linux though, so someone please correct me if I'm wrong.)  While NI is taking steps to assure the LVTN packages work correctly, I doubt they want to take on that level of responsibility.  Also, there are lots of packages outside the LVTN available to end users which NI has limited ability to influence. 

Java and MacOS have both encountered their own versions of dependency hell, though I'm not familiar with the solutions they came up with.  If anyone knows how their solutions differ from the sxs/version number solution I'm proposing I'd love to hear it.

0 Kudos
Message 22 of 71
(2,009 Views)

package line - a package line is a group of separate, incompatible packages that are intended to be an upgrade path.  I'd consider LDM.v1-1.0.0.1.vip and LDM.v2-2.0.0.2.vip part of the same line, even though they are not the same package.

Why not LDM-1.0.0.1.vip and LDM-2.0.0.1? I'm not following the need for V1 ans V2. The version number is already there. Is this so you can install both at the same time? At the VI level, what's happening? Are the VIs namespaced differently? Are they installed into different locations?

Currently VIPM doesn't support package lines or families, and only allows a single package instance from a package to be installed at any given time.  That's fine, as long as we take steps to avoid dependency hell.  The simplest solution I've found is what I described earlier--to create new packages with small variations in the package name, namespaces, and install locations.  I do that by leaving the descriptive part of the name the same and appending a version number to the package name, namespace, and install point.

If not for namespacing, then how would VIPM install 2 different VIs with the same name to the same location?

Perhaps a more elegant solution would be for VIPM to group families and lines together.  A collapsable tree is the first thing that comes to mind, but I'm sure there are other good solutions as well.  When building a package I would tag it with the family/line it belongs to, and VIPM would display the package under that branch.  Multiple packages with the same tag show up under the branch.

Sure, VIPM could do that. We've been thinking about the concept of products. Where a product would contain several packages. That would be similer to your family idea. But I still don't follow how this would solve things.

I'm getting the feeling that you want some form of package merging. Where basically two packages can install over each other? Hmm, not really because then how would you access the older versions. I'm confused.

0 Kudos
Message 23 of 71
(2,009 Views)

I'm enjoying the conversation, but for the purposes of more efficient communication, maybe you guys should have a conference call together, and then come back to us with a summary?  I feel like you're like ships passing in the night...





Copyright © 2004-2023 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
0 Kudos
Message 24 of 71
(2,009 Views)

Christopher Relf wrote:

I'm enjoying the conversation, but for the purposes of more efficient communication, maybe you guys should have a conference call together, and then come back to us with a summary?  I feel like you're like ships passing in the night...

Straaaangers in the niiight...

0 Kudos
Message 25 of 71
(2,009 Views)

MichaelAivaliotis wrote:

Christopher Relf wrote:

I'm enjoying the conversation, but for the purposes of more efficient communication, maybe you guys should have a conference call together, and then come back to us with a summary?  I feel like you're like ships passing in the night...

Straaaangers in the niiight...

...exchanging glances!





Copyright © 2004-2023 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
0 Kudos
Message 26 of 71
(2,009 Views)

I apologize if I haven't been clear.  Let me back up a bit and try laying it out point by point...

1.  As an api developer, I understand the ideal of never breaking backwards compatibility is not realistic.  At some point the software I release will not be fully compatible with prior versions.

2.  I recognize it is impossible to eliminate the possibility a user may need to have multiple incompatible versions of my software installed at the same time. 

Given 1 and 2, how do we allow users to have incompatible versions of my software installed on their computer?

3.  Each incompatible version needs to be installed in a unique directory with a unique namespace.

4.  Upgrading or downgrading a package removes the code installed by the prior package version (as it should,) so each incompatible version of my software needs a unique package as well.

Given 3 and 4, I decided to use a convention where the major version number is only rolled when backwards compatibility has been broken.  The major version number is appended to the install directory, the namespaces, and the package file name for the explicit purpose of allowing users to install multiple incompatible versions.  (LapDog.Messaging.v1, LapDog.Messaging.v2, etc.)

Why not LDM-1.0.0.1.vip and LDM-2.0.0.1? I'm not following the need for V1 ans V2. The version number is already there. Is this so you can install both at the same time?

Yes, exactly.  From a functional point of view it doesn't matter how I change the install directory, namespaces, and package file name. It only matters that they do change when an incompatible version is released. I could have chosen to append greek letters, a datestamp of the initial release, or something like LDM.current, LDM.now_this_is_current, LDM.this_is_the_real_current_package. I could have completely changed the name with each incompatible release and removed version info: LapDog.Messaging, LapFrog.Messaging, LapLog.Messaging. Functionally all these options are identical.

I chose to append the major version number because it's clear and concise.  Users are able to see LDM.v1 and LDM.v2 are related by virtue of the similar names.  Hopefully they will understand v2 is an upgrade to v1.  As you pointed out, it can cause some confusion because the major version number exists in both the name and the version number field.  (I plan to mitigate the confusion by making sure the numbers match.)

At the VI level, what's happening?  Are the VIs namespaced differently? Are they installed into different locations?

Each major release (incompatible version) is a completely separate entity--different palette, different install directory, different namespace. 

If not for namespacing, then how would VIPM install 2 different VIs with the same name to the same location?

It can't.  To clarify, I'm not criticizing VIPM or the way it works.  I think it is doing everything correctly right now.  My concern at the moment is more about making people aware of the impending problem and coming up with a collective plan for dealing with it.  The plan may or may not include changing VIPM.  It probably will include some standards on how people use VIPM.  It almost certainly will entail rethinking what it means to be a "package."

Currently the prevailing thought seems to favor the idea that all revisions to an api should be distributed as version updates to the same package.  There's kind of an implied understanding that the package and the package line are the same thing.  However, this thinking prevents users from having multiple incompatible versions simultaneously installed (sxs installation.)  That is what causes dependency hell.

Right now the only way to allow sxs installation is for package developers to create new packages, new namespaces, and new install locations when compatibility is broken.  That requires breaking the mental link between packages and package lines.  Using sxs installation, the package is simply a deployment mechansim for a single major version.  It does not define the entire package line.

(BTW, the sxs scheme I am proposing does not allow users to install multiple versions of the same package.  That would require creating something similar to the GAC and inserting an abstraction layer between Labview and package code.  This scheme requires much more thought on the part of the package developer.)

I'm getting the feeling that you want some form of package merging. Where basically two packages can install over each other? Hmm, not really because then how would you access the older versions. I'm confused.

No, not package merging.  Packages as the basic unit of deployment are good.  At some point in the future VIPM may merge how packages are presented to the user, but I don't see any reason to merge the packages themselves or change the mechanics of how they work.

For example, even though LDM.v1 and LDM.v2 are separate packages, because they are the part of the same package line it may make sense to list them on a single line and use a right click menu to select which packages to install similar to how the current "Install Other Version..." selection works but allowing multiple selections.  (Not suggesting it, just presenting it as an example.)

We've been thinking about the concept of products. Where a product would contain several packages. That would be similer to your family idea. But I still don't follow how this would solve things.

It doesn't help the immediate problem.  The purpose of families and lines is to clean up the package list and help communicate intent.  How do users know Package B is intended to replace Package A?  I use naming and version numbers right now, but that might not be the best long term solution.  The underlying package mechanics don't change at all.  Families, lines, and/or products are abstractions presented to the user.

0 Kudos
Message 27 of 71
(2,009 Views)

This sounds a lot like the python 'Virtual env' package, that let's you create a virtual environment where a  seperate set (non-global) of python packages and scripts is loaded.

It's used by applications that need a specific version of a package.

By using virtual environments you can install v 1.x of a package for application A and 1.y for application B.
These applications are then loaded into there own application space.
The best I can describe thins in LabVIEW terms are Application Instances, but linking this to VIPM is tricky at best.

Another word might be 'Virtual user.lib'

Ton

Free Code Capture Tool! Version 2.1.3 with comments, web-upload, back-save and snippets!
Nederlandse LabVIEW user groep www.lvug.nl
My LabVIEW Ideas

LabVIEW, programming like it should be!
0 Kudos
Message 28 of 71
(2,009 Views)

I'm not familiar with python or the virtual env package.  Based on your description, it sounds like that is a different solution to the problem of dependency hell.  What I'm proposing right now is not nearly that complicated.  I'm just saying, "let's create a set of namespace, install point, and package name conventions that allows users to simultaneously install multiple versions of the same package line."

VIPM use and the LVTN are growing.  I believe we--the advanced developer community--need to have a short term plan for preventing dependency hell or we run the risk of creating problems for less experienced LV users.  Once that is established we can look at longer term solutions (virtual user.lib, VIP GAC, whatever...) that provide a systemic approach to mitigating dependency hell rather than relying on each package developer to do the right thing.

Message 29 of 71
(2,009 Views)

I wanted to clarify something about the "official" stance of SxS on the LVTN.  I didn't mean to imply that any developer should or should not use SxS.  I can fully understand (and my assumptions are confirmed after this discussion thread) that both methods are very important in different situations.  The important thing in terms of the LabVIEW Tools Network is that whichever route you do choose it is easy to understand for the average user. 

If you choose NOT to use side-by-side, changes should be handled gracefully, wrapped appropriately, mutated etc.  If you DO choose to use side-by-side, just ensure that upgrade procedures are well documented and users have an easy understanding of which version to install under what situation. 

I still think this thread is important in deciding a best-practice going forward, but know that there is no definitive best answer in the perspective of the LabVIEW Tools Network and Compatible With LabVIEW program.

0 Kudos
Message 30 of 71
(2,009 Views)