Developer Center Discussions

cancel
Showing results for 
Search instead for 
Did you mean: 

Avoiding 'vip hell' with VIPM packages

I feel that the discussion is trailing off into territory that only the NI LabVIEW core development team can address with changes to LabVIEW itself and is out of VIPM's jurisdiction.

I agree. In fact the solutions I'm trying to focus on are a combination of best practices for package developers and tools for package users, so they aren't really in VIPM's jurisdiction either.  I see it as an issue for the package building community.  That said, I posted in this community for the specific purpose of including JKI and NI in the discussion and any input from either is extremely valuable.  I don't control your resource allocation and don't expect either of you to drop what you're doing and work on this, so I'm aiming for solutions that do not require your participation.

Either JKI or NI could be part of a better integrated solution if the resources are available and you chose to sign up for it.  Maybe the resources aren't available and you contribute comments like, "it will be easier to integrate the solution into our product in the future if you do xyz."  I certainly have not explored all of VIPM's capabilities, so just having you chip in with the things VIPM currently can do has been helpful.

The solution is simple. Package A and B should include package C as internal dependencies. This can be done easily with VIPM Pro, BTW. If you don't want to depend on an external package and want to be isolated from the changes of C, then don't.

1. This solution requires the end user to have control over the package A and B builds.  If I'm using package A from vendor A and package B from vendor B, and neither of them have included package C as an internal dependency, how do I fix that?  I suppose we could make it a best practice for package developers to always make dependent packages internal.  That would solve the problem, except for...

2. As Kosta pointed out, the package C license may not permit redistribution in that way.  What does the package A and B user do then? 

3. I had VIPM Pro at my previous employer and appreciate the additional functionality it provides.  However, at $1,000 per license it's not going to be a viable solution for everyone who encounters this problem.  On top of that, as a consumer I'm not going to be very happy if I'm asked to pay $1k to fix a problem that occurred due to the oversight of others--especially if there are fairly simple ways the problem could have been avoided.

0 Kudos
Message 61 of 71
(1,521 Views)

I'm sorry, but there's something that's reallyt bugging me about this thread: can we please get the title of this thread changed?  I understand the nod to DLL hell, but it doesn't do anyone any good to have the almost overt criticsm toward VIPM.  I suggest it change to something like "Implementing Side-by-Side Component Installations in LabVIEW" or something similar.





Copyright © 2004-2023 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
0 Kudos
Message 62 of 71
(1,521 Views)

I don't see any criticism towards VIPM in the title and I certainly didn't intend there to be any, but if others are interpreting it that way then it probably should be changed.

The point of the discussion is to prevent dependency conflicts by whatever means we can.  SxS installation is just the mechanism we're talking about right now.  How about, "Avoiding dependency hell with packages"?  Is that sufficiently distant from VIPM?

(Is there a way for me to change the title or does an admin have to do it?)

0 Kudos
Message 63 of 71
(1,521 Views)

Daklu wrote:

How about, "Avoiding dependency hell with packages"?  Is that sufficiently distant from VIPM?

No, because we're talking about dependancies in components, not necessarily packages.  As Mike said, this is a core issue - IMHO anything that could be donw in VIPM right now would be a dirty workaround of an issue that's outside its scope.





Copyright © 2004-2023 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
0 Kudos
Message 64 of 71
(1,521 Views)

No, because we're talking about dependancies in components, not necessarily packages.

Fair enough.  How about "Avoiding dependency hell with shared components"?  Packages are a common way to share components and the distribution mechanism I'm primarily concerned about, but they are not the only way.

As Mike said, this is a core issue

I disagree.  The problem is not JKI's or NI's fault, nor is it their responsibility to fix.  (Though I believe it is in both of their interests to see the problem fixed.)  Dependency hell exists (imo) because developers who are distributing reusable code do not expect users to ever require multiple versions at the same time.  That's my impression anyway.  It could be they do expect it but choose to ignore those use cases (and the users who run into them) in exchange for easy upgradability... I don't really know.

There are ways to fix the problem that do not require any participation by either JKI or NI. Simply creating a new package instead of issuing an updated version of the same package solves that problem. Maybe having package builders create a separate interface and use dependency injection or dynamic vi calls to specify which version to use will work. Maybe there are ways to "install" the conflicting versions in separate app instances like Ton mentioned. Maybe it's a 3rd party GAC-like framework other component developers can use to facilitate the process. Maybe it's a combination of solutions.

I fully agree NI is in the best position to provide a comprehensive plan and the only player able to provide an enforcable solution to the general problem.  I'm encouraged by the interest NI employees have expressed in this thread, but at the same time they (understandably) have not signed up to implement anything or (also understandably) talked about how--or even if--they expect to deal with it in the future.  Integrating a solution into Labview's source code is probably a fairly risky and time consuming undertaking, and they are (again unstandably) reluctant to reveal their intentions right now.

IMHO anything that could be donw in VIPM right now would be a dirty workaround of an issue that's outside its scope.

If JKI chooses to wait and see how things play out before making any changes to VIPM, that's fine.  I have no problem with that.  If they want to help shape the solution as it applies to packages that's fine too.  The only feature I've specifically asked for in VIPM to help manage the issue is the ability to arbitrarily group packages together, and Michael indicated he was already looking at that for other reasons. 

0 Kudos
Message 65 of 71
(1,521 Views)

Daklu wrote:

...or it could just enforce that the package developer create a project library before building the VIP.

Nope... don't like.  If putting code in a project library is a best practice, write it up as a best practice... but let the user decide if they want to follow it.  Every time you impose restrictions on a user to "help" them do something right you are removing flexibility another user would find beneficial.

Somewhere between starting to read this thread and my brain turning to mush (about an hour later) I stopped reading and started thinking (somewhat ironically, it was just before I got to LabBEAN's post) and the idea I came up with is basically the same. I have no real experience with .NET or with properly managing conflicting dependencies in LV, so everything came from reading this thread.

Based on the criteria put forth, I also think that a proper solution would have to be implemented in LV itself. It would seem that you agree, since you said earlier "Personally I'd rather wait a couple years for a good native solution than get an insufficient solution sooner".

These criteria are as follows:

  1. We need to be able to install different versions of a package SxS.
  2. Every component needs to remember which version it uses.
  3. It should be relatively easy to replace the version a component depends on.

This basically brought me to the same thing LabBEAN suggested, which seems to be similar to the .NET manifest. The reason I quoted what you said above is that I think that libraries actually might be best suited for this, since they're already namespaced and versioned.

So here's a crude example. For this to work, I want you to ignore VIPM for the moment and just think about LV:

Dependencies.PNG

What we see here is that project VIs call VIs from A 1.0.1 and VIs in B.lvlib call VIs from A 1.1.5.

Here are some thoughts on this:

  1. LV will need to have a mechanism for recognizing that a library has multiple versions and where the code for each version is. That mechanism will be in the library.
  2. The linking to a VI in the library will not be to the actual VI, but to the library and to the fully qualified name of the VI. The linking will include the version number of the library.
  3. As you can see in the image, each component selects one version of the library, but different components can select different versions of the libraries. This allows you to create a library and know that it will keep working with the same version of the library it depends on, even though someone else is working on another version in parallel.
  4. Note that in this example, both the project and the library will need a dialog like the one LabBEAN refered to, where they will need to specify which version of the library they want to use. The best thing about this is that switching to another version of the library is as simple as opening that dialog (or right clicking the dependent library) and selecting "use version X". This is basically the same as the manifest files.
  5. Because the linking is to the library and not the actual file, this will allow LV to immediately change the linking for all callers to the correct VIs, as long as the VI names don't change. This also gives you static linking to the VI, so as far as the end user is concerned nothing changed - you double click a VI in the diagram and LV automatically opens the correct VI. I think that's an important point.
  6. One problem is with what I refered to as "components" which hold the versions of the dependencies. I used projects and libraries here as these components, but this becomes an issue if you open a VI not in a project or if you have a VI which is not a member of a library. One suggestion I can make is adding this dependency data to VIs as well. If a VI is in a library, the library data overrides it.
  7. Likewise, if a VI which is not in a library is opened in a project which has different version information, I can see two options -

    The simpler one is to dictate that all naked VIs in a project must have the same version and to display a conflict resolution dialog when opening the VI.

    The second is not to include the dependncy version data in the project at all. The problem with this is that if you want to modify the dependency for naked VIs, you have to do it for each VI separately.

This was a bit stream of consciousness, so I hope it's clear and that it doesn't repeat too much what has gone before (it has been quite a long thread).


___________________
Try to take over the world!
0 Kudos
Message 66 of 71
(1,521 Views)

Based on the criteria put forth, I also think that a proper solution would have to be implemented in LV itself. It would seem that you agree, since you said earlier "Personally I'd rather wait a couple years for a good native solution than get an insufficient solution sooner".

Yeah, but we may disagree on what constitutes an "insufficient" solution. In the context of what I was saying, an insufficient solution is one integrated into Labview which doesn't allow dependency users a way to use the combination of dependencies they need. Once NI implements *any* solution, making changes will be difficult. My concern is implementing an insufficient native solution now may get in the way of implementing a good native solution later. Point being, it's likely to be quite some time before a good native solution is implemented.

However, the problem exists now.  Why do we need to wait for a "proper" native solution when we can relatively easily create a workable--if less seamless and convenient--solution right now?  When (if?) a native solution is implemented best practices can be altered to fit the new solution.

These criteria are as follows:

  1. We need to be able to install different versions of a package SxS.
  2. Every component needs to remember which version it uses.
  3. It should be relatively easy to replace the version a component depends on.

I think the list is a step in the right direction, but I don't think it quite captures the entire solution space accurately.  I don't have a better list to propose yet.  This feels a little too.... I don't know... restrictive? Specific?  I can't put my finger on it just yet.

--------------

@DavidStaab

From post #38

Designing for SxS installation would fix my problems without doubt.

1.  The typedefs from the dependency trickle up to my componenet's interfaces, and some of those typedefs changed while keeping their names.

I've been thinking more about your problem.  As I understand it, you have a tier 2 conflict but you own and control the source code of the tier 1 component.  In many cases that situation can be resolved using Michael's suggestion of rebuilding your T1 package and making the T2 package from NI a private dependency.  When you install the new T1 package VIPM also installs the T2 package, putting it in a new location with a new namespace, and linking all your T1 vis to the "private" T2 installation.  That's how I think it works anyway... I haven't used private package dependencies. 

StaabDependencyTree.PNG

In your case, it wouldn't work as well because you are using T2 elements in your application code, and the application code wouldn't know the original T2.lvlib has been moved and renamed something else.  You'd still need to go into all your applications that use your component and redirect the links to the new private copy.  (I'd make sure T2 (v2) is not installed when doing that.)  It's not ideal, but it sounds to me like a more manageable solution than either,

(A) remaining dependent on the older package and suffering version conflicts when I put my component into any project that uses the newer one, or (B) spending a lot of nights and weekends to refactor my component and test it with the new dependency.

At least this allows you to solve the problem incrementally, which is the most we can hope for given the current state of the technology and best practices.  Still, if the license doesn't allow redistributing modified code or you didn't own the T1 package source I think you'd be stuck.

Message 67 of 71
(1,521 Views)

Daklu, thanks for detailing the approach that Michael suggested. The diagram is especially helpful.

Kosta guesed correctly: the license for the T2 component is my sticking point right now. Unfortunately, everything published on any part of NI.com, whether by NI or someone else, is subject to NI's T&C which says that they own the code and rights to its use in source distributions. I'm working with the manager of the group that developed my dependency to get a custom license for redistribution.

Thinking more about it, this approach of wrapping dependencies seems to have a chicken-and-egg problem: When I create a new product that depends on another, I either have to preemptively redistribute it in my own package in anticipation of its being revved a some point in the future, or I have to assume that it'll never see a major revision that could break my code or a client application. Since the second option is simply myopic, it seems like the right answer is to always encapsulate dependencies inside my own distributions...that sort of defeats the purpose of being able to dynamically link dependencies and define package configurations in the first place, doesn't it?

0 Kudos
Message 68 of 71
(1,521 Views)

Thinking more about it, this approach of wrapping dependencies seems to have a chicken-and-egg problem: When I create a new product that depends on another, I either have to preemptively redistribute it in my own package in anticipation of its being revved a some point in the future, or I have to assume that it'll never see a major revision that could break my code or a client application. Since the second option is simply myopic, it seems like the right answer is to always encapsulate dependencies inside my own distributions...that sort of defeats the purpose of being able to dynamically link dependencies and define package configurations in the first place, doesn't it?

Previously I've talked about a standard sxs model (every version has a unique namespace) and a custom sxs model (end user does a Save As to customize the namespace.)  I'll call this a private sxs model (each component has a private copy of all dependent components.) 

Yeah, the private sxs model does make package configurations moot in addition to consuming an inordinate amount of disk space and system memory with a lot of duplicate code.  It also requires a bunch of adapters if parts of the T2 dependency ever have to interact with another instance of the T2 dependency.  (Think about T1 components that use the Actor Framework or LDM as a T2 component.  Making them a private dependency and giving each T2 copy a unique namespace reduces their utility.) 

I'm starting to think no one model is the best solution.  They all have pros and cons.  I'm am becoming more convinced that, at the very least, a component's namespace should be changed when backwards compatibility is broken.  And these three sxs models all deal with fairly fixed compile time linking. I haven't fully explored using the palette vis as an interface and invoking specific versions using dependency injection or dynamic vi calls.

Maybe what we need to do is figure out which installation models make the most sense for different user scenarios?  Or to look at it from the other side, which user scenarios are not supported by each installation model?  I've got an incomplete list of user scenarios in my head.  I need to capture them and figure out how to communicate them to everyone else.  I also don't have a good feel for how things like password protection, removing block diagrams, wrapping it in an llb, etc. affect an end user's ability alter the linking.

0 Kudos
Message 69 of 71
(1,521 Views)

Yeah, the private sxs model does make package configurations moot

Not really. At some point, your project will be using multiple packages that represent several add-ons and tools. You could package anything really, including other NI tools and even drivers on IDNET for internal use. Package configurations allow you to have a running snapshot of what those reuse libraries are for your project and allow you to roll-back or forward your LabVIEW development environment. It would only be moot, if you could represent your entire reuse library in one single package, which is a stretch, given the sheer number of existing or future packages.

0 Kudos
Message 70 of 71
(1,521 Views)