LabVIEW Tools Network Developer Center Discussions

Showing results for 
Search instead for 
Did you mean: 
Reply

Avoiding 'vip hell' with VIPM packages

Re: Avoiding 'vip hell' with VIPM packages

I know, I know.

0 Kudos
Message 41 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

"Save as" approach works when you have the rights to save another package under a different name and re-distribute the IP from the other package indirectly.  In other words, "Save as" approach may be limited by licensing.

Kosta
0 Kudos
Message 42 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

Daklu wrote:


As a practical matter, I'm concerned the lack of existing LV infrastructure support will result in a very negative experience for the end user if we pursue that option too soon. As you pointed out, switching versions takes a lot of effort, and I'm guessing most users would be fine with automatically upgrading all their Foo 1.0 code to Foo 1.1.

SxS is relatively novel in our context, but it seems to offer promise to solve problems associated with automatic upgrades.  The level of effort switching takes depends on the circumstances, including the manner and extent to which the APIs are used, and availability of any scripting tools to help with the explicit upgrade (and possibly downgrade).  Your guess that most users would be fine with automatic upgrading is supported by prior art. I am interested in how SxS would best address the rest of the users, and whether it can be made to work well for all the users.

Daklu wrote:


They will not be happy if they have to search and replace to receive every simple bug fix. I've thought about creating scripts to automatically search and replace user code when the APIs are compatible, but that is still a cumbersome solution. Someone upgrading from Foo 1.0.3 to Foo 1.0.19 will have to install all 16 intermediate versions and run their upgrade scripts in order.

(Edit: Hmm... maybe it would be possible to write mutation scripts recursively so the search and replace parameters are mutated through all 16 versions but the actual search and replace operation is only done once. That would eliminate having to install all the intermediate packages as long as the mutation code contained the entire history.)

If the interfaces are consistent in all Foo's between 1.0.0 and 1.0.19, I expect that you could write a single script for all upgrade and downgrade scenarios that does not require presence of any intermediate versions.

I am curious about number 19 from the example.  How often do you envision producing bug fix versions?

Daklu wrote:

The trick is making the upgrade (and downgrade) code replacement automatic for most people, but optional in those cases where a specific user application shouldn't automatically upgrade to the new package version. [...] (And as Jim alluded to early in the thread, it's very easy to change relink an application to a different version of the dependent assembly.)

How would we define "user"?  I am asking because some users may enjoy knowing about dependencies, but others will rely on a high-level library specifically so that they don't have to deal with lower-level details.  The main benefit for all users from the high-level library existing with tightly specified versions of low-level libraries would be quality.  I am assuming here that the high-level library gets tested thoroughly along the low-level libraries it depends on.

I have worked in an environment where it was easy to relink to a different version of a dependent library (with all assumptions listed in Daklu's earlier post), but it was not automatic, meaning that the programmer had to opt in for changes to take place.  I appreciated control I had over timing of opting in.

Daklu wrote:

There's nothing (that I'm aware of) even remotely similar to the GAC currently in Labview, and I expect implementing similar functionality in LV dev environment would be quite time consuming. On the positive side, this is primarily a dev environment issue rather than a run time issue, so maybe that would simplify things somewhat.

  [...]

All that being said, you mentioned you see changing package versions "as a separate problem." Do you have something else in mind for simplifying the process of changing which package version a project uses?

I don't know enough about GAC to comment on whether I'd like to see it emulated in LabVIEW.  The time for writing scripting tools for changing called versions should diminish with experience.

I agree that this is mostly a dev environment issue, but that it may also be a run time issue.  When I was describing "a separate problem", I was referring indeed for something easier on the user than brute force search-and-replace.

Kosta
0 Kudos
Message 43 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

In other words, "Save as" approach may be limited by licensing.

Excellent point.  I hadn't considered that.

I am curious about number 19 from the example.  How often do you envision producing bug fix versions?

It was just an arbitrarily large number I chose to demonstrate the amount of work it would take to switch between versions.  It's not outside the realm of possibility if a large package was very stable over a period of several years, but it's not something I would expect to occur very often.  Off the top of my head I think Foo 1.19.0 (api additions that do not break backwards compatibility) is more likely than Foo 1.0.19.

The trick is making the upgrade (and downgrade) code replacement automatic for most people, but optional in those cases where a specific user application shouldn't automatically upgrade to the new package version.

How would we define "user"? 

The "user" I am referring to above is simply the person who is using a package (produced by themselves or an arbitrary 3rd party) to develop their own software.  The "user application" is the code base they are writing that depends on the package.  That code could be intended to be deployed as an executable, a new package, a packed project library, or just a set of vis executed from the dev environment.

I am asking because some users may enjoy knowing about dependencies, but others will rely on a high-level library specifically so that they don't have to deal with lower-level details.  The main benefit for all users from the high-level library existing with tightly specified versions of low-level libraries would be quality.

I absolutely agree there are different types of users with different needs.  Usability and/or marketing teams often create personas (link, link, and link) to give the different types of users a concrete representation.  They can help keep the discussions and dev efforts focused in the right direction.  Do you know if NI's marketing has developed personas for Labview users, and if so, can they be shared with the tool development community?  That information would be very valuable for us.

I have worked in an environment where it was easy to relink to a different version of a dependent library (with all assumptions listed in Daklu's earlier post), but it was not automatic, meaning that the programmer had to opt in for changes to take place.  I appreciated control I had over timing of opting in.

So I can better understand where you're coming from, do you mind if I ask what environment that was and what your role is at NI?  I feel pretty safe assuming you're an experienced programmer, but I've been wrong in the past and I'll be wrong in the future.  The majority of my programming experience is in Labview with .Net (mainly c#) and VB6 as the runner ups.  I've also done assembly (Atmel AVR & Motorola 68HC11) and way back in college I did some c++, though I'd be hard pressed to produce anything in c++ these days.

(Edit - Rereading the paragraph above I realize it comes across like credential chest-thumping.  That's not how I mean it.  I'm just looking for some common experience to put the discussion in a concrete context.)

Like you, I appreciate being able to control the links to dependencies.  I think most software developers do.  I could be wrong, but I don't think most Labview users are software developers--they are users.  By that I mean they use LV as a tool to do their job, but writing code is not their primary job responsibility.  They are very smart people--engineers, scientists, academics--but they do not have software design knowledge or experience.  For these users automatic minor revisions and bugfix updates should be completely transparent (assuming they downloaded and installed the updated packages with VIPM.)

I don't know enough about GAC to comment on whether I'd like to see it emulated in LabVIEW.  The time for writing scripting tools for changing called versions should diminish with experience.

Let me try and clarify the terminology a little bit more.  (I welcome any contributions to defining terminology for this discussion.)

sxs - Any deployment strategy that allows multiple versions of the same package to be installed simultaneously.  This can be done right now with VIPM by changing the package name every time, but nobody does it because it is very cumbersome for the package users.

WinSxS - Microsoft's technology layer allowing sxs deployment for native code assemblies.  Depending on context, it can also refer to the file store where those assemblies are located.  (Typically c:\windows\winsxs.)  I'll use "WinSxs" to refer to the file store and "WinSxS layer" to refer to the technology as a whole.

GAC - The virtual store where Microsoft keeps all versions of managed code assemblies.  I've been using the term interchangably to mean the GAC itself and the entire technology layer the GAC is part of.  From here on out I'll use "GAC" to refer to the virtual store and "GAC layer" to refer the technology layer as a whole.

Assembly Redirection - This is the technique Microsoft uses to allow end users to easily relink to specific assembly versions.  It uses a combination of built-in technologies located in the OS or CLR (such as the SxS Manager and Fusion engine) and files end users can edit (manifests, config files, etc.) to customize the Fusion engine behavior to meet their requirements.  (More info here, here, and here.)  To me "redirection" implies something that happens dynamically.

Dependency Indirection - I'll use this term to refer to the concept of "indirectly" referring to a dependency, regardless of what layer that indirection takes place or whether the indirection is static or dynamic. The WinSxS layer and the GAC layer are dynamic dependency indirection implementations at the OS or CLR level. VIPM utilizes static dependency indirection at the file system level.

If we want to allow sxs package installation, something along the lines of the GAC has to be implemented.  It could be something as simple as what I'm doing by installing different package versions in different folders somewhere in vi.lib, or it could be something more complex similar to assembly redirection. 

Even if package developers get faster at writing version scripts, there are a couple reasons why I would prefer to find a better solution:

1.  The update script will have to be run against every application that uses the package instead of just installing the new package and having the fix being automatically applied the next time the project is loaded.  Repetitive manual intervention is prone to errors, even if we make the intervention as easy as possible.

2.  The Labview developer community is very small compared to the more popular languages, meaning we fewer and smaller teams developing packages--especially open source packages.  OpenG is the largest open source Labview project I'm aware of.  How many people are actively contributing to it?  (I actually don't know, but I'm sure it's much less than they'd like.)  Now that NI is shipping VIPM with Labview, I believe open source packages can become a driving force behind extending Labview's native apis with higher level functionality.  However, the process of preparing a package for release needs to be as streamlined as we can reasonably make it to encourage participation and ongoing support.

I agree that this is mostly a dev environment issue, but that it may also be a run time issue.

I'm not quite seeing how it could be a run time issue.  Static dependencies in built executables don't ever refer back to vi.lib do they?  Hmm... I suppose there could be issues if VIPM were used to deploy and manage run time components on a target system.  I've never used it for that but it might be a valid use case.  Anyone have any thoughts about that?

When I was describing "a separate problem", I was referring indeed for something easier on the user than brute force search-and-replace.

Keeping it close to the chest, aren't you?    I've been thinking about ways to use directory junctions to allow users to easily switch between multiple versions.  It allows more dynamicism with dependency indirection at the file system level, but I'm not sure it gives end users any real advantages over what VIPM does today.

0 Kudos
Message 44 of 71 (22 Views)
Reply
0 Kudos
Highlighted

Re: Avoiding 'vip hell' with VIPM packages

"This can be done right now with VIPM by changing the package name every time, but nobody does it because it is very cumbersome for the package users."

Why is this cumbersome?

0 Kudos
Message 45 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

MichaelAivaliotis wrote:

"This can be done right now with VIPM by changing the package name every time, but nobody does it because it is very cumbersome for the package users."

Why is this cumbersome?

Because sxs installation requires different namespaces, and changing the namespace with every bugfix would require the package user to manually replace every vi from Foo 1.0.0 with the equivalent vi from Foo 1.0.1 on their applications whenever a package update were issued.  The search and replace process can be scripted, but I don't think it can be entirely automated without inserting some kind of dynamic dependency redirection layer (similar to MS's assembly redirection technique) between the package user and the installed package code.

0 Kudos
Message 46 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

I understand. I thought that there was a VIPM issue you were reffering to. Basically the problem is there regardless of VIPM.

==

I don't see this any different than installing applications with Windows Installer. Some call them msi installers. Some companies choose to allow multiple versions of their app to be installed sxs. Others force an upgrade. For example LV allows multiple versions to be installed. Microsoft office or Photoshop not.

You also have this choice with VIPM. You can call your package: "My Package 2011" or "My package 2012" (allowing sxs). Or you can call it "My Package" (and not allow sxs).

How you as a end user of the package handle migration from one package to another is a much larger issue. VIPM is not currently tackling it directly. I'm not convinced that it should. VIPM does a lot of smart things but fixing your broken code by figuring out what the intention of the package builder wanted to do in a migration is not one of them. Now, if you want to add post-install actions that fix other people code for them and you have an infinite amount of time on your hands then VIPM provides hooks to do that.

PS. VIP hell? If this is VIP hell, then what did we have before vip files? I would argue that we had chaos.  We are far from hell right now.

0 Kudos
Message 47 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

Michael, I don't see any of this as a referendum on your product. The phrase "VIP hell" is, as Daklu admitted, a direct copy of the years-old common phrase "DLL hell" that describes the same problem with other types of binaries in Windows. It means nothing more than that. The point of the discussion is to identify all the issues surrounding distribution of incompatible constituent code in LV, and VIPM is without doubt the most common way to do that today.

0 Kudos
Message 48 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

I understand. I thought that there was a VIPM issue you were reffering to. Basically the problem is there regardless of VIPM.

Correct.  VIPM isn't at fault for this issue; neither is NI.  I consider it part of the natural growing pains that accompany increased code sharing among the community.  Windows, MacOS, Linux, and Java all went through their phases where they had to deal with dependency hell in one form or another.  Let's create a plan now instead of waiting for it to become a pain point for LV users.

PS. VIP hell? If this is VIP hell, then what did we have before vip files? I would argue that we had chaos. We are far from hell right now.

As Staab said, it's just a phrase to refer to problems that arise from not being able to install incompatible dependency versions sxs.  To be sure, we aren't in VIP hell right now (except for Staab, who is quietly dealing with his own mini hell.)  However, if we don't figure out how to prevent it from happening I believe it is almost certain we will be in 1-5 years.

Before vip files there wasn't the same potential for VIP hell because there wasn't nearly the same amount of code sharing going on.  The risk of VIP hell rises substantially with code sharing and using shared components to build other shared components.  No sharing, no hell.  Lots of sharing, lots of hell.

How you as a end user of the package handle migration from one package to another is a much larger issue. VIPM is not currently tackling it directly. I'm not convinced that it should. VIPM does a lot of smart things but fixing your broken code by figuring out what the intention of the package builder wanted to do in a migration is not one of them. Now, if you want to add post-install actions that fix other people code for them and you have an infinite amount of time on your hands then VIPM provides hooks to do that.

Yes, there is a workable solution to the problem right now without changing Labview or VIPM.  I do believe better long term solutions are available if we can agree on a course of action and NI and JKI are willing and able to devote some resources to the problem.  Right now the one thing I know I'd like to see in future VIPM versions is the ability to present a package line as a single package in VIPM.  I don't know what the best way to present that to the user is, but I know I don't want to see 20 lines of packages listing Foo.v1.0.0 through Foo.v1.19.0 every time I open VIPM.  It sounds like you are already investigating something along those lines.

I agree it's not VIPM's job to fix broken code or to act as some kind of dynamic dependency redirection layer.  Still, there could be changes to VIPM that would help users manage their sxs installation requirements.  Maybe a plug in mechanism that allows package developers to add custom right-click options to their packages.  Maybe it would be global options that allow end users to optionally change the namespace and install point for any given package instance.  I don't really know what the best long term solution will look like so I can't say what changes VIPM would have to make to simplify sxs installation.

I don't see this any different than installing applications with Windows Installer. Some call them msi installers. Some companies choose to allow multiple versions of their app to be installed sxs. Others force an upgrade. For example LV allows multiple versions to be installed. Microsoft office or Photoshop not.

The difference is in how they're intended to be used.  VI packages are components; LV, Office, and Photoshop are applications.  If I wrote a package that depended on Photoshop 7 that had some features you like and Staab wrote a package that depended on Photoshop 8 that had some features you like, then you'd be stuck choosing my package or his package, but you couldn't use both at the same time.  But people don't expect to use Photoshop as a component in an application, so very few people care that you cannot do sxs installation.  People *do* expect to use packages as components, and with that expectation the number of opportunities for version conflicts skyrockets.

0 Kudos
Message 49 of 71 (22 Views)
Reply
0 Kudos

Re: Avoiding 'vip hell' with VIPM packages

This is a good discussion.  It is important to be able to move to the latest tool version without relinking callers, and even if there are no major API changes (which seem eventually inevitable), it is imperative to have the option to service a fielded product while keeping it on the original tool version.

Thinking long term, would this move us in the right direction?

(1) VIPM could store VIs for installed packages like this:

vi.lib\My Company\My Tool #1\1.0.0.2\mt1_Foo.vi

vi.lib\My Company\My Tool #1\1.0.2.7\mt1_Foo.vi

vi.lib\My Company\My Tool #1\2.0.1.9\mt1_Foo.vi

(2) VIPM could display the Family (package line) in the LV Tools Network package list.

(3) VIPM could allow package users to dynamically select versions within a Family to install SxS or uninstall.

If package users wanted to do a blind upgrade for My Tool #1, they could deselect all versions in the package line except the latest OR click an "Install Only Latest" button.  At any time, users could go into VIPM and reconfigure the installed versions.

(4) LabVIEW could load the latest detected version of the package line into the palettes by default upon initial launch, but this could be overridden for an individual project if a package user wanted to load a legacy version for that project.

If a package user wanted to switch to a different package version for a project where they've configured a specific package version, they could close all VIs in the project, open a configuration dialog from within the project, and select a different version.  After the package user clicks "Apply New Version", LV could iterate through affected project VIs and update dependency paths.  If the package user discovers incompatibilities, they can revert back to the previous version using the same method.

Kosta wrote:

I agree that this is mostly a dev environment issue, but that it may also be a run time issue.

Daklu wrote:

I'm not quite seeing how it could be a run time issue.  Static dependencies in built executables don't ever refer back to vi.lib do they?  Hmm... I suppose there could be issues if VIPM were used to deploy and manage run time components on a target system.  I've never used it for that but it might be a valid use case.  Anyone have any thoughts about that?

The Biometric Login Toolkit VIPs install both edit time tools and an EXE (server), but theoretically the latest server version could support any previous API version.  There could be instances where package developers want to install SxS servers, but I'm not sure that this would affect VIPM or LabVIEW since each version of the API should be able to target the correct server.

0 Kudos
Message 50 of 71 (22 Views)
Reply
0 Kudos