The question at hand is one of philosophy. It is entirely possible for two rational people to come to completely different conclusions on this issue. We would like to hear our customers' opinions in order to help us choose the best course for the LabVIEW experience as a whole.
LabVIEW 2009 just released. It includes a revision to the configuration file VIs. We have a palette full of new VIs to replace the old palette of VIs. The new palette VIs preserved all the functionality of the old, but they add some additional features and/or they have some performance improvements. The upgrade path appears clean, no problems there. In general, this looks like a successful refactoring of this palette of VIs.
The key word in the preceding sentence is "palette." Many of LabVIEW's palette VIs rely upon subVIs that are not in the palettes. Those subVIs sit on disk in vi.lib. In the case of the configuration file VIs, there used to be a subVI for functionality X. In the revised VIs, there is no such subVI. There was a second subVI for functionality Y. That subVI still exists, but the all the configuration file VIs are part of a project library now, and that Y.vi is marked "private." This means that no block diagram can use that subVI other than the VIs that are members of the library -- which means users cannot use that subVI directly in their own code.
The fact that Y.vi is now private caused a problem for at least one user who had been using Y.vi in the prior LV version. Their VI loaded broken in the new version, and the user had no way of fixing the problem except by directly editing the vi.lib library file to make the Y.vi public. Although that fixed the VI on their machine, it meant the VI was not portable to other machines with the unmodified vi.lib. No one was burned by X.vi going missing, but we can imagine a hypothetical user who relied upon X.vi. That person doesn't even have the workaround of editing the library file.
Now, had the configuration VIs been a library at the time they were first released, both X.vi and Y.vi could have been marked private. No user would have been able to use X.vi or Y.vi at all, so when those VIs were changed and/or removed, no user could possibly have been affected. This is one of the major benefits of marking functionality private. If you don't intend to support the functionality in the future, private prevents your users from developing a dependency on that functionality.
On the other hand, had the VI been private, customers might have been stuck, unable to do whatever task they needed to do because NI hadn't thought to expose some functionality. An easy example of this is the "bold face string" subVI that is part of error.llb. It is used by the General Error Handler dialog to display a string containing <b> and </b> tags as a string indicator with the appropriate text in bold face. Is that something that the General Error Handler needs in order to do its job? Yes. Is it something that we would want to expose as part of the error handling interface? Of course not. So it would have been private. But many people have discovered that subVI over the years and used it quite happily in their own code.
What NI faces is the classic problem of any API developer -- what do you make public, and what do you make private? In the past we have relied upon the expectation that if something isn't in the palettes, it will only be found by users who go digging through our subVIs, which tended to be our more advanced users, and those users would be more understanding of the fact that they were using "unexposed functionality." But that's a really poor principle to rely upon, especially as LabVIEW grows to be used in ever more mission-critical activities. Good API design says that we should be marking anything not in the palettes as private. But we may not want to hide potentially valuable functionality from users who go digging.
Where do you think the balance should be? Should VIs not in the palettes be marked as private? Should we continue assuming that our customers who use on-disk publics will be adaptable if their code breaks during an upgrade? Should we try to maintain all of our subVIs, even the ones no longer needed by their original callers, on the off chance that someone somewhere is using it?
Hi Aristos. Thanks for posting this - I'm at least one of the users raising this question.
I understand both sides of the debate. I can certainly see that NI wouldn't want to maintain all of the underlying code. In fact, when I link to that I understand I'm taking the risk that it will change, as it did with this refactor. I'm glad you are refactoring some of the palette because I believe it needs it in places.
So, my side is really more of a question of extending functionality. In the configuration vi's case, I would like to add a method to save the configuration back to disk without closing the reference. Currently, the only save function is in the close method. I would prefer to stick to the architecture that's implemented, and I would prefer to use the NI subvi's, since then upgrading to the next LabVIEW will hopefully provide me with enhancements / fixes. In this particular case, I would like to use the reference to queue convert and the file saver.
I can easily, and will probably for now, copy the subvi's, renamed, to my own source control and go on my way. The main issue with this would be upgradeability - each version release I will have to re-copy the new vi's or check them. Then again, I took the risk so in theory I should be doing that anyway (and I do typically now with a unit test framework).
One thought I've had - although I'm torn on this as well, is to extend the new "friend" concept in libraries and allow me to accept the risk by making my own library that adds your library as my friend. Maybe it's more of a distant relative or an "acquantance" 🙂
Anyway, I'll be interested to hear other's opinions on the matter. As you've mentioned, Aristos, there are lots of hidden jewels that are down deep that I'm sure others use. I can certainly see NI saying "Only Palette vi's are supported."
Thanks for listening!
wow, that is a tricky one... on the one hand, as you say, from an API maintanence point it is totally wise to have all of the internal functions private, but I know I have definately used a couple of vi.lib vi's in the past (such as the variant utility Get Numeric Info to programatically get a list of enum strings without having to create a control / property node, or the non-simple "Write Registry Key" for using datatypes not supported by the "Simple" vi.).
I guess the way I would do it in my own APIs is to keep the truely core functionality private, but leave open some of the utility VIs public but not on the palettes. For example, in the case of the Registry VI's, keep the "Write Registry Key.vi" (non-simple) public but have the "Registry Handle Master.vi" private.
IMHO, an even better solution could be applied to LVOOP based libraries - keep the core VI's Protected, so that if somebody wants to extend the class to add in additional functionality they could have a safe, reliable way to do so, without having to edit the vi.lib files directly.
Should VIs not in the palettes be marked as private? Yep.
Should we continue assuming that our customers who use on-disk publics will be adaptable if their code breaks during an upgrade? Nope, you should make it so the code doesn't break.
Should we try to maintain all of our subVIs, even the ones no longer needed by their original callers, on the off chance that someone somewhere is using it? Yes, but not indefinitely. Trying to maintain permanent backwards compatibility leads to bloated code that is hard to maintain. Don't make that promise.
There are really two questions here:
Regarding the first question, there has been a tacit agreement in place that those VIs are not officially supported. As painful as it may be for end users I think the only reasonable answer is to make them private and offer workarounds where you can. In the case of "Bold Face String," which clearly should not be exposed through error.lvlib, might it be possible to create a StringFormat.lvlib that included VIs for making text bold, italicized, etc? It wouldn't be a drop in replacement but it would at least offer the customer a viable workaround, which is the most any reasonable developer can hope for when using unsupported VIs. However, NI should also provide an analysis tool that scans source code files for unsupported VIs that have changed behavior or are no longer available in new releases. That will help prevent inconvenient surprises.
If it is possible to continue to use the .llbs as interfaces (little 'i') and refactor their functionality into the .lvlibs I'd be all for that. My sense is that the amount of developer time required to create and maintain the backwards compatibility isn't justified by the number of customers who need continued access to those VIs.
I don't have a good answer to the second question. Now that NI has the ability to make VIs private in lvlibs, if a VI is available for the public to use (regardless of how difficult it is to find) then NI has a responsibility to not break backwards compatibility. As a rule if I break backwards compatibility in a reusable code module I've developed I'll rename it with a v2 suffix and deploy both, marking v1 as obsolete. This lets developers know they should update their code to v2 but allows them to do so on their own schedule. I'm strictly small-scale though... I have no idea how well this would work across the entire Labview API.
Perhaps the extra functionality can be exposed to advanced developers through a separate interface (little 'i') lvlib marked as a friend to the first. Make interface available via a freely available addon (or maybe with a product activation code) with the explicit agreement that these lvlibs can be revised frequently; possibly every release. Since the interface lvlib is much smaller than the functional code contained in the private members of the implementing lvlib, using a versioning system to deploy multiple versions simultaneously carries less cost. When a new interface version is released, continue to maintain backwards compatibility of the previous version for x number of Labview releases, after which new version may no longer be compatible.
We were also hit by this issue (Config VIs core SubVIs that has become private, among other things, broke the Right Click Framework in LV 2009).
This is a tricky question to answer.
There might be an in between solution (although this will require more work from NI) where:
I know that NI used this approach in the past (I think it was when you rewrote the queue to convert them to primitive AQ), and I think this could satisfy both side.
The only major drawback is that you end up with two libraries for the same functionality and that you can not prevent user from using the legacy one instead of the new one in new code (although updating the documentation and clearly specifying that this will be phased out over time should help in that regard).
I agree, I like the .lvlib format.
We also lost two functions: 1) Writing config data to disk without closing the reference and 2) Getting the config file path from the refnum.
As other routines get packaged into libraries, I hope NI is consulting with Alliance Members, etc on what functions get exposed to the user.
I think the two mentioned above are pretty common and I'm disappointed they got missed.
AQ, you are asking the $64k question every API developer goes through. I see both sides of the issue for NI as I live with old API issues in my non-LabVIEW code. Let me just throw out a few ideas to think about. This is some of the thought process I go through when I have my API hat on. I'll use LabVIEW lingo even though the API experience is in text languages:
It's tricky, because the very items that you think would have no value, such as the convert reference to queue in the GOOP architecture, are the items I'm needing to be public to extend functionality.
Extending a class could almost be a seperate conversation from the hidden jewels that have been mentioned in this thread. Perhaps an architecture change in LabVIEW could enable extending classes. Maybe the memory be reference will turn into the reference GOOP architecture of choice, and that would solve the issue. I haven't played with it much or read too many forums lately - maybe it's already the hot ticket.
I want NI to notice that 2 people now have requested the Config VI's have a write to disk without closing. I also have a function that looks up the filename as Nicholsj mentioned, so there's 2 votes for that one as well.
We do not only refactor the code of Configuration File VIs, but also changed its behavior to be more compliant to common practices. That's why its subVIs changed so much. The subVI changes do break some of the customer's applications. To aviod this happens again, we decided to make all its subVI private.
If a customer want to use any of its subVIs, he has to keep a copy of the subVI, and maintain the copy himself in the future. It losts the upgradeability. But it guarantees an application works on the future LabVIEW version. For most customers, keep their application work is the most important thing, have better performance is the second.
If one VI is required by lots of customers, then it is worth for NI to take some effort to make it an API VI and continuing maintaining it in future LV. If the VI is only required by two or three customers, then it is reasionalbe for the customers to maintain it themselfs. I suggest that everyone posts his subVI requirements on the LabVIEW idea exchange forum. So that NI can count how many customers want the VI. This will help NI add more API VIs.
Thanks for taking the time to address this issue.
I've only skimmed the other replies to this post, so I'm sorry if I'm repeating or not considering others' feedback.
First, in the case of the config VIs, (in the context of your X.vi and Y.vi examples, AQ) I was burned by both X.vi going missing and Y.vi becoming private.
There are really two use cases for violating the encapsulation of the config (and other vi.lib) VIs:
1) to call its private utilities
2) to extend its functionality
And, I've done both.
Regarding the refactoring of existing vi.lib code to use LVLIBs, I generally think that new versions of LabVIEW should not break users' code (regardless of whether users are calling vi.lib stuff that's not in the palettes). Any stuff that you want to make private should be duplicated and left in a deprecated LLB (with the same VI names) so that users' code can find it. And, if deprecated vi.lib VIs are going to be removed from vi.lib, users should be notified well in advance of the LabVIEW release (maybe a year/release in advance). The deprecated VIs should have their icon color changed and their descriptions should be amended to describe the deprecation, upgrade path, and timeline for removal from vi.lib.
My solution is probably going to be to copy config.llb from LabVIEW 8.6.1 into my project sources and relink all my projects to this version (actually, I'm just going to create a VI Package and install it with VIPM, but that's another topic). NI could (and probably should) have saved me this trouble by simply deprecating the old config.llb and doing all its refactoring in a duplicate config2.llb. If and when I'm ready to move to the new config2.llb, I'll manually relink/upgrade my code to call this new and improved version.
Note: I'll add that OpenG has a Library VI Deprecation Process that works quite well.
As an aside, my thought on palettes = public (and everything else is private): I don't think that the palettes should pulled into the discussion of public APIs, moving forward. The palettes make it convenient for users to find public functions, but the palettes seem to be driven by marketing and the inherent limitations/constraints of the palettes, and I don't think the palettes should constrain the API developer's decision about what's public or private.
Regarding my extensions to the config.llb, here are just a couple of the additions that are most efficiently implemented by calling private, internal VIs of config.llb. These address the use-case of operating on configuration strings in memory that don't exist in a file on disk: