collaboration on project (SVN)

[PeterFoerster]

Hello all,

I have a question regarding the workflow for collaboratively working on a project in LabVIEW using Subversion. First of all a bit of information about our projects:

We usually work on LabVIEW projects with 1-3 people at the same time. Even when it's just 1 person working a project, we have seen a great benefit in using Subversion, as it let's us roll back to previous versions and see what our first ideas were or transfer a new version of the software to the test bench easily and without creating loads of working copies and getting all confused. In this case, you don't really have the problem of two versions of the program existing at the same time except, when you make changes and don't commit properly. Our solution for this so far has been not to do this.

When the projects grow and more people work on a project, the scenario of several people working in the same vi becomes real.

We have had the case that my colleague couldn't commit his revision because someone had committed another revision before him and there was a mismatch.

My question now is: How do you tackle this problem?

So far I have not found a good way to solve this:

- VIewpoint TSVN Diff from within LabVIEW works great for comparing the last revision to whatever I am working with. Changes have to be made manually, though, which can be a lot of work in bigger projects. I still think this is the method with the best results.

- Using the LabVIEWs merge tool requires copying VIs from different revisions, which is also not the most comfortable way to go.

- Using the TSVN merge in the explorer doesn't do anything even with a small primitive example (why would it, though, it just compares text files...)

I'd be grateful for any input you can give me on this topic. Maybe you can share some insight into how you handle these issues. These are not show stoppers for us, just little problems that make an otherwise awesome tool imperfect. All the problems this has caused could be resolved quickly, too, because we had subversion to back us up. All this would have been a nightmare with USB Sticks and copies.

Cheers,

Pete

[Tim_McClung]

I'd ask one question first: What is the reason that your VI's are so large that multiple different pieces of code need modification at the same time?  You should have a iron clad reason for this or your code needs restructure.

Secondly:

 I think most of it comes from practice and developing an organizational dicipline that works for you.  I don't the same engineer should ever be working in the same VI.  I don't trust text based merges much less graphical.  LabVIEW VI's should be small.  Very small.  If they are big enough that two people need access to the same location I believe a reconsideration of your architecture may be in order. Not that your design is wrong, but you should be absolutely confident that moving forward there is no other way to implement your project.  There are reasons that you might have to, like a Real Time development project, but I'd strongly recommend trying to create some division of labour around creating SubVI's independently of eachother.   This is kindof answering your question by not answering your question, but I think the roadblock you are running into is a great example of why SOLID programming practices really save you time and headaches.  

[RavensFan]

I'd argue that Peter never said that his VI's were large,  just that his project is large.

[Tim_McClung]

Valid point.

One thing that has helped me at the project level is splitting compiled code and source so that a change to one VI doesn't ripple through your entire project.  This can cause endless (and needless) conflict issues.  If VI's are conflicting needlessly this often has been the cause for me.

[Tim_McClung]

I apoligize that I insunuated the your code doesn't follow those pratices. That was not my intention.  SVN is tough to work with and I've had lots of experience with stuff breaking even with a small team.  I've experienced all the same things you've mentioned, perhaps the correct way to word my previous message is that my only solution is just to work around it.  Try to get things as small as possible.  I don't mean to comment on the quality of your code or ability to design and I haven't found a solution for this, perhaps that's a comment on my coding ability more than anything else.  

Regards,

Tim

[Intaris]

We need to differentiate between two different sources of "changes" in LabVIEW files.

1) Two developers have made functional changes to code at the same time, and the first one to commit "wins".  The second one must somehow resolve the problem.

2) LabVIEW decides to save a VI with a different timestamp (or conditional disable) even though the code hasn't REALLY changed.

In case 1) you need to try to minimise these cases as they can be messy and time-consuming to resolve (not to mention prone to error).

In case 2) I have written a tool a long time ago to automatically retrieve a list of files marked as modified and then one-by-one do an automated LVCompare on them to see if the code has REALLY changed or if it's just LabVIEW messing about again (Yeah, RT Deploys, I'm looking at you).  I've mentioned this tool a few times and need to ask at work if I can release it.  It's not pretty, but it does the job and might serve as a springboard for a better end produce.

If you can more or less eliminate 2) then a few rules for 1) should solve 90% of the problems.  The remaining 10% just require patience and effort to solve.

[PeterFoerster]

Don't worry Tim, no harm done! After reading your first post, I took a look at my code (and my life's decisions thus far ) and came to the conclusion that it's good, the way it is. The reason my main VI is so "big" is that we are running several processes (FPGA communication, massive DAQmx, User Events...) in parallel. We have an 8 Core System with hyperthreading, which we need for processing power.

You are right in saying that my VI could be smaller (it's 1.2 Screens Down and 0.7 across) and I could easily do this by packing up the loops in their own subVIs, but I prefer it as is, because everyone can get the basic idea right away.

Even if I packed everything up into subVIs it wouldn't really change the core of my problem: different people working on different parts of the code that might be located in the same folder and get messed up on commit.

From the answers so far I gather that I'm not the only person having these problems and the best way is to work on different parts of the project. We do this already by developing the functionality as APIs in their own separate libraries, but sometimes people are done and they think: "Maybe I'll just implement this in the main project real quick..." Which is when the trouble starts.

By the way, do you guys commit globally or single folders? I've felt that committing single folders works well if you do it once or twice, but after that it can get messy quickly, too.

Another problem I've encountered are the .aliases and .lvlps files. I've excluded them from the versioning, but somehow they always mess up the "up to date status" of the folder.

[RavensFan]

---

@PeterFoerster wrote:

You are right in saying that my VI could be smaller (it's 1.2 Screens Down and 0.7 across) and I could easily do this by packing up the loops in their own subVIs, but I prefer it as is, because everyone can get the basic idea right away.

---

I'm impressed that  you got it that small and it is almost within an entire screen.  As good as the goal is to get it entirely within 1 screen, it can be very difficult to do, especially when you are still in the middle of developing.  The key thing is that things don't get so big where it is so many screens high or wide that you get lost in it moving around the block diagram.

[dan_u]

We try to avoid committing files that LabVIEW changed (e.g. due to typedef changes). Only commit what is actually modified by the developer. This usually limits the number of "real" conflicts. That of course requires keeping track of the modifications (should be done anyway in order to write the svn log) or using tools that tell you what actually has been modified ((programmatic) LVCompare).

Especially with RT targets this is not simple because you need to save the modified files (modified by LabVIEW) when deploying. And it leads to the situation that lots of local files appear as modified, and thus cause conflicts when updating (if they are not reverted before the update).

Separating the source code from the compiled code did not help much in this regard as far as I can tell. Changing a typedef'd enum still modifies the source of all VIs using that typedef.

Regarding the .lvlps and .aliases files, we also don't put them under version control. But they don't mess up the status of the folder. Do you probably have the option "Unversioned files mark parent folder as modified" selected in the TortoiseSVN settings (in the Icon Overlays page)?

I'm also curious if there are better ways to deal with these situations.

[Bob_Schor]

A few miscellaneous comments.

• I generally create a Repository for every major LabVIEW Project.  I'm also the guy who gets my colleagues clued in to Subversion, so (surprise!) they create Repositories at the Project Level, too.
• My practice is to do a Commit or Update at the Project level.  I tend to not use "hooks" built into LabVIEW for Subversion, but to use the Windows File Explorer extensions that TortoiseSVN creates.  I right-click the top-level Project folder and choose Commit or Update, as needed.
• At some point, I right-click the Project folder, choose Properties/Subversion/Properties.  I then create the following Properties on this top-level folder by choosing the Edit button:  svn-global ignores, *.lvlps and *.aliases; tsvn:logminsize, 40.  After I define the Property, I make sure that Apply Property Recursively is checked.  This sets these properties on all of the files/folders in this Repository.  What the Global Ignores do is to omit .lvlps and .aliases entries from the Repository, and to require a 40-character Revision entry before you can do a Commit.  Note that you may still have to "clean out" any .lvlps or .aliases file already committed -- just right-click it, choose the TortoiseSVN entry from the DropDown, and then choose Unversion and Add to Ignore List.  This removes the copy in the Repository.
• The time one gets into trouble with Conflicts are when someone modifies a TypeDef or a utility VI that everyone uses.  If you modify the TypeDef, for example, and I modify a VI that uses this TypeDef, if you commit first, I'll get a conflict when I try to Commit.  However, I'll be given the opportunity to do an Update (which will give me the modified TypeDef, then do the Commit (which should now be "safe").
• When things really get messed up is when someone forgets to Update at the beginning of the day or Commiit at the end of the day.  [I'm usually the guilty party -- Bob-at-work makes a bunch of changes, forgets to Commit, goes home, makes a bunch of changes, Commits, goes back to work, forgets to Update, makes more changes, and a Mess Ensues.  Unfortunately, the only guy to blame is Bob ...
• A bunch of years ago, I (also) independently developed a routine that would let me point to two folders with the same Project Tree and have LabVIEW run LVCompare on all of the VIs that had the same names, producing both lists of unique files in each of the two Folders, a list of files that were "identical" (meaning LVCompare considered them to be the same), and files that LVCompare considered different.  When you are dealing with thousand-VI (or even hundred-VI) Projects, this saves a lot of grief.  There have been Ideas on the Idea Exchange (I believe I posted at least one) for NI to make this functionality more available ...

Bob Schor