NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Type List Order Numbers (typelistordernum) change when opening a sequence file in Test Stand


@ATE Coder wrote:

Hi Doug,

 

Thanks for your help.

 

The types do reside in type palette file, which is part of the project. Indeed, the type resolution happen but this is far from silent :).  Obviously, after resolving what test stand perceives as a conflict, the sequence file changes. Now imagine this in a version control system. Two stations were updated and all files get changed for no apparent reason with the programmer having any clue as to what specific changes took place. Nice, right?

 

Now you may say why save the file. Is this really a viable solution?  Those files will have to be edited at some point at which time a conflict will ensue which source is unknown.

 

Does this type resolution not seem to you like a solution seeking a problem?

 

David

 

 


I'm still not fully understanding the problem you are hitting. What do you mean by "at which time a conflict will ensue"? What sort of conflict is ensuing? Do you mean that you are wanting to merge changes made in parallel by multiple developers into your source control system? If so are you using TestStand 2010 or higher? If you are using TestStand 2010 or higher you should use the teststand file differ to do the merging. It has a 3 way merge feature and understands type conflict resolution. If this is not the problem you mean then please explain in more detail why it matters if someone saves the file at some point, because if you don't mean the 3 way merge case, I'm not sure why it would be a problem.

 

If you are hitting the 3-way merge issue, please let me know what source control software you are using and what version of TestStand and I might be able to explain how to integrate TestStand's 3-way merge tool with your source control system.

 

-Doug

0 Kudos
Message 11 of 22
(972 Views)
In short, what I am trying to say is that because of this `conflict` resolutions, files get changed for no apparent reason, which is a problem for those of us trying to keep track of real changes, which is hard enough as it is.
Placing Test Stand under version control has been quite a straggle.
0 Kudos
Message 12 of 22
(968 Views)

I think it would be of value to dig deeper in what is going on.

 

Looking at the XML sequence files, I see that Test Stand holds a copy of the entire type in the sequence file. When a type changes in the palette (ini) file, it is time stamped allowing Test Stand to identify this type as different from those saved in sequence files. Until these conflicts are resolved, the sequence file includes hidden 'changes'. These changes manifest themselves when the files are loaded.

 

The corollary for a version controlled environment is: load all sequence files before checking the 'changed' files. Obviously, this pauses some issues as it is hard to tell which files changed.

 

Presently, sequence file analysis does not detect these conflicts.

 

Here is what NI could do:

  1. Revamp the analyzer to detect type conflicts;
  2. Add a tool that will detect and resolve all conflicts in the workspace;
  3. Allow adding types as reference top the palette files instead of as copies.

 

 

 

 

0 Kudos
Message 13 of 22
(965 Views)

It is not obvious to me why a file being marked as modified is a problem. I'd like to understand your use case so that I can either help you come up with a workaround or alternative process for handling source control, or have some reasonable idea of how it can be addressed with changes to teststand. In order to do so I need to understand, in as much detail as possible, exactly why it's a problem for you that your files are marked as modified when they are loaded after a type they use has been modified. In our workflow it is generally not a problem, so there must be something different that you require or are doing. Please explain in as much detail as possible what exactly the problem is and why it's a problem.

 

Thanks,

-Doug

0 Kudos
Message 14 of 22
(962 Views)

@ATE Coder wrote:
In short, what I am trying to say is that because of this `conflict` resolutions, files get changed for no apparent reason, which is a problem for those of us trying to keep track of real changes, which is hard enough as it is.
Placing Test Stand under version control has been quite a straggle.


Why is it a problem for "keeping track of real changes"? What is your process for doing so? Not everyone does things the same way, so it would be helpful if you could explain to me exactly what your process is and why and how the behavior of TestStand is causing problems for you. It's possible there is already a solution to your problem which I might be able to suggest if I better understood what your problem is.

 

-Doug

0 Kudos
Message 15 of 22
(960 Views)

@ATE Coder wrote:

I think it would be of value to dig deeper in what is going on.

 

Looking at the XML sequence files, I see that Test Stand holds a copy of the entire type in the sequence file. When a type changes in the palette (ini) file, it is time stamped allowing Test Stand to identify this type as different from those saved in sequence files. Until these conflicts are resolved, the sequence file includes hidden 'changes'. These changes manifest themselves when the files are loaded.

 

The corollary for a version controlled environment is: load all sequence files before checking the 'changed' files. Obviously, this pauses some issues as it is hard to tell which files changed.

 

Presently, sequence file analysis does not detect these conflicts.

 

Here is what NI could do:

  1. Revamp the analyzer to detect type conflicts;
  2. Add a tool that will detect and resolve all conflicts in the workspace;
  3. Allow adding types as reference top the palette files instead of as copies.

 

 

 

 


1. If you don't like automatic conflict resolution you can disable it, however I don't think the alternative of having to manually resolve all changes is a good one.

2. There is an update sequence files tool in the tools menu which can operate on directories and can update all of the files in a particular directory. I don't recommend doing this as part of the development process though. I only recommended updating all of the files like this before building a deployment in order to maximize performance.

3. There is an important technical reason why type definitions are saved in sequence files. The sequence file was written with an expectation that certain types with a certain structure existed. In order to enforce such an expectation when the file is loaded, the file needs to know exactly what the definitions of the types it depends on were at the time the file was saved so it can tell if the currently loaded versions of the type match those expectations. Also, there is no guarantee that the type is even loaded already when a file is loaded, thus in order to guarantee a file is even loadable, it has to store all of the types on which it depends so that they can be recreated at load time if they do not already exist on the machine loading them.

 

Hope this helps explain things,

-Doug

0 Kudos
Message 16 of 22
(959 Views)

Doug,

Thank you very much for trying to understand my frustrations.

You ask why is it a problem that the IDE changes? It is because I did find myself releasing a package with unresolved conflicts. So, if I push 100 files to 100 developers and all 10000 files change for no apparent reason we could have 10000 commits to the repository saying: dha?! on top of 10000 potential merges to deal with. Moreover, a version that was thought to be sealed, is now open. Go explain that to the configuration manager.

 

This is the exaggerated scenario of what I had do deal will trying to push a project of modest size to 2 production stations late at night.

 

I guess my problem is that I have been spoiled by Visual Studio, where, when a conflict occurs anywhere in a solution (=workspace), I am directed to these conflicts and the project will not compile unless these conflicts are resolved. In the Test Stand IDE conflicts become evident only if the sequence is 'loaded'. So, unless one is able to load all sequence files after every change, one could release files with hidden conflicts. Any change to a pallet type could cause such a conflict and, therefore, would require a full 'load' of the entire workspace, which is impossible even in a mostly sized environment.

 

BTW, I believe we have similar issues in LabVIEW. However, there my metaphor is that one is editing DLLs and one, therefore, is stuck with both the pros and cons of such an endeavor. Separating code from object, which is now possible in LabVIEW, alleviated some of these issues.

 

Also, maybe this explains why LabVIEW loads its files when starting. If Test Stand had done the same, then a global conflict resolution would be forced at the source and one would not be likely to push files with unresolved conflicts.


In short, our work is hard enough as it is.

 

Thanks again,

 

David

0 Kudos
Message 17 of 22
(951 Views)

I'm still not sure I'm completely following your situation.

 

When you say you get a conflict, do you mean the conflict resolution dialog is displayed? Or do you just mean that the sequence file is marked as modified because it is converted automatically to use the version of the type in the type palette file when it is loaded? The former should not be happening and is avoidable if you follow best practices (which I can explain in more details if needed), and the later should not affect how the sequence files run or work in anyway, they should still run fine.

 

When you say deploy, do you mean deploy to a production environment where no one should be modifying the files anymore, or do you mean something different? If you mean deploy to developers, what kind of source control are you using that deploys to developers and exactly what does it do? I am not aware of any source control that behaves like this. Most source control systems that I am familiar with require the developer to "get" the latest files from a shared depot and require them to checkout files they wish to edit and then check them in when they are done editing them (which commits them back to a shared depot). Also why would the developers change the files? You mean they are resaving all of the files because they are marked as modified and resaving them automatically commits them back to the depot?

 

We could potentially add a feature to not mark files as modified for automatic type conflict resolution. The files would still really get modified when loaded because they would have to be converted to the latest version of the type in the type palette, but we could potentially hide this detail from the user by not marking the files as modified. The file would still get written out with the latest version of the type if it was modified and resaved for another reason though. Would this address your needs? It sounds like it would, but some things you previously said made it sounds like it might not, so if not, please explain why it does not.

 

In the meantime, if I am understanding your use case correctly, I would recommend either:

1) Using the tool menu tool to update all sequence files to the latest types before deploying (though I'm not exactly sure what you mean by deploying since it sounds like you mean something different than the usage of the term that I am familiar with).

or

2) instruct your developers not to save files just because they are marked as modified when opened. They should only save them when they actually make some sort of edit to the file.

 

-Doug

0 Kudos
Message 18 of 22
(942 Views)

I think we (like we here at work) and also we like you and I are making progress. I added my comments below each of your reply paragraphs (in italics).

 

When you say you get a conflict, do you mean the conflict resolution dialog is displayed? Or do you just mean that the sequence file is marked as modified because it is converted automatically to use the version of the type in the type palette file when it is loaded? The former should not be happening and is avoidable if you follow best practices (which I can explain in more details if needed), and the later should not affect how the sequence files run or work in anyway, they should still run fine.

 

Both.  

 

On the former, please let me know of any procedures I could follow to prevent this from happening.

On the latter, we found that the Analyzer does see this conflicts. So, our solution at this time is:

  1. Turn on analysis of sub-sequences;
  2. Analyze before committing the code to version control.

Obviously, this is not guarantied to happen every time so we should expect to see conflicted files. 

 

Also why would the developers change the files? You mean they are resaving all of the files because they are marked as modified and resaving them automatically commits them back to the depot?

 

We use GIT, which works like you described: good job.  So, once a file with conflicts slips through it has to be saved. Otherwise, the IDE will not run.  What NI could do is:

  • Highlight what the conflict is --for example, instead of marking the file as modified, mark it as having a conflict;
  • Tag the conflict (e.g., highlight the conflicting type in red);
  • Explain the conflict (e.g., right click on the conflict and click description...); and
  • Allow the programmer to resolve the conflict (e.g., right click and select 'Use type from the <pallet file>.ini'.

We could potentially add a feature to not mark files as modified for automatic type conflict resolution. The files would still really get modified when loaded because they would have to be converted to the latest version of the type in the type palette, but we could potentially hide this detail from the user by not marking the files as modified. The file would still get written out with the latest version of the type if it was modified and resaved for another reason though. Would this address your needs? It sounds like it would, but some things you previously said made it sounds like it might not, so if not, please explain why it does not.

 

I prefer to see the IDE as less rather than more proactive. I would much prefer if the IDE will tag the file as having a conflict, point to the conflict and allow me to resolve it as I described about.

 

In the meantime, if I am understanding your use case correctly, I would recommend either:

1) Using the tool menu tool to update all sequence files to the latest types before deploying (though I'm not exactly sure what you mean by deploying since it sounds like you mean something different than the usage of the term that I am familiar with).

or

 

I tried to use the tool but decided to pass because I was unclear as to what type will be used to replace the conflicting types. The tool mentions the loaded type. But which is the loaded part is unclear.

 

2) instruct your developers not to save files just because they are marked as modified when opened. They should only save them when they actually make some sort of edit to the file.

 

 This I would not do. The programming work is hard enough as is and in my experience this directive is self defeating -- it will not work because it is hard to follow.

 

0 Kudos
Message 19 of 22
(934 Views)

On a second note I have to take back in part the statement that the Analyzer finds the conflicts because I just made a change in the pallet types and ran the analysis on the to level sequence file with the 'Analyze Subsequences' option turned on and it reported no conflicts. Yet opening a subsequence file where the modified type was used showed the file as modified.

 

So it looks like something is amiss in how the Analyzer works.

0 Kudos
Message 20 of 22
(932 Views)