Repeating VIA tests gives the same failures

[cbutcher]

I didn't see anything on this board via the search, but maybe I missed something - anyhow...

If I run VIA on a project/library/bunch of VIs etc and get some failures, then go ahead and fix those (with the Results Window open), then click "Done", Don't Save, and "Yes" for return to VIA, then rerun the tests (i.e. click "Analyze") I get all the same results - the VIs appear to be cached from the previous run.

Does everyone else observe this? Can it be reasonably changed?

[StefanLemmens]

I have also seen this before. You have to close the VIA  and restart it to make it see your changes.

It would indeed be a lot easier if it wouldn't cache the VI's but reloads them every run.

Best Regards,

Stefan Lemmens

Intersoft Electronics

[Darren]

I have never seen this. I frequently fix VIs while the Results Window is open, the re-analyze to make sure I didn't miss anything. If you can post a simple project and steps that reproduce the issue I will look into it.

[Darren]

Also, another thing you can try. The VI Analyzer engine uses caching to speed up performance. You can see if turning off the cache will fix the issue. To turn off the cache, add the following INI token (no LabVIEW restart required):

VIAnalyzerEngineCacheSize=0

[cbutcher]

---

@Darren wrote:

I have never seen this. I frequently fix VIs while the Results Window is open, the re-analyze to make sure I didn't miss anything. If you can post a simple project and steps that reproduce the issue I will look into it.

---

I figured it was a common workflow (I often try to do it) but I wasn't able to immediately reproduce it in a very simple project. Here's an example image (although it doesn't show much, and doesn't prove anything).

I initially tried in a project with just a single VI - it correctly updated there. I then added the VI to a library, created a new failure, and ran again. I then fixed the failure, reran, but again it was updated.

In the project pictured above (46 items according to Property Inspector Free, mostly VIs and a few class files all in one library) using a VI in the library updated correctly, but VIs in the class did not. I ran the VIA on the library file (excluding 43 additional items from JKI VI Tester and tests).

I'll pack the project up into a 2 commit git repository and send it to you via PM, if that's OK? I don't know if checking out the second commit and then comparing is different though, so feel free to make changes yourself 🙂

[Darren]

I tried for a long time and couldn't reproduce the issue. I noticed in your Steps.txt file that you are right-clicking the library in the project to analyze it. Can you try the following analysis mechanisms to analyze the same library and see if the problem reproduces:

1. With no files open, Tools > VI Analyzer > Analyze VIs > Start a new VI Analyzer task and add the folders on disk containing the files you want to analyze.

2. From your project window, Tools > VI Analyzer > Analyze VIs > Analyze the current project.

When you right-click the library in the project, it goes through a different code path than the two ways listed above. I'd like to know if your problem is specific to the library-right click code path, or if it happens to you no matter how you perform the analysis.

[cbutcher]

Thanks for trying. Here are my comments and responses today:

0. I first used the right click menu to check I could reproduce my own issue on this computer (different PC). To speed things up, I selected only the "Documentation" group of tests. I didn't see the issue (I had initially 26 failures, fixed one, reran and had 25). I then closed VI Analyzer, repeated with all tests (except Real Time) and saw the issue reported in this thread (the fix I made in this case was an overlapping wire, but I don't think it matters).

1. I used the Start a new VI Analyzer task method, did not save my configuration, and started with the "Current Folder" (so all of them) and then excluded Testing and .git. I then selected all test groups except Real Time. I fixed 2 of 68 issues, then returned to the menu and repeated, again clicking No (Do you want to save your changes?). Following the testing, I again was told I had 68 issues, and the two I had fixed again appeared. Double clicking opens the diagram, which clearly shows the new comment (fewer than 1 comment).

2. Again I excluded the Testing folder, and fixed 1 of 55 issues (I'm not sure why the total is so different... This time I checked and saw 40 VIs were tested but I didn't check other methods). Following the same repeat method (Done, No, Yes, Analyze) I again had 55 with 1 seemingly outdated. I fixed a couple more issues and then clicked back on the project window and used the Save All shortcut (Ctrl+Shift+S) and repeated the process. This also didn't make a difference, although a small dialog (with no text) did quickly pop up and close, so perhaps(?) something was saved. There was no dirty dot in the project window though...

3. I set the ini key you mentioned (VIAnalyzerEngineCacheSize=0) and did not restart LabVIEW. The VI Analyzer was closed at this point. I then reran the test using the method from (2.). I initially had 52 failures, from which I fixed one from "Create Comments Channels.vi" (in the library, but not a class member) and one from two different VIs in classes (same name, DD, wanted to avoid my own confusion) so 3 in total (all the Debugging Enabled test, which I have mixed feelings about for these VIs, but it's easy...). Repeating the VI Analyzer tests gave me 51 failures. The failure from Create Comments Channels.vi was updated correctly, whilst the class member VIs did not update.

In conclusion so far, it appears that

• small numbers of tests/VIs (not sure which, maybe product, maybe one or the other) do not exhibit this, but just 40 VIs making a total of 4000 tests is enough to demonstrate this (not a desperately large project)
• VIs in a single level library are fine (at least for me). Nested libraries are untested, except in-so-far that classes are special libraries
• VIs in a class in a library do not update, using any of the methods you suggested or the one I previously used (right-click menu)
• VIAnalyzerEngineCacheSize=0 does not change the results for method 2. I didn't retest the other methods but expect probably the same results
• Using the Save All method (at least via shortcut) for the project doesn't affect the results (compared with just Ctrl+S for VIs before closing them, after opening via double click in the VIA Results Window)

Hopefully this is at least partly helpful, although I'm unsure what I did differently to you such that you didn't see this 😕

[Darren]

Thanks so much for the thorough testing and reporting of results. I have one other thing for you to try whenever you get a chance. I noticed that you have all the code in the project marked as 'separate compiled code from source'. Can you turn this setting off for all your source files (right-click top-level project item > Properties > Mark Existing Items) and repeat your tests and see if the results differ?

[cbutcher]

I unmarked all items through the project properties and then saved all (90 items updated in dialog, seems about right).

Running using method 2 (Analyze this project, following Tools menu selection) gave me 49 failures (4000 total tests, 40 VIs)

Fixing 2 (one in Create Comments Channels.vi, one in a class VI) gave me 48 failures on repeated test via the same set of menu choices.

So it appears that the same is true regardless of the "separate compiled code from source", unless I should have also mass compiled the directory first?

[Darren]

I was finally able to reproduce your issue pretty reliably. I have filed myself Bug 961442 to investigate further. I will try to get a fix for the next LabVIEW release but no promises, as I still haven't narrowed down a root cause yet.