Offline Results Processing Utility (ORPU) performance issues

[Corey.Rotunno]

Jeff,

  I am going to follow up on my issues using other methods (phone support).

 I am not sure Trent understood my issue, this is not an issue of processing files on a network, or writing reports to the network. This is not about processing files at all, actually. In my previous posts, when I refer to "output" folder this is not where my reports are being stored. This is simply the folder that the ORPU moves the .tsr when it is done.

My issue is only about having already processed .tsr files on the network, period. It cannot be the intended behavior that the ORPU becomes unusable just because I store my .tsr's on the network. 

The processing of my files has no problems, it happens quickly without issue. It is when I open the ORPU interface it dogs down, even when I am not asking it to process anything.

By the way your comment "something like a VI/script that looks at the network folder, finds the files there, copies them to a local drive, and runs the ORPU from there".....this is exactly what the ORPU is doing for me. If I want to "re-process" a .tsr that is on the network, it first moves it to my local "unprocessed directory", processes it, then moves the .tsr back to the network "output" folder.

Thanks,

[WireWeaver]

Corey,

I see what you mean now. The problem is that ORPU wasn't designed for use with network drives. The inbox and outbox are checked once per second to see if they changed and update the display accordingly. We ship the source code if you want to look at the details:
%TestStand%\Components\Models\TestStandModels\Offline Results Processing Utility

I created CAR 636104 to investigate usability improvements in the future. The best workaround is still to use a local outbox and move the files later. If you want to change and rebuild the utility yourself, you need to use the project from 2014 because of this problem.

Have a good one!

Trent

[Corey.Rotunno]

Thanks Trent! I am employing that work around immediately. When I get a chance I will dig around the source, thanks for that info. Keeping my fingers crossed that something comes out of the CAR, the ORPU is pretty promising and it would be great to be able to have our whole test engineering department use it as the go-to tool for reprocessing past results in a seamless manner.

Thanks again,

[BoKnows]

It looks like Trent has got things to as much of an extent as they can get in the current state.  I did want to point out a couple of things.

"Period" is a dangerous word to use in anything dealing with software.  With an abstracted implementation, it's hard to say exactly what's taking place.  In this case, Trent mentioned the polling taking place is causing the problem.  There's a world of difference between ideal and expected.  Given the context Trent provided, I'd expect you'd see a bit of a tradeoff finding a fix for this.  Any fix you create will reduce the responsiveness of the UI to new files being added to the directory external to the ORPU.  This may be acceptable in some cases and not in others. "Simply" reducing the polling in this case will add another undesired side effects.  If the tool wasn't designed with network drives in mind, it very easily could be expected/intended behavior without being ideal behavior.  Ideally, it'd work amazing on both network and local drives while still being quick to respond to change.

My comment, when applied as intended, is not accurately depicted by your statement.  The ORPU is not solely taking on that task.  The script I mentioned would make temporary copies locally and remove the bottleneck we see when the ORPU polls the network drive for changes.  The other tasks handled by ORPU continue to process and the frustrating behavior you're seeing is mitigated.  It may cause an initial network hit.  But, it limits the behavior to the start of the application rather than continuous unresponsiveness.  We also mentioned this as being a workaround.  The nature of calling it a workaround means it isn't the ideal implementation but rather something to look at doing in the absence of the ideal solution.

Hopefully the CAR will be looked at and addressed in a timely manner or you'll get a chance to tinker with the source and find a way that works best for your application and group.

[WireWeaver · Accepted solution]

One last note -  I changed this line in in DeserializationConfiguration.cs:

mInboxTimer.Interval = 1000;

recompiled the project (TS 2014), and saw an immediate difference when running ORPU. 15000 seemed usable enough for me. Fair warning - I haven't tested this yet for any other unintended consequences, so consider it more of a hack than a fix.

Assuming there are no other consequences - would exposing this interval through a setting or ini file be good enough for your use case? Edits beyond that start changing some core functionality of this feature, which is possible - but takes up developer time that could be spent improving other features.

[Corey.Rotunno]

Trent,

  Wow, thanks...that's above and beyond. Something along those lines is exactly what I was hoping for. In my use case, refreshing the "outbox" when the interface first loads is probably enough...a button to be able to force a refresh would be icing on the cake. Or as you said, being able to change the interval via an .ini file would also be a good solution for me. Only the polling interval of the outbox is a concern in my use case; the inbox is local and rarely has any files in it...if it does it's 2 or 3 files max. Keeping a frequent polling interval on the inbox would be preferable.

  I greatly appreciate your follow ups and taking the time to look into it.

Thanks again,

[Corey.Rotunno]

Jeff,

  Thanks for the follow up, I understand your point and you are right. From a developer's standpoint the behavior I am seeing is probably the expected behavior. From a user's perspective (mine specifically) I wish it was a little different. But alas, the ORPU was not specifically designed for my use case. Your advice and time spent understanding my issue is appreciated.

Take care, 

[WireWeaver · Accepted solution]

I stumbled across this post again and wanted to mention that this behavior was improved in TestStand 2017 by default. The interval is now dynamically updated based on the time it takes to update.

I'll also mention that depending on what you're trying to do, SystemLink Test Module may be worth looking into. The latest version has a store & forward mechanism that will locally buffer test results until you get a network connection, then upload them.