Upper limits on TestSockets used with Batch model?

[Bob Rafuse]

What is highest number of TestSockets people have successfully used in
conjuction with the Batch model under TestStand 2.0 or TestStand 3.0?

We often have requirements where we need to test a large number of UUT in
parallel. In TestStand 1.0, we implemented our own mods to TestStand to
allow us to test multiple UUT in parallel with a single sequence execution.
It meant our test steps and libraries had to handle a lot of what TestStand
does automatically for you, but it was the only way we could get acceptable
throughput given the hardware available at the time. Its not uncommon for
us to see configurations of as many as 64 UUT being tested in parallel. For
one configuration, we tested 192 UUT (3 test sequence files each te
sting 64
UUT) in parallel.

However, since hardware is so much faster and cheaper these days, we're
looking to (possibly) move back towards using more stock TestStand
components.

I've tested the Batch model with a simple test sequence file on a P4 2.4GHz
with 128 MB RDRAM running Windows 2000 Pro and TestStand 3.0. For
TestSocket counts up to around 32, things seem acceptable. If I set the
number of TestSockets to 64, things go south. Most of the available RAM
gets chewed up, causing a lot of disk thrashing and (understandably) overall
PC performance becomes sluggish. Even if I disable report generation and
try to minimize the amount of results being logged to the database, the load
on the CPU and memory is understandably high. Higher TestSocket counts
yield increasingly sluggish performance.

Due to budgetary constaints, I do not presently have access to a newer or
more powerful box. So I was wondering if anyone could share their
experiences with using larger numbers of TestSoc
kets with the Batch model.
How many TestSockets did you use? What hardware were you running on? What
throughputs could you achieve?

Any and all input anyone could offer would be greatly appreciated.

---
Bob

[Scott_Richardson]

Bob -
When an operating system creates a thread it preallocates the stack space for it. TestStand internally specifies that the stack space for each thread/execution is 1MB. This allows for a sequence call stack depth of about 300.

I ran a simple test that using TestStand 3.0 and Windows XP on a system that has 512 MB. The system allows for a paging file of up to 1.5GB. When I ran the test the system failed to create threads after about 875 and the paging file for the processes was maxed out.

The best way to get more throughput on your system will be to add more memory. 128MB is the lower limit for a Windows 2000 based system.

I know that you will get much better performance if the executions that you create in TestStand hide their windows, or
minimally that they do not trace. The specify module for sequence call steps have an option to initially hide and disable tracing. You might want to consider this during your testing.

Scott Richardson (NI)