How do I guard against lost data logging to a remote database?

[wendell@fluke.com]

We're currently running approximately 100 test stations, each station will have an instance of TestStand running. Any data loss is unacceptable. Ideally we would like to log test data from each station to a centralized database. The problem: this is a high volume 3 shift a day environment and the network is never operational 100% of the time. How do I log locally to avoid data loss but migrate test data to the centralized Oracle database in the background during those times the network is operational? What are other TestStand users doing to guard against data loss during network outages?

[RayFarmer]

Hi,

You could log locally, and have a central task that sniffs the data from each of the local areas. So it wouldn't matter if the network went down.

Regards
Ray Farmer

[_xs_]

---

@Wendell wrote:
How do I log locally to avoid data loss but migrate test data to the centralized Oracle database in the background during those times the network is operational? What are other TestStand users doing to guard against data loss during network outages?

---

In our company I have used two different methods for collecting test data. On our intranet I have each test system log the results locally. I then have a small helper utility check the network status and transfer these log files into the central database every few minutes. The utility checks the network status first and only starts a transaction if the network is up and running.

For test sytems at our CM, we have them put the log files onto an FTP server and I have a scheduled task pull these files down every morning. I'm still a little concerned about the risk of connecting a test system to the network in China. For us, the risks outweigh the benefits of networking these systems. That may change in a couple of months as our installed base of test systems grows.

I've been reading TestPro News to try to get a better understanding of these issues lately.

Message Edited by Support on 05-26-2010 01:48 PM

[drohacek]

Microsoft Message Queuing (MSMQ) is a very general Microsoft technology that can be used to solve this type of problem. TestStand does not have any MSMQ support built in, so you will have to either go with a 3rd party solution or create your own.

Check out the Microsoft Message Queuing (MSMQ) Center for more information.

[Gemini]

Wendell,

I saw someone mentioning VI Technology's Arendar later in this thread. I would recommend you take a look at it. I do want to mention that I do work for VI Technology.

Two reasons you may be particularl interested in Arendar:

1. Arendar uses Microsoft Message Queuing (MSMQ which was mentioned later in this thread) to ensure network failsafe data transfer.

2. Arendar comes with a TestStand XML file parser so that you can merge/compare/analyze data from multiple stations. There is close to a zero integration effort since the parser not only imports data but also builds the database schema. If the XML file format changes, the importer is intelligent and modifies the schema as required (only additions, no deletions). The schema can be locked at any time.

Message Edited by Support on 05-26-2010 01:48 PM

[ADL]

Wendell,

We provide a Test Data Enterprise Management Appliance, called IntraStage (www.intrastage.com) it is designed to offer a low IT resistance path solution. We provide an appliance (physical box) which you plug into your network. You then point to theTestStand databases using a Wizard Interface, and you are then ready to report through internet explorer without any additional software licences.

Enough of the sales your individual questions are addressed below:

Q. We're currently running approximately 100 test stations, each station will have an instance of TestStand running. Any data loss is unacceptable.

A. The product was designed to suck data from the TestStand Databases when the network is up, network down time has no effect on the data, as SQL transactions ensure the data is only removed from the rigs when the data has been copied to the central storage area.

Q. Ideally we would like to log test data from each station to a centralized database.

A. Thats what we envision our product being used for, and then backup procedures are included at the server location.

Q. The problem: this is a high volume 3 shift a day environment and the network is never operational 100% of the time. How do I log locally to avoid data loss but migrate test data to the centralized Oracle database in the background during those times the network is operational? What are other TestStand users doing to guard against data loss during network outages?

A. As a generic answer you have to do exactly what has been described above. Log locally on each sttion, but have a central server "applaince" pull the data from the rigs.

Good Luck if yu rquire any further information or a demo on IntraStage, please give me a call Andy Long (858) 342 6795