Version 1.1.2 released with a bugfix for reading files that were generated by a big-endian processor.
Standard Test Data Format (STDF) is a file format developed by Teradyne to be used for logging data on automated test systems. It provides a common format for storage and transmission of all types of data in almost every system design. It is an active standard that has been widely accepted by ATE manufacturers in many industries. The attached library provides a means for both writing and reading STDF files that is conformant with version 4 of the specification.
The STDF format is designed for minimal size on disk. To achieve this, the file is structured as a linked list of binary records. A minimal configuration is required for the file to be valid, but beyond this "initial sequence", almost any number or order of records can be present. Additionally, each record has lists of required and optional fields; the optional fields may be omitted from any record under certain circumstances.
This arrangement lends itself very nicely to an Object-Oriented design. The three basic elements of the library are the Record class, the STDF Writer class, and the STDF Reader class. The Writer and Reader classes each manage a file and a list of records. When creating an STDF file, records are created, populated with pertinent test data, and stored in a writer. Similarly, when reading a file, the reader scans records from the file on disk and returns them to you.
The STDF Writer, STDF Reader, and Record classes are the basis of the STDF IO library.
To accommodate the 26 different kinds of records defined in the STDF specification, the design is extended using LVOOP Dynamic Dispatching: each type of Record class is defined as a child class. This allows the library's programming interface to dynamically change the data members of a record based on its subtype, without the client (you) having to change the set of VIs and functions being used.
The Class Property Node (a LV 2010 feature) dynamically adapts to the record type it is given.
These design elements are sufficient for use in a single, linear test sequence writing to a single STDF file on disk. However, many test applications operate in parallel or need to manage multiple data files at once. In response to this need, a wrapper API class called the Test Data Manager was created. The Test Data Manager encapsulates the STDF Writer and Record classes, instantiating and juggling them as necessary to provide a simple multi-access, multi-element management interface.
The Test Data Manager class provides multi-file management and a multi-access client interface.
The Test Data Manager class is an ESF session, which provides global access via the Obtain method and by-reference handling via the Data Value Reference (DVR) handle that is passed between its method VIs. Now Records can be accessed and modified by multiple parallel processes, and they can be written to one or more files as appropriate.
As a client developer, you don't really need to understand any of the underlying design decisions in order to use the STDF library. The end result is a palette of functions that behaves like many file I/O APIs, except that it is designed to manage multiple files and provide access to multiple clients:
The STDF palette provides the VIs and node needed for most test applications.
The File I/O subpalette provides VIs used for lower-level write access and read access. These palettes lack the VIs needed for multiple-access designs (Checkin/Checkout and Obtain/Release). The Records subpalette provides VIs needed to create and manipulate records. The Examples palette provides simple code snippets that demonstrate the use of the various palette functions.
The File I/O and Records palettes provide low-level APIs for managing STDF files.
As NI TestStand is a critical component in most automated test systems, consideration was given to the STDF library's use in that environment, as well. A beneficial artifact of deriving the Test Data Manager class from ESF is that it provides an interface that can be accessed directly from TestStand, if needed. The DVR handle passed between the class's methods is recognized as a simple U32 integer type. When designing a TestStand sequence, the developer has the flexibility of creating a Test Data Manager object and passing its handle to code modules or allowing code modules to access the object using the Obtain and Release VIs.
The session's DVR handle is seen as a U32 in TestStand, making it easy to pass between code modules as a parameter.
An essential usability feature of this library is its reliance on the Class Property Node to dynamically populate itself with the appropriate data members for each record type. Without this, a palette of over 500 accessor VIs would have to be made accessible to the user! A showstopper bug affecting the Class Property Node was fixed in LabVIEW 2010 SP1; as such, this library requires LV 2010 SP1 (LV 10.0.1) to function.
This library is dependent on the Extensible Session Framework. The ESF package can be downloaded from its NI Community document here.
wikipedia link-back added from http://en.wikipedia.org/wiki/Standard_Test_Data_Format
Thanks, Chris. I didn't even think of that.
No worries
This is an impressive suite and I can see how useful it is for what I would call end of line testing.
I have developed a very similar architecture for polymorphicly managing waveform signals with different data types (Analogue, Digital, Communications in both Synchronus and asynchronus).
I had a reasonable look through the stdf specification but didn't find anything on acquired signals.
Have you considered incorporating anything like this into the standard?
Hi Timmar -
The STDF standard was written and is still managed by Teradyne, Inc. I don't have any control over what goes into it. If you're asking about this library's feature set, I tend to prefer an atomic model of library construction, with each type of task or data management having a dedicated piece just for that thing. I only want this library to facilitate logging test data and parameters to an STDF file on disk (as well as reading them from an STDF file). I think it'd be too much work to anticipate where the data/params come from, what types they're stored in, and how they're handled in the system.
It looks like I will have a companion set of classes to manage data acquisition and analysis.
I might use the STDF implimentation as a roadmap for some of the more difficult activities and attempt to align as much of the funtionality and naming as possible.
Have you thought about using the TDMS format? There are primative built in to LabVIEW for reading/writing as well as a TDMS file viewer (which is open, so if it doesn't fit your needs you can use it as a template). Also, it's well suited to waveform data, and DAQmx now has stream-to-disk in TDMS format functionality built in.
cheers,
Chris
Thanks Chris, We are already using TDMS, Not all data is from Daq MX and not all data types are known at coding time. Most of our Comms Data is asynchronus and comes in the form: Data type:Data Length: [Data]. At the moment we have to Manipulate the data into a form that tdms can use.
We have 5 major and 6 minor data types and Much like STDF polymorpic data classes are a great solution to the problem.
"this library requires LV 2010 SP1 (LV 10.0.1) to function"
It looks like LV2010 SP1 is not available until 2/21/2011 (Monday)
LV2010 SP1 has been spotted in the wild, here: http://digital.ni.com/src.nsf/title/44B20C6F97AD2FD78625783900697619?OpenDocument
Hi all -
I've uploaded v1.1.1 of the STDF library. After gathering feedback from three users over a few weeks, I identified and fixed several critical bugs. Because of this, I've removed v1.0.0 from the document because it was fundamentally broken.
I also added a few small features that were commonly requested. The full changelist can be viewed in the VI Package installer, but here's a quick list of the added features:
This update is not fully backward-compatible with v1.0.0 because the utility functions were changed. The core functions, however, are backward-compatible.
Please upgrade to v1.1.1 and discontinue using v1.0.0 in your applications!
Hi,
I am new in programming with LabView.
At the moment I'm a student in a company and I should program a STDF Converter.
As an example I have a .txt (later I have to convert a .dat and .ept file) file with should be converted to an .stdf File.
I loaded the "STDF Package" with the VI Package Manager and I tried to understand the functions but it's still too hard for me.
I tried to get knowledge about this package with "learning by doing" but I couldn't get any results.
I also couldn't find any examples how to use the "STDF"-VI's.
You can say I fail at the bases.
My System:
Windows XP
LabView 2011
Country: Germany
Hi David
i´m working on STDF-Output and use also GDR records.
Inside the GDR record i use :
1) 1 Byte
2) 4 Byte Float
3) String
4) String
Between 1) and 2) there is a 0x00 when looking into the output *.std!
Do you know where it comes from?
My System
Win7 (64)
Labview 2010 SP1
Country:Germany
Rolibaerli,
Because of the way the GDR is structured, multi-byte numeric data types, when preceded by their GEN_DATA data type byte, will begin on an odd byte. This library automatically inserts a 0x00 pad field before all U*2, U*4, I*2, I*4, R*4 and R*8 data types to ensure the required even byte alignment of multi-byte numeric data.
Please see page 63 of the STDF spec linked above for further explanation and an example scenario.
--Michael
Is it possible with this library to open an existing STDF-File and append some records to it directly?
Or do I have to read out all records with the reader VIs first and write them including the new records afterwards?
Martin
Martin,
This library doesn't control the file mark offest. If you select "open" for Operation input to the Add File VI, the file marker will be at the default location (the beginning of the file in this case). It looks like the Write Cached Records VI of the Deferred Write class needs to be modified to call the Set File Position Funtion VI to set the file marker to the end of the file before writing the records.
I've confirmed this works and will see that the fix gets into the next update.
--Michael
Michael,
thank you for your response. I think this will work for appending records at the end of the file.
Do you think it would be possible to add records in the mid of the file, too?
What we want to do is appending additional test results (PTR records) to previously added parts with existing records in the file during a 2nd test run.
Martin
Martin,
This only works for appending to the end. If you were to set the file marker to the middle of the file, it would overwrite from that point forward. The only way to "insert" records would be to read the file out from a midpoint forward to the end, move the file marker back to the insertion point, append the last half of the file to the new records and write the new+end records to file.
Also keep in mind that inserting new records may affect the validity of some result records, such as the MRR. You may in fact need to update result records that exist at the end of the file, which would require that you rewrite them anyway.
It might be cleaner and easier to just create a new STDF file for subsequent retests. The MIR record contains a field for a lot retest code to distinguish retests from first pass.
--Michael
I try to save an example code(stdfex_Create Initial Sequence.vi) as another VI without changing anything.
Then I copy this VI to another computer, run this VI and LabVIEW CRASH!
Does anyone have the same problem?
Test Environment:
Windows 7 & LabVIEW 2011
VIPM 2011.0.1 (build 1693)
ESF 1.1.0.5
NI STDF IO Library 1.1.2.14
-Adam
I have not seen this problem or heard of others seeing this problem. What flavors of Win 7 and LabVIEW are you using, 32-bit or 64-bit? You mentioned that you are copying the VI from one computer to another. Do both computers have the identical Test Environment you describe? My initial thought is that there is something different between the two computers.
--Michael
I actually just ran into this exact problem yesterday also. I was working on my laptop which was running LV 2011 SP1 & Win 7-x64, and transferred it over to a production machine running LV 2011 & Win 7-x86. I'll respond if I find out more.
Michael,
My initial thought is the same as yours. My all computers have the identical test environment as described. This issue can be reproduced on three different computers.
I am using Professional version Windows 7 with SP1 and 32-bit LabVIEW 2011.
-Adam
I was able to quickly solve this problem yesterday by using the "Mass Compile" on the STDF library.
Unless pgans replies with some additional information, I'm not sure how this issue relatese to this library specifically. This may be a more general LabVIEW issue exposed by components of this (or the ESF) library....maybe LabVIEW classes. I would recommend getting in touch with an AE to help diagnose this.
--Michael
Thanks pgans! Can you and Adam confirm you are using LabVIEW 2011 SP1 with the f1 patch. Go to Help >> About LabVIEW... and see if it says Version 11.0.1f1. My suspicion is that this has to do with something that changed in the f1 patch that recently rolled out.
Hi Michael-
The computer where the code is being developed is 2011 SP1 f1 (11.0.1f1), and the deployment system is running 2011 (11.0).
The program was creating the error during the Property node call.
Also, we are now successfully creating STDF files to our company standard in LabVIEW. This library was a huge help.
-Patrick
Michael,
Thanks for your input.
I upgraded my all systems to LabVIEW 2011 SP1 f1 last night, but I got the same error.
Patrick,
I guess MIR property node is the culprit. I will try your suggestion later.
-Adam
Hi pgans,
Thanks for suggestion and it did work.
Here is my workaround,
1. Develop the VI with LabVIEW 2011 SP1 f1
2. Copy this VI to another computer with LabVIEW 2011 installed
3. Mass compile this VI
4. Run this VI
You can do it reversely, using LabVIEW 2011 for development and LabVIEW 2011 SP1 f1 for deployment.
But I still can not reproduce the above steps between the identical systems, either with LabVIEW 2011 SP1 f1 or LabVIEW 2011.
-Adam
Hi,
I'm looking into using STDF on our test stations and this library seems to be perfect for our intended use. However, I cannot install the library using VIPM (the Labview package manager). I get the error message telling me that "This package is not compatible with your operating system or any LabVIEW versions installed on your computer".
I'm using Labview 2012 (32bit) on a Windows7 (64bit) computer. I installed the ESF package without problems. Any idea?
Thanks
Try copying the VI package to your local hard drive and installing it from there. There was a problem (which I thought was resolved) where VIPM gave this error if the package was installed from a network drive.
evoHge,
I can confirm that the package works with Labview 2012 (32bit) on Windows7 (64bit). Give Michael's suggestion a try and let us know if you're still having problems.
Thank you for replies.
I downloaded it to a local drive the first time, but for some reason it seems like the download was not complete. I got no errors from Internet Explorer when downloading the first time, i just coincidentally noticed that the file size was about 1MB smaller than what is says in this post.
I downloaded once more and made sure the file size was correct and the package installed without problems. Maybe an idea to add a checksum to the vip file format?
Hi,
I've been using this library for some days now and so far I think it is very useful and easy to use. Thank you!
The only thing I'm missing so far is the possibility to append existing stdf files. I'm using the basic file I/O blocks since we do not use paralell testing and I would like to keep it as simple and basic as possible. I would therefore like to open and append results to one stdf file as I'm testing parts.
Is there some way of moving the write pointer with the existing blocks? I cannot access the file reference sine it is private to the STDF writer class and I'm reluctant to create a child class since I'm not familiar with Labview OOP and how easy it is to deploy this on multiple testers.
Thanks
Have a look on page 1 at my reply to Martin on Oct. 10, 2011
I'm having an odd behavior (which I'm assuming is LV related and not specific to this package.)
I'm using the STDF package in TestStand and wrote a wrapper library just to allow for easier use for other engineers in our company not as familiar with LV.
While in TestStand, if I call LV modules and specify a Project Path, the STDF library will not work. If I call LV modules and just access VI's directly (leave blank the Module: Project Path) everything works perfectly fine. I am calling the STDF write records in each of the VI's and am passing the Session handle into and out of the VI. The error is happening in the "Checkout Record (poly)" VI.
The error that I get is:
Error 1556 occurred at In Place Element Structure in STDF.lvlib:Test Data Manager.lvclass:Checkout Record-PTR.vi->ST-STDF.lvlib:Log Parametric Test.vi->ASK5_0100.lvlib:Test - Sensitivity.vi->ASK5_0100.lvlib:Test - Sensitivity.vi.ProxyCaller
Possible reason(s):
LabVIEW: The reference is invalid. This error might occur because the reference has been deleted.
Hi pgans,
Specifying the project for the VI will call the VI in the context of the project. It's possible to call the same VI (even set to non-reentrant) in both the project context and the standalone context, and have both exist in memory. LabVIEW uses the project for namespacing and allows multiple instances of the same, non-reentrant VI to exist in memory at the same time. If you were to specify the project for some, but not all, of your TestStand steps, it's possible to be calling the VIs in different contexts. For instance, if you created the session in a standalone context (no project specified), and then tried to access the session later in the context of the project (by specifying the project in TestStand), the inplace element structure might be trying to access a session that hasn't been created in its context.
Make sure either all or none of the TestStand steps specify the LabVIEW project, but not a mix of both.
Hi Michael,
Sorry for the late response, but thanks for the information. This makes complete sense.
I keep getting the following error and do not know how to handle it given the black boxes of OO design:
ERROR: Exception: (68) in ToRootData::Initialize
Not enough bytes read from file!
Any suggestions? It looks like i create a file, but that file is always 1kb..
Make sure you are creating a file in a location that the OS doesn't require admin privileges to write. Also make sure that once you've added records, you explicitly write those records to file before closing the file. You can refer to the examples that install with the library. I'd also make sure you can run those examples.
One handy tool I use just to validate the record structure of an STDF file is a freeware tool called STDF Explorer. It's lacking features and has a few quirks, but it serves the purpose of seeing what's in the file.
On LabVIEW 2014 there are compile errors with some of the low-level vis. They want you to set these vis to be re-entrant (they are currently not). I did that and no more error. NOt sure if this will affect the behavior of the functions, but it weems to write basic STDF files for me with no issues.
The error seems to be related to some of the class accessor vis.
I thought this would be easy to change my existing test program to stdf. I've written stdf parsers before in C but I can't figure out what I'm doing wrong with this library. When I drill down I get to ESF blocks so it's probably an install problem. Do you have any additional documentation for using this stdf library? Additional example programs would also be great.
I want to highlight few steps that I needed to use this library.
1. Download ESF(Extensible Session Framework) library first. If downloaded correctly, ESF.llb was added in the following directory.
C:\Program Files (x86)\National Instruments\LabVIEW 201x\project\ESF.llb
2. Download STDF library. If downloaded correctly, STDF.llb was added in the following directory.
C:\Program Files (x86)\National Instruments\LabVIEW 2018\user.lib\STDF
In function palette, STDF icon was under User Library.
3. In order to use Test Data Manager class(which can allow multiple write access), I needed to make two SubVIs in reentrant. They are inside [STDF.lvlib:Test Data Manager:lvclass:Obtain Session.vi] and [STDF.lvlib:Test Data Manager:lvclass:Release Session.vi] To get the subVIs, follow the error list.
4. In order to use STDF Reader or Writer classes, refer to examples, stdfex_Read All PTRs.vi or stdfex_Copy Record Data.vi. They work without any modification.
Dear all,
works also with LV2020.
The point (3): two SubVIs in reentrant is really important.
How-to: https://zone.ni.com/reference/en-XX/help/371361R-01/lvconcepts/reentrancy/