LabVIEW Development Best Practices Documents

cancel
Showing results for 
Search instead for 
Did you mean: 

STDF (Standard Test Data Format) IO Library


6/6/11 Update

Version 1.1.2 released with a bugfix for reading files that were generated by a big-endian processor.

Overview

Standard Test Data Format (STDF) is a file format developed by Teradyne to be used for logging data on automated test systems. It provides a common format for storage and transmission of all types of data in almost every system design. It is an active standard that has been widely accepted by ATE manufacturers in many industries. The attached library provides a means for both writing and reading STDF files that is conformant with version 4 of the specification.

This Library's Design

The STDF format is designed for minimal size on disk. To achieve this, the file is structured as a linked list of binary records. A minimal configuration is required for the file to be valid, but beyond this "initial sequence", almost any number or order of records can be present. Additionally, each record has lists of required and optional fields; the optional fields may be omitted from any record under certain circumstances.

This arrangement lends itself very nicely to an Object-Oriented design. The three basic elements of the library are the Record class, the STDF Writer class, and the STDF Reader class. The Writer and Reader classes each manage a file and a list of records. When creating an STDF file, records are created, populated with pertinent test data, and stored in a writer. Similarly, when reading a file, the reader scans records from the file on disk and returns them to you.

writer_reader_record.PNG

The STDF Writer, STDF Reader, and Record classes are the basis of the STDF IO library.

To accommodate the 26 different kinds of records defined in the STDF specification, the design is extended using LVOOP Dynamic Dispatching: each type of Record class is defined as a child class. This allows the library's programming interface to dynamically change the data members of a record based on its subtype, without the client (you) having to change the set of VIs and functions being used.

atr_record.PNG hbr_record.PNG

The Class Property Node (a LV 2010 feature) dynamically adapts to the record type it is given.

These design elements are sufficient for use in a single, linear test sequence writing to a single STDF file on disk. However, many test applications operate in parallel or need to manage multiple data files at once. In response to this need, a wrapper API class called the Test Data Manager was created. The Test Data Manager encapsulates the STDF Writer and Record classes, instantiating and juggling them as necessary to provide a simple multi-access, multi-element management interface.

tdm.PNG

The Test Data Manager class provides multi-file management and a multi-access client interface.

The Test Data Manager class is an ESF session, which provides global access via the Obtain method and by-reference handling via the Data Value Reference (DVR) handle that is passed between its method VIs. Now Records can be accessed and modified by multiple parallel processes, and they can be written to one or more files as appropriate.

How to Use This Library

As a client developer, you don't really need to understand any of the underlying design decisions in order to use the STDF library. The end result is a palette of functions that behaves like many file I/O APIs, except that it is designed to manage multiple files and provide access to multiple clients:

  • You can open one or more files for writing using the Add File VI, and you can close them using the Remove File VI.
  • You can create and gain access to data records using the Checkout Record VI, and you can relinquish access using the Checkin Record VI. Records are written to a file using Store Records, and they can be removed from the session using Delete Records.
  • To provide simultaneous access to the API session from multiple processes, the Obtain Session and Release Session VIs are provided in the palette.

stdf_palette.png

The STDF palette provides the VIs and node needed for most test applications.

The File I/O subpalette provides VIs used for lower-level write access and read access. These palettes lack the VIs needed for multiple-access designs (Checkin/Checkout and Obtain/Release). The Records subpalette provides VIs needed to create and manipulate records. The Examples palette provides simple code snippets that demonstrate the use of the various palette functions.

file_io_palette.pngrecords_palette.png

The File I/O and Records palettes provide low-level APIs for managing STDF files.

As NI TestStand is a critical component in most automated test systems, consideration was given to the STDF library's use in that environment, as well. A beneficial artifact of deriving the Test Data Manager class from ESF is that it provides an interface that can be accessed directly from TestStand, if needed. The DVR handle passed between the class's methods is recognized as a simple U32 integer type. When designing a TestStand sequence, the developer has the flexibility of creating a Test Data Manager object and passing its handle to code modules or allowing code modules to access the object using the Obtain and Release VIs.

call_from_ts.PNG

The session's DVR handle is seen as a U32 in TestStand, making it easy to pass between code modules as a parameter.

Requirements

An essential usability feature of this library is its reliance on the Class Property Node to dynamically populate itself with the appropriate data members for each record type. Without this, a palette of over 500 accessor VIs would have to be made accessible to the user! A showstopper bug affecting the Class Property Node was fixed in LabVIEW 2010 SP1; as such, this library requires LV 2010 SP1 (LV 10.0.1) to function.

This library is dependent on the Extensible Session Framework. The ESF package can be downloaded from its NI Community document here.

References

David Staab, CLA
Staff Systems Engineer
National Instruments
Comments
Active Participant crelf Active Participant
Active Participant

wikipedia link-back added from http://en.wikipedia.org/wiki/Standard_Test_Data_Format





Copyright © 2004-2018 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
David S.
NI Employee

Thanks, Chris. I didn't even think of that.

David Staab, CLA
Staff Systems Engineer
National Instruments
Active Participant crelf Active Participant
Active Participant

No worries





Copyright © 2004-2018 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
Active Participant Timmar
Active Participant

This is an impressive suite and I can see how useful it is for what I would call end of line testing.

I have developed a very similar architecture for polymorphicly managing waveform signals with different data types (Analogue, Digital, Communications in both Synchronus and asynchronus).

I had a reasonable look through the stdf specification but didn't find anything on acquired signals.

Have you considered incorporating anything like this into the standard?

iTm - Senior Systems Engineer
uses: LABVIEW 2012 SP1 x86 on Windows 7 x64. cFP, cRIO, PXI-RT
David S.
NI Employee

Hi Timmar -

The STDF standard was written and is still managed by Teradyne, Inc. I don't have any control over what goes into it. If you're asking about this library's feature set, I tend to prefer an atomic model of library construction, with each type of task or data management having a dedicated piece just for that thing. I only want this library to facilitate logging test data and parameters to an STDF file on disk (as well as reading them from an STDF file). I think it'd be too much work to anticipate where the data/params come from, what types they're stored in, and how they're handled in the system.

David Staab, CLA
Staff Systems Engineer
National Instruments
Active Participant Timmar
Active Participant

It looks like I will have a companion set of classes to manage data acquisition and analysis.

I might use the STDF implimentation as a roadmap for some of the more difficult activities and attempt to align as much of the funtionality and naming as possible.

iTm - Senior Systems Engineer
uses: LABVIEW 2012 SP1 x86 on Windows 7 x64. cFP, cRIO, PXI-RT
Active Participant crelf Active Participant
Active Participant

Have you thought about using the TDMS format? There are primative built in to LabVIEW for reading/writing as well as a TDMS file viewer (which is open, so if it doesn't fit your needs you can use it as a template). Also, it's well suited to waveform data, and DAQmx now has stream-to-disk in TDMS format functionality built in.

cheers,

Chris





Copyright © 2004-2018 Christopher G. Relf. Some Rights Reserved. This posting is licensed under a Creative Commons Attribution 2.5 License.
Active Participant Timmar
Active Participant

Thanks Chris, We are already using TDMS,  Not all data is from Daq MX and not all data types are known at coding time.  Most of our Comms Data is asynchronus and comes in the form: Data typeSmiley Very Happyata Length: [Data].  At the moment we have to Manipulate the data into a form that tdms can use.

We have 5 major and 6 minor data types and Much like STDF polymorpic data classes are a great solution to the problem.

iTm - Senior Systems Engineer
uses: LABVIEW 2012 SP1 x86 on Windows 7 x64. cFP, cRIO, PXI-RT
Member Gleichman
Member

"this library requires LV 2010 SP1 (LV 10.0.1) to function"

It looks like LV2010 SP1 is not available until 2/21/2011 (Monday)

Active Participant Jim_Kring Active Participant
Active Participant
David S.
NI Employee

Hi all -

I've uploaded v1.1.1 of the STDF library. After gathering feedback from three users over a few weeks, I identified and fixed several critical bugs. Because of this, I've removed v1.0.0 from the document because it was fundamentally broken.

I also added a few small features that were commonly requested. The full changelist can be viewed in the VI Package installer, but here's a quick list of the added features:

  • Added "INVALID" to record types enum, making this value default. Now when errors occur or a record type is not specified, it will default to "INVALID". (This may break existing code. It did on one of my tests.)
  • Added a "Last Record" indicator to "Read Next Record.vi". Its value is TRUE when the last record in the file is being returned by the VI. The next call into the VI should return an EOF exception from the underlying File I/O calls.
  • Added a "Delete after store" boolean control to "Store Records.vi". This option will remove each record from the Test Data Manager after storing it to disk.

This update is not fully backward-compatible with v1.0.0 because the utility functions were changed. The core functions, however, are backward-compatible.

Please upgrade to v1.1.1 and discontinue using v1.0.0 in your applications!

David Staab, CLA
Staff Systems Engineer
National Instruments
Member lambda89
Member

Hi,

I am new in programming with LabView.

At the moment I'm a student in a company and I should program a STDF Converter.

As an example I have a .txt (later I have to convert a .dat and .ept file) file with should be converted to an .stdf File.

I loaded the "STDF Package" with the VI Package Manager and I tried to understand the functions but it's still too hard for me.

I tried to get knowledge about this package with "learning by doing" but I couldn't get any results.

I also couldn't find any examples how to use the "STDF"-VI's.

You can say I fail at the bases.

My System:

Windows XP

LabView 2011

Country: Germany

Member Rolibaerli
Member

Hi David

i´m working on STDF-Output and use also GDR records.

Inside the GDR record i use :

1) 1 Byte

2) 4 Byte Float

3) String

4) String

stdf_0x00_error.jpg

Between 1) and 2) there is a 0x00 when looking into the output *.std!

stdf_0x00_error_hex.jpg

Do you know where it comes from?

My System

Win7 (64)

Labview 2010 SP1

Country:Germany

Not applicable

Rolibaerli,

Because of the way the GDR is structured, multi-byte numeric data types, when preceded by their GEN_DATA data type byte, will begin on an odd byte.  This library automatically inserts a 0x00 pad field before all U*2, U*4, I*2, I*4, R*4 and R*8 data types to ensure the required even byte alignment of multi-byte numeric data.

Please see page 63 of the STDF spec linked above for further explanation and an example scenario.

--Michael

Member MHn
Member

Is it possible with this library to open an existing STDF-File and append some records to it directly?

Or do I have to read out all records with the reader VIs first and write them including the new records afterwards?

Martin

Member MichaelChaney
Member

Martin,

This library doesn't control the file mark offest.  If you select "open" for Operation input to the Add File VI, the file marker will be at the default location (the beginning of the file in this case).  It looks like the Write Cached Records VI of the Deferred Write class needs to be modified to call the Set File Position Funtion VI to set the file marker to the end of the file before writing the records.

capture.JPG

I've confirmed this works and will see that the fix gets into the next update.

--Michael

Member MHn
Member

Michael,

thank you for your response. I think this will work for appending records at the end of the file.

Do you think it would be possible to add records in the mid of the file, too?

What we want to do is appending additional test results (PTR records) to previously added parts with existing records in the file during a 2nd test run.

Martin

Member MichaelChaney
Member

Martin,

This only works for appending to the end.  If you were to set the file marker to the middle of the file, it would overwrite from that point forward.  The only way to "insert" records would be to read the file out from a midpoint forward to the end, move the file marker back to the insertion point, append the last half of the file to the new records and write the new+end records to file.

Also keep in mind that inserting new records may affect the validity of some result records, such as the MRR.  You may in fact need to update result records that exist at the end of the file, which would require that you rewrite them anyway.

It might be cleaner and easier to just create a new STDF file for subsequent retests.  The MIR record contains a field for a lot retest code to distinguish retests from first pass.

--Michael

Member Meng-Shiang.Lin
Member

I try to save an example code(stdfex_Create Initial Sequence.vi) as another VI without changing anything.

Then I copy this VI to another computer, run this VI and LabVIEW CRASH!

Does anyone have the same problem?

crash.png

Test Environment:

Windows 7 & LabVIEW 2011

VIPM 2011.0.1 (build 1693)

ESF 1.1.0.5

NI STDF IO Library 1.1.2.14

-Adam

Member MichaelChaney
Member

I have not seen this problem or heard of others seeing this problem.  What flavors of Win 7 and LabVIEW are you using, 32-bit or 64-bit?  You mentioned that you are copying the VI from one computer to another.  Do both computers have the identical Test Environment you describe?  My initial thought is that there is something different between the two computers.

--Michael

Member pgans
Member

I actually just ran into this exact problem yesterday also. I was working on my laptop which was running LV 2011 SP1 & Win 7-x64, and transferred it over to a production machine running LV 2011 & Win 7-x86. I'll respond if I find out more.

Member Meng-Shiang.Lin
Member

Michael,

My initial thought is the same as yours. My all computers have the identical test environment as described. This issue can be reproduced on three different computers.

I am using Professional version Windows 7 with SP1 and 32-bit LabVIEW 2011.

-Adam

Member pgans
Member

I was able to quickly solve this problem yesterday by using the "Mass Compile" on the STDF library.

Member MichaelChaney
Member

Unless pgans replies with some additional information, I'm not sure how this issue relatese to this library specifically.  This may be a more general LabVIEW issue exposed by components of this (or the ESF) library....maybe LabVIEW classes.  I would recommend getting in touch with an AE to help diagnose this.

http://www.ni.com/support/

--Michael

Member MichaelChaney
Member

Thanks pgans!   Can you and Adam confirm you are using LabVIEW 2011 SP1 with the f1 patch.  Go to Help >> About LabVIEW... and see if it says Version 11.0.1f1.  My suspicion is that this has to do with something that changed in the f1 patch that recently rolled out.

Member pgans
Member

Hi Michael-

The computer where the code is being developed is 2011 SP1 f1 (11.0.1f1), and the deployment system is running 2011 (11.0).

The program was creating the error during the Property node call.

Also, we are now successfully creating STDF files to our company standard in LabVIEW. This library was a huge help.

-Patrick

Member Meng-Shiang.Lin
Member

Michael,

Thanks for your input.

I upgraded my all systems to LabVIEW 2011 SP1 f1 last night, but I got the same error.

Patrick,

I guess MIR property node is the culprit. I will try your suggestion later.

-Adam

Member Meng-Shiang.Lin
Member

Hi pgans,

Thanks for suggestion and it did work.

Here is my workaround,

1. Develop the VI with LabVIEW 2011 SP1 f1

2. Copy this VI to another computer with LabVIEW 2011 installed

3. Mass compile this VI

4. Run this VI

You can do it reversely, using LabVIEW 2011 for development and LabVIEW 2011 SP1 f1 for deployment.

But I still can not reproduce the above steps between the identical systems, either with LabVIEW 2011 SP1 f1 or LabVIEW 2011.

-Adam

Member evoNge
Member

Hi,

I'm looking into using STDF on our test stations and this library seems to be perfect for our intended use. However, I cannot install the library using VIPM (the Labview package manager). I get the error message telling me that "This package is not compatible with your operating system or any LabVIEW versions installed on your computer".

I'm using Labview 2012 (32bit) on a Windows7 (64bit) computer. I installed the ESF package without problems. Any idea?

Thanks

Member vishots.com
Member

Try copying the VI package to your local hard drive and installing it from there. There was a problem (which I thought was resolved) where VIPM gave this error if the package was installed from a network drive.

Member MichaelChaney
Member

evoHge,

I can confirm that the package works with Labview 2012 (32bit) on Windows7 (64bit).  Give Michael's suggestion a try and let us know if you're still having problems.

Member evoNge
Member

Thank you for replies.

I downloaded it to a local drive the first time, but for some reason it seems like the download was not complete. I got no errors from Internet Explorer when downloading the first time, i just coincidentally noticed that the file size was about 1MB smaller than what is says in this post.

I downloaded once more and made sure the file size was correct and the package installed without problems. Maybe an idea to add a checksum to the vip file format?  

Member evoNge
Member

Hi,

I've been using this library for some days now and so far I think it is very useful and easy to use. Thank you!

The only thing I'm missing so far is the possibility to append existing stdf files. I'm using the basic file I/O blocks since we do not use paralell testing and I would like to keep it as simple and basic as possible. I would therefore like to open and append results to one stdf file as I'm testing parts.

Is there some way of moving the write pointer with the existing blocks? I cannot access the file reference sine it is private to the STDF writer class and I'm reluctant to create a child class since I'm not familiar with Labview OOP and how easy it is to deploy this on multiple testers.

Thanks

Member MichaelChaney
Member

Have a look on page 1 at my reply to Martin on Oct. 10, 2011

Member pgans
Member

I'm having an odd behavior (which I'm assuming is LV related and not specific to this package.)

I'm using the STDF package in TestStand and wrote a wrapper library just to allow for easier use for other engineers in our company not as familiar with LV.

While in TestStand, if I call LV modules and specify a Project Path, the STDF library will not work. If I call LV modules and just access VI's directly (leave blank the Module: Project Path) everything works perfectly fine. I am calling the STDF write records in each of the VI's and am passing the Session handle into and out of the VI.  The error is happening in the "Checkout Record (poly)" VI.

The error that I get is:

Error 1556 occurred at In Place Element Structure in STDF.lvlib:Test Data Manager.lvclass:Checkout Record-PTR.vi->ST-STDF.lvlib:Log Parametric Test.vi->ASK5_0100.lvlib:Test - Sensitivity.vi->ASK5_0100.lvlib:Test - Sensitivity.vi.ProxyCaller

Possible reason(s):

LabVIEW:  The reference is invalid. This error might occur because the reference has been deleted.

Member MichaelChaney
Member

Hi pgans,

Specifying the project for the VI will call the VI in the context of the project.  It's possible to call the same VI (even set to non-reentrant) in both the project context and the standalone context, and have both exist in memory.  LabVIEW uses the project for namespacing and allows multiple instances of the same, non-reentrant VI to exist in memory at the same time.  If you were to specify the project for some, but not all, of your TestStand steps, it's possible to be calling the VIs in different contexts.  For instance, if you created the session in a standalone context (no project specified), and then tried to access the session later in the context of the project (by specifying the project in TestStand), the inplace element structure might be trying to access a session that hasn't been created in its context.

Make sure either all or none of the TestStand steps specify the LabVIEW project, but not a mix of both.

Member pgans
Member

Hi Michael,

Sorry for the late response, but thanks for the information. This makes complete sense.

Member proph
Member

I keep getting the following error and do not know how to handle it given the black boxes of OO design:

ERROR: Exception: (68) in ToRootData::Initialize

          Not enough bytes read from file!

Any suggestions? It looks like i create a file, but that file is always 1kb..

Member MichaelChaney
Member

Make sure you are creating a file in a location that the OS doesn't require admin privileges to write.  Also make sure that once you've added records, you explicitly write those records to file before closing the file.  You can refer to the examples that install with the library.  I'd also make sure you can run those examples.

One handy tool I use just to validate the record structure of an STDF file is a freeware tool called STDF Explorer.  It's lacking features and has a few quirks, but it serves the purpose of seeing what's in the file.

Not applicable

On LabVIEW 2014 there are compile errors with some of the low-level vis.  They want you to set these vis to be re-entrant (they are currently not).  I did that and no more error.  NOt sure if this will affect the behavior of the functions, but it weems to write basic STDF files for me with no issues.  

The error seems to be related to some of the class accessor vis.

Member jasontedf
Member

I thought this would be easy to change my existing test program to stdf. I've written stdf parsers before in C but I can't figure out what I'm doing wrong with this library.  When I drill down I get to ESF blocks so it's probably an install problem. Do you have any additional documentation for using this stdf library? Additional example programs would also be great.

Member Y.Kwon
Member

I want to highlight few steps that I needed to use this library.

1. Download ESF(Extensible Session Framework) library first. If downloaded correctly, ESF.llb was added in the following directory.

   C:\Program Files (x86)\National Instruments\LabVIEW 201x\project\ESF.llb

 

2. Download STDF library.  If downloaded correctly, STDF.llb was added in the following directory.

   C:\Program Files (x86)\National Instruments\LabVIEW 2018\user.lib\STDF

   In function palette, STDF icon was under User Library. 

 

3. In order to use Test Data Manager class(which can allow multiple write access), I needed to make two SubVIs in reentrant. They are inside [STDF.lvlib:Test Data Manager:lvclassSmiley Surprisedbtain Session.vi] and [STDF.lvlib:Test Data Manager:lvclass:Release Session.vi] To get the subVIs, follow the error list.   

 

4. In order to use STDF Reader or Writer classes, refer to examples, stdfex_Read All PTRs.vi or stdfex_Copy Record Data.vi. They work without any modification.

Contributors