LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Corrupted TDMS Index File - Deletion No Solution

LV Users,

 

I have been doing a large amount of digging for a post that adequately answers the issue of corrupted TDMS index files and have not found a true solution so here's my problem.

I am running tests that create TDMS files in the size range of approximately 75,000-101,000 Kb.

I am regularly (more than 50% of the time) running into TDMS files that will not open. Upon using the Excel Importer, I get a pop-up window with "USI Encountered An Exception (80)".

 

My first solution, from other forum posts, is to attempt to delete the index file (which works approx. 60% of the time).

For the instances in which this does not work, when I attempt to open the TDMS file I get a pop-up window with "USI Encountered an Exception" "TDS Err Not TDS File (-2503)".

When clicking "OK" in that error window, I get an additional error window that shows my full filepath followed by "(TDMS), Code 80".

When clicking "OK" in that error window, I get the original "USI Encountered An Exception (80)" error.

 

In Summation, I am unable to discover what the problem is with my files.

I have some files that are repaired by deletion of the index file and some that are not.

 

Does anyone have a theory on how I can repair this problem? I absolutely must find a way to stop these corruptions, as this LabVIEW testing system was installed specifically to increase reliability of data.

 

P.S. I have not attached any files as they are very large. If you would like to see any files, please advise as to where they should be directed.

 

Thank You!!

 

 

0 Kudos
Message 1 of 17
(4,002 Views)

I have had a lot of issues with TDMS in the past. Could you post the part of the code where you write your data files? Maybe there's something funky going on there.

 

-Seth


@chris8302 wrote:

LV Users,

 

I have been doing a large amount of digging for a post that adequately answers the issue of corrupted TDMS index files and have not found a true solution so here's my problem.

I am running tests that create TDMS files in the size range of approximately 75,000-101,000 Kb.

I am regularly (more than 50% of the time) running into TDMS files that will not open. Upon using the Excel Importer, I get a pop-up window with "USI Encountered An Exception (80)".

 

My first solution, from other forum posts, is to attempt to delete the index file (which works approx. 60% of the time).

For the instances in which this does not work, when I attempt to open the TDMS file I get a pop-up window with "USI Encountered an Exception" "TDS Err Not TDS File (-2503)".

When clicking "OK" in that error window, I get an additional error window that shows my full filepath followed by "(TDMS), Code 80".

When clicking "OK" in that error window, I get the original "USI Encountered An Exception (80)" error.

 

In Summation, I am unable to discover what the problem is with my files.

I have some files that are repaired by deletion of the index file and some that are not.

 

Does anyone have a theory on how I can repair this problem? I absolutely must find a way to stop these corruptions, as this LabVIEW testing system was installed specifically to increase reliability of data.

 

P.S. I have not attached any files as they are very large. If you would like to see any files, please advise as to where they should be directed.

 

Thank You!!

 

 




0 Kudos
Message 2 of 17
(3,995 Views)

It's been a few years since I did much with TDMS, but when I did I was often generating files that were upwards of 100 MB.   I had a smallish number of channels, but logged for up to 30 minutes with a multi-kHz sample rate.  In other words, millions of samples per channel.

 

Either Excel itself (I think we were still stuck corporately at the 2003 version) or the plugin would undoubtedly have choked on that much data.  I'm not sure how many rows the newer versions of Excel allow, but I've never really thought of Excel as a reasonable tool for large datasets.

 

I'm guessing maybe your errors relate to your quantity of data, be it the # of samples (Excel rows) or the # of channels (Excel columns).  Either Excel or the plugin aren't equipped to handle that much.   I suspect that the index file may have nothing to do with the error.

 

I wound up making my own TDMS-to-ASCII converter in LabVIEW to put the data into a more transparent and compatible format that other tools could subsequently parse.  Another workaround would be to change your TDMS writing code to close out your file and start a new one whenever the # samples approaches the limit of Excel and/or the plugin.

 

-Kevin P

CAUTION! New LabVIEW adopters -- it's too late for me, but you *can* save yourself. The new subscription policy for LabVIEW puts NI's hand in your wallet for the rest of your working life. Are you sure you're *that* dedicated to LabVIEW? (Summary of my reasons in this post, part of a voluminous thread of mostly complaints starting here).
0 Kudos
Message 3 of 17
(3,991 Views)

I think Kevin is onto something, another question: Can you open these "corrupt" TDMS files from within LV or does that cause problems as well?

 

Mike...


Certified Professional Instructor
Certified LabVIEW Architect
LabVIEW Champion

"... after all, He's not a tame lion..."

For help with grief and grieving.
0 Kudos
Message 4 of 17
(3,989 Views)

 

This is a picture of the code that ultimately saves the data to TDMS.The VI on the right is, as shown, a subVI of the left VI.

 

This can not, seemingly, be the problem as most all files work fine and even a good percentage of the broken files work again after deletion of the index.

 

This being said, I also do not believe that pre-2007 Excel limits are the problem for the same reason. I do get a warning that not all data can be shown due to Excel limits and it makes me choose parameters regarding where to start displaying data, but I am not worried about the fact that I can not display all data at once in Excel......I can fix this in a variety of ways.

 

What I am worried about is why I keep getting so many corrupt files.

 

 

0 Kudos
Message 5 of 17
(3,985 Views)

Sorry, I guess the pic didn't show.

 

I'll attach it here.

0 Kudos
Message 6 of 17
(3,984 Views)

Kevin,

 

I'm curious about your TDMS--ASCII converter. How did you do that and how are you post processing?

 

Mark,

 

The files do not open in LV either. Actually, when we attempt to open them in our LV "Data Plotter" tool, all of Labview completely crashes.

We have also tried to open the files in DCM_TDMS Data Viewer; they will not open here either.

0 Kudos
Message 7 of 17
(3,981 Views)

Ok, so we know that the files themselves are toast. Any idea if the problem is size related?

 

Mike...


Certified Professional Instructor
Certified LabVIEW Architect
LabVIEW Champion

"... after all, He's not a tame lion..."

For help with grief and grieving.
0 Kudos
Message 8 of 17
(3,976 Views)

Mike,

 

We see no correlation between data size and the problem. Our data sets can exceed pre-2007 Excel limits, but that doesn't seem to be the problem and really we're only typically logging at somewhere between 0.1 - 1.0 Hz with about 260 channels. And we have tried to open the same files in Excel 2007 with no luck as well as through other types of TDMS viewers.

 

When it comes to the files that do not work even after deletion of the index file, there is one strange observation.

We have been seeing index files that are typically about 72% the size of the main TDMS file (ex: Index file size is 73,000Kb and main TDMS file is 101,000Kb)

However, when we find a file that still will not open after we delete the index file, we find that the new index file that is automatically created when we attempt to open the main file is much smaller (like 45%).

 

I have no idea what that means, and I have no way of determining the quality of an index file as the only way I've found to look at one is in notepad and they all seem to look like the same ugly mess in notepad.

0 Kudos
Message 9 of 17
(3,969 Views)

The biggest problem I had with TDMS files is when learing hey are not like most files where I can use the same file ref in prallel thread to write one place and read the same file in another. So Q to you...

 

Does your app write to the same file in more than one place or is all of it done in only one place.

 

How is your disk health?

 

It hard as hell to fix a hardware problem with software.

 

Ben

Retired Senior Automation Systems Architect with Data Science Automation LabVIEW Champion Knight of NI and Prepper LinkedIn Profile YouTube Channel
0 Kudos
Message 10 of 17
(3,966 Views)