I have a Windows machine that makes a TDMS file, then it FTPs it over to the RT system, where it is then read. I'm seeing data now that when read back on the RT, doesn't match the data that was written on Windows. Attached is two VIs, one for making the Windows TDMS file, then you need to copy it over to the RT system, and run the second VI which will read the file, and tell you if the data matches (there is a string constant of the expected result).
What am I doing wrong here? Is there some weird Linux thing in regards to reading certain strings?
You can see the 4th byte doesn't match.
LabVIEW 2020 SP1, RT Linux on cDAQ 9136
EDIT: Okay so this is an extended ascii isn't it? On Windows I write 0x80, and on Linux it reads it as "?". Do I just need to only use the first 7 bits and have a padding routine to only use those? Isn't this still a bug? I mean the TDMS read is supposed to just return the bytes but LabVIEW is changing them.
I tried your TDMS file on a 9627 and got:
Error 6 occurred at TDMS Open in Read back TDMS On RT.vi
LabVIEW: (Hex 0x6) Generic file I/O error.
So, at least it's a bit more upfront about it's inability 😆
You didn't ask but your "write" VI works fine on the 9627
Ugh, okay thanks for trying. I have a set of code that can flatten clusters to TDMS channels. I ended up updating the code to basically not use the 8th bit, but instead add padding. I posted the possible change in the comments here but haven't rolled it into anything official yet. It does add overhead, and it might not be a common use case.
Either this is a bug in how it handles reading extended strings, or I'd ask for a feature that allows me to write an array of bytes to a single sample of a channel.
I think it has to be a bug, at least what I'm seeing on the 9627. Either RT is mistaken and the file is correct and we shouldn't get an error 6 when we read it or Windows is generating a corrupt TDMS.
Interestingly, if I create the file on the RT target and read it back on the Windows side, I reproduce the behavior you saw on the RT side. Windows gives up trying to interpret the euro symbol (I can see it in the hex of the TDMS) and just outputs a question mark (ASCII 63). So, there's an issue in that direction too. Either RT is goofing the representation of the euro symbol or windows is goofing the interpretation of 0x80
I attached my TDMS from RT.
Bumping this this thread again. NI is this a bug? I have a set of code that reproduces the issue. Why is the TDMS read function returning data that isn't what was written?
Sooooo, this suddenly became a problem for me. So, I poked into it a bit.
I wrote a script to write 256 tdms files on a LV-RT target.
Each tdms file writes 1 character type.
I then FTPd these back to Windows and inspected.
I found that the following characters appear to be the troublesome characters:
More specifically, when these characters are written to a TDMS file in RT, and then read back in Windows, Windows returns the character as "?"
It's super weird to me that the troublesome characters don't appear to be the entire extended ascii character set. Just a portion of it.
For maximum optimization purposes, I'm going to try to implement a padding strategy that only pads these characters instead of the full extended ascii set.
All of this work assumes that the issue here is strictly limited to singular "do not use" characters. i.e. I didn't test and patterns of characters.
I suspect that this is a safe assumption.
Lastly, I confirmed that these files all read-back in RT without issue.
So, this just confirms our observation that the issue happens going from Windows to RT, or from RT back to Windows.
I have not confirmed which characters are invalidated going from Windows down to RT.
This is probably a helpful thing to do
I've attached my padding escape strategy for anyone who finds it useful.
I've verified that this appears to work for TDMS files written in RT, FTPd to Windows, and read in Windows.
Again, I haven't tested going from Windows to RT
So I did actually come up with a strategy already and posted it in a thread where this issue was first discovered, I'll have to do a test to see who's is better with my bet being yours using the cached version. It looks like we came up with very similar solutions. Still I made this thread mostly as a question to NI of "Why does it do this, is it a bug?" but NI seems to be somewhat absent from the forums for a while now and maybe opening a support ticket with NI is the only way to get their attention.
One issue is my escape strategy isn't compatible with yours, and I've been using mine for a little while. So if yours does beat mine I'll need to have more version information to make sure my old stuff works and your new stuff does.
EDIT: Well yours uses much less bytes than mine somehow. If speed is even close I'd go with yours.
EDIT2: And yours is faster to...okay update coming soon.