reading from streaming file

[phmk]

I have a system in which an iMac continuously streams data to a text file (close to 100 MB within 1 hour).  I am using LabVIEW 2020 on a Win PC to read the streaming file (file sharing via SMB enabled on the iMac) while the streaming is going on, search for certain keywords and extract the latest information fitting the search criteria.  The whole operation runs in a while-loop to repeat every 1 to 3 seconds.  When the while-loop executes for the first time, everything works as intended.  On the second and subsequent runs, all I get is the content from the first file read operation.  Any information appended to the streaming file in the meantime doesn't appear at all.  Similarly, using Get File Size in the while-loop continues returning the size as determined in the first iteration.

It doesn't make a difference whether the file open and file close operations are outside the while-loop or inside.

Is this a Windows file buffering or a LabVIEW issue?  And if so, is there a way around it?

Thanks for any help with this,

Paul K

[billko]

The file is open for write by the computer.  You might be able to flush the file buffer to force a disk write before trying to read, but I don't know if that will work, not sure how to do it, and it still involves some trickery to figure out what you already read.

It would be a lot simpler if you could stream the data to your LabVIEW app, instead.

[phmk]

I just wanted to give an update on this issue and bring some closure to it.

First, Bill’s suggestion (to stream directly to my LV app) is not feasible as the app collecting the data and streaming it to file is a non-LV 3^rd party app.

Second, digging through Mac forums, I realized it appears to be essential to state the OS version (Apple v 12.0.1 Monterey).

Third, I was wrong with my initial statement that location of ‘File open’ and ‘File close’ doesn’t matter.  When these operations are inside the while-loop, the while-loop repeatedly returns the same data before, after a delay of at least 10 seconds, a current update of the streaming file content is received.  Digging through forums about the SMB protocol, 10 seconds appear to be the default setting for several SMB protocol buffers that could explain this behavior, for example how long to keep the file contents in memory.  While these SMB server settings can be tweaked on a Windows machine by editing their corresponding registry entries, no such thing is possible on a Mac.

As my application requires updates more often then every 10 seconds at best, I gave up on file sharing via SMB and explored other sharing options on the Mac.  Turns out that ‘Remote login’ on the Mac is essentially a SFTP server (earlier Apple OS had an option to also use FTP, but not 12.0.1 anymore).  So, after upgrading LV, I am now using the SFTP functions in LV 2021 to get at the content of the streaming file.  This finally gives me current updates of the streaming file with each iteration of the while-loop (ruling out a buffer/cache issue with writing the streaming file on the Mac).  The SFTP protocol is noticeably slower than SMB but I can now run my LV app long enough before I have to pause it to reset the streaming file on the Mac.  I am wondering how much of the speed difference between SMB and SFTP is due to the encryption/decryption involved with SFTP.  Maybe one day, when I am in the mood to torture myself, I’ll try again to get a 3^rd party FTP server installed on the Mac.

Paul