05-14-2018 03:16 PM
I am writing a program that reads in large binary data files and processes the data. I have written the code for reading the binary file in and it works. The problem I am running into is that the program takes ~15 seconds to read the data in (88672 Rows). A co-worker wrote a routine (Not in LabVIEW) and it reads the file in ~0.25 Seconds.
I don't come from a computer programming background so my question is this. Am I doing something fundamentally wrong handling the data in LabVIEW, or is there something else in LabVIEW that is causing it to run so much slower?
I have attached a PNG of the VI that I am using to read in the file. I cannot attach the binary file I am reading in as it is proprietary. The file contains an ASCII header (281 Bytes), and then there are 88672 rows of 16 bit signed integers. Each row contains 32 values, but I only keep 12 values and delete the rest.
Thanks for any help!
Solved! Go to Solution.
05-14-2018 03:33 PM
You should try to read all the data at once, then use "re-shape array" to get it to look like what you want. That should eliminate the loop altogether.
However, if you do have a loop and you know how many times you want it to run (like in your example) you should be using a For loop.
05-14-2018 03:56 PM
You can try something like this. You don't need "Get File Size" since you know how exactly how much data you there is, but it's there just in case.
05-14-2018 05:16 PM
I created my own program reading in the file in one read, and now it reads the entire file in ~10 ms.
In your VI, why use a type cast on the data array when you already know that it is a I16?
Thanks for the help!
05-14-2018 05:28 PM
I'd have to check the help for "Read Binary File", but that is just so that the remaining number of bytes is the number of bytes that we read.
Otherwise, if you are reading I32, then the number of I32s you read would be the number of bytes divided by 4. I don't know which way has better performance, but I just did the way that looked clearer to me.
05-14-2018 05:33 PM - edited 05-14-2018 05:37 PM
@adekruif wrote:In your VI, why use a type cast on the data array when you already know that it is a I16?
You shouldn't. Instead, wire an I32 (based on the snippet) to the data type input on the Read Binary File. You can also wire a -1 to the length input to read all that is left in the file.
05-14-2018 05:51 PM
@Gregory wrote:
On mobile so I can't code it up, but the file is can be reduced to a Read From Text File to read the header (281 characters) and then Read From Binary File with -1 for the length and an I32 for the data type. Also close the file when done.
05-15-2018 02:53 AM
Cleaned according to @Crossrulz tips.
/Y
05-15-2018 03:57 AM
@Yamaeda wrote:
@Cleaned according to @crossrulz tips.
Now just get rid of the Type Cast and you are there.
05-15-2018 04:56 AM
Oops, missed that! Yes that Type cast does nothing. 🙂
/Y