LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

LabVIEW speed when reading in large binary file

Solved!
Go to solution

I am writing a program that reads in large binary data files and processes the data. I have written the code for reading the binary file in and it works. The problem I am running into is that the program takes ~15 seconds to read the data in (88672 Rows). A co-worker wrote a routine (Not in LabVIEW) and it reads the file in ~0.25 Seconds. 

 

 

I don't come from a computer programming background so my question is this. Am I doing something fundamentally wrong handling the data in LabVIEW, or is there something else in LabVIEW that is causing it to run so much slower?

 

I have attached a PNG of the VI that I am using to read in the file. I cannot attach the binary file I am reading in as it is proprietary. The file contains an ASCII header (281 Bytes), and then there are 88672 rows of 16 bit signed integers. Each row contains 32 values, but I only keep 12 values and delete the rest. 

 

 

Thanks for any help!

0 Kudos
Message 1 of 12
(4,114 Views)
Solution
Accepted by topic author adekruif

You should try to read all the data at once, then use "re-shape array" to get it to look like what you want. That should eliminate the loop altogether.

 

However, if you do have a loop and you know how many times you want it to run (like in your example) you should be using a For loop.

Message 2 of 12
(4,105 Views)

You can try something like this. You don't need "Get File Size" since you know how exactly how much data you there is, but it's there just in case.Capture.PNG

Message 3 of 12
(4,088 Views)

I created my own program reading in the file in one read, and now it reads the entire file in ~10 ms. 

 

In your VI, why use a type cast on the data array when you already know that it is a I16?

 

Thanks for the help!

0 Kudos
Message 4 of 12
(4,064 Views)

I'd have to check the help for "Read Binary File", but that is just so that the remaining number of bytes is the number of bytes that we read. 

 

Otherwise, if you are reading I32, then the number of I32s you read would be the number of bytes divided by 4. I don't know which way has better performance, but I just did the way that looked clearer to me.

0 Kudos
Message 5 of 12
(4,054 Views)

@adekruif wrote:

In your VI, why use a type cast on the data array when you already know that it is a I16?


You shouldn't.  Instead, wire an I32 (based on the snippet) to the data type input on the Read Binary File.  You can also wire a -1 to the length input to read all that is left in the file.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
Message 6 of 12
(4,051 Views)

@Gregory wrote:

Capture.PNG


On mobile so I can't code it up, but the file is can be reduced to a Read From Text File to read the header (281 characters) and then Read From Binary File with -1 for the length and an I32 for the data type.  Also close the file when done.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
Message 7 of 12
(4,042 Views)

Cleaned according to @Crossrulz tips.

Forums read bin cleaned.png

/Y

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 8 of 12
(4,004 Views)

@Yamaeda wrote:

@Cleaned according to @crossrulz tips.

Forums read bin cleaned.png


Now just get rid of the Type Cast and you are there.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
0 Kudos
Message 9 of 12
(3,993 Views)

Oops, missed that! Yes that Type cast does nothing. 🙂

/Y

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 10 of 12
(3,982 Views)