LabVIEW
LabVIEW: Memory is full
Solved!
Hi everyone!
I implemented a full image processing algorithm in LabVIEW using quite a lot of Matlab code imported throughout Mathscript nodes and when I have finished implementing the algorithm I found out that it works fine for a few number of pictures, but runs out of memory if I give him more than 25-30. My intentions are to run it through a database of approximately 1200 pics, so this very small amount that it can handle looks to be a pretty drastical problem for me. How can I make the algorithm to scale better and handle hundreds or even thousands of pictures?
Here you have my top level VI that just simply runs through a folder containing the images and compares every one of them to a reference image. Expected result is to have an array of similarity scores between each image from the folder and this reference one.
I have two subVI's that I figured that they may cause the problem. The Retrieve Properties VI does the most of the job which takes on the image as an input, calls lots of Matlab code from .m files and outputs a binary template and mask for the image. Then the Hamming Distance VI takes on the images from the folder and the template and mask from the reference picture, runs the Retrieve Properties VI for the image from the folder and does some logical operations (XOR, AND) on the templates and masks and outputs a double precision score. This subVI is called for every image in the folder and this is how it looks:
As I mentioned previously, the algorithm works fine for a small number of pictures, but when I give him more than 25-30 I get the error that "Memory is full". I've read some posts on this issue and most of them said to reduce number of front panel elements, arrays, copies of arrays and every large data structure that uses contiguous allocation, so I tried to do so. My only element on the front panel is the array with the final scores, however I cannot manipulate the Matlab code too much as it is not my code, I just have an authorization to use it.
As a final information that could be relevant I should mention the size of the pictures: they are 150x200 grayscale pictures with the size of around 22-25 kB's of data.
Does anyone have an idea how to make this algorithm work on a higher scale of pictures?
1. On your top level VI, use an autoindexing output tunnel instead of building the array on your own. The way you are doing it, you have to reallocate an array with each iteration. With an autoindexing output tunnel, LabVIEW can preallocate the array size since it knows how many times that loop will run. Therefore, only 1 array will be allocated instead of N.
2. It looks like your Matlab code is nothing more than a couple of FOR loops. Why not just implement those in LabVIEW?
When I ask a mechanic to tell me what is wrong with my car I don't just bring him the radio. Giving us the full source (not pictures of the source) will probably help. But as mentioned the most efficient way to do the work, is to have LabVIEW do the work, not have LabVIEW call out to external libraries and do the work. Native G code is best.
Also without the full picture I can't say for sure, but are these properties it is retreiving better read using the native LabVIEW image processing functions, or even IMAQ?
@kemenesendre wrote:
From the point of view of implementing it in LabVIEW I thought that it should work fine by calling it from Matlab code. I mean, why reinvent the wheel if I have the tools for using the existing code?
Yeah I get that point, and in this case it sounds like you have a decent amount of Matlab code, so that probably isn't a feasible option. But if you were on a Matlab forum, and you had Matlab code that was calling ActiveX functions in Excel to do math operations, and returning the result, then I would expect that forum to also say you should probably have implemented it in Matlab, instead of calling out to an external library.
Is it possible to do operations in chunks? Maybe perform part of the Matlab calls, return a result, then feed that into another Matlab call? Without knowing where the memory allocation is taking place, and without knowing where the large amounts of data are being allocated, it is hard to recommend anything with much confidence.
Still based on the code I see I wouldn't expect the number of files being process to matter. You only are taking one double for each image processed. So even if it took 1GB of RAM to process an image, once that image is done being processed, all the memory should be de-allocated, other than the one number which was a result of that image. Basically what I am saying is if it can process 5 images, it should be able to process 5000 (or more) Are you sure it isn't crapping out on one particular image? What happens if you try to just process the one image it fails to process?
To be honest I didn't close any file, because I don't use any open node and about the others I didn't find in the documentation whether they implicitly call close or I should do it. I just read the data from the image with the IMAQ ReadFile function and give it to the IMAQ ImageToArray to create a 2D array of the image content that my Matlab code processes. Here is how it looks withouth the Mathscript node that does the image processing and outputs the template and the mask I use later on.
