Preallocating memory for images

[Orbital]

I was wondering if anyone could explain why these VIs use different amounts of RAM please? I'm using LV2012 64bit and Windows 7, and watching Labview's memory usage in task manager. The first VI doesn't really seem to be using much at all, which I don't understand. If these codes snippets include the front panel's default values, I would check the No. Images control before you run (may use lots of memory).

Thanks for any help!

[Justin.]

Hey Orbital,

I used the performance monitor (not perfmon) that is built into LabVIEW. You can access this through Tools > Profile > Performance and Memory.
This may be interesting for you to look at. I'll post the results and my educated guess at what is happening to hopefully explain. 

In short (tl;dr), it is the fact that you are copying the image in every loop in the slower performing VI, labview must create an image and then reserve another memory space for the copy, increasing your RAM usage.

When you open the text document you will find the most valuable data listed under "Expanded VI Data"
Here it lists the memory usage and time taken for different functions in each VI and as a whole over the two VIs.

In the "Fast performing VI," the IMAQ Create takes more time to execute than in the slower VI.
I would guess this describes the difference between how your VI looks and how LabVIEW actually executes the code. LabVIEW inherently disregards "useless" code when it executes but will always leave it on the block diagram for the user to understand. 

For example, if you coded "numeric control > +1 > -1 > numeric indicator," when you press run, LabVIEW doesn't actually do that summation (unless you run in highlight execution) it just gives you the same value as there is no point processing that equation.

Following from this argument, I would suggest that when you create an image and then set a new image size, as in the "faster performing VI," LabVIEW just creates an image with those parameters in one movement, rather than creating the image and then changing the image size as the block diagram data flow might suggest. Hence the slightly larger memory usage here.

However, the other parameters in the slow performing VI are what makes it that much harder to process. It takes less memory to create the image, but almost equal memory to copy every image and then you have to dispose of twice the amount of images (the original and copy) all leading up to more memory usage at execution time.

I hope the above makes sense. The fact remains that the slower VI uses more memory because of the copy function, doubling your amount of image. However hopefully my further guess at the reason you get all these values is sensical and maybe gives you more of an insight into LabVIEW.

Thanks Orbital. Performance .txt file linked below.

[BlueCheese]

Because the Vision Image data type is a refnum and the memory is managed inside of the Vision DLL, any memory used for Vision image buffers is not actually counted in LabVIEW's own memory accounting.

Eric

[Orbital]

Hi Justin,

Thanks for your reply. I think I'm even more confused now though.

---

In short (tl;dr), it is the fact that you are copying the image in every loop in the slower performing VI, labview must create an image and then reserve another memory space for the copy, increasing your RAM usage.

---

Presumably, creating a memory space for each copy is the desired behaviour in this case? These are examples of initialisation code that would run first, and then the main code would be placing images quickly into memory. I'm trying to preallocate memory with these examples - I'm not experienced with preallocating but I think I should be doing this.

---

However, the other parameters in the slow performing VI are what makes it that much harder to process. It takes less memory to create the image, but almost equal memory to copy every image and then you have to dispose of twice the amount of images (the original and copy) all leading up to more memory usage at execution time.

---

I don't understand this. I thought the slow VI would create one image ("InitialImage1") at the desired resolution, and then it would create x copies of that image in memory (where x is the number of loop iterations). So the total number of images created is x+1. Why do you end up with "twice the amount of images"? Twice compared to what?

My expectation was that the fast VI would create spaces in memory for x images at the desired resolution. Because of that I thought that the two methods would have near identical memory usage (x vs x+1 images).

Is the (memory) size of an image given by number of pixels multiplied by bits per pixel? If so it seems that the fast VI isn't using nearly enough memory.

Also I don't understand the results of the performance monitor. What are the "blocks"? And why does it show much lower memory usage than task manager? When I used task manager I was looking at the memory usage before and during running of the VI, and looking at the difference.

Thanks for your help.

Edit: I thought I'd explain the motivation for this thread. I've been basing the fast VI on part of the code here:

http://forums.ni.com/t5/LabVIEW/queueing-the-IMAQ-image-data-type/m-p/1110228#M490517

The code in that post isn't complete but I'm going to use it as a starting point for what I want to do. I'm just concerned that because the "Prepare memory for images" loop doesn't seem to use much memory, that it might not be a good way to preallocate, and that maybe my slow VI is a better method. If Windows thinks the memory is free, then that seems to suggest to me that it hasn't been preallocated, but I don't understand enough about the workings to justify one VI over the other at the moment.

[Orbital]

---

@BlueCheese wrote:

Because the Vision Image data type is a refnum and the memory is managed inside of the Vision DLL, any memory used for Vision image buffers is not actually counted in LabVIEW's own memory accounting.

---

Can you explain further? Both VIs use the Vision Image data type, so I don't see how that accounts for any differences in the measured memory usage.

Also, when using task manager, the memory used by "LabVIEW.exe" scales with the number of images I create. Maybe you were talking about using the performance monitor, but I don't understand that at the moment.

Thanks for your help.

[rolfk · Accepted solution]

I think the real reason is that the IMAQ Set Image Size VI doesn't necessarily allocate the image buffer just yet but only defines the dimensions of the image. The buffer is allocated/resized when you actually add real data to the image.

In the second case the image is created with real data and hence will allocate the necessary memory in the IMAQ buffer and then you copy the image a number of times, which of course will create a full copy of the entire data.

In the first case you create an array of images that CAN contain up to x * y pixels but doesn't physically contain those pixels yet. LabVIEW uses many memory optimization tricks throughout, one of them is that data buffers that are empty, are in fact canonically created as a NULL pointer and all the different routines are typically prepared to recognize and treat this as an empty buffer instead of crashing on it.

[Orbital]

Hi Rolf,

Thank you for your reply. Your explanation makes sense to me in the context of what I was seeing when I was doing this work. I'll mark your reply as the answer 🙂

I ended up going with the "copy" method from my original post, to force LabVIEW to allocate memory ahead of acquisition. The rest of the application had two loops (one for image acquisition, one for saving acquired images to disk), and used queues and the pre-allocated images to create a circular buffer.

The above was necessary because I was trying to acquire uncompressed images (though low resolution) at 1 kHz without losing any frames, and the available hard drive wasn't that great. Although I think later on I managed to get hold of some much faster storage.

For some context - I used this software in an experiment on decreasing entropy which was recently published:

Fairly boring video (YouTube) 

Publication 

[Andrey_Dmitriev]

Dear colleagues,

Unfortunately some slightly wrong sentences was written above, so sorry about this. The proper and correct answer, which will explain observed behaviour  is "Demand paging". This is fundamental property of the Operating System and PC architecture.

This statement is partially correct:

---

@rolfk wrote:

I think the real reason is that the IMAQ Set Image Size VI doesn't necessarily allocate the image buffer just yet but only defines the dimensions of the image. The buffer is allocated/resized when you actually add real data to the image.

---

But this happened not at LabVIEW level, but at Operating System level if under "buffer" we will assume physical memory and not virtual memory. There are no special memory optimization tricks (at least not known for me). The data (image) buffers are empty after IMAQ Create as expected, but after calling IMAQ SetImageSize the memory is allocated and the pointer is valid.

Let me try to explain.

The Windows OS implements a virtual memory system based on a flat (linear) address space that provides each process (LabVIEW as well) with the illusion of having its own large, private address space. Virtual memory provides a logical view of memory that might not correspond to its physical layout. Please do not confuse between Physical Memory and Virtual Memory. At run time, the memory manager (with assistance from hardware) translates, or maps, the virtual addresses into physical addresses, where the data is actually stored. Something like that

In our case the physical address space is divided into 4KB pages.

The Windows memory manager uses a demand-paging algorithm to load pages into memory. In demand paging, a page is brought into memory only when a request for it occurs, not in advance. When a reference is made to an address on a page not present in main memory, it is called a page fault. When an application receives a page fault, the memory manager loads into memory the faulted page. The set of pages that a program is actively using, called the Working Set. The amount of pages-backed virtual address space in use, called Commit Size.

When allocating memory with IMAQ Set Size, the memory is allocated in virtual address space, but not loaded into the physical memory immediately. Instead, the memory is placed in RAM only when the application writes to the given memory address using IMAQ Copy (or any other access call, IMAQ FillImage, for example). The process of moving pages into physical memory incurs page faults.

Here is nothing special related to LabVIEW or Vision Development Module memory optimization techniques. No any tricks, lazy or deferred allocation, etc. It is just how the OS works itself.

For better understanding I would like to recommend to proceed to the details page of the Task Manager and enable Commit Size, Working Set and Page Faults columns. Or (better) use Process Explorer from Sysinternals Suite. There are three important things here related to Virtual and Physical memory:

Now I will prepare simple VI, where I'll allocate 256 MB Image and check what happened with memory:

As you can see, immediately after IMAQ SetImageSize the virtual memory was allocated (yes, the memory was allocated) and Commit Size was increased from 273 MB to 536 MB - this is my 256 MB image plus some overhead from LabVIEW environment. But the physical memory was occupied only after IMAQ FillImage. Also you can see roughly 65K Page Faults (because each page is 4KB and ratio 256 MB / 4 KB will give me 65K pages).

If you need even more deeper and detailed overview of the memory, then I would like to recommend to use VMMap tool from SysInternals Suite.

Here you will see the same behavior:

Initial memory snapshot:

The screenshots after MAQ Set Image Size call and after IMAQ FillImage - please see in attachment, because I'm unable to insert here

I hope this explains everything. Trivial malloc (which will HeapAlloc) should work exactly in same way. The explanation above is very basic and there are lot of "fine details", like how translation lookaside buffer (TLB) and memory-management unit (MMU) are involved, but for more detailed explanation I would like to recommend to read two great and awesome books:

1. "Windows Internals, Part 1: System architecture, processes, threads, memory management" from Pavel Yosifovich, Mark Russinovich, David Solomon and Alex Ionescu. (especially Chapter 5)

2. "Modern Operating Systems" from Andrew S. Tanenbaum and Herbert Bos (especially Chapter 3)

Most information provided above just quotes from these books and Wikipedia.

The last small question is "how large penalty caused by page faults"? Not really much and can be easily checked (VI is attached as well)

For my test I will allocate 1000 1MB images (8bit 1024x1024), then fill all with zeroes. So, the time for first IMAQ FillImage call is roughly doubled, but 290 ms vs 147 ms for 1000 images is not really much (the example will allocate 1GB memory). Usually this not caused any bottlenecks, but if you prefer to avoid page faults in critical areas - just "initialize" the images with IMAQ FillImage (which is slightly easier than using IMAQ Copy). But massive continuous page faults should be avoided, of course.

sincerely yours,

Andrey.

PS. Unfortunately I ca't embed more screenshots to the post (looks like bug in Editor), so they just attached to the post. If you have any questions - feel free to ask me.