Matching memory image line alignments between IMAQ Images and non-IMAQ cameras—bytes and borders

[vekkuli]

I recently ran into the subject of different alignment blocks for storing images in the memory. In NI Vision you get involved if you happen to shove in the IMAQ GetImagePixelPtr VI or in case you read about it in, for example, the IMAQ Vision Concepts Manual. In LabVIEW it appears in terms of LineWidth(Pixels) and Pixel Size (Bytes)—the IDS uEye camera manual talks about pitch and elsewhere, such as .NET Bitmap, it might appear as stride. I'm sure most of you have been familiar with the topic already for a good time. My question then is about dealing with the alignment between IMAQ Image format and non-IMAQ cameras when they don't match by default, and image border.

Right, feel free to correct me and add, I most certainly haven't got everything right to the detail. For all I know, the alignment is decided by what we want the pitch, or the stride, to be divisible by. In IMAQ we get the stride by multiplying the LineWidth(Pixels) with the Pixel Size (Bytes) giving us the total number of bytes reserved for each image scan line. In the Vision Concepts manual it is written "The alignment regions ensure that the first pixel of the image is 8-byte aligned in memory" and "The horizontal resolution and line width may be the same length if the horizontal resolution is a multiple of 8 bytes and the border size is 0". Perhaps I get something entirely wrong but the stride always seems to be a multiple of 64, not 8. And we're still talking about bytes, not bits; it would make more sense if it was a multiple of 64-bits in the 64-bit OS. I don't know what's going on here but the bottom line is that the IMAQ pitch is a multiple of 64. My camera drivers, on the other hand, go for a 4-byte alignment, i.e. the pitch is a multiple of 4. For me, this all would add up if, sort of, the camera was going for the 32-bit packing and IMAQ for the 64-bit packing. The drivers are supposedly 64-bit, though, as well.

None of this would really matter at all if I didn't care about the performance so much. I don't have any problems gettings images one way or another, say, copying the camera images into another array, reshaping and using IMAQ ArrayToImage or ArrayToColorImage, but that stuff is just a massive waste of memory and CPU time. I'd rather find a smarter way. Using the GetImagePixelPtr we can get the pointer to the IMAQ image data which allows us to directly insert the data from the camera memory slot into the IMAQ Image container omitting buffer allocations. In my case, as I use the camera .NET interface, I create a System.IntPtr from the IMAQ Image pointer and pass that to the camera driver as the copy destination.

If you use IMAQ Image border size 0 and the pitches happen to match, the above works just fine without any extra tinkering—and sweet lord of darkness, embrace the speed. However, if the pitches don't match, or if you want to include the border, you run into problems with the alignment. Here's where I am not sure anymore what would be the right way to go. I've learned, at least, that I can force the camera driver memory allocation to match the IMAQ one by giving it a "false", larger, image width. It still seems to jump the lines approriately, I don't know why but hell, I'm not complaining. I'm not sure yet if this would be a universal solution. The other thing is that since I want to use the NI Vision functions I'm going to need that border. I can't figure out if there's anyway to match the alignments when there's a border. Should I then settle to create two IMAQ containers for each image: one without the border to receive the camera images quickly and one with the border to simply get the data from the other one. Would this work smoothly? Any better ideas?

I think this is enough for now. If anyone joins in for the discussion, I'd be happy to expand.

[GuenterMueller]

---

@vekkuli wrote:

Should I then settle to create two IMAQ containers for each image: one without the border to receive the camera images quickly and one with the border to simply get the data from the other one.

---

Have you tried setting the border size to something different than zero after the image acquisition using "IMAQ ImageBorderSize VI"?  I have never used it, but for what other reason should it be there?

[Andrey_Dmitriev]

If you looking for most efficient way for transfer image data from camera to IMAQ I would like to suggest to write wrapper function which will perform aligned copy.

Refer to example http://zone.ni.com/reference/en-XX/help/370281P-01/imaqvision/imaq_getimagepixelptr_example/

Let's say Width and Height - size of your image, CamImagePtr is pointer to image from camera, LVLineWidth - is IMAQ Image width (which is aligned to 64 byte boundaries) and CamLineWidth is from cam which is aligned to 4 bytes, then copy function will looks like that (assumed 8 bit image):

for (int y=0; y<Height; ++y) {
    memcpy (LVImagePtr + y * LVLineWidth, (char*)CamImagePtr + y * CamLineWidth, Width);
}

[vekkuli]

---

@GuenterMueller wrote:

Have you tried setting the border size to something different than zero after the image acquisition using "IMAQ ImageBorderSize VI"?  I have never used it, but for what other reason should it be there?

---

Good that you noticed. I thought about it, I haven't tried it yet—and it might actually work—but here's what I believe might be the issue: images with different border sizes have an entirely different structure in the memory container which means that changing the border size with the IMAQ ImageBorderSize VI would probably have to go through the memory manager for reallocation, and even if the IMAQ Image reference is appropriately updated, the pointer given to the camera driver becomes invalid.

[GuenterMueller]

---

@vekkuli wrote:

---

@GuenterMueller wrote:

Have you tried setting the border size to something different than zero after the image acquisition using "IMAQ ImageBorderSize VI"?  I have never used it, but for what other reason should it be there?

---

Good that you noticed. I thought about it, I haven't tried it yet—and it might actually work—but here's what I believe might be the issue: images with different border sizes have an entirely different structure in the memory container which means that changing the border size with the IMAQ ImageBorderSize VI would probably have to go through the memory manager for reallocation, and even if the IMAQ Image reference is appropriately updated, the pointer given to the camera driver becomes invalid.

---

Right you are. That why I said: "Have you tried setting the border size to something different than zero after the image acquisition".  Don't set it to something different than zero before doing the IMAQ.

[vekkuli]

---

@GuenterMueller wrote:

Right you are. That why I said: "Have you tried setting the border size to something different than zero after the image acquisition".  Don't set it to something different than zero before doing the IMAQ.

---

Just to get rid of any confusion: we have the allocated memory for the camera transfer, then we have another container for the IMAQ Image and its pointer into which the camera block memory is copied, similarly as in Andrey_Dmitriev's response. If I change the border size on this IMAQ Image, even if it was after, it will invalidate the pointer for the next cycle. Did you mean that I should make a copy first? Because then I might as well preallocate that copy with the border earlier.

[vekkuli]

---

@Andrey_Dmitriev wrote:

If you looking for most efficient way for transfer image data from camera to IMAQ I would like to suggest to write wrapper function which will perform aligned copy.

Refer to example http://zone.ni.com/reference/en-XX/help/370281P-01/imaqvision/imaq_getimagepixelptr_example/

Let's say Width and Height - size of your image, CamImagePtr is pointer to image from camera, LVLineWidth - is IMAQ Image width (which is aligned to 64 byte boundaries) and CamLineWidth is from cam which is aligned to 4 bytes, then copy function will looks like that (assumed 8 bit image):

for (int y=0; y<Height; ++y) {
    memcpy (LVImagePtr + y * LVLineWidth, (char*)CamImagePtr + y * CamLineWidth, Width);
}

---

If my alignment trick doesn't work with all the images, I will certainly try this out. I never used the LabVIEW C stuff, though, and so far I have tried to go without. I usually only go for those nodes with some C++ libraries. In any case, I believe that the above bit of code only works if both images have the same border, as in the example the "C image" is created with a border of 3 ( testImage = imaqCreateImage (IMAQ_IMAGE_U8, 3); ) just like the LV image. Or is it possible to copy directly into an image with a border? Perhaps I'm not seeing it right.

[Andrey_Dmitriev]

---

vekkuli wrote:

If my alignment trick doesn't work with all the images, I will certainly try this out. I never used the LabVIEW C stuff, though, and so far I have tried to go without.

---

Here is small snippet, how to do this without C stuff:

http://forums.ni.com/t5/LabVIEW/VI-IMAQ-Create-wenn-ich-die-Bordergr%C3%B6%C3%9Fe-ungleich-0-eingebe...

Andrey.

[vekkuli]

---

@Andrey_Dmitriev wrote:

---

vekkuli wrote:

If my alignment trick doesn't work with all the images, I will certainly try this out. I never used the LabVIEW C stuff, though, and so far I have tried to go without.

---

Here is small snippet, how to do this without C stuff:

http://forums.ni.com/t5/LabVIEW/VI-IMAQ-Create-wenn-ich-die-Bordergr%C3%B6%C3%9Fe-ungleich-0-eingebe...

Andrey.

---

Andrey, thank you, much appreciated! I wonder why I didn't run into this thread earlier. It really takes care of the border problems as well? If I get it right it computes the alignment for each line when copying. I'm sure to give it a try.

[BlueCheese]

---

@vekkuli wrote:

Right, feel free to correct me and add, I most certainly haven't got everything right to the detail. For all I know, the alignment is decided by what we want the pitch, or the stride, to be divisible by. In IMAQ we get the stride by multiplying the LineWidth(Pixels) with the Pixel Size (Bytes) giving us the total number of bytes reserved for each image scan line. In the Vision Concepts manual it is written "The alignment regions ensure that the first pixel of the image is 8-byte aligned in memory" and "The horizontal resolution and line width may be the same length if the horizontal resolution is a multiple of 8 bytes and the border size is 0". Perhaps I get something entirely wrong but the stride always seems to be a multiple of 64, not 8. And we're still talking about bytes, not bits; it would make more sense if it was a multiple of 64-bits in the 64-bit OS. I don't know what's going on here but the bottom line is that the IMAQ pitch is a multiple of 64. My camera drivers, on the other hand, go for a 4-byte alignment, i.e. the pitch is a multiple of 4. For me, this all would add up if, sort of, the camera was going for the 32-bit packing and IMAQ for the 64-bit packing. The drivers are supposedly 64-bit, though, as well.

---

The line alignment has nothing to do with the bitness. I think the Concepts Manual is likely out-of-date, as the alignment is neither 8-bytes anymore nor constant between different platforms. The various items that cause this alignment to be certain values include:

- Cache line alignment (64-bytes on x86/x64). This is critical to high-speed DMA transfer performance of framegrabbers as well as random access to different lines by the CPU. Also, if cache lines were shared between lines then it can  affect things like parallel processing if different CPUs are working on different lines.

- Vectorized instructions like SSE require larger alignment such as 16-bytes to get optimum performance. Forcing the lines to be aligned makes operations significantly faster than using unaligned instructions.

Eric