ni vision ring acquisition in visual studio? (image* v. img)

[contracultured]

does anyone have an example visual studio c++ solution for doing ring acquisition using ni vision libraries (v.  imaq)?  there is a cvi example out there but if anyone knows of a visual studio c++ version it would be a great help.  seems like if one wants to both use the vision libraries and deliver arrays to other processing (opencv in my case)....it's best to capture image* v. img.  the conversion of image* to array does not seem offhand to be computationally intensive whereas array to image* is a bit hit.

also, if anyone has experience delivering imaq img (pointer to ring acq buffer) to opencv ipl (intel performance primative) array this would also be a great help to me.

i'm much better at and happier with labview but have to dig into text for a moment....

thanks,

steve

[David S.]

Hi Steve -

The CVI example you're referring to is the only one provided by NI at this time.  I also noticed that you're currently working on a service request with another member of my team (Angela).  Because she's already familiar with this issue, I'll let her continue to work with you as an NI representative.  However, this thread will remain open so other members can help you out.  Hopefully some of the gurus on our forums have some example code that they can lend you!

[contracultured]

thanks for joining in, officially i'll follow up with angela but for anyone: 

my goal here overall is to do intel opencv  haar cascade classifier face detection using imaq 1428 and megapixel cameras.   i am downsampling capture for detection then cropping found face region from the full resolution image- resulting in a high res face.  i am also intending to track found faces by programmatically resizing the detection roi once a face is found (thus speeding up processing).  when no face is found in the resized smaller roi, i'll zoom roi back out in steps to full size.    i need to be in visual studio c++  because some of the opencv data structures of opencv will not run in cvi.   i'm at the point of committing myself to recoding one of the cvi image* acq examples to visual studio c++ and doing imagetoarray to load the ipl image array v.  working through getting img into ipl (there are already visual studio examples of low level ring with img)...to decide my path i've summarized my questions below:

there seem to be three options:

1.  ring with imaq loading image* into the buffers, nivision for display and filtering, imagetoarray to deliver images to opencv ipl (cvi example)
2.  ring with ni  vision  that loads  image* into the buffers, nivision for display and filtering, imagtoarry to deliver images to opencv ipl.  (cvi example)
3.  ring with imaq loading img into the buffers.  img loaded into ipl array, opencv for filtering and display (visual studio example)

is there a preference in terms of speed? 

if (1)  i have run the cvi example that  and it works fine, except that my preference is to capture with my 10bit camera and i get skewing of the image (my 1428 card writes the 10bit array as 16bit so i need to create image as 16 bit)..  i just can't seem then to unpack how to get a good image back.  in labview and in c i use imaqcast to change to 8bit with 2 shift bits.  however it seems like the manner in which the buffer is loaded with the image is the issue.  i'm trying to think at a bit level but can't figure it out....i do fine with my 8bit camera, declaring the imaqimagecreate as u8.

if (2)  i have managed to load ipl image and to do acquisition, just wondering if this is as fast as the other options, based on others' experience and knowledge of underlying factors.

if (3)  does anyone have any experience loading  img into ipl (intel performance primitive)  format array for opencv processing?  i'd be missing ni vision(because i do not want to take the hit of doing arraytoimage)  but could probably manage.

* * *
anyone's feedback would be greatly appreciated.  i'll share results for those of you playing with opencv.

steve









steve

[Harold Bien]

Having not worked with OpenCV before but looked into it awhile ago, I can only give some very basic hints/advice that may or may not help.
First, I'm not sure what the exact format is for an OpenCV array, but in C++ via NI-Vision (IMAQ) you can copy the acquired image into a buffer. This will basically be of type pointer to byte(i.e., char), e.g. "char *Buffer" [see their example LLgrab.c or LLsnap.c]. At this point, the image is in raw form and you have to decide what you wish to do with it. OpenCV probably has their own array structures, but I'll bet that it permits import of a raw binary "stream" of sorts. The major performance issue here, if any, would be if OpenCV insists on copying data from your buffer into its own structure.

If this is the case, then you may be better off simply letting OpenCV stream it from the acquisition buffer, assuming that it can do so before the buffer gets over-written. If it takes too much time, you can look into the low-level buffer management routines to lock (imgSessionExamineBuffer?) one buffer while you copy it into the OpenCV structure and then release it back into the ring. You have to make sure you have enough buffers and you operate fast enough to not drop any frames.

I'm also working with a 10-bit digital camera, and the major issue I've noticed is bit alignment. If I specify least significant bit (LSB) alignment, I get weird looking pictures. See if you can control your camera as to bit alignment and see which works for you (MSB worked for me, 10-bit->16-bit).

Hope this helps.

[contracultured]

thanks for your response.  just for your information or others, i investigated the two options.  ring to img where i used imaqExamineBuffer to point to the recently acquired img and a similarly structured ring acquisition where i loaded the ring with image*.  i was able in both cases to deliver these images to the ipl (intel performance primitive) image format for opencv processing.  roughly speaking, the image*
acquisition is about 50% more cpu intensive in my experience with a 1380 x 1018 sized image.  however, with a 10 bit camera i was not able to figure out how to do the casting to 8bits with the img array so i ended up opting to capture to image* and do imaqImageToArray before delivering to OpenCv IPL image.  the advantage of doing this for me is that i am also using the NI Vision libraries so i had everything i might want at my disposal. 

if someone gets into the situation of needing to deliver imaq acquired images to opencv post and i can send a code bit to handle this....

if by chance anyone has detail information on how to take a pointer to imaq acquired image buffer and cast that from 10bit to 8bit that would be helpful in the future.


steve

[Harold Bien]

I can only assume that the "image *" (pointer to type 'image') is some sort of structure used by OpenCV? If that is indeed the case, then it would be rather dangerous to pass that into a ring buffer as the data gets written into it strictly in a binary sense, i.e. it does not invoke OOP/insertion functions. The cast to (void *) alone tells you that you are now meddling with the internal structure of the struct or class itself. If, on the other hand, the "image" is nothing more than a typecast for a standard type, say, "char" or "unsigned char", then you are safe.

Both options should have been identical in CPU utilization - in both cases, IMAQ will directly write to the buffers treating it as nothing more than a simple storage bin of bytes. The fact that you see a 50% increase in using "image *" probably reflects then some OOP process, namely, a function call, being interposed. Not sure how this is done/possible as IMAQ has no idea what an "image *" is. On the other hand, if you are instead using imgSessionExamineBuffer() to lock a buffer, and then copy that buffer vis memcpy() or using a member function of 'image', then that's where the CPU utilization will spike. The OOP methods, unless it's a static function, incur a function call penalty for each access, and this can get extremely busy if you are using accessors/mutators like good OOP practice dictates.

My advice would be to try as much as possible to stream data to buffers which you can then hopefully directly attach to OOP objects. See if 'image' type, if not native, has methods that would permit you to attach a simple (char []) or (unsigned char[] - note that this is pretty much identical in C speak to char * or unsigned char *) to use as it's memory source.

As for "casting" from 10-bit to 8-bit, this is simply not possible. A 10-bit image capture requires 2-bytes per pixel, whereas 8-bit only requires 1. First, assuming you could live with the reduce dynamic range (8-bits permit values from 0..256, 10-bit from 0..1024), you would have to decide what you wish to lop off. Let's say you have dim images, and all values are <=256. If this is the case, then you only need the lower byte. To see why casting is not possible, you would need to read every _other_ byte (since each pixel in your original acquisition required 2 bytes), skipping the "high" byte of the 2-byte word (assuming PC architecture, big endian). Alternatively, you could choose to read the high byte, skipping every other byte but starting at offset 1 instead of 0. Either way, you see it's not trivial to convert from 10-bits to 8-bits, and this is not even considering the loss in dynamic range, e.g. if you had values between, say, 120 through 500 - how would you store that into a byte that only holds 0..256? Even if you did a baseline subtraction (expensive - every pixels needs to get subtracted) you would still end up with values from 0..480.

Your best option would be to find routines that can deal with 16-bit images (there are hard to find, only TIFF/PNG supports 16-bit grayscale for storage, and VS.Net I just found out lets you create objects for them, but you can't save/display/print them). Then it's a trivial matter to "cast" from 10-bit to 16-bit, as you have increased dynamic range (no truncation to worry about) and it's all the same to the system - 2 bytes per pixel. You just need to make sure that the function is expecting 16-bit (2-byte or word sized) values, e.g. short wordArray[]; __int16 wordArray[]; System::Int16 wordArray[]; etc. depending on your platform/compiler/etc.

Good luck.