RAW image conversion to bmp

[Ash535]

HI,

I am currently in the process of developing a vision system that uses a USB3.0 Camera module with an Aptina CMOS sensor - the image output is RAW but is wrapped in a YUY2 format to make it UVC compliant. So when I use Labview to capture an image I get a greenish image and the output format is recognized as YUY2 and no other options are available. The camera uses Directshow drivers for the capture. I have been struggling to convert this RAW image to a useable RGB32 format image that I can save and do some post processing on.

I have the vendor's souce code and invoke a C++ DLL but it crashes Labview while trying to run the raw to bmp subroutine - I have narrowed it down to an input/output mismatch - the input is a pointer to the image buffer and uses the BYTE (unsigned char) data type. The expected output is a 3D array with 4 x 2592 x 1944 for the 4 color planes R, G, B, Alpha 8 bits each for the RGB32. I have tried pretty much all possible combinations of image handle, array data pointer, C++ string pointer and array options as input for the buffer but unable to get the DLL to work. Any inputs are appreciated here.

Has anyone else encountered this Greenish image when capturing a RAW image and displaying it? Any ideas of how I could do the conversion in Labview?

Some more details of the image - used the Get ImagePxl Pointer to find out more details

2592 x 1944 pixels

 Line width : 2608

Border size - 3 pixels

Pixel size 4 bytes

Transfer max size : 20279808 bytes

So the image buffer is a 4 bytes per pixel

Thanks.

[BlueCheese]

You should be able to do this without a DLL. I'd suggest using IMAQdx's "Get Image Data" to get the raw image buffer without any conversion as a 1D array of U8. Re-shape it to an 2D array of the sensor size. Then call "IMAQ Bayer To RGB" to do the de-bayering.

Eric

[alexderjuengere]

hi,

could you post a sample image?

Regards,
Alex

[Ash535]

Hi Eric, thanks for the suggestion - I had tried that before and i tried it again. Gives me a blue pic if I convert it into a 2D array of the dimensions and change array to Image. Also the Bayer to RGB gives the same image.

The image data is supposed to be RAW 16 bits from the camera but somehow LABVIEW is giving me a 1D array of 4 bytes per pixel instead of two bytes as if it reads that as a YUY2 format and makes it a RGB32 image .

Hi Alex, I have a attachment of the type of image in my first message itself.

Thanks,

Ashwin.

[BlueCheese]

Ah, you are correct, my suggestion won't work. On Linux RT, this would work, but on Windows, the implementation is slightly different. Since it goes through DirectShow there, we present a capture pin to DirectShow to receive the data. Since the camera says it is color/YUV, we present an RGB32 media type and DirectShow does the conversion before it even gets to IMAQdx.

The only workaround I can think would be to try to back-convert from RGB to YUV (assuming the fake YUV data was converted using the reverse of the formula you use), then treat the new YUV data as your raw Bayer image. The problems with this approach are two-fold: you'll first have to guess the exact coefficients and formula used for the original conversion Windows is using. Since these are pretty standard (maybe slightly different depending on gamma correction) maybe this is ok. However, the YUV-to-RGB conversion also clips/saturates the RGB values during the conversion, this may cause saturated values to lose data.

It may be easier to use a different camera that behaves properly... 🙂

[Ash535]

Thanks Eric, that's probably what I was thinking initially that I wasn't having any options to change drivers. Now is there any way I can add a Bayer filter that I have added to my Directshow filters and pass the data through the capture pin before i get it into Labview?

I am stuck with this camera for the form factor. it is a 26 x 26mm camera and not many are around for the application that I am trying to build it for. Any possibility of invoking a C executable inside Labview that will do the capture image and save as bmp outside of Labview? I am relatively new to Labview but have lot of programming experience in Fortran/C++. My choice of Labview is to enable a lot of post image processing in the lines of pattern recognition, differential maps, measurement checks etc.

Ashwin.

[BruceAmmons]

Basler has the acA2500-14uc with the same resolution and 29mm x 29mm for $500 USD.  Does the three extra mm make a difference?  Their cameras work great.

Bruce

[BlueCheese]

Sorry, I don't know of an easy way to inject your own filters in that path. The DirectShow graph if pretty much rendered automatically from IMAQdx's perspective.

Are you sure you can't find a replacement camera? Basler has a Dart model with that same sensor that is pretty close in size:

http://www.baslerweb.com/media/documents/BAS1501_dart_Broschuere_EN_SAP5052_web.pdf

There's also Ace models of similar size...

Eric

[Ash535]

Thanks Bruce, But my max size can only be a 27mm x 27mm max and I already have the hardware ready to go. The camera works great on the manufacturer's software. Hope I can make something work.

[BlueCheese]

Seems like the Dart might fit your max dimensions, but if you're determined to make the software side much harder than I'm not going to stop you 😉 Keep in mind that using a standardized machine vision camera also gives you far greater control of the camera settings than UVC can provide.

As far as interfacing with the vendor SDK or other software in LabVIEW, you certainly can. You can either invoke other executables (and pass arguments/stdin/stdout) or call DLLs. The latter is usually preferred from an overhead perspective while the former can be easier to to isolate issues since they run in different processes, but if that is not critical then you could certainly use either route.