Stereo correspondence using already rectified images

[Robst]

Hello:

I have a set of rectified and undisorted stereo images, and want to calculate the disparity map. I wanted to test the Stereo Correspondence VIs, Block Matching and Semi Global Block Matching, but surprinsingly I found that you cannot use them without a valid Stereo session, because the VIs also rectify the images. I think the correspondence VIs should only do the matching, the rectification should be the responsability of another module. This fact limits the usability of the correspondence VIs.

I think that as a workaround, you can generate and load a stereo session file with "neutral" values so that R, T and the Essential and Fundamental matrices have values such that the rectification routines (and maybe the undistortion routines too) don't modify the already rectified images. But the file format used seems to be propietary, so you cannot edit the files.

Is there a way to use the correspondence VIs with already rectified images?

Can somebody, maybe from NI, supply a "neutral" stereo session file?

Thanks in advance for your reply.

[JuDa]

Hi Robst:

Could you share your rectified images to test some "neutral" Stereo Files?? In case this doesn´t work correctly I´ve already contacted the Vision Development Team to check if they can help us with a different solution.

[Robst]

Hello JuDa:

Thank you for your reply. I've attached to this message two images that are famous in the stereo-matching community. They conform the Tsukuba pair. The images are rectified and the ground truth of the discovery is available.

Please let me know how the test goes

Best regards!

[JuDa]

Hi Robst:

After several Tests and talking with the Vision Development Team we found out that currently we cannot separate the Rectification from the Correspondance VIs they are programed together in the Algorithm. However our team has taken great interest in your application and would like to gather more information to provide a workaround and maybe a new VI for new Vision Development Module. Could you help us answering the following questions??

1) General description of the application
2) Camera specs (resolution, monochrome/color?)
3) Computer/controller specs (processor, RAM)
4) If you use the current Correspondance VI Is there distortion or modification in the images with a second rectification?? How was the distortion from the "2nd" rectification measured? Can he provide some sample images of the original rectified images vs. the 2nd rectification so we can see how much distortion is introduced?
5) Is there a particular framerate that they are trying to achieve?
6) How did you generate the rectified images?

With this we can develop a better Vision Development Module for all the users.

Thanks a lot for the information.

[Robst]

Hello JuDa:

Thank you for your reply.  It's a shame that in the current implementation you cannot use separately the correspondence VIs. Below are the answers to the questions you asked me.

1) General description of the application

R: We are working on perceiving the environment using an stereo camera for driver assistance in cars, also in some other robotics applications.

2) Camera specs (resolution, monochrome/color?)

R. We use monochrome cameras with a pixel resolution of 1296 x 966

3) Computer/controller specs (processor, RAM)

R. Currently we are using a computer with a Core i7 processor and 8 GB RAM. For triggering and data acquisition of other sensors, we use a sbRIO.

4) If you use the current Correspondance VI Is there distortion or modification in the images with a second rectification?? How was the distortion from the "2nd" rectification measured? Can he provide some sample images of the original rectified images vs. the 2nd rectification so we can see how much distortion is introduced?

R. Right now I don't have all the equipment, so I cannot generate a new set of images. For the images that I already have, I used a chessboard-like calibration pattern to perform calibration and rectification, so I cannot use them to generate a stereo session to be able to use the Correspondence VIs. I could do some tests when I have with me again all the equipment, and a circles grid pattern.

5) Is there a particular framerate that they are trying to achieve?

R. Right now we are more focused on obtaining good-quality disparity maps than on very high speed processing. Eventually we would like to be able to calculate several disparity maps per second.

6) How did you generate the rectified images?

R. I generated the rectified images using OpenCV's calibration and rectification routines.

Best regards,

[JuDa]

Hi Robst:

Our team is already looking into these functions and are looking for a workaround to achieve this.

Although they are recommending performing the entire operation with the Vision Development Module functions, from the acquisiton, rectification, correct the distortion and Disparity map. The algorithms and functions used will deliver great image quality and Disparity maps. 

If you want when you have all your equipment ready we can perform some test running the entire test with LabVIEW and review the results.

If you have any other questions let me know.

PS. I´ll let you know the answers from our development team.

[lizhuo_lin]

Hi JuDa:

Can you publish the content and format in *.calib file used by stereo vision model? Our team use matlab to calibrate stereo system  arealdy, but the calibration result is not able used in LabVIEW. 

About the calibration, I use too much time to learn how to calibrate my stereo system well. Finally I found some trick to make good calibration, you can read my this post

 http://forums.ni.com/t5/Machine-Vision/How-to-calibrate-a-stereo-vision-system-with-tow-cameras/m-p/...

Does this necessary? If yes, why don`t you add into stereo vision example?

Compute whole disparity image waste time, Could your team develop some function that only find some given points` disparity? In some apply, only few points` 3D coordinates are needed.

Thanks!

[JuDa]

Hi lizhuo_lin:

Can you post the question in a new discussion so we can have better tracking of the questions and we can attend each one properly???

Thankyou