Machine Vision

cancel
Showing results for 
Search instead for 
Did you mean: 

How to get 3D coordinates of reflective markers using two cameras?

Hi,

 

I am very new to LabVIEW (in fact to any coding at all) and helping my adviser to get the 3D coordinates of a few reflective markers using two cameras. I am able to read the marker coordinates (x, y) from two cameras simultaneously by processing the data in real-time using codes generated from vision assistant. However, we want to get the depth position by triangulating the markers. I have seen stereo vision doing something similar to this, but I think the stereo vision may not work with our calibration frame (markers) and we don’t need the whole depth image, but only the maker’s z coordinates. I also want to use Region of Interest to mask out other regions that are creating reflections. However, I am not sure if triangulation would work if we select region of interest (as the origin of the camera coordinates would change after selecting ROI). I saw this link http://kwon3d.com/theory/dlt/dlt.html#3d where they used DLT (direct linear transformation) method, but it is too much to code from the beginning. Is there a subVI in LabVIEW or some sort of prewritten code that can be customized? Can anyone please give me some advice on how to solve this problem?

 

Thank you

0 Kudos
Message 1 of 27
(7,210 Views)

Hello,

 

this is indeed stereo vision (two cameras from different viewpoints). So the steps you need to take are exactly the same as for a stereo setup:

 

1. calibrate your system (intrinsic and extrinsic calibration) - intrinsic calibration will correct for optical image center and lens distortion, the extrinsic calibration should give you the baseline distance

 

2. you need to rectify the images and find the pixel location of your markers on both images. When you rectify the images, you can determine the disparity of the same object from both images along the horizontal scanline.  Calculate the disparity (d = x2 - x1).

 

3. Use the equation Z = f*B/d to detemine the distance (where f is the focal lenght in pixels - from intrinsic calibration, B is the baseline in meters and d is disparity in pixels.

 

4. Use projective camera model to determine the X and Y:

 

X = u*Z/f and Y = v*Z/f

 

u, v are coordinates on the reference image, you need to account for the optical center and distortion.

 

If you have LV2012 (i think) or later, you can use the stereo library to calibrate the system, rectify the images and then use the equations above to determine the 3D coordinates.

 

For calibration, you can check here:

 

https://decibel.ni.com/content/blogs/kl3m3n/2013/07/26/stereo-setup-parameters-calculation-and-labvi...

 

Also, there are a lot of resources online...

 

Tell us how it goes.

 

Best regards,

K


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
0 Kudos
Message 2 of 27
(7,185 Views)

Hi Klemen,

 

Thank you so much for your reply. I understand the intrinsic and extrinsic camera parameters but, I am not familiar how to get these parameters of the marker using labview. I used the calibration grid and calibration.vi to calibrate but I didnt understand how should I calibrate using markers so that I can get the focal length(fx, fy) and optical center (cx, cy) of the marker so that I can calculate the X, Y, Z. 

Can you please help me.

 

Thank you

0 Kudos
Message 3 of 27
(7,152 Views)

Hi Klemen,

 

Since I know the (x,y) of the marker wrt the left camera and (x,y) of the marker wrt to the right camera can I just do xleft - xright to get the disparity instead of doing the calibration to get the disparity? 

 

Thank you.

0 Kudos
Message 4 of 27
(7,137 Views)

Hello,

 

you need to do the calibration anyway, because you need the baseline distance (distance between the cameras).

Also your stereo setup is not "ideal". The ideal system makes some assumptions such as: both cameras have the same focal length, they are parallel to each other, etc..

You need to calibrate to consider all this.

 

First, do you have Stereo Library? Second, did you check the link I gave you? There is a  calibration program that will do the stereo calibration for you - including distortion and perspective correction. You just need to acquire some images (I can attach the program for image acquisition using two cameras to the link I provided - tell me if you need it!) in different positions using a dot pattern grid. Can be found in "C:\Program Files (x86)\National Instruments\Vision\Documentation". You need to acquire a couple of images (let's say 6 or 7), making sure that on the first pair of images all the dots on the grid are visible. Then, try to cover as much area as possible.

 

The program also has an option to save the stereo calibration file, so I suggest saving it. After the calibration, you can make a new application, where you can extract the baseline distance, focal distance, and find your marker position (calculate disparity). Since you know the positions of the markers, I think you do not need to rectify the images. Image rectification is helpful when doing block matching based on some metric, so that the 2D correspondence search is reduced to 1D problem. But you would still need to consider distortion and perspective projection. Image rectification (and stereo calibration) in Labview (based from what I see in the documentation) does this for you.

 

Also, make sure that the cameras setup does not change during and after the calibration.

 

Best regards,

K


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
0 Kudos
Message 5 of 27
(7,119 Views)

Hi Klemen,

 

Thank you for the brief description. I acquired some images of the calibration grid using image acquition and then I used them in the calibration.vi to do the calibration, then finally it gave me an option to save the stereo calibration file. I think the calibration file should be saved in the same folder where I had the images right? I was not able to find where the calibration file was saved.

0 Kudos
Message 6 of 27
(7,095 Views)

Hello,

 

the calibration file is in the folder where the "Left Images" and "Right Images" folders are under the name "StereoCalibrationData.calib". You can check the path in the block diagram...

 

What is your calibration quality? Did you remember to modify the spacing between dots according to your setup?

 

Best regards,

K


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
0 Kudos
Message 7 of 27
(7,083 Views)

Hi Kelmen 

 

You said' the extrinsic calibration should give you the baseline distance'  I already get the calibration file,  and there is a snap of Binocular stereo calibration info and my calibration file, my stereo system baseline is 100mm around. 

 

Can you tell me where is the baseline distance in binocular stereo calibration info? 

 

Thank you!

CLD
CAU
Download All
0 Kudos
Message 8 of 27
(7,043 Views)

@lizhuo_lin wrote:

Hi Kelmen 

 

You said' the extrinsic calibration should give you the baseline distance'  I already get the calibration file,  and there is a snap of Binocular stereo calibration info and my calibration file, my stereo system baseline is 100mm around. 

 

Can you tell me where is the baseline distance in binocular stereo calibration info? 

 

Thank you!


Hello,

 

after the calibration, the first element (Tx) in the translation vector should give you the baseline distance. Looking at your file, if I am not mistaken, you are calibrating in units of cm and your baseline is ~12 cm (120 mm)?

 

Best regards,

K


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
Message 9 of 27
(7,036 Views)

Hi Klemen

 

Yes! My baseline is ~12cm, the unit I selected in calibration program is cm.

CLD
CAU
0 Kudos
Message 10 of 27
(7,022 Views)