How to find moving object/target displacement (how much X, how much Y) using a USB camera in an acquired set of images?

[AmarMakana]

Hi,

       I am trying to do this for one of my projects. I need help for the Image Acquisition and Analysis part.

I have a SCORBOT and I am trying to guide it with the help of a USB camera mounted on top of it (gripper) using LabVIEW. Here the camera will be acquiring set of images continuously, and will be looking for a red object (imagine a red circle on a white paper) it has to pick/point. I have to see the displacement (which I will be changing manually) of the red object in each frame or for every set of frames and correspondingly drive motors of the SCORBOT (in X, Y directions) through NI DAQ 6216. I have found a way to send voltage signals to the SCORBOT through DAQ device, but I am not sure how to start or proceed in the Image Processing part.

It is like I have to give the change in the red object's position as a feedback to my robot and make it reach that position.  I can start off even with an algorithm if someone can guide me. Thanks. 

My webcam is from Logitech and is being shown under NI IMAQdx devices in my Measurement and Automation Explorer.

Amar.

[A.Rohde]

I would do the following: All steps have LabVIEW functions for them if you have the Vision Developement Module.

1) A color camera gives you a red a green and a blue image. Look at the red image channel.

2) Do a threshold so you only find the strongest red color (you now have a black and white image, representing red or not red)

3) Do a labeling so you get an array of objects

4) Find the object of the largets size

5) Find this objects centers of mass/ center of gravity (CoG).

If you do 1-5 in each frame then you can calculate: Current CoG - Last CoG to get the movement in the (x,y) direction in image coordinates.

If you need to get it in the robots frame then you need to create a coordinate transformation between camera coordinate system and the robots coordinate system. But lets deal with the algorithm first and look at the coordinate transformation second. Let me know if you have problems with any of the 5 above steps.

Best Regards

Anders Rohde

[petitOhu]

@A.Rohde : thank you for your answer, this must work, with only one change. Do not use the red channel if the target is red on white, because this two color admit a high red value. But green or blue should work.

Regards,

[A.Rohde]

petitOhu --> You are right. You have to check that green and blue channel is not high at the same time.

Simpel solution would just be to use the IMAQ ColorThreshold .vi this deals with this and does step (1+2) in one step.

Best Regards,

Anders Rohde

[AmarMakana]

Mr. Anders Rohde,

 Thanks for the step by step algorithm. I will try that and come back with questions. 

Amar.

[AmarMakana]

Another question is how do we channalize or get the RGB components in channels? Is it through property nodes? Can you show me an example?

Amar.

[A.Rohde]

Hi Amar

Please see attached VI for a demonstration of the basic elements of the Vision Developement Module.

Requires LabVIEW 2011 and Vision Developement Module to be installed.

I haven't used much time on styling either the Front Panel og Block Diagram but I hope this guides you in the right direction on how to find all red objects, and end up with the center of mass of the largest object.

VI tested vi the attached Color Identification.vi. Note that for a noisy image you would normally use some dialation/erosion operations on the binary image.

Best Regards,

Anders Rohde

Applications Engineer

National Instruments Denmark

[AmarMakana]

Thanks Mr. Rohde,

                 Can you please down convert the vi. to 2010 SP1. I have everything with this version. 

Also, I programmed a vi following your algorithm. I will post it too, but it is for continuous buffering. I am looking to use a sequence and extract frames from it and perform the same. Next I can try to do it continuously probably on a sequence of two frames to know the co-ordinate change.

Amar.

[AmarMakana]

Hi,

        I got the down converted vi. from the forums. But I am not able to open the vi. still. It says Load Error Code 3, Unable to open front panel.

Amar.

[AmarMakana]

Can you please take a look at this vi. I made it following your algorithm. I am stuck at labelling actually. After labeling, how to get the largest area object in the image? And centroid and center of gravity or center of mass. Are these one and the same? I did something with centroid in the vi. Please take a look.

Thanks.

Amar.