Machine Vision

cancel
Showing results for 
Search instead for 
Did you mean: 

Locating Billiard Balls

Hi.

 

I'm working on mobile robotics project that has to collect randomly scattered billiard balls on a 2x4 meter court. To do it successfully the robot has to locate and identify all the 16 billiard balls. I've tried many different solutions on the Vision Assistant through the last months and I've found one way that works fine since the balls aren't touching each other on the image. The problem is that there's a high probability of the balls being side to side, becoming a big unknown object, during the robot task.

 

To solve this problem I searched and tested the watershed algorithm and the advanced morphology separate objects, both options only cracked the particles where they shouldn't without giving the expected results. 

 

Let me explain some important points:

- Neither the lightning is uniform nor it has the same intensity along the court, due to shadows from the structures on the court and the distance from the point to light sources;

- Using external lightning like a 10 W LED hasn't helped much (pictures below);

- The court (which is the image background) is white like the cue and stripped balls;

- Thresholding the grayscale image didn't work well, so basically I've color thresholded the court (and the white part of the balls) and inversed the resulting binary image to find the balls (except by the cue ball that had to be found with other solution) using binary shape matching. The problem is, as I said before, when balls are in contact with each other.

- The watershed algorithm is not an option because it cracks the image as the binary particles are never perfectly round.

- The vision process is divided in two parts: locating the balls and identifying it. Our problem is on locating it, so all we need is to build something that give us the ROI or the center coordinates of each ball on the picture.

 

Some solutions that I think that would be nice but couldn't make work:

- Using Find Circular Edge to find multiple circular edges on the same image.

- Using Pattern or Geometric Matching.

- Solving it on a RGB or grayscale image.

 

Hope you can help me to find out some way to make it work.

 

Best regards,

Theo

0 Kudos
Message 1 of 27
(3,833 Views)

I would consider using color thresholding to find each color of ball.  Use hue and saturation color planes, not luminance.   For each color, there should be a solid ball and stripe ball.  Each colored ball would be a single blob, unless they are touching.  Erosion would probably take care of that problem.  You know the size and shape of a ball, so that should be fairly easy to fit around the blob you find.  Then all you have to do is find the white cue ball.

 

It looks like using color classification and segmentation might be useful, also.  You have to train each color, then it figures out which one is the closest color to what you have defined.  I think it would quickly highlight each color of ball.

 

Bruce

Bruce Ammons
Ammons Engineering
Message 2 of 27
(3,801 Views)

Bruce,

Thanks a lot for helping.

 

Thresholding each color would reduce the possibility of touching balls to only two balls on the worst case, which would be easy to solve. It's a very nice idea, but the issue here is that each color change from one picture to another and similar colors, like red and orange or like blue and purple can't be thresolded whitout also thresholding other color depending on where the picture is taken.

 

Even though we've already found out a solution to identify the balls, I appreciate your suggestion. Our goal now is to locate the balls position whitout depending on the court lightning. Can we do it based only on the circular shape of the balls?

0 Kudos
Message 3 of 27
(3,783 Views)

Do you have the possibility to paint the course say in green? It would make it easier to separate the background from the balls.

Message 4 of 27
(3,781 Views)

 


ChristopheC escreveu:

Do you have the possibility to paint the course say in green? It would make it easier to separate the background from the balls.


Hi Christophe,

It would be nice if we could, but finding the balls on the white court is exactly the challenge. 

0 Kudos
Message 5 of 27
(3,781 Views)

When I looked at the hue color plane, I got pretty good separation between the colors.  It also doesn't depend on the lighting, since hue is light independent.  Saturation is also important, since that will tell you how green or how blue the ball is.  Probably needed to separate the light and dark blue balls.  Both had pretty good contrast with the background, as well.

 

My instinct would be to come up with a search that uses a circular mask the size of the balls, and also uses the hue and saturation information.  This would come up with best case matches for each ball.  Probably threshold for one color, then locate the two circles that best match, then next color, etc.

 

Bruce

Bruce Ammons
Ammons Engineering
Message 6 of 27
(3,759 Views)

BruceAmmons escreveu:

When I looked at the hue color plane, I got pretty good separation between the colors.  It also doesn't depend on the lighting, since hue is light independent.  Saturation is also important, since that will tell you how green or how blue the ball is.  Probably needed to separate the light and dark blue balls.  Both had pretty good contrast with the background, as well.

 

My instinct would be to come up with a search that uses a circular mask the size of the balls, and also uses the hue and saturation information.  This would come up with best case matches for each ball.  Probably threshold for one color, then locate the two circles that best match, then next color, etc.

 

Bruce


Just to make sure we are talking about the same image processing tool: NI Vision Assistant > Processing Functions: Color > Color Threshold > RGB/HSL/HSV/HSI. When you say Hue and Saturation you're talking about Color Thresholding tool and not about Color Plane Extraction, right?

 

I could also got good separation between colors but only for the yellow and blue balls, all the other colors can't be color thresholded without thresholding other colors depending on the image, e.g. if you calibrate the orange on color threshold based on picture c.png and test the calibrated threshold on a.png you'll see that you thresholded the yellow ball too (and this is not good). The script has to work on all the following images a.png, c.png, d.png and f.png, otherwise I would have to have one calibration for each picture taken (which means more than 30 different setups)

 

Probably I forgot to tell one important thing: Not every photo will have the 16 balls on it, some pictures will probably have no balls, since the robot has to search for randomly scattered billiard balls and this task will require it to acquire more than 30 images.

 

 If you got good results and want to help me even more, please share the Vision Assistant script (code) so I check and learn from it.

Thank you.

0 Kudos
Message 7 of 27
(3,749 Views)

Hi, I'm posting other images taken from the billiard balls whitout the LED lightning. Maybe it can help us.

 

If the probrem could be solved on this condition it'd be much better because of the project efficiency when it comes to battery power.

 

🙂

0 Kudos
Message 8 of 27
(3,668 Views)

The color classification tools might help you figure out how to threshold the individual colors.  It worked very well to identify the colors, but I didn't try it on multiple images.  If your lighting is fixed, the balls should be the same color in every image.  You would want to ignore the intensity values, but the hue and saturation for each color should be pretty consistent.

 

You could also define the white ball and the background as two different colors, which might help locate the white ball after finding all the others.

 

Bruce

Bruce Ammons
Ammons Engineering
Message 9 of 27
(3,660 Views)

BruceAmmons escreveu:

The color classification tools might help you figure out how to threshold the individual colors.  It worked very well to identify the colors, but I didn't try it on multiple images.  If your lighting is fixed, the balls should be the same color in every image.  You would want to ignore the intensity values, but the hue and saturation for each color should be pretty consistent.

 

You could also define the white ball and the background as two different colors, which might help locate the white ball after finding all the others.

 

Bruce


I'll check the color classification tools to see if there is something that can help. 

 

The issue here is exactly this, the lighting changes depending on which place of the court the balls are. This makes Hue a Saturation change a lot from one picture to another, making it impossible to threshold a single color on all the images.

 

Unfortunately it's not allowed to change the background color.

 

Anyway, thank you for helping me.

0 Kudos
Message 10 of 27
(3,651 Views)