Shiny Object - Vision Challenge

Catherine_Bernier · July 2016

When using vision in robotic applications, the surface finish can be a challenge. More than often, we hear that a too shiny surface will cause the camera to not detect the part or to see inaccurate shapes.

Our Robotiq Camera isn't an exception, but there are ways to work around this issue.

For example, using external lighting and making sure that what is recognized is not related to a flash can help you in locating the shiny part you want.

We plan on doing a video showing how the surface finish of a part can affect the part recognition.

Image: https://us.v-cdn.net/6027406/uploads/editor/57/g6slf0l6y8c0.jpg

Did you ever had challenges with surface finish of a part when using vision systems?

Please share your experiences!

Catherine_Bernier · July 2016

So here is my conclusion:

By teaching a shiny part, the light reflection the camera sees on the part is defined as a feature. Therefore, the software of the camera tries to locate the parts with those "false" features. If the part has the same flash on it, the software can recognize it, but if it differs or even if you choose a similar part but painted (black hinge), it won't recognize it.

The opposite is also true. Teaching the Camera with a painted part can also cause the software to not detect the shiny part with more features due to the light reflection.

As a robotic user using vision, did you ever get some challenges with shiny objects?

Catherine_Bernier · July 2016

I want to invite @Vincent_Paquin and @Nicolas_Lauzier to share their expertise on this matter.

BeachChE · July 2016

I did have a job trying to use a camera system to detect shiny parts that would vary within a group of 35 pieces. The difference was so great at time that the camera would miss over half of the parts. I tried polarized lighting, flash lighting and constant lighting, none of it worked. Finally scrapped the project.

tylermartin · July 2016

We have used an IR camera or a red flash. it lights parts well without overexposing them. cognex sells their camera's with a red flash available to add in place of the white light.

Ryan_Weaver · July 2016

One other way to handle this would be to use a backlight - that way you're looking at the high contrast silhouette of the parts vs. the actual surface. If you use a threshold tool, you should be able to black out enough of the interior pixels regardless of surface finish and be able to look at a consistent outline. Backlighting is not always an option though.

Alexandre_Pare · July 2016

Yes, from experience this can be quite a challenge. Using Infrared lighting or Dome Diffuse lighting helped me make some shinny parts work with vision in past projects. Here are two lighting guides that I like to refer to!
8 tips for optimal machine vision lighting
A practical guide to machine vision lighting

If anyone have some hardware in hand and can try some of these techniques on shinny parts, it would be nice to see picture results!

audreybg · July 2016

I agree with @Ryan_Weaver that the use of a backlight would probably be the easiest solution if it can be used in the application. Infrared lighting is another great option. A book that has helped me a lot in the past to manage all kinds of surface appearances (shinny objects, but also glass/texture/etc): Light: science & magic. Since it explains the reflections, family of angles, etc, it will help you narrow down the selection of lights you should try out and should save you some time. A great book to add to your library

Nicolas_Lauzier · July 2016

Our vision software was designed with simple use cases in mind. Shiny objects may indeed be a challenge and as such may require some precautions. In some cases, our system might not be usable.

One way to improve results is to make sure that there is not reflection during the teaching phase. As such, the system will only learn the features which are always visible. That said, if reflections are too important at run-time, it is possible that the system will not find all parts.

Grady_Turner · August 2016

@Catherine_Bernier @Ryan_Weaver

Cameras I have used with some sort of feature finding function allow a high level of customization with contrasts. So you can effectively tell the camera that you're looking for certain ranges so that it only finds feature boundaries, and not all of the "garbage" from reflections.

Catherine_Bernier · August 2016

@Grady_Turner I agree that to be able to select which parts of the image or adjust the threshold of the detection is a valuable feature and we took great note of it during the training. It's in our backlog so we will keep you updated about this improvement.

Discussion

Shiny Object - Vision Challenge

Comments