Lights, camera, … and then reflections on the surface of your shiny objects! Suddenly your flawless robot vision setup can't detect objects. How do you deal with these annoying lighting problems? Here are ten great solutions.
The Robotiq Wrist Camera URCap version 1.7 is now the fastest and most intuitive vision system for Universal Robots.
Shiny objects are possibly the most challenging objects for a robot vision system. All it takes is one annoying reflection on the surface of the material and the reliability of the entire robot's task can be compromised.
Robot vision (and the more general "computer vision") is all about lighting. Whatever type of sensor you are using, the lighting will have a huge effect on whether or not it will be possible to detect the objects. Pick the wrong type of light and you could end up not being able to detect anything at all.
This is why shiny materials can be such trouble. The lighting you choose can completely change the appearance of the objects. This whitepaper from Microscan shows how a shiny ball bearing looks entirely different under 7 different lighting conditions. As Robotiq's Catherine Bernier explains, light reflections are seen by the robot's vision system as "features" (i.e. things that should be detected) but they aren't real physical features so they confuse the whole detection process.
How can we overcome these challenges and improve the detection of shiny objects?
Thankfully, there are some great solutions out there. Here are 10 solutions to improve robot vision with shiny objects.
A backlight is one which sits underneath the object. We've used them before in our trade fair demos as they are a good way to control the light when the ambient lighting is unpredictable. They provide the vision sensor with a very clear outline of the object, which will show up as dark in the image.
Backlights are particularly useful with shiny objects because the light is coming from behind the object. This reduces chance of reflections on the object's surface.
Not all frequencies of light will interact with a surface in the same way. For example, ultraviolet (UV) light has a short wavelength which means that it will reflect strongly off the material's surface. Infra-red (IR) light, on the other hand, has a long wavelength and so will reflect less strongly. With some materials, it's even possible to "look through" the material with IR light. It is for this reason that IR light is often used for inspection applications.
Of course, you don't need to change the type of light completely. Sometimes you can reduce the effects of light reflections by simply changing the color of the lighting. For example, visible-spectrum red light will interact less strongly with shiny surfaces than white light. An easy way to change lighting color is to cover the light with a colored "gel" (a thin, transparent film).
Changing the light color can also improve detection in other ways. For example, if your robot vision system uses monochrome images for detection, as many systems do, adding a blue light will make blue features lighter in the monochrome image and orange features darker. This effect happens because orange is the opposite color to blue on a color-wheel.
Normal white light contains light waves which are oriented in many different planes. Polarization is a process by which only light waves in one plane are allowed to pass through a polarizing filter.
Polarized light is often used in computer vision applications as shining a polarized light onto the shiny surface can reduce the reflections. However, polarized light is not always the best solution. You have to precisely position the polarized light or the reflections can get much worse.
If your vision system uses strobed lighting, this can have an effect on the reflections from shiny objects. Sometimes flashed light can reduce the reflections by "overwhelming" the glare caused by ambient lighting. Other times, the flash can make the reflections worse.
The solution is to test out your own setup and see whether changing the properties of the flash solves your problem.
Often, a reflection on the surface of a shiny object is caused by a single light source — e.g. a strip-light in the ceiling, a window, etc. It can seem a strange idea to add more light sources in an attempt to reduce the problem, but this is actually a good solution.
As with adding a flash, additional ambient lighting can "overwhelm" the reflections from individual light sources. Of course, you need to make sure that the light that you add doesn't create more reflections.
Imagine someone shines a torch in your eye. It's hard to look at the bright light, right? Now imagine a bright, grey sky on a very cloudy day. Even though the sky is bright, it's easier to look at than the torch, isn't it?
This is an example of the difference between a hard lighting source (the torch) and diffuse/soft lighting source (the overcast sky). The worst reflections on shiny objects are caused by harsh, single point light sources. By diffusing this light, you can drastically reduce the reflections. One common way to diffuse light is to add a diffuse dome.
Robot users sometimes think that more advanced (and costly) sensors will solve their problems. For dealing with shiny objects, the assumption is often that 3D laser scanners and 3D vision will not have the same problems as 2D vision.
However, although advanced sensors are necessary in some cases, they aren't immune from the problems caused by shiny objects. Researchers from Huazhong University of Science and Technology explain that even 3D laser scanners are affected by spurious reflections — they had to develop an entire new algorithmic solution to overcome these issues. Sometimes, more advanced technology just means more complex problems!
One of the real secrets to improving vision detection of all objects — not just shiny ones — is to make sure that your teaching phase is good. The detection phase can be quite robust to lighting changes as long as the vision system has a good model (aka template) of the object it needs to detect.
To ensure your system has a good template of your object, you should reduce reflections to a minimum when you are teaching the robot vision system. You may find that the system is able to detect the objects better even in the presence of reflections.
How do you improve the teaching phase? One way is to use our favorite "secret trick" for creating a flawless object template. You can learn about this trick in our article An Unusual Trick That Improves Robot Vision Teaching
What lighting problems have you encountered with robot vision? Tell us in the comments below or join the discussion on LinkedIn, Twitter, Facebook or the DoF professional robotics community.