Bin Picking the Easy Way vs the Hard Way With Robot Vision

It's one of the trickiest robotic tasks in the world. But, you don't need a complex solution to solve it. There's a hard way to do bin picking with robot vision and an easy way.

The Robotiq Wrist Camera URCap version 1.7 is now the fastest and most intuitive vision system for Universal Robots.

Bin picking is "robotics quest for the holy grail", said Guillame Roberts here on the blog back in 2013. Although 6 years have passed since he wrote that, robotic bin picking is still almost as elusive as it was back then.

Over the past two decades, a lot of research and development has gone into giving robots the ability to recognize jumbled objects in a box and pick them up individually. This task — which is so simple for us humans — has revealed the limitations of almost every aspect of robotic manipulation, including: robot vision, multi-fingered grasping, artificial intelligence, and trajectory planning.

It's true that bin picking is now much more possible than it was 5 years ago. Technological advances in all the related fields have provided solutions to may of the challenges which previously made it impossible. However, there's still a long way to go. Last year, Amazon discontinued their yearly picking challenge because it wasn't producing results.

Despite the advances, most current solutions are complex and require a lot of extra technology, such as advanced 3D vision setups and machine learning.

But, it doesn't have to be like this. Last year, teams here at Robotiq revealed that you don't have to go for the hard solutions to bin picking. Even with simple robot vision, you can achieve robust bin picking with a robot.

What is bin picking?

Bin picking is a robotic manipulation task that involves detecting objects which are arranged in a highly unstructured manner and picking them up individually.

The classic example is to have objects piled on top of each other inside a box. The robot a vision system to detect individual objects. It then uses trajectory planning to grasp each object one by one and remove it from the box.

This is a very challenging task for robot vision.

Challenges include:

• Occlusion — Some objects are partly or completely hidden by the objects on top of them.
• Lighting — The objects cast shadows on each other which further hides them from the camera.
• Edge detection — It is unclear where one object starts and the other finishes, which makes it hard to detect the outline of each individual object.

These issues occur using both 2D and 3D vision. However, they are especially problematic in 2D robot vision as they can make it almost impossible to detect individual objects.

• Read more: Top 10 Challenges for Robot Vision

The hard way to do bin picking: Complex 3D vision

With complete control over the camera settings and model definition, the software can adapt to any situation.

There are various commercial solutions for bin picking, but they tend to be quite complex and costly. They can also be quite inflexible — you need to add a lot of extra "stuff" around the robot for the system to work (e.g. fixed cameras, lighting, etc).

A lot of the available solutions use 3D vision. These require advanced processing and purchasing extra technology to make them work.

For example, a typical bin picking setup might include:

• 3D laser scanner(s) — These use laser light to capture a 3D depth image. The sensor produces a "point cloud" of the objects and surrounding area.
• Stereoscopic vision sensors — These involve using dual cameras to create a 3D image of the environment. They can be used alone or together with a laser scanner to improve the detection accuracy.
• 3D object detection — These algorithms attempt to find objects within the 3D point cloud. Some use CAD models of the objects that are being detected.
• Fixed lighting — Some systems require extra lighting to provide consistent illumination of the scene.

The complexity of these systems varies quite a lot. Some are small, self-contained units which are mounted above the detection area. Others require you to set up various sensors and lights at precise locations around the area. Whatever method they choose, all of them require more technology than a simple Robot Camera, which increases the cost of them.

Once the technology is set up and the system has been trained, the detection can be quite robust. You program the robot to receive object locations from the sensor and then use trajectory planning to pick the objects up.

The easy way to do bin picking: Simplify

There is an easier way to achieve bin picking.

It only requires a simple Robot Camera.

This is a technique that was used by the participants at last year's Robotiq User Conference Technical Challenge. It allows you to achieve complex bin picking without the need for complex sensing technology.

The trick to this technique is to simplify the detection step. Instead of trying to detect objects when they are piled up on top of each other, move them to a place where they can be more easily detected by a normal, 2D vision sensor.

Here's how it works.

1. Use the robot gripper to grab a "handful" of the objects that you want to pick. There's no need to detect the objects for this. Simply move the gripper into the box and grasp.
2. Drop the objects onto a flat surface.
3. Use the robot vision sensor to detect individual objects on the surface.
4. Pick up each object one by one.

That's it!

The real value of this technique is that it is so easy to implement. It's basically just a pick and place operation with an extra step.

Of course, this technique doesn't work for every single object. You need to have objects which don't have to be grasped in an exact same way every time. However, if your objects are so fragile that they require delicate handling, you probably won't have them piled into a box in the first place!

The secret of this technique, as with many good robot programming tricks, is… simplify.

Simpler robot solutions always have a better chance of success.

What tricks have you used to simplify a robot picking operation? Tell us in the comments below or join the discussion on LinkedIn, Twitter, Facebook or the DoF professional robotics community.