Performance Metrics for Robotic Randomized Bin Picking

In March 2012, the Performance Metrics for Intelligent Systems (PerMIS'12) Workshop was held at the University of Maryland. During this event, hosted by the National Institute of Standards and Technology (NIST), a panel discussion focused on a very interesting topic: Technology Readiness for Randomized Bin Picking. In this article, I would like to highlights some of the interesting points that were discussed by the panel. All the information presented here is from a report about the panel discussion published by NIST members (see reference at the end of this article).

Robot performing bin-picking (image from youtube user stefanovkrll)

What is Randomized Bin Picking?

Randomized bin picking is one of the greatest challenges to automated manufacturing today. Basically, a robot has to locate and grasp objects that have been placed randomly in a container. The robot then moves the grasped object to the next manufacturing cell. Having the parts presented in a random way has enormous advantages as it removes the need for specifically designed parts handling systems. This is especially true if there are various parts to be handled by the same robot. Ideally, a standard robot (with a standard gripper and vision system) would be able to be used for different applications with only minor software modifications.

Randomized bin picking is far from being a solved problem as many challenges remain, mostly related to:

• sensing: being able to distinguish the shape, position and orientation of the object on the top of the pile is a quite difficult task for a vision system.
• issues with the hardware involved: the lack of dexterity/adaptability of the gripper/robot system makes it difficult to pick objects when the number of grasping possibilities is reduced.
• integration: according to the authors of the report, this might be the most important challenge. Indeed, the integration cost is usually predominant compared to the hardware cost, due to the fine tuning required to make the solution functional for a particular application.

How to Measure the Quality of a Solution?

The report explains the potential metrics that could be used to evaluate a solution for randomized bin picking applications. The first one is speed, which is defined as: the time required to pick an object in the bin or the numbers of object which can be picked for a given amount of time. The second one is efficiency, which is a mixture of time efficiency (time required to locate the object vs cycle time), grasping quality and percentage of success. The third one is accuracy, which is the measurement of error in object recognition and part pose estimation.

The authors of the report also present the Technology Readiness Level (TRL), which is a process originally used by NASA to evaluate if a given technology is reliable enough to be used in the field. Basically, the TRL of a technology tells the level of qualification of the solution (a TRL of 1 corresponds to the basic principles being observed whereas a TRL of 9 is equivalent to a fully functional system proven by vetting). The authors explain how this scale could be modified to be applied to manufacturing applications such as randomized bin picking.

Particularities of Randomized Bin Picking

The most interesting part of the report (in my opinion) is the description of the panel discussion itself, in which the panelists identified several aspects specific to randomized bin picking. Here are some highlights of this discussion:

• Rough parts have to be handled differently from finished parts. Therefore, a solution which is effective at the beginning of the manufacturing line might not be the best at the end of it where the product is almost finished.
• The cost of inter-process component transfer has to be reduced by decreasing the complexity of the systems responsible for these steps. 
• Structured bin picking in which the objects are always presented in the same way can be considered a solved problem. The remaining challenges are associated with randomized bin picking in which no assumptions can be made regarding the object's position and orientation.
• The cost of robot bin picking solutions is not driven by the cost of the robot itself, but rather by the integration cost (adapting the solution to the specific application). From this observation, we could conclude that adding (relatively) expensive sensors might be acceptable if it helps to reduce the integration time and cost of the solution.
• It is desirable to have humans and robots working together on the production line. This means that a solution for randomized bin picking should be safe enough to have humans standing around it.
• In addition to the expected quality metrics such as speed and throughput, it was recommended to also evaluate agility and repurposing, i.e. the ability to switch from different products using the same setup. 

Adaptive Grippers and Randomized Bin Picking

Grippers are only one component of a randomized bin picking solution. However, as it was explained by the authors of the report, there are numerous advantages of having standard hardware components which are effective in various applications. A gripper which can adapt to objects of various shapes (such as our 2-Finger Adaptive Robot Gripper) is therefore very interesting for randomized bin picking!

In conclusion, lets just say that randomized bin picking is another sub-field of robotics which is craving new ideas that have the potential to improve efficiency in a number of manufacturing processes. Lets hope that someone will seize these opportunities!

Thanks for reading!