Using The Kinect For Robotic Manipulation

Many observers of robotics predicted that cheap sensors will be a key component for the field to progress. It appears this might be happening right now. Microsoft Kinect, originally developed for the Xbox 360 video game console, is literally invading the electronics market and now reaching into robotics. At a low $150 cost, the sensor device scored a Guinness World Record for "fastest selling consumer electronics device" last winter and now its application reach by amateurs and professionals in robotics is fast multiplying.

Technical Description:

• Central camera is an RGB camera (8-bit VGA resolution of 640X480) for image acquisition
• Two-side camera (11-bit VGA resolution of 640X480) infrared lasers with CMOS image sensing for 3D depth sensors
• Microphone array
• Proprietary connector USB-like with additional power
• Motorized tilt on the base
• Sensor field view of 57° horizontally and 43° vertically ±27° from the motor
• Programmable LED

Software for Kinect
Microsoft released a Software Development Kit for RDS (Robotic Developer Studio) on July 13th, for free. Microsoft is now active in the "Kinect hacking" community, many developers have been using Google ROS or third party support and Microsoft perhaps hopes to turn the tide.

Usage for Manipulation
The Kinect is, of course, used for machine vision setup, but there are two main features it can provide. Shape recognition via the standard camera and distance evaluation via the two infrared laser cameras. The method used in object manipulation is the following:

1. Evaluate the environment, find the desired object, the surrounding environment and the obstacles
2. Grasp the object
3. Manipulate the object in the known environment

Here is a great example of how the Kinect is used in robotic manipulation:

Other uses of machine vision provided by Kinect in robotics are:

• Human recognition: Tracking the movement of humans to replicate it with robots (Teleoperation)
• Environment recognition: Mapping the environment surrounding a robot (Navigation)
• Telepresence: Providing vision and movement replication

When robotics can piggyback on consumer market applications or use cases, it typically brings down the cost which is a good thing for the robotics industry. What do you think the other missing blocks are for a true robotics renaissance to happen on a large scale?

Read our other post on Microsoft in robotics?