Maybe some of you have seen the following video from the guys at Rensselaer Polytechnic Institute (RPI) where a robot replicates the movements of somebody by using Kinect. It was very interesting to see how a Kinect could perform on an industrial robot.
Today, guys from this robotics research lab are going a step further by integrating a Kinect, electric grippers, force/torque sensors and Firewire cameras to do teleoperation through motion detection. The robot uses vision and Alvar tags to align with the surfaces of the box, then applies squeeze control (using position accommodation) for a secure grasp. The user stands in front of the Kinect (with simulation and Kinect skeletal command shown) to provide motion input to the box. The robot autonomously applies the required force to prevent the box from falling.
The key of this integration is the Robot Raconteur (RR). Robot Raconteur is an open-source project intended to provide a solution to distributed control and component interfaces. It is precisely designed for the scenario of an engineer wanting to control a component from a high level language (refer to this blog post to learn more about Robot Raconteur). So this team of researchers has built a RR service for the Motoman High Speed Controller to handle its peripherals. Here is the overall architecture:
Using Kinect frees users from all the constraints related to mechanical interfaces, such as a joystick, keyboard, etc. It is much more natural and intuitive that way.
Laws for control have also been implemented to make sure the box doesn't slip or fall. The arms "squeeze" the box with 10N force towards each other (the other degrees of freedom are under position control). Since the robot position is the only command, the robot's position accommodation control can be used to emulate a generalized damper (essentially integral force feedback). The robot motion is also limited to avoid excessive motion induced force from causing the robot to lose its grip.
To learn more about other research projects which include our electric grippers visit our research webpage.