G3DB: A database of successful and failed grasps with RGB-D images, point clouds, mesh models and gripper parameters

Abstract

Autonomous grasping is a problem that receives continuous attention from our community because it is both key to many applications, and difficult to solve. The complexity of robot grasping is counter-intuitive. For us humans, planning a grasp is a trivial task that requires neither considerable effort nor time to solve, it is difficult to imagine the tremendous challenges that our brain has to overcome in order to allow us to interact with objects with such ease. Firstly, much information is missing. The weight, mass distribution, or friction of an object are impossible to measure prior to manipulating an object, and these properties have a dramatic impact on the behavior of a grasp. We thus have to infer from past experience or common sense the most likely values for them, and adapt to actual object properties during the grasp. Another key piece of information that is missing is the object’s 3D shape. Humans and mobile robots perceive the world from a single viewpoint, making the back and often the sides of an object inaccessible through our senses. We are thus forced to consider grasps for which at least one finger comes in contact with a part of the object that we cannot perceive. Further, the faces of an object that we do perceive are sampled through noisy sensors yielding unreliable depth or color. To complete the list of challenges associated with grasping, we note that grasps are parametrized in a highdimensional space: six parameters to fix the position and orientation of the gripper, plus the parameters that define the shape of the hand – 25 for a human hand, 4 in the case of the Barrett hand considered in this paper. The space to explore to plan a grasp is high dimensional, and, given the mechanical complexity of grasping, good solutions are sparsely distributed.