The Use of Modular Approaches for Robots to Learn Grasping and Manipulation

  • Bidan Huang

Student thesis: Doctoral ThesisPhD


Modular approaches are widely used methods in AI and engineering. This approach reduces the difficulty of solving a complex problem by subdividing the problem into several smaller parts, i.e. modules, and tackle each independently. In this dissertation, we show how modular approaches can simplify grasping and manipulation problems of service robots. We use the modular approach to tame the difficulties in solving three main research problems in this field: grasp planning, object manipulation and reach motion planning. Different from industrial controlled environments, service robots have to handle abrupt changes and uncertainties occurring in dynamic and cluttered human centered environments. Planning behaviours in such an environment needs to be fast and adaptive to changing context. Programming robot with adaptive behaviours usually is a difficult task and takes a long time. By adopting modular approaches, the task difficulty is reduced as well as the programming time.The proposed approach is based on the method of imitation learning, sometimes referred to as the Programming by Demonstration (PbD). In this framework, we first let human or robot demonstrates possible solutions of the problem. After collecting the demonstrations, we extract multiple modules from the data. Each module represents a part of the system and their corresponding demonstrations are modeled with a statistical method. According to the environment condition, a set of appropriate modules are chosen to provide the final solution.In this dissertation, we present three different modular approaches in tackling three subareas in robot grasping and manipulation: grasp planning, object manipulation adaptive control and planning reaching motions. In Chapter 3, we propose a fast method for computing grasps for known objects and extend this method by a modular approach to work with novel objects. We implemented this method with two different robot hands: the Barrett hand and the iCub hand, and show that the computation time is always in the millisecond scale. In Chapter 4, we present our modular approach in extracting adaptive control strategies using human demonstrations of object manipulation tasks. We successfully implement this method to teach a robot an manipulation tasks: opening bottle caps. In Chapter 5, we present a method to model reachingmotion primitives that would allow humans to modulate robot motions by verbal commands.This method is implemented to perform a bimanual lifting task. We show that the method can generate new motions to lift boxes with different sizes and at different locations. These three studies show that robot grasping and manipulation problems can indeed be divided into modules, the solutions of which can be combined to provide a whole solution to the original problems. With modular approaches, new solutions for novel scenarios can be integrated to the original solution without difficulty. This approach allows robots to accumulate their skills. In summary, we contribute three modular and learning hybrid methods in this dissertation: (1) a fast method for grasp planning; (2) a method to extract human manipulation skills from demonstrations for object manipulation; (3) a method to recognize motions and generatemotions according to human commands.
Date of Award2 Jul 2015
Original languageEnglish
Awarding Institution
  • University of Bath
SupervisorJoanna Bryson (Supervisor) & Aude Billard (Supervisor)


  • Robotics
  • Modular approaches
  • learning (artificial intelligence)

Cite this