Deep learning based robot cognitive architecture for collaborative assembly tasks

As the manufacturing industry becomes more agile, the use of collaborative robots capable of safely working with humans is becoming more prevalent, while adaptable and natural interaction is a goal yet to be achieved. This work presents a cognitive architecture composed of perception and reasoning modules that allows a robot to adapt its actions while collaborating with humans in an assembly task. Human action recognition perception is performed using convolutional neural network models with inertial measurement unit and skeleton tracking data. The action predictions are used for task status reasoning which predicts the time left for each action in a task allowing a robot to plan future actions. The task status reasoning uses a recurrent neural network method which is developed for transferability to new actions and tasks. Updateable input parameters allowing the system to optimise for each user and task with each trial performed are also investigated. Finally, the complete system is demonstrated with the collaborative assembly of a small chair and wooden box, along with a solo robot task of stacking objects performed when it would otherwise be idle. The human actions recognised are using a screw driver, Allen key, hammer and hand screwing, with online accuracies between 83-92 %. User trials demonstrate the robot deciding when to start collaborative actions in order to synchronise with the user, as well as deciding when it has time to complete an action on its solo task before a collaborative action is required.