Abstract
This work presents a framework for Human-Robot Collaboration (HRC) in assembly tasks that uses multimodal sensors, perception and control methods. First, vision sensing is employed for user identification to determine the collaborative task to be performed. Second, assembly actions and hand gestures are recognised using wearable inertial measurement units (IMUs) and convolutional neural networks (CNN) to identify when robot collaboration is needed and bring the next object to the user for assembly. If collaboration is not required, then the robot performs a solo task. Third, the robot arm uses time domain features from tactile sensors to detect when an object has been touched and grasped for handover actions in the assembly process. These multimodal sensors and computational modules are integrated in a layered control architecture for HRC collaborative assembly tasks. The proposed framework is validated in real-time using a Universal Robot arm (UR3) to collaborate with humans for assembling two types of objects 1) a box and 2) a small chair, and to work on a solo task of moving a stack of Lego blocks when collaboration with the user is not needed. The experiments show that the robot is capable of sensing and perceiving the state of the surrounding environment using multimodal sensors and computational methods to act and collaborate with humans to complete assembly tasks successfully.
Original language | English |
---|---|
Title of host publication | 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Proceedings |
Place of Publication | U. S. A. |
Publisher | IEEE |
Pages | 1266-1271 |
Number of pages | 6 |
ISBN (Electronic) | 9781665452588 |
DOIs | |
Publication status | Published - 31 Oct 2022 |
Event | 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Prague, Czech Republic Duration: 9 Oct 2022 → 12 Oct 2022 |
Publication series
Name | Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics |
---|---|
Volume | 2022-October |
ISSN (Print) | 1062-922X |
Conference
Conference | 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 |
---|---|
Country/Territory | Czech Republic |
City | Prague |
Period | 9/10/22 → 12/10/22 |
Bibliographical note
Funding Information:This work was supported by The Engineering and Physical Sciences Research Council (EPSRC), The Republic of Turkey Ministry of National Education and the Centre for Autonomous Robotics (CENTAUR).
Keywords
- assembly tasks
- human-robot collaboration
- vision and touch sensing
- wearable sensing
ASJC Scopus subject areas
- Electrical and Electronic Engineering
- Control and Systems Engineering
- Human-Computer Interaction