A Shared Control Strategy for Reach and Grasp of Multiple Objects Using Robot Vision and Noninvasive Brain-Computer Interface

Yang Xu, Heng Zhang, Linfeng Cao, Xiaokang Shu, Dingguo Zhang

Research output: Contribution to journalArticlepeer-review

Abstract

It is ambitious to develop a brain-controlled robotic arm for some patients with motor impairments to perform activities of daily living using brain-computer interfaces (BCIs). Despite much progress achieved, this mission is still very challenging mainly due to the poor decoding performance of BCIs. The problem is even exacerbated in the case of noninvasive BCIs. A shared control strategy is developed in this work to realize flexible robotic arm control for reach and grasp of multiple objects. With the intelligent assistance provided by robot vision, the subject was only required to finish gross reaching movement and target selection using a simple motor imagery-based BCI with binary output. Along with the user control, the robotic arm, which identified and localized potential targets within the workspace in the background, was capable of providing both trajectory correction in the reaching phase to reduce trajectory redundancy and autonomous grasping assistance in the phase of grasp. Ten subjects participated in the experiments containing one session of two-block grasping tasks with fixed locations and another one of randomly placed three-block grasping tasks. The results of the experiments demonstrated substantial improvement with the shared control system. Compared with the single BCI control, the success rate of shared control was significantly higher (p < 0.001 for group performance), and moreover, the task completion time and perceived difficulty were significantly lower (p < 0.001 for group performance both), indicating the potential of our proposed shared control system in real applications.

Original languageEnglish
JournalIEEE Transactions on Automation Science and Engineering
DOIs
Publication statusE-pub ahead of print - 16 Nov 2020

Keywords

  • Brain-computer interface (BCI)
  • Control systems
  • Grasping
  • Manipulators
  • motor imagery (MI)
  • Process control
  • Robot sensing systems
  • robot vision
  • robotic arm
  • Robots
  • shared control.
  • Task analysis

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this