Multimodal sensing and machine learning for soft and hard texture roughness recognition using sliding exploratory procedures

Texture roughness perception is crucial for autonomous robots to perform manipulation, quality inspection, and material discrimination in unknown environments. This work proposes an approach to combine vibration and force data using the VibroTact sensor for texture roughness classification. Vibration and force data are first processed by CNN and ANN models, and then combined using a Bayesian framework. This approach is evaluated by recognizing 15 textures with different roughness (7 soft and 8 hard textures) using individual ANN and CNN models, and is compared against the Bayesian combination of both methods. Texture data is collected by mounting the VibroTact sensor on a robotic arm and using three sliding exploratory procedures (vertical, diagonal, and circular sliding). The texture roughness recognition results achieve 100% accuracy using the combined approach, which improves the performance of individual ANN and CNN models which range from 87.50% to 100% accuracy. The results also show that diagonal and vertical sliding are optimal for recognizing hard and soft textures, respectively. This approach demonstrates its potential for industrial robotics applications that require texture discrimination.