Soft Tactile Sensor with Multimodal Data Processing for Texture Recognition

In this letter, a soft tactile device with an alternative design approach is presented in tasks of texture recognition using multimodal data processing. This device integrates multiple layers and sensing elements in a soft device. The top layer is covered with a flexible piezoresistive material. The bottom layer is comprised of a soft case and a seven-axis chip within capable of measuring acceleration, angular velocity, and pressure data. The soft tactile sensor is validated with texture recognition tasks using data collected from five textures slid on the sensor with a robotic arm. These experiments are key to validate and characterize the sensor design by analyzing both individual and combined piezoresistive, accelerometer, and angular velocity signals with Bayesian methods. The results show that the recognition accuracy achieved by the sensor is related to the type and combination of data modalities. The highest accuracy achieved is 99.43% by combining piezoresistive and accelerometer data, while the lowest accuracy of 90.12% is obtained with angular velocity data alone. Overall, this work shows that the proposed multimodal soft tactile sensor can improve the performance of recognition tasks by the systematic use of multimodal data.