Main_File.pdf (5.63 MB)
Download file

Improving the Representation and Extraction of Contact Information in Vision-based Tactile Sensors Using Continuous Marker Pattern

Download (5.63 MB)
posted on 2023-07-10, 05:52 authored by Mingxuan LiMingxuan Li, Yen Hang Zhou, Tiemin Li, Yao Jiang

Tactile perception has been a hot topic of research in robotics. Robots sense the shape, material, distributed force, slip during contact, and use the multi-modal contact information to control grasping and manipulation. For vision-based tactile sensors, the contact representation and extraction determine the quality of the raw tactile information, and therefore serve a significant role in the robot perception system. This article highlights for the first time the importance of raw representation and extraction in visuotactile perception, and proposes a new multicolor CMP method for enhancing the performance of vision-based tactile sensors. Based on the principle of continuous marker pattern (CMP), the multicolor CMP method is optimized in the pattern and algorithm design. Regarding information representation, we present a new type of marker pattern based on RGB triangles and a preferred layout. In terms of information extraction, we propose a series of extraction strategies with the adaptive growing algorithm (AGA) and the spin-search algorithm (SSA) as the cores. The experiments reveal that the multicolor CMP method achieves improved precision and reliability compared to the former CMP method.


National Natural Science Foundation of China under Grant 51705274

A grant from the Institute for Guo Qiang, Tsinghua University

National Training Program of Innovation and Entrepreneurship for Undergraduates under Grant 202210003017

Tsinghua University Initiative Scientific Research Program

iCenter Star of Innovation Program from the Fundamental Industry Training Center, Tsinghua University


Email Address of Submitting Author

ORCID of Submitting Author


Submitting Author's Institution

Tsinghua University

Submitting Author's Country

  • China

Usage metrics