TechRxiv_final.pdf (11.68 MB)
Download file

CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts

Download (11.68 MB)
posted on 2023-08-07, 13:34 authored by Fuyuki TokudaFuyuki Tokuda, Akira SeinoAkira Seino, Akinari Kobayashi, Kazuhiro KosugeKazuhiro Kosuge

This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator’s motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme.

 *The complete version of this preprint paper is in proceedings of ICRA 2023. Please refer to "". 


Innovation and Technology Commission of the HKSAR Government under the InnoHK initiative

JC STEM Lab of Robotics for Soft Materials funded by The Hong Kong Jockey Club Charities Trust


Email Address of Submitting Author

ORCID of Submitting Author

Submitting Author's Institution

Centre for Transformative Garment Production

Submitting Author's Country

  • Japan

Usage metrics