Home › Research › Visual servoing for positioning and flattening fabric

Visual servoing for positioning and flattening fabric

F. Tokuda, A. Seino, A. Kobayashi and K. Kosuge, “CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts,” Proc. IEEE ICRA, pp. 748-754, 2023.

Dual-arm setup for fabric positioning and flattening. Left: overall layout with the projector and robots A and B. Right: close-up of the end-effectors with the camera, force sensors and sponges.
Dual-arm setup for fabric positioning and flattening. Left: overall layout with the projector and robots A and B. Right: close-up of the end-effectors with the camera, force sensors and sponges.

Moving a piece of fabric also changes its shape. On plain fabric, wrinkles are also hard to see in an ordinary image.

I used structured lighting to make the surface relief visible, and fed grayscale images together with force/torque measurements into a CNN that outputs dual-arm motions. The training data were collected by moving the real robot.

Real-robot experiments tested whether the robot could position and flatten fabric at the same time, including wrinkles that did not appear in the training data.

Video

If the video does not load, open it in Google Drive.

Back to Research