F. Tokuda, A. Seino, A. Kobayashi, K. Tang and K. Kosuge, “Transformer Driven Visual Servoing for Fabric Texture Matching Using Dual-Arm Manipulator,” IEEE Robotics and Automation Letters, vol. 11, no. 2, pp. 1522-1529, 2026.
When two fabric pieces are stacked for sewing, their surface patterns have to line up, not just their outlines. A lifted piece deforms easily, which makes it hard to adjust the relative position and orientation until the patterns match.
In this work I added a Difference Extraction Attention Module (DEAM), which picks out differences between the target and current images, to a Transformer-based estimator of the pose offset between the two pieces. Visual servoing on that estimate is combined with dual-arm impedance control, so the fabric stays taut while it is aligned and placed on the table.
The model was trained on synthetic images and tested on the real robot with several fabrics of different textures. It does not need to be trained on the specific texture beforehand.