Sparsh Paper Code and Artifact Record

Sparsh Paper Code and Artifact Record

Version and venue

The arXiv record shows version 1 submitted on October 31, 2024. The OpenReview record identifies the work with the 8th Annual Conference on Robot Learning in 2024.

Public quantitative claims should name the reviewed version because the accessible HTML and appendix contain an aggregation discrepancy.

Training data

The paper says the authors curated about 661,000 images from YCB-Slide, Touch-Slide, Touch-and-Go, and ObjectFolder data. Seventy percent, or about 462,700 images, were used for self-supervised pretraining. The abstract and project material describe this as more than 460,000 tactile images.

The sensor families represented are DIGIT, GelSight 2017 with markers, and GelSight Mini without markers. Background subtraction is used for DIGIT and GelSight Mini. The repository says the DIGIT pretraining mix contains about 360,000 samples and provides reference no-contact images for preprocessing.

Model family

Sparsh adapts MAE, DINO, DINOv2, I-JEPA, and V-JEPA approaches to tactile images. The main encoders use a ViT-B/14 architecture with roughly 86 million parameters.

MAE reconstructs masked pixel content. DINO and DINOv2 use self-distillation between student and teacher representations. I-JEPA predicts target representations from context in an image. V-JEPA applies a related predictive approach over short video clips.

The paper reports that DINO and I-JEPA were generally the most competitive across its benchmark, while different models led on particular tasks. V-JEPA was strongest on the reported slip-detection comparison, and MAE performed well on textile recognition.

Temporal input

Image-based variants concatenate the current tactile frame with a frame five samples earlier. At 60 frames per second this spans about 80 milliseconds. V-JEPA uses four frames spanning about 100 milliseconds.

This design embeds a short history. It does not give the encoder indefinite memory or guarantee causal control behavior.

Evaluation architecture

For most TacBench tasks, the authors freeze the pretrained encoder and train an attentive pooling module plus a small supervised decoder. Force-field visualization uses a dense DPT-style decoder and photometric losses. The bead-maze task conditions a Diffusion Policy on Sparsh features and robot proprioception.

The use of labeled downstream data is central. Sparsh reduces the need to train every feature extractor from scratch. It does not remove the need for labels, task design, ground truth, or policy demonstrations.

Compute and throughput

The paper states that all models were trained for 150 epochs on eight Nvidia A100 80GB GPUs. The paper reports backbone inference rates from 60 to 112 frames per second, depending on the model, measured on an Nvidia RTX 3080.

Those numbers cover the reported setup and backbone. They do not establish full-system latency, memory, energy, or cost for a new sensor, decoder, robot, or deployment.

Artifact state and rights

The official repository was archived on April 1, 2026. It provides code, checkpoints, dataset instructions, and task configurations in read-only form.

The repository's license file is CC BY-NC 4.0. The public content should not imply that every dataset, pretrained weight, dependency, sensor design, or commercial use inherits one simple permission. Rights review is required before reuse.