CoTracker3 Paper and Code Record

CoTracker3 Paper and Code Record

Version record

The first arXiv version was submitted on October 15, 2024. The later ICCV 2025 record lists the paper in the conference proceedings on pages 6013 through 6022.

Quantitative statements should name the reviewed version. The public explainer should prefer the ICCV version of record for publication identity while using the arXiv HTML where it provides accessible method detail.

Research contribution

CoTracker3 combines a simplified point-tracker architecture with a semi-supervised training recipe. The authors pretrain on synthetic Kubric data, use frozen point trackers as teachers to generate pseudo-labels for real videos, and fine-tune a student model on those trajectories.

The paper reports teacher models drawn from CoTracker3 online and offline, CoTracker, and TAPIR. It randomly selects a frozen teacher for each batch rather than averaging all teacher predictions. Query points are sampled with a preference for features suitable for tracking.

The architecture computes multi-scale convolutional features, compares local feature neighborhoods through 4D correlation, and iteratively updates point coordinates, visibility, and confidence with a transformer. Cross-track attention allows points to inform one another.

Online and offline modes

The online model processes overlapping windows forward through time. The offline model processes the video as a single window and can track forward and backward relative to the query. The paper says the offline model handles occlusion and long-term visible tracks better, but its maximum frame count is memory bound.

The repository says online operation is more memory efficient and supports longer streams. That does not establish latency or throughput for an arbitrary resolution, number of queries, GPU, queue, or consumer.

Reported comparison

The paper's 1,000-times-less-data statement compares 15,000 pseudo-labeled real videos used for CoTracker3 fine-tuning with 15 million real videos used by BootsTAPIR. Both comparisons sit on top of synthetic pretraining. It is a data-volume and reported benchmark comparison, not a claim about total compute, cost, energy, or production efficiency.

The paper also reports two times fewer parameters than CoTracker and 27 percent faster operation than LocoTrack under its evaluated setup. These numbers require exact paper context and should not be generalized to every workload.

Limitation

The paper names teacher quality and diversity as a central limitation. Scaling performance saturates after the student absorbs the available teacher knowledge. The project page also shows featureless surfaces such as sky and water as failure cases.

Code and license

The official repository provides checkpoints, demos, evaluation code, and training scripts. It strongly recommends a GPU for local use.

The repository states that most CoTracker code is licensed under CC BY-NC, with some portions under separate terms. A team must review code, checkpoint, data, dependency, and media rights before commercial or production use.