Point Tracking Test Set Framework

Point Tracking Test Set Framework

Population and split

Define the target camera, scene, motion, subject, duration, resolution, frame rate, and operating condition before collecting clips. Split by source sequence, capture session, subject, site, or another grouping that prevents near-duplicate frames from crossing partitions.

A random frame split is invalid for most video evaluation because adjacent frames share the same motion, appearance, and scene.

Annotation policy

The record must define how query points are chosen, which physical surface location they represent, when a point becomes visible or occluded, how out-of-frame states are labeled, how re-entry is handled, and how ambiguous surfaces are treated.

Annotation tools may interpolate with optical flow, but a human reviewer should correct drift and hard sections. The TAP-Vid project reports that flow-assisted annotation reduced effort while still requiring refinement.

Coverage

Quality control

Pilot the policy with multiple annotators. Measure disagreement in coordinates and visibility. Resolve examples through an adjudication guide. Recheck a hidden sample after every tool or policy change.

Version corrections instead of silently replacing them. Record the original label, reason, reviewer, date, and affected benchmark result.

Data card

The test set needs a data card covering purpose, collection, rights, consent where required, privacy, retention, access, population, exclusions, splits, annotation, quality, known gaps, metrics, and change history.

Boundary

Public benchmark licenses do not grant rights to internal or customer video. Data rights and privacy review must cover capture, annotation, storage, model access, screenshots, failure reels, and publication.