CoTracker

Point tracking gets a joint treatment in CoTracker, the transformer from Meta AI and Oxford VGG that follows quasi-dense sets of pixels through a video together rather than independently, letting correlated points support each other through occlusions. The third release, CoTracker3, arrived in October 2024 with a simplified architecture and a pseudo-labeling pipeline over real videos that the authors report reaching state-of-the-art accuracy with 1,000x less training data than prior trackers, and a matching Kubric dataset followed in January 2025. Points can be selected manually or sampled on a grid, processed in online mode for streaming video or offline mode for full clips. Usage is a torch.hub one-liner or a local install, with a Hugging Face Space demo and a Colab notebook for trying it without setup, and a GPU is strongly recommended for local runs. Most of the code and the checkpoints are released under CC-BY-NC 4.0, restricting use to non-commercial research, with some files under MIT and Apache-2.0. Video editing, robotics, and 3D reconstruction pipelines rely on it wherever long-range pixel correspondence matters.