Grounded SAM 2

Type a phrase such as 'hippo' or 'person holding a cup' and this IDEA Research pipeline returns boxes, pixel masks, and object IDs that persist from frame to frame in video. Locally it pairs Grounding DINO 1.0, loaded through Hugging Face Transformers or from the original checkpoint, or Microsoft Florence-2 with Meta's SAM 2 and SAM 2.1, passing detections on as box, point, or mask prompts. Demo scripts handle still images, your own MP4 files, live camera feeds, fresh IDs for objects that appear mid-clip, reverse tracking, and JSON dumps with RLE-encoded masks. IDEA's newer detectors, DINO-X and Grounding DINO versions 1.5 and 1.6, plus SAHI tiling for dense 4K scenes, work only through a token-gated cloud API. Expect Python 3.10, PyTorch 2.3 or newer, and a CUDA toolkit for building the custom deformable-attention op, or use the bundled Docker setup. The repository is Apache-2.0, so it fits auto-labeling jobs, dataset bootstrapping, and vision prototypes where the cloud models stay optional.