The 2026 Gaussian Splatting and 3D Reconstruction Toolchain

Turning a folder of photos into a renderable 3D scene now means wiring together at least four separate models, and the licenses on half of them will stop you from shipping. This is a component-level map of the 2026 toolchain: camera pose estimation, Gaussian optimization, depth and tracking priors, export formats, viewers, and the cases where classic photogrammetry is still the correct answer. It assumes you are choosing libraries to build on, not shopping for a scanning app.

The pipeline, stage by stage

Every Gaussian splatting pipeline reduces to the same five stages, and each one is independently replaceable:

1. Capture: images or video frames, plus whatever metadata the camera bothered to record.
2. Pose: per-image extrinsics and intrinsics, usually with a sparse point cloud to initialize from.
3. Optimization: fit anisotropic 3D Gaussians to the images through a differentiable rasterizer.
4. Compression and export: turn tens of millions of floats into something a browser will load.
5. Rendering: a viewer that depth-sorts and rasterizes splats at frame rate.

Stages 2 and 3 are where the churn is. Nearly all of it is PyTorch plus hand-written CUDA, so the practical blocker on a new machine is almost always a matching torch and CUDA build rather than the reconstruction code itself.

Pose estimation: COLMAP is still the floor

COLMAP remains the reference structure-from-motion implementation and it is still the thing every other tool is measured against. It is under the new BSD license, which is one of the reasons it has survived as infrastructure. Version 4.0.0 landed in March 2026 and folded the GLOMAP global mapper in as a first-class alternative to the incremental and hierarchical mappers, reachable through the global_mapper command or automatic_reconstructor --mapper GLOBAL. The standalone GLOMAP repository was archived on 9 March 2026 because the work moved upstream. The 4.x line also replaced FreeImage with OpenImageIO for roughly 2.5x faster image I/O, and added ALIKED feature extraction via ONNX plus a LightGlue matcher. The latest tag at the time of writing is 4.1.1, published 17 July 2026.

Two practical notes. The OpenImageIO swap changes decoded pixel values slightly, so a pipeline with cached features or hard-coded reproducibility assertions will not reproduce bit-for-bit across the 3.x/4.x boundary. And the global mapper estimates intrinsics through view graph calibration from two-view geometries, which can yield different camera parameters than the incremental pipeline's self-calibration. If you compare splat quality across mappers, hold the intrinsics fixed or you are measuring two things at once.

Pose-free reconstruction: VGGT and the pointmap lineage

The interesting alternative is to skip correspondence search entirely. VGGT is a feed-forward transformer that takes one to hundreds of views and predicts camera parameters, depth maps, point maps, and 3D point tracks in a single pass, in under a second for small inputs. It won the CVPR 2025 best paper award, and it changed what a reasonable first attempt at a hard capture looks like: you get a rough pose graph in seconds instead of a COLMAP run that may or may not converge overnight.

Memory is the constraint, and it grows roughly linearly with frame count. The VGGT paper reports runtime and peak GPU memory for point map estimation on a single H100 with flash attention v3, at 336x518 input resolution:

Input frames	1	10	20	50	100	200
Time (s)	0.04	0.14	0.31	1.04	3.12	8.75
Peak memory (GB)	1.88	3.63	5.58	11.41	21.15	40.63

A 16 GB consumer card lands you somewhere around 70 frames at that resolution. Two things have shifted since the paper. A 15 May 2026 fix removed redundant intermediate tensors and lets VGGT run on roughly 2 to 3 times more input frames for the same budget, and VGGT-Omega reports using only about 30 percent of the GPU memory of its predecessor. Anything written before mid-2026 understates how many frames you can fit.

DUSt3R is the model that started this line: dense pairwise pointmap regression followed by global alignment, with three checkpoints at 224 and 512 resolution, the 512 DPT head being the one worth using. Its successor MASt3R bolts on a local feature head, metric pointmaps, and a more scalable global alignment, and MASt3R-SfM adds retrieval-based pair selection so you are not paying quadratic cost on pair construction. The catch is the global alignment step: it is an optimization over all pairs, and it dominates both runtime and memory once you go past a few dozen images. These models are transformer encoder stacks in the same self-supervised ViT lineage as DINOv2, which is why they inherit the same resolution and token-count economics.

The license trap under pose-free

This is where teams get hurt. The pose-free models are research artifacts first and products second, and the licensing reflects that.

DUSt3R and MASt3R are both CC BY-NC-SA 4.0. Non-commercial, share-alike, and you are additionally expected to comply with the licenses of every public training dataset and base checkpoint involved. That is not a technicality you can paper over with an internal memo.

VGGT is more subtle. On 29 July 2025 the repository license was updated to permit commercial use, excluding military applications, but only the VGGT-1B-Commercial checkpoint carries those rights. The original VGGT-1B weights remain non-commercial. Two checkpoints, one repository, similar reported performance, completely different legal status. If your inference code does VGGT.from_pretrained("facebook/VGGT-1B") because that is what the README quickstart shows, you have pulled the non-commercial weights into a commercial product.

VGGT-Omega, released 18 May 2026 and billed on its project page as a CVPR 2026 best paper finalist, is worse news again for product teams: its Hugging Face model card is tagged cc-by-nc-4.0. It is the better model and it is not usable in a commercial product.

If you need commercially clean feed-forward geometry, MapAnything from Meta and Carnegie Mellon is the current answer: Apache-2.0 code with two published checkpoints that share an identical API and differ only in training data and license. facebook/map-anything-apache is Apache-2.0 and trained on the permissively licensed subset, facebook/map-anything is CC BY-NC 4.0 and trained on the fuller mix, thirteen datasets in the paper. Splitting the release that way is the pattern the rest of the field should copy.

Training splats: gsplat is the default backend

The original 3D Gaussian Splatting release from Inria and MPII is still the reference implementation and still the most restrictive component in a typical stack. Its license states the software may be used non-commercially, for research and evaluation purposes only, with commercial use requiring explicit consent from the licensors. It also asks for 24 GB of VRAM to train to paper evaluation quality and a CUDA GPU with compute capability 7.0 or higher. The October 2024 update was substantive: an accelerated rasterizer worth about 1.6x with the default optimizer and about 2.7x with the new sparse Adam optimizer, plus depth regularization, anti-aliasing, and exposure compensation.

gsplat is the clean-room reimplementation and the one to actually build on. Apache-2.0, maintained by the Nerfstudio project, documented in JMLR volume 26 (2025), and reporting up to 4x less GPU memory with up to 15% less training time than the official implementation. The paper is more conservative than the README, claiming up to 10% less training time alongside the same 4x memory figure. Either way, the memory reduction is the number that matters, because it moves full-quality training from a 24 GB card down to consumer hardware.

# match your existing torch + CUDA build first
pip install gsplat

# or from source
pip install git+https://github.com/nerfstudio-project/gsplat.git

# or a prebuilt wheel, e.g. PyTorch 2.0 with CUDA 11.8
pip install ninja numpy jaxtyping rich
pip install gsplat --index-url https://docs.gsplat.studio/whl/pt20cu118

The plain pip install gsplat path JIT-compiles CUDA kernels on first run, which is the single most common source of "it hung" reports. Use the prebuilt wheel index if you can match your torch and CUDA versions.

Worth knowing: the last tagged gsplat release is v1.5.3 from July 2025, but the main branch has moved substantially since. NVIDIA's 3DGUT was integrated in April 2025 along with arbitrary batching over scenes and viewpoints in May 2025. Through 2026 the changelog adds PPISP as an alternative to the bilateral grid (January), LiDAR rasterization (March), AccuTile conservative ellipse-based tile intersection and NCore v4 capture support (April), a native CUDA MCMC inject_noise op and an experimental HiGS inference-only rendering path (May), and a rasterization pass worth roughly 30 percent on an A100 plus Gaussian ID rasterization, new camera models, and CUDA 13 support (June). If you are on the PyPI release you are a year behind.

Nerfstudio wraps gsplat as splatfacto and gives you the dataparser, viewer, and export plumbing. It is Apache-2.0, so the whole gsplat plus Nerfstudio stack is commercially usable, but note that the last PyPI release is 1.1.5 from November 2024. Install from git if you want the current gsplat features.

Outside the CUDA world, Brush is an Apache-2.0 Rust trainer built on the Burn framework with a WebGPU backend. It runs on macOS, Windows and Linux, on AMD, NVIDIA and Intel cards, on Android, and in a browser, and it ships dependency-free binaries with no CUDA setup. LichtFeld Studio is a desktop workspace with training, live inspection, splat editing with undo/redo, a Python plugin system, MCP resources and tools for automation, and export to PLY, SOG, SPZ, or a standalone HTML viewer. It is developed in public under GPLv3, which matters if you were planning to embed it.

Capture requirements decide everything downstream

No optimizer recovers from a bad capture, and the failure usually surfaces as a pose failure rather than a blurry splat.

Capture guides converge on 70 to 80 percent overlap between adjacent frames, so each feature appears in three to five images, with coverage from low, mid, and high positions. Object-scale captures want roughly 100 to 200 photos in a spiral; room scale wants 200 to 500 or more. A methodical 80-photo capture with consistent overlap beats a casual 300-photo capture with gaps, every time.

Lock exposure and focus before you start walking. Auto-exposure flicker between frames is a direct attack on feature matching, and autofocus hunting changes your intrinsics mid-sequence. On a real camera: manual mode, 1/500s or faster to kill motion blur, f/8 to f/11 for depth of field, lowest usable ISO.

For object captures on a turntable, background subtraction matters more than resolution. rembg is enough for clean studio setups; SAM 2 is the better choice when you need temporally consistent masks propagated across a video, for example to remove a person who walked through your scene.

Depth and tracking as priors, not products

Monocular depth is a regularizer in this stack, not an output. Depth supervision fills in low-texture regions where photometric loss has no gradient, and it stabilizes the geometry behind specular surfaces.

Depth Pro from Apple is the sharpest option for single frames: a 2.25-megapixel depth map in 0.3 seconds on a standard GPU, producing metric depth with absolute scale without needing camera intrinsics metadata. That last property is the useful one, because it means you can get a metrically scaled prior from a phone photo with stripped EXIF. The licensing is the problem: the code carries Apple's proprietary sample-code style license and the weights on Hugging Face are tagged apple-amlr, not an OSI-approved license. Read it before you build on it.

Depth Anything V2 is still widely deployed and well supported across inference runtimes, but check which checkpoint you pulled before assuming it is permissive: only Depth-Anything-V2-Small is Apache-2.0, and Base, Large and Giant are all CC-BY-NC-4.0. Depth Anything 3, released 14 November 2025 with any-view support and metric variants, splits the same way: DA3-Small, DA3-Base, DA3Metric-Large and DA3Mono-Large are Apache-2.0 while the Giant, Large and Nested checkpoints are CC BY-NC 4.0. For a commercial pipeline that needs metric depth, DA3Metric-Large is the one to reach for. MiDaS is worth knowing about mainly because older pipelines and ControlNet-era tooling still assume its relative-depth output convention.

Point tracking is the other prior, and it earns its place in dynamic and validation work rather than in static scene training. CoTracker 3 (October 2024) ships offline and online variants: offline processes a whole video in one window and scores 76.9 delta-avg-vis on TAP-Vid DAVIS, online uses a sliding window for streaming or long footage and scores 76.7. It was trained with roughly 1000x less data than the previous top performers, which is the reason the model is small enough to run alongside a splat trainer. Practical uses: sanity-checking a camera solve by tracking known-static points, sourcing correspondences for dynamic splat rigs, and rejecting frames where tracks collapse. The license is the usual Meta research pattern, mostly CC-BY-NC with portions under separate terms. Track Anything covers the adjacent case of interactive video object segmentation when what you need is a mask, not a point.

Most of these ship a Gradio demo app, which is genuinely the fastest way to check whether a model handles your footage before you write any integration code.

Export formats: PLY is a liability at scale

Uncompressed PLY at spherical harmonics degree 3 stores 62 floats per Gaussian, 248 bytes, covering position, an unused normal, DC color, 45 higher-order SH coefficients, opacity, scale, and a rotation quaternion. For a 4-million-Gaussian indoor scene that is roughly 990 MB. Nobody is streaming that.

Format	Size vs uncompressed PLY	License	Notes
PLY, SH3	1x, 248 bytes per Gaussian	n/a	Universal interchange, unusable for delivery
KSplat	Level 1 drops position, scale, rotation and SH to 16-bit; level 2 takes SH to 8-bit	MIT	Tied to the three.js GaussianSplats3D viewer
SPZ	Around 10x smaller	MIT	Niantic; since v4 each attribute stream is ZSTD compressed in parallel
SOG v2	Around 15x to 20x smaller	Open spec, MIT tooling	PlayCanvas; WebP texture payload, GPU-ready at load
glTF KHR_gaussian_splatting	Container, not a codec	Khronos	Still a release candidate in the glTF registry as of July 2026

SPZ, open sourced by Niantic under MIT, was the first format to treat splats as a delivery problem rather than a dump of training state, and the project reports spz files typically around 10x smaller than the corresponding ply with minimal visual difference. Version 4 uses a 32-byte plaintext header and compresses each attribute stream independently and in parallel with ZSTD, which improves both compression and decompression throughput.

SOG, short for Spatially Ordered Gaussians, is PlayCanvas's answer and the more aggressive one: quantized Gaussian properties packed into WebP textures, row-major with co-located pixel coordinates, so the payload uploads to the GPU without load-time repacking. The spec puts it at roughly 15x to 20x smaller than PLY. Version 2 defines 16-bit positions split across two images, a smallest-three quaternion encoding, codebook-indexed scales and colors, and optional higher-order SH through a palette. It ships either as a directory of files, which is what you want while authoring, or as a zipped bundle, and readers must unzip and resolve through meta.json in both layouts.

The standards picture is finally moving. KHR_gaussian_splatting sits in the glTF extension registry as a release candidate written against glTF 2.0, contributed by Cesium, Niantic Spatial, Esri, Khronos, NVIDIA, Huawei, and Autodesk. It defines position, a unit quaternion rotation, per-axis scale, opacity, and spherical harmonics up to degree 3, attached to a mesh primitive: a client that sees the extension on a primitive treats that primitive as a Gaussian splatting field. As of July 2026 it is still a release candidate rather than a ratified extension, so engine support is ahead of the paperwork.

For viewing and editing, SuperSplat is the pragmatic default in a browser and it is MIT licensed. Cesium, one of the extension's contributors, covers the geospatial case by pushing the same data through 3D Tiles.

Where photogrammetry still wins

A Gaussian splat is a rendering representation. It has no surface, no topology, no watertight volume, and no guarantee of metric scale unless you supplied one. Classic photogrammetry triangulates geometry and produces meshes and point clouds you can measure, export to CAD, or drop into a GIS or BIM workflow. If someone downstream is going to take a dimension off your output, you want a mesh.

Meshroom is the open-source option, MPL2 licensed, with 2025.1.0 released 19 August 2025. That release added a plugin architecture with sub-process isolation for Python nodes, a script editor with hot reload, and new pipelines for color calibration, raw to EXR, object reconstruction, LiDAR processing, and multi-view photometric stereo. Notably, MeshroomHub now carries experimental ML plugins for Gaussian splatting, depth estimation, and optical flow, so the two worlds are converging inside one node graph rather than staying separate pipelines.

If you do need geometry out of a splat, the two established routes are 2D Gaussian Splatting, which flattens Gaussians to surfels and extracts a mesh by rendering depth and running TSDF fusion, and SuGaR, which keeps 3D Gaussians but adds a regularization term pulling them onto the surface before Poisson reconstruction. 2DGS generally wins on sharp edges and fine detail. Neither matches a well-run photogrammetry solve for absolute metric accuracy, and both add a training pass.

A separate branch of the field skips capture entirely and generates 3D from one image. TRELLIS is the strongest permissive option: the models and the majority of the code are MIT, with two CUDA submodules under their own terms. Hunyuan3D 2 is technically excellent but its Tencent community license excludes the European Union, the United Kingdom, and South Korea from the licensed territory and requires a separate grant above 1 million monthly active users. SAM 3D Objects reconstructs shape geometry, texture, and layout from a single image and is built for occlusion and clutter rather than clean studio subjects; Meta reports it outperforming prior 3D generation models in human preference tests on real-world objects and scenes. It ships under the SAM License, which grants a royalty-free worldwide license that does cover commercial use, subject to acceptable-use and export-control terms and with no monthly-active-user threshold. DreamGaussian is the earlier optimization-based route and is mostly of historical interest now, though it is still a readable implementation if you want to understand score distillation into a Gaussian representation.

How to choose

Start from the license constraint, not the benchmark. If the output is commercial, your stack is COLMAP or MapAnything's Apache checkpoint for pose, gsplat under Apache-2.0 for training, an Apache-2.0 Depth Anything 3 checkpoint for depth priors, and SPZ or SOG for delivery. Every one of those is unambiguous. The moment you reach for the original Inria implementation, DUSt3R, MASt3R, VGGT-Omega, CoTracker, or the non-commercial VGGT-1B checkpoint, you are in research-only territory and you should know it before the code is in a release branch, not after.

If it is research or internal evaluation, the ordering flips: VGGT-Omega for pose, DUSt3R or MASt3R when you want dense pointmaps, gsplat main branch for the current rasterization work, and whatever depth model scores best on your scenes.

Two things to watch through the rest of 2026. The first is KHR_gaussian_splatting ratification, because a ratified glTF extension is what turns splats from a viewer-specific asset into something an engine pipeline can treat as normal geometry. The second is whether the feed-forward models keep splitting their releases into commercial and non-commercial checkpoints the way VGGT and MapAnything have. That pattern is annoying but honest, and it is much better than the alternative of a single restrictive license that quietly makes the best model unusable.