Triton Inference Server
NVIDIA inference serving platform for deploying AI models at scale.
About
Triton Inference Server by NVIDIA is an inference-serving platform for deploying models from many frameworks, including TensorRT, PyTorch, TensorFlow, ONNX, OpenVINO, Python, and vLLM. It offers dynamic batching, model ensembles, concurrent execution, and metrics, and runs across cloud, data center, edge, and embedded devices on NVIDIA GPUs or x86 and ARM CPUs. Released under the BSD-3-Clause license.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- AI Deployment & MLOps
- Price
- Free
- Platform
- Local/Desktop
- Difficulty
- Advanced (4/5)
- License
- BSD-3-Clause
- Minimum VRAM
- 8 GB
- Added
- Apr 3, 2026
Related Tools
Local AI API platform that runs LLMs on your hardware with OpenAI-compatible API.
Self-hosted Go gateway that routes LLM traffic across providers with failover, caching, and guardrails.
Open-source orchestrator for AI training and inference across clouds, Kubernetes, and bare metal.
Kubernetes-native workflow orchestration platform for machine learning and data pipelines.
Open-source AI gateway that routes requests to more than 1,600 LLMs through one API with guardrails and caching.
Framework for building production-ready AI application services.