GGML

Much of the local LLM world, including llama.cpp, whisper.cpp and the apps built on them, does its math through ggml, a compact C and C++ tensor library started by Georgi Gerganov and now under the ggml-org umbrella. You describe a graph of tensor operations, let ggml work out the memory it needs before anything runs, and hand the graph to a backend: hand-tuned vector code for the common CPU families, or offload to CUDA, HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, CANN, MUSA, OpenVINO and more. It also defines the GGUF model file format and a family of quantized weight types, integer formats spanning roughly 1.5 to 8 bits per weight plus the 4-bit floating point MXFP4 and NVFP4 layouts, so large models fit in consumer RAM. It builds with plain CMake and no external libraries, and the examples folder covers GPT-2, GPT-J, MNIST, YOLO and Segment Anything. Core work happens in the llama.cpp repository and syncs here, with regular tagged releases. MIT licensing makes commercial embedding easy. It is a building block for systems programmers writing inference engines; to simply run a model, use llama.cpp or a front end on top of it.