Weights Merging (mergekit)

Arcee AI's mergekit combines the weights of two or more fine-tuned language models into one checkpoint with no gradient training. You describe a merge in a YAML file and run the mergekit-yaml command; the config names source models or layer slices, a method, per-tensor parameters, and how to reconcile tokenizers and chat templates. Methods range from plain weighted averaging and SLERP to task-vector approaches like TIES, DARE, DELLA, Model Breadcrumbs and Model Stock, plus passthrough for stacking layers into larger frankenmerges. Extra commands build mixture-of-experts models from dense ones, extract LoRA adapters from a fine-tune, chain multi-stage merges, merge raw PyTorch checkpoints and transplant tokenizers, and an in-memory Python API covers scripted use. Tensors load lazily, so merges run on an ordinary CPU, and GPU acceleration needs only a mid-range card (the authors cite 8 GB). Install by cloning the repo and running pip install. The toolkit is licensed LGPL-3.0, which allows commercial use, and contributors sign a CLA. A good fit for model tinkerers and small labs testing blends without paying for training runs.