BitNet
Frameworkmicrosoft/BitNet
Official inference framework for fast, lossless 1-bit LLMs on CPU and GPU.
Overview
bitnet.cpp is Microsoft's official inference framework for 1-bit LLMs such as BitNet b1.58, delivering fast and lossless ternary model inference on CPUs and GPUs. It provides optimized kernels that achieve significant speedups and large energy savings on x86 and ARM, enabling even 100B-parameter models to run on a single CPU.
README Preview
\n\n# bitnet.cpp\n\n[](https://opensource.org/licenses/MIT)\n\n[](https://huggingface.co/collections/microsoft/bitnet)\n[](https://arxiv.org/abs/2502.11880)\n[](https://demo-bitnet-h0h8hcfqeqhrf5gf.canadacentral-01.azurewebsites.net/)\n[](https://github.com/microsoft/BitNet/blob/main/gpu/README.md)\n\n\n\n\n\n📰 News\n\n07/23/2026: 📣 We released VibeASR.cpp — a real-time multilingual ASR inference engine on CPU using BitNet I2_S quantization, achieving RTF Code] [Models] [Report] \n\n07/20/2026: 📣 We released BitNet-embedding-0.6B and BitNet-embedding-270M on Hugging Face — the first 1-bit embedding models that deliver competitive embedding quality with significantly faster inference on CPUs.\n- **1.42x to 2.28x speedup** over F16 on BitNet-embedding-0.6B prefill (8 threads)\n- **1.32x to 1.74x speedup** over F16 on BitNet-embedding-270M prefill (8 threads)\n- Supports I2_S conversion with optimized kernels on x86 CPUs\n- Lossless inference with 2 bits per weight\n\n07/16/2026: 📣 Released [BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization](docs/bitnet-embeddings-i2s-guide.md) — detailed guide for converting and running BitNet embedding models with optimized I2_S kernels.\n\n01/15/2026: 📣 Released [BitNet CPU Inference Optimization](https://github.com/microsoft/BitNet/blob/main/src/README.md) — parallel kernel implementations with configurable tiling and embedding quantization support, achieving **1.15x to 2.1x** additional speedup over the original implementation.\n\n05/20/2025: 📣 Released [BitNet Official GPU inference kernel](https://github.com/microsoft/BitNet/blob/main/gpu/README.md) — extending 1-bit inference beyond CPUs.\n\n04/14/2025: 📣 Released [BitNet Official 2B Parameter Model](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T) on Hugging Face — the first official BitNet b1.58 model trained with 4T tokens.\n\n02/18/2025: 📑 [Bitnet.cpp: Efficient Edge Inference for Ternary LLMs](https://arxiv.org/abs/2502.11880) — system-lev
FAQ (1)
TroubleshootingHow to fix missing llama-cli.exe (FileNotFoundError WinError 2) after building BitNet on Windows?
The build succeeds but llama-cli.exe is never generated because llama.cpp is included as a Git submodule, which makes CMake set LLAMA_BUILD_EXAMPLES=OFF by default (the CLI is treated as an 'example' target). Fix: re-run the CMake configuration step with the flag enabled before building:
cmake -B build -DLLAMA_BUILD_EXAMPLES=ON .
cmake --build build --config Release
Alternatively, explicitly build only the CLI target: cmake --build build --config Release --target llama-cli. After rebuilding, llama-cli.exe will appear in build/bin/Release and python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "Test" -cnv will work. A permanent fix to setup_env.py was submitted in PR #477.