项目简介
bitnet.cpp 是微软官方的 1-bit 大模型推理框架，支持 BitNet b1.58 等三值模型在 CPU 与 GPU 上快速、无损推理。其优化内核在 x86 和 ARM 平台上带来显著加速与能耗降低，甚至可在单颗 CPU 上运行千亿参数模型，推动本地设备部署大模型。
README 预览
\n\n# bitnet.cpp\n\n[](https://opensource.org/licenses/MIT)\n\n[](https://huggingface.co/collections/microsoft/bitnet)\n[](https://arxiv.org/abs/2502.11880)\n[](https://demo-bitnet-h0h8hcfqeqhrf5gf.canadacentral-01.azurewebsites.net/)\n[](https://github.com/microsoft/BitNet/blob/main/gpu/README.md)\n\n\n\n\n\n📰 News\n\n07/23/2026: 📣 We released VibeASR.cpp — a real-time multilingual ASR inference engine on CPU using BitNet I2_S quantization, achieving RTF Code] [Models] [Report] \n\n07/20/2026: 📣 We released BitNet-embedding-0.6B and BitNet-embedding-270M on Hugging Face — the first 1-bit embedding models that deliver competitive embedding quality with significantly faster inference on CPUs.\n- **1.42x to 2.28x speedup** over F16 on BitNet-embedding-0.6B prefill (8 threads)\n- **1.32x to 1.74x speedup** over F16 on BitNet-embedding-270M prefill (8 threads)\n- Supports I2_S conversion with optimized kernels on x86 CPUs\n- Lossless inference with 2 bits per weight\n\n07/16/2026: 📣 Released [BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization](docs/bitnet-embeddings-i2s-guide.md) — detailed guide for converting and running BitNet embedding models with optimized I2_S kernels.\n\n01/15/2026: 📣 Released [BitNet CPU Inference Optimization](https://github.com/microsoft/BitNet/blob/main/src/README.md) — parallel kernel implementations with configurable tiling and embedding quantization support, achieving **1.15x to 2.1x** additional speedup over the original implementation.\n\n05/20/2025: 📣 Released [BitNet Official GPU inference kernel](https://github.com/microsoft/BitNet/blob/main/gpu/README.md) — extending 1-bit inference beyond CPUs.\n\n04/14/2025: 📣 Released [BitNet Official 2B Parameter Model](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T) on Hugging Face — the first official BitNet b1.58 model trained with 4T tokens.\n\n02/18/2025: 📑 [Bitnet.cpp: Efficient Edge Inference for Ternary LLMs](https://arxiv.org/abs/2502.11880) — system-lev
常见问题 (1)
故障排除如何修复在 Windows 上构建 BitNet 后缺少 llama-cli.exe（FileNotFoundError WinError 2）的问题？
故障排除
如何修复在 Windows 上构建 BitNet 后缺少 llama-cli.exe（FileNotFoundError WinError 2）的问题？构建成功，但 llama-cli.exe 从未生成，因为 llama.cpp 是作为 Git 子模块引入的，这会导致 CMake 默认设置 LLAMA_BUILD_EXAMPLES=OFF（CLI 被视为"示例"目标）。修复方法：在构建之前重新运行 CMake 配置步骤并启用该标志：
cmake -B build -DLLAMA_BUILD_EXAMPLES=ON .
cmake --build build --config Release
或者，也可以只显式构建 CLI 目标：cmake --build build --config Release --target llama-cli。重新构建后，llama-cli.exe 将出现在 build/bin/Release 中，并且 python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "Test" -cnv 将可以正常运行。针对 setup_env.py 的永久修复已在 PR #477 中提交。