whichllm
CLI ToolAndyyyy64/whichllm
Finds the best local LLM for your hardware using real benchmarks.
Overview
Auto-detects GPU/CPU/RAM, ranks models from HuggingFace by benchmark quality and speed. Supports simulation for hardware planning, JSON output, and instant chat with selected models.
README Preview
# whichllm\n\n[](https://pypi.org/project/whichllm/)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n[](https://github.com/Andyyyy64/whichllm/actions/workflows/test.yml)\n[](https://github.com/sponsors/Andyyyy64)\n\n**Find the best local LLM that actually runs on your hardware.**\n\nAuto-detects your GPU/CPU/RAM and ranks the top models from HuggingFace that fit your system.\n\n[日本語版はこちら](docs/README.ja.md)\n\n## Quick start\n\nRun the recommendation command once, with no project setup.\n\n```bash\nuvx whichllm@latest\n```\n\nSimulate a GPU before you buy hardware.\n\n```bash\nuvx whichllm@latest --gpu "RTX 4090"\n```\n\nInstall it when you use it often.\n\n```bash\nuv tool install whichllm\nuv tool upgrade whichllm # update an existing install\n```\n\nOther install paths.\n\n```bash\nbrew install andyyyy64/whichllm/whichllm\npip install whichllm\n```\n\n## Common workflows\n\nAfter install, run `whichllm` directly. For one-off runs, replace `whichllm`\nwith `uvx whichllm@latest`.\n\n```bash\n# Best models for this machine\nwhichllm\n\n# Pretend you have a specific GPU\nwhichllm --gpu "RTX 4090"\n\n# Compare upgrade candidates\nwhichllm upgrade "RTX 4090" "RTX 5090" "H100"\n\n# Find the GPU needed for a model\nwhichllm plan "llama 3 70b"\n\n# Start a chat with a model\nwhichllm run "qwen 2.5 1.5b gguf"\n\n# Print copy-paste Python\nwhichllm snippet "qwen 7b"\n\n# Return JSON for scripts\nwhichllm --top 1 --json\n```\n\n\n\n## See it\n\n```text\n$ whichllm --gpu "RTX 4090"\n\n#1 Qwen/Qwen3.6-27B 27.8B Q5_K_M score 92.8 27 t/s\n#2 Qwen/Qwen3-32B 32.0B Q4_K_M score 83.0 31 t/s\n#3 Qwen/Qwen3-30B-A3B 30.0B Q5_K_M score 82.7 102 t/s\n```\n\nThe 32B model **fits your card fine** — whichllm still ranks the 27B #1,\nbecause it scores higher on real benchmarks and is a newer generation.\nA size-only "what fits?" tool would hand you the bigger one. That gap is\nthe whole point of whichllm. (Note #3: a MoE model
FAQ (5)
TroubleshootingHow do I fix whichllm saying “No model found matching” for a specific Hugging Face repo ID?
Upgrade whichllm to v0.5.18 or later, which adds exact Hugging Face repo ID lookup:
python -m pip install -U "whichllm>=0.5.18"If installed via Homebrew, upgrade the whichllm tap/formula, then run:
whichllm plan ilsp/Llama-Krikri-8B-InstructFor JSON output:
whichllm plan ilsp/Llama-Krikri-8B-Instruct --jsonWhen you pass an exact owner/repo ID, v0.5.18+ fetches that repository directly from Hugging Face if it is not in the cached model catalog. It also respects the HF_ENDPOINT environment variable.
TroubleshootingWhy is Qwen3.8-27B missing from whichllm recommendations and how do I fix it?
Upgrade whichllm to **v0.5.18 or later**. Older releases excluded Qwen/Qwen3.8-27B because its Hugging Face image-text-to-text tag was treated as a vision-only model, collapsed Qwen minor versions into one qwen3-27b family, and used stale benchmark/lineage data that ranked Qwen3.6 higher.
Run the fixed release with:
uvx whichllm@0.5.18or:
uvx whichllm@latestOn v0.5.18, checkpoint versions and organization identity are preserved during grouping, and newer Qwen releases are included. In the maintainer's M2 Max 96 GB comparison, Qwen3.8-27B moved from absent in the top ten to second place. On older versions, using the vision or any profile/include_vision=True only works around the fetch filter; the ranking and grouping issues still require the upgrade.
TroubleshootingHow do I fix whichllm crashing with UnicodeDecodeError when reading models.json or benchmarks.json?
Upgrade whichllm to version 0.5.17 or newer, which treats a non-UTF-8 cache file as a corrupted cache and refreshes it instead of crashing:
python -m pip install --upgrade whichllmIf installed through Homebrew:
brew update
brew upgrade whichllmAs a temporary workaround on older versions, delete the corrupted models.json and/or benchmarks.json files from the whichllm cache directory, then run whichllm again so the cache is rebuilt.
TroubleshootingHow do I fix HTTP 429 rate-limit errors when paginating the archived Open LLM Leaderboard during benchmark collection?
Upgrade to whichllm v0.5.20 or later. This release honors numeric and HTTP-date Retry-After headers, uses source-specific bounded retry settings for the archived Open LLM Leaderboard, and preserves already-fetched pages if a later page is still rate-limited after retries. Rerun the collection; partial results are cached for 24 hours, and a cached run will not repeat the fetch warning. Use --refresh to force fetching scores again. If the first leaderboard page fails with 429, the source still fails because no pages have been retained yet.
TroubleshootingHow do I fix outdated AA Intelligence Index scores and rankings after AA reworked its index?
Upgrade to **whichllm v0.5.20 or later**. v0.5.15 retuned _AA_INDEX_MIN/_AA_INDEX_MAX and refreshed the AA fallback snapshot for the reworked scale; v0.5.20 completed the fix by letting finite live AA scores replace stored scores even when they decrease or become zero. Models absent from the live response retain their stored score. After upgrading, run with --refresh to bypass any existing benchmark cache and force fresh data: whichllm --refresh (or the equivalent command for your installation).