forge
SHA-256A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows, offering guardrails, proxy mode, and WorkflowRunner.
Smart Download
Download Download Version
v0.9.2 · 19.4 MB
A reliability layer for self-hosted LLM tool-calling that improves accuracy dramatically and works as a drop-in proxy.
Core Features
- Three usage modes: proxy server, WorkflowRunner, and guardrails middleware
- Automatic rescue parsing, retry nudges, and response validation with zero required steps
- Drop-in proxy speaking both OpenAI chat-completions and Anthropic Messages APIs
- Supports Ollama, llama-server, Llamafile, vLLM, and Anthropic backends
What It Can't Do
- •Not an agent orchestrator—does not handle multi-agent graphs or cross-agent coordination
- •Not a coding harness—use domain-specific agent frameworks, and let forge be the reliability proxy beneath
- •Anthropic benchmark numbers are from v0.6.0; not re-run for v0.7.0 due to cost
- •Local model quality depends on hardware and model choice; consult the model guide for best results
Use Cases
- Enhance existing harnesses like opencode, Continue, aider, or Claude Code with local model guardrails
- Build reliable multi-step tool-calling workflows with structured context management
Detailed Introduction
Forge is a Python framework that makes self-hosted LLM tool-calling and multi-step agentic workflows reliable. You give it a set of tools, and the model calls them in any order; structure is opt-in via required_steps, prerequisites, and terminal_tool. Its guardrails—rescue parsing, retry nudges, response validation—work even with zero required steps. Unlike agent orchestrators or coding harnesses, Forge sits inside one agentic loop, making tool calls rock-solid. It can be used as a drop-in proxy server (OpenAI and Anthropic compatible) for existing harnesses like opencode or Claude Code, as a WorkflowRunner for building directly, or as guardrails middleware inside your own loop. Supports OpenAI-compatible endpoints, Ollama, llama-server, Llamafile, vLLM, and Anthropic backends. Compared to generic agent frameworks, Forge focuses purely on reliability, delivering measurable accuracy gains on 8B local models and even improving frontier models like Claude Sonnet 4.6.
Tags
Getting Started
Download installer
Click the button above to download the installer for your system
Install the software
Double-click the downloaded installer and follow the prompts
Install via pip: pip install forge-guardrails
Start a local LLM backend (llama-server, Ollama, etc.)
Run the proxy: python -m forge.proxy --backend-url http://localhost:8080 --port 8081
- Install via pip: pip install forge-guardrails
- Start a local LLM backend (llama-server, Ollama, etc.)
- Run the proxy: python -m forge.proxy --backend-url http://localhost:8080 --port 8081
SHA-256 checksum verified
Checksum extracted from GitHub official Release page
SHA256 Checksum
c49654ccac62c51335e501f3a491d8f39007c8c25790b7a85ce0f251a5a1d252This checksum is extracted from the GitHub Release page. Verify file integrity after download.
All SHA-256 checksums on this platform are extracted from the project's official GitHub Release page, without any modification. You can independently verify them on the GitHub Releases page.
Open Source Transparency
View GitHub SourceUninstall Info
If installed via pip, run 'pip uninstall forge-guardrails'. For the standalone proxy, delete the binary and configuration directory (e.g., ~/.forge-proxy).
No Extra Dependencies
Ready to use after download. No additional runtime required.
Similar Projects
ragflow
RAGFlow is a leading open-source RAG engine that fuses deep document understanding with agentic workflows, offering precise, citation-grounded answers. Unlike LangChain's DIY approach, RAGFlow provides a turnkey, enterprise-ready platform with a visual interface and template-based chunking, reducing hallucinations and setup complexity.
unsloth
Unsloth is the first desktop app to run and train AI models locally. It supports a wide range of models including Qwen3.8, Kimi K3, MiniMax-H3, Gemma 4, DeepSeek-V4, and FLUX. Unlike most local model tools that focus only on inference, Unsloth adds a full training stack: fine-tune LLMs, diffusion, TTS, and embedding models 2× faster with 70% less VRAM. It also supports LoRA, QLoRA, RL, GRPO, DPO, and FP8. The app includes built-in Agents & Tools integration (Claude Code, Codex, MCP), private search and RAG, image/video diffusion, and audio support. With hardware support for CPU, NVIDIA, AMD, Intel, macOS, and multi-GPU setups, plus remote access via Cloudflare, Unsloth is a comprehensive local AI studio. It can export to GGUF, NVFP4, FP8 and offers an OpenAI-compatible API for easy deployment.
LocalAI
LocalAI is the open-source AI engine to run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. Drop-in API compatibility with OpenAI, Anthropic, and ElevenLabs.