outlines
Librarydottxt-ai/outlines
Guarantees structured outputs from large language models.
Overview
Outlines is a Python library that ensures LLMs produce outputs conforming to a specified structure, such as JSON, regex, or Pydantic models. It supports multiple model backends and integrates seamlessly with existing workflows.
README Preview
\n\n\n\n\n\n 🗒️ *Structured outputs for LLMs* 🗒️\n\nMade with ❤👷️ by the team at [.txt](https://dottxt.co)\nTrusted by NVIDIA, Cohere, HuggingFace, vLLM, etc.\n\n\n[![PyPI Version][pypi-version-badge]][pypi]\n[![Downloads][downloads-badge]][pypistats]\n[![Stars][stars-badge]][stars]\n\n\n[![Discord][discord-badge]][discord]\n[![Blog][dottxt-blog-badge]][dottxt-blog]\n[![Twitter][twitter-badge]][twitter]\n\nThe .txt API is currently in early access. **[Request access here →](https://h1xbpbfsf0w.typeform.com/to/fwQNWmS8?utm_source=github&utm_medium=organic&utm_campaign=outlines)**\n\n\n\n\n## 🚀 Building the future of structured generation\n\nWe're working with select partners to develop new interfaces to structured generation.\n\nNeed XML, FHIR, custom schemas or grammars? Let's talk.\n\nAudit your schema: share one schema, we show you what breaks under generation, the constraints that fix it, and compliance rates before and after. Sign up [here](https://h1xbpbfsf0w.typeform.com/to/rtFUraA2?typeform).\n\n## Table of Contents\n\n- [Why Outlines?](#why-outlines)\n- [Quickstart](#quickstart)\n- [Real-World Examples](#real-world-examples)\n - [🙋♂️ Customer Support Triage](#customer-support-triage)\n - [📦 E-commerce Product Categorization](#e-commerce-product-categorization)\n - [📊 Parse Event Details with Incomplete Data](#parse-event-details-with-incomplete-data)\n - [🗂️ Categorize Documents into Predefined Types](#categorize-documents-into-predefined-types)\n - [📅 Schedule a Meeting with Function Calling](#schedule-a-meeting-with-function-calling)\n - [📝 Dynamically Generate Prompts with Re-usable Templates](#dynamically-generate-prompts-with-re-usable-templates)\n- [They Use Outlines](#they-use-outlines)\n- [Model Integrations](#model-integrations)\n- [Core Features](#core-features)\n- [Other Features](#other-features)\n- [About .txt](#about-txt)\n- [Community](#community)\n\n\n\n## Why Outlines?\n\nLLMs are powerful but their outputs are unpredictable
FAQ (5)
TroubleshootingHow to fix stale cache hits or wrong regex output when changing JsonSchema whitespace_pattern in Outlines?
JsonSchema.__eq__ compares only the parsed schema JSON and omits whitespace_pattern, so schemas with different patterns are treated as identical and cache/deduplication can reuse the wrong logits processor. Fix equality to compare the pattern as well:
def __eq__(self, other):
if not isinstance(other, JsonSchema):
return False
try:
self_dict = json.loads(self.schema)
other_dict = json.loads(other.schema)
return (
self_dict == other_dict
and self.whitespace_pattern == other.whitespace_pattern
)
except json.JSONDecodeError:
return (
self.schema == other.schema
and self.whitespace_pattern == other.whitespace_pattern
)
Until patched, avoid caching processors by JsonSchema alone; use a cache key such as (schema.schema, schema.whitespace_pattern) or clear the relevant cache when the pattern changes.
TroubleshootingWhy is the API Reference empty after migrating a Python package to a src layout with MkDocs?
Update scripts/gen_ref_pages.py so mkdocs-gen-files scans the new package location. Replace the old path root / "outlines" with root / "src" / "outlines". After the change, rebuild the docs with MkDocs and verify that pages such as site/api_reference/models/index.html are generated. The issue occurs because the src-layout migration moved the package from outlines/ to src/outlines/, leaving the old glob empty.
TroubleshootingWhy does Outlines' json_schema_dict_to_dataclass generate `age: int = None` for optional fields and how do I fix it?
In src/outlines/types/json_schema_utils.py, json_schema_dict_to_dataclass assigns None as the default for non-required fields but does not wrap their type in Optional. Update the converter so non-required properties use Optional[typ]:
from typing import Optional
for property, details in properties.items():
typ = schema_type_to_python(details, "dataclass")
if property not in required:
typ = Optional[typ]
defaults[property] = None
annotations[property] = typ
This makes an optional JSON Schema integer generate age: Optional[int] = None, matching the Pydantic converter and allowing null in structured generation. As a temporary workaround, manually declare non-required dataclass fields as Optional[T] = None, or use convert_to(schema, "pydantic") until the fix is applied.
TroubleshootingHow to fix Outlines `Literal[1.5]` incorrectly matching strings like `"1x5"`?
The numeric value is converted to a regex without escaping, so the . in 1.5 acts as a regex wildcard. In src/outlines/types/dsl.py, change numeric term construction from Regex(str(value)) to Regex(re.escape(str(value))), ensuring re is imported:
import re
# Inside python_types_to_terms handling for int/float:
term = Regex(re.escape(str(value)))
This produces 1\.5, so Literal[1.5] matches only "1.5" and rejects "1x5". Until the fix is released, use an explicitly escaped regex such as Regex(r"1\.5") instead of a float literal constraint.
TroubleshootingHow do I fix outlines.types.ipv6 accepting invalid leading-zero IPv4-mapped IPv6 addresses?
The ipv6 regex reuses the old IPv4 octet pattern, so suffixes such as ::ffff:01.1.1.1 or ::ffff:00.0.0.0 incorrectly match.
Fix it in src/outlines/types/__init__.py by replacing both occurrences of the old octet pattern in the ipv6 definition:
Old pattern:25[0-5]|2[0-4][0-9]|1?[0-9]{1,2}
New pattern:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]
This must be updated in both IPv4-mapped branches:
- ::ffff:a.b.c.d
- x:x:x:x::a.b.c.d
The new pattern accepts canonical octets such as 0, 1, 42, and 192, but rejects leading-zero forms such as 00, 01, and 007.
As a temporary workaround before patching, validate generated addresses with Python's standard library:
import ipaddress
def is_valid_ip(value: str) -> bool:
try:
ipaddress.ip_address(value)
return True
except ValueError:
return False
After applying the regex fix, ::ffff:192.168.1.1 and 1:2:3:4::5.6.7.8 still match, while ::ffff:01.1.1.1, ::ffff:00.0.0.0, and 1:2:3:4::5.6.007.8 are rejected.