OpenEnv
Frameworkhuggingface/OpenEnv
Framework for creating and using agentic execution environments for RL training.
Overview
OpenEnv provides a standard for interacting with agentic execution environments via Gymnasium-style APIs. It includes tools for environment creators and a CLI for deployment, enabling isolated, secure, and easy-to-use environments for RL post-training.
README Preview
# OpenEnv: Agentic Execution Environments\n\nAn e2e framework for creating, deploying and using isolated execution environments for agentic RL training, built using Gymnasium style simple APIs.\n\n\n \n \n \n \n \n \n\n\n---\n\n**Featured Example:** Train LLMs to play BlackJack using [torchforge](https://meta-pytorch.org/torchforge/) (PyTorch's agentic RL framework): [`examples/grpo_blackjack/`](examples/grpo_blackjack/)\n\n**Zero to Hero Tutorial:** End to end tutorial from our [GPU Mode](tutorial/README.md) lecture and other hackathons.\n\n## Quick Start\n\nInstall the OpenEnv package:\n\n```bash\npip install openenv\n```\n\nInstall an environment client (e.g., Echo):\n\n```bash\npip install git+https://huggingface.co/spaces/openenv/echo_env\n```\n\nThen use the environment:\n\n```python\nimport asyncio\nfrom echo_env import CallToolAction, EchoEnv\n\nasync def main():\n # Connect to a running Space (async context manager)\n async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as client:\n # Reset the environment\n result = await client.reset()\n print(result.observation.echoed_message) # "Echo environment ready!"\n\n # Send messages\n result = await client.step(\n CallToolAction(\n tool_name="echo_message",\n arguments={"message": "Hello, World!"},\n )\n )\n print(result.observation.result) # "Hello, World!"\n print(result.reward)\n\nasyncio.run(main())\n```\n\n**Synchronous usage** is also supported via the `.sync()` wrapper:\n\n```python\nfrom echo_env import CallToolAction, EchoEnv\n\n# Use .sync() for synchronous context manager\nwith EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as client:\n result = client.reset()\n result = client.step(\n CallToolAction(\n tool_name="echo_message",\n arguments={"message": "Hello, World!"},\n )\n )\n print(result.obseFAQ (5)
TroubleshootingWhy does MCPToolClient with mode="production" use WebSocket /ws instead of HTTP /mcp JSON-RPC?
The production transport flag is initialized incorrectly. In src/openenv/core/mcp_client.py, replace self.use_production_mode = False with:
self.use_production_mode = mode_lower == "production"This keeps the flag consistent with the normalized _mode value. As a temporary workaround, set it manually after creating the client:
client = MCPToolClient(..., mode="production")
client.use_production_mode = TrueAfter the fix, list_tools() should call tools/list over HTTP /mcp, call_tool() should call tools/call over /mcp, and simulation mode should continue using the WebSocket /ws step() path. Update regression tests to assert the selected transport rather than only checking for a type="step" message.
TroubleshootingHow do I fix MCPToolClient production mode using WebSocket instead of the HTTP /mcp JSON-RPC transport?
In src/openenv/core/mcp_client.py, replace the hard-coded self.use_production_mode = False in the MCPToolClient constructor with:
self.use_production_mode = mode_lower == "production"This makes list_tools() and call_tool() use the HTTP /mcp JSON-RPC methods tools/list and tools/call when mode="production", while simulation mode continues using the WebSocket step() path.
As a temporary workaround without changing the source code, set the flag immediately after initializing the client:
client = MCPToolClient(mode="production")
client.use_production_mode = TrueUpdate tests/core/test_mode_selection.py to assert that production mode calls /mcp with tools/list and tools/call, and that simulation mode still sends the WebSocket type="step" message.
TroubleshootingHow to fix Docker cleanup validation failing on Docker 29 and Podman because 'No such container' error text differs?
Do not rely on matching inspect or rm error text. Verify removal by listing the container and treating an empty result as success:
docker ps --all --filter "name=^/<container-name>$" --format '{{.Names}}'An empty result means the container is gone; a non-zero exit means the container engine is unreachable. For Podman, start containers with:
--stop-timeout 0so docker rm --force does not wait for the default stop timeout before SIGKILL. Also avoid asserting CapDrop == ["ALL"], because Podman reports dropped capabilities individually. Instead, verify that the container's capability bounding set (CapBnd) is empty; CapEff can be 0 for non-root users even when bounding capabilities remain. Add engine-specific unit tests for Docker 29 and Podman output.
informationalWhat is the status of OpenEnv RFC 005, and how does HarnessEnvironment differ from openenv.core.harness?
RFC 005 is still in effect and has been implemented on main in PRs #1097–#1100. For the RFC's multi-turn external-harness pattern, use HarnessEnvironment with AgenticHarnessAdapter. The external harness runs inside the environment, OpenEnv injects tools over MCP, and each step() call represents one conversational turn while the harness keeps context across turns in the episode. A worked Claude Code on τ²-bench example is available in the "Evaluate Claude Code in an Environment" tutorial and examples/claude_code_harness_eval.
The APIs introduced by #471/#652 under openenv.core.harness—such as ResourceSession, HarnessAdapter, MCPHarnessAdapter, CLIHarnessAdapter, and build_harness_rollout_func—are a separate session/runtime layer, not the RFC 005 implementation. Use them for lower-level session/resource management; use HarnessEnvironment/AgenticHarnessAdapter for RFC 005-style multi-turn agent-in-environment rollouts. Updated harness documentation organizes these options by which component owns the agent loop.
TroubleshootingHow do I fix the duplicated Python .venv layer that doubles Docker image size after openenv init?
The generated Dockerfile copies /app/env/.venv explicitly and then copies /app/env, which already contains .venv, creating two ~500 MB layers. Upgrade OpenEnv to a release containing the server Dockerfile template fix, or patch the generated Dockerfile manually. Remove the separate virtualenv copy, keep only the project copy, and create a symlink:
# Remove this line:
# COPY --from=builder /app/env/.venv /app/.venv
# Copy the project once, including its embedded .venv:
COPY --from=builder /app/env /app/env
# Provide compatibility for paths expecting /app/.venv:
RUN ln -s /app/env/.venv /app/.venvIf your template uses --chown, preserve those flags on the remaining COPY. Rebuild without cache and verify the layers:
docker build --no-cache -t your-image .
docker history your-image
# or
dive your-imageThe virtualenv should now appear in only one large layer, with the symlink adding negligible size.