miles
Frameworkradixark/miles
Enterprise-grade RL framework for large-scale LLM/VLM post-training.
Overview
Miles is an enterprise-grade reinforcement learning framework for LLM and VLM post-training, forked from slime. It combines SGLang for high-throughput rollout with Megatron-LM for scalable training, featuring fully async RL, fast weight updates, and low-precision training.
README Preview
\n\n\n\n### **Enterprise-Grade Reinforcement Learning for Large-Scale Model Post-Training**\n\n[](https://miles.radixark.com/)\n[](https://github.com/radixark/miles)\n[](https://miles.radixark.com/docs)\n[](LICENSE)\n[](https://slack.sglang.ai)\n\n| [**Website**](https://miles.radixark.com/) | [**Documentation**](https://miles.radixark.com/docs) | [**Quick Start**](https://miles.radixark.com/docs/getting-started/quick-start) | [**Supported Models**](https://miles.radixark.com/docs/models) | [**Miles Diffusion**](https://github.com/radixark/miles_diffusion) | [**Blog**](https://www.lmsys.org/blog?filter=miles) | [**Slack**](https://slack.sglang.ai) (`#miles-rl`) |\n\n\n\n--------------------------------------------------------------------------------\n\n## News\n\n- [2026/08] 🔥 Miles v0.1 is released! Read the blog post here: [Miles v0.1: Production-level Post-training](https://www.lmsys.org/blog/2026-08-18-miles-v0-1).\n- [2026/07] Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles ([blog](https://www.lmsys.org/blog/2026-07-29-mxfp8-nvfp4-rl)).\n- [2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 ([blog](https://www.lmsys.org/blog/2026-07-27-kimi-k3-day0-support)).\n- [2026/07] On-policy distillation lands in Miles ([blog](https://www.lmsys.org/blog/2026-07-18-opd-support-in-miles)).\n- [2026/07] 🔥 SGLang and Miles add day-0 support for Inkling, a frontier multimodal model ([blog](https://www.lmsys.org/blog/2026-07-15-inkling-day0-support)).\n- [2026/07] DeepSeek-V4 Flash RL training comes to AMD Instinct MI355X with Miles ([blog](https://www.lmsys.org/blog/2026-07-10-rocm-miles-dsv4)).\n- [2026/06] SGLang and Miles add day-0 support for NVIDIA Nemotron 3 Ultra ([blog](https://www.lmsys.org/blog/2026-06-04-nvidia-run-nemotron-3-ultra)).\n- [2026/05] No token left behind: token-in-token-out in Miles ([blog](https://www.lmsys.org/blog/2026-05-13-no-token-left-behind)).\n- [2026/04] Updating 1 T parameters in seconds: P2P we
FAQ (2)
TroubleshootingHow to fix SGLang router 'Invalid address: invalid socket address syntax' error when launching miles on a Slurm/Ray cluster?
This happens because sglang_router.launch_router --host only accepts IP literals (it parses into a Rust SocketAddr), but on Slurm-managed clusters Ray reports the node address as a hostname (e.g. node1234), so the router exits before binding and the run later fails with 'Server at <host>:<port> not ready after 120s'.
Fix: upgrade miles to a revision that includes PR #3356 (commit 9fecf0ff), which resolves the placed hostname to an IP literal for the router's bind address while keeping the hostname for connect-side URLs and distributed init. Verified working on GB200/GB300 Slurm setups with sglang-router 0.3.2.
Workaround on older revisions: make Ray advertise an IP literal instead of a hostname, i.e. start Ray with --node-ip-address=<node-IP> rather than $(hostname). Note that --miles-host-ip 0.0.0.0 is not a valid general workaround.
TroubleshootingHow to fix 'Process group MagicMock mock.cpu_group is not initialized in the world group map' error during Miles p2p weight-sync on Qwen3-235B-A22B (MoE)?
The crash happens because Miles' _create_cpu_replica() (in update_weight_from_distributed/p2p.py) builds a throwaway CPU-only replica to derive sharding layout, stubbing tp_group.cpu_group as a MagicMock. SGLang model construction then initializes MultimemAllGatherer (triton_symm_mem_ag), which probes the topology via torch.distributed.get_backend(tp_group.cpu_group) and fails on the unregistered mock group. Workarounds: 1) Apply the fix from PR #2605 — override the local P2P structural replica only with nnodes=1 (on current main, via get_context().override(...)). This short-circuits the MultimemAllGatherer topology probe while preserving TP/EP sharding geometry; note it is not yet fully validated for the 235B MoE multi-node run, and remaining MoE staging/layout limitations are documented in #2605. 2) Use broadcast weight-sync mode instead of p2p — Qwen3-235B-A22B broadcast completed successfully in the same test matrix.