Question 1

How do I fix MTPLX returning 507 memory-limit errors on every retry when idle sessions are still holding KV cache?

Accepted Answer

Upgrade to MTPLX 2.12.1 or later (2.12.2 is the latest validated release), then restart the MTPLX server. For a uv/CLI installation, run `uv tool upgrade mtplx`; for the app installation, quit and relaunch MTPLX. Verify with `mtplx --version`.

In 2.12.1+, before refusing a request near the memory limit, the engine releases its own buffer pool and then evicts RAM copies of conversations that are not currently generating, choosing the oldest and SSD-saved conversations first. Active generations are never evicted. If memory is still unavailable, the 507 response identifies what is holding memory and when a retry may succeed.

Temporary workaround on older versions: stop the idle session or restart the MTPLX server to release its live KV cache, reduce configured context size, or use a smaller model. On 128 GB Macs, 2.12.1+ also raises the default memory headroom by setting the default limit to 90 GiB.

Question 2

Why does MTPLX crash with HTTP 507 / out-of-memory on long prompts with Qwen3.8-27B hybrids on a 64GB Mac, and how to fix it?

Accepted Answer

This is a known bug in MTPLX 2.11.3: for non-QSA hybrid models (Qwen3.8-27B family with linear_attention + full_attention layers), the memory planner priced the prefill transient at 0 bytes/token, so it admitted a 262K context window whose actual prefill peak measured ~79 GiB — exceeding the 64 GiB machine and triggering allocation failures and HTTP 507 for clients. The #393 fix had only covered Flash-Next/QSA geometry.

**Fix:** Upgrade to MTPLX 2.12.1. The memory check now prices every request at its largest moment, including full-width rows while 4-bit paged KV pages are filled (a 250K-token prompt previously projected at 7.6 GiB actually needed 20.1 GiB). Memory pressure trims now also reach the session cache: a trim halves what the cache actually holds instead of targeting half its budget (which caused 421 consecutive no-op trims), while sparing conversations that are actively generating.

**Expected side effect after upgrading:** the planner now advertises smaller, honest max windows — e.g. ~60K tokens on a 48GB MacBook — instead of an overcommitted 262K. This is correct behavior, not a regression.

**Workarounds on 2.11.3 (not a fix):** cap the window at or below the dense-decode ceiling (15% of RAM ÷ 65,536 bytes/token ≈ 157K tokens on 64 GiB), and set `MTPLX_SESSION_BANK_MAX_BYTES=4G`, `MTPLX_SESSION_BANK_MAX_ENTRIES=12`, `MTPLX_MEMORY_LIMIT_BYTES=54G`, `MTPLX_CLEAR_CACHE_EVERY=auto`. Deep prompts may still shed by design.

Question 3

How do I stop MTPLXApp from deleting Hermes config.yaml sections like memory, providers, delegation, and auxiliary on every launch?

Accepted Answer

This is caused by MTPLXApp 1.0.4 (build 10012) regenerating `~/.hermes/profiles/mtplx/config.yaml` from a fixed template and preserving only app-known keys. There is no official fix in that version. Do not use `chflags uchg`: it prevents the rewrite but makes MTPLX show "Messaging needs repair" and refuse to start sessions. A working band-aid is a macOS LaunchAgent watching the config file and merging the missing YAML sections back immediately after each rewrite. 1. Install yq: ```bash brew install yq ``` 2. Save your custom Hermes settings, for example `~/.hermes/profiles/mtplx/custom-config.yaml`: ```yaml memory: provider: honcho providers: {} delegation: {} auxiliary: {} model: max_tokens: 8192 ``` 3. Create `~/bin/restore-mtplx-config.sh`: ```bash #!/bin/bash set -euo pipefail CFG="$HOME/.hermes/profiles/mtplx/config.yaml" SNIP="$HOME/.hermes/profiles/mtplx/custom-config.yaml" TMP="$(mktemp)" [ -f "$CFG" ] || exit 0 [ -f "$SNIP" ] || exit 0 yq eval-all '. as $item ireduce ({}; . * $item)' "$CFG" "$SNIP" > "$TMP" if cmp -s "$TMP" "$CFG"; then rm -f "$TMP" else mv "$TMP" "$CFG" fi ``` Make it executable: ```bash chmod +x ~/bin/restore-mtplx-config.sh ``` 4. Create `~/Library/LaunchAgents/local.mtplx-config-restore.plist`, replacing `/Users/you` with your actual home directory: ```xml Label local.mtplx-config-restore ProgramArguments /bin/bash /Users/you/bin/restore-mtplx-config.sh WatchPaths /Users/you/.hermes/profiles/mtplx/config.yaml RunAtLoad ``` 5. Load it: ```bash launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/local.mtplx-config-restore.plist ``` If it is already loaded, use `launchctl bootout gui/$(id -u)/local.mtplx-config-restore` first, then bootstrap again. The permanent application fix is for MTPLXApp to round-trip the existing YAML and overwrite only its own managed keys instead of rebuilding the whole file. Until that is released, the WatchPaths agent preserves user-added Hermes sections across launches.

Question 4

How to fix "vision requests must not use the session bank without content-keyed ids" when sending an image after tool calls in MTPLX?

Accepted Answer

This is a bug in MTPLX 2.12.0: streaming chat completions fail when a vision/image message follows a tool-call/tool-result turn, while the same request works without streaming. As a workaround, send the request with `"stream": false` in `/v1/chat/completions`, or disable streaming in the OpenAI-compatible client/provider configuration if possible. Retrying with streaming enabled does not resolve it. Keep the image and tool history unchanged; only switch streaming off until a fixed MTPLX release is available.

Question 5

How do I fix mtplx "insufficient memory: prompt projects against the engine's 96.0 GiB limit" on a 128GB Apple Silicon Mac when RAM is still free?

Accepted Answer

The 96 GiB value is mtplx's preflight memory budget, not the amount of RAM currently in use. First, increase Apple's GPU wired-memory cap: `sudo sysctl iogpu.wired_limit_mb=124518`. Then restart mtplx with a smaller host reserve. Because the variable is named `*_BYTES`, use raw bytes instead of `6G`; for a 6 GiB reserve: `MTPLX_HOST_MEMORY_ALLOWANCE_BYTES=6442450944 mtplx serve --model Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-Speed --model-id mtplx-flash-next-optimized-quality --host 0.0.0.0 --port 8080 --api-key my-secret-key`. Verify that the engine limit reported after restart is higher than 96 GiB. If a request is still slightly over the estimate and you accept possible macOS swap and slower performance, add `--allow-swap` to the `mtplx serve` command. Alternatives are enabling q8 KV quantization, shortening the prompt, or starting a new conversation.

MTPLX

Core Features

What It Can't Do

Use Cases

Tags

Getting Started

Similar Projects