mlx

Upgrade MLX to 0.32.3 or newer; the bug is fixed by PR #3922:

pip install --upgrade mlx

Verify with:

import mlx.core as mx
print(mx.__version__)

If you cannot upgrade, use sorted_indices=False, or pad the row count to a multiple of 64 and discard the padded output rows. The issue affects MLX 0.31.2 on Apple M5 Max/macOS 27; M4 Pro is unaffected.

mx.fast.gated_delta_update cast the decay g and beta to the dtype of q (typically bf16) before running the kernel. Since bf16 has steps of 1/256 just below 1, decay values like 0.9995 round to exactly 1.0, so the Gated DeltaNet state stops decaying entirely at those tokens. This was fixed in PR #4760: g and beta are now passed to the kernels in float32 (matching flash-linear-attention and vLLM behavior), cutting output error from ~4.8e-2 to ~2.4e-3 versus a float64 reference. Upgrade to an MLX version containing that fix. On older versions, the only workaround is to pass q, k, and v in float32 as well, which is about 30% slower.

This is caused by macOS revoking MLX's wired-memory residency shortly after the GPU becomes idle; MLX 0.32.x requests residency only once and does not renew it. It is fixed by upgrading MLX to a release containing PR #4633, which re-requests residency with a heartbeat. As a temporary workaround, run:

sudo sysctl iogpu.disable_wired_collector=1

Verify it with:

sysctl iogpu.disable_wired_collector

The setting is cleared after reboot. Alternatively, keep the GPU active with a tiny MLX operation roughly once per second; a 3-second interval was not sufficient in tests. This affects MLX 0.32.x on macOS 26+, including Ollama's Apple Silicon MLX runner.

This is caused by MLX issue #4601: the Float16 NEON comparison fallback reads only lane 0 and broadcasts that mask to every lane. In mlx/backend/cpu/simd/neon_fp16_simd.h, code such as return Simd<uint16_t, N>(*(uint16_t*)(&out)); must be changed to retain the full NEON vector, typically return Simd<uint16_t, N>(out);. Apply this change to all Float16 comparison overloads and the logical-negation overload, then clean-rebuild MLX and your target. After the fix, the reproducer reports sigmoid(1) = 0.731445, close to the expected 0.731059, instead of 0.268799. If you cannot patch immediately, compute the activation in Float32 and cast back, for example sigmoid(x.astype(float32)).astype(float16), or use an MLX release containing the upstream fix.

This is caused by a 32-bit Metal grid overflow in MLX's column/non-last-axis reduction kernel. For an inner reduction stride of 2^29, the launch size becomes 256 * ceil(2^29 / 32) = 2^32, overflowing the dispatch dimension and producing zero or partially zero output. It affects mx.sum(x, axis=1) for shapes such as (1, 32, 4096, 131072) on MLX 0.32.0.dev20260609+cc3f3e60; last-axis reduction uses a different code path and works correctly.

Immediate workaround: move the reduced axis to the last axis and reduce there:

y = mx.sum(mx.moveaxis(x, 1, -1), axis=-1)
mx.eval(y)

For a generic axis:

y = mx.sum(mx.moveaxis(x, axis, -1), axis=-1)

Alternatively, add slices explicitly:

y = None
for i in range(x.shape[1]):
    block = x[:, i:i+1]
    y = block if y is None else y + block
mx.eval(y)

Permanent fix: upgrade to an MLX release containing the "Fix Metal column reduction grid overflow" fix for issue #3784, or build MLX from a commit that includes that PR.