ds4

This is a Metal FP32 numerical-test failure, not an inference crash: sqrt(softplus(x)) is computed as log(1.0f + exp(x)), which loses precision for logits around -12 on pre-M5 GPUs. Update to the latest main, which includes PR #1044, then rebuild and rerun the model-free test:

git checkout main
git pull origin main
make clean
make tests/test_deepseek41_metal && ./tests/test_deepseek41_metal

The merged fix replaces the inaccurate 1.0f + exp(x) form with an accurate four-term log1p series for exp(x) < 1/32 in all four DeepSeek router transform sites, clamps the polynomial input, and compares Metal output against the shared accurate reference without widening the tolerance. On an older commit, a temporary workaround is to use log1pf(expf(v)) in the host reference and widen the test tolerance, or change the Metal kernel to log1p(exp(x)).

The errors occur because external launchers run ds4-server with a different working directory, and relative GGUF/Metal paths are resolved from that directory. Use a wrapper script that changes to the ds4 installation directory first:

#!/bin/sh
set -e
cd /path/to/ds4
exec ./ds4-server "$@"

Make it executable:

chmod +x /path/to/ds4/start-ds4.sh

Then point llama-swap at the wrapper. If your ds4-server build does not provide /health, use /v1/models as the health-check endpoint:

models:
  "deepseek-v4-flash":
    cmd: /path/to/ds4/start-ds4.sh
    proxy: http://127.0.0.1:5555
    checkEndpoint: /v1/models

Alternatively, upgrade to a release containing the path-resolution fix: relative model and Metal asset paths are then resolved relative to the executable when they are not found in the current working directory, and GET /health is available for liveness checks.

On M5-class GPUs, the automatic Metal 4 TensorOps path can produce inaccurate matmul2d prefill accumulations, causing logits to differ from the simdgroup/reference kernels and flipping greedy argmax results. Disable the automatic TensorOps route by running with DS4_METAL_DISABLE_METAL4=1, for example:

DS4_METAL_DISABLE_METAL4=1 ./ds4 -m gguf/GLM-5.3-Flash-Q2.gguf --metal --temp 0 -n 2 --ctx 32768 --prompt-file prompt.txt

Alternatively, update to a build containing the fix that withholds automatic TensorOps enablement on M5 devices. DS4_METAL_ENABLE_TENSOR=1 remains an explicit opt-in only for performance testing and should not be used when parity is required. Changing operand staging precision or matmul2d_descriptor::relaxed_precision does not resolve the issue.

Verify the workaround with:

DS4_TEST_MODEL=gguf/GLM-5.3-Flash-Q2.gguf \
DS4_TEST_VECTOR_FILE=tests/test-vectors/glm-openrouter/official.vec \
  ./ds4_test --metal-tensor-equivalence

With Metal 4 TensorOps disabled, M5 output should be bit-identical to pre-M5 reference machines across logprob dumps, decode-consistency checks, and eval traces.

Build PR #621 of ds4, download the experimental GGUF from Hugging Face PR #22, and provide the matching Vision encoder sidecar.

1. Download the model and encoder:

mkdir -p gguf

hf download antirez/deepseek-v4-gguf \
  DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf \
  --revision refs/pr/22 \
  --local-dir gguf

hf download antirez/deepseek-v4-gguf \
  DeepSeek-V4-Flash-Vision-Encoder.gguf \
  --revision main \
  --local-dir gguf

2. Verify the language-model checksum. On macOS use shasum -a 256; on Linux use sha256sum. Expected SHA-256:

a1d0fde967c910e37e4bea00c4db1b2e8a88978770828a95772bc66837386cb8

3. Fetch and build the PR head:

git clone https://github.com/antirez/ds4.git ds4-vision-q4
cd ds4-vision-q4
git fetch origin pull/621/head:pr-621
git checkout pr-621
git rev-parse HEAD

# Apple Silicon / Metal
make clean && make

# Other NVIDIA CUDA systems
make clean && make cuda-generic

# DGX Spark / GB10
make clean && make cuda-spark

# AMD Strix Halo / ROCm
make clean && make strix-halo -j"$(nproc)"

4. Inspect the model first:

MODEL=gguf/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf
VISION=gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf

./ds4 --cpu --inspect -m "$MODEL"

Expected metadata includes GGUF v3, 1328 tensors, 43 layers, 215 q4_k tensors, 130 q8_0 tensors, and 78.62 GiB file size.

5. Run with SSD streaming:

# Metal
./ds4 --metal --ssd-streaming --ctx 4096 --temp 0 \
  -m "$MODEL" --vision "$VISION"

# CUDA
./ds4 --cuda --ssd-streaming --ctx 4096 --temp 0 \
  -m "$MODEL" --vision "$VISION"

# ROCm
./ds4 --rocm --ssd-streaming --ctx 4096 --temp 0 \
  -m "$MODEL" --vision "$VISION"

For fully resident testing, remove --ssd-streaming. The 78.62 GiB model, vision encoder, KV cache, and runtime buffers must all fit in available memory.

In the interactive CLI, load a PNG or JPEG with:

/read /path/to/test-image.png

Test at least one ordinary photograph, one text-containing screenshot, one diagram or chart, and a follow-up question. Report the exact commit, backend, streaming/resident mode, crashes, image-understanding errors, quality drift, and performance.

The error occurs because Kimi is pointed at the OpenAI-style /v1/models endpoint, whose response uses object and data fields. Kimi’s custom provider registry requires provider objects with id, name, api, type, and models.

Upgrade ds4 to the release/build containing the Kimi-compatible registry PR, then add the dedicated /api.json endpoint instead:

kimi provider add http://spark:8000/api.json

Do not use:

kimi provider add http://spark:8000/v1/models

The new endpoint also parses query-string parameters sent by Kimi. If /api.json is unavailable in your build, either update ds4 or serve a Kimi-compatible registry JSON containing the required provider fields and add that URL.