deer-flow

The gateway fails because os.getenv("GATEWAY_PORT", "8001") returns the default only when the variable is unset; an empty but defined variable becomes "", and int("") raises ValueError.

Immediate workaround: unset the variable or provide a valid numeric port:

unset GATEWAY_PORT
uv run uvicorn app.gateway.app:app --port 8001

Or explicitly set it:

GATEWAY_PORT=8001 GATEWAY_HOST=0.0.0.0 uv run uvicorn app.gateway.app:app --port 8001

For Docker, do not use -e GATEWAY_PORT=; either omit the variable or pass a value:

docker run -e GATEWAY_PORT=8001 ...

In Docker Compose, provide a default to prevent unset shell variables expanding to an empty string:

GATEWAY_PORT: ${PORT:-8001}
GATEWAY_HOST: ${GATEWAY_HOST:-0.0.0.0}

Permanent code fix in backend/app/gateway/config.py: treat only None and "" as unset, while preserving GATEWAY_PORT=0:

port_raw = os.getenv("GATEWAY_PORT")
port = 8001 if port_raw is None or port_raw == "" else int(port_raw)
host = os.getenv("GATEWAY_HOST") or "0.0.0.0"

Patch list_files_in_dir in backend/packages/harness/deerflow/uploads/manager.py to tolerate the filesystem race. After os.scandir(), wrap entry.stat(follow_symlinks=False) in try/except OSError; skip the entry for transient lookup errors such as ENOENT, ENOTDIR, and ELOOP, but re-raise operational errors like EACCES. Also handle the directory disappearing before or during os.scandir by returning the entries already collected.

Example:

try:
    st = entry.stat(follow_symlinks=False)
except OSError as e:
    if e.errno in (errno.ENOENT, errno.ENOTDIR, errno.ELOOP):
        continue
    raise

Do not rely on entry.is_file() to prevent this: on POSIX it may use the cached d_type from scandir without checking the filesystem again. Add a regression test that simulates stat() raising FileNotFoundError and verifies the vanished entry is skipped while the remaining files are returned.

On Python 3.13.16+ and 3.14.8+, asyncio.to_thread() can resume without yielding to the event loop. A callback queued from the worker with call_soon_threadsafe() may therefore still be pending when RunJournal.flush() checks the buffer. flush() sees an empty buffer, _detach_runtime_dependencies() closes the journal, and the later callback returns early on _closed, silently dropping the last middleware event.

Fix flush() by yielding once before draining, then re-checking the closed state:

async def flush(self):
    # Let cross-thread call_soon_threadsafe callbacks land on the owner loop.
    await asyncio.sleep(0)

    if self._closed:
        return

    # Existing buffer-drain/queue-join logic follows here.

Keep await journal.flush() before _detach_runtime_dependencies(). For the observability/audit path, similarly drain pending foreign-thread Usage/Audit enqueues before flush() joins the queue and fence shutdown before marking the worker closed.

Verify with:

cd backend
PYTHONPATH=. uv run pytest tests/test_run_journal.py::test_cross_thread_middleware_events_are_serialized_on_owner_loop

The regression is caused by the CPython _chain_future change from gh-105836/GH-141696. Python 3.12.15 forces the extra loop iteration, so pinning to Python 3.12 is a temporary workaround; adding the await asyncio.sleep(0) fence fixes 3.12, 3.13, and 3.14.

Upgrade to a release containing the fix for E2B issue #6528 (fix(e2b): quote grep filename filters). The bug occurs because E2BSandbox.grep() builds an unquoted shell argument such as --include=release *.txt, so the shell splits or expands it. When patching the adapter, shell-quote the complete argument, not just part of it: use shlex.quote(f"--include={glob}") before appending it to the command run with sh -c. Prefer passing an argv array such as ["grep", f"--include={glob}", ...] if the transport supports it. This preserves spaces, apostrophes, double quotes, and literal $ characters while retaining wildcard semantics. As a temporary workaround, use a glob without shell metacharacters, such as *.txt, and filter matching paths in Python.

The Helm chart's nginx ConfigMap (deploy/helm/deer-flow/templates/configmap-nginx.yaml) has no dedicated location for the Live Browser endpoint /api/threads/<thread_id>/browser/stream. The request falls into the generic threads location, which strips Upgrade: websocket and rewrites Connection to close, so Gateway receives a plain HTTP request and the WebSocket handshake never completes.

Fix: add a dedicated location block to the nginx template in configmap-nginx.yaml, matching the one already present in the Docker/local nginx configs:

location ~ ^/api/threads/[^/]+/browser/stream {
    proxy_pass http://gateway;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_read_timeout 3600s;
}

Then redeploy with helm upgrade (or re-render via helm template and restart the nginx pod). A successful handshake returns HTTP/1.1 101 Switching Protocols. Note that make dev and the Docker stack already include this location, so the bug only shows in Helm-deployed environments. Gateway still enforces authentication and thread ownership after the upgrade.