open-notebook

Fix both issues in the source chat streaming and session-load paths.

1. Stop sending the full checkpointed message history during streaming. In api/routers/source_chat.py, around lines 376–384, do not iterate over every entry in result["messages"]. Emit only the latest AI message:

last_message = result["messages"][-1]
# Emit the ai_message event for last_message only

2. Restore context indicators when an existing session is loaded. In frontend/src/lib/hooks/use-source-chat.ts, update the session-load effect around lines 41–45:

setContextIndicators(currentSession.context_indicators ?? null);

The backend already returns context_indicators from GET session in api/routers/source_chat.py; the hook was only setting them from the SSE event. After these changes, subsequent streamed answers contain only the new answer, and the source count remains visible after a page refresh. Verified against main commit 3959eb0.

Batched repo_insert calls are not atomic: if a batch after the first one fails, earlier source_embedding rows remain. Because the embedded check only looks for one row (LIMIT 1), the source is incorrectly marked as fully embedded. Fix embed_source in commands/embedding_commands.py by wrapping the insert in try/except, deleting the source's embeddings on failure, and re-raising the original exception:

try:
    await repo_insert(...)
except Exception:
    await db.execute(
        "DELETE FROM source_embedding WHERE source = $source_id",
        {"source_id": source_id},
    )
    raise

Add a test that forces a later batch to fail, then assert that source_embedding contains zero rows for the source and the original exception is propagated unchanged.

The empty string is being removed before the PATCH request. In frontend/src/app/(dashboard)/notebooks/components/NotebookHeader.tsx, replace description: description || undefined with description: description. Apply the same change in frontend/src/app/(dashboard)/transformations/components/TransformationEditorDialog.tsx for the title and description fields. On the backend, update api/routers/notebooks.py and api/routers/transformations.py to distinguish an omitted field from a field explicitly set to ""; use Pydantic's model_fields_set or model_dump(exclude_unset=True) instead of checking only is not None. Verify with a PATCH containing description: "", which should clear the field, while omitting description should leave the existing value unchanged.

Use content-core 2.2 or later and configure YouTube cookies plus a residential proxy. Export YouTube cookies in Netscape cookies.txt format, mount the file into the container, and set:

-e CCORE_YOUTUBE_COOKIES_FILE=/run/secrets/cookies.txt \
-e CCORE_YOUTUBE_PROXY=http://user:password@residential-proxy:port \
-v /path/to/cookies.txt:/run/secrets/cookies.txt:ro

The proxy is used for transcript retrieval, the pytubefix fallback, and title lookup. Use a residential proxy; datacenter proxies are blocked. A missing or unreadable cookies file causes a ConfigurationError, so verify the mounted path and file permissions, then restart the worker/container.

With esperanto 2.28+, to_langchain() enforces ESPERANTO_LLM_TIMEOUT for every provider. If it is unset, the effective fallback is 60 seconds, so long responses or slow local models can fail. Upgrade to the Open Notebook release that defaults this value to 180 seconds, or set it explicitly before starting the API and worker:

export ESPERANTO_LLM_TIMEOUT=180

For slow local models, use a larger value and increase API_CLIENT_TIMEOUT as well, keeping it above the LLM timeout plus a buffer:

export ESPERANTO_LLM_TIMEOUT=600
export API_CLIENT_TIMEOUT=720

In code, the default can be applied before any model is created with:

os.environ.setdefault("ESPERANTO_LLM_TIMEOUT", "180")

An explicitly set ESPERANTO_LLM_TIMEOUT always takes precedence. STT/TTS timeout variables are separate and remain at esperanto's 300-second default.