agentmemory

Upgrade AgentMemory to v0.9.26 or later, which fixes the sharded index manifest loading for adapters that return undefined for missing keys:

npm install -g @agentmemory/agentmemory@0.9.26

Then run AgentMemory again. In v0.9.25, the error is cosmetic on first run: the index is rebuilt automatically shortly after startup by the debounced save process, so it can safely be ignored if you cannot upgrade immediately.

On agentmemory v0.9.24 with a fresh global install, override the runtime version check to use the already installed, consistent v0.16.1 engine and worker set:

AGENTMEMORY_III_VERSION=0.16.1 agentmemory

For npx, run:

AGENTMEMORY_III_VERSION=0.16.1 npx @agentmemory/agentmemory@latest

To make it persistent for the current shell:

export AGENTMEMORY_III_VERSION=0.16.1
agentmemory

On Ubuntu 24.04 aarch64, this boots successfully with iii-engine v0.16.1 and reports a healthy worker/REST service. Avoid blindly following the suggested v0.11.2 downgrade tarball: its iii-aarch64-unknown-linux-gnu.tar.gz contains only the iii binary, leaving v0.11.2 iii paired with v0.16.1 iii-worker, which creates another version mismatch. The permanent fix is for the installer to fetch the pinned engine version or for agentmemory to update its pin to v0.16.1.

In @agentmemory/agentmemory@0.9.27, getVisibleTools() in src/mcp/tools-registry.ts reads process.env.AGENTMEMORY_TOOLS directly instead of the merged env (loadEnvFile() + process.env), so values set only in ~/.agentmemory/.env are silently ignored and all 53 tools are exposed instead of the 8 core tools. Workarounds until the patch lands: (1) launch with the CLI flag agentmemory --tools core, or (2) export the variable in the launching shell, e.g. AGENTMEMORY_TOOLS=core agentmemory or configure it in your pm2 ecosystem file's env block, then pm2 restart agentmemory. Verify with (Invoke-RestMethod 'http://127.0.0.1:3111/agentmemory/mcp/tools').tools.Count — it should return 8. The permanent fix is changing process.env["AGENTMEMORY_TOOLS"] to getEnvVar("AGENTMEMORY_TOOLS") in getVisibleTools() and in src/mcp/standalone.ts.

This affects @agentmemory/agentmemory 0.9.22 through 0.9.24 when AGENTMEMORY_ALLOW_AGENT_SDK=true. The agent-sdk recursion guard sets the global process.env.AGENTMEMORY_SDK_CHILD, so concurrent summarize chunks see the flag and return an empty response, which is counted as a parse failure. As a workaround, force chunk summarization to run serially by adding SUMMARIZE_CHUNK_CONCURRENCY=1 to ~/.agentmemory/.env, then restart the worker:

echo 'SUMMARIZE_CHUNK_CONCURRENCY=1' >> ~/.agentmemory/.env

For a temporary shell run:

SUMMARIZE_CHUNK_CONCURRENCY=1 AGENTMEMORY_ALLOW_AGENT_SDK=true node your-worker.js

Then retry summarization. The permanent code fix is to replace the process-global env recursion guard with an in-process mechanism such as AsyncLocalStorage or a per-call flag, while keeping the env guard only for actual child-process boundaries.

Your reproduced numbers are correct; the published scorecard is wrong. In eval/runner/score.ts, Precision@5 is calculated per query as:

const topK = ranked.slice(0, k).map((r) => r.sessionId);
const gold = new Set(q.goldSessionIds);
const hits = topK.filter((id) => gold.has(id)).length;
const precisionAtK = k > 0 ? hits / k : 0;
const recallAtK = gold.size === 0 ? 0 : hits / gold.size;

The adapter score is the arithmetic mean across all 15 questions.

The coding-agent-life-v1 dataset contains 12 questions with one gold session and 3 questions with two gold sessions. Therefore the maximum possible P@5 is:

(12 × (1/5) + 3 × (2/5)) / 15 = 18/75 = 0.240

So:
- agentmemory-hybrid P@5=0.240, R@5=1.000 means every gold session was retrieved in the top 5.
- grep P@5=0.227, R@5=0.967 corresponds to 17 successful hits out of a maximum 18 top-5 hits; one two-gold question returned only one relevant session. Its macro-averaged recall is (14 + 0.5) / 15 = 0.9667.

Workaround: recalculate or regenerate the benchmark report using the current formula and replace the incorrect published P@5 value; any score above 0.240 for this dataset is impossible under this metric definition.