turbovec

Upgrade to TurboVec 1.1.1 or newer. This release includes fixes #556 and #558 for masked single-query searches on the staged AVX-512/ARM route used for indexes of 32,768+ vectors. As a temporary workaround on 1.1.0, force the classic kernels with:

export TURBOVEC_4BIT_PLANES=0
./your_search_binary

Or for one command:

TURBOVEC_4BIT_PLANES=0 ./your_search_binary

The regression occurred because the 1.1.0 staged route applied masks through a slow per-lane scalar path during sign scan and rescoring; selective masks could make a masked search slower than an unmasked scan.

This is a livelock in lock_write_gil_aware, not normal lock contention. It calls try_write(), and on WouldBlock acquires a blocking write lock inside py.detach() but immediately drops it before retrying. Concurrent searchers can reacquire the read lock first, so deletion may repeatedly lose the race.

Fix the swap_remove and IdMapIndex.remove call sites by performing the deletion inside one detached closure using the ordinary blocking lock_write:

py.detach(|| {
    let _guard = lock_write(&lock);
    // perform swap_remove/remove here; the guard is !Send
});

Do not acquire-and-drop the lock. If preserving the uncontended fast path matters, use one attached try_write(); if it returns WouldBlock, fall back to exactly one detached blocking acquire and execute the operation inside that closure. This matches the already-bounded add path.

A patched build reduced 8 concurrent swap_remove calls from 86.1 s total / 25.8 s worst to 1.13 s total / 144 ms p50. Upgrade to a release containing the fix for issue #484, or apply that PR. As a temporary workaround, serialize deletions with searches or pause search threads before calling swap_remove()/remove().

Upgrade TurboVec to 1.1.2, which fixes the staged-search recall regression on tightly clustered embeddings such as MedCPT and bge-small:

python -m pip install -U "turbovec>=1.1.2"

Affected versions are 1.1.0 and 1.1.1. In those releases, the first sign-bit stage keeps only max(256, 20 × k) candidates, so at k=10 it can discard the true nearest neighbor before rescoring on corpora with a strong shared vector direction. Version 1.0.0 was not affected.

If you cannot upgrade immediately, search with a larger k and truncate the results as a workaround, for example call search(query, k=64) and keep the first 10 results. On MedCPT 4-bit with TurboVec 1.1.1, this placed the true nearest neighbor in the top 10 for every tested query, whereas search(query, k=10) missed it for about 0.37–0.40% of queries.

After upgrading, verify recall against a full scan. On the reported 101K PubMed slices, 4-bit k=10 identical-result agreement improved from 82.6% to 99.1% for MedCPT and from 75.6% to 96.6% for bge-small, with recall nearly matching the full-index scan.

This happens because temp-file creation and stale-temp sweeping use different name prefixes. tmp_sibling truncates the destination basename when base + suffix exceeds NAME_MAX (255 bytes), but sweep_stale_tmps matches against the full, untruncated basename. Since the temp suffix is 21 bytes, truncation starts at a basename length of 235. The fix is to compute the same truncation point in sweep_stale_tmps and match <base[..cut]>.tmp., while retaining the existing is_our_tmp_suffix and 1-hour mtime guards. This was fixed in PR #524, with a follow-up covering the sync()/write_full path. Upgrade to a build containing those changes. As a workaround, keep destination basenames at 234 bytes or shorter. To manually reclaim only old temp siblings, first inspect:

find /path/to/dest -maxdepth 1 -type f -regextype posix-extended \
  -regex '.+\.tmp\.[0-9]+\.[0-9]+\.[0-9a-fA-F]{8}$' -mmin +60 -print

Then delete the verified files by replacing -print with -delete.

This is a known regression in TurboVec 1.0.0 for selective masks. The single-query block-parallel threshold counts all blocks in the index instead of the blocks allowed by the mask. Above 32,768 vectors (1,024 blocks), even a one-ID allowlist enters the parallel path and pays the Rayon pool-handoff cost, making 16-thread searches about 2× slower than one-thread searches.

Fix: upgrade TurboVec to 1.1.1 or newer, which keeps masked single-query searches on the fast route and includes the related masked staged-route fix:

cargo add turbovec@1.1.1
# or, with an existing lockfile
cargo update -p turbovec --precise 1.1.1
cargo build --release

Verify the resolved version with:

cargo tree -p turbovec

Temporary workaround if you cannot upgrade: run latency-sensitive selective single-query searches with one Rayon worker:

RAYON_NUM_THREADS=1 ./target/release/your-binary

This avoids the parallel handoff for selective masks, but it also disables process-wide Rayon parallelism and may slow dense or unmasked searches.