marin

The archived parse_log_elixir parser expects lines beginning with test name (...), but ExUnit emits lines such as * test handles arbitrary properties (0.1ms) [L#528]. Because of the leading *, the parser returns {} for named test statuses, so FAIL_TO_PASS lookup fails and the oracle scores 0.

Fix the trace by normalizing ExUnit's bullet marker before the archived parser runs, for example in experiments/post_training/task_curation/datasets/tasktrove/swe_rebench.py:

import re

def normalize_exunit_line(line: str) -> str:
    return re.sub(r"^\s*\*\s+", "", line)

Apply this to each log line before invoking parse_log_elixir. The example becomes test handles arbitrary properties (0.1ms) [L#528], allowing named passing and failing records to be detected. Regenerate or rebuild the verifier, then rerun test_state.py::test_patch_resolved; the passing oracle should score 1. Add regression tests for both passing and failing ExUnit traces. The fix is implemented in PR #10008.

The failure occurs because uv selects marin-rigging 0.2.0 from the pinned Marin Git workspace, even though PyPI has compatible releases. Force uv to use the published package by adding an override to config/external/MarinSkyRL/pyproject.toml:

[tool.uv]
override-dependencies = ["marin-rigging[secrets]>=0.2.139.dev37117265303"]

Use the higher version floor required by the pinned MarinSkyRL/TaskCompendium revision; the originally reported MarinSkyRL floor was >=0.2.123.dev35633925924. Then rerun:

uv lock --project config/external/MarinSkyRL --upgrade-package marinskyrl

Verify with uv lock -v that marin-rigging is resolved from PyPI rather than the Marin Git workspace. In testing, the reported revision resolved 198 packages with a published rigging release. This fix was implemented in PR #9931.

This happens because parse_board represents grids as tuples, but json.loads() restores metadata['output'] as lists. The arc_agi and rearc scorers compare these structures directly, so an otherwise correct grid receives the partial score 0.05.

Fix it by normalizing the JSON-decoded gold grid to tuples before calling score_answer():

metadata = entry["metadata"]
metadata["output"] = tuple(
    tuple(row) for row in metadata["output"]
)

For example:

entry = json.loads(json.dumps(entry))
entry["metadata"]["output"] = tuple(
    tuple(row) for row in entry["metadata"]["output"]
)

answer = format_board(entry["metadata"]["output"], dataset.board_format_opts)
print(dataset.score_answer(answer, entry))  # 1.0

Apply this normalization in Verifyit's JSON-loading/scoring boundary for both arc_agi and rearc. After the fix, correct grids score 1.0, wrong grids score 0.05, and malformed answers score 0.0. The TaskTrove ARC exclusion can then be removed, but previously staged rows must be reconverted separately for them to be admitted.

The failure is caused by the locked marin-iris-native==0.1.6.dev31131847556 wheel, which does not include the native proxy task-identity changes. Update and lock the native wheel to marin-iris-native==0.1.11.dev37977873285 or newer, as done in PR #9994/#10002. Update the dependency in lib/iris/pyproject.toml, refresh the lockfile (for example with uv lock), then reinstall with uv sync and rerun:

uv run pytest lib/iris/tests/cluster/controller/test_native_proxy.py -k test_native_listener_stamps_controller_identity_and_strips_caller_credentials

The updated wheel should include {'job_id': '/alice/train'} in the decoded verified_identity.

Upgrade RolloutEngine to a version containing PR #9980. The fix removes eager imports of shellbox.backends.local.machine (and the local TaskCompendium runtime) from lowering.py and machines.py, importing them only inside the lock-only grading path. It also replaces isinstance(factory, LocalMachineFactory) checks with factory.backend == Backend.LOCAL, allowing delegating factory wrappers.

If you are patching the code yourself:
1. Remove the top-level local backend imports.
2. Import the local backend lazily inside _acquire_machine and validate_lowered_task, only when the lock-only path is selected.
3. Guard the bubblewrap_bin import so macOS can import the package; raise a clear platform error only when a lock-only grader actually requires the local backend.
4. Dispatch on factory.backend == Backend.LOCAL instead of the concrete class.

Then update/reinstall the package and rerun tests, for example:

git pull
pip install -e lib/rolloutengine
pytest

Temporary workaround before upgrading: run the tests or type checks inside a Linux container.