e2e

Framework

tester-army/e2e

AI-driven end-to-end testing framework for web and mobile apps.

Overview

An open-source end-to-end testing framework where you describe goals in natural language and an AI agent drives web or mobile apps to reach them. Supports locators and assertions alongside agent steps, replays recorded actions without model calls, and works with Playwright browsers or iOS/Android simulators. Ships SDK, runner, CLI, and pluggable engine packages.

README Preview

\n\n\n  \n  \n  \n  \n\n\n# e2e\n\n[e2e](https://tester.army/e2e?utm_source=e2e&utm_medium=github&utm_campaign=readme_intro) is an end-to-end testing framework for web and mobile apps. Describe a goal in natural language and an agent drives the app to reach it. Check the result with locators and assertions in the same test.\n\n```ts\n// tests/checkout.e2e.ts\nimport { test, expect } from 'e2e';\n\ntest('a member upgrades to Pro', async ({ app, agent, screen }) => {\n  await app.open('/settings/billing');\n\n  await agent.act('upgrade the workspace to the Pro plan');\n  await agent.assert('the invoice preview shows a prorated amount');\n\n  await expect(screen.getByRole('status')).toContainText('Pro');\n});\n```\n\nAn agent step that a later assertion verifies records its actions, and the\nnext run replays them with no model calls until the app changes. Tests\nwithout agent steps need no model. Bring your own subscription, API key, or\nlocal model.\n\n## Quick start\n\n```bash\nnpx e2e init\n```\n\n`init` asks for an engine, web or mobile, and a model provider, then writes a\nconfig and an example test. The\n[quickstart](https://e2e.tester.army/docs/quickstart) covers the rest.\n\n## Packages\n\n| Package | What it does |\n| --- | --- |\n| [`e2e`](https://www.npmjs.com/package/e2e) | The SDK, runner, and CLI. |\n| [`@e2e-dev/web`](https://www.npmjs.com/package/@e2e-dev/web) | Browser engine: Chromium, Firefox, and WebKit through Playwright. |\n| [`@e2e-dev/mobile`](https://www.npmjs.com/package/@e2e-dev/mobile) | iOS and Android engine: simulators and emulators through agent-device. |\n| [`@e2e-dev/github`](https://www.npmjs.com/package/@e2e-dev/github) | Reporter that posts results as a pull request comment. |\n| [`@e2e-dev/kernel`](https://www.npmjs.com/package/@e2e-dev/kernel) | Kernel hosted browsers for the web engine. |\n| [`@e2e-dev/eas`](https://www.npmjs.com/package/@e2e-dev/eas) | EAS Simulators hosted iOS simulators and Android emulators for the mobile eng

FAQ (5)

Troubleshooting
Why does worker replacement drop the device claim on a provider-leased session in e2e mobile tests?

In e2e@0.15.1 / @e2e-dev/mobile@0.9.0, a failing unit retires the worker and AgentDeviceSurface.dispose() calls sessions.close(), releasing the agent-device claim before the replacement worker reopens it—letting another session steal the device. Fixed in PR #1209: when a slot holds a provider lease, worker replacement now keeps the session and its device claim open; only release or run finish closes it. Install the fix with npm i @e2e-dev/mobile@nightly e2e@nightly or upgrade to the next stable release after 0.15.1 / 0.9.0.

GitHub Issue #792
How-to
How to use different AI models in e2e.config.ts depending on which API key is set?

You don't need a built-in fallback API — use a plain conditional expression on environment variables directly in e2e.config.ts:

export default {
  agents: {
    default: {
      model: process.env.ANTHROPIC_API_KEY
        ? anthropic('claude-sonnet-5')
        : gateway('openai/gpt-6-luna-fast'),
    },
  },
} satisfies E2EConfig;

Each machine then picks the model based on the credential available locally (e.g., ANTHROPIC_API_KEY vs AI_GATEWAY_API_KEY) without editing the committed config. Note: a dedicated firstAvailable core API was proposed but rejected by the maintainers, because putting the model in the replay cache key would break replay across machines and fail at config load. Instead, the chosen model is recorded per recording (see issue #1192) without changing the cache key.

GitHub Issue #1116
Troubleshooting
How to capture per-step debug screenshots in tester-army e2e without enabling model vision or attributing them to the wrong action?

Don't take screenshots asynchronously from onStep — it's a nonblocking progress feed, so captures can land on a later screen. Instead: 1) Upgrade to e2e@0.21.2 and call session.report() to get the built-in step-screen trace with redacted, bounded per-step semantic diffs. 2) Use the onAction hook on agent.act() (added in #1219) — it runs awaited after each action, so calling app.screenshot() inside it is guaranteed to match that action: await agent.act('Open Settings', { onAction: async () => { await app.screenshot(); } }). 3) Artifact screenshots are now retained even after a secret fill (#1218), with existing masking and secret-taint rules still enforced. None of this enables model vision or sends pixels to the model.

GitHub Issue #1077
How-to
How to export a versioned debug report from a hosted e2e session for an external debug viewer?

Use session.report(), shipped in e2e@0.21.2 (PR #1156). After the session closes, it returns the session's stable report-1 document — the same schema built for the hosted trace — containing run/attempt/target identity, final SDK outcome, step timings/status/errors, recorded events, and existing artifact references. It is available for passed, SDK-failed, and timed-out sessions, so consumers can serialize/upload the document and render their own viewer without parsing Markdown, scanning artifact directories, or importing private runner internals. Existing secret redaction and retention policies remain authoritative (late-resolved secret redaction is tracked separately in issue #1157). The broader debug bundle with an artifact manifest and reasoning excerpts was descoped; open a focused issue if your viewer needs evidence that report-1 doesn't carry.

GitHub Issue #1074
Troubleshooting
Why does the e2e CLI exit with code 0 and report tests passed even though video recording cleanup failed?

In e2e 0.21.2, the CLI does not treat video cleanup failures (e.g., CLEANUP_TIMEOUT 'stopping the video timed out') as run failures, so a run with --video on can exit 0 with passed tests but no retained video artifacts. Workarounds: (1) Don't trust exit code 0 alone — wrap the CLI and inspect the run report for cleanup: "failed", secondaryErrors with code CLEANUP_TIMEOUT, and verify video artifacts actually exist. (2) To avoid the timeout itself, reduce screencast size (e.g., from 1440x1100 to 960x720), lower quality to 60, and increase cleanupTimeout from 30000 to 90000 ms. A fix is in progress to surface failed cleanup explicitly in the run result or exit status.

GitHub Issue #1231