E2E stack identity and two-shard execution¶
Slice 2 of the E2E throughput plan.
Slice 1 (PR #3473) cut the CI backend build from about 200 s to 40 s; the full
suite still exceeds the 45-minute contract because 59 tests need roughly
33 minutes of Playwright time and one runner executes them serially. This
slice makes two stacks able to coexist and splits the suite into two shards.
It does not add runner capacity: both shards run one after the other on the
existing pomegranate-01 lane until slice 3 activates a second listener.
No feature flag: this changes CI execution only. Browser coverage, retries, timeouts, performance budgets and trusted-source gates are unchanged.
Why a stack identity¶
CI today runs the stack at port offset 0 under the literal Compose project
syrf-e2e. Every step that inspects, budgets, monitors or cleans the stack
names syrf-e2e, syrf-e2e_default, syrf-e2e-<service> and ports 8080,
8081, 8083 and 15673 directly. The owner label is repo:run:attempt:job with
no shard component, and ci-resources.py aborts when any live object has a
different owner. Two runs on one host therefore cannot start, let alone run.
The local allocator already parameterises most surfaces from E2E_PORT_OFFSET
through E2E_STACK. The gap is that CI fixes the offset, E2E_STACK cannot be
set independently, the owner has no shard, and several steps rewrite files
inside the checkout.
Contract¶
E2E_STACK_ID = <run_id>-<run_attempt>-<shard> (shard is 1 or 2). It is
the only input; everything below derives from it.
| Surface | Today | After |
|---|---|---|
| Port offset | E2E_PORT_OFFSET=0 fixed in CI |
Allocated per shard from the reservation root; CI passes the resolved offset to every step through ports.sh exports, never literal numbers |
| Compose project / network / containers | syrf-e2e, syrf-e2e_default, syrf-e2e-* |
syrf-e2e-<stack-id> and derived names; E2E_STACK accepts an explicit override |
| Owner label | repo:run:attempt:e2e-tests |
repo:run:attempt:e2e-tests:<shard>; cleanup treats a live sibling shard of the same run and attempt as legitimate and still aborts on unknown owners |
| Job root | $RUNNER_TEMP/syrf-e2e-job (guarded) |
$RUNNER_TEMP/syrf-e2e-job/<stack-id>; guard updated; shared caches stay one level up |
| Checkout mutations | compose, deploy-lambda.sh, setup.sh, appConfig.env.json, e2e/.auth written in place |
Compose, mock-oidc/, ports.sh, setup.sh, teardown.sh, deploy-lambda.sh and service-env.sh are copied into <job root>/exec and patched and run from there; browser auth state lives in <job root>/exec/e2e/.auth (E2E_AUTH_DIR). No workflow step writes into the checkout after the build. Exception: ng serve still runs in the checkout, so runtime-config.ts writes the resolved appConfig.env.json overlay (and its backup) there and the Angular build cache stays shared; redirecting either would need src/services/web changes. Each CI job has its own checkout, so shards never share them |
| .NET service endpoints | overlay defaults (27018, 5673, 4566, 9000, 1434, 4200) | __ environment overrides from the resolved ports, as run-local.sh already does |
| Cookie names | literal syrf-session-syrf-e2e |
derived from E2E_STACK (already exported by ports.sh) |
| Feature-flag lock | keyed by GITHUB_RUN_ID |
keyed by E2E_STACK_ID; flags are per API instance, so the lock must be too |
| Results, report, artifacts | e2e/test-results, e2e/playwright-report, fixed artifact names |
<job root>/results, artifact names suffixed with the shard |
| PR comment | one result, one duration | per-shard row plus merged verdict |
Sharding¶
playwright test --project=@full --shard=<n>/2; smoke stays one unsharded job. The config setsfullyParallel: true, so Playwright shards by test group rather than by file: a serial describe group stays inside one shard, other tests of one file may land in different shards.- Implemented as the
e2e-testsmatrix ([1, 2]for full,[1]for smoke,max-parallel: 1,fail-fast: false). Each shard keeps the full 45-minute job contract and uploadse2e-shard-report-<n>(shard.json, the shard's JSON report and an unsharded--listinventory). - Not yet done: balancing by the measured healthy first-attempt durations from
handoverevidence (run 34905067814). Playwright 1.58 splits groups by count only; the Dockview, performance and annotation specs dominate, so the first main run's per-shard durations decide whether a weighted split is needed. merge-e2e-reports(ordinary Juniper route, read-only, no secrets, base-branch mergere2e/scripts/merge-shard-reports.cjs) is red when the job result is not success, any expected shard is missing, failed or cancelled, shards disagree on the inventory, or any inventoried test ran in zero or several shards (retries are results of one entry; the setup project runs in every shard and is excluded). The PR comment shows per-shard status, duration and test count plus the merged verdict, and is green only when both are.
Acceptance¶
- Contract tests reject any literal
syrf-e2ename or fixed port in a CI step and prove every surface above derives fromE2E_STACK_ID. - Local proof on one checkout: two stacks with different ids run at once; mutating a runtime flag in one does not affect the other; killing one owner mid-run leaves the other's containers, network and reservation intact; cleanup removes only the terminated stack.
- Report merger unit tests: missing, failed or cancelled shard is red; duplicate or absent test ids are red.
- First main run after merge (the workflow executes from main only): both shards complete their inventory, per-phase timings are reported, and the sum of shard Playwright time is compared with the serial baseline.
Out of scope¶
Second listener activation and host provisioning (slice 3, coordinated with the server-config owner), performance-lane isolation, prebuilt web bundle and recording policy (slice 4).