Verifying changes

A harness, not a test suite. It builds the app, boots it in Electron, and gives you a real page to click — so you can prove a claim instead of asserting one in prose.

npm run verify           # build, then boot the app and check it
npm run gate             # typecheck · lint · format:check · vitest · verify
npm run verify:packaged  # boots the *packaged* app (opt-in, see below)

What launchApp() gives you#

export interface Harness {
app: ElectronApplication;
page: Page;
target: LaunchTarget; // 'app' | 'fixture'  ← always check this
/** Host platform, for assertions about platform-specific chrome. */
platform: NodeJS.Platform;
/** Renderer console output, newest last. */
logs: string[];
/** Renderer console.error lines + uncaught exceptions. Assert this is empty. */
errors: string[];
/** Saves a PNG under e2e/.artifacts/screenshots and returns its absolute path. */
screenshot(name: string): Promise<string>;
close(): Promise<void>;
}

A typical check reads like this:

const harness = await launchApp();

try {
expect(harness.target).toBe('app');

await pasteInput(harness, 'A', beforeText);
await pasteInput(harness, 'B', afterText);
await harness.page.getByTestId('compare-button').click();

await expect(harness.page.getByTestId('screen-workspace')).toBeVisible();
await harness.screenshot('what-i-changed');
expect(harness.errors).toEqual([]);
} finally {
await harness.close();
}

The fixture fallback, and why target matters#

If out/main/index.js does not exist, the harness boots a minimal fixture Electron app instead — and logs loudly that it did:

if (target === 'fixture') {
console.log(
  '[harness] Real app not built (no out/main/index.js) — launching the fixture app.\n' +
    '[harness] Run `npm run build` first to verify the actual application.',
);
}

A pass against the fixture proves the harness works, not that the app works. So check harness.target — in an assertion, not by eye. A green run you did not look at is exactly how "verified" becomes untrue.

What counts as an error#

harness.errors collects renderer console.error lines, uncaught page exceptions, and genuinely fatal main-process output. That last filter is deliberately loose:

// Electron routes ordinary INFO logging to stderr, so only genuinely angry
// lines count as errors; everything else is kept in `logs` for debugging.
const looksFatal = (text: string): boolean =>
/\b(ERROR|FATAL)\b|Uncaught|Unhandled|\bError:/.test(text) && !/\bINFO\b/.test(text);

Electron logs INFO to stderr. Tighten that predicate and every run fails. Everything main printed is still in harness.logs, which is where to look when no window appears — main dying during load otherwise presents as an unexplained timeout.

Profiles and restarts#

Each launch gets a private --user-data-dir, because the app's single-instance lock would otherwise make the second launch in a run quit immediately, and because runs must not leak settings into each other.

Pass userDataDir to reuse one deliberately — hand the same path to two launches to test what survives a restart. The caller owns cleanup in that case, which is the whole point of passing one in.

Every feature keeps a permanent regression spec#

This is a standing rule, not a suggestion: finishing a feature means extending the suite, in the same commit. Add e2e/regression/<area>.spec.ts when the feature is a new area, extend the existing spec when it is not. Electron needs no browser download, so the whole suite runs in seconds.

Two of those earn a note#

full-loop does not launch a fresh app per area, and that is the point: the bugs it can catch are the ones that only exist between features — a store that does not reset when you switch engines, a toolbar slot two views both claim, change navigation left pointing at the previous result. The search store is the clearest case: the text view registers matches and gets an n/m badge, while JSON and folders use the same box as a filter and must not. Only a session that visits both proves the handover works.

scroll-perf is a measurement rather than an assertion shaped like one. Its automated check is a deliberate floor, not the budget — frame pacing on a shared CI box is not a developer's machine, and a test that fails because another process got the CPU teaches nothing. The printed p95 is the number worth reading.

Prefer assertions a screenshot cannot make#

Screenshots are for the human. Assertions are for the next change.

The regressions that slip through review are not the ones where the layout visibly collapsed — they are computed styles, ARIA state, token values and event ordering. So assert those:

• computed styles and CSS custom-property values, rather than "it looks right"
• aria-pressed, aria-selected, role, labels
• the order events arrived in, not just that they arrived
• literal data- attribute values — and note that React drops attributes whose value is undefined, which makes a false state unaddressable from both CSS and tests. Render a literal string instead

Then take a screenshot as well, and look at it. Porting the mockup's block-level elements to inline spans once collapsed stacked text onto one line; it passed every assertion and was only caught by a human reading the PNG. Screenshots land in e2e/.artifacts/screenshots/.

What the harness cannot do honestly#

The packaged smoke test is opt-in#

npm run package:mac      # build release/TwinScope-<version>-arm64.dmg
npm run verify:packaged  # sets TWINSCOPE_PACKAGED=1 and runs e2e/packaged.spec.ts

It boots the packaged app, runs a real comparison through the packaged engine host, checks the database lands in the user-data directory, and prints per-process memory. It is opt-in because release/ goes stale the moment source changes, and a stale pass is worse than no pass.

Test seams#

Under NODE_ENV=test — which the harness sets and nothing else does — main hangs globalThis.__twinscopeKillEngineHost where app.evaluate() can reach it, so the harness can prove that a worker crash is survivable. It is reachable from the test process only, never from the renderer.

The history module also exports a seam for pointing its database at a disposable path, used by unit tests that import it directly.