test(screenshots): stabilize electron screenshot output

This commit is contained in:
johannesjo 2026-05-09 14:38:49 +02:00
parent 989563dc79
commit 40467ad1c5
8 changed files with 152 additions and 108 deletions

View file

@ -12,7 +12,7 @@ import {
* at every required pixel size Playwright launches one context per project
* with the right viewport/deviceScaleFactor.
*
* Outputs land in `.tmp/screenshots/_master/<viewport>/<locale>/<theme>/`.
* Outputs land in `dist/screenshots/_master/<viewport>/<locale>/<theme>/`.
*
* Run all viewports: npm run screenshots:capture
* Run one viewport: npx playwright test --config e2e/playwright.store-screenshots.config.ts --project=desktopMaster

View file

@ -9,7 +9,7 @@ import { VIEWPORTS, type ViewportSpec } from './store-screenshots/matrix';
* fixture in `store-screenshots/fixture.ts` branches to Electron when
* `SCREENSHOT_MODE=electron` is set (the npm script does that).
*
* Outputs land in `.tmp/screenshots/_master_electron/` and feed the Mac App
* Outputs land in `dist/screenshots/_master_electron/` and feed the Mac App
* Store rule (and Flathub later). Run with:
*
* npm run screenshots:capture:electron

View file

@ -9,10 +9,10 @@ Reproducible app-store screenshots driven by Playwright + a single seed dataset.
npm run screenshots
# Or split:
npm run screenshots:capture # full web matrix → .tmp/screenshots/_master/
npm run screenshots:capture # full web matrix → dist/screenshots/_master/
npm run screenshots:capture:desktop # desktopMaster only
npm run screenshots:capture:mobile # iPhone/iPad/Android viewports only
npm run screenshots:capture:electron # Electron build → .tmp/screenshots/_master_electron/
npm run screenshots:capture:electron # Electron build → dist/screenshots/_master_electron/
npm run screenshots:electron # capture:electron + build (lands in dist/)
npm run screenshots:build # rebuild dist/ layout from existing masters
@ -31,7 +31,7 @@ npx playwright test --config e2e/playwright.store-screenshots.config.ts \
| `SP_SCREENSHOT_BG_DISABLE=1` | Drop background images entirely. |
| `SP_SCREENSHOT_BG_OVERLAY_OPACITY=80` | Drives the per-context "Darken/lighten background image for better contrast" slider (099). Default 80 for screenshots vs. 20 in the app. |
Master captures land in `.tmp/screenshots/_master/<viewport>/<locale>/<theme>/<scenario>/<name>.png`.
Master captures land in `dist/screenshots/_master/<viewport>/<locale>/<theme>/<scenario>/<name>.png`.
Per-store assets land in `dist/screenshots/<store>/<locale>/NN-name.png` (and the F-Droid `fastlane/...` layout).
## Scenario lineup
@ -114,24 +114,26 @@ for (const locale of LOCALES) {
## Electron pipeline (Mac App Store, Flathub)
Web Chromium captures don't look "native" on macOS — wrong fonts, wrong scrollbars, no traffic-lights. Flathub explicitly _requires_ native window chrome. So there's a parallel pipeline that runs the actual SP Electron build via Playwright's `_electron` API and captures via OS-level region tools (`screencapture` on macOS, `grim`/`import` on Linux).
Web Chromium captures don't look "native" on macOS — wrong fonts, wrong scrollbars, no traffic-lights. Flathub explicitly _requires_ native window chrome. So there's a parallel pipeline that runs the actual SP Electron build via Playwright's `_electron` API. macOS captures use Electron renderer screenshots plus a deterministic hiddenInset traffic-light overlay; Linux captures use OS-level region tools (`grim`/`import`) so Flathub gets real GTK chrome.
```bash
# Capture only — masters land in .tmp/screenshots/_master_electron/.
# Capture only — masters land in dist/screenshots/_master_electron/.
npm run screenshots:capture:electron
# Capture + build — masters under .tmp/, deliverables under dist/screenshots/
# Capture + build — masters and deliverables under dist/screenshots/
# (macappstore/, flathub/). Mirrors `npm run screenshots` for the web pipeline.
npm run screenshots:electron
```
Same scenarios, same fixture file — `store-screenshots/fixture.ts` branches on the `SCREENSHOT_MODE` env var (the npm script sets it). Each desktop spec runs unchanged in either mode.
The OS-level capture grabs the full window rect including titlebar, shadow, and traffic-lights / GTK decoration. Bounds come from `BrowserWindow.getBounds()`; on macOS Retina, 1440×900 points capture as 2880×1800 px. On Linux X11/Wayland bounds == pixels.
On macOS, Playwright-launched Electron is not always treated like a LaunchServices-started `.app`, and OS capture can miss AppKit's hiddenInset traffic lights even when the window content is correct. The fixture avoids that fragile path: it captures the renderer at the target 2560×1600 Retina size and composites the three traffic lights at AppKit's hiddenInset coordinates.
On Linux, the OS-level capture grabs the full window rect including titlebar, shadow, and GTK decoration. Bounds come from `BrowserWindow.getBounds()`; on X11/Wayland bounds == pixels.
Per-OS tooling (must be on PATH):
- **macOS**`screencapture` (built-in). **Requires Screen Recording permission** for the terminal you launch the run from: System Settings → Privacy & Security → Screen & System Audio Recording → enable for your terminal app, then quit and relaunch the terminal. Without permission `screencapture` errors with "could not create image from rect" and the fixture silently falls back to `page.screenshot()`, which produces PNGs with no traffic-lights — the captures are NOT submission-ready. The fallback prints a banner at first failure and a summary at end-of-run to make this loud.
- **macOS**no external capture tool; the fixture forces Retina-scale capture so the renderer screenshot lands at 2560×1600.
- **Linux X11** — ImageMagick (`apt install imagemagick`, ships `import`)
- **Linux Wayland**`grim` (`apt install grim`, wlroots-based compositors only)
@ -142,7 +144,7 @@ The Mac App Store store rule has `masterDir: 'electron'` in `STORE_RULES`, so th
- ✅ Foundation: matrix, seed builder, fixture (web + electron modes), helpers, post-processor
- ✅ 22 scenarios (3 hero + 6 mobile + 8 desktop + 5 tablet) covering planner, boards, schedule, focus, notes, project view
- ✅ Per-build `_preview.html` contact sheet under `dist/screenshots/` for one-click QA
- ✅ Electron-mode pipeline with OS-level chrome capture (`screencapture` / `grim` / `import`)
- ✅ Electron-mode pipeline with macOS traffic-light compositing and Linux OS chrome capture (`grim` / `import`)
- ✅ Mac App Store wired to source from `_master_electron/`
- ✅ Flathub STORE_RULE (single-gallery, sourced from `_master_electron/`)
- ✅ Snap JPEG re-encode under 2 MB cap (mozjpeg, automatic per-file)

View file

@ -1,7 +1,7 @@
/**
* Per-store layout builder.
*
* Reads master captures from `.tmp/screenshots/_master/<viewport>/<locale>/<theme>/<scenario>/<name>.png`
* Reads master captures from `dist/screenshots/_master/<viewport>/<locale>/<theme>/<scenario>/<name>.png`
* and writes per-store directory layouts to `dist/screenshots/`.
*
* Captures are made at each store's native pixel size (see playwright.store-screenshots.config.ts),
@ -20,11 +20,12 @@ import {
LOCALES,
MASTER_DIR_ELECTRON,
MASTER_DIR_WEB,
SCREENSHOTS_OUT_DIR,
STORE_RULES,
type StoreRule,
} from './matrix';
const OUT_DIR = path.resolve(__dirname, '..', '..', 'dist', 'screenshots');
const OUT_DIR = SCREENSHOTS_OUT_DIR;
const masterDirFor = (which: 'web' | 'electron'): string =>
which === 'electron' ? MASTER_DIR_ELECTRON : MASTER_DIR_WEB;
@ -190,6 +191,18 @@ const buildOneRule = async (
return { written, skipped };
};
const cleanDerivedOutput = (): void => {
fs.mkdirSync(OUT_DIR, { recursive: true });
const preserved = new Set([
path.basename(MASTER_DIR_WEB),
path.basename(MASTER_DIR_ELECTRON),
]);
for (const entry of fs.readdirSync(OUT_DIR)) {
if (preserved.has(entry)) continue;
fs.rmSync(path.join(OUT_DIR, entry), { recursive: true, force: true });
}
};
const main = async (): Promise<void> => {
if (!fs.existsSync(MASTER_DIR_WEB) && !fs.existsSync(MASTER_DIR_ELECTRON)) {
console.error('No master captures found.');
@ -199,8 +212,7 @@ const main = async (): Promise<void> => {
process.exit(1);
}
fs.rmSync(OUT_DIR, { recursive: true, force: true });
fs.mkdirSync(OUT_DIR, { recursive: true });
cleanDerivedOutput();
let total = 0;
for (const rule of STORE_RULES) {

View file

@ -0,0 +1,69 @@
import * as fs from 'fs';
import sharp from 'sharp';
/** Default outer width the matrix expects for `desktopMaster` (= 2560 device px). */
const TARGET_WIDTH_PX = 2560;
/** Default outer height the matrix expects for `desktopMaster` (= 1600 device px). */
const TARGET_HEIGHT_PX = 1600;
/** Retina device pixel ratio used by `desktopMaster`. */
const DEVICE_SCALE_FACTOR = 2;
/**
* AppKit's hiddenInset traffic-light geometry in points. With
* `titleBarStyle: 'hiddenInset'` the controls float over the web contents,
* so compositing must draw on top of the existing capture, not pad a titlebar.
*/
const LIGHT_RADIUS_PT = 6.5;
const LIGHT_CENTRE_Y_PT = 14;
const LIGHT_CENTRE_XS_PT = [24, 44, 64];
const LIGHT_FILLS = ['#ff5f57', '#febc2e', '#28c941'] as const;
/** Faint inner outline so the buttons read on light backgrounds. */
const LIGHT_OUTLINES = ['#e0443e', '#dea123', '#1aa334'] as const;
const buildTrafficLightsSvg = (): string => {
const width = (LIGHT_CENTRE_XS_PT[2] + LIGHT_RADIUS_PT + 2) * DEVICE_SCALE_FACTOR;
const height = (LIGHT_CENTRE_Y_PT + LIGHT_RADIUS_PT + 2) * DEVICE_SCALE_FACTOR;
const radius = LIGHT_RADIUS_PT * DEVICE_SCALE_FACTOR;
const cy = LIGHT_CENTRE_Y_PT * DEVICE_SCALE_FACTOR;
const lights = LIGHT_CENTRE_XS_PT.map((cxPt, i) => {
const cx = cxPt * DEVICE_SCALE_FACTOR;
return (
`<circle cx="${cx}" cy="${cy}" r="${radius}" fill="${LIGHT_FILLS[i]}" ` +
`stroke="${LIGHT_OUTLINES[i]}" stroke-width="${DEVICE_SCALE_FACTOR * 0.5}" />`
);
}).join('\n ');
return (
`<svg xmlns="http://www.w3.org/2000/svg" width="${width}" height="${height}">\n` +
' <filter id="s" x="-40%" y="-40%" width="180%" height="180%">\n' +
' <feDropShadow dx="0" dy="0.5" stdDeviation="0.75" flood-color="#000" flood-opacity="0.28"/>\n' +
' </filter>\n' +
` <g filter="url(#s)">\n` +
` ${lights}\n` +
' </g>\n' +
`</svg>`
);
};
/**
* Add macOS hiddenInset traffic lights to a desktop master capture. This is
* intentionally a deterministic overlay: Playwright-launched Electron is not
* always treated like a LaunchServices-started `.app`, and OS capture can
* therefore miss AppKit's button overlay even when the web contents are right.
*/
export const compositeMacTrafficLights = async (pngPath: string): Promise<void> => {
if (!fs.existsSync(pngPath)) return;
const meta = await sharp(pngPath).metadata();
if (!meta.width || !meta.height) return;
if (meta.width !== TARGET_WIDTH_PX || meta.height !== TARGET_HEIGHT_PX) {
throw new Error(
`Expected ${TARGET_WIDTH_PX}x${TARGET_HEIGHT_PX}, got ${meta.width}x${meta.height}`,
);
}
const composited = await sharp(pngPath)
.composite([{ input: Buffer.from(buildTrafficLightsSvg()), top: 0, left: 0 }])
.png()
.toBuffer();
fs.writeFileSync(pngPath, composited);
};

View file

@ -2,12 +2,12 @@
* Screenshot test fixture works in two modes selected by env var:
*
* SCREENSHOT_MODE=web (default) Playwright Chromium context
* Outputs to .tmp/screenshots/_master/
* Outputs to dist/screenshots/_master/
* SCREENSHOT_MODE=electron Real Electron build via Playwright `_electron`
* Outputs to .tmp/screenshots/_master_electron/
* Captures via OS-level region tool
* (screencapture / grim / import) so the PNG
* includes the native window chrome.
* Outputs to dist/screenshots/_master_electron/
* Captures macOS via renderer + deterministic
* traffic-light overlay, Linux via OS-level
* region tool (grim / import) for GTK chrome.
*
* Same `seededPage` / `screenshotMaster` API in both modes, so scenarios
* import from a single fixture file regardless of mode.
@ -34,6 +34,7 @@ import {
type Theme,
type ViewportName,
} from './matrix';
import { compositeMacTrafficLights } from './composite-mac-chrome';
import { writeSeedFile } from './seed/build-seed';
const run = promisify(execFile);
@ -82,24 +83,40 @@ type ScreenshotFixtures = {
let osChromeCaptureWarned = false;
/**
* Capture the focused Electron window's full screen-space rect, including
* native OS chrome (titlebar, traffic-lights / GTK decoration, shadow).
* `BrowserWindow.getBounds()` returns the OUTER frame in points (macOS) or
* pixels (Linux). The matching OS tool produces output at native resolution:
* - macOS Retina @2x: 1440×900 points 2880×1800 px PNG
* - Linux X11/Wayland: bounds == pixels 1:1
* Capture the focused Electron window.
*
* If the OS-level capture fails (commonly on macOS when the terminal lacks
* Screen Recording permission, error: "could not create image from rect"),
* we fall back to `page.screenshot()` so the rest of the single-session run
* still produces output. The fallback PNG won't include traffic-lights /
* native chrome, but every other scene downstream of the failure is salvaged.
* macOS: use renderer capture and add hiddenInset traffic lights
* deterministically. Playwright's `_electron.launch` does not always get the
* same AppKit treatment as a LaunchServices-started `.app`, and OS-level
* capture can miss the button overlay even when the content is correct.
*
* Linux: capture the full screen-space rect, including native GTK decoration.
*
* On Linux, `BrowserWindow.getBounds()` returns the outer frame in pixels, and
* the matching OS tool produces 1:1 output.
*
* If the Linux OS-level capture fails, we fall back to `page.screenshot()` so
* the rest of the single-session run still produces output. The fallback PNG
* won't include native GTK chrome, but every other scene downstream of the
* failure is salvaged.
*/
const captureWindowWithChrome = async (
electronApp: ElectronApplication,
page: Page,
outPath: string,
): Promise<void> => {
if (process.platform === 'darwin') {
await page.screenshot({
path: outPath,
type: 'png',
fullPage: false,
animations: 'disabled',
caret: 'hide',
});
await compositeMacTrafficLights(outPath);
return;
}
try {
// Beat for focus + paint to settle before the OS-level capture fires.
const settleAndCapture = async (bin: string, args: string[]): Promise<void> => {
@ -111,8 +128,7 @@ const captureWindowWithChrome = async (
} catch (err) {
// execFile throws { message, code, stdout, stderr }; surface stderr
// because the default Error message just says "Command failed: …" and
// hides the actual cause (e.g. "could not create image from rect"
// for missing Screen Recording permission).
// hides the actual cause from the OS capture tool.
const e = err as { stderr?: string; stdout?: string; message?: string };
const stderr = (e.stderr ?? '').toString().trim();
const stdout = (e.stdout ?? '').toString().trim();
@ -127,52 +143,7 @@ const captureWindowWithChrome = async (
}
};
if (process.platform === 'darwin') {
// Capture by CGWindowID rather than by screen rect:
// - Survives small displays that clamp the window below the
// configured outer height (e.g. setBounds 1280×800 ends up as
// 1280×780 because menu bar + dock leave only 780pt usable).
// - Captures the full window including the hiddenInset titlebar
// where the traffic lights sit, regardless of where the window
// ended up on screen or whether part of it is clipped by the
// screen edge.
// Resolve the window id from the main process via desktopCapturer:
// its source.id is "window:CGWindowID:0" on macOS, and we match by
// exact window title to find OUR window among all open windows.
const windowId = await electronApp.evaluate(
async ({ BrowserWindow, desktopCapturer }) => {
const w = BrowserWindow.getFocusedWindow() ?? BrowserWindow.getAllWindows()[0];
if (!w) throw new Error('No Electron window to capture');
w.show();
w.focus();
const targetTitle = w.getTitle();
const sources = await desktopCapturer.getSources({ types: ['window'] });
const match = sources.find((s) => s.name === targetTitle);
if (!match) {
const names = sources.map((s) => s.name).join(', ');
throw new Error(
`No window source matched title "${targetTitle}". Open windows: ${names}`,
);
}
const parts = match.id.split(':');
const id = Number.parseInt(parts[1] ?? '', 10);
if (!Number.isFinite(id)) {
throw new Error(`Invalid CGWindowID parsed from "${match.id}"`);
}
return id;
},
);
await settleAndCapture('screencapture', [
'-l',
String(windowId),
'-t',
'png',
outPath,
]);
return;
}
// ─── Linux: still capture by screen rect (no per-window equivalent
// ─── Linux: capture by screen rect (no per-window equivalent
// that's guaranteed to be on PATH). Output dims will match the
// window outer bounds. ─────────────────────────────────────────
const b = await electronApp.evaluate(({ BrowserWindow }) => {
@ -203,23 +174,18 @@ const captureWindowWithChrome = async (
if (!osChromeCaptureWarned) {
osChromeCaptureWarned = true;
const hint =
process.platform === 'darwin'
? '\n ↳ macOS: System Settings → Privacy & Security → Screen & System ' +
'Audio Recording → enable for the terminal you launched this from, ' +
'then quit and relaunch the terminal.'
: process.platform === 'linux'
? '\n ↳ Linux: ensure `grim` (Wayland) or ImageMagick `import` (X11) ' +
'is installed and on PATH.'
: '';
process.platform === 'linux'
? '\n ↳ Linux: ensure `grim` (Wayland) or ImageMagick `import` (X11) ' +
'is installed and on PATH.'
: '';
console.warn(
'\n' +
'════════════════════════════════════════════════════════════════════\n' +
'⚠ OS-LEVEL SCREEN CAPTURE FAILED — RENDERER FALLBACK ACTIVE\n' +
'════════════════════════════════════════════════════════════════════\n' +
` ${msg.split('\n')[0]}\n` +
' Captures will NOT include native window chrome (titlebar,\n' +
' traffic-lights). The Mac App Store / Flathub deliverables from\n' +
' this run are therefore NOT submission-ready.' +
' Captures will NOT include native GTK window chrome. The Flathub\n' +
' deliverables from this run are therefore NOT submission-ready.' +
hint +
'\n' +
'════════════════════════════════════════════════════════════════════\n',
@ -298,11 +264,14 @@ export const test = base.extend<ScreenshotFixtures>({
// launched as a child process (the same binary launched via `npm start`
// shows them fine). Only pass them on Linux.
const isMac = process.platform === 'darwin';
const deviceScaleFactor =
(testInfo.project.use as { deviceScaleFactor?: number }).deviceScaleFactor ?? 1;
const electronApp = await _electron.launch({
args: [
ELECTRON_MAIN,
`--custom-url=http://localhost:4242/`,
`--user-data-dir=${userDataDir}`,
...(isMac ? [`--force-device-scale-factor=${deviceScaleFactor}`] : []),
...(isMac ? [] : ['--disable-dev-shm-usage', '--no-sandbox']),
],
env: {

View file

@ -98,14 +98,10 @@ export const TABLET_VIEWPORTS = [
import * as path from 'path';
const repoRoot = path.resolve(__dirname, '..', '..');
export const SCREENSHOTS_OUT_DIR = path.join(repoRoot, 'dist', 'screenshots');
/** Roots that hold raw scenario captures, separated by capture pipeline. */
export const MASTER_DIR_WEB = path.join(repoRoot, '.tmp', 'screenshots', '_master');
export const MASTER_DIR_ELECTRON = path.join(
repoRoot,
'.tmp',
'screenshots',
'_master_electron',
);
export const MASTER_DIR_WEB = path.join(SCREENSHOTS_OUT_DIR, '_master');
export const MASTER_DIR_ELECTRON = path.join(SCREENSHOTS_OUT_DIR, '_master_electron');
/**
* Per-store derivation rules used by `build-store-assets.ts`. Each rule maps

View file

@ -12,26 +12,22 @@ const printOutputPath = (): void => {
// Re-surface the OS-capture-failed warning at end-of-run. The first-time
// banner from the fixture scrolls off during long runs, so a final summary
// ensures the user notices that the captures lack native window chrome.
// ensures the user notices that Linux captures lack native GTK chrome.
const marker = path.join(dir, '.os-capture-failed');
if (fs.existsSync(marker)) {
const hint =
process.platform === 'darwin'
? ' ↳ macOS: System Settings → Privacy & Security → Screen & System\n' +
' Audio Recording → enable for the terminal you launched this from,\n' +
' then quit and relaunch the terminal.\n'
: process.platform === 'linux'
? ' ↳ Linux: install `grim` (Wayland) or ImageMagick `import` (X11).\n'
: '';
process.platform === 'linux'
? ' ↳ Linux: install `grim` (Wayland) or ImageMagick `import` (X11).\n'
: '';
console.warn(
'\n' +
'════════════════════════════════════════════════════════════════════\n' +
'⚠ THIS RUN USED THE RENDERER FALLBACK — NO NATIVE WINDOW CHROME\n' +
'⚠ THIS RUN USED THE RENDERER FALLBACK — NO NATIVE GTK CHROME\n' +
'════════════════════════════════════════════════════════════════════\n' +
' At least one capture fell back to page.screenshot() because the\n' +
' OS-level capture tool failed. The resulting PNGs do NOT contain\n' +
' the macOS traffic-lights / GTK titlebar — the Mac App Store and\n' +
' Flathub deliverables from this run are NOT submission-ready.\n' +
' the GTK titlebar — the Flathub deliverables from this run are NOT\n' +
' submission-ready.\n' +
hint +
'════════════════════════════════════════════════════════════════════\n',
);