* test(ci): schedule a mid-test snapshot 150ms after every beforeEach
Run 26401801404 (PR #7841 after merging develop) captured the dying
test's beforeEach node-report — be-0258, written 75ms after
socketio.ts > "Pad-wide settings creator gate different browsers"
entered — but no further state. The kill landed 321ms into the test
body, between 1 Hz heartbeat ticks, and the 100ms boundary throttle
prevented further beforeEach writes inside the same test. The report
we have shows only the listening server socket; the connections that
the test body creates (and that presumably precede the kill) never
get snapshotted.
Schedule an unref'd setTimeout from beforeEach that fires 150ms after
the test entered. If it's still the running test at fire time (i.e.
slow enough that the death window applies), capture a node-report
from INSIDE the test body — the moment when the real TCP / socket.io
activity is in flight. Fast tests (<150ms) skip the write because
afterEach has already cleared currentTest by the time the timer
fires.
Result on the next reproduction of the death pattern:
- be-NNNN report at +75ms (beforeEach, body not yet started)
- mt-MMMM report at +150ms (mid-test, body in flight, before kill
at +320ms)
- kill, no further reports
Cost: only slow tests (>150ms) generate an mt report, so the
artifact size growth is bounded by the count of tests that take
longer than 150ms — typically a small minority. Locally verified
against a 3-test probe: 2 fast tests skipped, 1 300ms test produced
the expected mt-NNNN snapshot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): bump heartbeat from 1Hz to 5Hz
Run 26402211271 produced be-0207 (the dying test's beforeEach
snapshot) but no mid-test snapshot, even though setTimeout(150ms)
was scheduled and the test body lived another 280 ms after that
deadline. setTimeout under Windows-runner load is being starved
past the deadline — we already saw the previous test's mt fire at
+252 ms (102 ms late) when the deadline was 150 ms, so the dying
test's timer was likely scheduled to fire well after the kill at
+425 ms.
setInterval has fired reliably throughout the investigation
(every heartbeat in every run lands within ~1 s of schedule, even
when setTimeout misses). Bump heartbeat to 200 ms (5 Hz) so any
death window ≥200 ms is sampled inside the test body, independent
of how starved setTimeout is.
Cost on the Windows runner: the existing log shows writeReport
completes in <1 ms (from "Writing Node.js report" to "Node.js
report completed" timestamps), so 5 Hz adds ~5 ms/s of overhead.
Artifact growth: ~500 reports for a 100 s test phase (~25 MB raw,
~5 MB compressed). The setTimeout mid-test snapshot stays — it's
belt-and-suspenders cheap and fires for slow tests where the
heartbeat alone might not align with the death window.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: isolate mt/hb writes from `be` throttle + gate timer on canWriteReport
Qodo flagged two real issues on #7842:
1. The single shared `lastReportT` made `mt` writes poison the `be`
throttle window. Slow tests trigger an `mt` write at +150 ms, then
the test ends a few ms later, and the NEXT test's `beforeEach`
landed within the 100 ms throttle from the `mt` write — so its
own `be` snapshot was suppressed. That's the exact boundary
coverage the throttle is supposed to PROTECT. Local repro with a
180 ms slow test followed by a fast one confirmed: the fast
test's `be-0004` is now captured instead of swallowed.
Fix: split into `lastBoundaryT` used and updated only by `be`
writes. `hb` and `mt` pass `updateThrottle=false` and never
advance the boundary timestamp.
2. `setTimeout` was being scheduled in `beforeEach` for every test
even when `canWriteReport` is false (Linux backend matrix, local
dev). That's a wasted timer per test for no possible diagnostic
output. Gate the schedule itself on `canWriteReport`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): heartbeat + running-test pointer in backend test diagnostics
Backend tests silently die with code 255 mid-suite ~22% of the time on
develop (most often Windows-with-plugins, Node 24). Each kill lands
300±50 ms after the previous test's clean ✔ teardown line and produces
no failing-test marker, no error, no Mocha summary, and — despite the
unconditional handlers in `diagnostics.ts` — none of the JS-level death
events fire either. Recent example: run 26311025244 (`Windows with
Plugins (24)`); both attempts crashed at completely different "last
test" locations, so the dying test itself isn't to blame.
The existing diagnostics only set lastSeenTest in afterEach, so if the
kill lands during the NEXT test's setup or body — which is exactly the
~300ms gap we observe — the pointer reads as the previous (passing)
test. That hides whether we're between tests or inside one, and which
one.
Two changes:
1. Track currentTest in beforeEach as well as lastFinishedTest in
afterEach. Every diag line now carries both, so the death point is
bracketable regardless of which lifecycle phase the kill interrupts.
2. Add a 1Hz heartbeat that writeSyncs the running-test name plus
`process.memoryUsage()` (rss, heap) and the active-handle and
active-request counts. The interval is unref'd so it never holds the
event loop open by itself. Cost is roughly one extra log line per
second of mocha runtime (~60-120 lines per CI run).
When the next failure fires, the last heartbeat narrows the kill window
to ≤1s, the running pointer names the test on the rails at that moment,
and the handle/memory trace gives a sparkline that exposes sudden
spikes — a leaked socket, an unref'd timer, a runaway map — that
would otherwise be invisible at the runner-log level.
No behavior change on successful runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: heartbeat _getActiveHandles optional chain bug (Qodo #2)
Qodo correctly flagged `_getActiveHandles?.().length` as a latent
TypeError: `?.()` guards the call but the call's `undefined` return
on a missing method still hits `.length`, which throws. Since the
heartbeat fires on a setInterval inside the mocha bootstrap, a Node
build without the underscore-prefixed internals would take down the
whole backend test run.
Capture the array first, then read `.length` only when it actually
exists. -1 stays as the "API missing" sentinel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): per-test start diag + drop stray console.log noise
Follow-up to the heartbeat PR after run 26397693748 confirmed the
diagnostic works (the kill landed at importexportGetPost.ts
'Import authorization checks > authn anonymous !exist -> fail',
~300 ms after the previous test's ✔). Two cleanups so the next
failure pinpoints faster and reads cleaner:
1. diagnostics.ts: emit a `test start: <name>` diag line in the
mocha beforeEach hook, after setting the currentTest pointer.
The 1Hz heartbeat misses tests that take less than a second,
and the silent kills land ~300 ms after a test boundary —
precisely the gap where heartbeat resolution fails. A start
line per test gives sub-millisecond resolution on which test
was on the rails when the process died.
2. specs/api/importexportGetPost.ts: drop a stray
`console.log(importedPads)` debug leftover (and the duplicate
`await importEtherpad(records)` only present to feed it) in
the `malformed .etherpad files are rejected` block. The leftover
dumped a ~600-line reflection of a supertest Response object
to the CI log on every successful run, drowning the surrounding
test output and making the silent-kill window much harder to
read.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): write node-report on every heartbeat tick
Run 26398054688 narrowed the kill to a specific test
(pad.ts > Gets text on a pad Id and doesn't have an excess newline)
but the test body is a trivial supertest GET — the kill bypasses
all JS handlers, so we can't capture stack state at death.
Two failures across two runs share the shape: an agent.{get,post}
+ common.generateJWTToken() call dies ~300-600 ms after test start,
with no JS-visible cause. The next step is V8 + native stack.
Hook into the existing 1Hz heartbeat to call
process.report.writeReport(path) whenever a report directory is set.
The Windows backend-tests workflow already wires up
`--report-directory=${{ github.workspace }}/node-report` via
NODE_OPTIONS and uploads that directory as an artifact on failure,
so the rolling snapshots ride for free on the existing upload step.
Each report (~50 KB) contains:
- V8 + native call stacks for all threads
- libuv active handles (open TCP, timers, file handles)
- JS heap statistics
- resourceUsage + system info
- shared-object list
On the next reproduction the latest report before ELIFECYCLE will
sit ~0-1 s before the kill — enough to see whether the V8 stack
is inside jose's WebCrypto sign path, inside supertest's TCP
roundtrip, or somewhere unexpected entirely.
NODE_REPORT_DIR is also honored as an explicit override for local
repro / non-workflow runs.
Cost: ~6 files (~300 KB) per Windows backend-test failure, plus
~50 ms event-loop pause per heartbeat. No-op when neither env var
is set.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: writeReport with bare filename, not mixed-slash absolute path
Run 26398830249 exposed the path-separator bug in the previous commit:
every heartbeat tick on the Windows runner logged
Failed to open Node.js report file:
D:\a\etherpad\etherpad/node-report/hb-NNNN-...json
directory: D:\a\etherpad\etherpad/node-report (errno: 22)
— EINVAL. The workflow sets --report-directory with forward-slash
separators on Windows, then this code concatenated another `/` plus
the filename, producing a path Node's report writer rejects.
writeReport(fileName) takes a BARE filename and resolves it against
the configured report directory using the platform-correct separator
internally. Switch to that. For local repro overrides via
NODE_REPORT_DIR, push the path into process.report.directory (the
documented config knob) instead of joining it into the call site.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): write node-report on test boundaries, throttled to 4Hz
Run 26398985832 proved the heartbeat-only report cadence isn't tight
enough: the last report before the kill was hb-0013 at +16201ms,
~1.5 s before ELIFECYCLE at +17701ms — during which ~30 tests fired,
including the dying one (`authn anonymous !exist -> fail`). The
captured V8 stack is just our heartbeat code, not the dying test.
Move the writeReport call to a shared tryWriteReport() helper and
invoke it from BOTH the heartbeat AND mocha's beforeEach hook,
throttled to one report per 250 ms. That gives ≤250 ms resolution
on the kill window — close enough that the latest report captures
state from inside the dying test rather than from the test ~30
slots earlier. The heartbeat always writes (so we don't lose the
no-test-running ticks during setup); beforeEach only writes when
the throttle window has elapsed.
Cost ceiling: ~4 reports/sec × ~12 s test phase ≈ 48 reports
(~2.5 MB) per failing run. Each writeReport adds ~50 ms of
event-loop pause — at 4Hz that's 20% of wall time spent in
diagnostics, which is acceptable for a temporary debug-only
bootstrap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): drop beforeEach report throttle from 250ms to 100ms
Run 26399285213's rerun captured a sixth death point on the new 4Hz
cadence (`socketio.ts > Duplicate-author handling > cookie identity:
same-author second socket kicks the first`, kill at +45953ms, 271ms
after test start). The throttle suppressed the dying test's own
beforeEach: previous boundary write landed 128 ms earlier and the
next 31 ms after that, both inside the 250 ms window. Last captured
report (be-0100) is from the previous test.
100 ms is still well above the inter-test cadence in fast burst
suites (tests fire 2-5 ms apart, so 20-50 of them get throttled to a
single write, ceiling ~10 writes/sec). But it's tight enough that
any death-window neighbour ≥100 ms after the previous report — the
shape we keep observing — gets its own boundary snapshot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(admin): explain env-var substitution in /settings, surface auth errors (#7819)
Three small, env-var-only UX improvements driven by issue #7819, where a
Docker operator saved an ep_oauth block in the admin /settings raw view
and reported it "disappeared" — but the underlying confusion was that
settings.json on disk is a *template*, not the effective config. None of
these changes is visible to installs that don't use ${VAR} placeholders.
* Banner above the editor explaining the template/env-substitution model,
only rendered when the loaded file contains a ${VAR} placeholder. Tells
the operator that the file is not env-substituted in place and that the
Effective tab shows the live values.
* Effective tab in the mode toggle, read-only, also gated on ${VAR}. The
backend was already emitting redacted runtime settings as `resolved`
alongside every `load`; the SPA now exposes them so an operator can
verify what Etherpad is actually using.
* admin_auth_error event from the /settings socket handler. The handler
previously silently returned when the connecting session wasn't admin,
which made misrouted Traefik+SSO auth look like "save did nothing" with
no error path in the UI. Emit a dedicated event before dropping the
socket so the SPA can show a clear toast.
Tests:
- src/tests/backend/specs/admin/adminSettingsAuthError.ts — new spec for
the auth_error/disconnect contract.
- src/tests/frontend-new/admin-spec/adminsettings.spec.ts — new Playwright
test asserting the banner + Effective tab only appear after a ${VAR}
is added to settings.json, and that the Effective view is read-only +
shows [REDACTED] for secrets.
No behaviour change for installs without ${VAR} placeholders — banner,
Effective tab, and auth-error contract are all the same as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(admin): drop fragile pre-condition + add reconnect-loop guard (#7819)
CI's admin-UI workflow seeds settings.json by copying settings.json.template
verbatim, which contains ~30 \${VAR} placeholders. The new Playwright
test asserted "banner not present before adding placeholder" — true on a
fresh dev machine, false in CI. Drop that assertion: the negative path
is covered by the SettingsPage ENV_VAR_PATTERN regex itself; what
matters at the UI level is the positive path (banner + Effective tab
render correctly when placeholders are present), which this test still
exercises.
Also: the server's admin_auth_error path calls socket.disconnect(),
which the SPA's existing disconnect handler interprets as "io server
disconnect" and immediately reconnects — creating a reject/reconnect
loop. Track an authErrored flag and suppress the reconnect once an
auth_error has been received. Reset on successful connect, so a
legitimate re-auth path still works.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes#7835.
- src/locales/en.json: add `index.code` (referenced by src/templates/index.html
for the session-receive code input but never defined, producing a
"Couldn't find translation key" console error on the landing page).
- admin/src/utils/LoadingScreen.tsx, admin/src/pages/PadPage.tsx,
admin/src/pages/AuthorPage.tsx: every @radix-ui/react-dialog `Dialog.Content`
now has a `Dialog.Title` and `Dialog.Description` (visually hidden via
`@radix-ui/react-visually-hidden` where there is no visible heading),
silencing Radix's a11y console warnings on every admin page load.
- src/tests/backend-new/specs/template-l10n-keys.test.ts: regression
coverage — fails CI if any `data-l10n-id` in `src/templates/*.html` is
missing from `src/locales/en.json`.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related operator-facing docs gaps, both surfaced by #7819:
1. settings.json on disk is a *template*; env-var substitution happens
at load time in memory only. Operators repeatedly mistake the
templated file for a stale config because the docs never spell out
that the on-disk file is intentionally unchanged by env vars.
2. The default docker-compose.yml puts settings.json in the container's
writable layer with no host mount, which means admin /settings edits
are silently lost on `docker compose down && up`, `pull`, or
watchtower — but preserved across plain `restart`. Operators don't
reliably know which compose verbs recreate the container.
Adds two prose sections to doc/docker.md (explaining both gotchas, with
a recreate-vs-restart table) and a commented-out `./settings.json:…`
bind mount in both docker-compose.yml and the README compose example.
Bind mount is opt-in so existing setups behave identically.
No runtime change.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: page sessionstorage cleanup to avoid OOM (#7830)
SessionStore._cleanup() previously called `findKeys('sessionstorage:*',
null)`, materialising every session key into a single array. On decade-
old MariaDB installs with millions of sessions this OOMs the node
process within ~15 minutes — see #7830.
Switch to ueberdb2 6.1.0's findKeysPaged with a 500-key page size, and
yield to the event loop between pages so the DB driver can release each
page's buffered rows and request handlers can interleave.
The break is now driven by `page.length === 0` rather than `page.length
< CLEANUP_PAGE_SIZE` so a stubbed/throttled paged source still iterates
the full keyspace.
Adds a regression test that seeds 50 sessionstorage rows, monkey-patches
`DB.findKeysPaged` to use a 4-key page, runs cleanup, and asserts every
expired row is removed plus every valid row preserved across page
boundaries.
Closes#7830
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address Qodo review on #7831
Four follow-ups raised by Qodo on the session cleanup paging fix:
- DB.ts: fail-fast at init() if any required wrapper method (incl.
findKeysPaged) is missing, so a stale ueberdb2 pin surfaces at boot
rather than crashing the first cleanup run an hour later.
- SessionStore: bound a single _cleanup() run to 10 minutes. Under
sustained session creation the keyspace can grow faster than cleanup
drains it; without a budget the next scheduled run would never fire.
When the budget hits, log a warning and let the next run continue.
- SessionStore: log the defensive `page[0] <= after` cursor-stall break.
Previously the loop exited silently, leaving expired rows behind with
no operator-visible signal of the backend regression.
- Tests: the paged-cleanup regression test now removes both expiredSids
AND validSids in finally, so a failed assertion doesn't leak rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: note paged session cleanup in CHANGELOG + settings template
CHANGELOG.md picks up an entry under 3.1.0 Notable fixes describing the
OOM cause, the paged iteration, the 10-minute per-run budget, the
cursor-stall logging, and the fail-fast init guard.
settings.json.template's sessionCleanup comment adds the page-size,
budget, and pointer to #7830 so admins can reason about the new
behaviour from the template alone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: regenerate lockfile against ueberdb2 6.1.2
Now that ether/ueberDB#983 unblocked the publish workflow (OIDC trusted
publishing), ueberdb2 6.1.2 is live on npm and the `^6.1.0` pin in
src/package.json resolves cleanly. Resolves the ERR_PNPM_OUTDATED_LOCKFILE
that was blocking CI on this PR.
29 SessionStore backend tests still green against the published tarball.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(docker): admin save persists across container restart (#7819)
The OP reports the symptom on the official Docker image specifically.
Adds two layers of coverage to docker.yml's build-test job, driven from
inside a container started against the same TEST_TAG the existing
test-container step uses:
1. New mocha spec adminSettings_7819.ts under tests/container/specs/api
— authenticates against /admin, opens the /settings socket, saves an
augmented JSON with an ep_oauth-shaped top-level block, and asserts
the next load reply contains the marker. Intentionally leaves the
marker on disk so the workflow can inspect it.
2. docker.yml now `docker exec test grep`s for the marker after
test-container, then `docker restart`s the container, waits for the
health probe, and re-greps. Both checks must pass — the first proves
the socket-driven save actually touched the file inside the
container layer; the second proves an in-place restart doesn't reset
it. A recreate (docker rm + docker run) would wipe the file, but
that's expected (image layer) and out of scope.
Container is started with `-e ADMIN_PASSWORD=changeme1` so the existing
settings.json.docker provisions the admin user; pad.js doesn't touch
/admin so the existing API specs are unaffected. test-container timeout
bumped 5s → 30s to cover socket connect + save round-trip, and the
mocha discovery extension list now includes `ts` so the new spec is
picked up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(docker): authenticate via /admin-auth/ POST, surface auth/load failures fast (#7819)
CI failed on #7821 with a generic 20s mocha timeout because the spec
hit GET /admin/ to grab a session cookie. webaccess.ts only treats
paths starting with /admin-auth as requireAdmin — and the container
runs with REQUIRE_AUTHENTICATION=false (default), so GET /admin/ never
issued a Basic challenge and Set-Cookie was empty. The socket then
connected unauthenticated, adminsettings.ts's connection handler
returned early without binding any listeners, and the load() promise
hung until mocha killed the test with no useful diagnostic.
Switch to POST /admin-auth/ (always-requireAdmin, regardless of
settings.requireAuthentication). Assert a 2xx with at least one
Set-Cookie before proceeding. Add an 8s timeout + meaningful error
message to load() so the "session was not admin" failure mode reports
immediately instead of burning the suite budget.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(docker): replace splice with hand-built payload (#7819)
Last CI failed because the splice-after-last-} approach landed a comma
between an existing trailing-comma-before-comment and the close brace
of settings.json.docker, producing `, /* … */, "ep_oauth"` — invalid
JSON.
settings.json.docker uses jsonc `/* */` and `//` comments and a
trailing-comma-before-comment-before-close shape that's annoying to
patch from the test side, and the existing isJSONClean has zero
backend coverage so the splice is going through Etherpad's lenient
write path anyway.
Switch to a hand-built minimal-but-viable settings document containing
the ep_oauth block. Three properties hold:
- We're testing the WRITE path, not the synthesis path. Whatever
bytes we send, the next `load` must return verbatim.
- The post-save document must survive `docker restart` (the next
step in docker.yml) — minimal-but-viable means port/users/dbType
are present so Etherpad boots back up and HEALTHCHECK passes.
- The next `load` reply must equal the bytes we saved
(`reply.results === augmented`) — stronger than `.includes()`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The admin saveSettings socket had zero direct backend coverage and the
e2e 'restart works' test only checked the page renders after restart —
neither catches a deployment that resets settings.json on restart, nor
the user-visible workflow that triggered #7819 (add a top-level plugin
block via Raw, save, watch it disappear).
Adds three backend specs for the saveSettings socket:
- payload is written byte-for-byte to settings.settingsFilename
- augmenting existing JSON with a new top-level block round-trips
through the next load reply
- /* */ comments survive the write path
Adds one e2e spec mirroring the #7819 workflow: open Raw, prepend an
ep_oauth-shaped top-level block, save, restartEtherpad(), re-login,
confirm the block is still in Raw and surfaces as its own Form-view
section ('Ep oauth' from humanize()).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ep_readonly_guest is archived (read-only on GitHub) and its
authenticate hook unconditionally swaps req.session.user with a
read-only guest, even when the request carries an HTTP Authorization
header. That silently demoted admin login attempts and stalled the
anonymizeAuthorSocket tests for 14 min/run on every with-plugins CI
matrix (#7795). The pre-fix theory blamed ep_hash_auth.handleMessage;
the actual hook trace is a red herring — handleMessage only fires on
the /pad namespace and never on /settings.
ep_guest is the maintained successor (same authors, same purpose).
1.0.72 on npm already includes the "defer to basic auth / admin
paths" fix backported to ep_readonly_guest by intent here. Swapping
the matrix unblocks the anonymizeAuthorSocket suite on Linux,
Windows, and the upgrade-from-latest-release workflow.
The runtime probe added in #7796 stays — it still catches any other
authenticate-hook plugin that rejects the test's plain-text
credentials (e.g. a future ep_hash_auth-style hashed-only plugin).
Reattribute its comment so future readers don't chase ep_hash_auth.
Closes#7795.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: design for URL base-path support (#7802)
Spec covers the architecture, header handling rules, components touched,
backwards-compatibility story, risks, and test plan for honoring
X-Forwarded-Prefix / X-Ingress-Path under trustProxy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: amend #7802 spec with discovery of pre-existing proxy-path helpers
After exploring the codebase, much of the proposed architecture is
already in place (sanitizeProxyPath, padBootstrap.js basePath derivation,
admin SPA rewrite). Spec now reflects the actual delta: header source
expansion, /manifest.json prefix-awareness, socialMeta proxyPath honoring,
and template URL touch-ups for index/timeslider/pad/export.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(plan): implementation plan for URL base-path support (#7802)
Adds the bite-sized TDD task list to ship X-Forwarded-Prefix /
X-Ingress-Path support: extends sanitizeProxyPath, makes /manifest.json
and socialMeta prefix-aware, touches up the remaining leading-slash
URLs in index/pad/timeslider/export templates, fixes a pre-existing
manifest .. count bug. Drops the originally-proposed <base href>
belt-and-braces after discovering it'd break the existing relative
URLs in pad.html/timeslider.html and wouldn't help plugin DOM
injection anyway (path-absolute URLs ignore <base>'s path component).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(proxy): accept X-Forwarded-Prefix and X-Ingress-Path under trustProxy (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(pwa): make /manifest.json honor sanitised proxy-path (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(social-meta): honor proxyPath in from-request og:url and og:image (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(templates): index.html manifest + jslicense links honor proxyPath (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(templates): pad.html reconnect/jslicense honor proxyPath; fix manifest .. count (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(templates): timeslider.html reconnect/jslicense honor proxyPath; fix manifest .. count (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(templates): export_html.html manifest honors proxyPath when available (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: end-to-end coverage for X-Forwarded-Prefix / X-Ingress-Path (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(settings): trustProxy also enables X-Forwarded-Prefix / X-Ingress-Path (#7802)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: spec for admin/settings resolved runtime values (#7803)
Side-channel resolved+redacted settings alongside raw file blob.
Form view dropdowns and env pill chips reflect actual runtime values
instead of falling back to template defaults. Save round-trip is
unchanged so ${VAR:default} literals stay intact on disk.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: implementation plan for admin/settings resolved runtime (#7803)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(admin): add redactor for resolved settings payload (#7803)
Pure helper that walks the live settings module and replaces known
sensitive paths (users.*.password, dbSettings.password,
sso.clients[*].client_secret, sessionKey, …) with [REDACTED] sentinel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(admin): emit redacted runtime settings on /settings socket load (#7803)
Existing 'results' raw-file blob is unchanged so the textarea editor
and saveSettings round-trip continue to preserve \${VAR:default}
literals on disk. New 'resolved' field carries the in-memory settings
module run through the redactor — admin SPA can use it to show actual
runtime values next to env-var placeholders.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(admin): show resolved runtime value on EnvPill (#7803)
Admin SPA now stores the resolved field from the /settings socket
payload and exposes useResolvedAt(path) to walk it. EnvPill renders a
"→ active value" chip when the path is resolved, or "→ ••••••" with a
redacted tooltip when the server returned the [REDACTED] sentinel.
Old-server fallback (undefined resolved) keeps current behaviour.
The admin test script glob now picks up .test.tsx alongside .test.ts
so the new EnvPill tests run under tsx --test.
Closes#7803.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(updater): resolve pad author from token cookie, not session.user
Etherpad does not populate an authorID into the express-session user
object for pad visitors, so resolveRequestAuthor() always returned null
in production, causing computeOutdated() to return EMPTY and the
pad-side gritter to never fire.
Replace the session-based lookup with a cookie-based path that mirrors
how the socket.io handshake resolves pad-visitor identity: read the
HttpOnly `token` (or `<prefix>token`) cookie and call
authorManager.getAuthorId(token, user) via dynamic import (same
circular-init guard pattern as the PadManager import).
Update the test harness to mock AuthorManager instead of injecting a
fake req.session.user.author, and to set req.cookies.token directly.
All 9 cases continue to pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(openapi): clarify admin spec scope includes pad-side endpoints
/api/version-status is a public pad-side endpoint but lives in the
admin OpenAPI document because it shares the same internal route
registration. Add a note to info.description so downstream tooling
consumers are not misled into treating it as an admin-only route.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: design spec for #7799 outdated-notice redesign
Per-pad first-author gating, dismissable gritter, minor-or-more rule, drop vulnerable UI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: implementation plan for #7799 outdated-notice redesign
12 bite-sized tasks, TDD-first where applicable; closes the spec end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): add isMinorOrMoreBehind, drop major/vulnerable helpers
Adds isMinorOrMoreBehind(current, latest) which returns true only when
the latest release is at least one minor version ahead (patch-only deltas
return false). Removes isMajorBehind, parseVulnerableBelow, and
isVulnerable from versionCompare.ts — callers in updateStatus.ts,
VersionChecker.ts, and index.ts will be updated in subsequent tasks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(updater): drop vulnerable-below directive and state field
Remove VulnerableBelowDirective type, UpdateState.vulnerableBelow field, and
all related scraping/checking logic (parseVulnerableBelow, isVulnerable imports).
Clean up Notifier, OpenAPI schema, and all test fixtures to match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(updater): drop residual EmailSendLog vulnerable fields
Remove `vulnerableAt` and `vulnerableNewReleaseTag` from the
`EmailSendLog` interface, `EMPTY_STATE`, and the `isValidEmail`
validator — these backed the removed `vulnerable`/`vulnerable-new-release`
email kinds and are now dead code.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(updater): add firstAuthorOf helper
Export firstAuthorOf() from updateStatus.ts — finds the lowest-numbered
author attrib in a pad's pool, skipping empty-string placeholders.
Covered by 6 vitest cases in tests/backend-new/specs/hooks/express/.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(updater): add resolveRequestAuthor helper for HTTP GET
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(updater): pad-aware /api/version-status with first-author gating
Replace global badge cache with a per-(padId, authorId) LRU cache. The
new response shape is {outdated: 'minor' | null, isFirstAuthor: boolean};
the old 'severe'/'vulnerable' enum is dropped entirely. computeOutdated
now resolves the pad's first author and compares it against the session
author before returning outdated:'minor', so the notice is only shown to
the person who created the pad.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(updater): switch isSevere signal from major-only to minor-or-more behind
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(updater): end-to-end coverage for /api/version-status
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(openapi): /api/version-status pad-aware shape and gating
Add the /api/version-status GET operation to the admin OpenAPI spec with
the new pad-aware response shape: outdated enum reduced to [minor]|null,
isFirstAuthor boolean, and an optional padId query param.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore(pad): remove unused #version-badge template and CSS
* feat(pad): replace persistent badge with first-author outdated gritter
Renames pad_version_badge.ts → pad_outdated_notice.ts and rewrites it
as a fire-and-forget gritter notice that only shows when the API reports
outdated=minor AND the current user is the pad's first author. Wires
the new maybeShowOutdatedNotice() call into pad.ts immediately after
showPrivacyBannerIfEnabled().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(pad): playwright coverage for outdated notice gritter
Six Playwright specs exercise maybeShowOutdatedNotice: null response,
isFirstAuthor:false guard, positive appearance + text, X-dismiss,
500 server error tolerance, and 8 s auto-fade.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(pad): outdated-notice redesign + drop vulnerable-below docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore(test): remove stale specs for deleted #version-badge surface
Delete the GET /api/version-status describe block from the legacy mocha
spec (asserted outdated:null and outdated:'severe' — both no longer match
the new response shape). The new vitest spec at
tests/backend-new/specs/hooks/express/updateStatus.test.ts covers this
surface comprehensively.
Delete src/tests/frontend-new/specs/pad-version-badge.spec.ts entirely:
all three tests reference the #version-badge DOM element removed in Task 8
and stub 'severe'/'vulnerable' enum values that no longer exist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: clean stale references to vulnerable/severe in types, emails, docs
- Remove OutdatedLevel type (null|'severe') from types.ts — no consumers
remain after the badge redesign removed the severe tier.
- Fix Notifier severe-email body: was "more than one major release behind"
but isSevere now fires on minor-or-more, so update to "at least one
minor release behind the latest published version".
- Drop "vulnerability directives" from the /admin/update/status OpenAPI
description; replace with the actual response fields.
- Remove stale vulnerableBelow field from UpdateStatusPayload in
admin/src/store/store.ts — server no longer sends it.
- Fix docs/admin/updates.md: "pad-side badge" → "pad-side notice".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(l10n): short-circuit form controls in translateNode (no spurious warning)
`<select data-l10n-id="...">` with `<option>` element children — the
pattern used by ep_headings2, ep_align, ep_font_size, ep_font_family, …
— used to drop into the textContent branch of html10n.translateNode and
hunt for a text-node child to overwrite. There is none, so the loop
exited with `found = false` and emitted:
Unexpected error: could not translate element content for key ep_headings.style
The SELECT/INPUT/TEXTAREA aria-label fallback already lived inside the
same else-branch, *after* the warning, so the accessible name landed
correctly but the noisy console line still fired on every pad load.
Move the form-control case into its own `else if`, before the text-node
hunt: aria-label is the only sensible localization target for these
elements (a <select>'s text is its <option> labels, not its own name).
Closes the console warning reported on Etherpad 3.1.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): actually exercise the form-control short-circuit branch
Qodo's review on #7797 caught two real test bugs:
1. `pad.toolbar.bold.title` ends in `.title`, which is in html10n's
attribute allowlist. translateNode picks `prop = 'title'`, takes
the first branch (node[prop] = str.str), and never reaches the
textContent path where the warning lives. The test would pass
without my fix.
Switched the key to `pad.loading` — same stable pad-bundle
translation, but the suffix isn't in the allowlist, so `prop`
defaults to `textContent` and the test actually exercises the
regression path.
2. `page.waitForTimeout(50)` is a fixed sleep, but `html10n.localize`
runs its work inside a `build()` callback, so completion timing
depends on the loader. Replaced with a deterministic `html10n.mt
.bind('localized', …)` await.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(admin/pads): apply filter chip server-side, before pagination
Before: PadPage's filter chip (`active`/`recent`/`empty`/`stale`) ran
on the client AFTER the 12-row page slice was already on screen. On a
deployment with hundreds of pads it produced obviously wrong results
— click "empty pads" on page 1 with 100 empties and only the 0–12
empties within the current page passed the filter. thm reported this
on a 3.1.0 deployment.
Move the filter into `PadSearchQuery` so the `/settings` socket can
apply it before slicing:
1. pattern filter on names (cheap)
2. hydrate metadata for the matching pad universe iff a non-`all`
filter is set or a non-`padName` sort is requested
3. apply filter chip on the hydrated set
4. sort + slice → `total` reflects the filtered universe so the
pagination footer makes sense
The original handler also had a 4-way `if/else if` that duplicated the
hydrate-and-sort loop per `sortBy`. Folded those into one pipeline
with a single comparator switch.
Client side, `PadPage.tsx`:
- drop the client-side `filteredResults` filter (server already filters)
- chip click writes `filter` into searchParams (debounced refetch) and
resets `currentPage` to 0
- older clients that don't send `filter` keep working — server defaults
to `all`
Stats cards (totalUsers/activeCount/emptyCount) still count the visible
page only — that's a pre-existing UI limitation tracked separately.
Closes the regression thm reported.
Test plan
- `tsc --noEmit` clean (server + admin)
- New backend spec `padLoadFilter.ts` exercises filter:empty with
small `limit` to lock in the bug-fix, plus all/active/omitted cases
- `5 passing` locally on Node 25
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* address Qodo review on #7798
1. Functional setState updaters for every searchParams mutation
(Qodo bug 1). The debounced pattern handler captured a render-time
snapshot of searchParams; a faster chip click or sort change in
between would be silently reverted when the debounce fired. Now
every mutation merges against the latest state.
2. Concurrency-limited hydration (Qodo bug 3). The earlier draft
issued Promise.all over the full candidate set, fanning out to
thousands of in-flight padManager.getPad() reads on busy
deployments. New mapWithConcurrency() caps concurrent loads at 16
— empirically enough to saturate a single ueberDB driver without
pushing the event loop into back-pressure.
3. Test cleanup deletes the injected test-admin (Qodo bug 4). The
original snapshot/restore pattern saved `settings.users` by
reference; reassigning the same reference in after() left the
inserted key in place and could leak into later backend specs.
4. Document the new `filter` field on the `padLoad` socket query in
admin/README.md (Qodo rule violation 2).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summarises the changes since v3.0.0: Tier 4 autonomous-update-in-
maintenance-window lands for real (was forward-documented in 3.0.0),
real SMTP via nodemailer, Node engine preflight, rollback/preflight
email notifications, security hardening bundle (JWT, temp-file
tokens, token transfer, x-proxy-path sanitiser, Pad.appendRevision
author invariant, setPadRaw legacy rewrite), and the api/admin
backend specs that were silently skipped by the glob.
* test(backend): make the glob regression check Windows-safe
The previous version called execFileSync('npx', ...). On Windows
runners (and any Windows host where npx is installed as npx.cmd),
node's child_process.spawn does not auto-pick the .cmd shim, so
the call fails with spawnSync npx ENOENT. CI on develop went red
for "Windows without plugins" because of this — Linux passed.
Resolve mocha's JS entry directly via require.resolve and run it
under the current node process. No shell, no .cmd resolution,
identical behaviour on every platform.
Also normalise the absolute paths mocha prints to POSIX-relative
form so the toContain() assertions match on both Linux (which
emits forward slashes) and Windows (backslashes).
Verified locally that the test still fails when the package.json
glob is reverted to the broken tests/backend/specs/**.ts pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(backend): use path.relative for cross-platform glob normalization
Qodo flagged the prefix-strip + sep-split approach as brittle. On
Windows runners mocha can emit paths with mixed separators or
different drive-letter casing, in which case startsWith() misses
the prefix and the assertions fail against absolute paths.
path.relative(srcRoot, abs) handles drive-letter casing and mixed
separators consistently, then a final replace([\\/]) -> '/' yields
POSIX-relative paths regardless of how mocha printed them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(admin): skip anonymizeAuthorSocket suite when ep_hash_auth is installed
#7789 un-hid this suite from CI and immediately surfaced a 14-minute
stall on every with-plugins matrix run (Linux + Windows + the
'Upgrade from latest release' workflow). Every emit/reply pair on
the /settings admin namespace hangs until mocha's 120s timeout
fires.
Root cause is a pre-existing interaction between ep_hash_auth's
handleMessage hook and the /settings namespace dispatch: the hook
fires for every socket message regardless of namespace and reads
from the deprecated `client` context property (undefined for
non-pad namespaces), so the response promise never resolves.
Tracked separately in #7795.
Until that lands, gate the suite on require.resolve('ep_hash_auth').
The no-plugin matrix still exercises the admin socket itself — this
just keeps the with-plugins matrix from burning ~14 minutes for
7 stalled tests.
Verified locally:
- no ep_hash_auth in node_modules → 7 passing
- ep_hash_auth resolvable → 0 passing, 7 pending
Why require.resolve and not pluginDefs.plugins[...]: Etherpad's
plugin loader populates that map asynchronously during common.init.
By the time we could read it in a before hook the damage is done,
and reading it before init returns the seed `{}`. Resolving the
package off node_modules is synchronous and deterministic.
Refs #7795
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): probe at the application layer; restore settings safely on skip
Two Qodo follow-ups on this PR:
1) Replace the static `require.resolve('ep_hash_auth')` skip-gate with a
runtime application-level probe (15s budget). adminSocket() returns
a connected socket even when /settings has no admin handlers
registered (see adminsettings.ts:25 — non-admin sockets exit early
without binding listeners). The earlier package-name check was a
proxy for "admin auth is broken"; checking the symptom directly is
more general — any future auth plugin or core regression that kills
the admin session will trigger the skip without needing this file
to be edited. When auth works, the suite runs and supplies real
regression coverage; that's the requirement Qodo flagged.
2) Guard after() with a setupCompleted flag. The skip-via-this.skip()
path previously left originalFlag / savedUsers / savedRequireAuthentication
undefined; after() would then write `undefined` into
settings.gdprAuthorErasure.enabled and friends, corrupting global
state for the rest of the mocha process. Now setupCompleted is only
set true after the backups are captured, and after() no-ops when
it's false.
Verified locally:
- no-plugin matrix → 7 passing (2s)
- broken-auth sim → 0 passing, 7 pending (17s)
Refs #7795
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(updater): plan tier 4 — autonomous update in maintenance window (#7607)
Maps PR 4 of the auto-update design spec (§"Tier 4 — autonomous") to concrete
files, tasks, and verification steps. Subsequent commits scaffold against this
plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): MaintenanceWindow module — wall-clock window math for tier 4
Pure module: parseWindow, inWindow, nextWindowStart. Supports tz=local|utc
and cross-midnight ranges. Used by upcoming Scheduler + UpdatePolicy changes.
22 vitest unit tests cover format validation, same-day + cross-midnight
boundaries, and host-local vs UTC clock comparisons. DST handling is
absorbed by JS Date constructor's wall-clock normalization (documented in
the file header).
Refs #7607
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): tier 4 backend — window-gated UpdatePolicy + Scheduler
Wires MaintenanceWindow into the existing tier 3 backend so autonomous
updates only fire while `now` is inside `updates.maintenanceWindow`.
UpdatePolicy
- new optional `maintenanceWindow` input
- canAutonomous flips on only for git+tier=autonomous+parse-valid window
- new reasons `maintenance-window-missing` / `maintenance-window-invalid`
- rollback-failed still wins over window denial
Scheduler
- decideSchedule snaps scheduledFor forward to nextWindowStart when
canAutonomous + grace lands outside the window
- decideTriggerApply returns a new `{action: 'defer'}` when canAutonomous
+ fire-time is outside the window; carries nextStart for the runner
- canAutonomous=false preserves Tier 3 behavior unchanged
index.ts wires settings.updates.maintenanceWindow through both passes and
re-arms the timer on defer. Status endpoint surface (nextWindowOpensAt) +
admin UI picker land in a follow-up commit.
Settings adds `maintenanceWindow: {start, end, tz} | null`, defaulting to
null. settings.json.template / settings.json.docker document the shape.
Tests
- 22 vitest cases for MaintenanceWindow already cover the math
- 4 new UpdatePolicy cases for the window outcomes
- 6 new Scheduler cases for tier-4 schedule/trigger paths
- Full backend-new suite: 629 passed (35 files)
Refs #7607
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): tier 4 admin UI — window status, deferred subtitle, banner
GET /admin/update/status now returns:
- `maintenanceWindow`: the parsed window object (admin sessions only)
- `nextWindowOpensAt`: ISO of the next window opening when tier=autonomous
UpdatePage
- new "Maintenance window" section when tier=autonomous, shows current
window summary + next opens at, or "Not configured" when unset
- scheduled panel now appends a "deferred until <iso>" line when the
backend has snapped scheduledFor to the next window opening
UpdateBanner
- new variant when tier=autonomous and policy.reason is
`maintenance-window-missing` or `maintenance-window-invalid`, linking
to /admin/update
i18n
- 8 new keys under `update.banner.*`, `update.page.policy.*`,
`update.page.scheduled.*`, `update.window.*` (en.json only;
translations follow via the usual locale workflow)
Interactive picker is intentionally deferred — admins edit
`updates.maintenanceWindow` via the parsed JSONC settings editor (#7709).
A follow-up commit may add a thin write-through component if the JSONC
round-trip turns out to be too rough for typical operators.
Refs #7607
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(updater): tier 4 — window gate, DST notes, runbook §12 (#7607)
CHANGELOG: flip Tier 4 from "designed, not yet implemented" to current.
Document maintenanceWindow shape, snap-forward, defer-at-fire, and the
two missing/invalid policy reasons.
doc/admin/updates.md: new "Tier 4 — autonomous in a maintenance window"
section with config example, policy gating, DST/timezone notes, admin UI
behavior.
runbook: §12 walks a disposable VM through missing-window, malformed,
outside-window deferral, fire-at-opening, and window-closes-mid-grace.
Adds five sign-off checklist items.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(updater): tier 4 window-boundary integration (#7607)
Mocha integration covering the four scenarios called out in the spec
§"Tier 4 — autonomous":
- outside-window: decideSchedule snaps scheduledFor forward to the
next opening and the snapped value round-trips through saveState
- inside-window at fire-time: decideTriggerApply returns fire
- window-closes-mid-grace: decideTriggerApply returns defer with
nextStart at the next opening; persisted state moves forward
- cancel during deferred-grace: state returns to idle, and the next
decideSchedule pass re-emits a schedule snapped to the next opening
All 4 cases passing locally under tsx mocha.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): real SMTP via nodemailer (mail.* settings) (#7607)
Replaces the (would send email) stub introduced in PR #7601 with a
nodemailer-backed transport. The dependency is lazy-imported so installs
that don't set mail.host pay no runtime cost.
Settings additions
- new top-level mail block: host, port, secure, from, auth (user/pass)
- mail.host=null keeps the legacy log-only behaviour; the Notifier
still updates dedupe state so we don't re-evaluate every tick
- settings.json.template documents the shape inline
- settings.json.docker reads MAIL_HOST / MAIL_FROM / MAIL_PORT /
MAIL_SECURE from env so operators can configure via container env
Transport
- lazy import('nodemailer') on first send
- transport cached by host; settings reload picks up new host without
needing a restart
- send errors are swallowed (logged warn) so a transient SMTP failure
can never poison the surrounding updater state machine
- successful sends log at info; legacy "(would send email)" path
remains the visible signal when mail is disabled
Refs #7607
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): preflight checks target tag's engines.node (#7607)
Before mutating the working tree, runPreflight now reads the target tag's
package.json via `git show <tag>:package.json` and verifies that
process.versions.node satisfies its engines.node range. Failures land at
preflight-failed cleanly (no rollback needed — nothing has changed yet).
Motivation: a release that bumps the Node floor used to either fail
mid-`pnpm install` (which then rolls back successfully) or restart on the
new build and crash in the boot path (which then rolls back via the
health-check timer). Both paths recover, but they burn a drain + restart
cycle on a condition we can reject upfront.
Implementation
- new PreflightReason `node-engine-mismatch`
- new dep `readTargetEnginesNode(tag)` — runs the git-show as a child
process with stdio captured to a string; missing tag / missing file /
malformed JSON / missing engines.node all resolve to null (treated as
"no constraint, pass")
- uses existing semver dep with includePrerelease: true
- new PreflightInput field `currentNodeVersion`; threaded from
process.versions.node in both wirings (scheduler + manual apply)
- check runs *after* signature verification so we trust the package.json
- PreflightResult carries an optional `detail` string; applyPipeline
appends it to the lastResult.reason so the admin UI shows e.g.
"node-engine-mismatch: target requires Node >=26.0.0, running 25.0.0"
Tests: 6 new vitest cases (no engines.node, satisfies, fails below floor,
caret range, loose-spaced range, ordering after signature). Full
backend-new: 635 passed (was 629).
Refs #7607
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(updater): email admin on auto-rollback / preflight-failed (#7607)
Before this commit, only the terminal rollback-failed state emailed the
admin. Auto-recovered failures (rolled-back-install-failed, rolled-back-
build-failed, rolled-back-health-check, rolled-back-crash-loop) and pre-
flight-failed surfaced only via the /admin/update banner — so a 3am
autonomous update that failed because of, say, a Node engine bump would
roll back silently and stay invisible until the admin next logged in.
Notifier
- new EmailKinds: 'update-preflight-failed', 'update-rolled-back',
'update-rollback-failed'
- new pure decideOutcomeEmail(input) → {toSend, newState}
- dedupe key `<outcome>:<targetTag>` in EmailSendLog.lastFailureKey:
same outcome on same tag emits one email per cycle (kills retry-loop
spam); a different outcome or different tag resets the key
- rollback-failed always fires (terminal — overrides dedupe)
- state.ts validator + loadState backfill the new field for legacy
state files (Tier 1/2/3 installs upgrading in place)
Wiring
- new index.ts helper notifyApplyFailure() loads state, runs the pure
notifier, sends (via the nodemailer-backed sendEmailViaSmtp from the
previous commit), persists the new dedupe key — all best-effort
- schedulerTriggerApply: fires on applyUpdate returning preflight-failed
or rolled-back
- /admin/update/apply HTTP handler: same
- boot path in expressCreateServer: if state.lastResult is a failure
outcome we haven't already emailed about, fire then. Covers:
- health-check timeout rollback (timer expired between boots)
- crash-loop forced rollback caught on a later boot
- preflight-failed where the process didn't get to email before exit
- unacknowledged rollback-failed terminal
Tests
- 8 new vitest cases for decideOutcomeEmail (adminEmail=null, each
outcome's content, dedupe by tag, dedupe by outcome, rollback-failed
bypass)
- Full backend-new suite: 643 passed (was 635)
Refs #7607
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(updater): address Qodo review on tier 4
- UpdatePage: only show "deferred until" subtitle when scheduledFor
actually matches nextWindowOpensAt. The previous `scheduledFor >
now + 60s` heuristic misfired during a normal in-window 15-min
grace period.
- applyPipeline: return the enriched preflight reason (`reason:
detail`) instead of only `pf.reason`, so /admin/update/apply 409
bodies and failure-notify emails preserve diagnostics like the
Node engine mismatch detail.
- updater/index: key the cached nodemailer transport on the full
set of SMTP options (host + port + secure + auth) so runtime
changes to port/credentials via reloadSettings() invalidate
the cache.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(backend-tests): un-skip tests/backend/specs/{api,admin}/*
The pnpm test script's glob `tests/backend/specs/**.ts` only matched
files at the top level of tests/backend/specs/. Every spec under
tests/backend/specs/api/ (14 files) and tests/backend/specs/admin/
(2 files) has been silently skipped by CI — including the failing
tests reported in #7785, #7786, #7787, #7788.
Switch to passing the directories with `--extension ts --recursive`,
which makes mocha walk the tree the way --recursive is documented to.
Local run after this change picks up 16 additional spec files and
surfaces 6 newly-visible failures: 4 already filed (#7785–#7788) plus
one that wasn't yet filed (appendTextAuthor.ts:
"appendText without authorId does not attribute to any author").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(backend): regression check for tests/backend/specs/{api,admin} discovery
Read the pnpm test script from src/package.json, hand mocha the
same arguments under --dry-run --list-files, and assert that a
representative spec from tests/backend/specs/api/ and
tests/backend/specs/admin/ appears in the discovered list.
Locks in the glob fix from this PR: if anyone re-narrows the
script back to the previous tests/backend/specs/**.ts pattern
(which only matches depth 1), this vitest fails with a clear
"glob missed ..." message instead of letting the affected specs
silently drop out of CI.
Verified the test FAILS when the script is reverted to the
broken glob.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When an ordered-list level was the only consumer of olItemCounts,
closing any list at that depth (including an unordered list that
happens to share the level) reset olItemCounts[level] to 0. A later,
unrelated ordered list at the same depth then took the
"counter exists but is 0" branch in the ol-opening logic and emitted
`<ol class="...">` without the start attribute that line.start would
have supplied.
Round-trip: importing
<ul>...<ul>...</ul></ul><ol><li>x<ol><li>y</li></ol></li></ol>
exported the inner ol as `<ol class="number">` instead of
`<ol start="2" class="number">`, because the closing of the inner
bullet ul wrote olItemCounts[2]=0 before the outer ol even opened.
Gate the reset on line.listTypeName === 'number' so closing an
unordered list never touches the ol bookkeeping. Closing an actual
ordered list still resets, as #7470 intended.
Fixes#7786Fixes#7787
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(export): surface checkValidRev error message in response body
A non-numeric :rev (e.g. /p/foo/test1/export/txt) was reaching
checkValidRev, which throws CustomError('rev is not a number',
'apierror'). The error fell through the route handler's
.catch(next), so Express's default error renderer kicked in and
returned a 500 with the generic HTML page <title>Error</title> /
<pre>Internal Server Error</pre>. The thrown message never made it
to the body, so callers had no way to tell why the request failed.
Catch the apierror in the route handler and send err.message as a
text/plain 500 body. Other errors still propagate to next(err) so
unrelated failures keep their existing handling.
Also retitle the test (was "is 403" while asserting expect(500)) —
leftover label from an earlier expectation.
Fixes#7788
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(export): validate rev before attachment, broaden error catch
Qodo feedback on #7792:
1) ExportHandler.doExport set Content-Disposition (via res.attachment)
before calling checkValidRev. If the rev was invalid, the route-level
catch returned a plain-text 500 — but the attachment header was still
in place, so browsers offered to save the error message as a file.
Move checkValidRev to the top of doExport so an invalid rev never
touches the attachment header.
2) The catch only converted CustomError('...', 'apierror') into a
plain-text response. Other export errors (conversion failures, fs
issues, soffice problems) still fell through to Express's default
HTML renderer — non-deterministic for API callers and confusing
when they were already half-downloading a file.
Surface every export failure as a deterministic text/plain 500.
apierrors carry user-facing messages, so send err.message verbatim.
For other errors, log the full stack server-side and still emit
err.message (or 'Internal Server Error' if absent) so the response
body is never the Express HTML stack page. Also clear
Content-Disposition in the catch as a safety net.
Backend tests (importexportGetPost.ts): 58 passing, 0 failing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(API): exclude SYSTEM_AUTHOR_ID from listAuthorsOfPad
Pad.SYSTEM_AUTHOR_ID ('a.etherpad-system') is the synthetic author
Etherpad attributes inserts to when the HTTP API receives a call
without authorId (setText, setHTML, appendText, the server-side
import flows, and plugins like ep_post_data). It exists so the
changeset's text and attribs stay in sync — without ANY author
attribute, pad.atext drifts and clients fail setDocAText
reconciliation when loading the pad. See Pad.ts:96-105 for the
full rationale.
That bookkeeping detail was leaking through listAuthorsOfPad: a
pad whose only "contributor" is the system author still reported
one authorID, which the existing tests in pad.ts and
appendTextAuthor.ts (and presumably any caller that uses
listAuthorsOfPad to count real users) treat as a real participant.
Filter SYSTEM_AUTHOR_ID at the API surface so internal attribution
stays internal. getAllAuthors() and downstream callers (copy,
anonymize, atext verification) keep seeing the synthetic id —
this only narrows the public listAuthorsOfPad response.
Fixes#7785Fixes#7790
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(api): note that listAuthorsOfPad omits the system author
Match the runtime behaviour from the previous commit — the
synthetic 'a.etherpad-system' author used for unattributed inserts
is filtered out of the listAuthorsOfPad response.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* harden: assorted security tightening across server entry points
A bundle of defence-in-depth hardening picked up during an internal
audit pass. Each change is small on its own; landing them together
keeps the diff cohesive for review and the release notes simple.
Production-side changes:
- src/node/handler/APIHandler.ts: tighten the OAuth JWT validation
path on the HTTP API. Verify the signature before reading any
claim off the payload, and require the admin claim to be strictly
true (not just present). Switch the apikey comparison to
crypto.timingSafeEqual.
- src/node/handler/{Import,Export}Handler.ts: derive temp-file path
tokens from crypto.randomBytes(16) instead of Math.random.
- src/node/hooks/express/tokenTransfer.ts: enforce a 5-minute TTL
on transfer records, make redemption single-use (remove before
response), and drop the author token from the response body —
the HttpOnly cookie is the only delivery channel.
- src/node/utils/sanitizeProxyPath.ts (new): shared sanitiser for
the `x-proxy-path` header. Used by admin.ts (HTML/JS/CSS
substitution) and specialpages.ts (legacy timeslider redirect).
Strips characters outside [A-Za-z0-9_./-], collapses leading
`//+` to a single `/`, rejects `..` traversal. admin.ts also
emits Vary: x-proxy-path and Cache-Control: private, no-store.
- src/node/db/Pad.ts + src/node/utils/ImportHtml.ts: centralise
the "every insert op carries an author attribute" invariant in
Pad.appendRevision so all non-wire callers (setText, setHTML,
restoreRevision, plugin paths) get the same check the socket
handler already enforces. Pad.init and setPadHTML now
substitute SYSTEM_AUTHOR_ID when no author is supplied — same
pattern setText/spliceText already use.
Tests:
- src/tests/backend/specs/api/jwtAdminClaim.ts (5 cases)
- src/tests/backend/specs/tokenTransfer.ts (6 cases)
- src/tests/backend/specs/proxyPathRedirect.ts (5 cases)
- src/tests/backend/specs/padInsertAuthorInvariant.ts (4 cases)
- src/tests/backend-new/specs/sanitizeProxyPath.test.ts (14 vitest cases)
- src/tests/backend/common.ts: add generateJWTTokenAdminFalse helper.
Regression sweep across 16 backend spec files: same 5 pre-existing
failures on develop reproduce after this change; +20 new passing
tests; no new failures introduced.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* harden: rewrite imported .etherpad records to satisfy the insert-op invariant
Legacy .etherpad exports (and exports from older server-internal flows
that didn't substitute SYSTEM_AUTHOR_ID) can contain `+content` insert
ops without an `author` attribute. The previous commit's appendRevision
guard rejects that shape, but setPadRaw bulk-writes records directly
to the DB and never goes through appendRevision -- so a hand-crafted
.etherpad file could persist non-conforming data that any subsequent
setText / setHTML / restoreRevision call would then refuse to extend.
Add a pre-pass over the parsed import that:
- walks revs in numeric order, sanitising each changeset's `+` ops
against the cumulative pad pool (mutating the pool to register
SYSTEM_AUTHOR_ID when needed);
- re-applies each (post-sanitisation) changeset to a running atext
so the head atext and any key-rev meta.atext / meta.pool
snapshots are re-derived in lock-step with the rewritten revs
(otherwise pad.check's deep-equal at the end of setPadRaw would
fail on the attribute-number drift between the sanitised head
state and the now-stale key-rev snapshot);
- leaves already-conforming payloads untouched (no log noise on
good imports).
Returns the number of ops rewritten so the import can log a single
warning per legacy file rather than per-record. Pure-newline `+` ops
are exempted -- same whitelist as the wire-side guard.
Tests:
- tests/backend/specs/padInsertAuthorInvariant.ts: two new cases
drive setPadRaw end-to-end with a hand-crafted legacy payload
(asserts the head atext gains a `*N` attribute reference and the
pool registers `[author, a.etherpad-system]`) and with a
conforming payload (asserts the data round-trips unchanged).
Regression sweep across 17 backend spec files: 209 passing / 12
pending / 5 failing -- same 5 pre-existing failures present on
unmodified develop. +2 new passing tests; no new failures introduced.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
