export.ts uses await import('jszip'). Previously jszip was a
transitive of html-to-docx and not directly resolvable from src/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ep_markdown and similar plugins ship TypeScript source files and load
them via CJS require('./exportMarkdown') without an extension. Node's
CJS resolver does not recognise .ts by default. Register a Module._extensions
handler at plugins.ts load time that uses esbuild's synchronous transformSync
to compile .ts → CJS on demand. This shim is guard-checked so tsx/vite-node
environments (which already handle .ts) are not affected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix A: plugins calling require('ep_etherpad-lite/node/eejs/') with a
trailing slash need a real file at the wildcard-expanded path
(dist-cjs/node/eejs/.cjs, dist/node/eejs/.mjs). A postbuild script
creates these bridge files after tsdown finishes; the build and pretest
scripts now run it automatically.
Fix B: node/db/*.ts files transitively import ueberdb2 which is ESM-only
(no "require" export condition). Exclude these files from the CJS build
entry set and add specific ./node/db/* exports entries with only the
"import" condition, so CJS plugins get a clean ERR_PACKAGE_PATH_NOT_EXPORTED
instead of the confusing ueberdb2 "no exports main" error. Update
exports_map.ts to verify the new ESM-only behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Six require() calls were missed in the earlier ESM migration: three
top-level requires for native bindings (pdfkit, mammoth) and a transitive
CJS bridge for jszip, plus four lazy/optional requires (Cleanup,
PadManager — circular; html-to-docx, ExportPdfNative — optional).
Top-level pdfkit and mammoth go to static ESM import (both have
esModuleInterop-compatible export= types). JSZip remains on a
createRequire bridge because it is a transitive dependency not symlinked
into node_modules for ESM resolution — a minimal inline type replaces the
implicit any.
Lazy CJS bridge requires become dynamic await import() with .js
extensions, which lets Node's ESM loader resolve the modules correctly.
The enclosing functions (compactPad, anonymizeAuthor, doExport) are
already async. html-to-docx has no published types; a declare module
shim is added to globals.d.ts.
Fixes Cannot-find-module crashes for Cleanup and PadManager in the
compactPad and anonymizeAuthor test suites. All 40 tests in those two
files now pass.
Migrates the remaining 19 test files in tests/backend/specs to static ESM
imports: anonymizeAuthor, authorTokenCookie, colorutils, compactPad,
openapi-admin, padDeletionManager, padInsertAuthorInvariant,
proxyPathRedirect, pwaManifest, sessionIdCookie, settingsModalHeading,
socialMeta-unit, socialMeta, timesliderRedirect, tokenTransfer,
updateActions, updateStatus, updater-integration, and urlBasePath.
Key decisions:
- colorutils cast to `any` (source is @ts-nocheck with untyped empty object)
- compactPad/sessionIdCookie use default assert import for bare assert() calls
- plugin_defs uses default import (module uses export default)
- updateStatus/updateActions use top-level import for updateStatus module
and access _resetBadgeCacheForTests via the imported namespace
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates api/anonymizeAuthor, api/deletePad, api/jwtAdminClaim, and
run_cmd to static ESM imports. Also widens RunCMDOptions.stdio type to
accept string | (string | null | Function)[], matching its actual runtime
usage (opts.stdio = 'string' shorthand was previously untyped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates adminSettingsResolved, adminSettingsSave, anonymizeAuthorSocket,
authorSearch, and padLoadFilter to static ESM imports, removing top-level
require() calls so the files work under vite-node/vitest. Uses named {io}
import from socket.io-client (matching common.ts pattern) to avoid the
default-import callable issue.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tests/backend-new/specs/** are mock-heavy unit tests that need
per-file isolation (vi.mock must apply before the SUT is loaded,
and a shared module graph defeats it). tests/backend/specs/**
share rustydb and need the old isolate:false sequential model.
Split via test.projects (vitest 4): unit gets isolate:true +
parallelism; integration keeps the existing serial config.
Different sequence.groupOrder values (1 vs 2) are required by
vitest when projects have different maxWorkers settings.
Fixes 8 mock-related test failures in updateStatus, firstAuthorOf,
and updateCheck-optout that came in from the develop merge.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When internal code imported via the exports map (ep_etherpad-lite/node/x)
AND via a relative path (../../node/x), vite-node resolved two distinct
module instances. Prom-client top-level Counter() calls ran twice and
threw "metric already registered", cascading ~35 test failures. Fix adds a
resolve.alias in vitest.config.ts that rewrites ep_etherpad-lite/<subpath>
to the .ts source, so the two spellings collapse to one module instance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CJS plugins keep working unchanged via the require condition; ESM
plugins are an opt-in track using extension-explicit imports.
Documents two known limitations: trailing-slash node/eejs/ and CJS
require() of db modules (ueberdb2 is ESM-only).
The 'with plugins' jobs install ep_markdown / ep_readonly_guest / etc.
which require ep_etherpad-lite at install-time. The dist + dist-cjs
twins must exist before pnpm resolves those subpath imports.
Also run check:exports as a fast canary before plugin install.
predev builds once before the dev server starts; dev:watch keeps
tsdown running alongside the server. The pretest hook (added in
the tsdown setup commit) auto-builds before vitest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plugins that ship CJS-only entries (e.g. ep_readonly_guest's
ep_readonly_guest.cjs) and ESM-only entries previously hit the loader's
extensionless fallback path and failed because only .ts and .js were
tried. Add .cjs and .mjs to the candidate list.
The verifier walks the exports map and asserts each target exists.
First run caught that the '.' entry's require condition pointed at
dist-cjs/node/server.cjs which is never built (server.ts has
top-level await, excluded from the CJS build). Drop the require
sub-condition and the now-dead 'main' field; plugins consume
subpaths, not the package root.
The previous attempt added stub .cjs files at dist-cjs/node/eejs/.cjs
to make the trailing-slash require() resolve. But the stubs were
empty — plugins would resolve and then immediately crash calling
methods on an empty module. Worse than failing fast.
Accept that 'require("ep_etherpad-lite/node/eejs/")' (trailing
slash) is not supported by the exports map. Affected plugins must
drop the trailing slash to migrate. The bare form
'require("ep_etherpad-lite/node/eejs")' works as before.
Previous commit dropped the './node/eejs/' entry to silence DEP0155.
That warning applies only to folder mappings (string target ending in
/), not to exact-match keys with conditional object values. Real
plugins use the trailing-slash form (see PR #7605 CI logs).
Implementation notes:
- Re-adds './node/eejs/' exports key with object-condition value.
Node 24/26 still fires DEP0155 for the trailing-slash specifier
(the warning is tied to the caller's import path, not just the
exports key format), but resolution succeeds.
- tsdown build:done hooks emit dist-cjs/node/eejs/.cjs and
dist/node/eejs/.mjs stubs that Node's folder-pattern expansion
resolves to when the import suffix is empty ('eejs/' + '').
- Adds explicit './node/eejs/index.js' and './node/eejs/index'
entries so the ESM import('ep_etherpad-lite/node/eejs/index.js')
is not hijacked by the trailing-slash folder-prefix match.
- Adds 'ep_etherpad-lite/node/eejs/' to cjsResolvableSubpaths in
the exports_map spec.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Routes CJS plugins' require() calls to dist-cjs/*.cjs twins while
keeping ESM consumers on dist/*.mjs (tsdown emits .mjs for ESM). The
trailing-.js wildcard handles plugins that wrote require(...'.js')
with an explicit extension. tests/backend has only an import
condition because CJS build excludes it (top-level await).
Also fixes Settings.ts getEpVersion() to use a static JSON import
instead of a build-path-relative requireFromHere() call, which broke
when resolved from dist-cjs/node/utils/. Test file updated: split
cjsSubpaths into resolvable vs loadable sets since DB modules
transitively depend on ueberdb2 (ESM-only, no require condition).
Builds .ts sources to dist/*.mjs (ESM) and dist-cjs/*.cjs (CJS) so the
upcoming exports map can route plugins' require() calls to the CJS
twin while ESM consumers use the .mjs originals. No source code is
moved or rewritten. tsdown 0.22.0 emits .mjs for ESM regardless of
the outExtension callback; accept that convention.
The CJS entry set excludes node/server.ts (top-level await is not
valid in CJS) and tests/backend/** (common.ts transitively imports
server.ts). The ESM entry set includes both.
vi.setConfig({hookTimeout: N}) inside a before() callback is a no-op —
vitest reads hook timeouts before the hook runs. padLoadFilter genuinely
needs 120s for its setup, so pass it as the second arg. The two 60s
files drop the override entirely (matches the global default already
set in vitest.config.ts).
Three admin test files brought in by the develop merge (5afd466bb) still
used mocha-shape this.timeout/this.skip. Converts them to vi.setConfig
and ctx.skip per the pattern established in 95f753c80.
Both symbols are already exported at lines 300 and 348. The trailing
re-export at line 438 caused esbuild 'Multiple exports with the same
name' build errors in PR #7605 (Backend tests / Linux without plugins).
Captures the design for landing PR #7605 without breaking the existing
plugin ecosystem. Covers the three CI failure causes (duplicate export,
develop merge, plugin resolution) and concretes the dual-emit
ep_etherpad-lite package with an exports map, tsdown build, plugin
loader updates, and a fixture-based testing strategy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related operator-facing docs gaps, both surfaced by #7819:
1. settings.json on disk is a *template*; env-var substitution happens
at load time in memory only. Operators repeatedly mistake the
templated file for a stale config because the docs never spell out
that the on-disk file is intentionally unchanged by env vars.
2. The default docker-compose.yml puts settings.json in the container's
writable layer with no host mount, which means admin /settings edits
are silently lost on `docker compose down && up`, `pull`, or
watchtower — but preserved across plain `restart`. Operators don't
reliably know which compose verbs recreate the container.
Adds two prose sections to doc/docker.md (explaining both gotchas, with
a recreate-vs-restart table) and a commented-out `./settings.json:…`
bind mount in both docker-compose.yml and the README compose example.
Bind mount is opt-in so existing setups behave identically.
No runtime change.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: page sessionstorage cleanup to avoid OOM (#7830)
SessionStore._cleanup() previously called `findKeys('sessionstorage:*',
null)`, materialising every session key into a single array. On decade-
old MariaDB installs with millions of sessions this OOMs the node
process within ~15 minutes — see #7830.
Switch to ueberdb2 6.1.0's findKeysPaged with a 500-key page size, and
yield to the event loop between pages so the DB driver can release each
page's buffered rows and request handlers can interleave.
The break is now driven by `page.length === 0` rather than `page.length
< CLEANUP_PAGE_SIZE` so a stubbed/throttled paged source still iterates
the full keyspace.
Adds a regression test that seeds 50 sessionstorage rows, monkey-patches
`DB.findKeysPaged` to use a 4-key page, runs cleanup, and asserts every
expired row is removed plus every valid row preserved across page
boundaries.
Closes#7830
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address Qodo review on #7831
Four follow-ups raised by Qodo on the session cleanup paging fix:
- DB.ts: fail-fast at init() if any required wrapper method (incl.
findKeysPaged) is missing, so a stale ueberdb2 pin surfaces at boot
rather than crashing the first cleanup run an hour later.
- SessionStore: bound a single _cleanup() run to 10 minutes. Under
sustained session creation the keyspace can grow faster than cleanup
drains it; without a budget the next scheduled run would never fire.
When the budget hits, log a warning and let the next run continue.
- SessionStore: log the defensive `page[0] <= after` cursor-stall break.
Previously the loop exited silently, leaving expired rows behind with
no operator-visible signal of the backend regression.
- Tests: the paged-cleanup regression test now removes both expiredSids
AND validSids in finally, so a failed assertion doesn't leak rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: note paged session cleanup in CHANGELOG + settings template
CHANGELOG.md picks up an entry under 3.1.0 Notable fixes describing the
OOM cause, the paged iteration, the 10-minute per-run budget, the
cursor-stall logging, and the fail-fast init guard.
settings.json.template's sessionCleanup comment adds the page-size,
budget, and pointer to #7830 so admins can reason about the new
behaviour from the template alone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: regenerate lockfile against ueberdb2 6.1.2
Now that ether/ueberDB#983 unblocked the publish workflow (OIDC trusted
publishing), ueberdb2 6.1.2 is live on npm and the `^6.1.0` pin in
src/package.json resolves cleanly. Resolves the ERR_PNPM_OUTDATED_LOCKFILE
that was blocking CI on this PR.
29 SessionStore backend tests still green against the published tarball.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(docker): admin save persists across container restart (#7819)
The OP reports the symptom on the official Docker image specifically.
Adds two layers of coverage to docker.yml's build-test job, driven from
inside a container started against the same TEST_TAG the existing
test-container step uses:
1. New mocha spec adminSettings_7819.ts under tests/container/specs/api
— authenticates against /admin, opens the /settings socket, saves an
augmented JSON with an ep_oauth-shaped top-level block, and asserts
the next load reply contains the marker. Intentionally leaves the
marker on disk so the workflow can inspect it.
2. docker.yml now `docker exec test grep`s for the marker after
test-container, then `docker restart`s the container, waits for the
health probe, and re-greps. Both checks must pass — the first proves
the socket-driven save actually touched the file inside the
container layer; the second proves an in-place restart doesn't reset
it. A recreate (docker rm + docker run) would wipe the file, but
that's expected (image layer) and out of scope.
Container is started with `-e ADMIN_PASSWORD=changeme1` so the existing
settings.json.docker provisions the admin user; pad.js doesn't touch
/admin so the existing API specs are unaffected. test-container timeout
bumped 5s → 30s to cover socket connect + save round-trip, and the
mocha discovery extension list now includes `ts` so the new spec is
picked up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(docker): authenticate via /admin-auth/ POST, surface auth/load failures fast (#7819)
CI failed on #7821 with a generic 20s mocha timeout because the spec
hit GET /admin/ to grab a session cookie. webaccess.ts only treats
paths starting with /admin-auth as requireAdmin — and the container
runs with REQUIRE_AUTHENTICATION=false (default), so GET /admin/ never
issued a Basic challenge and Set-Cookie was empty. The socket then
connected unauthenticated, adminsettings.ts's connection handler
returned early without binding any listeners, and the load() promise
hung until mocha killed the test with no useful diagnostic.
Switch to POST /admin-auth/ (always-requireAdmin, regardless of
settings.requireAuthentication). Assert a 2xx with at least one
Set-Cookie before proceeding. Add an 8s timeout + meaningful error
message to load() so the "session was not admin" failure mode reports
immediately instead of burning the suite budget.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(docker): replace splice with hand-built payload (#7819)
Last CI failed because the splice-after-last-} approach landed a comma
between an existing trailing-comma-before-comment and the close brace
of settings.json.docker, producing `, /* … */, "ep_oauth"` — invalid
JSON.
settings.json.docker uses jsonc `/* */` and `//` comments and a
trailing-comma-before-comment-before-close shape that's annoying to
patch from the test side, and the existing isJSONClean has zero
backend coverage so the splice is going through Etherpad's lenient
write path anyway.
Switch to a hand-built minimal-but-viable settings document containing
the ep_oauth block. Three properties hold:
- We're testing the WRITE path, not the synthesis path. Whatever
bytes we send, the next `load` must return verbatim.
- The post-save document must survive `docker restart` (the next
step in docker.yml) — minimal-but-viable means port/users/dbType
are present so Etherpad boots back up and HEALTHCHECK passes.
- The next `load` reply must equal the bytes we saved
(`reply.results === augmented`) — stronger than `.includes()`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The admin saveSettings socket had zero direct backend coverage and the
e2e 'restart works' test only checked the page renders after restart —
neither catches a deployment that resets settings.json on restart, nor
the user-visible workflow that triggered #7819 (add a top-level plugin
block via Raw, save, watch it disappear).
Adds three backend specs for the saveSettings socket:
- payload is written byte-for-byte to settings.settingsFilename
- augmenting existing JSON with a new top-level block round-trips
through the next load reply
- /* */ comments survive the write path
Adds one e2e spec mirroring the #7819 workflow: open Raw, prepend an
ep_oauth-shaped top-level block, save, restartEtherpad(), re-login,
confirm the block is still in Raw and surfaces as its own Form-view
section ('Ep oauth' from humanize()).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
