* feat(export): native DOCX export via html-to-docx (opt-in)
Addresses #7538. The current DOCX export path shells out to LibreOffice,
which means every deployment that wants a Word download either installs
soffice (~500 MB) or loses that export. This PR adds a pure-JS
alternative: render the HTML via the existing exporthtml pipeline, then
feed it to the `html-to-docx` library in-process to produce a valid
.docx buffer — no soffice required, no subprocess spawn, no temp file
dance for the DOCX case.
Behavior:
- `settings.nativeDocxExport` (default `false`) gates the new path so
existing deployments see zero behavior change.
- When enabled, `type === 'docx'` requests skip the LibreOffice branch,
run `html-to-docx(html)`, and return the buffer with the
`application/vnd.openxmlformats-officedocument.wordprocessingml.document`
content-type.
- If the native converter throws, the handler falls through to the
existing LibreOffice path — so flipping the flag on is safe even on a
mixed-installation where soffice is still present as a backstop.
- Other export formats (pdf, odt, rtf, txt, html, etherpad) are
unchanged.
Files:
- `src/package.json`: `html-to-docx` dep (pure JS, no binary reqs)
- `src/node/handler/ExportHandler.ts`: new DOCX branch gated on the
setting, with fall-through on error
- `src/node/utils/Settings.ts`, `settings.json.template`,
`settings.json.docker`, `doc/docker.md`: wire up the new setting +
env var (`NATIVE_DOCX_EXPORT`)
- `src/tests/backend/specs/export.ts`: two new tests — asserts the
exported buffer is a valid ZIP (PK\x03\x04 signature) and the
response carries the correct content-type — both with
`settings.soffice = 'false'` to prove the path doesn't need soffice
at all.
Out of scope for this PR:
- Native PDF export (would need a PDF rendering step — separate
undertaking, and the issue acknowledges the `pdfkit`/puppeteer size
trade-off).
Closes#7538
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(7538): skip native DOCX test when html-to-docx isn't installed
The upgrade-from-latest-release CI job installs deps from the previous
release's package.json (before this PR adds html-to-docx) and then
git-checkouts this branch's code without re-running pnpm install.
Under that one workflow the new test can't find the module and fails
on the LibreOffice fallback, masking that the native path actually
works in every normal install.
Guard the describe block with require.resolve('html-to-docx'); Mocha's
this.skip() on before cascades to the sibling its. Regular backend
tests (pnpm install against this branch's lockfile) still exercise it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(7538): spec for soffice-free DOCX/PDF export and DOCX import
Captures the agreed scope expansion of PR #7568: replace the flag-gated
native DOCX path with a soffice-first selection cascade, add native PDF
export via pdfkit + a small htmlparser2-driven walker, and add native
DOCX import via mammoth. Also defines a shared HTML sanitizer
(stripRemoteImages) used by both export converters to close the
SSRF surface that Qodo flagged on the html-to-docx path.
The spec drops the nativeDocxExport setting and its env var; with
soffice configured, behavior is unchanged, and with soffice null,
docx/pdf export and docx import all work in-process. odt/doc/rtf
(and pdf import) keep needing soffice and are documented as such.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(7538): implementation plan for native DOCX/PDF and DOCX import
Bite-sized TDD task breakdown of the soffice-free export/import work:
rebase, deps, sanitizer, PDF walker, mammoth wrapper, ExportHandler
cascade, route guard, ImportHandler branch, UI fix, flag rollback,
verification + Qodo reply.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(7538): add pdfkit, htmlparser2, mammoth deps
Pure-JS, no native binaries:
- pdfkit ^0.18.0 (PDF rendering)
- htmlparser2 ^12 (SAX parser used by walker + sanitizer)
- mammoth ^1.12 (DOCX -> HTML for native import)
- @types/pdfkit ^0.17 (dev)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(7538): add stripRemoteImages HTML sanitizer
Drops <img src=> elements pointing at non-data, non-relative URLs to
prevent the DOCX/PDF converters from making outbound requests via
plugin-modified HTML. Closes Qodo finding #4 against the
html-to-docx path; will be wired into both export branches in
the cascade refactor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(7538): native PDF export via pdfkit + htmlparser2 walker
Renders pad HTML to a PDF Buffer in-process: headings, paragraphs,
lists, links, inline emphasis, data:-URI images. Remote images are
explicitly skipped at the walker (defense-in-depth on top of the
shared stripRemoteImages sanitizer).
PDFs are emitted with compress:false so accessibility/SEO indexers
that don't FlateDecode can still extract text. Pads are small enough
that the size cost is negligible.
Walker is 167 LOC, well under the spec's 500-LOC bail-out
threshold for switching to pdfmake+html-to-pdfmake+jsdom.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(7538): native DOCX import via mammoth
Wraps mammoth.convertToHtml so a soffice-less Etherpad can ingest
.docx files. Images are coerced to data: URIs at the converter
boundary so the import pipeline never sees a remote src=.
Includes a tiny generated DOCX fixture (heading, paragraph, list)
under tests/backend/specs/fixtures/ for both this wrapper test and
the upcoming end-to-end import test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(7538): soffice-first cascade in ExportHandler
Replaces the flag-gated DOCX branch with a deterministic dispatch:
soffice if configured, native DOCX/PDF otherwise, 5xx on native
error. Both native paths run plugin-modified HTML through
stripRemoteImages first.
Test changes:
- existing native DOCX block now sets soffice=null (was 'false', a
truthy non-null string that sidestepped the route guard); fixes
Qodo finding #3.
- new native PDF integration tests assert %PDF- header and
application/pdf content-type with soffice=null.
- new negative test: with soffice=null, /export/odt still returns
the 'not enabled' message.
- the legacy 500-on-export-error test now uses /bin/false so it
exercises the soffice error path explicitly (the cascade dropped
the ad-hoc 'false' string; .doc has no native path so this still
works as a soffice error probe).
Integration tests for native DOCX/PDF currently fail because the
/export route guard still treats both formats as soffice-only;
the next commit fixes that.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): allow docx/pdf through export guard without soffice
Tightens the no-soffice block to ['odt','doc'] only — formats with
no native path. docx and pdf are handed to ExportHandler, which
dispatches to the in-process converters. Closes Qodo finding #2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(7538): native DOCX import path in ImportHandler
When soffice is null and the upload is .docx, run mammoth and feed
the resulting HTML through setPadHTML. Other office formats
(pdf/odt/doc/rtf) are explicitly rejected with uploadFailed instead
of silently falling through to the ASCII-only path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): always show DOCX/PDF export links
Native paths (#7538) make DOCX and PDF available regardless of
soffice presence, so unconditionally render those links. ODT still
gates on exportAvailable. Closes Qodo finding #2 on the UI side.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(7538): drop nativeDocxExport flag
Selection is now purely soffice-presence-driven (cascade in
ExportHandler). The opt-in setting and its NATIVE_DOCX_EXPORT env
var are no longer needed -- soffice configured means soffice path;
soffice null means native path (DOCX, PDF, and DOCX import).
Reverts the additive surface introduced earlier in this PR.
Also updates the SOFFICE doc row to reflect that null no longer
means 'plain text and HTML only' -- docx/pdf export and docx
import now work natively without soffice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(7538): tighten link annotation assertion
CodeQL flagged the loose 'raw.includes("etherpad.org")' as
'incomplete URL substring sanitization' (a false positive in test
context, but worth fixing). Match the full /URI (host) form
instead -- it's both more accurate (we're verifying the PDF link
annotation structure) and CodeQL-clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): cleaner DOCX/PDF output + round-trip test coverage
DOCX:
- New extractBody helper drops <head>/<style> and the leading
newline inside <body> so html-to-docx doesn't render CSS or
prefix paragraphs with empty space.
- New wrapLooseLines pre-processor wraps loose pad lines in <p>
before the converter sees them. html-to-docx renders <br>
outside <p> as a new <w:p> (full empty line in Word); inside
<p> it correctly emits <w:br/> (soft break). Etherpad's HTML
uses bare <br> for every line, so this was making single
Enters look like double Enters in the Word output.
PDF:
- Walker SKIP_TAGS rejects head/style/script/title/meta/link
content -- prior version dumped CSS into the rendered PDF.
- New breakLine() helper combines flushLine() with moveDown(1).
pdfkit's text('', false) closes the continued run but does
NOT advance the cursor, so consecutive runs were stacking at
the same y-coordinate. <br>, end-of-block, and list items
now use breakLine().
- ontext collapses runs of whitespace and drops pure-whitespace
text nodes so pretty-printed source HTML doesn't render its
formatting newlines.
Round-trip:
- New backend test: pad text -> DOCX export -> DOCX import ->
new pad. Asserts content survives the trip.
- New PDF sanity test: extracts visible text from the PDF stream
and asserts the source pad text appears verbatim.
- 6 new unit tests for extractBody and wrapLooseLines plus 1 for
PDF walker SKIP_TAGS coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): CodeQL ReDoS + import.ts type error
- BR_PARA_RE was /(?:\s*<br>\s*){2,}/ -- two adjacent \s* runs can
match the same chars, so on '<br>\t<br>\t<br>...' the regex
backtracks exponentially. Re-anchored to match a fixed first <br>
followed by one or more additional <br>s, so each whitespace run
has exactly one home.
- import.ts: fetchBuffer was typed Promise<Buffer> but call sites
chained .expect(200) on it, which only works on supertest's Test
object. Return the Test (typed any) so the chain is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): plugin-aware HTML cleanup, PDF text-align, monospace
ep_headings2/ep_align emit one heading-styled blank-line block after
every styled line in the pad ('<h1 style=text-align:right></h1>'),
which both html-to-docx and our pdfkit walker render as a full empty
paragraph. Plus the pdfkit walker had no support for text-align or
monospace, so right/center alignment and 'code' lines rendered the
same as plain body text.
- New dropEmptyBlocks helper strips empty h1-h6/p/code/pre/div/
blockquote wrappers in preprocessing. Iterates so nested empties
collapse too. Applied before both DOCX and PDF conversion.
- PDF walker now reads style='text-align:left|center|right|justify'
on block elements (h1-6, p, div) and passes it as pdfkit's align
option. align is applied once per continued run, then reset on
flushLine so the next block can pick up its own value.
- PDF walker handles <code>, <tt>, <kbd>, <samp> as inline monospace
(Courier) and <pre> as block monospace (Courier + breakLine on
open/close).
11 new unit tests:
- 4 for dropEmptyBlocks (heading wrappers, code, nesting,
pass-through)
- 1 for PDF text-align (compares the BT matrix x for left vs right)
- 2 for Courier in <code> and <pre>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): separate adjacent heading-style blocks on import
Round-trip bug: ep_headings2 emits <h1>/<h2>/<code> in pad HTML.
Mammoth round-trips them as adjacent <h1>A</h1><h2>B</h2><p>C</p>
on import. Etherpad's server-side content collector has a default
_blockElems set of just {div, p, pre, li}, and ep_headings2 only
registers the CLIENT-side aceRegisterBlockElements -- not the
server-side ccRegisterBlockElements. So h1/h2/code end up being
treated as inline by the importer, and adjacent blocks merge into
a single pad line.
Fix: insert <br> after </h1>...</h6>/</code> when followed by
another block. Server-side workaround keeps this PR self-contained
regardless of plugin version. The right long-term fix is to extend
ep_plugin_helpers' lineAttribute factory to register both hooks
(filed as a follow-up).
Tests:
- 5 unit tests for separateAdjacentHeadingBlocks
- New end-to-end round-trip test asserts H1+H2+P land on three
separate pad lines after the import path.
Plus the prior PDF text-align/Courier/code commit also included
here:
- code/tt/kbd/samp inherit text-align from style attribute
- pre inherits text-align too
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): blank-line round-trip + DOCX code monospace + a==c tests
DOCX round-trip dropping blank pad lines:
- wrapLooseLines now emits an explicit <p></p> marker for each blank
line in a <br> run, instead of collapsing all gaps into a single
paragraph break. (N consecutive <br>s -> 1 paragraph boundary +
N-2 empty <p></p> markers, mapping to N-1 blank pad lines.)
- mammoth's docxBufferToHtml now passes ignoreEmptyParagraphs:false
so the empty <w:p> entries survive the import side. mammoth's
default of true was silently dropping them.
- dropEmptyBlocks no longer strips <p></p> -- that's the meaningful
marker for the round-trip. Empty <h1>/<code>/<pre>/<div>/
<blockquote> are still stripped (plugin noise).
DOCX <code> rendering as monospace:
- New applyMonospaceToCode wraps code/tt/kbd/samp/pre content in a
<span style="font-family:'Courier New', monospace">. html-to-docx
honors that and emits <w:rFonts w:ascii="Courier New".../>, which
Word renders as Courier. The bare <code> tag is otherwise just
a no-op for html-to-docx.
- Applied only on the DOCX export path (PDF walker already handles
monospace via Courier font selection).
Round-trip tests:
- New a==c suite: txt, etherpad, html, docx -- export from src,
import to dst, re-export and compare against the meaningful
invariant (line text for binary formats; trimmed body for HTML).
- HTML test tolerates one trailing <br> per round-trip because
setPadHTML appends a final <p> on import; this is pre-existing
core behavior, not our bug.
- DOCX test normalizes trailing newline run (same reason).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): preserve <w:jc> alignment through mammoth round-trip
mammoth doesn't expose Word's paragraph alignment (`<w:jc>`) when it
converts a docx to HTML -- there's no equivalent in its style-mapping
machinery. To keep alignment through DOCX round-trips we walk the
docx's document.xml directly, pull the `w:val` from each `<w:p>`'s
`<w:jc>`, and inject `style="text-align:..."` onto the matching
block element in mammoth's output by document order.
Word's w:jc accepts more values than CSS text-align; we map left/
start, center, right/end, both/justify/distribute and skip the rest
(start/end take left/right because we don't track ltr/rtl from the
docx for now).
Combines with the upstream ep_align PR (ether/ep_align#183) for the
full round-trip: this PR makes the mammoth output carry the
alignment style; ep_align#183 makes the importer pick it up.
Closes the alignment side of #7538.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): preserve <a href> inside <code>/<pre> in DOCX export
html-to-docx silently drops <a href> children of <code>/<pre> tags
(and of styled <span>s, but the <code> wrapper is the active offender
here). The pad-export HTML produced by ep_headings2 + ep_align uses
<code style='text-align:right'>...<a href>...</a>...</code> for each
'Code'-style line, which lost its links on every DOCX export.
Workaround: applyMonospaceToCode now drops the code/pre/tt/kbd/samp
wrapper entirely. The non-anchor content gets wrapped in monospace
spans; anchors are emitted unstyled so they keep their hyperlink. For
block-level usage (<pre>, or <code> with an inline style attr) we
emit a wrapping <p> and forward the text-align style. Run BEFORE
wrapLooseLines so the <p> doesn't get double-wrapped.
Tests added:
- inline <code> -> just a styled span (no <code> wrapper)
- <code style='text-align:right'> -> <p style> wrap
- <pre> -> always block-wrapped
- <tt>/<kbd>/<samp> -> inline span only
- regression: <a href> inside <code> survives html-to-docx round-trip
with both the URL in word/_rels/document.xml.rels AND a <w:hyperlink>
in the document body
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(7538): HTML import line-break drift between blocks
Etherpad's HTML export wraps each pad line in <p>...</p> (or <h1>,
<code>, etc.) and then appends a <br> between lines. The closing
block tag already ends the line for contentcollector, so the trailing
<br> is redundant -- and on import the server collector counts BOTH
as line breaks, doubling every blank line between paragraphs and
inserting an extra blank between adjacent headings.
Two fixes, both gated on the runtime block-element registry so they
don't double-trigger when the underlying plugin already handles
adjacency:
1. HTML import path now runs the new collapseRedundantBrAfterBlocks
helper before setPadHTML. Drops a single <br> immediately
following </p>/</h1-6>/</code>/</pre>/</div>/</blockquote>/</ul>/
</ol>/</li>/</table>/</tr>/</td>/</th>. Multiple consecutive
<br>s after a block keep all but the first (the rest still
represent intentional blank lines).
2. The DOCX-import separateAdjacentHeadingBlocks workaround now
checks whether 'h1' is in the runtime ccRegisterBlockElements
set before inserting <br>s. When ep_headings2 has the new server
hook (per ep_plugin_helpers#14 + the upcoming ep_headings2 PR),
the workaround correctly stays out of the way -- otherwise it
adds an extra blank line per heading transition.
Also fixed a subtle ts-check failure on the import.ts test changes
and a leftover implicit-any in ImportDocxNative's alignment
preserver.
Tests added:
- collapseRedundantBrAfterBlocks: 5 unit tests (each block tag,
whitespace tolerance, multiple <br> keeping intentional blanks)
- HTML import: 'does not introduce a blank line between H1 and H2',
'preserves blank-line count between H1 and H2 (realistic shape)'
reproduces the 5-blanks-where-2-expected bug from the user's
round-trip pad.
1054 backend tests pass locally (the 6 failures are the pre-existing
favicon/webaccess send@1.x dotfile-path issue from running under
.claude/, doesn't reach CI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(7538): skip plugin-dependent HTML import tests on no-plugin CI
The two new HTML-import-adjacency tests assume ep_headings2 (or
another plugin) has registered h1/h2 as server-side block elements
via ccRegisterBlockElements. Without that, contentcollector treats
<h1>/<h2> as inline and adjacent ones merge into a single pad line
-- making the assertions inapplicable.
CI's backend-tests job runs without plugins installed, so guard
the describe block with a runtime hooks.callAll() check and skip
when h1 isn't a registered block. Local dev with ep_headings2 (and
the local plugin patch wiring ccRegisterBlockElements) still
exercises both tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(padOptions): pass plugin-namespaced ep_* keys through applyPadSettings
Native pad-wide settings ride a single padOptions object: the server seeds
clientVars.initialOptions, the client mutates via pad.changePadOption(), and
the existing padoptions COLLABROOM message broadcasts changes. Plugins can't
use the same rail today because applyPadSettings (client) and
normalizePadSettings (server) silently drop any key not in their hardcoded
whitelist.
Add a passthrough loop that preserves keys matching /^ep_[a-z0-9_]+$/ on both
sides. Plugins can now stash their pad-wide values under their own namespace
(e.g. pad.padOptions.ep_table_of_contents = {enabled: true}) and inherit the
existing broadcast, persistence, creator-only-write enforcement, and
enforceSettings semantics for free.
A new src/node/utils/PluginCapabilities module exposes
padOptionsPluginPassthrough = true so plugins can feature-detect via
require() and fall back to per-user behavior on older cores.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Address Qodo review on PR #7698
Four concerns raised by Qodo (qodo-free-for-open-source-projects):
1. Feature flag — AGENTS.MD §52 requires new features behind a flag,
disabled by default. Add `enablePluginPadOptions` (default false) gating
the passthrough on both server (normalizePadSettings) and client
(applyPadSettings, via clientVars). Plugins detect the runtime state
through clientVars.enablePluginPadOptions; the static
PluginCapabilities flag stays as the "core can do this" signal.
2. Documentation — add a "Plugin-namespaced pad-wide options" section to
doc/plugins.md covering capability detection, the runtime flag, the
key namespace pattern, and the validation rules. Mirror the flag
description in settings.json.template.
3. Unbounded payload — values for ep_* keys are persisted with the pad and
broadcast to every connected client, so an unvalidated path was a
reliability hazard. Validate every ep_* value:
- Must round-trip through JSON.stringify (rejects functions, symbols,
BigInt, circular refs).
- Per-key serialized size capped at 64 KB.
- Combined ep_* size capped at 256 KB per pad.
Rejects drop the value with a console.warn line; the rest of the pad
settings round-trip cleanly.
4. PadOption type — add `[k: \`ep_${string}\`]: unknown` index signature
so the SocketIO message type matches runtime behavior; TS callers no
longer need unsafe casts to read plugin-namespaced keys.
Also extends the backend test suite with cases covering the runtime flag
(off/on), JSON-serializability rejection, per-key cap, and total cap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(snap-tests): assert_grep — use here-string to dodge pipefail SIGPIPE
`assert_grep` ran `printf '%s' "$out" | grep -q -F -- "$needle"` under
`set -o pipefail`. When grep matched early it closed its stdin, printf
got SIGPIPE on its next write (exit 141), and pipefail propagated the
broken-pipe failure to the pipeline — making `if` see non-zero and
falling into the FAIL branch even though grep itself succeeded.
Failure was timing-dependent: it only fired when `$out` was large enough
that printf hadn't flushed before grep exited. CI ubuntu-latest tipped
into the racy path on PR #7698 once `settings.json.template` grew by 11
lines (the new `enablePluginPadOptions` flag); the symptom was the
`Wrapper unit tests` step reporting `dbType rewritten to sqlite ✗` with
"got: /*…" output even though the seeded file did contain the needle.
Replace the pipe with a here-string so grep gets its input in one shot
with no pipe between processes — no SIGPIPE possible. The fail-message
`head -3` is converted to a here-string for the same reason.
Repro on a runner whose pipe-buffer flush is slower than grep's first
match would have hit the same flake on any PR; the bug isn't about
this particular template change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(socketio): don't kick authenticated duplicate-author sessions (#7656)
The CLIENT_READY handler kicks any prior socket whose authorID matches the
joining socket's, originally as a workaround for stale tabs in the same
browser (cookie-derived authorIDs were per-browser, so "same authorID, same
pad" reliably meant "page refresh / second tab in this browser").
With stable identities (basic auth, SSO, apikey, getAuthorId hook) the same
authorID can legitimately appear across windows or devices, so the kick
disconnects real concurrent sessions. Skip the kick when the joining socket
has req.session.user set; cookie-only sessions keep the existing behavior so
the userdup modal and the xxauto_reconnect path still work.
* fix(socketio): suppress USER_LEAVE when other same-author sockets remain
With the duplicate-author kick disabled for authenticated sessions, a single
authorID can legitimately span multiple sockets in one pad. handleDisconnect
was emitting USER_LEAVE on every socket close, which made clients (whose
presence is keyed by authorID) drop the author entirely even when another
socket of theirs was still online.
Only broadcast USER_LEAVE — and only run the userLeave hook — when the
disconnecting socket is the last one in the pad for that author.
Adds two backend tests:
- authenticated identity: closing one of two same-author sockets does NOT
emit USER_LEAVE on the other.
- different authors (regression): closing socket A still emits USER_LEAVE
for socket B.
Action of Qodo review feedback on PR #7678.
* docs: PR5 GDPR author erasure design spec
* docs: PR5 GDPR author erasure implementation plan
* feat(gdpr): AuthorManager.anonymizeAuthor — Art. 17 erasure
* test(gdpr): AuthorManager.anonymizeAuthor unit tests
* feat(gdpr): REST anonymizeAuthor on API version 1.3.1
* test(gdpr): REST anonymizeAuthor end-to-end
* docs(gdpr): right-to-erasure section + anonymizeAuthor example
* fix(gdpr): make anonymizeAuthor resumable on partial failure
Qodo review: the `erased: true` sentinel was written before the chat
scrub loop, so a throw during scrub left chat messages untouched
while subsequent calls short-circuited on `existing.erased` and never
finished. Split the write: zero the display identity first (still
hides the name), run the chat scrub, and only then stamp
`erased: true` so a retry resumes the sweep. Regression test
covers the partial-run → retry path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: PR4 GDPR privacy banner design spec
* docs: PR4 GDPR privacy banner implementation plan
* feat(gdpr): typed privacyBanner setting block + public getter exposure
* feat(gdpr): send privacyBanner config to the browser via clientVars
* feat(gdpr): privacy banner DOM (hidden by default)
* feat(gdpr): render privacy banner on pad load when enabled
* style(gdpr): privacy banner layout
* test+fix(gdpr): privacy banner Playwright + hidden-attr CSS override
* docs(gdpr): privacyBanner configuration section
* fix(gdpr): reject unsafe learnMoreUrl schemes
Qodo review: showPrivacyBannerIfEnabled assigned config.learnMoreUrl
directly to <a href>, so a misconfigured settings.privacyBanner.
learnMoreUrl of `javascript:alert(1)` or `data:…<script>…` would run
script on click. Validate via URL parsing and allow only http(s) /
mailto; everything else yields no link. Playwright regression guards
the four cases (javascript, data, https, mailto).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(privacy-banner): drop unneeded !important on [hidden] rule
Class+attribute selector already outranks `.privacy-banner { display: flex }`
on specificity (0,2,0 vs 0,1,0), so `!important` was redundant. Adds a
comment explaining why so a future reader doesn't put it back.
Per Sam's review on #7549.
* refactor(privacy-banner): render as a persistent gritter, not custom DOM
Drops the bespoke #privacy-banner template + ~50 lines of popup.css and
delegates to $.gritter.add({sticky: true, position: 'bottom'}). The
notice now matches every other gritter on the pad (theme variables,
shadow, animation, (X) close), sits in the bottom corner instead of
above the editor, and inherits dark-mode handling for free.
The two dismissal modes survive intact:
- dismissible: gritter closes on (X); before_close persists a flag
in localStorage so the notice is suppressed on subsequent loads.
- sticky: closes for the current session only; never persists; the
next pad load shows it again.
learnMoreUrl still goes through the same safeUrl() filter so a
javascript:/data:/vbscript: URL can't smuggle a script handler into the
anchor (Qodo's review concern remains addressed).
Tests: src/tests/frontend-new/specs/privacy_banner.spec.ts now drives
the real showPrivacyBannerIfEnabled via a __etherpad_privacyBanner__
test hook and asserts against the rendered gritter, instead of the
previous tests that mutated DOM by hand and never exercised the
function under test. Coverage adds: enabled=false short-circuit,
dismissible-flag-respected on subsequent show, sticky-ignores-flag,
sticky-close-does-not-persist, javascript: rejection, data: rejection,
and mailto: allow-list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(privacy-banner): noreferrer + validate dismissal (Qodo)
Two follow-ups from Qodo's review on #7549:
1. The Learn-more link now sets `rel="noreferrer noopener"` (was just
`noopener`). Without `noreferrer` the browser sends the pad URL as a
Referer to the operator-configured external policy site, which leaks
pad identifiers to a third party. Matches the rel pattern already
used by pad_utils.ts.
2. `privacyBanner.dismissal` is now validated in reloadSettings(): an
unknown value falls back to 'dismissible' with a `logger.warn`, in
the same shape as the existing ipLogging validation a few lines up.
The client also guards defensively (treats anything other than the
exact string 'sticky' as 'dismissible') so that hot-reload paths
that skip the server validator can't silently degrade a typo'd
'sticky' into "no close button persisted, no localStorage suppression".
Test added: spec asserts the rel attribute, and a new test exercises
the dismissal fallback (sets dismissal:'wat', asserts the gritter is
shown, the (X) closes it, and the dismissal flag is persisted — i.e.
the unknown value is treated like 'dismissible').
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(privacy-banner): gate test hook on webdriver, align doc with sticky behavior
Two follow-ups from Qodo's second review on #7549.
Rule violation: __etherpad_privacyBanner__ was published on every pad
load even when privacyBanner.enabled was false, so the disabled-by-
default feature still added an observable global. Gate the assignment
on `navigator.webdriver` — Playwright/ChromeDriver/Selenium set this
to true; production browsers do not — so the hook is only present for
tests and the disabled path is genuinely zero-side-effect.
Bug 3 (sticky still closable): doc/privacy.md previously claimed
`dismissal: "sticky"` removes the close button, but the gritter
implementation always renders (X). Aligning the doc with reality —
sticky now means "shows on every load, but closable for the session"
— rather than adding bespoke CSS to a vanilla gritter (matches the
"don't style it differently than other gritter messages" preference
that drove the gritter migration in 906e145).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(privacy-banner): allow-list keys before sending to clientVars (Qodo)
storeSettings() merges nested objects with _.defaults() and preserves
unknown nested keys, and TypeScript's Pick<> doesn't strip at runtime.
The previous wire path forwarded settings.privacyBanner by reference
into both clientVars and getPublicSettings(), so any extra keys an
operator typed (or pasted) under privacyBanner — credentials, internal
notes, anything — would have shipped to every browser on every pad
load.
Adds getPublicPrivacyBanner() in Settings.ts that returns a literal
with only {enabled, title, body, learnMoreUrl, dismissal}, and uses it
from both leak sites (PadMessageHandler.ts clientVars and
getPublicSettings()). Single source of truth for the wire shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: PR3 GDPR anonymous identity hardening design spec
* docs: PR3 GDPR anon identity implementation plan
* feat(gdpr): ensureAuthorTokenCookie helper — HttpOnly server-set author token
* feat(gdpr): set HttpOnly author-token cookie from the pad routes
* feat(gdpr): read author token from cookie first, keep message.token fallback
* feat(gdpr): stop generating the author token client-side
* test(gdpr): server sets + reuses the HttpOnly author-token cookie
* fix+test(gdpr): parse token cookie from handshake Cookie header
socket.io handshake doesn't run cookie-parser, so socket.request.cookies
is undefined. Parse the Cookie header directly in handleClientReady so
the HttpOnly token actually resolves. Playwright spec covers HttpOnly
attribute, reload-stability, and context-isolation.
* docs(gdpr): token cookie is now HttpOnly + server-set
* fix(gdpr): close two HttpOnly token bypasses
Qodo review:
- Timeslider still ran the pre-PR3 JS-cookie path: it read
Cookies.get('${cp}token') (which HttpOnly hides), then generated a
fresh plaintext token and overwrote the server's HttpOnly cookie with
it, and sent token in every socket message. Strip the token read/
write entirely from timeslider.ts and from the outgoing message
shape; the server reads the cookie off the socket.io handshake just
like on /p/:pad.
- tokenTransfer re-issued the author cookie without HttpOnly, undoing
the hardening the first time a user transferred a session. Re-set
it as HttpOnly + Secure (on HTTPS) + SameSite=Lax. Also stop
trusting the body-supplied token on POST: read it off req.cookies
server-side so the client never needs JS access to the token.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(pad): compactHistory() + compactPad CLI for DB-size reclaim
Fixes#6194. Long-lived pads with heavy edit history dominate the DB —
the issue describes a ~400 MB Postgres after two months with ~100
users. Etherpad keeps every revision forever, and removing arbitrary
middle revisions is unsafe because state is reconstructed by composing
forward from key revisions.
What's safe: collapse the full history into a single base revision
that reproduces the current atext. The existing `copyPadWithoutHistory`
already does this for a new pad ID — this PR lifts that same changeset
pattern into an in-place operation and wires up an admin CLI.
- `Pad.compactHistory(authorId?)` (src/node/db/Pad.ts): composes the
current atext into one base changeset, deletes all existing rev
records, clears saved-revision bookmarks, and appends the new rev 0.
Text, attributes, and chat history are preserved; saved-revision
pointers are cleared. Returns the number of revisions removed.
- `API.compactPad(padID, authorId?)` (src/node/db/API.ts): public-API
wrapper around compactHistory. Reports `{removed}` so callers can
log savings.
- `APIHandler.ts`: register `compactPad` under a new `1.3.1` version,
bump `latestApiVersion`.
- `bin/compactPad.ts`: admin CLI. Reports the current revision count,
calls compactPad via the HTTP API, and prints how many revisions
were dropped.
- `src/tests/backend/specs/compactPad.ts`: four backend tests cover
the empty-pad no-op, the text-preservation + head=0 contract,
saved-revision cleanup, and that subsequent edits continue to
append cleanly on top of the collapsed base.
The operation is destructive so admins must opt in explicitly; the CLI
prints the before-count, and the recommended pre-flight is an
`.etherpad` export (backup).
Closes#6194
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(compact): delegate to copyPadWithoutHistory via temp-pad swap
The initial compactHistory() implementation built a custom base
changeset and re-ran appendRevision against a reset atext — but the
changeset was packed with oldLength=2 (matching copyPadWithoutHistory's
dest-pad init state) while the reset atext was only length 1, so
applyToText tripped its "mismatched apply: 1 / 2" assertion and every
test failed with a Changeset corruption error.
Switch to the tested path instead: copy the pad via
copyPadWithoutHistory to a uniquely-named temp pad (inherits all its
attribute/pool/changeset correctness), read the temp pad's rev records
back, delete the old ones under our pad's ID, write the new records in
their place, update in-memory state to match, and remove the temp pad.
Errors at any step fall through with a best-effort temp-pad cleanup.
Contract shifts slightly: the collapsed pad is head<=1 rather than
head=0, matching the shape of a freshly-imported pad (seed rev 0 +
content rev 1). Tests updated to assert that invariant plus
text-preservation, saved-revision cleanup, and append-after-compact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(6194): match the head<=1 post-compact contract
Tests previously asserted head=0 exactly after compaction; the
temp-pad-swap path lands at head=1 (one seed rev plus one content
rev) matching the shape of a freshly-imported pad. Relax the
assertions to and derive the removed-count from
before-head minus after-head, so the tests still catch regressions in
text-preservation, saved-revision cleanup, and append-after-compact
without being tied to the exact implementation shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(6194): wrap existing Cleanup instead of duplicating it
Develop already ships a working revision-cleanup path under
`src/node/utils/Cleanup.ts` with two public helpers —
`deleteAllRevisions(padId)` (collapse full history via
copyPadWithoutHistory) and `deleteRevisions(padId, keepRevisions)`
(keep the last N). The admin-settings UI wires these up but neither
is exposed on the public API, and there's no CLI for operators who
want to run compaction outside the web UI. That's the gap this PR
now fills.
Changes from the prior revision of this PR:
- Drop `pad.compactHistory()` — it re-implemented what
`Cleanup.deleteAllRevisions` already does. Remove the duplicate.
- `API.compactPad(padID, keepRevisions?)` now delegates to Cleanup:
• keepRevisions null/undefined → deleteAllRevisions (full collapse)
• keepRevisions >= 0 → deleteRevisions(N) (keep last N)
Returns {ok, mode: 'all' | 'keepLast', keepRevisions?}.
- APIHandler `1.3.1`: signature updated to take `keepRevisions`
instead of `authorId`.
- `bin/compactPad.ts`: accepts `--keep N` for the keep-last mode,
shows before/after revision counts so operators see concrete
savings.
- Backend tests rewritten around the public API surface (mode
reporting, text preservation, input validation) rather than
internal method plumbing that no longer exists.
Net: strictly a thin public-API and CLI veneer over already-tested
Cleanup helpers. No new low-level logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(6194): assert content markers, not byte-exact atext
Cleanup.deleteAllRevisions internally calls copyPadWithoutHistory
twice (src → tempId, tempId → src with force=true), and each round
trip normalizes trailing whitespace. That meant my byte-exact
atext.text assertion failed in CI:
expected: '...line 3\n\n\n'
actual: '...line 3\n'
Swap the comparisons to use content markers (marker-alpha / beta /
gamma, keep-line-N). The test still catches the real regressions —
if compactPad lost content those markers would disappear — without
coupling to whitespace quirks of the existing Cleanup implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(6194): correct API param + document compactPad in http_api docs
The 1.3.1 entry in APIHandler registered `['padID', 'authorId']`, but
`API.compactPad` takes `(padID, keepRevisions)` and the CLI sends a
`keepRevisions` query param. APIHandler.handle dispatches by URL field
name, so the previous wiring silently dropped `keepRevisions` and never
ran the keep-last branch over HTTP.
- Register `['padID', 'keepRevisions']` so the handler forwards the
CLI/HTTP arg into the API function.
- Add HTTP-level dispatch tests that hit `/api/1.3.1/compactPad` with
and without `keepRevisions`. The direct `api.compactPad()` tests
bypass the handler and would have missed this regression.
- Document compactPad in `doc/api/http_api.md` and `http_api.adoc`,
and bump the documented latest version from 1.3.0 to 1.3.1 to match
`latestApiVersion`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(6194): add bin/compactAllPads for per-instance bulk compaction
`bin/compactPad <padID>` covers the case where you know which pad is
fat. For "reclaim space across the whole instance," composing
`listAllPads` + `compactPad` yourself is annoying; this script does it.
- Walks every pad on the instance and compacts it (full collapse, or
`--keep N` keep-last).
- Per-pad failures don't abort the run — they're logged, counted, and
the script exits 1 if any failed.
- `--dry-run` lists pads + revision counts without writing anything,
so operators can scope impact before committing.
- Reports `before → after` per pad and a total reclaimed count.
Deliberately not adding a `compactAllPads` HTTP API: bulk compaction
over a single HTTP request means one giant response and a long-held
connection. Operators who want this should run it locally, where they
can see progress and kill it cleanly. Staleness gating ("only pads
older than X days") is tracked separately as a follow-up.
Also registers `compactPad` and `compactAllPads` script aliases in
`bin/package.json` so they show up next to the other admin CLIs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(6194): cover the bin/compactAllPads loop logic
Previous commit added the script but only exercised it by hand. The
loop itself — error tolerance, dry-run gating, keep-last passthrough,
the empty-instance and listAllPads-failure paths — had no automated
coverage.
- Refactor compactAllPads.ts to export `runCompactAll(api, opts, logger)`
and `parseArgs(argv)`. The CLI shell wires them up to axios+APIKEY
for production; tests use an in-memory `CompactAllApi` so we don't
need to stand up the apikey-auth path in mocha.
- Add 9 specs covering: arg parsing, full-collapse iteration,
--keep N passthrough, --dry-run skipping writes, single-pad failure
not aborting the run, pre-flight count failure tolerated, a
listAllPads failure short-circuiting cleanly, the empty-instance
no-op, and a final end-to-end test that runs `runCompactAll`
against the real `/api/1.3.1/compactPad` handler over supertest+JWT
to catch contract drift between the CompactAllApi shape and the
HTTP endpoints.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(6194): address Qodo review — gate, integer check, SSL
Three valid concerns from the Qodo review on 75a08a13:
1. **cleanup.enabled gate.** The admin/Cleanup-socket path checks
`settings.cleanup.enabled` before doing anything destructive; the
public API was bypassing that gate. Now `compactPad` mirrors the
admin path's check and returns a clear apierror when disabled, so
exposing the API doesn't accidentally widen the cleanup-opt-in
surface.
2. **Number.isFinite → Number.isInteger.** `2.5` was finite and
non-negative, so the old check let it through into
`Cleanup.deleteRevisions`, which does revision-index arithmetic
that assumes integer math. Reject at the API boundary instead of
silently misbehaving.
3. **SSL-aware baseURL in the bin scripts.** Other bin scripts
hardcode `http://`, but the rest of the codebase uses
`settings.ssl ? 'https' : 'http'`. The compact CLIs now do the
same, so they work against HTTPS deployments. (Other bin scripts
carry the same bug but fixing them is out of scope for this PR.)
Tests:
- New spec: `rejects fractional keepRevisions` (2.5 with the old
check passed; the new one rejects).
- New spec: `refuses to run when cleanup.enabled is false`. The
existing API tests opt in via a before-hook + restore, so they
still cover the success path under the new gate.
- API docs (`http_api.md` + `http_api.adoc`) document the gate and
the new error message.
Skipped Qodo concerns:
- "Wrong compactPad parameters" — already fixed in 26e12ff7
(the param map now correctly says `keepRevisions`, not `authorId`).
- "Unbounded revision deletions" / "No session eviction" / changeset
base-length / padCreate hook — these all targeted the earlier
on-Pad implementation that was refactored away. The current code
wraps `Cleanup.deleteAllRevisions` / `deleteRevisions`, which
already handle concurrency, locking, and hook semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: PR1 GDPR deletion-controls design spec
First of five GDPR PRs tracked in #6701. PR1 covers deletion controls:
one-time deletion token, allowPadDeletionByAllUsers flag, authorisation
matrix for handlePadDelete and the REST deletePad endpoint, a single
token-display modal for browser pad creators, and test coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: PR1 GDPR deletion-controls implementation plan
13 TDD-structured tasks covering PadDeletionManager unit tests, socket
+ REST three-way auth, clientVars wiring, one-time token modal,
delete-with-token UI, Playwright coverage, and PR handoff.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(gdpr): scaffolding for pad deletion tokens
PadDeletionManager stores a sha256-hashed per-pad deletion token and
verifies it with timing-safe comparison. createPad / createGroupPad
return the plaintext token once on first creation, and Pad.remove()
cleans it up. Gated behind the new allowPadDeletionByAllUsers flag
which defaults to false to preserve existing behaviour.
Part of #6701 (GDPR PR1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix+test(gdpr): lazy DB access in PadDeletionManager + unit tests
Capturing DB.db at module-load time was null until DB.init() ran, which
broke importing the module outside a live server (including from the
test runner). Switch to DB.db.* at call time and add unit tests
exercising create/verify/remove plus timing-safe comparison.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(gdpr): three-way auth for socket PAD_DELETE
Creator cookie → valid deletion token → allowPadDeletionByAllUsers flag.
Anyone else still gets the existing refusal shout.
* feat(gdpr): optional deletionToken on programmatic deletePad
* feat(gdpr): advertise optional deletionToken on REST deletePad
* test(gdpr): cover deletePad authorisation matrix via REST
* feat(gdpr): surface padDeletionToken in clientVars for creators only
Revision-0 author on their first CLIENT_READY visit receives the
plaintext token; all subsequent CLIENT_READYs receive null because
createDeletionTokenIfAbsent is idempotent. Readonly sessions and any
other user never see the token.
* i18n(gdpr): strings for deletion-token modal and delete-with-token flow
* feat(gdpr): token modal + delete-with-token disclosure markup
* feat(gdpr): show deletion token once, allow delete via recovery token
* style(gdpr): modal + delete-with-token layout
* test(gdpr): Playwright coverage for deletion-token modal + delete-with-token
* fix(test): auto-dismiss deletion-token modal in goToNewPad helper
The token modal introduced in PR1 blocks clicks for every Playwright
test that creates a new pad via the shared helper. Add a one-line
dismissal so unrelated tests keep passing, and have the deletion-token
spec navigate inline via newPadKeepingModal() when it needs the modal
open to capture the token.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): dismiss deletion-token modal without focus transfer
Clicking the ack button transferred focus out of the pad iframe, which
made subsequent keyboard-driven tests (Tab / Enter) silently miss the
editor. Swap the click for a page.evaluate() that hides the modal and
nulls clientVars.padDeletionToken directly, leaving focus where it was.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(gdpr): PadDeletionManager race + document createPad/deletePad
Qodo review:
- createDeletionTokenIfAbsent() was a non-atomic read-then-write. Two
concurrent callers for the same pad could both return different
plaintext tokens while only the later hash was stored, leaving the
first caller with an unusable recovery token. Serialise per-pad via a
Promise chain and add a regression test that fires 8 concurrent
calls and asserts exactly one plaintext is emitted and validates.
- doc/api/http_api.md now documents createPad returning deletionToken
and deletePad accepting the optional deletionToken parameter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(gdpr): always render delete-with-token in settings popup
The rebase onto develop placed the delete-pad-with-token details inside
the pad-settings-section conditional, which is only rendered when
enablePadWideSettings is true AND the section is toggled visible.
Second-device recovery (typing the captured token on a fresh browser)
must work without pad-wide settings enabled, so move the details out
to sit alongside the existing pad_deletion_token.spec.ts expectations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(gdpr): require valid token when supplied, gate on auth, harden a11y/i18n
- PadMessageHandler: a supplied deletion token must validate; do not fall
back to the creator-cookie path when the token is wrong (was deleting
the pad anyway when the creator pasted a wrong token into the field).
- Skip token issuance + UI when requireAuthentication is on (creator
identity is stable, recovery token is redundant noise).
- Server emits messageKey instead of hardcoded English; both shout
handlers (inline alert and global gritter) localize via html10n.
- Suppress the global "Admin message" gritter for pad.deletionToken.*
shouts to avoid the "Admin message: undefined" duplicate.
- Token-modal a11y: role=dialog, aria-modal, aria-labelledby/describedby,
visually-hidden label on the token input, aria-live on Copy, focus to
the token input on open and restore on dismiss.
- Style the "Delete Pad with Token" disclosure to match the Delete pad
button; align the Copy/value row; pad the disclosure label.
Tests: Playwright now covers the creator-with-wrong-token path, asserts
no "Admin message" / "undefined" gritter on denial; backend API test
covers requireAuthentication suppressing the token.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: PR2 GDPR IP/privacy audit design spec
Second of five GDPR PRs (#6701). Audit identifies four log-sites that
leak IPs despite disableIPlogging=true, proposes a tri-state ipLogging
setting with a back-compat shim, and specifies a doc/privacy.md that
documents Etherpad's actual IP handling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: PR2 GDPR IP/privacy audit implementation plan
7 TDD-structured tasks: anonymizeIp helper + unit tests, tri-state
ipLogging setting with disableIPlogging deprecation shim, wiring
through 5 leaking log sites, clientVars.clientIp removal, access-log
integration test, doc/privacy.md, and PR handoff.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(gdpr): anonymizeIp helper with v4/v6/v4-mapped truncation
* feat(gdpr): tri-state ipLogging setting + disableIPlogging shim
* fix(gdpr): route every IP log site through anonymizeIp
Closes four leaks where disableIPlogging was silently ignored
(rate-limit warn, both auth-log calls in webaccess, import/export
rate-limit warn) and normalises the four that did honour the flag
onto the new ipLogging tri-state via the shared helper.
* chore(gdpr): drop dead clientVars.clientIp placeholder
Server side: remove the literal '127.0.0.1' assignments from both
clientVars and collab_client_vars. Type side: drop clientIp from
ClientVarPayload and ServerVar. pad.getClientIp now returns the same
'127.0.0.1' literal as a plugin-compat shim (pad_utils.uniqueId still
uses it as a prefix).
* test(gdpr): ipLogging modes + disableIPlogging shim
* docs(gdpr): operator-facing privacy and IP handling statement
* fix(gdpr): validate ipLogging at load + regression test for log sites
Qodo review:
- settings.ipLogging is loaded as a trusted union but nothing enforced
the shape. An unknown value (e.g. a typo or null) silently fell
through to anonymizeIp's "truncated" branch and emitted partially
redacted IPs. Fall back to "anonymous" with a WARN at load time.
- New regression test scans the four known log-sites for raw
req.ip / socket.request.ip / request.ip inside logger calls that
don't wrap through anonymizeIp / logIp, so a future edit that
re-introduces a raw IP fails CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes#5071. `/p/:pad/:rev/export/etherpad` has always ignored the rev
parameter and returned the full pad history, unlike the txt/html
export endpoints which use the same route but do respect rev. Users
wanting to back up or inspect a snapshot of a pad at a specific rev
got every later revision in the payload instead — both wasteful and
a surprise when the downloaded .etherpad blob contained content that
had supposedly been reverted.
Change:
- `exportEtherpad.getPadRaw(padId, readOnlyId, revNum?)` now takes an
optional revNum. When supplied, it clamps to `min(revNum, pad.head)`,
iterates only revs 0..effectiveHead, and ships a shallow-cloned pad
object whose `head` and `atext` reflect the requested snapshot. The
original live Pad is still passed to the `exportEtherpad` hook so
plugin callbacks see the real document.
- `ExportHandler` passes `req.params.rev` through on the `etherpad`
type, matching the existing behavior of `txt` and `html`.
- Chat history is intentionally left full (it is not rev-anchored).
Adds three backend regression tests under `ExportEtherpad.ts`:
- default (no revNum) still exports the full history
- explicit revNum limits exported revs and rewrites the serialized
head so re-import reconstructs the pad at that rev
- revNum above head is treated as full history, preventing accidental
truncation of short pads
Out of scope: `getHTML(padID, rev)` on the API side is already honoring
rev in current code (exportHtml.getPadHTML threads the parameter
through), so the earlier report on that API call appears to be
resolved. This PR does not touch it.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PadMessageHandler built the `pluginsSanitized` payload for clientVars by
aliasing `plugins.plugins` and then mutating each entry's `package` field
in place:
let pluginsSanitized: any = plugins.plugins;
Object.keys(plugins.plugins).forEach(function(element) {
const p: any = plugins.plugins[element].package;
pluginsSanitized[element].package = {name: p.name, version: p.version};
});
Because `pluginsSanitized` is a reference to `plugins.plugins`, the
assignment clobbered the server-side plugin registry. After the first
pad connection, every plugin's `package` object held only `{name,
version}` — `realPath`, `path`, and `location` were gone.
Minify.ts resolves `/static/plugins/ep_*/...` URLs via
`plugin.package.realPath`. Once the field disappeared, every subsequent
static asset request for a bundled plugin 500'd with:
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of
type string. Received undefined
at Object.join (node:path:1354:7)
at _minify (src/node/utils/Minify.ts:181:23)
Symptoms on Chromium: plugin CSS/JS assets fail to load (e.g.
/static/plugins/ep_font_size/static/css/size.css returns 500), so
plugins partially render or don't work at all. Firefox swallows the
resulting console errors quietly.
Fix: extract the sanitization into a pure helper `sanitizePluginsForWire`
that returns a fresh object graph and never touches the input. The
helper is covered by a new backend spec that:
* verifies the sanitized output has only {name, version} in `package`
* asserts the input registry's realPath/path/location survive the call
* runs the call repeatedly and confirms non-destructiveness
* mutates the returned copy and asserts the input is independent
Verified live with the dev server: before the fix, `/static/plugins/
ep_font_size/static/css/size.css` 500'd after visiting any pad; after
the fix it returns 200 both before and after pad connections.
* fix: allow undo of clear authorship colors without disconnect (#2802)
When a user clears authorship colors and then undoes, the undo changeset
re-applies author attributes for all authors who contributed text. The
server was rejecting this because it treated any changeset containing
another author's ID as impersonation, disconnecting the user.
The fix distinguishes between:
- '+' ops (new text): still reject if attributed to another author
- '=' ops (attribute changes on existing text): allow restoring other
authors' attributes, which is needed for undo of clear authorship
Also removes the client-side workaround in undomodule.ts that prevented
clear authorship from being undone at all, and adds backend + frontend
tests covering the multi-author undo scenario.
Fixes: https://github.com/ether/etherpad-lite/issues/2802
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use robust Playwright assertions in authorship undo tests
- Use toHaveAttribute with regex instead of raw getAttribute + toContain
- Check div/span attributes within pad body instead of broad selectors
- Use Playwright auto-retry (expect with timeout) instead of toHaveCount(0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: handle confirm dialog and sync timing in Playwright tests
- Add page.on('dialog') handler to accept the confirm dialog triggered
by clearAuthorship when no text is selected (clears whole pad)
- Use auto-retrying toHaveAttribute assertions instead of raw getAttribute
- Increase cross-user sync timeouts to 15s for CI reliability
- Add retries: 2 to multi-user test for CI flakiness
- Scope assertions to pad body spans instead of broad selectors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use persistent socket listeners to avoid missing messages in CI
Replace sequential waitForSocketEvent loops with single persistent
listeners that filter messages inline. This prevents race conditions
where messages arrive between off/on listener cycles, causing timeouts
on slower CI runners.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reject - ops with foreign author to prevent pool injection
The '-' op attribs are discarded from the document but still get added
to the pad's attribute pool by moveOpsToNewPool. Without this check, an
attacker could inject a fabricated author ID into the pool via a '-' op,
then use a '=' op to attribute text to that fabricated author (bypassing
the pool existence check).
Now all non-'=' ops (+, -) with foreign author IDs are rejected.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: use not.toHaveClass for cleared authorship spans
Addresses Qodo review: linestylefilter skips attribs with empty values,
so a span with author='' has no class attribute at all. The previous
negative-lookahead regex on the class attribute failed against a null
attribute and was flaky in CI. Switch to not.toHaveClass(/author-/),
which also passes when the attribute is missing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat!: replace Abiword with LibreOffice and add DOCX export (#4805)
The Abiword converter is dropped. Abiword's DOCX export is weak and the
project is niche on modern platforms; LibreOffice (soffice) is the
common deployment path and now serves as the sole converter backend.
DOCX is added as an export format and becomes the new target for the
"Microsoft Word" UI button. The /export/doc URL still works for legacy
API consumers.
BREAKING CHANGE: The 'abiword' setting, the INSTALL_ABIWORD Dockerfile
build arg, the abiwordAvailable clientVar, and the
#importmessageabiword UI element (with locale key
pad.importExport.abiword.innerHTML) are removed. Deployments relying on
Abiword must configure 'soffice' instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add docxExport feature flag and abiword deprecation WARN
- Add `docxExport: true` setting to opt out of DOCX (use legacy DOC)
- Pass `docxExport` to client via clientVars
- Use `docxExport` flag in pad_impexp.ts for Word button format
- Emit a specific WARN when deprecated `abiword` config is detected
- Update settings.json.template and settings.json.docker with docxExport
- Add docxExport to ClientVarPayload type in SocketIOMessage.ts
Agent-Logs-Url: https://github.com/ether/etherpad/sessions/9afc5291-73b2-4b66-b028-feed39e7056f
Co-authored-by: JohnMcLear <220864+JohnMcLear@users.noreply.github.com>
* refactor: extract wordFormat variable and improve docxExport comment
Agent-Logs-Url: https://github.com/ether/etherpad/sessions/9afc5291-73b2-4b66-b028-feed39e7056f
Co-authored-by: JohnMcLear <220864+JohnMcLear@users.noreply.github.com>
* fix: restore import-limitation message when no converter is configured
The abiword removal dropped both the #importmessageabiword DOM element
and its locale key, but Copilot's refactor still expected the show()
call to surface a message when exportAvailable === 'no'. Result: users
with no soffice binary got silent failure instead of an explanation.
Add #importmessagenoconverter back with updated, LibreOffice-focused
copy (new locale key pad.importExport.noConverter.innerHTML) and flip
the hidden prop when the client knows no converter is available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* i18n: inline English fallback for noConverter import message
The original abiword message existed in ~70 locale files and was
removed from all of them by this PR. The replacement key was only
added to en.json, so non-English users had an empty div until
translators localize. Follow the project's usual pad.html pattern
(e.g. line 146's "Font type:") and include the English text inside
the div as the fallback content; html10n replaces it when a
translation is available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Revert "i18n: inline English fallback for noConverter import message"
This reverts commit f336f24d. Follow the project convention: add the
new locale key to en.json only and let translations catch up via the
translation system, rather than putting inline fallback in the template.
* i18n: leave non-English locale files untouched
The PR had removed pad.importExport.abiword.innerHTML from ~82 locale
files alongside its removal from en.json. The replacement message uses
a new key (pad.importExport.noConverter.innerHTML) in en.json only, so
churning every localisation file for a key that is no longer referenced
produces useless translation diffs. Restore every non-en locale file to
its pre-PR state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: JohnMcLear <220864+JohnMcLear@users.noreply.github.com>
* docs(openapi): document apikey auth in openapi.json (#7532)
The API accepts the key via ?apikey=, ?api_key=, or the apikey header, but
only ?apikey= was advertised in /api-docs.json. /api/{version}/openapi.json
was worse: it hardcoded an OAuth2 scheme even when Etherpad was started in
apikey auth mode.
Switch both generators on settings.authenticationMethod and publish apiKey
schemes for the query (apikey, api_key) and header (apikey) variants. The
openapi.ts definition is now regenerated per request so runtime settings
are reflected.
The raw authorization: <key> header still works in code but is deliberately
not documented — pinning it in the spec would ossify a quirk.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(openapi): add apiKeyAlias/apiKeyHeader conditionally in RestAPI.ts
In SSO mode, apiKeyAlias and apiKeyHeader were always present in
securitySchemes even though they're only relevant when
authenticationMethod is 'apikey'. Mirror the pattern used for the sso
scheme: add these two schemes dynamically inside the apikey branch, and
mark them optional in the TypeScript type annotation.
Agent-Logs-Url: https://github.com/ether/etherpad/sessions/1d440432-7389-462e-9aac-9a3c027640e8
Co-authored-by: JohnMcLear <220864+JohnMcLear@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: JohnMcLear <220864+JohnMcLear@users.noreply.github.com>
* fix: capture head revision atomically with atext to prevent mismatched apply
When constructing CLIENT_VARS, pad.atext was captured at one point but
pad.getHeadRevisionNumber() was called later. If concurrent edits
advanced the revision between these two reads, the client received
initialAttributedText from rev N but rev=N+3, causing "mismatched apply"
errors when the next changeset arrived (expecting rev N+3 text).
Now captures headRev at the same time as atext and uses the captured
value consistently in CLIENT_VARS and sessionInfo.
Fixes#4040
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: flush missed revisions after socket joins pad room
During handleClientReady(), the server awaits the clientVars hook before
socket.join(). Any revisions appended during that await window are
broadcast to existing room members but the connecting socket misses them.
Call updatePadClients(pad) after joining to flush any such revisions.
Also adds a regression test that injects a slow clientVars hook and
verifies the connecting client receives catch-up changesets for edits
that occurred during the hook await window.
Fixes#4040
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: fix race condition in clientVars hook test
Listen for messages during handshake to avoid missing NEW_CHANGES that
arrive before the explicit waitForSocketEvent listener is attached.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: initialize sessionInfo.time before catch-up updatePadClients
The catch-up updatePadClients() call introduced in this PR could send
NEW_CHANGES with timeDelta=NaN because sessionInfo.time was never set
for new sessions. NaN poisons the client-side broadcast/timeslider
currentTime tracking.
Initialize sessionInfo.time to the timestamp of the snapshot revision
before the catch-up flush, with a fallback to Date.now() if the
revision date is unavailable.
Also strengthens the regression tests:
- Validate that initialAttributedText matches the pad AText at the
EXACT advertised rev (not just the latest pad text), using
pad.getInternalRevisionAText(rev).
- Add a load test that hammers the pad with concurrent edits while
multiple clients connect, asserting CLIENT_VARS consistency under
the exact race condition the fix is targeting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: replace open-ended load loop with bounded mid-handshake edit
The previous load test ran 'while (!stopLoad) await pad.setText(...)'
in the background while the test connected clients. This saturated
ueberDB's write queue and on shutdown the queued writes never drained,
hanging the mocha process for the full 6h GitHub Actions job timeout.
Replace it with a bounded approach: a clientVars hook lands 3 edits
mid-handshake (deterministic, no background loop, no shutdown hang).
Still exercises the exact race the fix targets — an edit advancing
the rev after the atext snapshot but before CLIENT_VARS is sent —
and asserts AText / rev consistency via getInternalRevisionAText.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: address remaining Qodo concerns on PR #7480
Addresses Qodo review items 1, 2, 5 from
https://github.com/ether/etherpad-lite/issues/comments/4194702740 :
- Concern 1 (no loadTesting reproduction test): the suite now toggles
settings.loadTest = true in before(), restores in after(). The
middle test also pre-populates the pad with 20 revisions before
connecting so we genuinely exercise a busy/loaded pad rather than a
fresh one.
- Concern 2 (no CLIENT_VARS / NEW_CHANGES delay test): the slow
clientVars hook in the middle test now has explicit setTimeout
delays before AND after the mid-handshake edits, so the race window
between atext snapshot and CLIENT_VARS send is observably wide
rather than relying on async scheduling alone. The test also
collects post-handshake messages and asserts a NEW_CHANGES catch-up
arrives when the pad advanced past the advertised rev.
- Concern 5 (test doesn't validate rev): both rev-consistency tests
use pad.getInternalRevisionAText(advertisedRev) and assert text and
attribs match, not just `pad.text() === clientVars.text`.
Concerns 3 (connect can miss revisions) and 4 (NaN timeDelta) were
already addressed in earlier commits on this branch via the catch-up
updatePadClients() call and the sessionInfo.time initialization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: make cookie names configurable with prefix setting
Add cookie.prefix setting (default "ep_") that gets prepended to all
cookie names set by Etherpad. This prevents conflicts with other
applications on the same domain that use generic cookie names like
"sessionID" or "token".
Affected cookies: token, sessionID, language, prefs/prefsHttp,
express_sid.
The prefix is passed to the client via clientVars.cookiePrefix in the
bootstrap templates so it's available before the handshake. Server-side
cookie reads fall back to unprefixed names for backward compatibility
during migration.
Fixes#664
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: default cookie prefix to empty string for backward compatibility
Changing the default to "ep_" would invalidate all existing sessions
on upgrade since express-session only looks for the configured cookie
name. Default to "" (no prefix) so upgrades are non-breaking — users
opt-in to prefixed names by setting cookie.prefix in settings.json.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address Qodo review — cookie prefix migration and fallbacks
- l10n.ts: Read prefixed language cookie with fallback to unprefixed
- welcome.ts: Use cookiePrefix for token transfer reads
- timeslider.ts: Use prefix for sessionID in socket messages
- pad_cookie.ts: Fall back to unprefixed prefs cookie for migration
- indexBootstrap.js: Pass cookiePrefix via clientVars to welcome page
- specialpages.ts: Pass settings to indexBootstrap template
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: escape regex metacharacters in cookie prefix, document Vite hardcode
- l10n.ts: Escape special regex characters in cookiePrefix before using
it in RegExp constructor to prevent runtime errors
- padViteBootstrap.js: Add comment noting the hardcoded prefix is
dev-only and must match settings.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* security: validate cookie prefix to prevent header injection
Reject cookie.prefix values containing characters outside
[a-zA-Z0-9_-] to prevent HTTP header injection via crafted cookie
names (e.g., \r\n sequences). Falls back to empty prefix with an
error log.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
numConnectedUsers in CLIENT_VARS was computed from roomSockets.length
before the new socket joined the room, so the joining user always saw
a count one less than the actual number. Added +1 to include the
joining user in the count.
Fixes#6145
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: wait for server confirmation before navigating after pad delete
The delete pad handler navigated to '/' immediately after sending the
PAD_DELETE message. Firefox (and some mobile Chrome) would close the
WebSocket before the message reached the server, causing the delete to
silently fail.
Now the client waits for the server's {disconnect: 'deleted'} response
before navigating. Also awaits pad.remove() on the server side to
ensure the operation completes before the response is sent.
Fixes: https://github.com/ether/etherpad-lite/issues/7306
Fixes: https://github.com/ether/etherpad-lite/issues/7311
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: handle non-creator delete and add timeout fallback
- Listen for 'shout' event to show error when non-creator tries to
delete (server sends shoutMessage instead of deleting)
- Add 5-second timeout fallback in case the server doesn't respond
(socket dropped, server crashed, etc.)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add initial code for revision cleanup
* Some improvements - code cleanup
* Cleanup logging
* Add button in admin backend to cleanup revisions of a specific pad
* Disable cleanup by default and show errors in admin area
* Improve cleanup code
* Load revisions for cleanup in parallel
* Consider saved revisions during pad cleanup
* New option to make pad names case-insensitive
fixes#3844
* fix helper.gotoTimeslider()
* fix helper.aNewPad() return value
* Update src/node/utils/Settings.js
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
* remove timeout
* rename enforceLowerCasePadIds to lowerCasePadIds
* use before and after hooks
* update with socket specific test
* enforce sanitizing padID for websocket connections
- only enforce for newly created pads, to combat case-sensitive pad name hijacking
* Added updated package.json file.
---------
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: SamTV12345 <40429738+samtv12345@users.noreply.github.com>
