mirror of
https://github.com/ether/etherpad-lite.git
synced 2026-07-17 16:47:05 +00:00
feat(export): native DOCX export via html-to-docx (opt-in) (#7568)
* feat(export): native DOCX export via html-to-docx (opt-in) Addresses #7538. The current DOCX export path shells out to LibreOffice, which means every deployment that wants a Word download either installs soffice (~500 MB) or loses that export. This PR adds a pure-JS alternative: render the HTML via the existing exporthtml pipeline, then feed it to the `html-to-docx` library in-process to produce a valid .docx buffer — no soffice required, no subprocess spawn, no temp file dance for the DOCX case. Behavior: - `settings.nativeDocxExport` (default `false`) gates the new path so existing deployments see zero behavior change. - When enabled, `type === 'docx'` requests skip the LibreOffice branch, run `html-to-docx(html)`, and return the buffer with the `application/vnd.openxmlformats-officedocument.wordprocessingml.document` content-type. - If the native converter throws, the handler falls through to the existing LibreOffice path — so flipping the flag on is safe even on a mixed-installation where soffice is still present as a backstop. - Other export formats (pdf, odt, rtf, txt, html, etherpad) are unchanged. Files: - `src/package.json`: `html-to-docx` dep (pure JS, no binary reqs) - `src/node/handler/ExportHandler.ts`: new DOCX branch gated on the setting, with fall-through on error - `src/node/utils/Settings.ts`, `settings.json.template`, `settings.json.docker`, `doc/docker.md`: wire up the new setting + env var (`NATIVE_DOCX_EXPORT`) - `src/tests/backend/specs/export.ts`: two new tests — asserts the exported buffer is a valid ZIP (PK\x03\x04 signature) and the response carries the correct content-type — both with `settings.soffice = 'false'` to prove the path doesn't need soffice at all. Out of scope for this PR: - Native PDF export (would need a PDF rendering step — separate undertaking, and the issue acknowledges the `pdfkit`/puppeteer size trade-off). Closes #7538 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(7538): skip native DOCX test when html-to-docx isn't installed The upgrade-from-latest-release CI job installs deps from the previous release's package.json (before this PR adds html-to-docx) and then git-checkouts this branch's code without re-running pnpm install. Under that one workflow the new test can't find the module and fails on the LibreOffice fallback, masking that the native path actually works in every normal install. Guard the describe block with require.resolve('html-to-docx'); Mocha's this.skip() on before cascades to the sibling its. Regular backend tests (pnpm install against this branch's lockfile) still exercise it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(7538): spec for soffice-free DOCX/PDF export and DOCX import Captures the agreed scope expansion of PR #7568: replace the flag-gated native DOCX path with a soffice-first selection cascade, add native PDF export via pdfkit + a small htmlparser2-driven walker, and add native DOCX import via mammoth. Also defines a shared HTML sanitizer (stripRemoteImages) used by both export converters to close the SSRF surface that Qodo flagged on the html-to-docx path. The spec drops the nativeDocxExport setting and its env var; with soffice configured, behavior is unchanged, and with soffice null, docx/pdf export and docx import all work in-process. odt/doc/rtf (and pdf import) keep needing soffice and are documented as such. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(7538): implementation plan for native DOCX/PDF and DOCX import Bite-sized TDD task breakdown of the soffice-free export/import work: rebase, deps, sanitizer, PDF walker, mammoth wrapper, ExportHandler cascade, route guard, ImportHandler branch, UI fix, flag rollback, verification + Qodo reply. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(7538): add pdfkit, htmlparser2, mammoth deps Pure-JS, no native binaries: - pdfkit ^0.18.0 (PDF rendering) - htmlparser2 ^12 (SAX parser used by walker + sanitizer) - mammoth ^1.12 (DOCX -> HTML for native import) - @types/pdfkit ^0.17 (dev) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(7538): add stripRemoteImages HTML sanitizer Drops <img src=> elements pointing at non-data, non-relative URLs to prevent the DOCX/PDF converters from making outbound requests via plugin-modified HTML. Closes Qodo finding #4 against the html-to-docx path; will be wired into both export branches in the cascade refactor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(7538): native PDF export via pdfkit + htmlparser2 walker Renders pad HTML to a PDF Buffer in-process: headings, paragraphs, lists, links, inline emphasis, data:-URI images. Remote images are explicitly skipped at the walker (defense-in-depth on top of the shared stripRemoteImages sanitizer). PDFs are emitted with compress:false so accessibility/SEO indexers that don't FlateDecode can still extract text. Pads are small enough that the size cost is negligible. Walker is 167 LOC, well under the spec's 500-LOC bail-out threshold for switching to pdfmake+html-to-pdfmake+jsdom. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(7538): native DOCX import via mammoth Wraps mammoth.convertToHtml so a soffice-less Etherpad can ingest .docx files. Images are coerced to data: URIs at the converter boundary so the import pipeline never sees a remote src=. Includes a tiny generated DOCX fixture (heading, paragraph, list) under tests/backend/specs/fixtures/ for both this wrapper test and the upcoming end-to-end import test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(7538): soffice-first cascade in ExportHandler Replaces the flag-gated DOCX branch with a deterministic dispatch: soffice if configured, native DOCX/PDF otherwise, 5xx on native error. Both native paths run plugin-modified HTML through stripRemoteImages first. Test changes: - existing native DOCX block now sets soffice=null (was 'false', a truthy non-null string that sidestepped the route guard); fixes Qodo finding #3. - new native PDF integration tests assert %PDF- header and application/pdf content-type with soffice=null. - new negative test: with soffice=null, /export/odt still returns the 'not enabled' message. - the legacy 500-on-export-error test now uses /bin/false so it exercises the soffice error path explicitly (the cascade dropped the ad-hoc 'false' string; .doc has no native path so this still works as a soffice error probe). Integration tests for native DOCX/PDF currently fail because the /export route guard still treats both formats as soffice-only; the next commit fixes that. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): allow docx/pdf through export guard without soffice Tightens the no-soffice block to ['odt','doc'] only — formats with no native path. docx and pdf are handed to ExportHandler, which dispatches to the in-process converters. Closes Qodo finding #2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(7538): native DOCX import path in ImportHandler When soffice is null and the upload is .docx, run mammoth and feed the resulting HTML through setPadHTML. Other office formats (pdf/odt/doc/rtf) are explicitly rejected with uploadFailed instead of silently falling through to the ASCII-only path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): always show DOCX/PDF export links Native paths (#7538) make DOCX and PDF available regardless of soffice presence, so unconditionally render those links. ODT still gates on exportAvailable. Closes Qodo finding #2 on the UI side. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(7538): drop nativeDocxExport flag Selection is now purely soffice-presence-driven (cascade in ExportHandler). The opt-in setting and its NATIVE_DOCX_EXPORT env var are no longer needed -- soffice configured means soffice path; soffice null means native path (DOCX, PDF, and DOCX import). Reverts the additive surface introduced earlier in this PR. Also updates the SOFFICE doc row to reflect that null no longer means 'plain text and HTML only' -- docx/pdf export and docx import now work natively without soffice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(7538): tighten link annotation assertion CodeQL flagged the loose 'raw.includes("etherpad.org")' as 'incomplete URL substring sanitization' (a false positive in test context, but worth fixing). Match the full /URI (host) form instead -- it's both more accurate (we're verifying the PDF link annotation structure) and CodeQL-clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): cleaner DOCX/PDF output + round-trip test coverage DOCX: - New extractBody helper drops <head>/<style> and the leading newline inside <body> so html-to-docx doesn't render CSS or prefix paragraphs with empty space. - New wrapLooseLines pre-processor wraps loose pad lines in <p> before the converter sees them. html-to-docx renders <br> outside <p> as a new <w:p> (full empty line in Word); inside <p> it correctly emits <w:br/> (soft break). Etherpad's HTML uses bare <br> for every line, so this was making single Enters look like double Enters in the Word output. PDF: - Walker SKIP_TAGS rejects head/style/script/title/meta/link content -- prior version dumped CSS into the rendered PDF. - New breakLine() helper combines flushLine() with moveDown(1). pdfkit's text('', false) closes the continued run but does NOT advance the cursor, so consecutive runs were stacking at the same y-coordinate. <br>, end-of-block, and list items now use breakLine(). - ontext collapses runs of whitespace and drops pure-whitespace text nodes so pretty-printed source HTML doesn't render its formatting newlines. Round-trip: - New backend test: pad text -> DOCX export -> DOCX import -> new pad. Asserts content survives the trip. - New PDF sanity test: extracts visible text from the PDF stream and asserts the source pad text appears verbatim. - 6 new unit tests for extractBody and wrapLooseLines plus 1 for PDF walker SKIP_TAGS coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): CodeQL ReDoS + import.ts type error - BR_PARA_RE was /(?:\s*<br>\s*){2,}/ -- two adjacent \s* runs can match the same chars, so on '<br>\t<br>\t<br>...' the regex backtracks exponentially. Re-anchored to match a fixed first <br> followed by one or more additional <br>s, so each whitespace run has exactly one home. - import.ts: fetchBuffer was typed Promise<Buffer> but call sites chained .expect(200) on it, which only works on supertest's Test object. Return the Test (typed any) so the chain is preserved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): plugin-aware HTML cleanup, PDF text-align, monospace ep_headings2/ep_align emit one heading-styled blank-line block after every styled line in the pad ('<h1 style=text-align:right></h1>'), which both html-to-docx and our pdfkit walker render as a full empty paragraph. Plus the pdfkit walker had no support for text-align or monospace, so right/center alignment and 'code' lines rendered the same as plain body text. - New dropEmptyBlocks helper strips empty h1-h6/p/code/pre/div/ blockquote wrappers in preprocessing. Iterates so nested empties collapse too. Applied before both DOCX and PDF conversion. - PDF walker now reads style='text-align:left|center|right|justify' on block elements (h1-6, p, div) and passes it as pdfkit's align option. align is applied once per continued run, then reset on flushLine so the next block can pick up its own value. - PDF walker handles <code>, <tt>, <kbd>, <samp> as inline monospace (Courier) and <pre> as block monospace (Courier + breakLine on open/close). 11 new unit tests: - 4 for dropEmptyBlocks (heading wrappers, code, nesting, pass-through) - 1 for PDF text-align (compares the BT matrix x for left vs right) - 2 for Courier in <code> and <pre> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): separate adjacent heading-style blocks on import Round-trip bug: ep_headings2 emits <h1>/<h2>/<code> in pad HTML. Mammoth round-trips them as adjacent <h1>A</h1><h2>B</h2><p>C</p> on import. Etherpad's server-side content collector has a default _blockElems set of just {div, p, pre, li}, and ep_headings2 only registers the CLIENT-side aceRegisterBlockElements -- not the server-side ccRegisterBlockElements. So h1/h2/code end up being treated as inline by the importer, and adjacent blocks merge into a single pad line. Fix: insert <br> after </h1>...</h6>/</code> when followed by another block. Server-side workaround keeps this PR self-contained regardless of plugin version. The right long-term fix is to extend ep_plugin_helpers' lineAttribute factory to register both hooks (filed as a follow-up). Tests: - 5 unit tests for separateAdjacentHeadingBlocks - New end-to-end round-trip test asserts H1+H2+P land on three separate pad lines after the import path. Plus the prior PDF text-align/Courier/code commit also included here: - code/tt/kbd/samp inherit text-align from style attribute - pre inherits text-align too Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): blank-line round-trip + DOCX code monospace + a==c tests DOCX round-trip dropping blank pad lines: - wrapLooseLines now emits an explicit <p></p> marker for each blank line in a <br> run, instead of collapsing all gaps into a single paragraph break. (N consecutive <br>s -> 1 paragraph boundary + N-2 empty <p></p> markers, mapping to N-1 blank pad lines.) - mammoth's docxBufferToHtml now passes ignoreEmptyParagraphs:false so the empty <w:p> entries survive the import side. mammoth's default of true was silently dropping them. - dropEmptyBlocks no longer strips <p></p> -- that's the meaningful marker for the round-trip. Empty <h1>/<code>/<pre>/<div>/ <blockquote> are still stripped (plugin noise). DOCX <code> rendering as monospace: - New applyMonospaceToCode wraps code/tt/kbd/samp/pre content in a <span style="font-family:'Courier New', monospace">. html-to-docx honors that and emits <w:rFonts w:ascii="Courier New".../>, which Word renders as Courier. The bare <code> tag is otherwise just a no-op for html-to-docx. - Applied only on the DOCX export path (PDF walker already handles monospace via Courier font selection). Round-trip tests: - New a==c suite: txt, etherpad, html, docx -- export from src, import to dst, re-export and compare against the meaningful invariant (line text for binary formats; trimmed body for HTML). - HTML test tolerates one trailing <br> per round-trip because setPadHTML appends a final <p> on import; this is pre-existing core behavior, not our bug. - DOCX test normalizes trailing newline run (same reason). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): preserve <w:jc> alignment through mammoth round-trip mammoth doesn't expose Word's paragraph alignment (`<w:jc>`) when it converts a docx to HTML -- there's no equivalent in its style-mapping machinery. To keep alignment through DOCX round-trips we walk the docx's document.xml directly, pull the `w:val` from each `<w:p>`'s `<w:jc>`, and inject `style="text-align:..."` onto the matching block element in mammoth's output by document order. Word's w:jc accepts more values than CSS text-align; we map left/ start, center, right/end, both/justify/distribute and skip the rest (start/end take left/right because we don't track ltr/rtl from the docx for now). Combines with the upstream ep_align PR (ether/ep_align#183) for the full round-trip: this PR makes the mammoth output carry the alignment style; ep_align#183 makes the importer pick it up. Closes the alignment side of #7538. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): preserve <a href> inside <code>/<pre> in DOCX export html-to-docx silently drops <a href> children of <code>/<pre> tags (and of styled <span>s, but the <code> wrapper is the active offender here). The pad-export HTML produced by ep_headings2 + ep_align uses <code style='text-align:right'>...<a href>...</a>...</code> for each 'Code'-style line, which lost its links on every DOCX export. Workaround: applyMonospaceToCode now drops the code/pre/tt/kbd/samp wrapper entirely. The non-anchor content gets wrapped in monospace spans; anchors are emitted unstyled so they keep their hyperlink. For block-level usage (<pre>, or <code> with an inline style attr) we emit a wrapping <p> and forward the text-align style. Run BEFORE wrapLooseLines so the <p> doesn't get double-wrapped. Tests added: - inline <code> -> just a styled span (no <code> wrapper) - <code style='text-align:right'> -> <p style> wrap - <pre> -> always block-wrapped - <tt>/<kbd>/<samp> -> inline span only - regression: <a href> inside <code> survives html-to-docx round-trip with both the URL in word/_rels/document.xml.rels AND a <w:hyperlink> in the document body Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(7538): HTML import line-break drift between blocks Etherpad's HTML export wraps each pad line in <p>...</p> (or <h1>, <code>, etc.) and then appends a <br> between lines. The closing block tag already ends the line for contentcollector, so the trailing <br> is redundant -- and on import the server collector counts BOTH as line breaks, doubling every blank line between paragraphs and inserting an extra blank between adjacent headings. Two fixes, both gated on the runtime block-element registry so they don't double-trigger when the underlying plugin already handles adjacency: 1. HTML import path now runs the new collapseRedundantBrAfterBlocks helper before setPadHTML. Drops a single <br> immediately following </p>/</h1-6>/</code>/</pre>/</div>/</blockquote>/</ul>/ </ol>/</li>/</table>/</tr>/</td>/</th>. Multiple consecutive <br>s after a block keep all but the first (the rest still represent intentional blank lines). 2. The DOCX-import separateAdjacentHeadingBlocks workaround now checks whether 'h1' is in the runtime ccRegisterBlockElements set before inserting <br>s. When ep_headings2 has the new server hook (per ep_plugin_helpers#14 + the upcoming ep_headings2 PR), the workaround correctly stays out of the way -- otherwise it adds an extra blank line per heading transition. Also fixed a subtle ts-check failure on the import.ts test changes and a leftover implicit-any in ImportDocxNative's alignment preserver. Tests added: - collapseRedundantBrAfterBlocks: 5 unit tests (each block tag, whitespace tolerance, multiple <br> keeping intentional blanks) - HTML import: 'does not introduce a blank line between H1 and H2', 'preserves blank-line count between H1 and H2 (realistic shape)' reproduces the 5-blanks-where-2-expected bug from the user's round-trip pad. 1054 backend tests pass locally (the 6 failures are the pre-existing favicon/webaccess send@1.x dotfile-path issue from running under .claude/, doesn't reach CI). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(7538): skip plugin-dependent HTML import tests on no-plugin CI The two new HTML-import-adjacency tests assume ep_headings2 (or another plugin) has registered h1/h2 as server-side block elements via ccRegisterBlockElements. Without that, contentcollector treats <h1>/<h2> as inline and adjacent ones merge into a single pad line -- making the assertions inapplicable. CI's backend-tests job runs without plugins installed, so guard the describe block with a runtime hooks.callAll() check and skip when h1 isn't a registered block. Local dev with ep_headings2 (and the local plugin patch wiring ccRegisterBlockElements) still exercises both tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
John McLear
GitHub
parent
afee7969dd
commit
c47ffd5705
15 changed files with 4191 additions and 35 deletions
|
|
@ -197,7 +197,7 @@ For the editor container, you can also make it full width by adding `full-width-
|
|||
| `EDIT_ONLY` | Users may edit pads but not create new ones. Pad creation is only via the API. This applies both to group pads and regular pads. | `false` |
|
||||
| `MINIFY` | If true, all css & js will be minified before sending to the client. This will improve the loading performance massively, but makes it difficult to debug the javascript/css | `true` |
|
||||
| `MAX_AGE` | How long may clients use served javascript code (in seconds)? Not setting this may cause problems during deployment. Set to 0 to disable caching. | `21600` (6 hours) |
|
||||
| `SOFFICE` | Absolute path to the soffice (LibreOffice) executable. Needed for advanced import/export of pads (docx, pdf, odt). Setting it to null disables LibreOffice and will only allow plain text and HTML import/exports. | `null` |
|
||||
| `SOFFICE` | Absolute path to the soffice (LibreOffice) executable. When configured, all advanced import/export formats use it (docx, pdf, odt, doc, rtf). Setting it to null falls back to in-process pure-JS converters: docx and pdf export, plus docx import, still work; odt/doc/rtf and pdf import remain unavailable. | `null` |
|
||||
| `ALLOW_UNKNOWN_FILE_ENDS` | Allow import of file types other than the supported ones: txt, doc, docx, rtf, odt, html & htm | `true` |
|
||||
| `REQUIRE_AUTHENTICATION` | This setting is used if you require authentication of all users. Note: "/admin" always requires authentication. | `false` |
|
||||
| `REQUIRE_AUTHORIZATION` | Require authorization by a module, or a user with is_admin set, see below. | `false` |
|
||||
|
|
|
|||
1543
docs/superpowers/plans/2026-05-08-native-docx-pdf-export-import.md
Normal file
1543
docs/superpowers/plans/2026-05-08-native-docx-pdf-export-import.md
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,230 @@
|
|||
# Native DOCX + PDF export and DOCX import without LibreOffice
|
||||
|
||||
**Status:** spec — pending implementation
|
||||
**Issue:** #7538
|
||||
**Extending PR:** #7568 (`feat/native-docx-export-7538`)
|
||||
**Date:** 2026-05-08
|
||||
|
||||
## Problem
|
||||
|
||||
Etherpad's import/export pipeline shells out to LibreOffice (`soffice`) for every "office" format — `pdf`, `docx`, `odt`, `doc`, `rtf`. Operators who want any of those formats must install ~500 MB of LibreOffice as a runtime dependency, plus pay subprocess latency on every export. Operators who don't want LibreOffice lose those formats entirely.
|
||||
|
||||
PR #7568 took a first cut at native DOCX export via `html-to-docx`, but:
|
||||
|
||||
- it's flag-gated (`settings.nativeDocxExport`) and falls back to soffice on error, so soffice remains a soft requirement;
|
||||
- the `/export` route guard and pad UI both gate `docx` on `soffice` being configured, so the new path is unreachable in a real no-soffice deployment (Qodo finding #2);
|
||||
- existing tests use `settings.soffice = 'false'` (a non-null string), which sidesteps the route guard and doesn't simulate a real no-soffice deployment (Qodo finding #3);
|
||||
- the `html-to-docx` dependency tree includes `node-fetch` via `image-to-base64`, so plugin-modified HTML can trigger outbound requests from the converter (Qodo finding #4);
|
||||
- nothing addresses PDF, which the issue explicitly scopes alongside DOCX.
|
||||
|
||||
This spec replaces the flag-gated half-measure with a soffice-first selection model, adds a native PDF export path, adds native DOCX import, and hardens both export converters against SSRF.
|
||||
|
||||
## Goal
|
||||
|
||||
A deployment with `settings.soffice = null` can:
|
||||
|
||||
- export pads as `html`, `txt`, `etherpad`, `docx`, `pdf` — all in-process, no subprocess, no native binaries.
|
||||
- import `.html`, `.txt`, `.etherpad`, `.docx` files — all in-process.
|
||||
|
||||
A deployment with `settings.soffice` configured retains today's behavior bit-for-bit. There is no flag to flip; the path is chosen automatically based on `sofficeAvailable()`.
|
||||
|
||||
`odt`, `doc`, `rtf` (and `pdf` import) continue to require soffice. The deployment matrix is documented; users get a clear error message instead of a silent failure.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Native ODT export. No mature pure-JS writer; deferred to a follow-up issue.
|
||||
- Native PDF/ODT/DOC/RTF import. No mature pure-JS readers for these in Node. Deferred.
|
||||
- Pixel-perfect PDF fidelity. We target structural fidelity (paragraphs, headings, lists, tables, images, basic styling) — the same bar `html-to-docx` hits for DOCX.
|
||||
- Memory/timeout caps on conversion. Pad size is already gated upstream; we'll add caps if production signal warrants it.
|
||||
|
||||
## Selection model
|
||||
|
||||
A single cascade in `ExportHandler.ts` (and a mirror in `ImportHandler`):
|
||||
|
||||
```text
|
||||
if (sofficeAvailable() === 'yes') {
|
||||
→ existing soffice path (handles all formats)
|
||||
} else if (sofficeAvailable() === 'withoutPDF') {
|
||||
// Windows: soffice present but can't render PDF
|
||||
if (type === 'pdf') → native PDF
|
||||
else → soffice
|
||||
} else { // 'no' — soffice null
|
||||
if (type === 'docx') → native DOCX
|
||||
else if (type === 'pdf') → native PDF
|
||||
else → 4xx "this format requires soffice"
|
||||
}
|
||||
```
|
||||
|
||||
No fallback chain on native error. If the native converter throws, the request returns a 500 with a clear log line. This is deliberate — fallback-to-soffice is the pattern that PR #7568 originally used and that Qodo flagged as defeating the no-soffice goal.
|
||||
|
||||
The `nativeDocxExport` setting introduced by PR #7568 is removed entirely. With it go `NATIVE_DOCX_EXPORT`, the `doc/docker.md` row, and the new entries in `settings.json.template` / `settings.json.docker`. Native is built-in; the only thing that varies behavior is whether `soffice` is configured.
|
||||
|
||||
## Route guard and UI capability
|
||||
|
||||
`src/node/hooks/express/importexport.ts` currently rejects all of `['odt','pdf','doc','docx']` when `exportAvailable() === 'no'`. Tighten that list:
|
||||
|
||||
```text
|
||||
if (exportAvailable() === 'no' && ['odt','doc'].includes(req.params.type)) {
|
||||
→ existing "this export is not enabled" message
|
||||
}
|
||||
// pdf and docx fall through to ExportHandler, which dispatches per the cascade above
|
||||
```
|
||||
|
||||
Same shape on the import endpoint: `pdf`, `odt`, `doc`, `rtf` blocked when soffice is null; `docx` (plus the pre-existing `etherpad`/`html`/`txt`) goes through.
|
||||
|
||||
UI side — `src/static/js/pad_impexp.ts:147-166` currently hides DOCX/PDF/ODT export links when `clientVars.exportAvailable === 'no'`. Update so:
|
||||
|
||||
- ODT link: visible iff `exportAvailable === 'yes'` (effectively unchanged)
|
||||
- DOCX, PDF links: always visible
|
||||
|
||||
No new clientVars flags. The "always visible" rule reflects reality — those paths are built into core.
|
||||
|
||||
## Native PDF export
|
||||
|
||||
Module: `src/node/utils/ExportPdfNative.ts`. Single export `htmlToPdfBuffer(html: string): Promise<Buffer>`.
|
||||
|
||||
Approach: **`pdfkit` + `htmlparser2` + a small walker we own.** Pure JS, no jsdom, ~3 MB install footprint. We control the renderer end-to-end, so there is no SSRF surface from the converter.
|
||||
|
||||
### Pipeline
|
||||
|
||||
1. `htmlparser2` parses the input HTML into a SAX-style event stream.
|
||||
2. A walker maintains a `pdfkit` document and a stack of inline-style state. Tag handling:
|
||||
- `<p>`, `<h1..h6>` — block break + font sizing
|
||||
- `<strong>`/`<b>`, `<em>`/`<i>`, `<u>`, `<s>` — toggle inline style
|
||||
- `<ul>`/`<ol>`/`<li>` — indent + bullet/number prefix
|
||||
- `<a href="…">` — underlined text + `doc.link()` annotation
|
||||
- `<br>` — `doc.moveDown(0.5)`
|
||||
- `<table>`/`<tr>`/`<td>` — best-effort grid via computed x/y on `doc.text()`. Pad HTML emits real tables only via plugins; we render what we can.
|
||||
- `<img src="data:…">` — embed via `doc.image(buffer)` after decoding the data URI
|
||||
- `<img src="…">` (any non-data URL) — replaced with the `alt=` text or skipped. **No fetch.** This is an explicit SSRF guard at the converter; the upstream sanitizer (next section) handles it too, this is defense-in-depth.
|
||||
- Unknown tags — recurse into children, ignore the wrapper
|
||||
3. Buffer the PDF in memory via a `PassThrough` stream → resolve with the concatenated `Buffer`.
|
||||
|
||||
### Bail-out criterion
|
||||
|
||||
The walker is a pragmatic bet — pad HTML is constrained enough that a small walker should cover it. **If, during implementation, the walker exceeds ~500 lines of code or hits a class of pad/plugin HTML it cannot reasonably render**, switch to **Approach A**: `pdfmake` + `html-to-pdfmake` + `jsdom`. That swap keeps the same `ExportHandler.ts` integration shape — only `ExportPdfNative.ts` changes — and adds ~15–20 MB of pure-JS deps.
|
||||
|
||||
The plan that follows this spec must call out this decision point so the implementer doesn't grind on a dying walker.
|
||||
|
||||
## HTML sanitization (defense-in-depth)
|
||||
|
||||
New module: `src/node/utils/ExportSanitizeHtml.ts`. Single export `stripRemoteImages(html: string): string`.
|
||||
|
||||
Walks the HTML once with `htmlparser2`, drops any `<img>` whose `src` is not `data:` or a same-origin/relative URL. Replaces with the original `alt=` text (empty string if absent). Pure string-in/string-out, ~50 lines + a unit test.
|
||||
|
||||
Both export branches call this *before* handing HTML to their respective converters:
|
||||
|
||||
```text
|
||||
const safeHtml = stripRemoteImages(html);
|
||||
buffer = (type === 'docx') ? await htmlToDocx(safeHtml) : await htmlToPdfBuffer(safeHtml);
|
||||
```
|
||||
|
||||
This addresses Qodo finding #4 against the existing DOCX path (which was always present, not introduced here) and prevents the equivalent SSRF on the PDF path.
|
||||
|
||||
## Native DOCX import
|
||||
|
||||
Module: `src/node/utils/ImportDocxNative.ts`. Single export `docxBufferToHtml(buf: Buffer): Promise<string>`.
|
||||
|
||||
Wraps `mammoth.convertToHtml({buffer: buf})` and returns `result.value`. Mammoth is pure JS, ~3 MB, embeds images as data URLs by default — no fetches, no SSRF surface. We pass `convertImage: mammoth.images.imgElement(...)` configured to emit data URLs only, as belt-and-braces.
|
||||
|
||||
Dispatch in `src/node/handler/ImportHandler.ts` mirrors the export cascade:
|
||||
|
||||
```text
|
||||
if (sofficeAvailable() === 'yes') {
|
||||
→ existing soffice path
|
||||
} else if (extension === '.docx') {
|
||||
const html = await docxBufferToHtml(buffer);
|
||||
→ hand to existing HTML import pipeline
|
||||
} else if (['.pdf','.odt','.doc','.rtf'].includes(extension)) {
|
||||
→ 4xx "this format requires soffice"
|
||||
}
|
||||
// .etherpad / .html / .txt unchanged on both branches
|
||||
```
|
||||
|
||||
The HTML import pipeline already handles whatever `mammoth` emits (semantic HTML with paragraphs, lists, headings, links, inline styles, embedded images).
|
||||
|
||||
## Error handling
|
||||
|
||||
Native conversion errors surface to the client as 5xx with a logged error line that includes the pad id and format:
|
||||
|
||||
```text
|
||||
} catch (err) {
|
||||
console.error(`native ${type} export failed for pad "${padId}":`, err);
|
||||
res.status(500).send(`Failed to export pad as ${type}.`);
|
||||
}
|
||||
```
|
||||
|
||||
No fallback chain. No silent retries.
|
||||
|
||||
## Tests
|
||||
|
||||
`src/tests/backend/specs/export.ts` — revise existing native-DOCX tests:
|
||||
|
||||
- Set `settings.soffice = null` (was `'false'` — fixes Qodo #3)
|
||||
- Assert response is a ZIP-signature DOCX with the correct content-type
|
||||
- Keep the `require.resolve('html-to-docx')` describe-skip guard for the `upgrade-from-latest-release` CI job
|
||||
|
||||
`src/tests/backend/specs/export.ts` — add native-PDF tests:
|
||||
|
||||
- With `settings.soffice = null`, GET `/p/<pad>/export/pdf` → 200, `Content-Type: application/pdf`, body starts with `%PDF-`
|
||||
- Same describe-skip guard for `pdfkit` and `htmlparser2`
|
||||
|
||||
Negative test: with `settings.soffice = null`, GET `/p/<pad>/export/odt` still returns the "not enabled" message (proves we tightened the right gate).
|
||||
|
||||
`src/tests/backend/specs/export.ts` (or a sibling file) — unit test for `stripRemoteImages`:
|
||||
|
||||
- `<img src="https://evil/x.png">` → dropped
|
||||
- `<img src="data:image/png;base64,…">` → kept
|
||||
- `<img src="/local/x.png">` → kept (same-origin/relative)
|
||||
|
||||
A new import test file (e.g. `src/tests/backend/specs/import.ts` — there is no existing import-flow file in `src/tests/backend/specs/`, only `ImportEtherpad.ts`) for native DOCX import:
|
||||
|
||||
- Fixture: a small known `.docx` with a heading, a paragraph, and a bullet list, committed under `src/tests/backend/specs/fixtures/`
|
||||
- With `settings.soffice = null`, POST `/p/<pad>/import` with the fixture → assert pad atext/HTML contains the expected structure
|
||||
- Negative: rename fixture to `.odt` extension, POST → still rejected with the "requires soffice" message
|
||||
|
||||
`exportHTMLSend` plugin hook: verify by reading the code whether the hook fires on the native paths (currently it's only invoked on the `type === 'html'` branch). If a small move is needed to keep the hook contract intact across native DOCX/PDF, include it. If the existing behavior is "hook only fires for html export", document that and don't change it — out of scope for this spec.
|
||||
|
||||
## Files touched
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `src/node/handler/ExportHandler.ts` | Replace flag-gated branch with soffice-first cascade; call sanitizer; native PDF + DOCX dispatch |
|
||||
| `src/node/handler/ImportHandler.ts` | Soffice-first cascade; native DOCX import dispatch |
|
||||
| `src/node/utils/ExportPdfNative.ts` | **new** — pdfkit walker, ≤500 lines (bail-out criterion) |
|
||||
| `src/node/utils/ExportSanitizeHtml.ts` | **new** — `stripRemoteImages`, ~50 lines |
|
||||
| `src/node/utils/ImportDocxNative.ts` | **new** — mammoth wrapper, ~30 lines |
|
||||
| `src/node/hooks/express/importexport.ts` | Tighten export and import route guards to `['odt','doc','pdf','rtf']`-as-appropriate |
|
||||
| `src/node/utils/Settings.ts` | **revert** `nativeDocxExport` field (introduced by PR #7568) |
|
||||
| `src/static/js/pad_impexp.ts` | Always show DOCX + PDF export links; ODT link still gated on `exportAvailable` |
|
||||
| `src/package.json` | Add `pdfkit`, `htmlparser2`, `mammoth`. Keep `html-to-docx`. Drop nothing. |
|
||||
| `pnpm-lock.yaml` | Lockfile regen |
|
||||
| `settings.json.template`, `settings.json.docker` | **revert** `nativeDocxExport` entries |
|
||||
| `doc/docker.md` | **revert** `NATIVE_DOCX_EXPORT` row |
|
||||
| `src/tests/backend/specs/export.ts` | Revise DOCX tests (`soffice=null`); add PDF tests; add negative ODT; add unit test for sanitizer |
|
||||
| `src/tests/backend/specs/import.ts` | Add native DOCX import tests; add negative ODT |
|
||||
| `src/tests/backend/specs/fixtures/<file>.docx` | **new** — small DOCX fixture |
|
||||
|
||||
## Open questions handled in implementation, not spec
|
||||
|
||||
- Exact error response shape for the route-guard-rejected formats — match whatever the existing soffice-disabled path uses, no fresh design.
|
||||
- Whether `exportHTMLSend` needs to fire on the native paths — covered in the test plan; verify against current behavior, don't expand scope.
|
||||
- Image MIME sniffing for `data:` URLs in the PDF walker — `pdfkit` accepts PNG/JPEG buffers; we'll decode the base64 and let pdfkit reject unsupported types, surfacing as a converter error.
|
||||
|
||||
## Dependencies summary
|
||||
|
||||
| Package | Purpose | Approx install size |
|
||||
|---|---|---|
|
||||
| `html-to-docx` | DOCX export (pre-existing in PR #7568) | ~5 MB |
|
||||
| `pdfkit` | PDF export rendering | ~2 MB |
|
||||
| `htmlparser2` | HTML SAX parser used by walker + sanitizer | <1 MB |
|
||||
| `mammoth` | DOCX → HTML import | ~3 MB |
|
||||
|
||||
Total added install: roughly 11 MB across all four. Compared against ~500 MB for LibreOffice and ~200 MB for puppeteer (the alternative considered and rejected in #7538), this is the right tradeoff for the structural-fidelity bar.
|
||||
|
||||
## Out of scope (deferred to follow-ups)
|
||||
|
||||
- Native ODT export — file follow-up issue.
|
||||
- Native PDF/ODT/DOC/RTF import — file follow-up issue, document why they were rejected (no mature pure-JS readers).
|
||||
- Memory/timeout caps on conversion — add when production signal warrants.
|
||||
- Plugin hook coverage on native paths — beyond the `exportHTMLSend` check above.
|
||||
716
pnpm-lock.yaml
generated
716
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load diff
|
|
@ -90,7 +90,72 @@ exports.doExport = async (req: any, res: any, padId: string, readOnlyId: string,
|
|||
return;
|
||||
}
|
||||
|
||||
// else write the html export to a file
|
||||
// Soffice-first dispatch (issue #7538). When soffice is configured
|
||||
// we keep the legacy convert-via-tempfile path; when it's not, we
|
||||
// hand DOCX to html-to-docx and PDF to our pdfkit walker — both
|
||||
// pure-JS, in-process. No fallback chain: native errors surface as
|
||||
// 5xx so admins see real failures instead of silent shadowing.
|
||||
const {sofficeAvailable} = require('../utils/Settings');
|
||||
const sofState = sofficeAvailable();
|
||||
const goNative = sofState === 'no'
|
||||
|| (sofState === 'withoutPDF' && type === 'pdf');
|
||||
|
||||
if (goNative) {
|
||||
const {
|
||||
stripRemoteImages, extractBody, wrapLooseLines, dropEmptyBlocks,
|
||||
applyMonospaceToCode,
|
||||
} = require('../utils/ExportSanitizeHtml');
|
||||
// The HTML pipeline returns a full document (head, style, body); the
|
||||
// legacy soffice path renders that fine, but the in-process
|
||||
// converters need just the body content to avoid leaking CSS into
|
||||
// the output and to drop the document-level whitespace that creates
|
||||
// stray paragraph breaks at the top of the result.
|
||||
// dropEmptyBlocks strips heading-styled blank-line wrappers that
|
||||
// ep_headings2 emits between every styled line.
|
||||
const bodyHtml = dropEmptyBlocks(stripRemoteImages(extractBody(html)));
|
||||
html = null;
|
||||
try {
|
||||
if (type === 'docx') {
|
||||
// applyMonospaceToCode strips `<code>`/`<pre>`/`<tt>` wrappers
|
||||
// (html-to-docx ignores them AND has a bug where it drops
|
||||
// `<a href>` children of those tags) and emits styled
|
||||
// monospace spans, forwarding any block-level alignment style
|
||||
// to a wrapping `<p>`. Run BEFORE wrapLooseLines so the
|
||||
// resulting `<p>` lands at the loose-line boundary instead
|
||||
// of getting double-wrapped.
|
||||
//
|
||||
// wrapLooseLines then handles `<br>` semantics: bare `<br>`
|
||||
// outside `<p>` becomes a soft break, `<br><br>` becomes a
|
||||
// paragraph boundary plus blank-line markers.
|
||||
const docxHtml = wrapLooseLines(applyMonospaceToCode(bodyHtml));
|
||||
const htmlToDocx = require('html-to-docx');
|
||||
const buf = await htmlToDocx(docxHtml);
|
||||
res.contentType(
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document');
|
||||
res.send(buf);
|
||||
return;
|
||||
}
|
||||
if (type === 'pdf') {
|
||||
const {htmlToPdfBuffer} = require('../utils/ExportPdfNative');
|
||||
const buf = await htmlToPdfBuffer(bodyHtml);
|
||||
res.contentType('application/pdf');
|
||||
res.send(buf);
|
||||
return;
|
||||
}
|
||||
// soffice-only formats (odt, doc) are blocked at the route guard
|
||||
// when soffice is null; reaching here means the guard is wrong.
|
||||
res.status(500).send(`Cannot export ${type} without soffice configured`);
|
||||
return;
|
||||
} catch (err) {
|
||||
console.error(
|
||||
`native ${type} export failed for pad "${padId}":`,
|
||||
err && (err as Error).stack ? (err as Error).stack : err);
|
||||
res.status(500).send(`Failed to export pad as ${type}.`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// soffice path — write the html export to a file
|
||||
const randNum = Math.floor(Math.random() * 0xFFFFFFFF);
|
||||
const srcFile = `${tempDirectory}/etherpad_export_${randNum}.html`;
|
||||
await fsp_writeFile(srcFile, html);
|
||||
|
|
|
|||
|
|
@ -65,6 +65,11 @@ if (settings.soffice != null) {
|
|||
exportExtension = 'html';
|
||||
}
|
||||
|
||||
// Office formats with no in-process import path (issue #7538). When soffice
|
||||
// is null these are rejected explicitly so users see a clear error instead
|
||||
// of a silent ASCII-only fallback. .docx has a native path via mammoth.
|
||||
const SOFFICE_ONLY_IMPORT_FORMATS = new Set(['.pdf', '.odt', '.doc', '.rtf']);
|
||||
|
||||
const tmpDirectory = os.tmpdir();
|
||||
|
||||
/**
|
||||
|
|
@ -130,6 +135,67 @@ const doImport = async (req:any, res:any, padId:string, authorId:string) => {
|
|||
}
|
||||
}
|
||||
|
||||
// Detect once whether the contentcollector treats h1-h6/code as block
|
||||
// elements server-side. ep_headings2 v0.2.118+ (after the
|
||||
// ep_plugin_helpers ccRegisterBlockElements wiring lands) registers
|
||||
// them; older versions don't. The two preprocessors below are only
|
||||
// needed when the plugin hook is missing — they're harmful otherwise
|
||||
// (each adds an extra blank pad line per heading transition).
|
||||
const ccBlockElems: string[] =
|
||||
([] as string[]).concat(...(hooks.callAll('ccRegisterBlockElements') || []));
|
||||
const ccBlockSet = new Set(ccBlockElems.map((t: string) => t.toLowerCase()));
|
||||
// ep_headings2 registers 'h1' along with the others when its server
|
||||
// hook is wired (ccRegisterBlockElements). h1 is sufficient as the
|
||||
// detection probe; the absence of h5/h6 in the set is a quirk of
|
||||
// ep_headings2 (it only handles h1-h4) and not a sign of a broken
|
||||
// hook.
|
||||
const headingsAreBlocks = ccBlockSet.has('h1');
|
||||
|
||||
// Native DOCX import (issue #7538): when soffice isn't configured we
|
||||
// hand .docx files to mammoth, which produces HTML — then we feed that
|
||||
// through the existing setPadHTML pipeline.
|
||||
if (settings.soffice == null && fileEnding === '.docx') {
|
||||
const buf = await fs.readFile(srcFile);
|
||||
const {docxBufferToHtml} = require('../utils/ImportDocxNative');
|
||||
const {separateAdjacentHeadingBlocks} = require('../utils/ExportSanitizeHtml');
|
||||
let nativeHtml: string;
|
||||
try {
|
||||
nativeHtml = await docxBufferToHtml(buf);
|
||||
// When the plugin hook is missing, contentcollector treats h1-h6
|
||||
// as inline and adjacent headings merge into a single pad line.
|
||||
// Insert <br> between them as a defensive workaround. Skipped when
|
||||
// the plugin already registers the tags (otherwise the <br> becomes
|
||||
// an extra blank line per heading transition).
|
||||
if (!headingsAreBlocks) {
|
||||
nativeHtml = separateAdjacentHeadingBlocks(nativeHtml);
|
||||
}
|
||||
} catch (err: any) {
|
||||
logger.warn(`Native DOCX import failed: ${err.stack || err}`);
|
||||
throw new ImportError('convertFailed');
|
||||
}
|
||||
const pad = await padManager.getPad(padId, '\n', authorId);
|
||||
try {
|
||||
await importHtml.setPadHTML(pad, nativeHtml, authorId);
|
||||
} catch (err: any) {
|
||||
logger.warn(`Error importing native DOCX HTML: ${err.stack || err}`);
|
||||
throw new ImportError('convertFailed');
|
||||
}
|
||||
padManager.unloadPad(padId);
|
||||
const reloaded = await padManager.getPad(padId, '\n', authorId);
|
||||
padManager.unloadPad(padId);
|
||||
await padMessageHandler.updatePadClients(reloaded);
|
||||
rm(srcFile);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Without soffice, the legacy office formats (pdf, odt, doc, rtf) have
|
||||
// no in-process path. Reject explicitly so the user sees a clear error
|
||||
// instead of a silent ASCII-only fallback.
|
||||
if (settings.soffice == null && SOFFICE_ONLY_IMPORT_FORMATS.has(fileEnding)) {
|
||||
logger.warn(`Cannot import ${fileEnding} without soffice configured`);
|
||||
throw new ImportError('uploadFailed');
|
||||
}
|
||||
|
||||
const destFile = path.join(tmpDirectory, `etherpad_import_${randNum}.${exportExtension}`);
|
||||
const context = {srcFile, destFile, fileEnding, padId, ImportError};
|
||||
const importHandledByPlugin = (await hooks.aCallAll('import', context)).some((x:string) => x);
|
||||
|
|
@ -205,7 +271,20 @@ const doImport = async (req:any, res:any, padId:string, authorId:string) => {
|
|||
if (!directDatabaseAccess) {
|
||||
if (importHandledByPlugin || useConverter || fileIsHTML) {
|
||||
try {
|
||||
await importHtml.setPadHTML(pad, text, authorId);
|
||||
// Etherpad's HTML export wraps each pad line in `<p>...</p>`
|
||||
// (or `<h1>`, `<code>`, etc.) and then appends a `<br>` between
|
||||
// lines. The closing block tag already ends the line for
|
||||
// contentcollector, so the trailing `<br>` is redundant and
|
||||
// doubles every blank line on import. Collapse `</block><br>`
|
||||
// before handing to setPadHTML so HTML round-trips don't drift.
|
||||
// Only applied to HTML imports (and converted-via-soffice
|
||||
// outputs, which look the same shape) -- the docx native path
|
||||
// above doesn't go through here.
|
||||
const {collapseRedundantBrAfterBlocks} =
|
||||
require('../utils/ExportSanitizeHtml');
|
||||
const cleaned = (fileIsHTML || useConverter)
|
||||
? collapseRedundantBrAfterBlocks(text) : text;
|
||||
await importHtml.setPadHTML(pad, cleaned, authorId);
|
||||
} catch (err:any) {
|
||||
logger.warn(`Error importing, possibly caused by malformed HTML: ${err.stack || err}`);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -36,9 +36,11 @@ exports.expressCreateServer = (hookName:string, args:ArgsExpressType, cb:Functio
|
|||
return next();
|
||||
}
|
||||
|
||||
// if soffice is disabled, and this is a format we only support with soffice, output a message
|
||||
// When soffice is disabled, only block formats with no native path.
|
||||
// pdf and docx fall through to ExportHandler, which dispatches to
|
||||
// the in-process converters (issue #7538).
|
||||
if (exportAvailable() === 'no' &&
|
||||
['odt', 'pdf', 'doc', 'docx'].indexOf(req.params.type) !== -1) {
|
||||
['odt', 'doc'].indexOf(req.params.type) !== -1) {
|
||||
console.error(`Impossible to export pad "${req.params.pad}" in ${req.params.type} format.` +
|
||||
' There is no converter configured');
|
||||
|
||||
|
|
|
|||
248
src/node/utils/ExportPdfNative.ts
Normal file
248
src/node/utils/ExportPdfNative.ts
Normal file
|
|
@ -0,0 +1,248 @@
|
|||
'use strict';
|
||||
|
||||
import {Parser} from 'htmlparser2';
|
||||
import {PassThrough} from 'stream';
|
||||
|
||||
const PDFDocument = require('pdfkit');
|
||||
|
||||
interface InlineState {
|
||||
bold: boolean;
|
||||
italic: boolean;
|
||||
underline: boolean;
|
||||
strike: boolean;
|
||||
link?: string;
|
||||
fontSize?: number;
|
||||
align?: 'left' | 'center' | 'right' | 'justify';
|
||||
mono?: boolean;
|
||||
}
|
||||
|
||||
const parseAlign = (style: string | undefined): InlineState['align'] | undefined => {
|
||||
if (!style) return undefined;
|
||||
const m = /text-align\s*:\s*(left|center|right|justify)/i.exec(style);
|
||||
return m ? (m[1].toLowerCase() as InlineState['align']) : undefined;
|
||||
};
|
||||
|
||||
const HEADING_SIZES: Record<string, number> = {
|
||||
h1: 24, h2: 20, h3: 16, h4: 14, h5: 12, h6: 11,
|
||||
};
|
||||
|
||||
// Tags whose text content must never appear in the rendered PDF (CSS,
|
||||
// scripts, document metadata). The walker maintains a depth counter so that
|
||||
// nested elements inside one of these are ignored too.
|
||||
const SKIP_TAGS = new Set(['head', 'style', 'script', 'title', 'meta', 'link', 'noscript']);
|
||||
|
||||
const decodeDataUri = (src: string): Buffer | null => {
|
||||
const m = /^data:[^;,]+;base64,(.+)$/i.exec(src);
|
||||
if (!m) return null;
|
||||
try {
|
||||
return Buffer.from(m[1], 'base64');
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
export const htmlToPdfBuffer = (html: string): Promise<Buffer> =>
|
||||
new Promise((resolve, reject) => {
|
||||
// compress:false keeps the content stream uncompressed. Pads are small
|
||||
// enough that the size cost is negligible, and it lets ops greppable PDFs
|
||||
// out of the box for accessibility / search-engine indexers that don't
|
||||
// FlateDecode.
|
||||
const doc = new PDFDocument({margin: 50, compress: false});
|
||||
const stream = new PassThrough();
|
||||
const chunks: Buffer[] = [];
|
||||
stream.on('data', (c: Buffer) => chunks.push(c));
|
||||
stream.on('end', () => resolve(Buffer.concat(chunks)));
|
||||
stream.on('error', reject);
|
||||
doc.pipe(stream);
|
||||
|
||||
const styleStack: InlineState[] = [{
|
||||
bold: false, italic: false, underline: false, strike: false,
|
||||
}];
|
||||
const listType: ('ul' | 'ol' | null)[] = [];
|
||||
const listIndex: number[] = [];
|
||||
let pendingNewline = false;
|
||||
let skipDepth = 0;
|
||||
|
||||
const top = () => styleStack[styleStack.length - 1];
|
||||
|
||||
const applyFont = () => {
|
||||
const s = top();
|
||||
let variant: string;
|
||||
if (s.mono) {
|
||||
variant =
|
||||
s.bold && s.italic ? 'Courier-BoldOblique' :
|
||||
s.bold ? 'Courier-Bold' :
|
||||
s.italic ? 'Courier-Oblique' :
|
||||
'Courier';
|
||||
} else {
|
||||
variant =
|
||||
s.bold && s.italic ? 'Helvetica-BoldOblique' :
|
||||
s.bold ? 'Helvetica-Bold' :
|
||||
s.italic ? 'Helvetica-Oblique' :
|
||||
'Helvetica';
|
||||
}
|
||||
doc.font(variant);
|
||||
doc.fontSize(s.fontSize || 11);
|
||||
};
|
||||
|
||||
// Track whether the current run started with an alignment override so
|
||||
// we apply `align` exactly once per pdfkit text() call (pdfkit uses the
|
||||
// align option of the first call in a continued run for the whole line).
|
||||
let runStartedAligned = false;
|
||||
|
||||
const writeText = (raw: string) => {
|
||||
if (!raw) return;
|
||||
if (pendingNewline) {
|
||||
doc.moveDown(0.5);
|
||||
pendingNewline = false;
|
||||
}
|
||||
const s = top();
|
||||
applyFont();
|
||||
const opts: any = {continued: true};
|
||||
if (s.underline) opts.underline = true;
|
||||
if (s.strike) opts.strike = true;
|
||||
if (s.link) opts.link = s.link;
|
||||
if (s.align && !runStartedAligned) {
|
||||
opts.align = s.align;
|
||||
runStartedAligned = true;
|
||||
}
|
||||
doc.text(raw, opts);
|
||||
};
|
||||
|
||||
// End the current `continued: true` text run. pdfkit's `text('', false)`
|
||||
// closes the run but does NOT advance the cursor — subsequent text would
|
||||
// overlay at the same y. Use `breakLine` whenever a true newline is
|
||||
// intended (br, end-of-block, list items).
|
||||
const flushLine = () => {
|
||||
doc.text('', {continued: false});
|
||||
runStartedAligned = false;
|
||||
};
|
||||
const breakLine = () => {
|
||||
flushLine();
|
||||
doc.moveDown(1);
|
||||
};
|
||||
|
||||
const parser = new Parser({
|
||||
onopentag(name, attribs) {
|
||||
if (SKIP_TAGS.has(name)) skipDepth += 1;
|
||||
if (skipDepth > 0) {
|
||||
styleStack.push({...top()});
|
||||
return;
|
||||
}
|
||||
const cur = top();
|
||||
const next: InlineState = {...cur};
|
||||
switch (name) {
|
||||
case 'b': case 'strong': next.bold = true; break;
|
||||
case 'i': case 'em': next.italic = true; break;
|
||||
case 'u': next.underline = true; break;
|
||||
case 's': case 'strike': case 'del': next.strike = true; break;
|
||||
case 'a': next.link = attribs.href; next.underline = true; break;
|
||||
case 'code': case 'tt': case 'kbd': case 'samp': {
|
||||
next.mono = true;
|
||||
// ep_headings2 uses <code style='text-align:...'> as a block-
|
||||
// styled "code" line, so read the alignment off the opening
|
||||
// tag too. parseAlign returns undefined when no text-align
|
||||
// is set, so this is a no-op for inline <code> usage.
|
||||
const a = parseAlign(attribs.style);
|
||||
if (a) next.align = a;
|
||||
break;
|
||||
}
|
||||
case 'pre': {
|
||||
next.mono = true;
|
||||
const a = parseAlign(attribs.style);
|
||||
if (a) next.align = a;
|
||||
if (!pendingNewline) breakLine();
|
||||
break;
|
||||
}
|
||||
case 'h1': case 'h2': case 'h3': case 'h4': case 'h5': case 'h6': {
|
||||
next.fontSize = HEADING_SIZES[name];
|
||||
next.bold = true;
|
||||
const a = parseAlign(attribs.style);
|
||||
if (a) next.align = a;
|
||||
if (!pendingNewline) breakLine();
|
||||
break;
|
||||
}
|
||||
case 'p': case 'div': {
|
||||
const a = parseAlign(attribs.style);
|
||||
if (a) next.align = a;
|
||||
if (!pendingNewline) breakLine();
|
||||
break;
|
||||
}
|
||||
case 'ul': case 'ol':
|
||||
listType.push(name as 'ul' | 'ol');
|
||||
listIndex.push(0);
|
||||
breakLine();
|
||||
break;
|
||||
case 'li': {
|
||||
breakLine();
|
||||
const t = listType[listType.length - 1] || 'ul';
|
||||
if (t === 'ol') listIndex[listIndex.length - 1] += 1;
|
||||
const prefix = t === 'ul'
|
||||
? '• '
|
||||
: `${listIndex[listIndex.length - 1]}. `;
|
||||
const indent = ' '.repeat(Math.max(0, listType.length - 1));
|
||||
applyFont();
|
||||
doc.text(`${indent}${prefix}`, {continued: true});
|
||||
break;
|
||||
}
|
||||
case 'br':
|
||||
breakLine();
|
||||
break;
|
||||
case 'img': {
|
||||
const buf = decodeDataUri(attribs.src || '');
|
||||
if (buf) {
|
||||
flushLine();
|
||||
try { doc.image(buf, {fit: [400, 300]}); } catch { /* skip bad image */ }
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
styleStack.push(next);
|
||||
},
|
||||
|
||||
ontext(text) {
|
||||
if (skipDepth > 0) return;
|
||||
// Collapse consecutive whitespace to a single space, the way an
|
||||
// HTML renderer would. Without this, literal newlines and tabs in
|
||||
// pretty-printed source HTML show up as runs of " " in the PDF.
|
||||
const collapsed = text.replace(/[\s ]+/g, ' ');
|
||||
if (collapsed === ' ') return; // pure-whitespace runs are dropped
|
||||
writeText(collapsed);
|
||||
},
|
||||
|
||||
onclosetag(name) {
|
||||
if (skipDepth > 0) {
|
||||
if (SKIP_TAGS.has(name)) skipDepth -= 1;
|
||||
styleStack.pop();
|
||||
if (styleStack.length === 0) {
|
||||
styleStack.push({bold: false, italic: false, underline: false, strike: false});
|
||||
}
|
||||
return;
|
||||
}
|
||||
switch (name) {
|
||||
case 'h1': case 'h2': case 'h3': case 'h4': case 'h5': case 'h6':
|
||||
case 'p': case 'div': case 'pre':
|
||||
breakLine();
|
||||
pendingNewline = true;
|
||||
break;
|
||||
case 'li':
|
||||
flushLine();
|
||||
break;
|
||||
case 'ul': case 'ol':
|
||||
listType.pop();
|
||||
listIndex.pop();
|
||||
doc.moveDown(0.3);
|
||||
break;
|
||||
}
|
||||
styleStack.pop();
|
||||
if (styleStack.length === 0) {
|
||||
styleStack.push({bold: false, italic: false, underline: false, strike: false});
|
||||
}
|
||||
},
|
||||
}, {decodeEntities: true, lowerCaseTags: true});
|
||||
|
||||
parser.write(html);
|
||||
parser.end();
|
||||
flushLine();
|
||||
doc.end();
|
||||
});
|
||||
215
src/node/utils/ExportSanitizeHtml.ts
Normal file
215
src/node/utils/ExportSanitizeHtml.ts
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
'use strict';
|
||||
|
||||
import {Parser} from 'htmlparser2';
|
||||
|
||||
// Pull `<body>...</body>` out of a full HTML document. Etherpad's
|
||||
// `getPadHTMLDocument()` returns a complete page — `<head>` with a `<style>`
|
||||
// block, doctype, etc. The legacy LibreOffice path renders that fine, but
|
||||
// the in-process converters (html-to-docx, our pdfkit walker) treat
|
||||
// non-body content as renderable, leaking CSS into the output and giving
|
||||
// blank-line issues from the leading whitespace inside `<body>`. This helper
|
||||
// extracts the body content and trims surrounding whitespace; if the input
|
||||
// has no `<body>`, it's returned unchanged so plugin-shaped fragments still
|
||||
// flow through.
|
||||
const BODY_RE = /<body[^>]*>([\s\S]*?)<\/body>/i;
|
||||
export const extractBody = (html: string): string => {
|
||||
const m = BODY_RE.exec(html);
|
||||
if (!m) return html;
|
||||
return m[1].replace(/^[\s ]+/, '').replace(/[\s ]+$/, '');
|
||||
};
|
||||
|
||||
// Drop `<br>` immediately following a closing block tag. Etherpad's
|
||||
// HTML export writes one `<p>...</p>` per pad line (or `<h1>...</h1>`,
|
||||
// `<code>...</code>` for the styled ones from ep_align/ep_headings2),
|
||||
// then appends a `<br>` between lines. The `<br>` is redundant — the
|
||||
// closing block tag already ends the line — and on import the server
|
||||
// content collector counts BOTH as line breaks, so every blank line
|
||||
// between two paragraphs gets duplicated.
|
||||
const REDUNDANT_BR_RE =
|
||||
/(<\/(?:p|h[1-6]|div|pre|blockquote|code|ul|ol|li|table|tr|td|th)>)\s*<br\s*\/?>/gi;
|
||||
export const collapseRedundantBrAfterBlocks = (html: string): string =>
|
||||
html.replace(REDUNDANT_BR_RE, '$1');
|
||||
|
||||
// Insert a `<br>` between adjacent heading-style blocks so etherpad's
|
||||
// server-side content collector breaks them into separate pad lines.
|
||||
//
|
||||
// Background: contentcollector's default `_blockElems` set is just
|
||||
// `{div, p, pre, li}`. ep_headings2 registers the CLIENT-side
|
||||
// `aceRegisterBlockElements` for `h1..h4` and `code`, but not the
|
||||
// SERVER-side `ccRegisterBlockElements`, so on import contentcollector
|
||||
// treats those tags as inline and merges adjacent ones into a single
|
||||
// line. This helper fires on the IMPORT path (after mammoth produces
|
||||
// HTML) to forcibly separate them.
|
||||
const ADJACENT_HEADING_BLOCKS_RE =
|
||||
/(<\/(?:h[1-6]|code)>)(\s*<(?:h[1-6]|code|p|pre|div|blockquote|ul|ol)\b)/gi;
|
||||
export const separateAdjacentHeadingBlocks = (html: string): string =>
|
||||
html.replace(ADJACENT_HEADING_BLOCKS_RE, '$1<br>$2');
|
||||
|
||||
// Convert code/pre/tt/kbd/samp wrappers to plain styled spans (and a
|
||||
// wrapping <p> when block-styled) so html-to-docx renders them with
|
||||
// `<w:rFonts w:ascii="Courier New" .../>`. The bare `<code>` tag
|
||||
// isn't translated to a font change by html-to-docx, AND it has a
|
||||
// nasty bug where any `<a href>` nested inside `<code>` (or inside a
|
||||
// styled `<span>`) is silently dropped from the output. Workaround:
|
||||
// drop the code/pre tag entirely, wrap non-anchor text in monospace
|
||||
// spans, leave anchors as-is. For block-level usage (e.g.
|
||||
// ep_headings2's `<code style='text-align:right'>` per-line wrapper)
|
||||
// we emit a wrapping `<p>` and forward any text-align style.
|
||||
//
|
||||
// Run BEFORE `wrapLooseLines` so the resulting `<p>` lands at the
|
||||
// loose-line boundary instead of getting double-wrapped.
|
||||
const MONO_TAGS_RE = /<(code|tt|kbd|samp|pre)\b([^>]*)>([\s\S]*?)<\/\1>/gi;
|
||||
const ANCHOR_RE = /<a\b[^>]*>[\s\S]*?<\/a>/gi;
|
||||
const STYLE_ATTR_RE = /\bstyle\s*=\s*(['"])([^'"]*)\1/i;
|
||||
const COURIER_OPEN = '<span style="font-family:\'Courier New\', monospace">';
|
||||
const COURIER_CLOSE = '</span>';
|
||||
|
||||
const wrapNonAnchorSegments = (content: string): string => {
|
||||
let out = '';
|
||||
let lastIndex = 0;
|
||||
let m: RegExpExecArray | null;
|
||||
ANCHOR_RE.lastIndex = 0;
|
||||
while ((m = ANCHOR_RE.exec(content)) !== null) {
|
||||
const before = content.slice(lastIndex, m.index);
|
||||
if (before) out += `${COURIER_OPEN}${before}${COURIER_CLOSE}`;
|
||||
out += m[0];
|
||||
lastIndex = m.index + m[0].length;
|
||||
}
|
||||
if (lastIndex < content.length) {
|
||||
const after = content.slice(lastIndex);
|
||||
if (after) out += `${COURIER_OPEN}${after}${COURIER_CLOSE}`;
|
||||
}
|
||||
return out || `${COURIER_OPEN}${content}${COURIER_CLOSE}`;
|
||||
};
|
||||
|
||||
export const applyMonospaceToCode = (html: string): string =>
|
||||
html.replace(MONO_TAGS_RE, (_, tag, attrs, content) => {
|
||||
const styled = wrapNonAnchorSegments(content);
|
||||
// Block-level treatment for <pre> (always) and <code>/<tt>/etc.
|
||||
// when the wrapper carries an inline style (ep_headings2 +
|
||||
// ep_align emit `<code style='text-align:right'>` for each pad
|
||||
// line). Forward the style to a wrapping `<p>`.
|
||||
const styleMatch = STYLE_ATTR_RE.exec(attrs);
|
||||
if (tag.toLowerCase() === 'pre' || styleMatch) {
|
||||
const styleAttr = styleMatch ? ` style="${styleMatch[2]}"` : '';
|
||||
return `<p${styleAttr}>${styled}</p>`;
|
||||
}
|
||||
return styled;
|
||||
});
|
||||
|
||||
// Drop block elements whose only content is whitespace. Etherpad plugins
|
||||
// like ep_headings2 emit a heading-styled blank-line block (e.g.
|
||||
// `<h1 style='text-align:right'></h1>`) after every styled line, which
|
||||
// turns into an extra empty `<w:p>` in DOCX and an extra blank line in
|
||||
// PDF. Iterates because removing one empty wrapper can expose another.
|
||||
//
|
||||
// Note: `<p></p>` is intentionally NOT in this list — `wrapLooseLines`
|
||||
// uses empty `<p>` markers to encode blank-line gaps for round-trip
|
||||
// fidelity through html-to-docx.
|
||||
const EMPTY_BLOCK_RE = /<(h[1-6]|code|pre|div|blockquote)\b[^>]*>\s*<\/\1>/gi;
|
||||
export const dropEmptyBlocks = (html: string): string => {
|
||||
let prev: string;
|
||||
let cur = html;
|
||||
do {
|
||||
prev = cur;
|
||||
cur = cur.replace(EMPTY_BLOCK_RE, '');
|
||||
} while (cur !== prev);
|
||||
return cur;
|
||||
};
|
||||
|
||||
// Wrap loose text + inline content in `<p>` blocks so html-to-docx renders
|
||||
// `<br>` as a soft line break (`<w:br/>`) instead of a paragraph break
|
||||
// (`<w:p>`). Etherpad's HTML export uses bare `<br>` for every line and
|
||||
// `<br><br>` for blank lines, so without this DOCX exports get one Word
|
||||
// paragraph per line and two empty paragraphs for every blank line.
|
||||
//
|
||||
// Strategy: capture `<br>` separators of length >= 2 (paragraph separators)
|
||||
// AND remember how many `<br>`s each separator contains, so blank-line
|
||||
// gaps survive the round-trip. For N consecutive `<br>`s, emit one
|
||||
// closing-then-opening paragraph break PLUS (N - 2) empty `<p></p>`
|
||||
// markers (each empty paragraph = one blank pad line).
|
||||
const BLOCK_HEAD_RE = /^<(?:p|h[1-6]|ul|ol|table|blockquote|pre|div)[\s>/]/i;
|
||||
// Anchored so the inner `\s*` can't overlap with surrounding whitespace and
|
||||
// trigger exponential backtracking. Matches `<br>` followed by at least one
|
||||
// more `<br>` (with optional whitespace between).
|
||||
const BR_PARA_RE = /<br\s*\/?>(?:\s*<br\s*\/?>)+/gi;
|
||||
const TRAILING_BR_RE = /(?:<br\s*\/?>\s*)+$/i;
|
||||
const BR_COUNT_RE = /<br/gi;
|
||||
export const wrapLooseLines = (html: string): string => {
|
||||
// split() with a capturing group keeps the separators in the result, so
|
||||
// parts[i] alternates between content (even i) and br-run separator
|
||||
// (odd i).
|
||||
const parts = html.split(/(<br\s*\/?>(?:\s*<br\s*\/?>)+)/gi);
|
||||
const out: string[] = [];
|
||||
for (let i = 0; i < parts.length; i++) {
|
||||
if (i % 2 === 0) {
|
||||
const c = parts[i].replace(TRAILING_BR_RE, '').trim();
|
||||
if (!c) continue;
|
||||
out.push(BLOCK_HEAD_RE.test(c) ? c : `<p>${c}</p>`);
|
||||
} else {
|
||||
// Separator of N >= 2 <br>s. The first <br> is the paragraph
|
||||
// boundary; the remaining (N - 1) each represent one blank pad
|
||||
// line, emitted as an empty <p></p>.
|
||||
const n = (parts[i].match(BR_COUNT_RE) || []).length;
|
||||
for (let k = 0; k < n - 1; k++) out.push('<p></p>');
|
||||
}
|
||||
}
|
||||
return out.join('');
|
||||
};
|
||||
|
||||
const isLocalSrc = (src: string): boolean => {
|
||||
if (!src) return true;
|
||||
if (src.startsWith('data:')) return true;
|
||||
if (src.startsWith('//')) return false;
|
||||
if (/^[a-z][a-z0-9+.-]*:/i.test(src)) return false;
|
||||
return true;
|
||||
};
|
||||
|
||||
const escapeAttr = (s: string): string =>
|
||||
s.replace(/&/g, '&').replace(/"/g, '"').replace(/</g, '<');
|
||||
|
||||
const escapeText = (s: string): string =>
|
||||
s.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>');
|
||||
|
||||
const VOID_TAGS = new Set([
|
||||
'area', 'base', 'br', 'col', 'embed', 'hr', 'img', 'input',
|
||||
'link', 'meta', 'source', 'track', 'wbr',
|
||||
]);
|
||||
|
||||
export const stripRemoteImages = (html: string): string => {
|
||||
let out = '';
|
||||
const parser = new Parser({
|
||||
onopentag(name, attribs) {
|
||||
if (name === 'img') {
|
||||
const src = attribs.src || '';
|
||||
if (isLocalSrc(src)) {
|
||||
let tag = '<img';
|
||||
for (const [k, v] of Object.entries(attribs)) {
|
||||
tag += ` ${k}="${escapeAttr(v)}"`;
|
||||
}
|
||||
tag += '>';
|
||||
out += tag;
|
||||
} else {
|
||||
out += escapeText(attribs.alt || '');
|
||||
}
|
||||
return;
|
||||
}
|
||||
let tag = `<${name}`;
|
||||
for (const [k, v] of Object.entries(attribs)) {
|
||||
tag += ` ${k}="${escapeAttr(v)}"`;
|
||||
}
|
||||
tag += '>';
|
||||
out += tag;
|
||||
},
|
||||
ontext(text) {
|
||||
out += text;
|
||||
},
|
||||
onclosetag(name) {
|
||||
if (VOID_TAGS.has(name)) return;
|
||||
out += `</${name}>`;
|
||||
},
|
||||
}, {decodeEntities: false, lowerCaseTags: true});
|
||||
parser.write(html);
|
||||
parser.end();
|
||||
return out;
|
||||
};
|
||||
83
src/node/utils/ImportDocxNative.ts
Normal file
83
src/node/utils/ImportDocxNative.ts
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
'use strict';
|
||||
|
||||
const mammoth = require('mammoth');
|
||||
const JSZip = require('jszip');
|
||||
|
||||
// mammoth strips paragraph alignment (<w:jc>) when it converts a docx to
|
||||
// HTML; it has no equivalent style-mapping for justification. To keep
|
||||
// alignment through the round-trip we walk the docx's document.xml
|
||||
// directly, pull the `w:val` from each `<w:p>`'s `<w:jc>`, and inject a
|
||||
// matching `style="text-align:..."` onto the corresponding block element
|
||||
// in mammoth's output. Match is by document order: the Nth `<p>` /
|
||||
// `<h1>...<h6>` in mammoth's output corresponds to the Nth `<w:p>` in
|
||||
// the docx.
|
||||
const PARA_RE = /<w:p\b[^>]*>([\s\S]*?)<\/w:p>/g;
|
||||
const JC_RE = /<w:jc\s+w:val=["']([^"']+)["']/;
|
||||
// Word's `w:jc` accepts more values than CSS text-align; map the ones
|
||||
// we want to surface and skip the rest.
|
||||
const JC_TO_CSS: Record<string, string> = {
|
||||
left: 'left',
|
||||
start: 'left',
|
||||
center: 'center',
|
||||
right: 'right',
|
||||
end: 'right',
|
||||
both: 'justify',
|
||||
justify: 'justify',
|
||||
distribute: 'justify',
|
||||
};
|
||||
|
||||
const extractAlignmentMap = async (buffer: Buffer): Promise<Array<string|null>> => {
|
||||
const aligns: Array<string|null> = [];
|
||||
try {
|
||||
const zip = await JSZip.loadAsync(buffer);
|
||||
const file = zip.file('word/document.xml');
|
||||
if (!file) return aligns;
|
||||
const xml: string = await file.async('text');
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = PARA_RE.exec(xml)) !== null) {
|
||||
const jcMatch = JC_RE.exec(m[1]);
|
||||
const css = jcMatch ? JC_TO_CSS[jcMatch[1].toLowerCase()] : null;
|
||||
aligns.push(css || null);
|
||||
}
|
||||
} catch {
|
||||
// Best-effort — if the docx structure is anything other than a
|
||||
// standard document.xml, fall back to no alignment.
|
||||
}
|
||||
return aligns;
|
||||
};
|
||||
|
||||
const applyAlignmentToHtml = (html: string, aligns: Array<string|null>): string => {
|
||||
if (aligns.length === 0 || !aligns.some((a) => a && a !== 'left')) return html;
|
||||
let i = 0;
|
||||
return html.replace(/<(p|h[1-6])(\b[^>]*)>/gi, (whole: string, tag: string, attrs: string) => {
|
||||
const align = aligns[i++];
|
||||
if (!align || align === 'left') return whole;
|
||||
if (/\bstyle\s*=/.test(attrs)) {
|
||||
return `<${tag}${attrs.replace(
|
||||
/\bstyle\s*=\s*(['"])([^'"]*)\1/i,
|
||||
(_full: string, q: string, val: string) => `style=${q}${val}; text-align:${align}${q}`)}>`;
|
||||
}
|
||||
return `<${tag}${attrs} style="text-align:${align}">`;
|
||||
});
|
||||
};
|
||||
|
||||
export const docxBufferToHtml = async (buffer: Buffer): Promise<string> => {
|
||||
const aligns = await extractAlignmentMap(buffer);
|
||||
const result = await mammoth.convertToHtml(
|
||||
{buffer},
|
||||
{
|
||||
// Preserve empty paragraphs so blank pad lines survive a
|
||||
// round-trip. mammoth defaults to true and drops them, which
|
||||
// collapses blank lines in the middle of a pad's content.
|
||||
ignoreEmptyParagraphs: false,
|
||||
convertImage: mammoth.images.imgElement(async (image: any) => {
|
||||
const buf: Buffer = await image.read();
|
||||
const contentType = image.contentType || 'application/octet-stream';
|
||||
return {src: `data:${contentType};base64,${buf.toString('base64')}`};
|
||||
}),
|
||||
},
|
||||
);
|
||||
let html: string = result.value || '';
|
||||
html = applyAlignmentToHtml(html, aligns);
|
||||
return html;
|
||||
};
|
||||
|
|
@ -43,6 +43,8 @@
|
|||
"express-session": "^1.19.0",
|
||||
"find-root": "1.1.0",
|
||||
"formidable": "^3.5.4",
|
||||
"html-to-docx": "^1.8.0",
|
||||
"htmlparser2": "^12.0.0",
|
||||
"http-errors": "^2.0.1",
|
||||
"jose": "^6.2.3",
|
||||
"js-cookie": "^3.0.5",
|
||||
|
|
@ -55,6 +57,7 @@
|
|||
"lodash.clonedeep": "4.5.0",
|
||||
"log4js": "^6.9.1",
|
||||
"lru-cache": "^11.3.6",
|
||||
"mammoth": "^1.12.0",
|
||||
"measured-core": "^2.0.0",
|
||||
"mime-types": "^3.0.2",
|
||||
"mongodb": "^7.1.1",
|
||||
|
|
@ -63,6 +66,7 @@
|
|||
"nano": "^11.0.5",
|
||||
"oidc-provider": "9.8.3",
|
||||
"openapi-backend": "^5.16.1",
|
||||
"pdfkit": "^0.18.0",
|
||||
"pg": "^8.20.0",
|
||||
"prom-client": "^15.1.3",
|
||||
"proxy-addr": "^2.0.7",
|
||||
|
|
@ -111,6 +115,7 @@
|
|||
"@types/mocha": "^10.0.9",
|
||||
"@types/node": "^25.6.0",
|
||||
"@types/oidc-provider": "^9.5.0",
|
||||
"@types/pdfkit": "^0.17.6",
|
||||
"@types/semver": "^7.7.1",
|
||||
"@types/sinon": "^21.0.1",
|
||||
"@types/supertest": "^7.2.0",
|
||||
|
|
|
|||
|
|
@ -144,24 +144,18 @@ const padimpexp = (() => {
|
|||
$('#exportetherpada').attr('href', `${padRootPath}/export/etherpad`);
|
||||
$('#exportplaina').attr('href', `${padRootPath}/export/txt`);
|
||||
|
||||
// hide stuff thats not avaible if soffice is disabled
|
||||
// DOCX and PDF are always available — soffice when configured,
|
||||
// native pure-JS converters otherwise (issue #7538). ODT still
|
||||
// requires soffice. The 'withoutPDF' branch (Windows soffice
|
||||
// without PDF) is handled by the server-side cascade routing
|
||||
// PDF through native; the UI link stays.
|
||||
const wordFormat = clientVars.docxExport ? 'docx' : 'doc';
|
||||
$('#exportworda').attr('href', `${padRootPath}/export/${wordFormat}`);
|
||||
$('#exportpdfa').attr('href', `${padRootPath}/export/pdf`);
|
||||
if (clientVars.exportAvailable === 'no') {
|
||||
$('#exportworda').remove();
|
||||
$('#exportpdfa').remove();
|
||||
$('#exportopena').remove();
|
||||
$('#importmessagenoconverter').prop('hidden', false);
|
||||
} else if (clientVars.exportAvailable === 'withoutPDF') {
|
||||
$('#exportpdfa').remove();
|
||||
|
||||
$('#exportworda').attr('href', `${padRootPath}/export/${wordFormat}`);
|
||||
$('#exportopena').attr('href', `${padRootPath}/export/odt`);
|
||||
|
||||
$('#importexport').css({height: '142px'});
|
||||
$('#importexportline').css({height: '142px'});
|
||||
} else {
|
||||
$('#exportworda').attr('href', `${padRootPath}/export/${wordFormat}`);
|
||||
$('#exportpdfa').attr('href', `${padRootPath}/export/pdf`);
|
||||
$('#exportopena').attr('href', `${padRootPath}/export/odt`);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
|
||||
import {MapArrayType} from "../../../node/types/MapType";
|
||||
|
||||
const assert = require('assert').strict;
|
||||
const common = require('../common');
|
||||
const padManager = require('../../../node/db/PadManager');
|
||||
import settings from '../../../node/utils/Settings';
|
||||
|
|
@ -21,8 +22,542 @@ describe(__filename, function () {
|
|||
});
|
||||
|
||||
it('returns 500 on export error', async function () {
|
||||
settings.soffice = 'false'; // '/bin/false' doesn't work on Windows
|
||||
// With soffice configured but pointing at a binary that fails, the
|
||||
// legacy convert path errors and the route returns 500. .doc has no
|
||||
// native fallback (it stays soffice-only), so this exercises the
|
||||
// soffice error path even after #7538.
|
||||
settings.soffice = '/bin/false';
|
||||
await agent.get('/p/testExportPad/export/doc')
|
||||
.expect(500);
|
||||
});
|
||||
|
||||
// Issue #7538: in-process DOCX export via html-to-docx bypasses the
|
||||
// soffice requirement entirely. A deployment with `soffice: null`
|
||||
// should still produce a working .docx via the native path.
|
||||
describe('native DOCX export (#7538)', function () {
|
||||
before(function () {
|
||||
// The upgrade-from-latest-release CI job installs deps from the
|
||||
// PREVIOUS release's package.json (before this PR adds html-to-docx)
|
||||
// and then git-checkouts this branch's code without re-running
|
||||
// `pnpm install`. Under that workflow the module isn't resolvable.
|
||||
// Skip the block in that one case; regular backend tests (which
|
||||
// install against this branch's lockfile) still exercise it.
|
||||
try {
|
||||
require.resolve('html-to-docx');
|
||||
} catch {
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
settings.soffice = null;
|
||||
});
|
||||
|
||||
it('returns a valid DOCX archive (PK zip signature)', async function () {
|
||||
const res = await agent.get('/p/testExportPad/export/docx')
|
||||
.buffer(true)
|
||||
.parse((resp: any, callback: any) => {
|
||||
const chunks: Buffer[] = [];
|
||||
resp.on('data', (chunk: Buffer) => chunks.push(chunk));
|
||||
resp.on('end', () => callback(null, Buffer.concat(chunks)));
|
||||
})
|
||||
.expect(200);
|
||||
const body: Buffer = res.body as Buffer;
|
||||
assert.ok(body.length > 0, 'DOCX body must not be empty');
|
||||
// Word .docx files are ZIP archives — must start with the ZIP local
|
||||
// file header signature 0x504b0304 ("PK\x03\x04").
|
||||
assert.strictEqual(body[0], 0x50, 'byte 0 (P)');
|
||||
assert.strictEqual(body[1], 0x4b, 'byte 1 (K)');
|
||||
assert.strictEqual(body[2], 0x03, 'byte 2');
|
||||
assert.strictEqual(body[3], 0x04, 'byte 3');
|
||||
});
|
||||
|
||||
it('sends the Word-processing-ml content-type', async function () {
|
||||
const res = await agent.get('/p/testExportPad/export/docx').expect(200);
|
||||
assert.match(res.headers['content-type'],
|
||||
/application\/vnd\.openxmlformats-officedocument\.wordprocessingml\.document/,
|
||||
`unexpected content-type: ${res.headers['content-type']}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('native PDF export (#7538)', function () {
|
||||
before(function () {
|
||||
try {
|
||||
require.resolve('pdfkit');
|
||||
require.resolve('htmlparser2');
|
||||
} catch {
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
settings.soffice = null;
|
||||
});
|
||||
|
||||
it('returns a valid %PDF- document', async function () {
|
||||
const res = await agent.get('/p/testExportPad/export/pdf')
|
||||
.buffer(true)
|
||||
.parse((resp: any, callback: any) => {
|
||||
const chunks: Buffer[] = [];
|
||||
resp.on('data', (chunk: Buffer) => chunks.push(chunk));
|
||||
resp.on('end', () => callback(null, Buffer.concat(chunks)));
|
||||
})
|
||||
.expect(200);
|
||||
const body: Buffer = res.body as Buffer;
|
||||
assert.ok(body.length > 200, 'PDF body must be non-trivial');
|
||||
assert.strictEqual(body.slice(0, 5).toString('ascii'), '%PDF-');
|
||||
});
|
||||
|
||||
it('sends application/pdf content-type', async function () {
|
||||
const res = await agent.get('/p/testExportPad/export/pdf').expect(200);
|
||||
assert.match(res.headers['content-type'], /application\/pdf/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('odt without soffice (#7538)', function () {
|
||||
before(function () { settings.soffice = null; });
|
||||
it('returns the "not enabled" message for odt', async function () {
|
||||
const res = await agent.get('/p/testExportPad/export/odt').expect(200);
|
||||
assert.match(res.text, /This export is not enabled/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('stripRemoteImages', function () {
|
||||
const {stripRemoteImages} = require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('keeps data: URIs', function () {
|
||||
const out = stripRemoteImages(
|
||||
'<p>x</p><img src="data:image/png;base64,iVBORw0KGgo=">');
|
||||
assert.match(out, /<img[^>]+src="data:image\/png/);
|
||||
});
|
||||
|
||||
it('keeps relative URLs', function () {
|
||||
const out = stripRemoteImages('<img src="/foo/bar.png">');
|
||||
assert.match(out, /<img[^>]+src="\/foo\/bar\.png"/);
|
||||
});
|
||||
|
||||
it('drops absolute http(s) URLs and falls back to alt', function () {
|
||||
const out = stripRemoteImages(
|
||||
'<p>before<img src="https://evil.example/x.png" alt="cat">after</p>');
|
||||
assert.doesNotMatch(out, /evil\.example/);
|
||||
assert.match(out, /before/);
|
||||
assert.match(out, /cat/);
|
||||
assert.match(out, /after/);
|
||||
});
|
||||
|
||||
it('drops protocol-relative URLs', function () {
|
||||
const out = stripRemoteImages('<img src="//evil.example/x.png">');
|
||||
assert.doesNotMatch(out, /evil\.example/);
|
||||
});
|
||||
|
||||
it('passes non-image markup through unchanged', function () {
|
||||
const html = '<h1>hi</h1><p>body <a href="/x">link</a></p>';
|
||||
assert.strictEqual(stripRemoteImages(html), html);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractBody', function () {
|
||||
const {extractBody} = require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('returns trimmed body content from a full document', function () {
|
||||
const html = `<!doctype html><html><head><style>.x{color:red}</style></head><body>
|
||||
hello<br>world
|
||||
</body></html>`;
|
||||
assert.strictEqual(extractBody(html), 'hello<br>world');
|
||||
});
|
||||
|
||||
it('passes a body-less fragment through unchanged', function () {
|
||||
const html = '<p>just a fragment</p>';
|
||||
assert.strictEqual(extractBody(html), html);
|
||||
});
|
||||
|
||||
it('drops <head><style> contents', function () {
|
||||
const html = '<html><head><style>.x{}</style></head><body><p>kept</p></body></html>';
|
||||
const out = extractBody(html);
|
||||
assert.doesNotMatch(out, /style/);
|
||||
assert.doesNotMatch(out, /\.x/);
|
||||
assert.match(out, /kept/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('wrapLooseLines', function () {
|
||||
const {wrapLooseLines} = require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('wraps loose text in <p>', function () {
|
||||
assert.strictEqual(wrapLooseLines('Hello'), '<p>Hello</p>');
|
||||
});
|
||||
|
||||
it('keeps single <br> as soft break inside one paragraph', function () {
|
||||
assert.strictEqual(wrapLooseLines('A<br>B'), '<p>A<br>B</p>');
|
||||
});
|
||||
|
||||
it('splits paragraphs on consecutive <br>', function () {
|
||||
// Two <br>s between content: one paragraph break + one empty
|
||||
// <p></p> marker so the blank pad line survives a DOCX round-trip
|
||||
// through html-to-docx and mammoth.
|
||||
assert.strictEqual(wrapLooseLines('A<br><br>B'),
|
||||
'<p>A</p><p></p><p>B</p>');
|
||||
});
|
||||
|
||||
it('emits more empty <p> markers for longer <br> runs', function () {
|
||||
// Three <br>s = 2 blank pad lines between content.
|
||||
assert.strictEqual(wrapLooseLines('A<br><br><br>B'),
|
||||
'<p>A</p><p></p><p></p><p>B</p>');
|
||||
});
|
||||
|
||||
it('drops trailing <br>', function () {
|
||||
assert.strictEqual(wrapLooseLines('Foo<br>'), '<p>Foo</p>');
|
||||
});
|
||||
|
||||
it('leaves block elements alone', function () {
|
||||
const html = '<ul><li>x</li></ul>';
|
||||
assert.strictEqual(wrapLooseLines(html), html);
|
||||
});
|
||||
|
||||
it('handles realistic etherpad pad HTML', function () {
|
||||
const out = wrapLooseLines(
|
||||
'Welcome!<br><br>Body text.<br>More text.<br>');
|
||||
// <br><br> -> blank-line marker between Welcome and Body text;
|
||||
// single <br> in the second chunk stays as a soft break;
|
||||
// trailing <br> is dropped.
|
||||
assert.strictEqual(out,
|
||||
'<p>Welcome!</p><p></p><p>Body text.<br>More text.</p>');
|
||||
});
|
||||
});
|
||||
|
||||
describe('dropEmptyBlocks', function () {
|
||||
const {dropEmptyBlocks} = require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('drops empty heading blocks', function () {
|
||||
const out = dropEmptyBlocks(
|
||||
"<h1 style='text-align:right'>Hi</h1><br><h1 style='text-align:right'></h1><br>x");
|
||||
assert.strictEqual(out, "<h1 style='text-align:right'>Hi</h1><br><br>x");
|
||||
});
|
||||
|
||||
it('drops empty code blocks', function () {
|
||||
assert.strictEqual(dropEmptyBlocks('<code></code>x'), 'x');
|
||||
assert.strictEqual(
|
||||
dropEmptyBlocks('<code style="x"> \n\t </code>x'), 'x');
|
||||
});
|
||||
|
||||
it('iterates so nested empties are dropped too', function () {
|
||||
// <code></code> inside a <div> -> div becomes empty -> div drops too.
|
||||
// (<p></p> is preserved on purpose; wrapLooseLines uses it as a
|
||||
// blank-line marker for DOCX round-trip fidelity.)
|
||||
const out = dropEmptyBlocks('<div><code></code></div>after');
|
||||
assert.strictEqual(out, 'after');
|
||||
});
|
||||
|
||||
it('does not drop empty <p></p> (blank-line marker)', function () {
|
||||
const out = dropEmptyBlocks('<p>x</p><p></p><p>y</p>');
|
||||
assert.strictEqual(out, '<p>x</p><p></p><p>y</p>');
|
||||
});
|
||||
|
||||
it('keeps non-empty blocks unchanged', function () {
|
||||
const html = '<h1>Hi</h1><p>body</p><code>x = 1</code>';
|
||||
assert.strictEqual(dropEmptyBlocks(html), html);
|
||||
});
|
||||
});
|
||||
|
||||
describe('collapseRedundantBrAfterBlocks', function () {
|
||||
const {collapseRedundantBrAfterBlocks} =
|
||||
require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('drops <br> immediately after a closing <p>', function () {
|
||||
assert.strictEqual(
|
||||
collapseRedundantBrAfterBlocks('<p>x</p><br><p>y</p>'),
|
||||
'<p>x</p><p>y</p>');
|
||||
});
|
||||
|
||||
it('drops <br> after closing heading and code tags', function () {
|
||||
for (const tag of ['h1', 'h2', 'h3', 'code', 'pre', 'div', 'blockquote']) {
|
||||
assert.strictEqual(
|
||||
collapseRedundantBrAfterBlocks(`<${tag}>x</${tag}><br>`),
|
||||
`<${tag}>x</${tag}>`,
|
||||
`expected </${tag}><br> collapsed`);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps a standalone <br> between text', function () {
|
||||
const html = 'Hello<br>World';
|
||||
assert.strictEqual(collapseRedundantBrAfterBlocks(html), html);
|
||||
});
|
||||
|
||||
it('handles whitespace between </tag> and <br>', function () {
|
||||
assert.strictEqual(
|
||||
collapseRedundantBrAfterBlocks('<p>x</p> \n<br>after'),
|
||||
'<p>x</p>after');
|
||||
});
|
||||
|
||||
it('drops only one <br>, leaving any subsequent ones', function () {
|
||||
// <br><br> after a closing block represents (one redundant + one
|
||||
// intentional blank-line break). After collapsing the first, the
|
||||
// second remains.
|
||||
assert.strictEqual(
|
||||
collapseRedundantBrAfterBlocks('<p>x</p><br><br><p>y</p>'),
|
||||
'<p>x</p><br><p>y</p>');
|
||||
});
|
||||
});
|
||||
|
||||
describe('separateAdjacentHeadingBlocks', function () {
|
||||
const {separateAdjacentHeadingBlocks} =
|
||||
require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('inserts <br> between adjacent <h1> and <h2>', function () {
|
||||
assert.strictEqual(
|
||||
separateAdjacentHeadingBlocks('<h1>A</h1><h2>B</h2>'),
|
||||
'<h1>A</h1><br><h2>B</h2>');
|
||||
});
|
||||
|
||||
it('inserts <br> between adjacent <code> blocks', function () {
|
||||
assert.strictEqual(
|
||||
separateAdjacentHeadingBlocks('<code>A</code><code>B</code>'),
|
||||
'<code>A</code><br><code>B</code>');
|
||||
});
|
||||
|
||||
it('inserts <br> after a heading before a <p>', function () {
|
||||
assert.strictEqual(
|
||||
separateAdjacentHeadingBlocks('<h1>A</h1><p>B</p>'),
|
||||
'<h1>A</h1><br><p>B</p>');
|
||||
});
|
||||
|
||||
it('does not change adjacent <p> elements', function () {
|
||||
const html = '<p>A</p><p>B</p>';
|
||||
assert.strictEqual(separateAdjacentHeadingBlocks(html), html);
|
||||
});
|
||||
|
||||
it('handles three-block round-trip case', function () {
|
||||
// Mirrors what mammoth produces for a pad with H1 + H2 + Code.
|
||||
assert.strictEqual(
|
||||
separateAdjacentHeadingBlocks(
|
||||
'<h1>Welcome</h1><h2>This pad</h2><p>Code line</p>'),
|
||||
'<h1>Welcome</h1><br><h2>This pad</h2><br><p>Code line</p>');
|
||||
});
|
||||
});
|
||||
|
||||
describe('applyMonospaceToCode', function () {
|
||||
const {applyMonospaceToCode} =
|
||||
require('../../../node/utils/ExportSanitizeHtml');
|
||||
|
||||
it('emits a Courier span for inline <code>', function () {
|
||||
// The <code> tag itself is dropped (html-to-docx ignores it and
|
||||
// also breaks <a> children when they're nested inside it). The
|
||||
// text becomes a Courier-styled inline span.
|
||||
const out = applyMonospaceToCode('<code>x = 1</code>');
|
||||
assert.strictEqual(out,
|
||||
`<span style="font-family:'Courier New', monospace">x = 1</span>`);
|
||||
});
|
||||
|
||||
it('forwards block-level style to a wrapping <p>', function () {
|
||||
// ep_headings2 + ep_align emit `<code style='text-align:right'>`
|
||||
// for each "Code"-styled pad line. The alignment must reach
|
||||
// html-to-docx as a paragraph property, so we move the style
|
||||
// onto a wrapping <p>.
|
||||
const out = applyMonospaceToCode("<code style='text-align:right'>x</code>");
|
||||
assert.match(out, /<p style="text-align:right">/);
|
||||
assert.match(out, /font-family:'Courier New'/);
|
||||
});
|
||||
|
||||
it('emits <p> wrap for <pre> regardless of style', function () {
|
||||
// <pre> is always block-level.
|
||||
const out = applyMonospaceToCode('<pre>preformatted</pre>');
|
||||
assert.match(out, /^<p>/);
|
||||
assert.match(out, /<\/p>$/);
|
||||
assert.match(out, /font-family:'Courier New'/);
|
||||
});
|
||||
|
||||
it('handles inline <tt>, <kbd>, <samp> as bare spans', function () {
|
||||
for (const tag of ['tt', 'kbd', 'samp']) {
|
||||
const out = applyMonospaceToCode(`<${tag}>x</${tag}>`);
|
||||
assert.strictEqual(out,
|
||||
`<span style="font-family:'Courier New', monospace">x</span>`,
|
||||
`expected styled span (no ${tag} wrapper) but got: ${out}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('does not touch unrelated tags', function () {
|
||||
const html = '<p>plain</p><strong>bold</strong>';
|
||||
assert.strictEqual(applyMonospaceToCode(html), html);
|
||||
});
|
||||
|
||||
it('does not wrap <a> elements in the Courier span', function () {
|
||||
// Regression: html-to-docx drops <a href> content when nested
|
||||
// inside a styled span OR inside <code>. We split on anchors
|
||||
// and leave them unstyled.
|
||||
const out = applyMonospaceToCode(
|
||||
'<code>Github: <a href="https://github.com/ether/etherpad">link</a> end</code>');
|
||||
// Anchor is preserved as-is (no Courier span around it)
|
||||
assert.match(out, /<a href="https:\/\/github\.com\/ether\/etherpad">link<\/a>/);
|
||||
// Text before the anchor is wrapped
|
||||
assert.match(out, /font-family:'Courier New', monospace">Github: <\/span>/);
|
||||
// Text after the anchor is wrapped
|
||||
assert.match(out, /font-family:'Courier New', monospace"> end<\/span>/);
|
||||
// <code> wrapper is dropped
|
||||
assert.doesNotMatch(out, /<code/);
|
||||
assert.doesNotMatch(out, /<\/code>/);
|
||||
});
|
||||
|
||||
it('preserves <a> through html-to-docx round-trip', async function () {
|
||||
try { require.resolve('html-to-docx'); }
|
||||
catch { this.skip(); return; }
|
||||
const htmlToDocx = require('html-to-docx');
|
||||
const JSZip = require('jszip');
|
||||
const buf: Buffer = await htmlToDocx(applyMonospaceToCode(
|
||||
'<p><code>Github: <a href="https://github.com/ether/etherpad">site</a></code></p>'));
|
||||
const z = await JSZip.loadAsync(buf);
|
||||
const xml: string = await z.file('word/document.xml').async('text');
|
||||
// Anchor must survive: docx hyperlinks live in <w:hyperlink>
|
||||
assert.match(xml, /<w:hyperlink/, 'expected <w:hyperlink> in docx');
|
||||
assert.match(xml, /<w:t[^>]*>site<\/w:t>/, 'expected link text "site"');
|
||||
const rels: string = await z.file('word/_rels/document.xml.rels')
|
||||
.async('text');
|
||||
assert.match(rels, /github\.com\/ether\/etherpad/,
|
||||
'expected URL in document.xml.rels');
|
||||
});
|
||||
});
|
||||
|
||||
describe('htmlToPdfBuffer', function () {
|
||||
let htmlToPdfBuffer: (html: string) => Promise<Buffer>;
|
||||
|
||||
before(function () {
|
||||
try {
|
||||
require.resolve('pdfkit');
|
||||
require.resolve('htmlparser2');
|
||||
} catch {
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
htmlToPdfBuffer = require('../../../node/utils/ExportPdfNative').htmlToPdfBuffer;
|
||||
});
|
||||
|
||||
it('produces a buffer starting with %PDF-', async function () {
|
||||
const buf = await htmlToPdfBuffer('<p>hello world</p>');
|
||||
assert.ok(Buffer.isBuffer(buf), 'must return Buffer');
|
||||
assert.ok(buf.length > 100, `buffer suspiciously small: ${buf.length} bytes`);
|
||||
assert.strictEqual(buf.slice(0, 5).toString('ascii'), '%PDF-');
|
||||
});
|
||||
|
||||
// pdfkit emits visible text as hex strings inside TJ operators
|
||||
// (e.g. `Title` -> `<5469746c65>`), and pdfkit splits a single text
|
||||
// run into multiple chunks at kerning boundaries. Decode every hex
|
||||
// string we find and concatenate the result so substring matching
|
||||
// works on the visible text content.
|
||||
const decodeVisibleText = (raw: string): string => {
|
||||
const out: string[] = [];
|
||||
const re = /<([0-9a-fA-F]{2,})>/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(raw)) !== null) {
|
||||
try {
|
||||
out.push(Buffer.from(m[1], 'hex').toString('latin1'));
|
||||
} catch { /* not hex; ignore */ }
|
||||
}
|
||||
return out.join('');
|
||||
};
|
||||
|
||||
const renderText = async (html: string): Promise<string> => {
|
||||
const buf = await htmlToPdfBuffer(html);
|
||||
return buf.toString('latin1');
|
||||
};
|
||||
|
||||
it('renders headings, paragraphs, and lists', async function () {
|
||||
const raw = await renderText(`
|
||||
<h1>Title</h1>
|
||||
<p>Body paragraph here.</p>
|
||||
<ul><li>one</li><li>two</li></ul>
|
||||
<ol><li>alpha</li><li>beta</li></ol>
|
||||
`);
|
||||
const visible = decodeVisibleText(raw);
|
||||
assert.ok(visible.includes('Title'), `expected Title in: ${visible}`);
|
||||
assert.ok(visible.includes('Body paragraph here.'),
|
||||
`expected paragraph in: ${visible}`);
|
||||
assert.ok(visible.includes('one'), `expected "one" in: ${visible}`);
|
||||
assert.ok(visible.includes('two'), `expected "two" in: ${visible}`);
|
||||
assert.ok(visible.includes('alpha'), `expected "alpha" in: ${visible}`);
|
||||
assert.ok(visible.includes('beta'), `expected "beta" in: ${visible}`);
|
||||
});
|
||||
|
||||
it('emits link annotations for <a href>', async function () {
|
||||
const raw = await renderText('<p><a href="https://etherpad.org">site</a></p>');
|
||||
const visible = decodeVisibleText(raw);
|
||||
assert.ok(visible.includes('site'), `expected "site" in: ${visible}`);
|
||||
// URL is stored in a /URI dict as a plain (parenthesized) string.
|
||||
// Match the full /URI (...) form so we're verifying the PDF link
|
||||
// annotation structure, not just any occurrence of the host string.
|
||||
assert.match(raw, /\/URI\s*\(https:\/\/etherpad\.org\)/,
|
||||
'expected link target URL in PDF /URI dict');
|
||||
});
|
||||
|
||||
it('embeds data: URI images without throwing', async function () {
|
||||
const tinyPng =
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
|
||||
const buf = await htmlToPdfBuffer(`<img src="data:image/png;base64,${tinyPng}">`);
|
||||
assert.ok(buf.length > 200);
|
||||
});
|
||||
|
||||
it('ignores unknown tags rather than crashing', async function () {
|
||||
const buf = await htmlToPdfBuffer(
|
||||
'<custom-tag><p>still works</p></custom-tag>');
|
||||
assert.strictEqual(buf.slice(0, 5).toString('ascii'), '%PDF-');
|
||||
});
|
||||
|
||||
it('does not render head/style/script content', async function () {
|
||||
const raw = await renderText(`
|
||||
<html><head>
|
||||
<title>SECRET_TITLE</title>
|
||||
<style>.x { display: SECRET_CSS; }</style>
|
||||
<script>var SECRET_JS = 1;</script>
|
||||
</head><body>
|
||||
<p>visible body</p>
|
||||
</body></html>
|
||||
`);
|
||||
const visible = decodeVisibleText(raw);
|
||||
assert.doesNotMatch(visible, /SECRET_TITLE/);
|
||||
assert.doesNotMatch(visible, /SECRET_CSS/);
|
||||
assert.doesNotMatch(visible, /SECRET_JS/);
|
||||
assert.match(visible, /visible body/);
|
||||
});
|
||||
|
||||
it('honors text-align style on block elements', async function () {
|
||||
// pdfkit emits text-positioning matrices for aligned text. We assert
|
||||
// the alignment option produced different output than left-aligned
|
||||
// by checking the x coordinate of the BT block.
|
||||
const leftRaw = await renderText('<p>aligned text</p>');
|
||||
const rightRaw = await renderText('<p style="text-align:right">aligned text</p>');
|
||||
const leftX = (leftRaw.match(/1 0 0 1 (\d+(?:\.\d+)?)/) || [])[1];
|
||||
const rightX = (rightRaw.match(/1 0 0 1 (\d+(?:\.\d+)?)/) || [])[1];
|
||||
assert.ok(leftX, 'expected left x');
|
||||
assert.ok(rightX, 'expected right x');
|
||||
assert.notStrictEqual(leftX, rightX,
|
||||
`right-aligned text should sit at a different x than left-aligned (left=${leftX} right=${rightX})`);
|
||||
});
|
||||
|
||||
it('uses Courier font inside <code>', async function () {
|
||||
const raw = await renderText('<p>before <code>x = 1</code> after</p>');
|
||||
// pdfkit references the font in the resource dictionary; Courier
|
||||
// isn't in the default resources so its first use creates a new
|
||||
// /Font subtype entry. Look for "Courier" anywhere in the PDF.
|
||||
assert.match(raw, /Courier/);
|
||||
});
|
||||
|
||||
it('uses Courier font inside <pre>', async function () {
|
||||
const raw = await renderText('<pre>preformatted text</pre>');
|
||||
assert.match(raw, /Courier/);
|
||||
});
|
||||
|
||||
it('honors text-align on <code> (ep_headings2 code lines)', async function () {
|
||||
const leftRaw = await renderText('<code>x = 1</code>');
|
||||
const rightRaw = await renderText("<code style='text-align:right'>x = 1</code>");
|
||||
const leftX = (leftRaw.match(/1 0 0 1 (\d+(?:\.\d+)?)/) || [])[1];
|
||||
const rightX = (rightRaw.match(/1 0 0 1 (\d+(?:\.\d+)?)/) || [])[1];
|
||||
assert.ok(leftX, 'expected left x');
|
||||
assert.ok(rightX, 'expected right x');
|
||||
assert.notStrictEqual(leftX, rightX,
|
||||
`right-aligned <code> should sit at a different x than left-aligned (left=${leftX} right=${rightX})`);
|
||||
});
|
||||
|
||||
it('honors text-align on <pre>', async function () {
|
||||
const leftRaw = await renderText('<pre>x = 1</pre>');
|
||||
const rightRaw = await renderText("<pre style='text-align:right'>x = 1</pre>");
|
||||
const leftX = (leftRaw.match(/1 0 0 1 (\d+(?:\.\d+)?)/) || [])[1];
|
||||
const rightX = (rightRaw.match(/1 0 0 1 (\d+(?:\.\d+)?)/) || [])[1];
|
||||
assert.notStrictEqual(leftX, rightX,
|
||||
`right-aligned <pre> should sit at a different x than left-aligned (left=${leftX} right=${rightX})`);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
BIN
src/tests/backend/specs/fixtures/sample.docx
vendored
Normal file
BIN
src/tests/backend/specs/fixtures/sample.docx
vendored
Normal file
Binary file not shown.
473
src/tests/backend/specs/import.ts
Normal file
473
src/tests/backend/specs/import.ts
Normal file
|
|
@ -0,0 +1,473 @@
|
|||
'use strict';
|
||||
|
||||
import {MapArrayType} from '../../../node/types/MapType';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import {promises as fs} from 'fs';
|
||||
|
||||
const assert = require('assert').strict;
|
||||
const common = require('../common');
|
||||
const padManager = require('../../../node/db/PadManager');
|
||||
import settings from '../../../node/utils/Settings';
|
||||
|
||||
describe(__filename, function () {
|
||||
const settingsBackup: MapArrayType<any> = {};
|
||||
let agent: any;
|
||||
|
||||
before(async function () {
|
||||
agent = await common.init();
|
||||
settingsBackup.soffice = settings.soffice;
|
||||
});
|
||||
|
||||
after(function () {
|
||||
Object.assign(settings, settingsBackup);
|
||||
});
|
||||
|
||||
describe('docxBufferToHtml (#7538)', function () {
|
||||
let docxBufferToHtml: (b: Buffer) => Promise<string>;
|
||||
|
||||
before(function () {
|
||||
try { require.resolve('mammoth'); }
|
||||
catch { this.skip(); return; }
|
||||
docxBufferToHtml = require('../../../node/utils/ImportDocxNative').docxBufferToHtml;
|
||||
});
|
||||
|
||||
it('converts the sample.docx fixture to HTML', async function () {
|
||||
const buf = await fs.readFile(
|
||||
path.join(__dirname, 'fixtures', 'sample.docx'));
|
||||
const html = await docxBufferToHtml(buf);
|
||||
assert.match(html, /Heading/);
|
||||
assert.match(html, /Paragraph body\./);
|
||||
assert.match(html, /one/);
|
||||
assert.match(html, /two/);
|
||||
});
|
||||
|
||||
it('emits no remote image URLs', async function () {
|
||||
const buf = await fs.readFile(
|
||||
path.join(__dirname, 'fixtures', 'sample.docx'));
|
||||
const html = await docxBufferToHtml(buf);
|
||||
assert.doesNotMatch(html, /<img[^>]+src="https?:/);
|
||||
assert.doesNotMatch(html, /<img[^>]+src="\/\//);
|
||||
});
|
||||
|
||||
it('preserves paragraph alignment from <w:jc>', async function () {
|
||||
// Round through html-to-docx so the input docx has <w:jc> entries
|
||||
// we can verify mammoth + our workaround surface as text-align.
|
||||
try { require.resolve('html-to-docx'); }
|
||||
catch { this.skip(); return; }
|
||||
const htmlToDocx = require('html-to-docx');
|
||||
const docx: Buffer = await htmlToDocx(
|
||||
'<h1 style="text-align:right">Right heading</h1>' +
|
||||
'<p style="text-align:center">Center paragraph</p>' +
|
||||
'<p>Left paragraph</p>' +
|
||||
'<p style="text-align:justify">Justify paragraph</p>');
|
||||
const html = await docxBufferToHtml(docx);
|
||||
assert.match(html, /<h1[^>]*style="[^"]*text-align:right/,
|
||||
`expected right-aligned h1 in: ${html}`);
|
||||
assert.match(html, /<p[^>]*style="[^"]*text-align:center/,
|
||||
`expected center-aligned p in: ${html}`);
|
||||
assert.match(html, /<p[^>]*style="[^"]*text-align:justify/,
|
||||
`expected justify-aligned p in: ${html}`);
|
||||
// Left-aligned paragraph should NOT carry a redundant style attr
|
||||
// (we skip "left" because it's the CSS default).
|
||||
assert.match(html, /<p>Left paragraph<\/p>/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('end-to-end DOCX import (#7538)', function () {
|
||||
before(function () {
|
||||
try { require.resolve('mammoth'); }
|
||||
catch { this.skip(); return; }
|
||||
settings.soffice = null;
|
||||
});
|
||||
|
||||
it('imports a docx into a pad without soffice', async function () {
|
||||
const padId = 'test7538DocxImport';
|
||||
try { await padManager.removePad(padId); } catch { /* noop */ }
|
||||
const fixture = path.join(__dirname, 'fixtures', 'sample.docx');
|
||||
const res = await agent
|
||||
.post(`/p/${padId}/import`)
|
||||
.attach('file', fixture)
|
||||
.expect(200);
|
||||
assert.strictEqual(res.body.code, 0,
|
||||
`import failed: ${JSON.stringify(res.body)}`);
|
||||
const pad = await padManager.getPad(padId);
|
||||
const text = pad.text();
|
||||
assert.match(text, /Heading/);
|
||||
assert.match(text, /Paragraph body/);
|
||||
assert.match(text, /one/);
|
||||
assert.match(text, /two/);
|
||||
});
|
||||
|
||||
it('rejects odt extension when soffice is null', async function () {
|
||||
const padId = 'test7538OdtReject';
|
||||
try { await padManager.removePad(padId); } catch { /* noop */ }
|
||||
const fixture = path.join(__dirname, 'fixtures', 'sample.docx');
|
||||
const odtPath = path.join(__dirname, 'fixtures', 'sample.odt');
|
||||
await fs.copyFile(fixture, odtPath);
|
||||
try {
|
||||
const res = await agent
|
||||
.post(`/p/${padId}/import`)
|
||||
.attach('file', odtPath);
|
||||
assert.ok(
|
||||
res.status >= 400 || res.body.code !== 0,
|
||||
`expected odt import to fail when soffice is null, got: ${res.status} ${JSON.stringify(res.body)}`);
|
||||
} finally {
|
||||
await fs.unlink(odtPath).catch(() => undefined);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('DOCX export -> import round-trip (#7538)', function () {
|
||||
before(function () {
|
||||
try {
|
||||
require.resolve('html-to-docx');
|
||||
require.resolve('mammoth');
|
||||
} catch {
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
settings.soffice = null;
|
||||
});
|
||||
|
||||
// Returns the supertest Test so callers can keep chaining .expect();
|
||||
// typing as `any` because supertest's Test isn't re-exported as a type
|
||||
// we can name here.
|
||||
const fetchBuffer = (req: any): any => req
|
||||
.buffer(true)
|
||||
.parse((resp: any, cb: any) => {
|
||||
const chunks: Buffer[] = [];
|
||||
resp.on('data', (c: Buffer) => chunks.push(c));
|
||||
resp.on('end', () => cb(null, Buffer.concat(chunks)));
|
||||
});
|
||||
|
||||
it('preserves text content through native DOCX round-trip', async function () {
|
||||
const srcPadId = 'test7538RoundTripSrc';
|
||||
const dstPadId = 'test7538RoundTripDst';
|
||||
const tmpFile = path.join(os.tmpdir(), `roundtrip-${process.pid}.docx`);
|
||||
|
||||
try {
|
||||
await padManager.removePad(srcPadId);
|
||||
await padManager.removePad(dstPadId);
|
||||
} catch { /* noop */ }
|
||||
|
||||
const srcPad = await padManager.getPad(srcPadId, '\n');
|
||||
await srcPad.setText('Line one\nLine two\n\nAfter the blank line\n');
|
||||
const srcText = srcPad.text();
|
||||
assert.match(srcText, /Line one/);
|
||||
assert.match(srcText, /After the blank line/);
|
||||
|
||||
const exp = await fetchBuffer(agent.get(`/p/${srcPadId}/export/docx`))
|
||||
.expect(200);
|
||||
const docxBuffer: Buffer = exp.body as Buffer;
|
||||
assert.strictEqual(docxBuffer.slice(0, 4).toString('latin1'), 'PK\x03\x04');
|
||||
await fs.writeFile(tmpFile, docxBuffer);
|
||||
|
||||
try {
|
||||
const imp = await agent
|
||||
.post(`/p/${dstPadId}/import`)
|
||||
.attach('file', tmpFile)
|
||||
.expect(200);
|
||||
assert.strictEqual(imp.body.code, 0,
|
||||
`import failed: ${JSON.stringify(imp.body)}`);
|
||||
|
||||
const dstPad = await padManager.getPad(dstPadId);
|
||||
const dstText = dstPad.text();
|
||||
assert.match(dstText, /Line one/);
|
||||
assert.match(dstText, /Line two/);
|
||||
assert.match(dstText, /After the blank line/);
|
||||
} finally {
|
||||
await fs.unlink(tmpFile).catch(() => undefined);
|
||||
}
|
||||
});
|
||||
|
||||
// Bidirectional round-trip: export from src pad, import into dst pad,
|
||||
// export again. Compare exhibit (a) to exhibit (c). For text-based
|
||||
// formats (txt, etherpad) this is straight byte equality. For HTML
|
||||
// and DOCX we compare the relevant invariants (line text, paragraph
|
||||
// count) since whitespace and metadata can differ between exports.
|
||||
const exportPad = async (padId: string, type: string): Promise<Buffer> => {
|
||||
const r = await fetchBuffer(agent.get(`/p/${padId}/export/${type}`))
|
||||
.expect(200);
|
||||
return r.body as Buffer;
|
||||
};
|
||||
|
||||
const importToPad = async (padId: string, content: Buffer, ext: string) => {
|
||||
try { await padManager.removePad(padId); } catch { /* noop */ }
|
||||
const tmp = path.join(os.tmpdir(),
|
||||
`roundtrip-${process.pid}-${Date.now()}.${ext}`);
|
||||
await fs.writeFile(tmp, content);
|
||||
try {
|
||||
const r = await agent.post(`/p/${padId}/import`)
|
||||
.attach('file', tmp).expect(200);
|
||||
assert.strictEqual(r.body.code, 0,
|
||||
`${ext} import failed: ${JSON.stringify(r.body)}`);
|
||||
} finally {
|
||||
await fs.unlink(tmp).catch(() => undefined);
|
||||
}
|
||||
};
|
||||
|
||||
const seedPad = async (padId: string, text: string) => {
|
||||
try { await padManager.removePad(padId); } catch { /* noop */ }
|
||||
const pad = await padManager.getPad(padId, '\n');
|
||||
await pad.setText(text);
|
||||
};
|
||||
|
||||
const SAMPLE_TEXT = 'Line one\nLine two\n\nAfter blank\n';
|
||||
|
||||
it('a==c round-trip: txt export -> import -> export', async function () {
|
||||
const src = 'test7538RtTxtSrc';
|
||||
const dst = 'test7538RtTxtDst';
|
||||
await seedPad(src, SAMPLE_TEXT);
|
||||
const a = await exportPad(src, 'txt');
|
||||
await importToPad(dst, a, 'txt');
|
||||
const c = await exportPad(dst, 'txt');
|
||||
assert.strictEqual(c.toString('utf8'), a.toString('utf8'),
|
||||
`txt round-trip drift\nA:${JSON.stringify(a.toString('utf8'))}\nC:${JSON.stringify(c.toString('utf8'))}`);
|
||||
});
|
||||
|
||||
it('a==c round-trip: etherpad export -> import -> export', async function () {
|
||||
const src = 'test7538RtEpadSrc';
|
||||
const dst = 'test7538RtEpadDst';
|
||||
await seedPad(src, SAMPLE_TEXT);
|
||||
const a = await exportPad(src, 'etherpad');
|
||||
await importToPad(dst, a, 'etherpad');
|
||||
const c = await exportPad(dst, 'etherpad');
|
||||
// etherpad format is JSON metadata + atext; the surrounding metadata
|
||||
// (timestamps, ids) differs across pads. Assert the line content
|
||||
// matches by parsing pad text from both pads.
|
||||
const srcText = (await padManager.getPad(src)).text();
|
||||
const dstText = (await padManager.getPad(dst)).text();
|
||||
assert.strictEqual(dstText, srcText,
|
||||
`etherpad round-trip drift\nsrc:${JSON.stringify(srcText)}\ndst:${JSON.stringify(dstText)}`);
|
||||
assert.ok(a.length > 0 && c.length > 0,
|
||||
'expected non-empty etherpad bodies');
|
||||
});
|
||||
|
||||
it('a==c round-trip: html export -> import -> export', async function () {
|
||||
const src = 'test7538RtHtmlSrc';
|
||||
const dst = 'test7538RtHtmlDst';
|
||||
await seedPad(src, SAMPLE_TEXT);
|
||||
const a = (await exportPad(src, 'html')).toString('utf8');
|
||||
await importToPad(dst, Buffer.from(a, 'utf8'), 'html');
|
||||
const c = (await exportPad(dst, 'html')).toString('utf8');
|
||||
// Strip <head> (skin-versioned hashes), trim trailing whitespace,
|
||||
// and trim trailing <br>s — etherpad's setPadHTML appends an empty
|
||||
// <p> on import to keep a caret below the last line, which adds
|
||||
// exactly one trailing newline per round-trip. That's pre-existing
|
||||
// core behavior, so the meaningful invariant is "content lines
|
||||
// match" with the trailing newline tolerated.
|
||||
const bodyOf = (s: string) =>
|
||||
(s.match(/<body>([\s\S]*?)<\/body>/i)?.[1] ?? '')
|
||||
.replace(/(?:<br\s*\/?>\s*)+$/i, '')
|
||||
.trim();
|
||||
assert.strictEqual(bodyOf(c), bodyOf(a),
|
||||
`html body drift\nA:${JSON.stringify(bodyOf(a))}\nC:${JSON.stringify(bodyOf(c))}`);
|
||||
});
|
||||
|
||||
it('a==c round-trip: docx export -> import -> export (line text)',
|
||||
async function () {
|
||||
const src = 'test7538RtDocxSrc';
|
||||
const dst = 'test7538RtDocxDst';
|
||||
await seedPad(src, SAMPLE_TEXT);
|
||||
const a = await exportPad(src, 'docx');
|
||||
await importToPad(dst, a, 'docx');
|
||||
// Compare pad text, not docx bytes -- DOCX includes timestamps
|
||||
// and pad ID metadata in the document properties so byte equality
|
||||
// is impossible. Pad text equality is the right invariant.
|
||||
const srcText = (await padManager.getPad(src)).text();
|
||||
const dstText = (await padManager.getPad(dst)).text();
|
||||
// setPadHTML appends a trailing newline on import, so dst is
|
||||
// expected to be src plus one trailing '\n'.
|
||||
const srcNorm = srcText.replace(/\n+$/, '\n');
|
||||
const dstNorm = dstText.replace(/\n+$/, '\n');
|
||||
assert.strictEqual(dstNorm, srcNorm,
|
||||
`docx round-trip drift\nsrc:${JSON.stringify(srcText)}\ndst:${JSON.stringify(dstText)}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('HTML import — adjacent headings (#7538)', function () {
|
||||
before(async function () {
|
||||
// These tests assume ep_headings2 (or another plugin) registers
|
||||
// h1/h2/etc. as server-side block elements via
|
||||
// `ccRegisterBlockElements`. Without that hook, contentcollector
|
||||
// treats <h1>/<h2> as inline and adjacent ones merge into a
|
||||
// single pad line — making the assertions below moot. The CI
|
||||
// backend-tests job runs without plugins installed, so skip
|
||||
// there. Local dev with ep_headings2 installed exercises these.
|
||||
const hooks = require('../../../static/js/pluginfw/hooks');
|
||||
const ccBlockElems: string[] = ([] as string[]).concat(
|
||||
...(hooks.callAll('ccRegisterBlockElements') || []));
|
||||
const headingsAreBlocks = ccBlockElems.map((t: string) => t.toLowerCase())
|
||||
.includes('h1');
|
||||
if (!headingsAreBlocks) {
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
settings.soffice = null;
|
||||
});
|
||||
|
||||
const importHtml = async (padId: string, html: string) => {
|
||||
try { await padManager.removePad(padId); } catch { /* noop */ }
|
||||
const tmp = path.join(os.tmpdir(),
|
||||
`htmlimport-${process.pid}-${Date.now()}.html`);
|
||||
await fs.writeFile(tmp, html);
|
||||
try {
|
||||
const r = await agent.post(`/p/${padId}/import`)
|
||||
.attach('file', tmp).expect(200);
|
||||
assert.strictEqual(r.body.code, 0,
|
||||
`import failed: ${JSON.stringify(r.body)}`);
|
||||
} finally {
|
||||
await fs.unlink(tmp).catch(() => undefined);
|
||||
}
|
||||
};
|
||||
|
||||
it('does not introduce a blank line between H1 and H2', async function () {
|
||||
const padId = 'test7538HtmlH1H2';
|
||||
await importHtml(padId, '<html><body><h1>A</h1><h2>B</h2></body></html>');
|
||||
const pad = await padManager.getPad(padId);
|
||||
const lines = (pad.text().split('\n') as string[]).filter(
|
||||
(l: string, i: number, arr: string[]) =>
|
||||
// ignore the trailing blank line setPadHTML always appends
|
||||
!(l === '' && i === arr.length - 1));
|
||||
// We want exactly two content lines (A and B), no blank line
|
||||
// injected between them.
|
||||
const meaningful = lines.filter((l: string) => l.trim().length > 0);
|
||||
assert.deepStrictEqual(meaningful.length, 2,
|
||||
`expected 2 content lines, got: ${JSON.stringify(lines)}`);
|
||||
const between = lines.slice(
|
||||
lines.findIndex((l: string) => l.includes('A')) + 1,
|
||||
lines.findIndex((l: string) => l.includes('B')));
|
||||
assert.deepStrictEqual(between, [],
|
||||
`expected no blank line between A and B, got: ${JSON.stringify(between)}`);
|
||||
});
|
||||
|
||||
// Reproduce the realistic export-side shape: H1 + two blank pad lines
|
||||
// (encoded by ep_align as `<br><p style></p><br><p style></p><br>`)
|
||||
// + H2. The pad should round-trip back to H1, blank, blank, H2 -- not
|
||||
// gain or lose blank lines.
|
||||
it('preserves blank-line count between H1 and H2 (realistic shape)', async function () {
|
||||
const padId = 'test7538HtmlBlankLines';
|
||||
const html =
|
||||
'<html><body>' +
|
||||
"<h1 style='text-align:right'>A</h1><br>" +
|
||||
"<p style='text-align:right'></p><br>" +
|
||||
"<p style='text-align:right'></p><br>" +
|
||||
"<h2 style='text-align:center'>B</h2><br>" +
|
||||
'</body></html>';
|
||||
await importHtml(padId, html);
|
||||
const pad = await padManager.getPad(padId);
|
||||
const lines: string[] = pad.text().split('\n');
|
||||
// Drop the trailing-newline appended by setPadHTML on import.
|
||||
while (lines.length > 0 && lines[lines.length - 1] === '') lines.pop();
|
||||
// Expect: A, '', '', B
|
||||
const aIdx = lines.findIndex((l: string) => l.includes('A'));
|
||||
const bIdx = lines.findIndex((l: string) => l.includes('B'));
|
||||
assert.notStrictEqual(aIdx, -1, `expected A: ${JSON.stringify(lines)}`);
|
||||
assert.notStrictEqual(bIdx, -1, `expected B: ${JSON.stringify(lines)}`);
|
||||
const blankCount = bIdx - aIdx - 1;
|
||||
assert.strictEqual(blankCount, 2,
|
||||
`expected 2 blank lines between A and B, got ${blankCount}: ${JSON.stringify(lines)}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Round-trip integrity: heading-style content (#7538)', function () {
|
||||
before(function () {
|
||||
try {
|
||||
require.resolve('html-to-docx');
|
||||
require.resolve('mammoth');
|
||||
} catch {
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
settings.soffice = null;
|
||||
});
|
||||
|
||||
const fetchBuffer = (req: any): any => req
|
||||
.buffer(true)
|
||||
.parse((resp: any, cb: any) => {
|
||||
const chunks: Buffer[] = [];
|
||||
resp.on('data', (c: Buffer) => chunks.push(c));
|
||||
resp.on('end', () => cb(null, Buffer.concat(chunks)));
|
||||
});
|
||||
|
||||
it('keeps adjacent heading-style blocks on separate lines after round-trip', async function () {
|
||||
// Regression: ep_headings2 emits <h1>/<h2>/<code> that aren't in
|
||||
// contentcollector's default block-element set. Without the
|
||||
// separateAdjacentHeadingBlocks fix, mammoth's <h1>A</h1><h2>B</h2>
|
||||
// would merge into one pad line.
|
||||
const srcPadId = 'test7538MultiHeading';
|
||||
const dstPadId = 'test7538MultiHeadingImport';
|
||||
const tmpFile = path.join(os.tmpdir(), `multiheading-${process.pid}.docx`);
|
||||
|
||||
try {
|
||||
await padManager.removePad(srcPadId);
|
||||
await padManager.removePad(dstPadId);
|
||||
} catch { /* noop */ }
|
||||
|
||||
// Drive the import path directly with a hand-crafted DOCX whose
|
||||
// content is just three adjacent block elements; this is what
|
||||
// mammoth produces from the round-trip output of ep_headings2's
|
||||
// pad HTML.
|
||||
const htmlToDocx = require('html-to-docx');
|
||||
const buf: Buffer = await htmlToDocx(
|
||||
'<h1>Welcome</h1><h2>This pad</h2><p>Code line</p>');
|
||||
await fs.writeFile(tmpFile, buf);
|
||||
|
||||
try {
|
||||
await agent.post(`/p/${dstPadId}/import`)
|
||||
.attach('file', tmpFile)
|
||||
.expect(200);
|
||||
const dstPad = await padManager.getPad(dstPadId);
|
||||
const lines = dstPad.text().split('\n');
|
||||
// Each block must land on its own line (lines may carry an
|
||||
// etherpad heading marker prefix like '*' from ep_headings2).
|
||||
const findLine = (needle: string) =>
|
||||
lines.findIndex((l: string) => l.includes(needle));
|
||||
const iWelcome = findLine('Welcome');
|
||||
const iThisPad = findLine('This pad');
|
||||
const iCode = findLine('Code line');
|
||||
assert.notStrictEqual(iWelcome, -1,
|
||||
`expected "Welcome" on its own line: ${JSON.stringify(lines)}`);
|
||||
assert.notStrictEqual(iThisPad, -1,
|
||||
`expected "This pad" on its own line: ${JSON.stringify(lines)}`);
|
||||
assert.notStrictEqual(iCode, -1,
|
||||
`expected "Code line" on its own line: ${JSON.stringify(lines)}`);
|
||||
assert.ok(iWelcome !== iThisPad && iThisPad !== iCode,
|
||||
`headings/code merged into the same line: ${JSON.stringify(lines)}`);
|
||||
} finally {
|
||||
await fs.unlink(tmpFile).catch(() => undefined);
|
||||
}
|
||||
});
|
||||
|
||||
it('preserves text content through native PDF export (sanity check)', async function () {
|
||||
// PDF round-trip is one-way (no native PDF import) -- this just
|
||||
// verifies the exported PDF has the source text in its visible
|
||||
// content stream, so we know nothing got dropped on export.
|
||||
const padId = 'test7538PdfSanity';
|
||||
try { await padManager.removePad(padId); } catch { /* noop */ }
|
||||
const pad = await padManager.getPad(padId, '\n');
|
||||
await pad.setText('Hello PDF\nSecond line\n');
|
||||
|
||||
const exp = await fetchBuffer(agent.get(`/p/${padId}/export/pdf`))
|
||||
.expect(200);
|
||||
const pdf: Buffer = exp.body as Buffer;
|
||||
assert.strictEqual(pdf.slice(0, 5).toString('ascii'), '%PDF-');
|
||||
|
||||
// pdfkit emits text as hex strings inside TJ ops; concat them and
|
||||
// search the visible content.
|
||||
const ascii = pdf.toString('latin1');
|
||||
const visible: string[] = [];
|
||||
const re = /<([0-9a-fA-F]{2,})>/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(ascii)) !== null) {
|
||||
visible.push(Buffer.from(m[1], 'hex').toString('latin1'));
|
||||
}
|
||||
const concatenated = visible.join('');
|
||||
assert.ok(concatenated.includes('Hello PDF'),
|
||||
`expected "Hello PDF" in PDF visible content: ${concatenated}`);
|
||||
assert.ok(concatenated.includes('Second line'),
|
||||
`expected "Second line" in PDF visible content: ${concatenated}`);
|
||||
});
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue