etherpad-lite/src/node/utils
John McLear 67e542d2b9
fix(editor): preserve U+00A0 non-breaking space (#3037) (#7585)
* fix(editor): preserve U+00A0 non-breaking space (#3037)

Non-breaking spaces were silently normalized to regular spaces at every
ingestion point, so typed/pasted/imported nbsps never reached the
changeset and users could not glue words against line-wrap in French or
other languages that require nbsp typography.

Removed the four strip sites that replaced U+00A0 with U+0020:
  - src/node/db/Pad.ts cleanText
  - src/static/js/contentcollector.ts textify
  - src/static/js/ace2_inner.ts textify
  - src/static/js/ace2_inner.ts importText raw-text guard

Updated both processSpaces functions (domline and ExportHtml) to tokenize
U+00A0 as a separate unit, emit it verbatim as  , and treat it as
content (not whitespace) for the run-collapse bookkeeping so adjacent
regular-space runs aren't miscounted.

Added backend round-trip tests for spliceText and setText, and extended
the cleanText case table. Updated the existing contentcollector and
importexport specs whose expectations encoded the previous buggy
behavior; they now assert genuine nbsp preservation.

Verified manually in Firefox: clipboard U+00A0 → paste → pad → getText
returns c2 a0; getHTML emits `100 km`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(contentcollector): collapse display-artifact nbsp runs on DOM read-back

processSpaces is a lossy one-way display transform: leading/trailing
spaces and all-but-the-last of a run get rendered as &nbsp; so HTML
doesn't collapse them. When incorporateUserChanges reads text back from
the DOM, those display-artifact nbsps were being stored in the changeset
model instead of being normalized back to plain spaces.

This broke handleReturnIndentation, whose /^ *(?:)/ regex only matches
ASCII spaces: auto-indent after `foo:\n` produced 4 spaces instead of
the expected prev-indent (2) + THE_TAB (4) = 6, because the previous
line's model had nbsps where it used to have spaces.

Fix: in contentcollector.textify, collapse any [  ]+ run back to
plain spaces UNLESS the run is pure U+00A0 AND strictly interior to
word chars. That preserves user-intended typographic nbsps like
"100 km" while undoing the one-way display transform.

Updated 7 contentcollector tests and 7 importexport tests whose
assertions needed to reflect the new rule (boundary/mixed runs collapse;
pure-interior nbsp runs preserve).

Fixes the Playwright regression in indentation.spec.ts:117 that the
previous commit introduced.

* fix(contentcollector): canonicalize nbsp runs at line assembly, not per text node

Addresses Qodo code review feedback on PR #7585.

## Bug fix — nbsp lost at DOM text-node boundary

The previous approach ran the "collapse display-artifact nbsp" rule inside
textify(), which is called per individual DOM TEXT_NODE. A user-intended
nbsp sitting at a text-node boundary (e.g., <span>100</span><span>&nbsp;km
</span>) was incorrectly seen as non-interior (before === '' for the second
text node) and normalized back to a regular space.

Fix: move the canonicalization out of textify() and run it on each
fully assembled line string inside cc.finish(). The rule remains:

    [  ]+ run  ->  plain spaces
                   UNLESS pure U+00A0 AND strictly interior to non-ws chars

It is length-preserving, so attribute offsets and line lengths are
unaffected.

Added a regression test (contentcollector.spec.ts) for the cross-span
case.

## Docs concern

Reverted the type-only addition of spliceText to PadType. spliceText
is an existing Pad runtime method; the backend test now uses a cast
(`(pad as any).spliceText`) so the PR does not expand the declared
public type surface, avoiding a separate documentation requirement.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 08:57:30 +01:00
..
AbsolutePaths.ts chore: migrated settings to esm6 (#7062) 2025-08-04 22:42:50 +02:00
checkValidRev.ts Added typescript to etherpad 2024-02-05 21:13:02 +01:00
Cleanup.ts chore: moved first files to esm (#7061) 2025-08-04 19:59:28 +02:00
Cli.ts chore: use args from cmd (#7112) 2025-09-03 21:01:20 +02:00
customError.ts Added typescript to etherpad 2024-02-05 21:13:02 +01:00
ExportEtherpad.ts Added typescript to etherpad 2024-02-05 21:13:02 +01:00
ExportHelper.ts Feat/changeset ts (#6594) 2024-08-18 12:14:24 +02:00
ExportHtml.ts fix(editor): preserve U+00A0 non-breaking space (#3037) (#7585) 2026-04-23 08:57:30 +01:00
ExportTxt.ts feat!: replace Abiword with LibreOffice and add DOCX export (#7539) 2026-04-19 09:08:22 +01:00
ImportEtherpad.ts Moved to ts (#6593) 2024-08-17 20:14:36 +02:00
ImportHtml.ts Feat/changeset ts (#6594) 2024-08-18 12:14:24 +02:00
LibreOffice.ts chore: migrated settings to esm6 (#7062) 2025-08-04 22:42:50 +02:00
Minify.ts chore: migrated settings to esm6 (#7062) 2025-08-04 22:42:50 +02:00
MinifyWorker.ts Feat/frontend vitest (#6469) 2024-08-16 22:55:42 +02:00
NodeVersion.ts chore: moved first files to esm (#7061) 2025-08-04 19:59:28 +02:00
padDiff.ts fix: createDiffHTML API fails with "Not a changeset: undefined" (#7445) 2026-04-04 08:25:26 +01:00
path_exists.ts Moved path_exists and promises to es6 2024-08-18 19:52:21 +02:00
promises.ts Moved path_exists and promises to es6 2024-08-18 19:52:21 +02:00
randomstring.ts chore: moved first files to esm (#7061) 2025-08-04 19:59:28 +02:00
run_cmd.ts chore: migrated settings to esm6 (#7062) 2025-08-04 22:42:50 +02:00
sanitizePathname.ts Feat/frontend vitest (#6469) 2024-08-16 22:55:42 +02:00
Settings.ts chore: Rename some occurences of etherpad-lite to etherpad (#7552) 2026-04-19 16:53:57 +02:00
SettingsTree.ts Enforce 2-space indentation across codebase (#7426) 2026-04-01 19:48:45 +01:00
Stream.ts Added typescript to etherpad 2024-02-05 21:13:02 +01:00
tar.json Changeset: New API to simplify attribute processing 2021-11-21 04:11:41 -05:00
toolbar.ts Fix ESM/CJS interop for Settings module breaking plugin compatibility (#7421) 2026-04-01 11:39:40 +01:00
UpdateCheck.ts chore: migrated settings to esm6 (#7062) 2025-08-04 22:42:50 +02:00