fix(editor): preserve U+00A0 non-breaking space (#3037) (#7585)

* fix(editor): preserve U+00A0 non-breaking space (#3037)

Non-breaking spaces were silently normalized to regular spaces at every
ingestion point, so typed/pasted/imported nbsps never reached the
changeset and users could not glue words against line-wrap in French or
other languages that require nbsp typography.

Removed the four strip sites that replaced U+00A0 with U+0020:
  - src/node/db/Pad.ts cleanText
  - src/static/js/contentcollector.ts textify
  - src/static/js/ace2_inner.ts textify
  - src/static/js/ace2_inner.ts importText raw-text guard

Updated both processSpaces functions (domline and ExportHtml) to tokenize
U+00A0 as a separate unit, emit it verbatim as  , and treat it as
content (not whitespace) for the run-collapse bookkeeping so adjacent
regular-space runs aren't miscounted.

Added backend round-trip tests for spliceText and setText, and extended
the cleanText case table. Updated the existing contentcollector and
importexport specs whose expectations encoded the previous buggy
behavior; they now assert genuine nbsp preservation.

Verified manually in Firefox: clipboard U+00A0 → paste → pad → getText
returns c2 a0; getHTML emits `100 km`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(contentcollector): collapse display-artifact nbsp runs on DOM read-back

processSpaces is a lossy one-way display transform: leading/trailing
spaces and all-but-the-last of a run get rendered as &nbsp; so HTML
doesn't collapse them. When incorporateUserChanges reads text back from
the DOM, those display-artifact nbsps were being stored in the changeset
model instead of being normalized back to plain spaces.

This broke handleReturnIndentation, whose /^ *(?:)/ regex only matches
ASCII spaces: auto-indent after `foo:\n` produced 4 spaces instead of
the expected prev-indent (2) + THE_TAB (4) = 6, because the previous
line's model had nbsps where it used to have spaces.

Fix: in contentcollector.textify, collapse any [  ]+ run back to
plain spaces UNLESS the run is pure U+00A0 AND strictly interior to
word chars. That preserves user-intended typographic nbsps like
"100 km" while undoing the one-way display transform.

Updated 7 contentcollector tests and 7 importexport tests whose
assertions needed to reflect the new rule (boundary/mixed runs collapse;
pure-interior nbsp runs preserve).

Fixes the Playwright regression in indentation.spec.ts:117 that the
previous commit introduced.

* fix(contentcollector): canonicalize nbsp runs at line assembly, not per text node

Addresses Qodo code review feedback on PR #7585.

## Bug fix — nbsp lost at DOM text-node boundary

The previous approach ran the "collapse display-artifact nbsp" rule inside
textify(), which is called per individual DOM TEXT_NODE. A user-intended
nbsp sitting at a text-node boundary (e.g., <span>100</span><span>&nbsp;km
</span>) was incorrectly seen as non-interior (before === '' for the second
text node) and normalized back to a regular space.

Fix: move the canonicalization out of textify() and run it on each
fully assembled line string inside cc.finish(). The rule remains:

    [  ]+ run  ->  plain spaces
                   UNLESS pure U+00A0 AND strictly interior to non-ws chars

It is length-preserving, so attribute offsets and line lengths are
unaffected.

Added a regression test (contentcollector.spec.ts) for the cross-span
case.

## Docs concern

Reverted the type-only addition of spliceText to PadType. spliceText
is an existing Pad runtime method; the backend test now uses a cast
(`(pad as any).spliceText`) so the PR does not expand the declared
public type surface, avoiding a separate documentation requirement.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
John McLear 2026-04-23 08:57:30 +01:00 committed by GitHub
parent de5feb2eb5
commit 67e542d2b9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 85 additions and 17 deletions

View file

@ -537,13 +537,16 @@ const _processSpaces = (s: string) => {
const doesWrap = true;
if (s.indexOf('<') < 0 && !doesWrap) {
// short-cut
return s.replace(/ /g, '&nbsp;');
return s.replace(/[ \u00a0]/g, '&nbsp;');
}
const parts = [];
s.replace(/<[^>]*>?| |[^ <]+/g, (m) => {
const parts: string[] = [];
s.replace(/<[^>]*>?|[ \u00a0]|[^ \u00a0<]+/g, (m) => {
parts.push(m);
return m
});
// U+00A0 is content for run-bookkeeping - it terminates a space run
// just like a word character would, so runs of regular spaces adjacent
// to a nbsp are not miscounted as one long run (issue #3037).
if (doesWrap) {
let endOfLine = true;
let beforeSpace = false;
@ -578,6 +581,9 @@ const _processSpaces = (s: string) => {
}
}
}
for (let i = 0; i < parts.length; i++) {
if (parts[i] === '\u00a0') parts[i] = '&nbsp;';
}
return parts.join('');
};