Commit graph

5 commits

Author SHA1 Message Date
Johannes Millan
8f274582e2
feat(plugins): Stage A keyed persistence with LWW (#7749) (#7763)
* feat(plugins): keyed persistence API for per-context LWW (Stage A Phase 1+3)

Add an optional `key` argument to `persistDataSynced` / `loadSyncedData`,
composed at the bridge transport boundary into `pluginId:key` entity ids.
Distinct keys now produce distinct ops that LWW-resolve per-entity,
enabling document-mode-style plugins to avoid cross-context blob
overwrites without changing existing keyless callers.

Phase 3: `removePluginUserData(pluginId)` now sweeps the full prefix
(legacy entry + every keyed entry), dispatching one delete per match
with the rule-6 setTimeout(0) trailer so remote replicas don't keep
keyed entries after uninstall. The reducer-only "smart prefix match"
shortcut is wrong (one op for the prefix only, remote keyed entries
leak) — see docs/plans/2026-05-23-stage-a-keyed-plugin-persistence.md
Phase 3.

Phase 4 (document-mode plugin-side migration of the legacy single-blob
entry) is left as a separate follow-up so the host change can be
reviewed in isolation.

Issue #7749

* feat(document-mode): migrate to keyed persistence (Stage A Phase 4)

Move from one synced blob under the bare plugin id to per-entity keyed
entries:
 - meta            — { enabledCtxIds: string[] }, owned by background.ts
 - doc:${ctxId}    — one entry per context, owned by the editor iframe
 - __meta__        — migration stamp

Each entry has its own LWW timestamp on the host, so a concurrent edit
in project A on Device 1 and project B on Device 2 no longer
whole-blob-collide.

The migration runs idempotently from both background.ts and editor.ts
(stamp-guarded), splits the legacy single blob into keyed entries, then
tombstones the legacy entry with an empty payload — giving LWW a
winning side against any offline device that still writes the old
shape.

flushSave / flushSaveSync no longer need to read+merge sibling state,
since each context's entry stands alone. The future-version blob guard
(isStorageUnreadable) is dropped — it referenced the wrapping blob's
version, which no longer exists; per-doc corruption still falls back
via isDocCorrupt.

Issue #7749

* fix(plugins): tighten Stage A keyspace at the boundaries

Multi-review surfaced three small gaps in the keyed-persistence rollout:

- The synchronous composeId throw covers the bridge's iframe and
  direct-API entry points, but three in-process callers
  (plugin-config.service, plugin.service, plugin-config-dialog) bypass
  the bridge and route directly into the persistence service. A
  user-installed plugin with `id: "evil:plugin"` passed manifest
  validation and would have collided with the legitimate `evil`
  plugin's keyed namespace — `removePluginUserData('evil')` would have
  over-matched the sweep. Reject the colon at install time in
  `validatePluginManifest`; keep the bridge throw as defense-in-depth.

- The new `key` arg at the bridge was typia-asserted on `data` but
  unchecked itself. A compromised iframe could pass a multi-megabyte
  string or a non-string value via postMessage. `data` is capped at
  1 MB, but the entity id composed from `key` would be stored verbatim
  in NgRx state, IndexedDB, the op-log, and on the sync wire — bypassing
  the data cap. Add `assertPluginPersistenceKey` with a 256-char cap.

- `_loadPersistedData` silently returned `null` when composeId threw,
  while `_persistDataSynced` rethrew. The asymmetry made a malformed
  pluginId look like "no data yet" on the load side, indistinguishable
  from a fresh install. Hoist composeId + key validation out of the
  load try/catch so it throws symmetrically.

Issue #7749

* fix(plugins): lower per-write cap to 256 KB

The pre-Stage-A 1 MB cap was sized for the old single-blob shape, where
one entry held every context's data. With the keyed split, each entity
gets its own write budget — 1 MB per write is wildly over-provisioned
for the realistic upper bound of plugin payloads (heavy document-mode
docs ~30–100 KB, configs and automations KB-scale).

256 KB keeps 2–5× headroom over realistic payloads while bounding the
per-plugin storage growth more tightly.

Document-mode's migration loop now skips oversized legacy docs instead
of aborting the whole run: a user whose legacy blob holds one ~500 KB
doc (legal under the old cap) keeps the other contexts migrated and
the original bytes preserved in the legacy entry. The success stamp
stays at migrated:0 in that case so a future build (or pruning of the
doc) can complete the migration without data loss.

Issue #7749

* test(plugins): e2e migration of legacy single-blob to keyed entries

The migration logic in document-mode is unit-tested against a mock
PluginAPI, which can't catch real-iframe quirks (postMessage handling
of undefined second args, commit-chain timing under the host's
per-entity rate limiter, hydration ordering against the op-log). Add
two end-to-end scenarios:

- Fresh install: enable the plugin, verify the __meta__ stamp lands
  at migrated:1 (the migration's final write — observing it implies
  every earlier step completed).
- Legacy blob: seed a pre-Stage-A single-blob entry via the e2e helper
  store, enable the plugin, verify the legacy entry is tombstoned,
  meta carries the enabledCtxIds, and each doc landed under its own
  doc:${ctxId} key.

Issue #7749

* chore(plugins): drop dead code and review-driven polish

Four small follow-ups from the multi-review pass:

- Don't log the plugin-supplied `key` value. Plugins may use user
  content (search queries, doc titles) as keys; the log history is
  exportable. Log `keyLen` instead, per CLAUDE.md rule 9.
- Delete `detectStaleLegacyWrite` and its 3 specs. Exported and
  fully tested, but zero non-test callers — banner UI is forbidden
  by project convention for transient-only messaging. If the need
  resurfaces, the implementation is four lines.
- Drop the `attemptedAt` field from `MigrationStamp`. It was written
  but never read; the success stamp is the only re-entry gate, and
  the resume path is just "re-run the loop" — re-writes are content-
  idempotent. Saves one rate-limited write per fresh migration.
- Update `docs/plans/2026-05-23-stage-a-keyed-plugin-persistence.md`
  with an implementation-status table referencing the shipping
  commits, so future readers don't have to dig through git.

Issue #7749
2026-05-23 22:23:17 +02:00
Johannes Millan
196e50b906
test: strengthen unit test assertions and revive disabled plugin specs (#7755)
* chore(plugins): re-bundle document-mode and document Stage A path

Reverts the unbundling from b0cae69ffe. Stage 0 (gzip + throttle, shipped
in 84625be849) handles size; document-mode remains opt-in per context, so
cross-context conflict risk is bounded. Stage A (keyed plugin-persistence
API, issue #7749) is the documented future path for closing the LWW gap
on different-context concurrent edits — picked up when conflicts are
observed in practice.

Design sketch with multi-reviewed phasing lives in
docs/plans/2026-05-23-stage-a-keyed-plugin-persistence.md. Predecessor
plan's "Future work" section now links to it.

* test(plugins): cover document-mode bundled load and PLUGIN_USER_DATA LWW

Follow-ups from the multi-review of 199e816479's re-bundling decision:

- E2E smoke test asserts document-mode appears in plugin management so
  a typo in BUNDLED_PLUGIN_PATHS fails loudly.
- Spec exercises PLUGIN_USER_DATA conflict resolution end-to-end, which
  previously relied on analogy to REMINDER (same array+null branch) but
  was never directly asserted after the migration off 'virtual'.
- Stage A plan risks: stale-editor-view gap surfaced by the review;
  PluginHooks.PERSISTED_DATA_UPDATE already exists in the API but is
  never dispatched host-side — wiring it is the path to a fix.
- background.ts: comment marks the known gap at the registerHook site.

* docs(plugins): plan for wiring PERSISTED_DATA_CHANGED hook

Designs the host-side wiring for the currently-dead
PluginHooks.PERSISTED_DATA_CHANGED so plugins can react to remote-driven
changes to their persisted data. Multi-reviewed twice; v4 trims scope to
host-only (no plugin adoption in this design) and preserves the
multi-review insights as seeds for the follow-up doc-mode adoption
tracked at issue #7752.

Implementation lands in a separate PR.

* test: strengthen unit test assertions and revive disabled plugin specs

Replace tautological assertions, setTimeout-without-expect patterns, and
placeholder `expect(true).toBe(true)` tests with real assertions across
~30 spec files. Revive 5 plugin spec files that were fully commented out
on master with live tests covering core behavior.

Production-side: extract pure helpers for testability:
- app.component: getBackgroundOverlayOpacity, getBackgroundImageBlur
- android-sync-bridge.effects: getSuperSyncCredentialBridgeCommand
- super-sync-server: export escapeHtml, SERVER_HELMET_CONFIG

Delete the always-skipped xdescribe placeholder
src/app/imex/sync/sync-fixes.spec.ts (412 LOC).
Rename operation-log-stress.spec.ts to .benchmark.ts to match its
header comment ("excluded from regular test runs").

No production behavior changes; no master commits reverted.
2026-05-23 18:31:53 +02:00
Johannes Millan
0c2420eeb3 feat(plugin): improve sync data size for plugins
- docs(plugins): remove delta-sync plan; track Stage A as GH issue
- fix(plugins): preserve teardown safety net + earlier base64 size gate
- fix(plugins): address second-round multi-review findings
- refactor(sync): drop unused isVirtualEntity host re-export
- fix(plugins): address multi-review findings in plugin persistence
- perf(plugins): delegate plugin-data codec to sync-core helpers, cap decompression
- docs(sync): mark delta-sync plan implementation status
- perf(plugins): gzip plugin user data at the persistence boundary
- perf(document-mode): raise save throttle and add focus-loss flush triggers
- fix(sync): resolve plugin-data LWW conflicts via array storagePattern
2026-05-23 16:36:04 +02:00
Johannes Millan
69b1cb2877 perf(document-mode): strip redundant chip content from synced data
The document-mode plugin persisted each taskRef/subTaskRef chip with its task title as inline content plus an isDone attr. Both are re-derived from the host task cache on load (migrateStoredDoc backfills content, refreshChipContentFromCache overwrites both), so storing them only inflates the synced blob -- and the title is its byte-heavy part.

Add a pure stripChipContent() that collapses every chip to a bare identity atom { type, attrs: { taskId } }, and apply it in flushSave / flushSaveSync before serializing. The live editor document keeps its inline content, so title write-back is unaffected; only the persisted copy shrinks. A bare-atom chip loads through the existing unchanged pipeline, so no schema bump or migration is needed. Roughly halves the per-change sync payload.
2026-05-22 20:21:19 +02:00
Johannes Millan
1c10ff67dd feat(document-mode): add TipTap-based document-mode plugin
A document-mode plugin under packages/plugin-dev/document-mode/ that
renders the active work context's tasks as an editable TipTap document.
Per-context doc state is persisted as a last-writer-wins JSON blob via
PluginAPI.persistDataSynced; task identity (title, done state, hierarchy)
stays in NgRx and is reached through PluginAPI.updateTask and the
ANY_TASK_UPDATE hook.

It registers a work-context header button and embeds itself into the
work-view embed slot added in the preceding commit. See
docs/plans/2026-05-21-document-mode-tiptap-plugin.md for the full design.

Squashed from the feat/doc-mode-v4 work; full per-step history is
preserved in the archive/doc-mode-v4-full-history branch and the
doc-mode-v4-pre-squash tag.
2026-05-22 17:33:22 +02:00