Commit graph

11 commits

Author SHA1 Message Date
Johannes Millan
99f923ad6c
docs(sync): fix stale schema-compat docs, add bump policy, rescope Task 6 (#9119)
* docs(sync): fix stale schema docs, add bump policy, rescope Task 6

* docs(sync): add severity-triage and schema-bump rules to AGENTS.md

* docs(sync): fix review findings in schema compat docs
2026-07-17 15:37:06 +02:00
Johannes Millan
8e810edbe7
fix(sync): make marked project deletions win LWW conflicts (#9009)
deleteProject cascade-deletes a project's tasks, notes, sections, repeat
config, and archive data in one reducer pass. When that op lost an LWW
conflict to a concurrent project edit, only the PROJECT entity was
reversed: every client resurrected an empty project and the winning
client's status-blind hydration replay cascaded its tasks away after a
restart (live state != post-restart replay).

Rather than recreate every cascaded entity (payload scales with project
size and cannot restore every side effect safely), give schema-v4
deleteProject operations explicit delete-wins precedence:

- new deleteProject actions carry a shared PROJECT_DELETE_WINS_MARKER; the
  shared LWW planner accepts a host-supplied delete-wins classifier. A
  marked remote delete is applied regardless of timestamps; a marked local
  delete is replaced with one op whose vector clock dominates both sides.
- historical unmarked (schema-v3) deletions keep timestamp-based LWW; the
  absence of the marker (never added by the no-op v3->v4 migration) is the
  real discriminator, and a schema v3->v4 barrier (mirroring v2->v3) makes
  older clients block on the newer-schema gate instead of mis-resolving.

Delete-wins plans reuse the archive-win resolution pipeline, so they
inherit its atomic persistence and losing-op rejection, and disjoint
merge leaves them untouched (the delete must win the whole entity).

Hardening from multi-agent review:

- union allTaskIds/noteIds across multiple concurrent marked deletes for
  the same project, so a single replacement cannot leave orphan tasks on
  clients that only receive it (the task reducer removes by allTaskIds).
- gate the classifier on the AUTHENTICATED payload projectId matching the
  plaintext entityId, so a tampered/replayed delete retargeted onto a live
  entity cannot silently drop a concurrent edit.
- guard a null/undefined delete payload in the classifier instead of
  throwing and wedging the conflict pass.
- pin the server's legacy-misc conflict alias to the fixed v1->v2 split
  boundary, not CURRENT_SCHEMA_VERSION, so this bump does not fabricate
  false GLOBAL_CONFIG:misc/tasks conflicts during rollout.
- bind the marker with a shared const (compiler-checked on producer and
  consumer) and rename _isArchivePlan -> _isWholeEntityWinPlan.

Documents the policy as ARCHITECTURE-DECISIONS.md #7.

Addresses #8997.
2026-07-14 19:58:33 +02:00
Johannes Millan
51bf689bd5
fix(sync): preserve multi-entity conflict recovery (#8990)
* chore: add project-scoped Angular MCP

* chore: update npm for release-age policy

* fix(sync): preserve LWW outcomes across clients

Distinguish replacement snapshots from partial merge operations, protect device-local sync configuration, recreate winning deletes, and resolve every conflicted entity in bulk operations.

Fixes #8956

* fix(sync): harden mixed LWW conflict replay

Preserve unaffected remote and local bulk intents, keep device-local sync settings during replacement replay, and gate replacement semantics behind schema v3.

* fix(sync): recover subtask subtree and harden LWW replay follow-ups

Follow-up hardening for #8956 after multi-agent review:

- Recreate a locally-winning parent's subtasks when a remote bulk delete
  is a mixed winner. The full remote delete is applied (cascade-deleting
  the parent's subtasks via handleDeleteTasks) but only the parent had a
  compensation op, so the subtree was silently lost across devices.
- extractUpdateChanges: scan array-valued payload props instead of guessing
  `${payloadKey}s`, so irregular bulk keys (e.g. taskUpdates) no longer
  return {} and drop a remote winner's changes.
- Degrade gracefully instead of throwing when a remote update wins over a
  local delete with no reconstructable base entity, matching the
  single-entity path (a permanent sync wedge is worse than the bounded
  divergence it already accepts).
- Remove dead code (unused `deleting` set + zero-caller wrapper), use the
  Set-based scoped-bulk-delete filter, type meta via LwwUpdateMode, and
  restore the withLocalOnlySyncSettings rationale comment.

Adds regression tests for the subtree-recovery and irregular-bulk-key paths.

* fix(sync): preserve conflict outcomes during replay

Persist replacement LWW operations and bulk-delete snapshots so reconstructed state matches the result applied live. Bump the op-log DB version to prevent older clients from opening the incompatible schema.

* fix(sync): persist conflict outcomes atomically

Write remote losers, local compensations, and final remote winners in one IndexedDB transaction so crashes cannot expose a partial replay order. Add real-store coverage for live/replay equivalence and transaction rollback.

* fix(sync): preserve multi-entity conflict recovery

* fix(sync): close review gaps in multi-entity conflict recovery

- recreate a winning parent's subtasks when a remote DELETE loses
  outright (single-entity or all-local-win bulk), so clients that
  applied the delete and status-blind hydration replay converge
- apply the combined resolution batch in durable seq order so a
  pending row reused from a prior failed attempt replays identically
  live and after a crash
- restamp converted remote updates carrying the v3 replacement
  envelope to the current schema version
- strip the virtual TODAY tag from LWW task payloads and shallow-merge
  patch-mode singleton payloads instead of replacing feature state
- pin the server snapshot fast-path spec to CURRENT_SCHEMA_VERSION
  (fixes the CI failure from the v2-to-v3 bump)

* test(sync): pin outright-losing delete convergence across clients

Three-way real-reducer/real-store convergence for the pure-loser path
(live == restart replay == originating client), covering both the
same-batch recreate exemption and the cross-batch recreate path.
Verified to fail against the pre-fix service.
2026-07-14 14:10:52 +02:00
Johannes Millan
eecf33a4e8 fix(sync): remediate multi-agent review findings
- USE_REMOTE: fix guaranteed self-deadlock — flushPendingWrites() was
  called while already holding the non-reentrant sp_op_log lock its own
  Phase 2 re-acquires, so every "Use Server Data" hung 30s and aborted.
  New OperationWriteFlushService.flushThenRunExclusive owns the bounded
  flush->lock->recheck barrier; ServerMigrationService reuses it, which
  also bounds its previously unbounded snapshot-cutoff retry loop.
- USE_REMOTE: make the rebuild crash-resumable. A raw_rebuild_incomplete
  META marker is set atomically with the baseline replacement and cleared
  after the replay commits; the next sync redoes the raw rebuild instead
  of the normal download (which excludes own ops server-side and would
  silently lose the un-replayed suffix). The resume keeps the first
  attempt's pre-replace backup instead of overwriting the single slot
  with the partial baseline.
- Deferred actions: serialize overlapping drains (two concurrent drains
  could mint two ops for one user intent), stop the drain at the first
  transient failure instead of persisting successors out of order (the
  older edit would win LWW everywhere via clock inversion), abandon
  permanently invalid actions after one attempt (typed
  PermanentDeferredWriteError) instead of retrying + snacking on every
  sync forever, and latch the buffer-size warnings to once per stuck
  window (previously a blocking dev dialog per action past 100); drop
  the no-op 5000 "hard cap" tier.
- writeOperation: post-append bookkeeping failures no longer propagate
  into the deferred retry loop, which re-appended the same action under
  a fresh op id (double-apply of additive payloads).
- remote-apply: full-state cleanup retains every full-state op of the
  current batch, so an archive_pending import can no longer be deleted
  before its archive retry (startup replay would lose a change already
  visible at runtime).
- Hydration: sanitize a malformed stored schemaVersion per-op instead of
  failing the whole boot into recovery; strict parsing (floor now 1,
  matching the server contract) stays on the receive/upload paths.
- Server: request fingerprints are computed lazily — only when a dedup
  entry exists, and otherwise after the rate-limit/pre-quota gates but
  before uploadOps mutates the parsed ops. Fixes the pre-gate hashing of
  full payloads (authenticated DoS-cost regression) and repairs three
  sync-fixes retry tests to model true identical-body retries.
- Delete dead code: failed-op-ids util (+spec) and MAX_VERSION_SKIP (the
  removed forward-compat band); fix the stale clock-merge parity comment.

sync-core 210/210, shared-schema 50/50, super-sync-server 825/825, and
all touched Angular specs pass.
2026-07-10 16:39:37 +02:00
Ivan Kalashnikov
b3da4e4850 fix: update CURRENT_SCHEMA_VERSION to 2 for upcoming migration 2026-01-19 13:16:48 +07:00
Ivan Kalashnikov
5002bae1c0 fix: add todo comment to bump CURRENT_SCHEMA_VERSION for upcoming migration 2026-01-19 01:09:00 +07:00
Ivan Kalashnikov
325e24f461 fix: revert CURRENT_SCHEMA_VERSION to 1 2026-01-19 00:03:33 +07:00
Ivan Kalashnikov
f2940fd7ae fix: remove outdated todo comment regarding schema version synchronization 2026-01-18 23:07:12 +07:00
Ivan Kalashnikov
bd2615e7d7 fix: update migration versions from 16 to 1 and 17 to 2 for consistency 2026-01-18 21:28:43 +07:00
Ivan Kalashnikov
92d7d4aafe fix: update CURRENT_SCHEMA_VERSION to 17 for new migrations 2026-01-18 21:08:31 +07:00
Johannes Millan
8dc8207da2 feat(sync): add shared schema versioning package for frontend/backend
Create @sp/shared-schema package with pure TypeScript migration functions
that work in both Angular frontend and Node.js backend environments.

Package contents:
- schema-version.ts: Version constants (CURRENT=1, MAX_SKIP=3)
- migration.types.ts: OperationLike, SchemaMigration interfaces
- migrate.ts: Pure functions (migrateState, migrateOperation, etc.)
- migrations/index.ts: Empty migrations array (ready for first migration)
- 22 unit tests covering all migration scenarios

Backend changes:
- Add snapshot_schema_version column to user_sync_state table
- Migrate snapshots during generateSnapshot if outdated
- Migrate operations during replayOpsToState if outdated
- Drop operations that return null from migration (removed features)

Frontend changes:
- Refactor SchemaMigrationService to use @sp/shared-schema
- Re-export constants for backwards compatibility
- All 20 existing tests pass

This enables coordinated schema migrations across client and server,
ensuring old snapshots and operations can be upgraded when the state
structure changes.

Rollout strategy: Deploy backend first, then frontend.
2025-12-12 20:47:44 +01:00