When a full-state op (SYNC_IMPORT/BACKUP_IMPORT) arrived from another
client, mergeRemoteOpClocks reset the local clock to a "minimal" form
but only preserved the current client's counter from the incoming op's
clock. If the current client had issued ops (e.g. GLOBAL_CONFIG) not
reflected in the incoming full-state op's clock, its counter was dropped,
causing subsequent ops to reuse the same counter value. Downstream clients
then saw these ops as EQUAL (duplicate) and skipped them silently.
Fix: take max(mergedClock[clientId], currentClock[clientId]) when
rebuilding the clock after a full-state op reset.
Also add __SP_E2E_BLOCK_WS_DOWNLOAD flag to WsTriggeredDownloadService
to allow E2E tests to block automatic WS-triggered downloads during
concurrent conflict scenarios.
Fix archive conflict test by blocking WS downloads on Client A during
the concurrent edit phase so it doesn't auto-receive B's rename via
WebSocket before archiving (restoring the intended conflict scenario).
Fix LWW singleton test to assert convergence rather than specific winner.
Fix renameTask helper to avoid Playwright/Angular re-render races.
Fix shepherd.js import paths that broke the Angular dev server build.
- Replace O(N) Prisma findMany with PostgreSQL SQL aggregate
(jsonb_each_text + GROUP BY + MAX) for snapshot vector clock
computation, avoiding loading all ops' clocks into Node.js memory
- Add defensive regex filter (^[0-9]+$) for non-numeric JSONB values
- Fix || to ?? for nullish coalescing in vector clock value lookups
- Remove unreachable CONCURRENT check in compareVectorClocks
- Remove unused VectorClockMetrics interface and measureVectorClock
- Add 7 new unit tests for SQL aggregate path verification
- Add 10 integration tests running actual SQL against PostgreSQL
Replace separate JWT_EXPIRY_MAGIC_LINK (365d) and JWT_EXPIRY_PASSKEY (7d)
constants with a single JWT_EXPIRY (365d). The auth method only matters
during login — once a JWT is issued, it represents a verified session
regardless of how the user authenticated.
On each successful sync request, the server now returns a fresh 14-day
JWT via X-Refreshed-Token response header. The client reads and stores
it automatically, preventing token expiry during active usage.
Server: export JWT_EXPIRY/getJwtSecret from auth, add createRefreshedToken(),
add onSend hook scoped to sync routes, expose header via CORS.
Client: store refreshed token in all 4 fetch methods, fix _getServerSeqKey
to hash only baseUrl (prevents full re-download on token change).
Operations with timestamps beyond maxClockDriftMs (60s) were being
permanently rejected, causing silent data loss when a client's clock
was slightly ahead.
Now these timestamps are clamped to now + maxClockDriftMs and accepted.
A TIMESTAMP_CLAMPED audit log is recorded for debugging.
- Add clamping logic in SyncService.processOperation()
- Remove future timestamp rejection from ValidationService
- Add comprehensive tests verifying clamped values and boundaries
- Update integration test to expect acceptance
Tombstones were used for tracking deleted entities but are no longer
needed with the operation log architecture. This removes:
- Tombstone table from Prisma schema
- Tombstone-related methods from SyncService
- Tombstone mocks and tests from all test files
- Database migration to drop tombstones table
The operation log now handles deletions through DEL operations,
making the separate tombstone tracking redundant.
Unit tests (6):
- Returns latestSnapshotSeq when SYNC_IMPORT exists
- Skips operations before SYNC_IMPORT for fresh clients (sinceSeq=0)
- Does NOT skip when sinceSeq is after SYNC_IMPORT
- Returns undefined latestSnapshotSeq when no full-state ops exist
- Handles BACKUP_IMPORT as a full-state operation
- Handles REPAIR as a full-state operation
Integration tests (5):
- HTTP endpoint includes latestSnapshotSeq in response
- HTTP endpoint skips pre-import ops for fresh clients
- HTTP endpoint does NOT skip when sinceSeq > SYNC_IMPORT seq
- HTTP endpoint omits latestSnapshotSeq when no full-state ops
- HTTP endpoint uses the latest full-state op when multiple exist
Code review fixes:
- Fix download memory limit to correctly return success: false
- Add logging when vector clock entries are stripped during sanitization
- Add integration test for partial batch upload recovery (network failure)
parentOpId support for conflict resolution:
- Add parentOpId field to Operation type for conflict chain tracking
- Add parent_op_id column to operations table with migration
- Update sync routes schema to accept parentOpId
- Store and retrieve parentOpId in sync service
Also adds PLUGIN_USER_DATA and PLUGIN_METADATA to allowed entity types.
Remove the acknowledgeOps mechanism and simplify server-side garbage
collection to pure time-based cleanup (delete ops older than 50 days).
Client-side changes:
- Remove acknowledgeOps from OperationSyncCapable interface
- Remove acknowledgeOps implementation from super-sync.ts
- Remove ACK call from download service (with explanatory comment)
- Remove ACK-related tests and mock server methods
Server-side changes:
- Remove ACK endpoint (POST /api/sync/devices/:clientId/ack)
- Remove updateDeviceAck and getMinAckedSeq from sync service
- Simplify deleteOldSyncedOpsForAllUsers to time-based only
- Update status endpoint (pendingOps now always 0)
Documentation updates:
- Update architecture diagrams to remove ACK references
- Update cleanup process to show 50-day time-based cleanup
This simplification was needed because fresh clients couldn't ACK
(device registration only happens on upload), causing 403 errors.
Time-based cleanup at 50 days is simpler and equally effective.
Also fixes DIAGNOSTIC test to expect server-side conflict detection
(test was written before server conflict detection was added).
Add two integration tests for gzip-compressed snapshot uploads:
- Verify compressed snapshots work through full route handler and sync
- Verify backwards compatibility with uncompressed snapshots
Tests simulate real multi-client scenarios where one client uploads
a compressed snapshot and another client retrieves it.
