When a full-state op (SYNC_IMPORT/BACKUP_IMPORT) arrived from another
client, mergeRemoteOpClocks reset the local clock to a "minimal" form
but only preserved the current client's counter from the incoming op's
clock. If the current client had issued ops (e.g. GLOBAL_CONFIG) not
reflected in the incoming full-state op's clock, its counter was dropped,
causing subsequent ops to reuse the same counter value. Downstream clients
then saw these ops as EQUAL (duplicate) and skipped them silently.
Fix: take max(mergedClock[clientId], currentClock[clientId]) when
rebuilding the clock after a full-state op reset.
Also add __SP_E2E_BLOCK_WS_DOWNLOAD flag to WsTriggeredDownloadService
to allow E2E tests to block automatic WS-triggered downloads during
concurrent conflict scenarios.
Fix archive conflict test by blocking WS downloads on Client A during
the concurrent edit phase so it doesn't auto-receive B's rename via
WebSocket before archiving (restoring the intended conflict scenario).
Fix LWW singleton test to assert convergence rather than specific winner.
Fix renameTask helper to avoid Playwright/Angular re-render races.
Fix shepherd.js import paths that broke the Angular dev server build.
- Replace O(N) Prisma findMany with PostgreSQL SQL aggregate
(jsonb_each_text + GROUP BY + MAX) for snapshot vector clock
computation, avoiding loading all ops' clocks into Node.js memory
- Add defensive regex filter (^[0-9]+$) for non-numeric JSONB values
- Fix || to ?? for nullish coalescing in vector clock value lookups
- Remove unreachable CONCURRENT check in compareVectorClocks
- Remove unused VectorClockMetrics interface and measureVectorClock
- Add 7 new unit tests for SQL aggregate path verification
- Add 10 integration tests running actual SQL against PostgreSQL
Add tests for previously untested vector clock pruning scenarios:
- Multi-preserve-ID pruning (two low-counter IDs both preserved)
- Overlapping/duplicate preserve IDs (Set deduplication)
- Preserve IDs missing from clock (silent skip, no crash)
- Snapshot vector clock aggregation + pruning integration
- Post-snapshot comparison correctness after pruning
- Server sanitizeVectorClock counter cap at 100M
- DoS cap at 2.5x MAX (reject, not prune)
- Invalid inputs (null, arrays, empty keys, long keys, negative/float values)
https://claude.ai/code/session_01GdsbKoo8eax394j2UyWUu1
Co-authored-by: Claude <noreply@anthropic.com>
- Cap sanitizeVectorClock values at 100M instead of MAX_SAFE_INTEGER (prevents adversarial clock inflation)
- Add early-exit optimization in compareVectorClocks for CONCURRENT detection
- Add test.setTimeout(180000) to token-expiry and lastseq-preservation E2E tests
- Remove dead code (unreachable guard) in compaction E2E test
- Extract shared config helpers (navigateToMiscSettings, toggleSetting, etc.) into e2e/utils/config-helpers.ts
- Fix Client B cleanup in provider-switch test (move to finally block)
- Add clarifying comments for raw SQL schema coupling, CORS header, and mergeRemoteOpClocks caller contract
- Extract enforceStorageQuota helper in sync routes to reduce duplication
- Add conflictType to conflict detection for structured error handling
- Add MAX_CLEANUP_ITERATIONS guard to freeStorageForUpload
Remove references to deleted docs/ai/ files, update stale comments about
protectedClientIds and pruning-aware comparison to reflect current REPLACE
semantics, fix MAX=30→20 heading, client_0..29→19 comment, use ?? over ||,
preserve lastSeq in server clean-slate path to prevent sequence reuse, and
add clarifying comments for mock limitations and validation guards.
- Add deterministic tie-breaking test for vector clock pruning with
equal counters (shared-schema)
- Remove last isLikelyPruningArtifact reference from entity versioning doc
- Use JWT_EXPIRY_PASSKEY (7d) for replaceToken instead of 365d magic link
expiry — token replacement is a security action, shorter lifetime is safer
- Replace localeCompare with locale-independent comparator in vector clock
tie-breaking to ensure deterministic behavior across environments
- Fix 5 additional stale MAX=30 references in docs and tests (now 20)
- Update authentication.md to reflect dual JWT expiry tiers
- Clean up isLikelyPruningArtifact references in docs and LEGACY_MAX in tests
- Fix client/server comparison divergence: remove isVectorClockEmpty
short-circuit that caused {} vs {a:0} to return LESS_THAN on client
but EQUAL on shared impl. Now delegates all comparisons to shared.
- Align server value cap with client: raise from 10M to MAX_SAFE_INTEGER
to prevent silent clock entry stripping on long-term usage.
- Make client validation require integers (Number.isInteger) matching
server-side validation.
- Add deterministic tie-breaking (localeCompare) to pruning sort for
equal counter values.
- Fix stale comments: MAX references and CLAUDE.md DoS cap (5x→2.5x).
Lower the cap to leave headroom for future increases and surface
size-related edge cases earlier. All pruning logic is MAX-agnostic
so this is a safe constant change with documentation updates.
At MAX=10, pruning triggered frequently enough (11+ unique client IDs from
reinstalls/new browsers) to require 4 defense layers compensating for
information loss: pruning-aware comparison, protected client IDs with
migration, isLikelyPruningArtifact heuristic, and same-client check.
At MAX=30, pruning almost never triggers (needs 31+ unique client IDs).
A 30-entry clock is ~500 bytes — negligible bandwidth. This allows removing
most defense layers while keeping two cheap backward-compat checks for old
10-entry pruned data still on servers.
Removed:
- Pruning-aware mode in compareVectorClocks (standard comparison now)
- Protected client IDs mechanism (storage, migration, preservation)
- selectProtectedClientIds function
- Clock normalization in SyncImportFilterService
Kept temporarily (backward compat with old 10-entry data):
- isLikelyPruningArtifact with LEGACY_MAX=10
- Same-client check (always mathematically correct)
Cover the full round-trip of limitVectorClockSize → compareVectorClocks →
isLikelyPruningArtifact with realistic MAX-entry clocks to guard against
regressions in the layered pruning heuristics.
Client-side pruning in OperationLogEffects dropped client IDs from
vector clocks (12→10 entries). Combined with the >= threshold in
compareVectorClocks' bothPossiblyPruned check, this caused false
CONCURRENT results and an infinite sync rejection loop.
- Remove limitVectorClockSize call from OperationLogEffects (server
already prunes AFTER comparison but BEFORE storage per arch doc #13)
- Change bothPossiblyPruned from >= to === MAX_VECTOR_CLOCK_SIZE
(a clock exceeding MAX was never pruned by limitVectorClockSize)
- Update test expectations and comments in both shared-schema and
client-side specs to reflect the corrected behavior
- Remove redundant subset condition in compareVectorClocks
- Add clarifying comment for conservative return in hasVectorClockChanges
- Batch per-key verbose logs into single summary log
- Add defensive warning for negative startingSeq in file-based sync
- Extract magic timeout constant in E2E test
- Update stale doc date
- Replace fragile VectorClockComparison[result] enum lookup with safe cast
- Return CONCURRENT instead of EQUAL when only one side has non-shared
keys in pruning-aware mode (safe direction: triggers LWW instead of
silent skip)
- Fix docs claiming clocks "reset" at MAX_SAFE_INTEGER (they throw)
- Add Logger.warn() in ValidationService when oversized vector clocks
are pruned server-side, improving observability for buggy clients
- Add tests for hasVectorClockChanges (previously zero coverage)
- Add symmetric LESS_THAN test for pruning-aware comparison mode
- Export MAX_LWW_REUPLOAD_RETRIES constant to eliminate magic number
coupling between sync-wrapper service and its test
- Add test verifying uploading client ID is preserved during server-side
clock pruning
- Add test for limitVectorClockSize when preserveClientIds exceeds MAX
Make hasVectorClockChanges pruning-aware by checking clock size before
logging missing keys — downgrade to verbose when pruning is likely,
warn when corruption is likely. Add return value assertion to LWW retry
exhaustion test, asymmetric pruning test cases, server-side pruning
tradeoff documentation, and fix stale "max 50" in docs.
When both vector clocks are at MAX size (pruning-aware mode) and shared
keys compare as equal, non-shared keys on both sides indicate genuinely
different causal histories. Returning EQUAL caused silent data loss by
skipping operations as duplicates. Returning CONCURRENT safely triggers
LWW conflict resolution instead.
Also fixes server snapshot pruning to preserve the requesting client's
ID and the snapshot author's client ID when limiting vector clock size.
Return CONCURRENT instead of EQUAL when two max-size clocks share no
keys (independent client populations). Fix misleading SyncStatus.InSync
return when LWW retry exhaustion leaves pending ops. Document known
limitation of size-based pruning heuristic. Add missing edge case tests.
Address issues found by code review agents after the LWW loop fix
(cb36c09538):
- Enforce MAX_VECTOR_CLOCK_SIZE on server upload to prevent oversized
clocks from buggy/adversarial clients
- Limit aggregated snapshotVectorClock in download service to prevent
unbounded growth with many clients
- Set UNKNOWN_OR_CHANGED status when LWW retry cap is exhausted instead
of misleadingly reporting IN_SYNC
- Wrap startPostSyncCooldown() in try-catch so endApplyingRemoteOps()
always runs even on failure
- Cap limitVectorClockSize output at MAX even when preserved IDs exceed
the limit
- Update stale doc comments referencing MAX_VECTOR_CLOCK_SIZE=8 to 10
Three root causes addressed:
1. Pruning asymmetry: Different clients preserve their own clientId
during vector clock pruning, producing different clock shapes.
When compared, pruned-away keys default to 0, causing false
CONCURRENT verdicts and infinite rejection loops. Fix: when both
clocks are at MAX_VECTOR_CLOCK_SIZE, compare only shared keys
(intersection) instead of the union.
2. Timing gap: Between endApplyingRemoteOps() and
startPostSyncCooldown(), isInSyncWindow() returns false, allowing
selector-based effects to fire and create TAG:TODAY operations.
Fix: start cooldown BEFORE ending remote ops flag.
3. No retry limit: LWW re-upload had no cap, enabling infinite loops.
Fix: cap at 3 retries, defer remaining to next sync cycle.
Also moves MAX_VECTOR_CLOCK_SIZE and limitVectorClockSize into
@sp/shared-schema so client and server share the same pruning
constant and algorithm.
When splitting one operation into misc and tasks operations:
- Both operations now get unique ID suffixes (_misc and _tasks)
- Skip logic verifies both conditions before skipping:
1. tasks config has migrated field (isConfirmBeforeDelete)
2. misc config has NO migrated fields remaining
This prevents:
- Potential operation ID conflicts in the sync log
- Data loss in partial migration scenarios
Adds test cases for partial migration and correct skip scenarios.
The previous commit changed module resolution to bundler mode which
fixed Angular tests but broke Node.js ESM imports in super-sync-server.
Switch to tsup bundler that outputs both ESM (.mjs) and CJS (.js) formats,
with proper exports field to support both environments:
- CJS for super-sync-server (Node.js commonjs)
- ESM for Angular/webpack bundler imports
- Change shared-schema tsconfig to use bundler moduleResolution
- Remove .js extensions from imports (incompatible with bundler mode)
- Fix tsconfig.spec.json to include both package path mappings
The shared-schema package was using NodeNext module resolution which
requires .js extensions in imports. Since the main Angular project
uses bundler resolution and imports directly from source files,
this caused webpack to fail finding the modules during tests.
Bug fix:
- Fix vector clock cache staleness in multi-tab scenarios by clearing
cache when acquiring operation write lock. Each browser tab has its
own in-memory cache, so Tab B's cache could be stale if Tab A wrote
while Tab B was waiting for the lock.
Shared code extraction (client/server consistency):
- Extract vector clock comparison to @sp/shared-schema
- Client wraps shared impl with null handling
- Server imports directly from shared
- Extract entity types to @sp/shared-schema
- Single source of truth for ENTITY_TYPES array
- Removes duplicated "must match" comments
Files:
- packages/shared-schema/src/vector-clock.ts (new)
- packages/shared-schema/src/entity-types.ts (new)
- src/app/op-log/store/operation-log-store.service.ts (cache clear)
- src/app/op-log/capture/operation-log.effects.ts (call cache clear)
