super-productivity/docs/sync-and-op-log/vector-clocks.md

Vector Clocks Architecture

1. Overview

Vector clocks track causality — "did this client know about that operation?" — rather than wall-clock time, which can drift between devices. They are the foundation of conflict detection and SYNC_IMPORT filtering in Super Productivity's sync system.

Core Type

interface VectorClock {
  [clientId: string]: number;
}

Each entry maps a client ID to a monotonically increasing counter. A clock with {A: 5, B: 3} means "this state includes A's first 5 operations and B's first 3 operations".

Constants

Constant	Value	Purpose
MAX_VECTOR_CLOCK_SIZE	20	Maximum entries in a pruned clock

At 6-char client IDs, a 20-entry clock is ~333 bytes — negligible bandwidth. A user needs 21+ unique client IDs (reinstalls/new browsers) before pruning triggers, which is extremely unlikely for a personal productivity app.

---

2. Core Operations

Three operations — compare, merge, and prune (limitVectorClockSize) — are implemented in the generic sync-core package (packages/sync-core/src/vector-clock.ts), used by both client and server. Two operations — initialize and increment — are client-only (src/app/core/util/vector-clock.ts), which also wraps the shared operations with null-handling and logging.

Create

initializeVectorClock(clientId) → { [clientId]: 0 }

Increment

incrementVectorClock(clock, clientId) → { ...clock, [clientId]: clock[clientId] + 1 }

Throws on overflow (approaching MAX_SAFE_INTEGER). The only recovery is a SYNC_IMPORT to reset clocks.

Compare

compareVectorClocks(a, b) → EQUAL | LESS_THAN | GREATER_THAN | CONCURRENT

Standard vector clock comparison. Missing keys are treated as zero.

Merge

mergeVectorClocks(a, b) → { [key]: max(a[key], b[key]) for all keys in a ∪ b }

Creates a new clock that dominates both inputs.

---

3. Where Vector Clocks Live

Per-Operation Clock

Every Operation carries a vectorClock field — the global clock state at the time the operation was created. This is the primary mechanism for causality tracking.

Global Clock Store

Stored in IndexedDB (SUP_OPS database, vector_clock object store) as a VectorClockEntry:

interface VectorClockEntry {
  clock: VectorClock; // Current global clock
  lastUpdate: number; // Timestamp of last update
}

The global clock is the single source of truth for the client's current causal knowledge. During local operation capture, it is updated atomically with operation writes (single IndexedDB transaction via appendWithVectorClockOverwrite). The remote merge path (mergeRemoteOpClocks) updates the clock in a separate write after reading the current state.

Snapshot Clock

The state_cache stores a vectorClock representing the clock at compaction time. This serves as a baseline for entities that haven't been modified since the last snapshot.

Entity Frontier

Per-entity latest clocks, computed on demand by VectorClockService.getEntityFrontier(). Built by scanning operations after the snapshot. Used for fine-grained conflict detection.

---

4. Vector Clock Lifecycle (Normal Operations)

Step 1: Local Operation Created

In operation-log.effects.ts:

1. VectorClockService.getCurrentVectorClock() reads the global clock from the vector_clock store
2. incrementVectorClock(currentClock, clientId) creates a new clock with the client's counter incremented
3. The operation is created with this full, unpruned clock
4. appendWithVectorClockOverwrite(op, 'local') writes the operation AND updates the global clock in a single atomic IndexedDB transaction

Key invariant: Normal operations carry full (unpruned) vector clocks. No client-side pruning happens during capture.

Step 2: Upload to Server

In sync.service.ts (processOperation):

1. ValidationService.validateOp() sanitizes the clock (DoS cap at 2.5×MAX = 50 entries) but does NOT prune
2. detectConflict() compares the full unpruned incoming clock against the existing entity clock
3. If accepted: limitVectorClockSize() prunes to MAX before storage, preserving the uploading client and, when present, the latest causal full-state author
4. The pruned clock is stored in the database

Step 3: Download by Other Clients

In operation-log-store.service.ts (mergeRemoteOpClocks):

1. Each downloaded operation's clock is merged into the local global clock
2. For full-state operations (SYNC_IMPORT/BACKUP_IMPORT/REPAIR), the global clock is replaced (not merged) with the import's clock, then remaining ops are merged on top — existing entries not present in the import's clock can be lost
3. For non-full-state downloads, the merge preserves all existing entries (inherits new entries without losing existing ones)

Key Insight

Normal operations are NEVER pruned client-side. The server prunes after comparison but before storage. This asymmetry is critical — see Section 6 for why.

---

5. Pruning

Why Pruning Exists

Clocks grow with each new client. Without bounds, a user who has used many devices would have ever-growing clocks. Pruning limits clocks to MAX_VECTOR_CLOCK_SIZE (20) entries.

The limitVectorClockSize Algorithm

Input: clock, preserveClientIds[]
If entries ≤ MAX: return clock unchanged
Otherwise:
  1. Add entries from preserveClientIds first (capped at MAX)
  2. Fill remaining slots with highest-counter entries (sorted descending)
  3. Return clock with exactly MAX entries

Implemented in packages/sync-core/src/vector-clock.ts. The client wrapper in src/app/core/util/vector-clock.ts adds logging and passes [currentClientId] as the preserve list.

When Pruning Happens (Exhaustive List)

Location	When	What's Preserved
Server processOperation()	After conflict detection, before storage	Uploading client + active full-state author
Server getOpsSinceWithSeq()	Aggregating snapshot vector clock	Requesting client
Client SyncHydrationService	Creating SYNC_IMPORT during conflict resolution	Current client only
Client ServerMigrationService	Creating SYNC_IMPORT during migration	Current client only
Client RepairOperationService	Creating REPAIR operation	Current client only
Client OperationLogSnapshotService	Saving snapshot to state cache	Current client only
Client OperationLogCompactionService	Compaction (saving snapshot + deleting old ops)	Current client only
Client OperationLogHydratorService	Restoring snapshot during hydration	Current client only
Client normal op capture	NEVER	N/A
Client SupersededOperationResolverService	NEVER (conflict resolution)	N/A

Pruning is Rare

With MAX=20, a user needs 21+ unique client IDs before pruning triggers. The server preserves both the uploader and the latest causal full-state author when they are present in the incoming clock. Preserving that boundary edge prevents high-counter historical clients from making a post-import operation appear concurrent to the importing client. Other pruned edges can still cause one extra server round-trip (false CONCURRENT → client resolves → re-uploads with >MAX clock → GREATER_THAN → accepted).

---

6. Conflict Detection & Resolution (Server Upload)

Server-Side Flow

1. Server finds the latest operation for the same entity (findFirst by entityType + entityId, ordered by serverSeq desc)
2. Compares incoming clock vs existing clock using the full unpruned incoming clock
3. Possible outcomes:
   • GREATER_THAN → accept (incoming op causally succeeds existing)
   • EQUAL + same client → accept (retry of same operation)
   • EQUAL + different client → reject (suspicious clock reuse)
   • CONCURRENT → reject (true conflict)
   • LESS_THAN → reject (superseded)
4. If accepted: prune clock, then store

Client-Side Resolution

When the server rejects an operation:

1. Client receives rejection with existingClock
2. SupersededOperationResolverService.resolveSupersededLocalOps():
   • Merges the global clock + all superseded ops' clocks + snapshot clock + extra clocks from force download
   • Calls mergeAndIncrementClocks() — no client-side pruning!
   • Creates new LWW Update ops with the merged clock
3. Re-uploads → server compares the full merged clock (which now has MAX+1 entries or more) → GREATER_THAN → accept
4. Server prunes the merged clock before storage

Critical Invariant: Server Must Prune AFTER Comparison

If the server pruned before comparison, it would be impossible to build a dominating clock when the entity clock already has MAX entries and the client's ID isn't among them.

Safety net: RejectedOpsHandlerService tracks resolution attempts per entity. After exceeding MAX_CONCURRENT_RESOLUTION_ATTEMPTS (3) consecutive failures, ops are permanently rejected.

---

7. SYNC_IMPORT / BACKUP_IMPORT / REPAIR Handling

The Core Rule: Clean Slate Semantics

An import is an explicit user action to restore all clients to a specific state. Operations without knowledge of the import are dropped:

Comparison	Meaning	Action
GREATER_THAN	Op created after seeing import	Keep
EQUAL	Same causal history as import	Keep
CONCURRENT	Op created without knowledge of import	Drop
LESS_THAN	Op is dominated by import	Drop

CONCURRENT ops are dropped even from unknown clients. This ensures a true "restore to point in time" semantic.

How Import Clocks Are Created

Source	Method	Clock Construction
BACKUP_IMPORT (clean slate)	BackupService	Fresh clock {newClientId: 1} — small, no pruning issues
Server migration	ServerMigrationService	Merge all local op clocks + global clock → increment → prune to MAX
Sync hydration (conflict resolution)	SyncHydrationService	Merge local clock + state cache clock + remote snapshot clock → increment → prune to MAX
Auto-repair	RepairOperationService	Get current global clock → increment → prune to MAX

Full-State Operations Skip Server Conflict Detection

In detectConflict(), operations with opType of SYNC_IMPORT, BACKUP_IMPORT, or REPAIR return { hasConflict: false } immediately. These operations replace entire state and don't operate on individual entities.

Multi-Entity Ops and Server-Side Conflict Detection (issue #8334)

An op may carry entityIds: string[] (batch actions: deleteTasks, moveToArchive, __updateMultipleTaskSimple, round-time-spent, task-repeat-cfg/board/issue-provider batches). detectConflict() checks the incoming op against all of its entityIds. The op must also be looked up by all of its entities once stored — otherwise a later stale write to a non-first entity would find no prior writer and be wrongly accepted as non-conflicting.

To make that symmetric, the operations row stores:

• entity_id — the client-supplied scalar. For batch ops the client sets it to entityIds[0] (operation-log.effects.ts), but the server does not enforce entity_id === entityIds[0]. It is the lookup key for single-entity ops and the first entity of multi-entity ops, and is also used by duplicate detection.
• entity_ids — the entity set for multi-entity ops only (a text[] column; populated via getStoredEntityIds(op), which returns [] for single-entity ops). Keeping single-entity rows out of this column keeps the GIN(entity_ids) index small and the array-branch lookup cheap.

The lookups in conflict.ts match a requested entity as the scalar entity_id or a member of entity_ids:

• detectConflictForEntity (single) — Prisma where: { OR: [{ entityId }, { entityIds: { has: entityId } }] }, ordered by server_seq. The OR spans the entity_id btree and the entity_ids GIN, so the planner uses a BitmapOr + sort bounded by the entity's stored version depth (op-log pruning keeps that small). If a real-Postgres EXPLAIN on a deep-history entity ever shows this hot path is a problem, the escalation is two ordered LIMIT 1 lookups (scalar btree + entity_ids GIN) taking the higher server_seq — the array side stays small because the column is multi-entity-only.
• detectConflictForEntities / prefetchLatestEntityOpsForBatch (batch) — raw SQL unnesting CASE WHEN cardinality(entity_ids) > 0 THEN entity_ids ELSE ARRAY[entity_id] END, with a entity_ids && ... OR entity_id = ANY(...) prefilter so the GIN(entity_ids) index (migration 20260613000001) and the existing entity_id btree stay usable.

Forward-only by design: rows written before migration 20260613000000 have an empty entity_ids array and fall back to the scalar entity_id (= first entity) in the CASE expression / scalar branch above — there is no UPDATE backfill. Entities 2..n of already-stored multi-entity ops were never persisted and are unrecoverable, so they remain invisible to conflict detection until that entity gets a fresh write. This residual is bounded: client-side LWW is unaffected (the client persists the full op and VectorClockService.getEntityFrontier() fans each op out to every entity), and the server only builds an authoritative snapshot from non-encrypted ops (replayOpsToState() throws on encrypted ops), so the pre-fix gap could only surface a stale value to a fresh client on non-encrypted self-hosted servers.

The SyncImportFilterService Algorithm

Implemented in src/app/op-log/sync/sync-import-filter.service.ts:

1. Find the latest full-state op — the last full-state op in the downloaded batch wins (server apply order); otherwise use the non-rejected local full-state entry with the greatest local sequence. UUIDv7 IDs are identities, not causal clocks, because a device clock can move backwards.
2. For each non-full-state operation in the batch:
   • Compare op.vectorClock vs import clock
   • GREATER_THAN or EQUAL → keep
   • CONCURRENT + same client as import + higher counter → keep (same-client check)
   • CONCURRENT against automatic REPAIR → keep and replay after the repair boundary
   • Otherwise → filter

Same-Client Check

If an op is from the same client that created the import, with a higher counter, it's definitely a post-import op. A client can't create ops concurrent with its own import — counters are monotonically increasing. This check is always correct and cheap (~15 lines).

---

8. Key Scenarios (Step-by-Step Traces)

Scenario 1: Two-Client Sync (No Conflicts)

Initial state: Client A and B both know about each other
  A's global clock: {A: 3, B: 2}
  B's global clock: {A: 3, B: 2}

Step 1: A creates a task
  A increments: {A: 4, B: 2}
  Op carries clock: {A: 4, B: 2}
  A's global clock updated to: {A: 4, B: 2}

Step 2: A uploads
  Server compares op clock {A: 4, B: 2} vs latest entity clock (none) → no conflict
  Server stores op (no pruning needed, 2 entries < MAX)

Step 3: B downloads
  B receives op with clock {A: 4, B: 2}
  B merges into global clock: max({A: 3, B: 2}, {A: 4, B: 2}) = {A: 4, B: 2}

Step 4: B creates a task
  B increments: {A: 4, B: 3}
  B's global clock updated to: {A: 4, B: 3}

Scenario 2: Concurrent Modification (Conflict Resolution)

Starting state: Both clients synced
  A's clock: {A: 3, B: 2}    B's clock: {A: 3, B: 2}

Step 1: Both modify the same task offline
  A creates op: {A: 4, B: 2}
  B creates op: {A: 3, B: 3}

Step 2: A uploads first → server accepts (no prior op for this entity)
  Server stores: {A: 4, B: 2}

Step 3: B uploads
  Server compares: {A: 3, B: 3} vs {A: 4, B: 2}
  A=3 < 4 (b greater), B=3 > 2 (a greater) → CONCURRENT → reject
  Server returns existingClock: {A: 4, B: 2}

Step 4: B resolves
  SupersededOperationResolverService merges:
    globalClock={A: 3, B: 3} + existingClock={A: 4, B: 2} + opClock={A: 3, B: 3}
    merged = {A: 4, B: 3}, incremented = {A: 4, B: 4}
  Creates new LWW Update op with clock {A: 4, B: 4}
  NO client-side pruning

Step 5: B re-uploads
  Server compares: {A: 4, B: 4} vs {A: 4, B: 2} → GREATER_THAN → accept
  Server stores (pruned if needed, but only 2 entries here)

Scenario 3: SYNC_IMPORT with Small Clock (Clean Slate)

Step 1: Client A does BACKUP_IMPORT (full data restore)
  Creates SYNC_IMPORT op with clock: {A: 1}
  Uploads to server

Step 2: Client B has been working offline
  If B never saw A's state: B's clock: {B: 5}
  Compare: {B: 5} vs {A: 1} → CONCURRENT → filtered ✓

  If B had previously synced with A: B's clock: {A: 3, B: 5}
  Compare: {A: 3, B: 5} vs {A: 1} → GREATER_THAN → kept ✓
  (B's ops were created with knowledge beyond the import point)

---

9. Invariants

Rules that must hold for the system to be correct. Use these to verify implementations and tests.

1. Normal ops carry full (unpruned) vector clocks. No pruning in operation-log.effects.ts.

2. Server prunes AFTER comparison, BEFORE storage. processOperation() calls limitVectorClockSize() after detectConflict() succeeds.

3. Client does NOT prune during conflict resolution. SupersededOperationResolverService sends full merged clocks; the server prunes after accepting.

4. compareVectorClocks produces identical results on client and server. Both import from @sp/sync-core. The client wrapper only adds null handling.

5. Full-state ops skip conflict detection on server. detectConflict() returns { hasConflict: false } for SYNC_IMPORT, BACKUP_IMPORT, and REPAIR.

6. CONCURRENT ops are FILTERED (not kept) against SYNC_IMPORT — unless identified as legacy pruning artifacts or same-client ops. Clean slate semantics — this is the explicit, correct behavior.

7. Global clock is REPLACED (not merged) on remote SYNC_IMPORT. mergeRemoteOpClocks() starts from the import's clock as the base, then merges remaining ops on top. This prevents clock bloat.

8. DoS cap is NOT pruning. sanitizeVectorClock() rejects clocks with > 2.5×MAX (50) entries entirely — it doesn't prune them down. This is a validation gate, not a size reduction.

---

10. Key Files Reference

Concept	File(s)
Core algorithms (compare, merge, prune)	packages/sync-core/src/vector-clock.ts
Compatibility re-export for existing shared-schema imports	(removed — imports now target @sp/sync-core directly)
Client wrappers (null handling, logging, validation)	src/app/core/util/vector-clock.ts
Global clock management, entity frontier	src/app/op-log/sync/vector-clock.service.ts
Operation capture (no pruning, atomic clock update)	src/app/op-log/capture/operation-log.effects.ts
Clock persistence	src/app/op-log/persistence/operation-log-store.service.ts
Import filtering + same-client check	src/app/op-log/sync/sync-import-filter.service.ts
Conflict resolution (no pruning, merges clocks)	src/app/op-log/sync/superseded-operation-resolver.service.ts
Conflict resolution (LWW logic, mergeAndIncrementClocks)	src/app/op-log/sync/conflict-resolution.service.ts
SYNC_IMPORT creation (sync hydration)	src/app/op-log/persistence/sync-hydration.service.ts
SYNC_IMPORT creation (server migration)	src/app/op-log/sync/server-migration.service.ts
REPAIR creation	src/app/op-log/validation/repair-operation.service.ts
Server: conflict detection + prune after comparison	packages/super-sync-server/src/sync/sync.service.ts
Server: DoS cap (sanitize, no pruning)	packages/super-sync-server/src/sync/services/validation.service.ts
Server: snapshot clock pruning during download optimization	packages/super-sync-server/src/sync/services/operation-download.service.ts

---

11. History & Rationale (why pruning is the way it is)

Decision-history behind the current pruning design (previously in a separate research doc, now git-only). Load-bearing context for anyone changing MAX_VECTOR_CLOCK_SIZE or the prune ordering.

Compare before pruning — and the bugs that proved it

Never prune a vector clock before using it in a comparison. Pruning removes information: a missing entry is ambiguous — "never knew about this client" vs "entry was pruned" — so a pre-pruned comparison returns CONCURRENT instead of EQUAL/causal. Two independent incidents established this:

• Riak #613: pruning before comparison caused "sibling explosion" — objects accumulated hundreds of siblings that could never resolve because pruned clocks always compared CONCURRENT.
• Super Productivity (Feb 2026): with MAX = 10, server pruning before comparison caused an infinite rejection loop — a client merges all clocks + its own ID (11 entries), the server prunes to 10, the non-shared key forces CONCURRENT, the server rejects, the client re-merges, the loop repeats.

Fix in both systems: compare the full unpruned clock, then prune only before storage. This is the invariant in §6 and §9.

Why MAX = 20 (the 10 → 30 → 20 evolution)

The original defense against the Feb-2026 loop was a 4-layer scheme (broad protected-client tracking, pruning-aware comparison, an isLikelyPruningArtifact heuristic, the same-client check) — symptom treatment. The root cause was that MAX = 10 was too small, making pruning frequent and interacting badly with SYNC_IMPORT.

Commit d70f18a94d raised MAX 10 → 30 (later reduced to 20 — a 20-entry clock is ~333 bytes, negligible) and removed the broad tracking and comparison heuristics. The server now has one narrow storage-only exception: it preserves the latest causal full-state author so post-import operations retain that boundary edge. isLikelyPruningArtifact was dropped (known false positives, unnecessary at MAX = 20). Only the same-client check remains — always mathematically correct during conflict comparison (monotonic counters are definitive) and independent of MAX. At MAX = 20, pruning needs 21+ distinct client IDs, extremely rare for a personal productivity app, so the pruning path is effectively dormant (see §5 "Pruning is Rare").

Future options (only if the server becomes the coordinator)

In a server-authoritative model, clock growth could be bounded without pruning via Dotted Version Vectors (bound to server vnodes, not devices), bounded reclaimable client IDs (needs a registration/retirement protocol), or periodic stable-cut GC (needs all-to-all clock reporting). None apply to the current dumb-relay model.