super-productivity/@docs/ai/operation-log-sync.md

Architecture V2: Operation Log & Event Sourcing

1. Executive Summary

Goal: Transition super-productivity from a state-snapshot sync model to an operation-based (event sourcing) model.

1.1. Current Problems Solved

Problem	Current Behavior	With Operation Log
Data Loss/Overwrite	"Last write wins" on whole file level overwrites granular changes	Per-operation merge enables granular conflict detection
Performance	Sync downloads full state and reloads entire app (reInit)	Only delta operations transferred and replayed
State Drift	Dual source of truth (NgRx vs. IndexedDB)	NgRx is single source, IndexedDB is derived
Conflict Resolution	User must choose entire file: local vs remote	Automatic merge of non-conflicting operations

1.2. New Paradigm

• NgRx is the Single Source of Truth.
• Persistence is a Log of Operations.
• Sync is the Exchange of Operations.

1.3. Key Benefits Over Current Implementation

The current sync system (see sync.service.ts) already uses vector clocks for conflict detection at the whole-data level. This proposal extends that to per-operation granularity:

Current:  [Device A State] vs [Device B State] → Conflict if concurrent
Proposed: [Op1, Op2, Op3]  vs [Op4, Op5]       → Merge non-conflicting, flag conflicts per-entity

---

2. Core Data Structures

2.1. The Vector Clock (Existing)

Already implemented in vector-clock.ts. No changes needed.

// Map of Client ID -> Counter
// e.g., { 'client_A': 15, 'client_B': 7 }
export type VectorClock = Record<string, number>;

2.2. The Operation Schema

All changes to the application state must be representable as a discrete operation.

export enum OpType {
  Create = 'CRT',
  Update = 'UPD',
  Delete = 'DEL',
  Move = 'MOV', // For list reordering
  Batch = 'BATCH', // For bulk operations (import, mass update)
}

export interface Operation {
  // IDENTITY
  id: string; // UUID v7 (time-ordered, better for sorting)

  // ACTION MAPPING
  actionType: string; // NgRx Action type (e.g., '[Task] Update')
  opType: OpType; // High-level operation category

  // SCOPE
  entityType: EntityType; // 'TASK' | 'PROJECT' | 'TAG' | 'NOTE' | 'GLOBAL_CONFIG' | 'SIMPLE_COUNTER'
  entityId: string; // ID of the affected entity (or '*' for global config)

  // DATA
  // Validated by Typia.
  // For large text fields (Notes), consider storing diffs instead of full content in future iterations.
  payload: unknown;

  // CAUSALITY & ORDERING
  clientId: string; // Device generating the op (reuse existing from vector-clock)
  vectorClock: VectorClock; // State of the world AFTER this Op
  timestamp: number; // Wall clock time (ISO 8601 or epoch ms)

  // META
  schemaVersion: number; // For future migrations
  parentOpId?: string; // For conflict resolution chains
}

export type EntityType =
  | 'TASK'
  | 'PROJECT'
  | 'TAG'
  | 'NOTE'
  | 'GLOBAL_CONFIG'
  | 'SIMPLE_COUNTER'
  | 'WORK_CONTEXT'
  | 'TASK_REPEAT_CFG'
  | 'ISSUE_PROVIDER'
  | 'MIGRATION';

2.3. The Persistent Log Entry

How operations are stored locally in IndexedDB.

export interface OperationLogEntry {
  seq: number; // Local, monotonic auto-increment (IndexedDB primary key)
  op: Operation;
  appliedAt: number; // When this op was applied locally (epoch ms)
  source: 'local' | 'remote'; // Origin of this operation
  syncedAt?: number; // When successfully synced to remote (null if pending)
}

export interface OperationLogStore {
  // Core read/write
  append(op: Operation, source?: 'local' | 'remote'): Promise<void>;
  hasOp(id: string): Promise<boolean>;
  getOpsAfterSeq(seq: number): Promise<OperationLogEntry[]>;
  getUnsynced(): Promise<OperationLogEntry[]>;
  getUnsyncedByEntity(): Promise<Map<string, Operation[]>>;
  getAppliedOpIds(): Promise<Set<string>>;
  markApplied(id: string): Promise<void>;
  markSynced(seqs: number[]): Promise<void>;

  // Compaction helpers
  deleteOpsWhere(predicate: (entry: OperationLogEntry) => boolean): Promise<void>;
  getLastSeq(): Promise<number>;
  saveStateCache(snapshot: {
    state: unknown;
    lastAppliedOpSeq: number;
    vectorClock: VectorClock;
    compactedAt: number;
  }): Promise<void>;
  loadStateCache(): Promise<{
    state: unknown;
    lastAppliedOpSeq: number;
    vectorClock: VectorClock;
    compactedAt: number;
  } | null>;

  // Frontiers
  getEntityFrontier(
    entityType?: EntityType,
    entityId?: string,
  ): Promise<Map<string, VectorClock>>;
  getCurrentVectorClock(): Promise<VectorClock>;
}

2.4. Entity-Level Conflict Tracking

For fine-grained conflict resolution:

export interface EntityConflict {
  entityType: EntityType;
  entityId: string;
  localOps: Operation[]; // Local ops affecting this entity
  remoteOps: Operation[]; // Remote ops affecting the same entity
  suggestedResolution: 'local' | 'remote' | 'merge' | 'manual';
  mergedPayload?: unknown; // If auto-mergeable
}

---

3. Architecture Layers

3.1. The Unified Flow

graph TD
    subgraph User Actions
        User[User Interaction] -->|Triggers| Action[NgRx Action]
    end

    subgraph Core State
        Action -->|Reducer| Store[NgRx Store]
    end

    subgraph Persistence Layer
        Action -.->|Effect: isPersistent| OpConverter[Operation Converter]
        OpConverter -->|Web Locks / Mutex| OpLog[(IndexedDB: op_log)]
        OpLog -->|Compaction Trigger| StateCache[(IndexedDB: state_cache)]
    end

    subgraph Sync Layer
        OpLog -->|Pending Ops| SyncUp[Sync Upload]
        SyncUp -->|HTTP/WebDAV| Cloud[(Remote Storage)]
        Cloud -->|Download| SyncDown[Sync Download]
        SyncDown -->|Filter Duplicates| MergeEngine[Merge Engine]
        MergeEngine -->|Non-Conflict| RemoteAction[Dispatch isRemote:true]
        MergeEngine -->|Conflict| ConflictResolver[Conflict Resolution UI]
        RemoteAction --> Store
    end

    subgraph Compaction
        StateCache -->|Load| Hydrate[Hydrate Store]
        OpLog -->|Replay Tail| Hydrate
    end

3.2. The Action Contract (Whitelisting)

Not all NgRx actions should be persisted. We need an explicit contract.

// In src/app/core/persistence/operation-log/persistent-action.interface.ts

export interface PersistentActionMeta {
  isPersistent?: boolean; // When false, the action is blacklisted and not persisted
  entityType: EntityType;
  entityId: string;
  opType: OpType;
  isRemote?: boolean; // TRUE if from Sync (prevents re-logging)
  isBulk?: boolean; // TRUE for batch operations
}

export interface PersistentAction<P = unknown> extends Action {
  type: string; // Standard NgRx action type
  meta: PersistentActionMeta;
  payload?: P;
}

// Helper type guard
export function isPersistentAction(action: Action): action is PersistentAction {
  return (action as PersistentAction).meta?.isPersistent !== false;
}

3.3. Action Blacklist Registry

Maintenance Note: To prevent developers from forgetting to blacklist new actions that should not be persisted, or unintentionally blacklisting actions that should, we should implement a unit test that scans all NgRx action definitions and asserts they are either explicitly listed in BLACKLISTED_ACTION_TYPES or are intended to be persistent.

// In src/app/core/persistence/operation-log/action-whitelist.ts

export const BLACKLISTED_ACTION_TYPES: Set<string> = new Set([
  // Explicitly EXCLUDED (not persisted):
  '[App] Set Current Worklog Task', // UI state
  '[Layout] Toggle Sidebar', // UI state
  '[Focus Mode] Enter/Exit', // transient
]);

---

4. Critical Workflows

4.1. The Write Path (Local User Action)

1. User clicks "Complete Task".
2. Component dispatches TaskActions.update({ id, changes: { isDone: true } }).
3. Reducer: Updates memory state immediately (optimistic UI).
4. OperationLogEffect (new):
   • Listens for actions matching PERSISTENT_ACTION_TYPES or with meta.isPersistent: true.
   • Ignores actions with meta.isRemote: true.
   • Converts Action → Operation object.
   • Increments local vector clock counter.
   • Writes to IndexedDB using LockService.

// In src/app/core/persistence/operation-log/operation-log.effects.ts

@Injectable()
export class OperationLogEffects {
  private clientId = this.getOrCreateClientId();

  persistOperation$ = createEffect(
    () =>
      this.actions$.pipe(
        filter((action) => (action as PersistentAction).meta?.isPersistent !== false), // Check for explicit opt-out
        filter((action) => !BLACKLISTED_ACTION_TYPES.has(action.type)), // Check against global blacklist
        filter((action) => !(action as PersistentAction).meta?.isRemote), // Always ignore remote actions
        concatMap((action) => this.writeOperation(action)),
      ),
    { dispatch: false },
  );

  private async writeOperation(action: PersistentAction): Promise<void> {
    try {
      await this.lockService.request('sp_op_log_write', async () => {
        const currentClock = await this.opLogStore.getCurrentVectorClock();
        const newClock = incrementVectorClock(currentClock, this.clientId);

        const op: Operation = {
          id: uuidv7(),
          actionType: action.type,
          opType: action.meta.opType,
          entityType: action.meta.entityType,
          entityId: action.meta.entityId,
          payload: action.payload,
          clientId: this.clientId,
          vectorClock: newClock,
          timestamp: Date.now(),
          schemaVersion: CURRENT_SCHEMA_VERSION,
        };

        await this.opLogStore.append(op);
      });
    } catch (e) {
      // 4.1.1 Error Handling for Optimistic Updates
      console.error('Failed to persist operation', e);
      this.notifyUserAndTriggerRollback(action);
    }
  }
}

4.2. The Read Path (Startup / App Load)

Loading thousands of operations is too slow. We use a Rolling Snapshot strategy.

sequenceDiagram
    participant App
    participant StateCache as IndexedDB: state_cache
    participant OpLog as IndexedDB: op_log
    participant Store as NgRx Store

    App->>StateCache: Load cached state + lastAppliedOpSeq
    StateCache-->>App: { state, lastAppliedOpSeq: 1500 }
    App->>Store: Hydrate with cached state
    App->>OpLog: Query ops WHERE seq > 1500
    OpLog-->>App: [Op 1501, Op 1502, Op 1503]
    loop Replay Each Op
        App->>Store: Dispatch action (isRemote: true)
    end
    App->>App: Ready

Implementation:

// In src/app/core/persistence/operation-log/operation-log-hydrator.service.ts

@Injectable({ providedIn: 'root' })
export class OperationLogHydratorService {
  async hydrateStore(): Promise<void> {
    // 1. Load snapshot
    const snapshot = await this.opLogStore.loadStateCache();

    if (snapshot) {
      // 2. Hydrate NgRx with snapshot
      this.store.dispatch(hydrateFromSnapshot({ state: snapshot.state }));

      // 3. Replay tail operations
      const tailOps = await this.opLogStore.getOpsAfterSeq(snapshot.lastAppliedOpSeq);

      for (const entry of tailOps) {
        const action = this.opToActionConverter.convert(entry.op);
        action.meta.isRemote = true; // Prevent re-logging
        this.store.dispatch(action);
      }
    } else {
      // Fresh install or migration - no snapshot exists
      await this.handleFreshInstallOrMigration();
    }
  }
}

4.3. The Sync Path (Bidirectional)

4.3.1. Upload (Local → Remote)

async uploadPendingOps(): Promise<void> {
  const pendingOps = await this.opLogStore.getUnsynced();

  if (pendingOps.length === 0) return;

  // Batch into chunks (e.g., 100 ops per file for WebDAV)
  const chunks = chunkArray(pendingOps, 100);

  for (const chunk of chunks) {
    const filename = `ops_${this.clientId}_${chunk[0].op.timestamp}.json`;
    await this.syncProvider.uploadFile(filename, JSON.stringify(chunk));

    // Mark as synced
    await this.opLogStore.markSynced(chunk.map(e => e.seq));
  }

  // Update remote manifest
  await this.updateRemoteManifest();
}

4.3.2. Download (Remote → Local)

async downloadRemoteOps(): Promise<void> {
  const manifest = await this.syncProvider.downloadFile('manifest.json');
  const appliedOpIds = await this.opLogStore.getAppliedOpIds();
  const appliedFrontierByEntity = await this.opLogStore.getEntityFrontier(); // latest applied vector per entity

  // Find new remote op files
  const newFiles = manifest.opFiles.filter(f => !this.isFileFullyApplied(f, appliedOpIds));

  for (const file of newFiles) {
    const opsData = await this.syncProvider.downloadFile(file);
    const remoteOps: Operation[] = JSON.parse(opsData);

    // Filter already-applied ops
    const newOps = remoteOps.filter(op => !appliedOpIds.has(op.id));

    // Check for conflicts at entity level
    const { nonConflicting, conflicts } = await this.detectConflicts(newOps, appliedFrontierByEntity);

    // Apply non-conflicting ops
    for (const op of nonConflicting) {
      // Persist remote op first so the log/frontier stays canonical
      // append must be idempotent or guarded
      if (!(await this.opLogStore.hasOp(op.id))) {
        await this.opLogStore.append(op, 'remote');
      }

      const action = this.opToActionConverter.convert(op);
      action.meta.isRemote = true;
      this.store.dispatch(action);

      // markApplied assumes the op is already in the log
      await this.opLogStore.markApplied(op.id);
    }

    // Handle conflicts
    if (conflicts.length > 0) {
      await this.conflictHandler.present(conflicts);
    }
  }
}

4.3.3. Conflict Detection Algorithm

async detectConflicts(
  remoteOps: Operation[],
  appliedFrontierByEntity: Map<string, VectorClock>,
): Promise<ConflictResult> {
  const localPendingOps = await this.opLogStore.getUnsyncedByEntity();
  const conflicts: EntityConflict[] = [];
  const nonConflicting: Operation[] = [];

  for (const remoteOp of remoteOps) {
    const entityKey = `${remoteOp.entityType}:${remoteOp.entityId}`;
    const localOpsForEntity = localPendingOps.get(entityKey) || [];
    const appliedFrontier = appliedFrontierByEntity.get(entityKey);

    // Build the frontier from everything we know locally (applied + pending)
    const localFrontier = mergeVectorClocks([
      ...(appliedFrontier ? [appliedFrontier] : []),
      ...localOpsForEntity.map((op) => op.vectorClock),
    ]);

    // Check if ops are concurrent (vector clock comparison)
    const vcComparison = compareVectorClocks(localFrontier, remoteOp.vectorClock);

    if (vcComparison === VectorClockComparison.CONCURRENT) {
      // True conflict - same entity modified independently
      conflicts.push({
        entityType: remoteOp.entityType,
        entityId: remoteOp.entityId,
        localOps: localOpsForEntity,
        remoteOps: [remoteOp],
        suggestedResolution: this.suggestResolution(localOpsForEntity, remoteOp),
      });
    } else {
      // One happened before the other - can be auto-resolved
      nonConflicting.push(remoteOp);
    }
  }

  return { nonConflicting, conflicts };
}

4.4. Multi-Tab & Cross-Process Coordination

We need a robust LockService that abstracts the locking mechanism, as navigator.locks (Web Locks API) is not reliable in all environments (e.g., older Android WebViews, some Capacitor configs).

Locking Strategy:

1. Primary: navigator.locks (Web Locks API) - Best performance, dead-lock free.
2. Fallback: localStorage mutex with timeout or simple file locks (for Electron if native locks fail).

// In src/app/core/persistence/operation-log/lock.service.ts
// Pseudocode for the service

@Injectable({ providedIn: 'root' })
export class LockService {
  async request(lockName: string, callback: () => Promise<void>): Promise<void> {
    if (navigator.locks) {
      return navigator.locks.request(lockName, callback);
    } else {
      // Fallback implementation using localStorage polling or similar
      return this.acquireFallbackLock(lockName, callback);
    }
  }
}

// In src/app/core/persistence/operation-log/multi-tab-coordinator.service.ts
@Injectable({ providedIn: 'root' })
export class MultiTabCoordinatorService {
  private broadcastChannel = new BroadcastChannel('sp_op_log');

  constructor(private lockService: LockService) {
    this.broadcastChannel.onmessage = (event) => {
      if (event.data.type === 'NEW_OP') {
        // Another tab wrote an operation - reload from DB
        this.handleRemoteTabOp(event.data.op);
      }
    };
  }

  async writeOperation(op: Operation): Promise<void> {
    await this.lockService.request('sp_op_log_write', async () => {
      await this.opLogStore.append(op);
      // Notify other tabs
      this.broadcastChannel.postMessage({ type: 'NEW_OP', op });
    });
  }
}

4.5. Compaction (Garbage Collection)

To prevent the log from growing infinitely.

Triggers:

• Every N operations (configurable, default: 500)
• On app close/background
• When log size exceeds threshold (e.g., 10MB)

Process:

Guardrail: never delete an operation that has not been confirmed as synced to remote storage, even if it is older than the retention window. Keep a trailing window (by seq) for conflict detection/hydration and require syncedAt before pruning.

async compact(): Promise<void> {
  await this.lockService.request('sp_op_log_compact', async () => {
    // 1. Get current NgRx state
    const currentState = await firstValueFrom(this.store.select(selectAllSyncState));

    // 2. Get current vector clock (max of all ops)
    const currentVectorClock = await this.opLogStore.getCurrentVectorClock();

    // 3. Write to state cache
    const lastSeq = await this.opLogStore.getLastSeq();
    await this.opLogStore.saveStateCache({
      state: currentState,
      lastAppliedOpSeq: lastSeq,
      vectorClock: currentVectorClock,
      compactedAt: Date.now(),
    });

    // 4. Delete old operations (keep recent for conflict resolution window)
    const retentionWindowMs = 7 * 24 * 60 * 60 * 1000; // 7 days
    await this.opLogStore.deleteOpsWhere(
      (entry) =>
        !!entry.syncedAt && // never drop unsynced ops
        entry.appliedAt < Date.now() - retentionWindowMs &&
        entry.seq <= lastSeq, // keep tail for conflict frontier
    );
  });
}

4.5.1. Remote Storage Cleanup

When performing a full snapshot upload to remote storage, we must also efficiently manage the remote file list.

• Create a new snapshot_TIMESTAMP.json on remote.
• Identify ops/ files that are fully contained within this new snapshot.
• Delete those old ops/ files from the remote provider to prevent directory listing performance degradation (WebDAV/Dropbox are slow with 1000+ files).

---

5. Cross-Model Relationships & Referential Integrity

Super Productivity has several cross-model relationships that require special handling in an operation-based sync system:

5.1. Entity Relationship Map

erDiagram
    TASK ||--o| PROJECT : "belongs to"
    TASK }|--o{ TAG : "has many"
    TASK ||--o| TASK : "parent/child"
    NOTE ||--o| PROJECT : "belongs to"
    TASK_REPEAT_CFG ||--o| PROJECT : "belongs to"
    ISSUE_PROVIDER ||--o| PROJECT : "belongs to"

    TASK {
        string id PK
        string projectId FK
        string parentId FK
        array tagIds FK
    }
    PROJECT {
        string id PK
        array taskIds
        array backlogTaskIds
    }
    TAG {
        string id PK
        array taskIds
    }

5.2. Reference Types & Handling Strategies

Relationship	Reference Field	On Foreign Delete	On Missing Reference
Task → Project	task.projectId	Orphan (keep task, null projectId)	Queue op, apply when project exists
Task → Tag	task.tagIds[]	Remove from array	Skip missing tag, log warning
Task → Parent Task	task.parentId	Cascade delete subtask	Queue op, apply when parent exists
Note → Project	note.projectId	Orphan (keep note)	Queue op, apply when project exists
TaskRepeatCfg → Project	cfg.projectId	Orphan or delete cfg	Queue op

5.3. Operation Ordering & Dependencies

When syncing operations, some must be applied before others. We use a dependency resolver:

interface OperationDependency {
  opId: string;
  dependsOn: {
    entityType: EntityType;
    entityId: string;
    mustExist: boolean; // true = hard dependency, false = soft (can apply without)
  }[];
}

// Example dependencies:
// - Create SubTask depends on parent Task existing (hard)
// - Create Task in Project depends on Project existing (soft - can orphan)
// - Add Tag to Task depends on Tag existing (soft - skip missing)

Dependency Resolution Algorithm:

async applyOperationsWithDependencies(ops: Operation[]): Promise<ApplyResult> {
  const pendingQueue: Operation[] = [];
  const applied: Operation[] = [];
  const failed: { op: Operation; reason: string }[] = [];

  // Sort operations by logical order
  const sortedOps = this.topologicalSort(ops);

  for (const op of sortedOps) {
    const deps = this.extractDependencies(op);
    const missingHardDeps = await this.findMissingHardDependencies(deps);

    if (missingHardDeps.length > 0) {
      // Check if dependency will be satisfied by a pending op
      const willBeSatisfied = missingHardDeps.every(dep =>
        pendingQueue.some(pendingOp =>
          pendingOp.entityType === dep.entityType &&
          pendingOp.entityId === dep.entityId &&
          pendingOp.opType === OpType.Create
        )
      );

      if (willBeSatisfied) {
        pendingQueue.push(op);
      } else {
        // Hard dependency missing and won't be created - queue for later sync
        this.queueForRetry(op, missingHardDeps);
      }
      continue;
    }

    // Handle soft dependencies (missing tags, etc.)
    const missingSoftDeps = await this.findMissingSoftDependencies(deps);
    if (missingSoftDeps.length > 0) {
      op.payload = this.removeMissingReferences(op.payload, missingSoftDeps);
      this.logWarning('Applied op with missing soft dependencies', { op, missingSoftDeps });
    }

    // Apply operation
    await this.applyOperation(op);
    applied.push(op);

    // Check if any pending ops can now be applied
    await this.processPendingQueue(pendingQueue, applied);
  }

  return { applied, pending: pendingQueue, failed };
}

5.4. Cascade Operations

When a parent entity is deleted, related entities must be handled:

// In operation-log.effects.ts

// Intercept delete operations and emit cascade ops
handleCascadeDelete$ = createEffect(() =>
  this.actions$.pipe(
    filter(isPersistentAction),
    filter(action => action.meta.opType === OpType.Delete),
    switchMap(action => this.generateCascadeOps(action)),
  )
);

private async generateCascadeOps(deleteAction: PersistentAction): Promise<Action[]> {
  const cascadeOps: Action[] = [];

  switch (deleteAction.meta.entityType) {
    case 'PROJECT':
      // Find all tasks in this project
      const tasks = await this.taskService.getAllTasksForProject(deleteAction.meta.entityId);

      // Option A: Orphan tasks (move to inbox)
      for (const task of tasks) {
        if (!task.parentId) { // Only main tasks
          cascadeOps.push(updateTask({
            task: { id: task.id, changes: { projectId: INBOX_PROJECT.id } },
            meta: {
              isPersistent: true,
              entityType: 'TASK',
              entityId: task.id,
              opType: OpType.Update,
              cascadeFrom: deleteAction.meta.entityId,
            }
          }));
        }
      }
      break;

    case 'TASK':
      // If parent task deleted, delete all subtasks
      const parentTask = await this.taskService.getByIdOnce$(deleteAction.meta.entityId);
      for (const subTaskId of parentTask.subTaskIds) {
        cascadeOps.push(deleteTask({
          task: { id: subTaskId },
          meta: {
            isPersistent: true,
            entityType: 'TASK',
            entityId: subTaskId,
            opType: OpType.Delete,
            cascadeFrom: deleteAction.meta.entityId,
          }
        }));
      }
      break;

    case 'TAG':
      // Remove tag from all tasks (soft cascade)
      // This is handled by the reducer, but we log it
      break;
  }

  return cascadeOps;
}

5.5. Conflict Resolution with Related Entities

When conflicts occur, we must consider the relationship graph:

interface RelatedConflict extends EntityConflict {
  relatedConflicts: EntityConflict[];  // Conflicts in related entities
  relationshipType: 'parent' | 'child' | 'reference';
}

// Example: Task updated locally, but its Project was deleted remotely
// This creates a "cascading conflict" that needs unified resolution

async detectRelatedConflicts(conflict: EntityConflict): Promise<RelatedConflict> {
  const relatedConflicts: EntityConflict[] = [];

  if (conflict.entityType === 'TASK') {
    const task = conflict.localOps[0].payload as Partial<Task>;

    // Check if project was modified/deleted
    if (task.projectId) {
      const projectConflict = await this.findConflictForEntity('PROJECT', task.projectId);
      if (projectConflict) {
        relatedConflicts.push({ ...projectConflict, relationshipType: 'parent' });
      }
    }

    // Check if parent task was modified/deleted
    if (task.parentId) {
      const parentConflict = await this.findConflictForEntity('TASK', task.parentId);
      if (parentConflict) {
        relatedConflicts.push({ ...parentConflict, relationshipType: 'parent' });
      }
    }

    // Check if any referenced tags were modified/deleted
    for (const tagId of (task.tagIds || [])) {
      const tagConflict = await this.findConflictForEntity('TAG', tagId);
      if (tagConflict) {
        relatedConflicts.push({ ...tagConflict, relationshipType: 'reference' });
      }
    }
  }

  return { ...conflict, relatedConflicts };
}

5.6. Atomic Multi-Entity Operations

Some user actions affect multiple entities atomically. These should be grouped:

// Example: Moving a task to another project affects:
// 1. Task.projectId
// 2. Source Project.taskIds (remove)
// 3. Target Project.taskIds (add)

interface AtomicOperationGroup {
  groupId: string; // UUIDv7
  operations: Operation[]; // All ops in this group
  rollbackOnPartialFailure: boolean;
}

// When creating ops for moveToProject:
const moveToProjectOps: AtomicOperationGroup = {
  groupId: uuidv7(),
  operations: [
    {
      id: uuidv7(),
      actionType: '[Task] Update',
      opType: OpType.Update,
      entityType: 'TASK',
      entityId: taskId,
      payload: { projectId: targetProjectId },
      // ... other fields
      groupId: groupId, // Link to group
    },
    // Note: Project.taskIds updates are derived, not stored as separate ops
    // The reducer handles this automatically
  ],
  rollbackOnPartialFailure: true,
};

5.7. Optimistic Updates with Rollback

For related entities, we need coordinated rollback:

class RelatedEntityRollbackService {
  private pendingRollbacks = new Map<string, RollbackState>();

  async applyWithRollback(ops: Operation[]): Promise<void> {
    const groupId = ops[0].groupId || uuidv7();
    const rollbackState: RollbackState = {
      originalStates: new Map(),
      appliedOps: [],
    };

    try {
      for (const op of ops) {
        // Capture original state before applying
        const originalState = await this.captureEntityState(op.entityType, op.entityId);
        rollbackState.originalStates.set(
          `${op.entityType}:${op.entityId}`,
          originalState,
        );

        // Apply operation
        await this.applyOperation(op);
        rollbackState.appliedOps.push(op);
      }

      // All succeeded - clear rollback state
      this.pendingRollbacks.delete(groupId);
    } catch (error) {
      // Rollback all applied operations in reverse order
      for (const appliedOp of rollbackState.appliedOps.reverse()) {
        const key = `${appliedOp.entityType}:${appliedOp.entityId}`;
        const originalState = rollbackState.originalStates.get(key);
        await this.restoreEntityState(
          appliedOp.entityType,
          appliedOp.entityId,
          originalState,
        );
      }
      throw error;
    }
  }
}

5.8. Handling Orphaned References During Sync

When downloading remote operations, we may encounter references to entities that don't exist locally:

enum OrphanHandling {
  QUEUE_AND_RETRY = 'queue', // Wait for the referenced entity to be created
  SKIP_REFERENCE = 'skip', // Apply op but skip the missing reference
  CREATE_PLACEHOLDER = 'placeholder', // Create a minimal placeholder entity
  REJECT = 'reject', // Fail the operation
}

const ORPHAN_HANDLING_POLICY: Record<string, OrphanHandling> = {
  'TASK.projectId': OrphanHandling.QUEUE_AND_RETRY,
  'TASK.parentId': OrphanHandling.QUEUE_AND_RETRY,
  'TASK.tagIds': OrphanHandling.SKIP_REFERENCE,
  'NOTE.projectId': OrphanHandling.QUEUE_AND_RETRY,
  'TASK_REPEAT_CFG.projectId': OrphanHandling.QUEUE_AND_RETRY,
};

5.9. Undo/Redo Handling

In an Event Sourcing architecture, "Undo" cannot simply revert the application state, as this would break the operation history chain and vector clocks.

Strategy: Compensating Transactions (Inverse Operations)

• When the user triggers "Undo", we identify the last operation(s) performed by the current client.
• We generate a new Inverse Operation that reverses the effect.
  • Original: Create Task A → Undo: Delete Task A
  • Original: Update Task A (title: 'Foo') [prev: 'Bar'] → Undo: Update Task A (title: 'Bar')
• This inverse operation is appended to the log with a new ID and updated vector clock.
• It syncs naturally to other devices like any other operation.

---

6. Integration with Existing pfapi

6.1. Coexistence Strategy

The operation log can be introduced alongside the existing sync system, not as a replacement:

Phase 1: Operation logging only (local persistence, no sync changes)
Phase 2: Dual-write (log ops + maintain current snapshot sync)
Phase 3: Operation-based sync (read from log for sync)
Phase 4: Remove snapshot sync (operation log is primary)

6.2. pfapi Integration Points

Existing Component	Integration Approach
MetaModelCtrl	Add op log sequence to meta
ModelSyncService	Keep for snapshot backup
SyncService.sync()	Add operation exchange phase
Vector Clock utilities	Reuse existing implementation

6.3. Remote Storage Format

For compatibility with existing WebDAV/Dropbox providers:

/superproductivity/
├── main.json              # Legacy: Full state snapshot (kept for backup)
├── meta.json              # Legacy: Metadata with vector clock
├── ops/                   # NEW: Operation logs directory
│   ├── manifest.json      # Index of all op files + last compaction
│   ├── ops_clientA_1700000000.json
│   ├── ops_clientA_1700001000.json
│   ├── ops_clientB_1700000500.json
│   └── ...
└── snapshots/             # NEW: Periodic full snapshots
    ├── snapshot_1700000000.json
    └── ...

---

7. Migration Strategy

7.1. Detection Phase

async detectMigrationNeeded(): Promise<MigrationType> {
  const opLogExists = await this.opLogStore.exists();
  const legacyDataExists = await this.legacyStore.hasData();

  if (!opLogExists && legacyDataExists) {
    return MigrationType.LEGACY_TO_OPLOG;
  }
  if (opLogExists && legacyDataExists) {
    return MigrationType.HYBRID; // During transition
  }
  if (!opLogExists && !legacyDataExists) {
    return MigrationType.FRESH_INSTALL;
  }
  return MigrationType.ALREADY_MIGRATED;
}

7.2. Genesis Operation

For existing users upgrading from the current snapshot-based system:

async migrateFromLegacy(): Promise<void> {
  // 1. Load all legacy data
  const legacyState = await this.pfapi.getAllSyncModelData();

  // 2. Create genesis operation
  const genesisOp: Operation = {
    id: uuidv7(),
    actionType: '[Migration] Genesis Import',
    opType: OpType.Batch,
    entityType: 'MIGRATION' as EntityType,
    entityId: '*',
    payload: legacyState,
    clientId: this.clientId,
    vectorClock: { [this.clientId]: 1 },
    timestamp: Date.now(),
    schemaVersion: CURRENT_SCHEMA_VERSION,
  };

  // 3. Write genesis op
  await this.opLogStore.append(genesisOp);

  // 4. Create initial state cache
  await this.opLogStore.saveStateCache({
    state: legacyState,
    lastAppliedOpSeq: 1,
    vectorClock: genesisOp.vectorClock,
    compactedAt: Date.now(),
  });

  // 5. Mark legacy as migrated (don't delete yet)
  await this.legacyStore.markMigrated();
}

7.3. Rollback Strategy

If issues are detected after migration:

async rollbackToLegacy(): Promise<void> {
  // 1. Check if legacy backup exists
  const legacyBackup = await this.legacyStore.getBackup();
  if (!legacyBackup) {
    throw new Error('No legacy backup available for rollback');
  }

  // 2. Restore legacy data
  await this.pfapi.importAllSyncModelData({ data: legacyBackup });

  // 3. Clear operation log
  await this.opLogStore.clearAll();

  // 4. Update migration status
  await this.legacyStore.markRolledBack();
}

---

8. Risks & Mitigations

Risk	Severity	Mitigation
Infinite Loops	High	Strict isRemote flag + action type blacklist + tests
Startup Slowness	Medium	Aggressive compaction (default: 500 ops, 7 day retention)
Lost Updates (Tabs)	High	Web Locks API + BroadcastChannel + Fallbacks
Data Bloat	Medium	Blacklist non-essential actions + automatic compaction
Remote File Explosion	Medium	Aggressive cleanup of remote op files during snapshot uploads
Ordering Glitches	Medium	Vector clocks + UUID v7 (time-ordered) + deterministic sort
Memory Pressure	Medium	Streaming replay for large op sets + pagination
Sync Provider Limits	Low	Chunk ops into max 1MB files + manifest index
Schema Evolution	Medium	schemaVersion field + forward-compatible payloads
Partial Sync Failures	High	Transaction-like batching + rollback on failure
Clock Skew	Low	Use vector clocks for ordering, wall clock only as tiebreaker
Dangling References	High	Dependency resolver + orphan handling policies (Section 5)
Cascade Failures	Medium	Atomic operation groups + coordinated rollback
Circular Dependencies	Low	Topological sort with cycle detection

---

9. Testing Strategy

9.1. Unit Tests

describe('OperationLogStore', () => {
  it('should append operations with correct sequence numbers');
  it('should query operations after a given sequence');
  it('should handle concurrent writes with Web Locks');
  it('should compact old operations correctly');
});

describe('OperationLogEffects', () => {
  it('should convert persistent actions to operations');
  it('should ignore remote actions (isRemote: true)');
  it('should increment vector clock on each operation');
});

describe('ActionBlacklist', () => {
  it('should ensure every ActionType is either blacklisted or implicitly persistent');
});

describe('ConflictDetector', () => {
  it('should identify concurrent changes to same entity');
  it('should auto-resolve sequential changes');
  it('should suggest merge for compatible changes');
});

describe('DependencyResolver', () => {
  it('should detect hard dependencies (subtask -> parent task)');
  it('should handle soft dependencies (task -> tag)');
  it('should queue operations with missing dependencies');
  it('should apply queued ops when dependencies arrive');
  it('should detect and reject circular dependencies');
});

describe('CascadeOperations', () => {
  it('should generate cascade deletes for subtasks when parent deleted');
  it('should orphan tasks when project deleted');
  it('should remove deleted tag from all task.tagIds');
});

9.2. Integration Tests

describe('Multi-Tab Sync', () => {
  it('should coordinate writes across browser tabs');
  it('should broadcast new operations to other tabs');
  it('should handle tab crashes gracefully');
});

describe('Remote Sync', () => {
  it('should upload pending operations');
  it('should download and apply remote operations');
  it('should detect and present conflicts');
  it('should handle operations referencing entities that arrive later');
  it('should resolve related entity conflicts together');
  it('should cleanup old remote op files after snapshotting');
});

describe('Cross-Model Integrity', () => {
  it('should maintain task-project relationship after sync');
  it('should maintain task-tag relationship after sync');
  it('should maintain parent-subtask relationship after sync');
  it('should handle project deletion with associated tasks during sync');
});

9.3. E2E Tests

describe('Operation Log E2E', () => {
  it('should persist task creation across page reload');
  it('should sync changes between two browser sessions');
  it('should handle offline changes and sync on reconnect');
  it('should migrate from legacy data correctly');
});

---

10. MVP Guardrails and Scope

Replay without side effects

• Introduce a replay dispatcher (or global guard) that prevents effects/notifications from firing for meta.isRemote/meta.isReplay.
• Add tests asserting hydration/sync replays do not call effects.

Canonical operation codecs

• Per-entity Typia codecs that normalize payloads to minimal diffs (no derived arrays or redundant fields).
• Persist the normalized patch shape, not raw action payloads. Ensure only non-blacklisted actions are persisted.
• Typia Integration: Ensure ts-patch is correctly configured for all build targets (Electron/Capacitor).

Deterministic reducers

• Audit reducers for randomness/time or hidden side effects; refactor or exclude non-deterministic actions from persistence.
• Property-test: random op streams applied via replay vs. direct reducer baseline must match.

Conflict frontier

• Track last-applied op/vector per entity; conflict detection compares remote ops against both unsynced local ops and the applied frontier (not just pending uploads).
• Order application by vector-clock comparison + local seq; do not rely on timestamps.

Snapshot/manifest contract

• Before deleting ops, publish a snapshot (state + vector clock + retained op range) to remote storage.
• Keep ops from that snapshot forward; new devices hydrate from the latest published snapshot + tail.

Atomic multi-entity application

• Add a batch/transaction path so grouped ops apply atomically (or rollback) instead of partial state on crash.

Dependency handling with bounds

• Queue missing-dependency ops with retry budget/diagnostics; process via topological order that tolerates concurrent creates/updates; move exhausted items to a “stuck” queue.

Cross-platform coordination (Critical)

• Feature-detect navigator.locks.
• Mandatory Fallback: Provide robust fallback (IndexedDB polling / LocalStorage mutex) for environments where Web Locks are flaky (WebView/iOS).

Upload/idempotency robustness

• Use content-hash filenames; write manifest last; reconcile uploaded-but-unmanifested files on startup; dedupe by op ID regardless of file overlap.

Migration slicing

• Emit per-entity genesis create ops sharing one frontier instead of a single huge batch payload; align the initial snapshot to that frontier.

11. Implementation Phases

Phase 1: Local Operation Logging (2-3 weeks)

• Create OperationLogStore (IndexedDB adapter)
• Create Operation types and per-entity codecs (Typia) with normalization
• Implement LockService with robust fallbacks
• Create OperationLogEffects (listen to persistent actions) with replay guard for effects
• Create action blacklist registry with Type Safety tests
• Add isPersistent: false meta to 10-15 core actions that should not be persisted
• Create OperationLogHydrator for startup (side-effect-safe replay path)
• Create compaction service with snapshot/manifest contract
• Unit tests + replay determinism property test

Phase 2: Multi-Tab Coordination (1 week)

• Integrate LockService for write coordination
• Implement BroadcastChannel for cross-tab notification with fallbacks
• Handle tab crash/unexpected close
• Integration tests

Phase 3: Dependency & Relationship Handling (1-2 weeks)

• Implement dependency resolver for operation ordering with bounded retries
• Implement orphan handling policies
• Implement cascade operation generation
• Implement atomic operation groups/batch apply with rollback
• Unit tests for cross-model scenarios

Phase 4: Conflict Detection (1-2 weeks)

• Implement entity-level conflict detection
• Implement related entity conflict detection
• Create conflict resolution UI component
• Implement auto-merge strategies where possible
• Add "suggest resolution" logic

Phase 5: Remote Sync Integration (2-3 weeks)

• Extend sync providers with op file support
• Implement op manifest handling
• Implement bidirectional op sync
• Implement remote storage cleanup (delete old op files)
• Handle partial sync failures
• Integrate with existing SyncService

Phase 6: Migration & Rollout (1-2 weeks)

• Implement legacy detection
• Implement genesis operation migration
• Implement rollback mechanism
• Feature flag for gradual rollout
• E2E tests for migration scenarios

Phase 7: Legacy Removal (Post-Stable)

• Remove dual-write mode
• Remove snapshot sync code
• Update documentation
• Announce breaking change (if needed)

---

12. Open Questions

1. Payload Size Limits: Should we split large payloads (e.g., long notes) into separate operations? Strategy: Monitor during MVP. Implement diffing for notes if bloat occurs.
2. Selective Sync: Can users choose which entities to sync (e.g., only project A)?
3. Audit Trail: Should we expose the operation log as a user-visible feature (activity history)?
4. Encryption: Should individual operations be encrypted, or only the full op files?
5. Cascade Delete Policy: When deleting a project, should tasks be orphaned or deleted? Current decision: Orphan tasks to Inbox.
6. Relationship Validation: Should we validate referential integrity on every operation, or trust the client?
7. Partial Relationship Sync: If a task references 3 tags but only 2 exist remotely, how do we handle the third?
8. Ordering Guarantees: Do we need strict causal ordering for related entities, or is eventual consistency acceptable?
9. Cross-Client Consistency: How do we handle the case where Client A deletes a project while Client B creates a task in that project (both offline)?

---

13. References

• Event Sourcing Pattern (Martin Fowler)
• CRDT Primer
• Vector Clocks Explained
• Web Locks API
• Existing Vector Clock Implementation
• Current Sync Service