Add 11 additional edge case tests for the server-side snapshot skip
optimization:
- SYNC_IMPORT at seq 1
- sinceSeq exactly equal to snapshotSeq
- Empty ops when all before snapshot
- excludeClient interaction with snapshot detection
- Gap detection: intentional skip vs real gaps
- Pagination after snapshot
- Multiple snapshots (use latest)
- Empty server state
- sinceSeq slightly before snapshotSeq
- sinceSeq at exactly snapshotSeq-1
Total: 17 unit tests covering the optimization thoroughly.
Unit tests (6):
- Returns latestSnapshotSeq when SYNC_IMPORT exists
- Skips operations before SYNC_IMPORT for fresh clients (sinceSeq=0)
- Does NOT skip when sinceSeq is after SYNC_IMPORT
- Returns undefined latestSnapshotSeq when no full-state ops exist
- Handles BACKUP_IMPORT as a full-state operation
- Handles REPAIR as a full-state operation
Integration tests (5):
- HTTP endpoint includes latestSnapshotSeq in response
- HTTP endpoint skips pre-import ops for fresh clients
- HTTP endpoint does NOT skip when sinceSeq > SYNC_IMPORT seq
- HTTP endpoint omits latestSnapshotSeq when no full-state ops
- HTTP endpoint uses the latest full-state op when multiple exist
When a fresh client downloads operations (sinceSeq=0), the server now
finds the latest full-state operation (SYNC_IMPORT, BACKUP_IMPORT, REPAIR)
and starts from there instead of from the beginning.
This optimization:
- Reduces bandwidth by not sending superseded operations
- Reduces server and client processing time
- Works with the client-side filtering fix (pre-import ops are filtered anyway)
The response now includes latestSnapshotSeq to indicate where the
effective state starts, which clients can use for optimization decisions.
Remove flawed same-client exception in _filterOpsInvalidatedBySyncImport().
The bug allowed pre-import operations from the importing client to bypass
timestamp filtering, causing state corruption when a fresh client joined
a sync group after a data import.
The assumption that "ops from the same client have knowledge of the import"
was wrong for pre-import operations. They reference the old state that the
import replaced. Now all pre-import ops are filtered by timestamp regardless
of which client created them.
The effect filter used a truthy check which filtered out 0 values,
preventing setStartOfNextDayDiff(0) from being called when resetting
the setting to its default value.
Add support for {entityType}Ids pattern (e.g., taskIds) in
validateUpdatePayload to prevent false "unusual structure" warnings
for actions like planTasksForToday and removeTasksFromTodayTag.
Update test expectations in map-archive-to-worklog.spec.ts and E2E tests
to match the new alphabetical sorting order for worklog entries.
The sorting feature sorts tasks alphabetically while keeping subtasks
grouped with their parent tasks. This changes the expected order in tests:
- Parent tasks now sorted alphabetically (MT1 before PT1)
- Subtasks sorted alphabetically within their group (SUB_B before SUB_C)
Two key fixes for LWW conflict resolution sync flow:
1. Add lwwUpdateMetaReducer to handle [ENTITY_TYPE] LWW Update actions
- When Client B receives an LWW Update from Client A, the meta-reducer
updates the NgRx store so the UI reflects changes without reload
- Supports TASK, PROJECT, TAG, NOTE, SIMPLE_COUNTER, TASK_REPEAT_CFG
2. Propagate needsReupload flag through sync flow
- autoResolveConflictsLWW now returns { localWinOpsCreated: number }
- downloadRemoteOps returns needsReupload flag when local-win ops created
- sync.service triggers second upload when needsReupload is true
- Ensures local-win operations are uploaded in the same sync cycle
Tests added:
- Unit tests for lwwUpdateMetaReducer (10 tests)
- Integration tests for LWW Update store application (12 tests)
- E2E tests for LWW conflict resolution scenarios (6 tests)
The short-syntax meta-reducer was updating timeSpentOnDay directly via
taskAdapter.updateOne() without recalculating the timeSpent total. This
caused the UI to display stale time values.
Now uses updateTimeSpentForTask() which:
- Calculates timeSpent from timeSpentOnDay
- Handles parent task time aggregation for subtasks
- Add caching for getUnsynced() with cache invalidation on markSynced/markRejected
- Optimize getAppliedOpIds() to incrementally update cache instead of full rebuild
- When new ops are added, only scan the new entries instead of the entire table
- Initial cache build still requires full scan, but subsequent calls are O(new_ops)
This significantly reduces sync latency by avoiding multiple full IndexedDB table
scans on each sync cycle. Before: 2-3 full scans × 300-500ms. After: Only scan
newly added operations (typically 0-5 ops = ~1ms).
Also fix lint errors in spec files (client-B -> clientB naming convention).
The meta-reducer was incorrectly positioned LAST in the array, but NgRx
composes meta-reducers such that FIRST = OUTERMOST. This caused
operationCaptureMetaReducer to receive already-modified state from
taskSharedCrudMetaReducer, resulting in beforeState === afterState
and no entity changes being detected for delete operations.
Moving it to FIRST position ensures it captures the original state
before any other meta-reducer modifies it.
Adds integration tests to verify correct ordering and prevent regression.
The documentation incorrectly described "vector clock dominance" filtering,
but the actual implementation uses UUIDv7 timestamp comparison. Updated
Section 2c to accurately reflect the implementation and explain why UUIDv7
is used (SYNC_IMPORT operations break vector clock causality detection).
* master:
fix(electron): use includes() instead of in operator for hostname check
fix(docker): use Debian-based nginx for ARM64 QEMU compatibility
16.6.1
build: add resolved URL and integrity for ical.js version 2.1.0
fix(ui): align time tracking button overlay (#5720)
fix(calendar): handle Office 365 updateTimezones crash (#5722)
fix(repeat): use fallback for undefined startDate (#5724)
build(welcome): update wording for issue claiming instructions
fix(docker): drop arm/v7 platform to fix QEMU build failure
The conflict resolution dialog and presentConflicts method have been removed
as all conflicts are now automatically resolved using Last-Write-Wins (LWW)
strategy. This removes the need for manual user intervention during sync.
Changes:
- Remove DialogConflictResolutionComponent and related files
- Remove presentConflicts method from ConflictResolutionService
- Remove presentConflicts tests and dialog timeout tests
- Remove D_CONFLICT_RESOLUTION translations from en.json
- Update spec files to reference autoResolveConflictsLWW instead
The remaining ConflictResolutionService now only contains:
- isIdenticalConflict (for detecting when both sides made same change)
- autoResolveConflictsLWW (automatic timestamp-based conflict resolution)
Replace manual conflict resolution dialogs with automatic Last-Write-Wins
(LWW) resolution based on operation timestamps:
- Add autoResolveConflictsLWW() to ConflictResolutionService
- When remote timestamp >= local: remote wins, local op rejected
- When local timestamp > remote: local wins, create new UPDATE op
with current entity state and merged vector clock
- Show non-blocking snackbar notification after resolution
- Create safety backup before resolving conflicts
Includes comprehensive test coverage:
- Unit tests for ConflictResolutionService
- Integration tests for LWW timestamp comparison and vector clock merging
- E2E tests for multi-client convergence scenarios
Add detailed LWW documentation with Mermaid diagrams explaining the
algorithm, outcomes, and tradeoffs vs manual resolution.
- Add autoResolveConflictsLWW to ConflictResolutionService spy
- Mock window.confirm to return true for fresh client confirmation
- Update conflict test assertion to check autoResolveConflictsLWW
Skip dependency checking and archive handling during local hydration
since operations were already validated when created and archive data
is already persisted. Also removes debug console.log statements and
reduces logging verbosity.
Diagram 2 corrections:
- Gap detection does NOT fetch snapshot - it resets sinceSeq=0 and
re-downloads all ops from the server
- Add "Server Migration" case: gap + empty server triggers SYNC_IMPORT
- Update Key Implementation Details with correct behavior
Diagram 1 cleanup:
- Remove misleading "OpEffects -->|flushYoungToOld| ArchiveYoung" arrow
(flush is handled by ArchiveOperationHandler, not direct OpEffects write)
Update Conflict Management section to show auto-resolve flow:
- Both DELETE same entity → auto-resolved as "remote"
- Identical payload (same opType + payload) → auto-resolved as "remote"
- Real conflicts → show user dialog
Add auto-resolve note to Key Implementation Details.
Replace fragmented sections 2, 2.1, 2.2, 2.3 with single comprehensive
master diagram showing:
- Client-side: sync loop, download/upload flow, conflict management
- Server-side: all 6 API endpoints with request/response details
- Server processing: validation, conflict detection, persistence
- PostgreSQL: all 4 tables with columns and write operations
- Connections: API -> processing -> database with operation types
Add quick reference tables for API endpoints and PostgreSQL tables.
- Update first diagram to show ArchiveModel contains both task: TaskArchive
and timeTracking: State (not just time tracking data)
- Add flushYoungToOld action flow from OpEffects to archive
- Expand Archive Data Flow Notes with flush mechanism details
- Update Section 8.2 dual-database diagram with full ArchiveModel structure
Replace incorrect "Multi-Tab Coordinator" with actual behavior:
- Single instance enforcement via BroadcastChannel (startup.service.ts)
- Web Locks API for critical sync operations (lock.service.ts)
Add section 8.2 showing the two separate IndexedDB databases:
- SUP_OPS: Operation log (ops table, state_cache)
- PFAPI: Archive data (archiveYoung, archiveOld), time tracking, legacy sync
Update archive operations flow diagram to explicitly show:
- PFAPI IndexedDB as the storage destination for archive data
- SUP_OPS IndexedDB for operation log entries
- Clear separation between the two storage systems
Add Archive Storage section to the first architecture diagram showing:
- ArchiveService writing to IndexedDB BEFORE dispatch
- Two-tier archive storage (archiveYoung/archiveOld)
- Time tracking data (timeSpentOnDay entries)
- Notes explaining the archive data flow and sync handling
- Fix silent save failures during sync by adding snackbar notification
- Fix fresh client race condition by using synchronous window.confirm()
- Simplify LockService to Web Locks API only (97%+ browser support)
- Add monotonicity warning when server sequence decreases
- Document vector clock corruption limitation and ModelCtrl cache bypass
- Add unit tests for UUID timestamp extraction from UUIDv7
- Update hybrid-manifest-architecture.md status to Implemented
- Add documentation index (README.md) for operation-log docs
- Add index for docs/ai/sync/ with document categorization
- Replace duplicate docs with redirect stubs to canonical locations
- Add effect rules, multi-entity rules, and config constants to operation-rules.md
- Update vector-clocks.md with operation log integration section
- Expand super-sync-server README with API details and security features
- Update pfapi sync README with operation log integration info
- Extract `filterNewOps()` to OperationLogStoreService for batch duplicate filtering
- Refactor `detectConflicts()` into smaller helpers: `_checkOpForConflicts()`,
`_checkEntityForConflict()`, `_buildEntityFrontier()`, `_adjustForClockCorruption()`
- Add UUID validation to `_extractTimestampFromUuidv7()` with clear error messages
- Extract `_isValidBackupId()` helper in SyncSafetyBackupService
- Add `hasTask()` method to TaskArchiveService, replace exception-based control flow
- Cache reducer chain in TaskArchiveService to avoid recreation per call
- Add `SYSTEM_TAG_IDS` constant to tag.const.ts for filtering system tags
- Extract `_hasNoUserCreatedTags()` helper using SYSTEM_TAG_IDS
- Add iteration limit (MAX_DOWNLOAD_ITERATIONS) to download loop
- Add user notification for archive operation failures
- Add JSDoc for SyncSafetyBackup interface and backup slot strategy comments
- Use i18n keys for error messages in restore dialog
Add ability to compress archived tasks by:
- Deleting subtask entities and merging timeSpent/timeSpentOnDay to parent
- Clearing notes from tasks older than 1 year
- Clearing non-essential issue fields (keeps issueId and issueType)
The compression operation is synced across clients via the operation log.
Users can trigger compression from Settings → Import/Export with a preview
showing affected tasks and estimated space savings.
When tasks scheduled for today with a time triggered the reminder dialog,
clicking "Add all to today" did not clear remindAt, causing the reminder
to repeatedly pop up. Now remindAt is always cleared when explicitly
adding tasks to today.
Two bugs prevented legacy data from being migrated to operation logs:
1. Migration services called getAllSyncModelData() which uses NgRx store
delegate. Since NgRx store is empty during init, this returned empty data.
Fix: Use getAllSyncModelDataFromModelCtrls() to read from 'pf' database.
2. Effects dispatching actions during app init created orphan operations
before migration ran. The lastSeq > 0 check then returned early, thinking
migration already happened.
Fix: Check for snapshot first, then check for Genesis operation, and
clear orphan operations if found before proceeding with migration.
Show single checkmark when local ops are synced to remote, and double
checkmark (done_all icon) when we also recently checked remote for
updates (within 1 minute). This applies only to Super Sync.
- Add SuperSyncStatusService to track remote check freshness and
pending ops status
- Update OperationLogDownloadService to mark remote checks
- Update OperationLogSyncService to track pending ops status
- Update SyncWrapperService to expose combined status observable
- Update main-header component to conditionally show done_all vs check
When filtering operations after a SYNC_IMPORT, the previous string
comparison `op.id > latestImport.id` could incorrectly filter out
operations created in the same millisecond due to random bits in UUIDv7.
This caused repeatable task instances to not sync when created
immediately after an import operation.
Fix: Extract the 48-bit timestamp from UUIDv7 and use >= comparison
to ensure same-millisecond operations are kept.
When receiving a SYNC_IMPORT operation from another client (e.g., after
a backup import), the archive data (archiveYoung, archiveOld) was only
loaded into NgRx state but never persisted to IndexedDB. On app restart,
IndexedDB had the old data, causing time tracking loss.
Fix adds _handleLoadAllData handler to ArchiveOperationHandler that
writes archive data to IndexedDB for remote SYNC_IMPORT operations.
Fixes: 1594 hours on Client A -> 70 hours on Client B after sync
- Remove ~750 lines of unused file-based sync code from upload/download services
- Delete OperationLogManifest interface (no longer needed)
- Remove manifest service references from integration tests
- Add unit tests for HydrationStateService
- Test TaskArchiveService.updateTask dispatches persistent action
- Test TaskArchiveService.updateTasks dispatches actions for each update
- Test isSkipDispatch option prevents dispatch
- Test isIgnoreDBLock is passed through to save
- Test ArchiveOperationHandler.handleUpdateTask for remote operations
- Test updateTask is now archive-affecting action
TaskArchiveService.updateTask() and updateTasks() now dispatch
persistent actions after saving to IndexedDB, enabling sync.
ArchiveOperationHandler handles remote updateTask operations
by updating the archive with isSkipDispatch to prevent loops.
Fix moveToArchive operations being rejected by server:
- deriveOpType() now uses action's declared opType instead of deriving from entity changes
- moveToArchive declares opType: Update but was being changed to Delete based on entity changes
- Server rejected because DEL operations require singular entityId, but bulk ops use entityIds
Additional sync improvements:
- Use OpLog.warn instead of devError for old snapshot format (expected transitional state)
- Add lock-based synchronization for conflict detection (replaces flushPendingWrites)
- Handle TODAY_TAG correctly when checking if client state is empty
- Add partial apply failure handling with validation and user notification
- Server: Add race condition prevention with REPEATABLE_READ isolation and re-check after seq allocation
- Server: Add concurrent snapshot generation lock to prevent duplicate work
Update tests to use LockService instead of OperationWriteFlushService
When both local and remote operations have the same effect (e.g., both
DELETE the same entity or both UPDATE with identical payloads), the
conflict is now auto-resolved as "remote" without showing the dialog
to the user. This reduces unnecessary user interaction when conflicts
are semantically identical.
- Add isIdenticalConflict() method to detect identical conflicts
- Add _deepEqual() helper for payload comparison
- Modify presentConflicts() to separate and auto-resolve identical conflicts
- Add comprehensive tests for identical conflict detection
Replace vector clock comparison with UUIDv7 timestamp comparison in
_replayLocalSyncedOpsAfterImport. This fixes state divergence after
restore/import operations.
When restoring with isForceConflict=true, a fresh vector clock with a
new client ID is created. Comparing clocks with no overlapping client
IDs resulted in CONCURRENT (not LESS_THAN), causing old ops to be
incorrectly replayed on top of restored state.
UUIDv7 IDs are time-ordered, so op.id < syncImportOp.id correctly
identifies ops created before the restore/import that should be
filtered out.
