Mirrors/super-productivity
1
0
Fork 0
mirror of https://github.com/johannesjo/super-productivity.git synced 2026-01-23 02:36:05 +00:00
Code Issues Projects Releases Wiki Activity
18269 commits 48 branches 590 tags 336 MiB
Commit graph

16 commits

Author SHA1 Message Date
Johannes Millan
038a722ed8 style: apply prettier formatting 2026-01-10 17:09:14 +01:00
Johannes Millan
9605177fc0 docs: enhance SuperSync vs file-based comparison with detailed diagrams 
Add comprehensive comparison diagrams:
- Side-by-side architecture showing upload/download flows
- Key conceptual differences (source of truth, sequencing, etc.)
- Detailed feature comparison table with "winner" column
- Trade-offs visualization (bandwidth, setup, history, etc.)
- Concurrent edit scenario showing gap detection vs piggybacking
 2026-01-08 11:31:07 +01:00
Johannes Millan
c7bc9d4978 docs: add detailed master architecture diagram for file-based sync 
Replace simple "Complete System Flow" with comprehensive diagram showing:
- Client-side sync loop, download flow, upload flow
- Conflict management with LWW auto-resolution
- First-sync conflict dialog and snapshot hydration
- Piggybacking mechanism on upload
- File provider layer with adapter methods
- Remote storage structure

Also add Quick Reference Tables for:
- File operations (download/upload methods)
- sync-data.json structure
- Key implementation details
 2026-01-08 11:12:42 +01:00
Johannes Millan
9f0adbb95c docs: fix outdated file paths and types in diagrams 
- Fix FileBasedSyncData type: remove non-existent lastSeq, add clientId
- Fix file paths: op-log/processing → op-log/apply
- Fix file paths: features/time-tracking → features/archive
- Fix file path: super-sync not supersync
- Fix vector-clock path: now in op-log/sync/
- Remove non-existent state-capture.meta-reducer.ts reference
- Remove pfapi-migration.service.ts (no longer exists)

docs: remove outdated .bak file references from diagrams

The backup file (sync-data.json.bak) is no longer created during
upload. It's only deleted as cleanup from legacy implementations.

docs: add sync comparison and simple sync flow diagrams

- Add 07-supersync-vs-file-based.md comparing the two sync approaches
- Add 08-sync-flow-explained.md with step-by-step sync explanation
- Remove consolidated unified-oplog-sync-diagrams.md
- Update diagrams README with new entries

docs(sync): reorganize diagrams into subfolder and update for unified architecture

- Create docs/sync-and-op-log/diagrams/ with topic-based diagram files
- Remove outdated PFAPI Legacy Bridge references from diagrams
- Update archive diagrams to use generic "Archive Database" naming
- Fix file paths from sync/providers/ to sync-providers/
- Update quick-reference Area 12 to show unified file-based sync
- Update README to reference new diagram locations

docs: update architecture docs to reflect PFAPI elimination

- Delete obsolete PFAPI documentation:
  - docs/sync-and-op-log/pfapi-sync-persistence-architecture.md
  - docs/sync-and-op-log/pfapi-sync-overview.md
  - docs/plans/pfapi-elimination-status.md

- Update sync-and-op-log/README.md:
  - Describe unified operation log architecture
  - Document file-based sync (Part B) and server sync (Part C)
  - Update file structure to reflect sync-providers location

- Update operation-log-architecture.md:
  - Rewrite Part B from "Legacy Sync Bridge" to "File-Based Sync"
  - Remove all PFAPI code examples and references
  - Update IndexedDB structure diagram (single SUP_OPS database)
  - Update architecture overview to show current provider structure
  - Add notes about PFAPI elimination (January 2026)

- Mark completed implementation plans:
  - replace-pfapi-with-oplog-plan.md - marked as COMPLETED
  - file-based-oplog-sync-implementation-plan.md - marked as COMPLETED

Also includes fix for file-based sync gap detection to handle
snapshot replacement (when "Use Local" is chosen in conflict resolution).
 2026-01-08 11:10:29 +01:00
Johannes Millan
4e49628eca chore: clean up leftover pfapi references after refactor 
- Remove dead test:shard:pfapi script from package.json
- Update AGENTS.md persistence layer path to op-log and sync
- Update documentation file paths in secure-storage.md,
  vector-clocks.md, and quick-reference.md
 2026-01-07 21:10:38 +01:00
Johannes Millan
1f8fe61c84 refactor: integrate pfapi into oplog 1 2026-01-07 13:30:09 +01:00
Johannes Millan
936f374bde refactor(sync): address code review findings for operation log 
Phase 1 - Fix archive circular dependency:
- Create ArchiveDbAdapter for direct IndexedDB access to archive data
- Update ArchiveOperationHandler to use ArchiveDbAdapter instead of
  lazy-injecting PfapiService, breaking the circular dependency chain
- Update tests to mock ArchiveDbAdapter

Phase 2 - Address testing gaps:
- Add 3 timeout exhaustion tests for compaction service (25s timeout)
- Add 5 clock skew edge case tests for conflict resolution:
  - Far future/past timestamps
  - Zero and negative timestamps
  - Client ID tie-breaker for identical timestamps

Phase 3 - Document cross-version migration (A.7.11):
- Add comprehensive implementation guide in architecture doc covering:
  - When to bump CURRENT_SCHEMA_VERSION
  - Operation transformation strategy
  - Conflict detection across versions
  - Backward compatibility guarantees
  - Migration rollout strategy
  - Example migration and testing requirements

All 1399 op-log tests pass.
 2026-01-02 13:48:22 +01:00
Johannes Millan
c29913aeac test(time-tracking): add comprehensive tests and documentation 
Add extensive tests and documentation for TimeTrackingState sync:

Unit tests (~22 new):
- merge-time-tracking-states.spec.ts: partial data merging, null
  handling, merge priority, scale testing
- sort-data-to-flush.spec.ts: flush completeness, LWW behavior,
  date boundaries
- time-tracking.reducer.spec.ts: context preservation, immutability

Integration tests (9 new):
- time-tracking-sync.integration.spec.ts: operation creation,
  SYNC_IMPORT with timeTracking, multi-client sync, archive flush

Server tests (18 new):
- time-tracking-operations.spec.ts: operation upload, conflict
  detection, snapshot generation, LWW semantics

Documentation:
- JSDoc for merge-time-tracking-states.ts and sort-data-to-flush.ts
- Section E.6 "Time Tracking Sync Semantics" in architecture docs
 2025-12-28 13:16:07 +01:00
Johannes Millan
d6dcc86ca0 refactor: reorganize operation-log files into src/app/op-log/ 
Move operation-log from src/app/core/persistence/operation-log/ to
src/app/op-log/ with improved subdirectory organization:

- core/: Types, constants, errors, entity registry
- capture/: Write path (Actions → Operations)
  - operation-capture.meta-reducer.ts
  - operation-capture.service.ts
  - operation-log.effects.ts
- apply/: Read path (Operations → State)
  - bulk-hydration.action.ts/.meta-reducer.ts
  - operation-applier.service.ts
  - operation-converter.util.ts
  - hydration-state.service.ts
  - archive-operation-handler.service.ts/.effects.ts
- store/: IndexedDB persistence
  - operation-log-store.service.ts
  - operation-log-hydrator.service.ts
  - operation-log-compaction.service.ts
  - schema-migration.service.ts
- sync/: Server sync (SuperSync)
  - operation-log-sync.service.ts
  - operation-log-upload.service.ts
  - operation-log-download.service.ts
  - conflict-resolution.service.ts
  - sync-import-filter.service.ts
  - vector-clock.service.ts
  - operation-encryption.service.ts
- validation/: State validation
  - validate-state.service.ts
  - validate-operation-payload.ts
- util/: Shared utilities
  - entity-key.util.ts
  - client-id.provider.ts
- testing/: Integration tests and benchmarks

This reorganization:
- Places op-log at the same level as pfapi for better visibility
- Groups files by responsibility (write path vs read path)
- Makes the sync architecture more discoverable
- Improves navigation for developers new to the codebase
 2025-12-27 17:52:11 +01:00
Johannes Millan
693e71deeb docs(sync): complete quick-reference with Areas 8-12 
Add visual summaries for remaining areas:
- Area 8: Meta-Reducers (atomic multi-entity changes)
- Area 9: Compaction (snapshot + garbage collection)
- Area 10: Bulk Application (single dispatch optimization)
- Area 11: Encryption (AES-256-GCM + Argon2id)
- Area 12: Legacy PFAPI Bridge (WebDAV/Dropbox)

Removes "Coming Soon" placeholder. All 12 areas now documented.
 2025-12-27 17:10:18 +01:00
Johannes Millan
e221afe81c docs(sync): add quick-reference.md with visual summaries 
Create a quick reference document with ASCII diagrams summarizing:
- Area 1: Write Path
- Area 2: Read Path (Hydration)
- Area 3: Server Sync (SuperSync)
- Area 4: Conflict Detection
- Area 5: Conflict Resolution (LWW)
- Area 6: SYNC_IMPORT Filtering
- Area 7: Archive Handling

Includes decision tables, key invariants, and file references.
Areas 8-12 placeholders for future additions.
 2025-12-27 16:42:38 +01:00
Johannes Millan
330ecce2c9 docs(sync): fix LWW timestamp and SYNC_IMPORT filtering sections 
- C.5: Correct "Current timestamp" to "Preserved maximum timestamp
  from local ops" - critical for correct LWW semantics
- C.7: Rewrite "Late-Joiner Replay" as "SYNC_IMPORT Filtering" to
  reflect actual "Clean Slate Semantics" implementation:
  - CONCURRENT ops are now DROPPED, not replayed
  - Uses SyncImportFilterService, not removed _replayLocalSyncedOpsAfterImport()
  - Vector clock comparison determines causality, not UUIDv7 timestamps
 2025-12-27 15:59:00 +01:00
Johannes Millan
8fff1325b4 docs: fix SYNC_IMPORT filtering section to match implementation 
- Fix CLAUDE.md path reference from docs/op-log/ to docs/sync-and-op-log/
- Rewrite section 2c in architecture diagrams to reflect actual implementation:
  - Was: UUIDv7 timestamp replay (removed feature)
  - Now: Vector clock filtering with clean slate semantics
- Update service reference from removed _replayLocalSyncedOpsAfterImport()
  to SyncImportFilterService.filterOpsInvalidatedBySyncImport()
- Clarify that CONCURRENT ops are dropped, not replayed
 2025-12-27 13:15:47 +01:00
Johannes Millan
c1575d7669 docs(sync): update operation-log docs to reflect current implementation 
- Update README.md with correct links and implementation status
- Add E2EE to implementation status (now complete)
- Update e2e-encryption-plan.md to mark as implemented
- Update operation-log-architecture.md:
  - Add missing service files to file reference section
  - Add E2EE to Part C and "Recently Completed" sections
  - Fix broken references, update last modified date
- Remove reference to non-existent tiered-archive-proposal.md
 2025-12-27 11:44:30 +01:00
Johannes Millan
39d6e15446 docs: improve structure 2025-12-27 11:32:54 +01:00
Johannes Millan
7227749ec4 test(sync): add tests for bulk dispatch edge cases 
Add comprehensive tests for operation applier testing gaps:

- Partial archive failure: verifies that when archive handling fails
  midway through a batch, previously processed ops are reported
  as successful while remaining ops are not applied
- Effects isolation: confirms that only bulkApplyOperations action
  is dispatched (not individual action types), which is the key
  architectural benefit preventing effects from firing on remote ops
- Multiple archive-affecting ops: ensures remoteArchiveDataApplied
  is dispatched exactly once when batch contains multiple
  archive-affecting operations

Total: 4 new test cases, 24 tests now passing
 2025-12-27 11:31:51 +01:00