super-productivity/docs
Johannes Millan 63d46f7a0b
feat(op-log): validate SQLite backend + IDB→SQLite migration + backend-aware init (#7931) (#7954)
* test(op-log): validate SqliteOpLogAdapter against a real sql.js engine

The 23 adapter specs ran only against an in-memory regex stand-in that
models the SQL shapes the adapter emits — it validates the translation
layer, not SQLite itself. Add sql.js (dev-only; never in the app bundle)
served into Karma, and run the behavioral contract against BOTH the fake
and a real SQLite engine.

This exercises genuine-engine behavior the stand-in could only model:
the real UNIQUE-constraint message -> ConstraintError mapping,
AUTOINCREMENT never reusing seq after clear(), compound-index + NULL
range handling, and real BEGIN IMMEDIATE rollback. 51/51 green.

B2 (translation-layer pass) per docs/sync-and-op-log/sqlite-migration.md.
The integration-harness second pass and the on-device real-engine run
remain.

* test(op-log): run store-port integration against real sql.js (B2 stage 2)

Parameterize the RemoteOperationApplyStorePort integration scenarios to
run against BOTH the default IndexedDB backend and a sql.js-backed
SqliteOpLogAdapter, exercising the store's COMPOSED flows (apply/mark/
merge-clock, partial-failure persistence, full-state import clearing,
vector-clock persistence) on a real SQL engine — not just the adapter in
isolation. 6/6 green.

Surfaced a real B3 wiring gap: OperationLogStoreService.init() is
IDB-shaped (opens+adopts an IndexedDB connection, never calls the
adapter's own init()). For a self-managing backend like SQLite the
tables would not exist. The sql.js setup creates them once on the shared
db to mirror the store-init change B3 must make on native (call
adapter.init() / skip the IDB open when the backend is SQLite).

* feat(op-log): add verified IDB->SQLite backend migration (C1)

One-time copy of the entire op-log from a source adapter (legacy
IndexedDB) to a dest adapter (SQLite) in a single dest transaction with
verify-before-commit: a mismatch in op count, last seq, or vector clock
throws and rolls the dest back, leaving it empty and the source
untouched.

Adapter-agnostic (talks only to the OpLogDbAdapter port), so it is
validated in CI with a real Chrome IndexedDB source + a sql.js SQLite
dest; the native @capacitor-community/sqlite dest behaves identically
through the same port. The generic iterate->put copy preserves ops seq
(incl. gaps) via the put-honors-seq path and writes singletons at their
out-of-line key uniformly, with no per-store special-casing.

Not wired into startup — Phase B3/C2 decide WHEN to run it (SQLite empty
+ legacy SUP_OPS present) and retain the IDB copy >= 1 release. 5/5
green, incl. seq-fidelity, AUTOINCREMENT-continues-past-migrated,
empty-source, non-empty-dest guard, and verify-rollback.

* docs(sync): record sql.js validation, C1, and the B1/B3 findings

Update the SQLite migration plan + follow-up backlog to reflect what
landed this pass and hand off the device-gated remainder:
- B2: real-engine (sql.js) adapter contract + store-port second pass are
  done in CI; only the on-device run remains.
- C1: the backend-migration algorithm + verify-before-commit are done
  and tested (real IDB -> sql.js); only the startup wiring remains.
- B3 finding: OperationLogStoreService.init() is IDB-shaped (opens+adopts
  IDB, never calls adapter.init()); native must call adapter.init() and
  skip the IDB open on SQLite.
- B1 perf note: bridge round-trips dominate on native; return lastId from
  the plugin's run response and add a runBatch/executeSet bulk path so
  appendBatch is one crossing, with RETURNING-seq for per-op seq.

* feat(op-log): make store init backend-aware for self-managing backends (B3)

OperationLogStoreService.init() and ArchiveStoreService._init() were
IDB-shaped: they unconditionally opened+adopted a WebView IndexedDB
connection and never called the adapter's own init(). For a self-managing
backend (SQLite) that meant (a) the adapter's tables were never created
and (b) it still touched the evictable WebView store this migration
exists to escape.

Now: when the adapter exposes no adoptConnection (i.e. it self-manages,
like SQLite), call adapter.init() and skip the IndexedDB open. The
adopt-connection (IndexedDB) path is unchanged. The new branch is dead in
production until B3 flips the native token, so this is risk-free now and
unblocks that flip.

Tested: two unit tests cover both branches (self-managing -> adapter.init,
no IDB open; IDB -> open+adopt, no adapter.init). The store-port
integration spec now drives the store fully on SQLite with _db undefined,
so its earlier pre-init workaround is removed. 521 persistence + 6
integration green.

* docs(sync): mark the B3 backend-aware init fix as landed

The store-init half of B3 (call adapter.init() / skip the IDB open for
self-managing backends) is implemented + CI-tested; only the device-gated
native token flip + SqliteDb wrapper remain.
2026-06-02 17:26:23 +02:00
..
long-term-plans docs(caldav): mark VEVENT two-way as shipped via the bundled plugin (#7879) 2026-06-01 14:31:45 +02:00
plans fix(ios): reconcile time tracking and focus mode on app resume (#7837) 2026-05-28 16:56:54 +02:00
promotion build: better organize stuff 2025-06-27 09:03:16 +02:00
research docs(task-repeat-cfg): revise RRULE migration plan and consolidate to one doc 2026-06-02 16:09:35 +02:00
screens docs/wiki content v0.8 (#7068) 2026-04-01 12:40:28 +02:00
sync-and-op-log feat(op-log): validate SQLite backend + IDB→SQLite migration + backend-aware init (#7931) (#7954) 2026-06-02 17:26:23 +02:00
wiki feat(plugins): fire PERSISTED_DATA_CHANGED hook on persisted-data changes (#7805) 2026-05-26 23:41:01 +02:00
add-new-integration.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
apple-release-automation.md ci: auto-submit iOS and macOS App Store builds for review (#7857) 2026-06-01 11:42:16 +02:00
build-and-publish-notes.md fix(ci): prevent duplicate unsigned exe files in GitHub Releases 2026-03-19 18:58:52 +01:00
documentation-guide.md Housekeeping for various docs, templates, and actions (#7451) 2026-05-02 13:58:25 +02:00
ENV_SETUP.md build: final approach 2025-07-14 20:52:51 +02:00
github-access-token-instructions.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
gitlab-access-token-instructions.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
how-to-rate.md feat(rate-dialog): action-aware prompt with feedback split 2026-05-04 17:27:11 +02:00
howto-refresh-snap-credentials.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
i18n-script-usage.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
legacy-webview-analysis.md 16.2.1 2025-11-01 13:22:58 +01:00
mac-app-store-code-signing-guide.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
performance-project-tag-report.md docs: add performance report 2025-11-13 18:23:50 +01:00
plugin-development.md Android soft-keyboard: fix dark-theme white flash on resize (#7839) 2026-05-28 16:56:21 +02:00
styling-guide.md fix(theme): apply velvet sidenav blur on inner element, not host 2026-05-14 14:41:08 +02:00
theming-contract.md feat(theme): primitive token layer + theme contract validator 2026-05-08 22:31:00 +02:00
TRANSLATING.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
unused-translations-analysis.md chore(i18n): remove additional 86 orphan translation keys 2026-01-09 14:59:05 +01:00
update-android-app.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00
update-mac-certificates.md docs: clean up and organize project documentation 2026-03-10 10:22:56 +01:00