Commit graph

400 commits

Author SHA1 Message Date
johannesjo
3bb07fc6d0 fix(sync): address package review feedback 2026-05-14 12:11:48 +02:00
johannesjo
6797e9eed2 refactor(sync): tighten extracted package surfaces
Combined polish from the post-extraction review:

- sync-core: strip NgRx-shaped types from EntityConfig/EntityRegistry;
  expose host extensions via generic param. Move StateSelector,
  PropsStateSelector, SelectByIdFactory, SelectById, EntityUpdateLike,
  EntityAdapterLike to a new app-side entity-registry-host.types.ts.
- sync-core: mark OpType.SyncImport/BackupImport/Repair as @deprecated;
  hosts should use createFullStateOpTypeHelpers().
- sync-providers: resolve provider.types.ts vs provider-types.ts
  duplication; inline implementation into the dashed canonical name.
- sync-providers: drop unused root barrel and "." export; consumers
  already use focused subpath barrels (/dropbox, /webdav, etc.).
- sync-providers: replace wildcard "@sp/sync-providers/*" tsconfig path
  alias with 11 explicit subpath entries matching package.json exports;
  deep-internal imports now fail at typecheck.
- sync-providers: move @sp/sync-core from dependencies to
  peerDependencies (kept in devDependencies for tests).
- both packages: add composite: true to enable project references;
  introduce tsconfig.build.json overlay so tsup DTS bundler still works.
- gitignore: ignore **/*.tsbuildinfo composite outputs.
2026-05-14 00:13:25 +02:00
Johannes Millan
4e1ace0786 refactor(sync): remove encryption test mocks 2026-05-13 17:26:55 +02:00
Johannes Millan
4b856b3411 refactor(sync-core): extract encryption primitives 2026-05-13 17:26:55 +02:00
Johannes Millan
faa18c1638 docs(sync): add plan to extract encryption primitives to @sp/sync-core 2026-05-13 14:11:15 +02:00
Johannes Millan
1af8538189 docs(sync): mark extraction plan complete 2026-05-13 14:11:15 +02:00
Johannes Millan
00098f52fb refactor(sync-providers): add tiered package exports 2026-05-13 14:11:15 +02:00
Johannes Millan
32ba4d144c perf(sync-providers): reduce webdav xml parser scans 2026-05-13 14:11:15 +02:00
Johannes Millan
5fa8260bb8 refactor(sync): finish package extraction polish 2026-05-13 14:11:15 +02:00
Johannes Millan
e6788c7b31 Merge branch 'feat/i-am-getting-this-for-super-sync-6d324d'
* feat/i-am-getting-this-for-super-sync-6d324d:
  feat(supersync): retry transient web fetch failures and surface them as warnings
2026-05-13 11:53:09 +02:00
Johannes Millan
e4f9a4e2e5 refactor(sync-providers): extract local file provider 2026-05-13 11:36:19 +02:00
Johannes Millan
8005b4ec52 feat(supersync): retry transient web fetch failures and surface them as warnings
Adds a 2-retry / 1s+2s backoff loop to the browser/Electron SuperSync
request path, mirroring the existing native retry. A new typed
NetworkUnavailableSPError replaces the previous string-shape contract
between provider and wrapper: the provider throws the typed error from
both web-retry exhaustion and native-failure handler, and the wrapper
matches via instanceof to show a transient WARNING snackbar
(F.SYNC.S.NETWORK_ERROR) without flipping into a hard ERROR state.

Drops the wrapper-side regex classifier (and its defensive
^HTTP\s+\d{3}\b filter), the now-unused isTransientNetworkError alias,
and dedupes the predicate call inside the web-retry catch. The single
remaining call site for the broad regex (operation-log-upload) now
imports isRetryableUploadError directly under its honest name.

Tests cover the 1s/2s cadence, retry-then-success, retry-exhaustion via
the typed error, and negative paths (AbortError, HTTP-status 5xx,
AuthFailSPError must NOT retry). The error class is added to the
cross-module identity safety net.
2026-05-13 11:20:02 +02:00
johannesjo
058b10033c docs(sync): update extraction plan after supersync slice 2026-05-13 02:04:41 +02:00
johannesjo
c40feef6d2 refactor(sync-providers): move SuperSync provider into package
Move SuperSync's provider class, model, and spec from
`src/app/op-log/sync-providers/super-sync/` into
`packages/sync-providers/src/super-sync/` behind the ports
established by slices 5-6 (logger, platform-info, web-fetch,
native-http-executor, credential-store) plus two new ports for this
slice: `SuperSyncStorage` (narrow, three methods for `lastServerSeq`
persistence) and `SuperSyncResponseValidators` (host-side wrapper
that keeps `@sp/shared-schema` out of the package boundary).

Privacy fixes applied inline per multi-review consensus:

- `AuthFailSPError(reason, body)` now only takes the extracted
  reason — body is no longer retained on `additionalLog`.
- `_handleNativeRequestError` drops the parenthesised
  `(${errorMessage})` interpolation from the user-facing message
  to avoid leaking hostname/URL on native-stack errors.
- Timeout `Error.message` drops the `path` interpolation
  (`excludeClient` clientId was leaking into the thrown message).
- `_extractServerErrorReason` caps the extracted server reason at
  80 chars to defend against future server contract drift.
- Thrown non-2xx errors use a fixed `HTTP <status> <statusText> —
<reason?>` form (web) or `HTTP <status> — <reason?>` (native)
  instead of embedding the raw response body in `Error.message`.

JSDoc invariants pinned on `getEncryptKey`, `getWebSocketParams`,
`_cachedServerSeqKey`, and `deleteAllData` response shape.

Compression switched to direct `@sp/sync-core` import (no
`CompressError` wrapping at SuperSync's call sites; the existing
app-side compression-handler still wraps for
`EncryptAndCompressHandlerService`).

Spec converted Jasmine → Vitest. `TestableSuperSyncProvider`
subclass gone — native-platform routing now tested via injected
`platformInfo` + `nativeHttpExecutor` mocks. Adds a new
`privacy regression: no user content in error/log surface`
describe block driven by an HTTP-error path with a body containing
plausible user content (taskId/title/accessToken).

App-side `super-sync.ts` collapses to a 50-line
`createSuperSyncProvider()` factory (no `extraPath` arg —
SuperSync explicitly ignored it). The `SyncProviderId.SuperSync`
enum stays app-side; `AssertSuperSyncId` conditional pins
enum-vs-constant drift at compile time. `super-sync.model.ts`
becomes a re-export shim. `sync-providers.factory.ts` updated.

`super-sync-restore.service.ts` narrows the package's
`RestorePoint<string>` return through the consensus "narrow at the
app shim" decision (single cast at the call boundary).

Bundle: CJS 95.15 KB / ESM 92.34 KB (up from 76.78 / 74.18). Right
in the performance reviewer's 95-100 KB estimate; under the
original 110 KB projection. Tiered barrel split remains a
documented deferral.

Package spec count: 273 (was 197, added 76 SuperSync specs).
All targeted app specs green: sync-wrapper (107), op-log (2513
across the directory), imex/sync (381).
2026-05-13 00:30:19 +02:00
johannesjo
3d0ff2aec7 docs(sync-core-plan): record SuperSync slice multi-review consensus
Folds the six-Claude-lens (correctness, security/privacy, architecture,
alternatives, performance, simplicity) review pass into the SuperSync
slice design doc. Closes all 8 open questions in-doc, records new
blockers the original draft undercounted, and trims the body per the
simplicity reviewer's recommendations.

Key revisions:
- Storage port is narrow `SuperSyncStorage` (3 methods), not the
  generic `KeyValueStoragePort` originally proposed. Architecture
  and alternatives both pushed back.
- Promoted helper renamed to `isRetryableUploadError` (intent-
  anchored) to avoid naming collision with package's existing
  `isTransientNetworkError`.
- Factory drops `extraPath` — SuperSync explicitly ignores it.
- Privacy A1 fix uses extracted-reason form (via
  `_extractServerErrorReason`, capped at 80 chars), not blanket
  body-drop — preserves 5xx debug context.

Four new privacy blockers added that the original sweep missed:
- `AuthFailSPError(reason, body)` retains body via
  `AdditionalLogErrorBase.additionalLog` (security + correctness
  both flagged independently — PR 5b did NOT scrub this for
  SuperSync's call site).
- `_handleNativeRequestError` re-throw embeds raw `errorMessage`
  (can leak hostname/URL from native-stack `.message`).
- Timeout `Error.message` embeds `path` with `excludeClient` query
  param (pseudonymous clientId).
- `_extractServerErrorReason` returns server `error` field uncapped.

Dissents recorded: alternatives reviewer preferred pre-split spec
before move, moving CompressError/DecompressError into sync-core
for strict behavior preservation, and barrel split as PR 7d.
2026-05-12 23:58:58 +02:00
johannesjo
6a1f569a09 docs(sync-core-plan): draft SuperSync slice design for multi-review
Captures the SuperSync provider move (slice 7) design surface before
implementation: which files move into @sp/sync-providers, which stay
app-side, which ports get reused from prior slices, which new ports
this slice introduces, the privacy sweep checklist, the suggested
commit shape, and the open questions for parallel multi-review.

Key design calls flagged for review:
- response-validators.ts stays app-side (banned @sp/shared-schema
  import). New SuperSyncResponseValidators port.
- localStorage access becomes a KeyValueStoragePort.
- isTransientNetworkError promoted from app sync-error-utils to the
  package as isTransientErrorMessage (distinct from the package's
  existing native-error-code-aware version).
- Compression imported directly from @sp/sync-core; CompressError
  wrapping dropped at SuperSync call sites (no instanceof catches
  exist).
- Two privacy A1 sites identified: _doNativeFetch:646-648 and
  _doWebFetch:582 embed response body in thrown Error.message; fix
  via fixed status-only message.
- Spec migration kept monolithic for the move (1553 lines), split
  deferred to a follow-up.

Multi-review consensus block left blank for the reviewer pass.
2026-05-12 23:50:12 +02:00
johannesjo
173218c151 docs(sync-core-plan): summarize Sixth Slice and renumber remaining
Folds the WebDAV + Nextcloud slice notes into the long-term plan in
the same format as prior slices and renumbers the Remaining Slice
Plan so SuperSync becomes slice 1 (next) and LocalFile slice 2.

Captures the consensus-driven decisions from the WebDAV slice
(port reuse instead of new `WebDavNativeHttpExecutor`, no-retry
preservation, monolithic spec move, Nextcloud generic widened,
`md5HashSync` → `hash-wasm` async, CORS heuristic tightened,
inline `registerPlugin` dropped) plus follow-up commit history
(namespace/server-format specs, xmldom devDeps move, CORS pattern
broadening).

Records the bundle-size delta (75.77 KB CJS / 73.23 KB ESM after
the slice) and the architectural deferrals (parser O(n) walk,
LocalFile's `md5HashPromise`).

Includes a more concrete plan for the SuperSync slice: port reuse,
response-validators staying app-side because of the
\`@sp/shared-schema\` boundary ban, localStorage storage port,
\`isTransientNetworkError\` promotion path, compression direct
\`@sp/sync-core\` import, and the privacy-sweep starting points.
2026-05-12 23:47:05 +02:00
Johannes Millan
cf4fbd22e3 docs(sync-core-plan): fold Gemini review dissent into consensus
Gemini's review eventually completed after a workspace-sandbox retry
and quota throttling. Its findings broadly affirm the Claude
consensus, with two dissents — both rejected with reasoning:

- Q1 (port reuse): Gemini wanted to keep the new WebDavNativeHttpExecutor
  port "for consistency with NativeHttpExecutor." Rejected — the
  architecture and simplicity reviewers verified by code reading that
  the existing NativeHttpExecutor already supports arbitrary methods,
  responseType: 'text', and maxRetries: 0. Naming consistency is a
  weak argument against actual port duplication.

- Q6 (retry policy): Gemini wanted a 2-attempt + explicit 423 Locked
  retry. Rejected — alternatives and simplicity flagged this as a
  behavior change masquerading as a refactor. Adapter has zero retries
  today; preserving that keeps the slice scope a move. Per-call-site
  maxRetries remains trivially available once we reuse the existing
  port.

Refresh the consensus header from "Claude-only consensus" to reflect
that Gemini did contribute.
2026-05-12 22:22:41 +02:00
Johannes Millan
9a00152c22 docs(sync-core-plan): record WebDAV slice multi-review consensus
Add a "Multi-review consensus (2026-05-12)" section between the goal
and the what-moves description, mirroring the Dropbox slice doc's
structure. Captures findings from four Claude reviewers (security,
architecture, alternatives, simplicity). Codex / Copilot / Gemini CLIs
were attempted but failed for environment reasons (sandbox, deny-tool
classifier, quota+workspace limits); noted in the consensus header.

Decisions revised:

- Open question 1: DROP the new WebDavNativeHttpExecutor port. Reuse
  NativeHttpExecutor — the existing port already supports responseType:
  'text', maxRetries: 0, and arbitrary methods (PROPFIND/MKCOL/MOVE).
  Auto-JSON-parse is a property of CapacitorHttp.request, not the port.
  The app injects a different adapter (wired to WebDavHttp plugin) of
  the same port. Commit 2 collapses to "wire app-side adapter."
- Open question 4: Drop inline registerPlugin in this slice — the
  canonical registration in capacitor-webdav-http/index.ts carries the
  web: () => import('./web') fallback; the inline one in
  webdav-http-adapter.ts:31 does not.
- Open question 5: Tighten CORS heuristic in this slice. Collapse the
  string-matching to a ~3-line "TypeError && message.includes('cors')"
  check and replace the ambiguous-error log with toSyncLogError +
  urlPathOnly meta. Closes a privacy leak (Firefox NetworkError embeds
  the URL) and removes 40 lines.
- Open question 6: Preserve no-retry behavior. Under open-question-1's
  port reuse, becomes a per-call-site maxRetries: 0 argument.
- Open question 8: Keep webdav-api.spec.ts monolithic. Match Dropbox
  precedent (~876 lines, single file move).
- Commit shape: Match Dropbox 5a/5b split. PR 6a = log helpers
  promotion; PR 6b = bulk move + adapter wiring + privacy sweep +
  Nextcloud generic widen + md5HashSync → hash-wasm + CORS tighten +
  spec migration.
- Factory shape: createWebdavProvider(extraPath?: string), not (deps).
  Deps are composed internally from app singletons, matching the
  Dropbox precedent at app-side dropbox.ts:31-43.

Decisions affirmed: md5HashSync → hash-wasm async (with one-line
benchmark in PR description), Nextcloud generic widening to union,
delete TestableWebDavHttpAdapter spec harness.

New privacy blockers surfaced (must fix in PR 6b): URL/basePath leak
via _buildFullPath at four error-construction call sites, PROPFIND
response body fed into HttpNotOkAPIError (contains user filenames),
testConnection returning raw e.message, _buildFullPath throwing
generic Error with raw path, A3 sweep undercount (10+ raw-error log
sites), new B3.4 invariant (FileMeta never enters a Log call site),
and a documented package-boundary invariant that response headers
are not logged or attached to errors.

Action items: PR 6a (helpers promotion) → PR 6b (bulk move). The
original "Open questions" section is preserved below the consensus
as a decision-log for review history, in the same style as the
Dropbox slice doc.
2026-05-12 22:22:41 +02:00
Johannes Millan
bd6863ac84 docs(sync-core-plan): draft WebDAV slice design for multi-review
Pre-implementation design doc for PR 5's WebDAV + Nextcloud slice,
mirroring the structure of the Dropbox slice design doc.

Captures the file-move surface (9 source files + specs), the new
WebDavNativeHttpExecutor port shape (callable type, divergent from
NativeHttpExecutor because of Capacitor WebDavHttp plugin transport
differences), the md5HashSync → hash-wasm migration choice, the
errorMeta / urlPathOnly log-helper promotion as a precursor commit,
the privacy A1/A3/B3.x sweep checklist, and the Nextcloud generic-
parameter / cast cleanup option.

Includes eight open questions framed for multi-review (port naming,
md5 strategy, generic parameter, registerPlugin cleanup scope, CORS
heuristic, retry policy, spec migration, file split). Suggested
three-commit shape (helpers promotion → port introduction → bulk
move) keeps each commit independently green.

No code changes; design doc only.
2026-05-12 22:22:41 +02:00
Johannes Millan
745551cacb docs(sync-core-plan): summarize Fifth Slice and renumber remaining
Add a "Current Fifth Slice" section mirroring the Current First through
Fourth Slice summaries, capturing PR 5a (provider error classes), PR 5b
(Dropbox provider proper), and the post-review cleanup pass.

Renumber the Remaining Slice Plan now that the Dropbox slice has shipped:
WebDAV + Nextcloud is now slice 1 (next), SuperSync slice 2, LocalFile
slice 3. The WebDAV entry calls out the WebDavNativeHttpExecutor port
shape, the errorMeta / urlPathOnly log-helper promotion, and the
A1/A3/B3.x privacy sweep carry-over.

Refresh the top-of-doc Status header so the shipped-vs-remaining
inventory matches the slice summaries.
2026-05-12 22:22:41 +02:00
Johannes Millan
91e53bb488 refactor(sync-providers): move provider error classes
Lift 12 provider-shared error classes (AuthFailSPError, InvalidDataSPError,
HttpNotOkAPIError, NoRevAPIError, RemoteFileNotFoundAPIError,
MissingCredentialsSPError, MissingRefreshTokenAPIError,
TooManyRequestsAPIError, UploadRevToMatchMismatchAPIError,
PotentialCorsError, RemoteFileChangedUnexpectedly, EmptyRemoteBodySPError)
plus AdditionalLogErrorBase and extractErrorMessage into
@sp/sync-providers. App-side sync-errors.ts becomes a re-export shim
so existing call sites and instanceof checks keep working.

The moved AdditionalLogErrorBase drops its constructor-time
OP_LOG_SYNC_LOGGER.log side effect (Option A from the slice design):
privacy responsibility shifts entirely to catch-site logging via the
injected SyncLogger port. A new app-side identity spec asserts the
constructor identity is preserved across import paths so future bundler
or tsconfig drift can't silently break instanceof catches.

HttpNotOkAPIError splits its parsed body excerpt off .message onto a
new opt-in .detail field; getErrorTxt forwards .detail to UI surfaces
so user-visible toasts remain unchanged while privacy-aware logger
paths see only "HTTP <status> <statusText>".

TooManyRequestsAPIError's constructor is narrowed to accept only
{ status, retryAfter?, path? } — closing a latent bearer-token leak
where Dropbox's _handleErrorResponse passed the raw Authorization
header through additionalLog. Callers in dropbox-api and
webdav-http-adapter updated accordingly.

Package gains "sideEffects": false to unlock tree-shaking through the
barrel for consumers that import only error classes.

Slice design and round-2 multi-review findings documented in
docs/plans/2026-05-12-pr5-dropbox-slice.md.
2026-05-12 20:32:08 +02:00
Johannes Millan
1fcb0d8d83 docs(sync-providers): document fourth slice and revised remaining plan
Capture what landed in the native-HTTP-retry slice and split the
deferred Dropbox + WebDAV provider moves into their own slices, with
the additional package ports (provider error classes, platform-info,
fetch-provider, WebDAV native HTTP) called out explicitly so the
shape can be designed and reviewed before the next slice ships.
2026-05-12 19:24:12 +02:00
Johannes Millan
f1f5b67789 docs(sync-core-extraction): add PR 7 optional polish section
Captures non-blocking follow-ups surfaced during the PR 5 provider lift:
PKCE dedup, dead-arg removal, test tightening, deprecated-alias retirement,
and eslint pattern cleanup.
2026-05-12 19:14:12 +02:00
Johannes Millan
6aac5c07de docs(sync-providers): plan remaining p5 slices 2026-05-12 19:14:12 +02:00
Johannes Millan
0a891d5c36 refactor(sync-providers): move pkce helper 2026-05-12 19:14:12 +02:00
Johannes Millan
6c777d5c20 refactor(sync-providers): move file sync envelope types 2026-05-12 19:14:12 +02:00
Johannes Millan
a97a15457b refactor(sync-providers): scaffold provider package 2026-05-12 19:14:12 +02:00
Johannes Millan
298928e6d2 refactor(sync-core): prepare core boundary for providers 2026-05-12 16:37:12 +02:00
Johannes Millan
debdedacc0 fix(electron): support today filter in local REST API 2026-05-12 16:33:59 +02:00
Johannes Millan
d4d3395c54 refactor(sync): address extraction review findings 2026-05-12 16:33:59 +02:00
Johannes Millan
4758464bd7 refactor(sync): sanitize validation failure logging 2026-05-12 16:33:59 +02:00
Johannes Millan
01d4e6be6f refactor(sync): sanitize menu tree repair logging 2026-05-12 16:33:59 +02:00
Johannes Millan
78f3a1e180 refactor(sync): sanitize typia auto-fix logging 2026-05-12 16:33:59 +02:00
Johannes Millan
8ccba12fda refactor(sync): prove ui and config port adapters 2026-05-12 16:33:59 +02:00
Johannes Millan
fd5a0b6945 refactor(sync-core): add ui and config port contracts 2026-05-12 16:33:59 +02:00
Johannes Millan
a59aa39931 feat(tasks): add notes panel shortcut 2026-05-12 12:48:45 +02:00
johannesjo
33099ec617 refactor(sync): continue logger and registry cleanup 2026-05-12 00:52:50 +02:00
johannesjo
6d4c69034b docs(sync): update sync-core extraction status 2026-05-12 00:52:50 +02:00
johannesjo
0c852c4cf4 refactor(sync): extract snapshot hydration planning 2026-05-12 00:52:50 +02:00
johannesjo
610fbc1c75 refactor(sync-core): extract download planning helpers 2026-05-11 23:27:45 +02:00
johannesjo
3c06157324 refactor(sync-core): extract upload planning helpers 2026-05-11 23:23:05 +02:00
johannesjo
8d897d461c refactor(sync-core): move remote apply coordinator behind ports 2026-05-11 23:17:13 +02:00
johannesjo
bc437c7881 test(sync-core): add port contract adapter coverage 2026-05-11 23:08:20 +02:00
johannesjo
1c337866d6 refactor(sync-core): add initial orchestration port contracts 2026-05-11 22:46:20 +02:00
johannesjo
b1512eb335 refactor(sync): route sync error diagnostics through sync logger 2026-05-11 22:45:05 +02:00
Johannes Millan
416d95161f refactor(sync-core): extract compression helpers 2026-05-11 21:41:02 +02:00
Johannes Millan
0b40a8bedb refactor(sync): extract sync file prefix helpers 2026-05-11 21:21:25 +02:00
Johannes Millan
d2be63c9c4 refactor(sync-core): configure full-state op classification 2026-05-11 20:36:48 +02:00
Johannes Millan
ba838eccf6 refactor(sync-core): extract sync import filter decision 2026-05-11 18:47:53 +02:00