Commit graph

725 commits

Author SHA1 Message Date
johannesjo
3bb07fc6d0 fix(sync): address package review feedback 2026-05-14 12:11:48 +02:00
johannesjo
f0706ac779 style(sync-core): format sync-file-prefix.spec import line 2026-05-14 01:11:37 +02:00
johannesjo
257c86d8e8 test(sync): pin vector-clock pruning, error-meta privacy, and sync-import edges
Fills three test gaps surfaced by the post-extraction review:

- vector-clock pruning correctness across clocks: 4 cases pinning that
  pruning legitimately flips GREATER_THAN to CONCURRENT/LESS_THAN when
  the dropped keys are still present in the comparison clock. This is
  the documented behavior (compareVectorClocks is intentionally not
  pruning-aware); the protocol handles flips server-side via the
  rejected-ops retry loop. preserveClientIds case also covered.

- error-meta privacy boundary: 22 new cases covering urlPathOnly (strip
  query/fragment/userinfo, preserve host+path+port, leave non-URLs
  intact) and errorMeta (no leakage of headers, response bodies, OAuth
  tokens, signed-URL params, user emails, or attached error fields).
  Real negative assertions (.not.toContain), not shape checks.

- sync-import-filter edge cases: 8 cases covering empty clocks on
  either side, op clock listing the import client at 0, same-client
  with equal counter (pinning the strict-greater-than boundary), and
  different-client knowledge above the import counter.

sync-core 195 -> 207 tests, sync-providers 319 -> 341 tests; no
production code changed.
2026-05-14 01:02:24 +02:00
johannesjo
3640e23d26 test(sync-providers): extract shared test helpers and prefer barrels
Adds tests/helpers/sync-logger.ts and tests/helpers/credential-store.ts
to centralize the noopLogger and CredentialStore mocks that were copy-
pasted across 8 spec files. createStatefulCredentialStore covers the
"load/upsert/clear with state" cases; createMockCredentialStore covers
bare vi.fn() ports. Spec sites that needed a unique mockResolvedValue
chain it after the helper, preserving behavior 1:1.

Also migrates 5 spec files from deep ../src/<file> paths to the
matching sub-barrel (../src/webdav, /http, /super-sync, /platform) for
symbols already exported there. No new barrel exports added — internal
types (WebDavHttpAdapter, WebdavApi, DropboxApi, etc.) stay on deep
paths because they are intentionally not part of the public surface.

super-sync.spec.ts keeps its own credential/logger mocks (special
__asPort wrapper and vi.spyOn against the live NOOP_SYNC_LOGGER) that
the generic helpers cannot reproduce without bloat.
2026-05-14 00:56:46 +02:00
johannesjo
813978becc refactor(sync-providers): decouple SuperSync provider from SP-specific host
Two coupling leaks the package shouldn't carry:

1. SUPER_SYNC_DEFAULT_BASE_URL was an implicit fallback inside
   SuperSyncProvider — an SP-specific URL baked into a "framework-
   agnostic" package. Make defaultBaseUrl a required SuperSyncDeps
   field; the host factory supplies the SP default. The constant stays
   exported as a suggested default for hosts targeting the SP-hosted
   server.

2. Consumers that wanted the WebSocket path had to do
   `provider as unknown as SuperSyncProvider` to call
   getWebSocketParams. Introduce SuperSyncWebSocketAccess interface +
   isSuperSyncWebSocketAccess structural guard; SuperSyncProvider
   implements it. sync-wrapper.service drops its cast in favor of the
   guard.

super-sync-restore.service still casts to SuperSyncProvider for the
restore path — same pattern would solve it, but out of scope here.
2026-05-14 00:48:27 +02:00
johannesjo
01f9dcf502 refactor(sync-core): drop redundant OperationStorePort
OperationStorePort overlapped with RemoteOperationApplyStorePort on the
two state-transition methods (markSynced/markApplied,
markRejected/markFailed) and had zero non-structural consumers — the
only implementer was OperationLogStoreService, which already exposes
the three methods as its own public surface. Removing the port leaves
the service contract intact and removes the verb-pair confusion noted
in the post-extraction review.

Spec contract test still drives the same state transitions; only the
local typing of the test fixture changes from the deleted interface to
Pick<OperationLogStoreService, ...>.
2026-05-14 00:41:26 +02:00
johannesjo
ab2d3c5234 refactor(sync): split oversized super-sync and conflict-resolution
sync-providers: extract request-ID hashing from super-sync.ts (1017 ->
918 lines) into a new request-id.ts. The helpers were free functions
already in disguise (none referenced this), so the move is mechanical.
HTTP plumbing (_doWebFetch/_doNativeFetch/_fetchApi*) stays as private
methods — it transitively touches 12 instance members and would need
either a wide context object or a separate http-client collaborator
class to extract cleanly. Left as a follow-up.

sync-core: split conflict-resolution.ts into three cohesive files:
- entity-frontier.ts now owns buildEntityFrontier and
  adjustForClockCorruption (per-entity vector-clock domain).
- extractEntityFromPayload and extractUpdateChanges move to
  operation.types.ts next to the existing extractActionPayload.
- conflict-resolution.ts keeps deep-equality, LWW planning,
  partitioning, and identical-conflict detection.

Public barrel exports unchanged; tests now import the moved symbols
from their new homes.
2026-05-14 00:37:19 +02:00
johannesjo
5089b8d987 fix(sync-providers): bound dropbox token refresh to single retry; share md5 rev helper
The five hand-rolled token-refresh blocks in Dropbox.{getFileRev,
downloadFile, uploadFile, removeFile, listFiles} recursed on themselves
after refresh. If the post-refresh call still saw a token error (real
case: the refresh token itself was revoked), the recursion would not
terminate. Consolidated into a single _withTokenRefresh helper that
attempts the call, refreshes once on a token error, retries once, then
lets the outer 401 classifier surface AuthFailSPError.

Same log message, same _isTokenError discriminator, same refresh call.
Same five sites still apply their post-call non-token error mapping
(NoRev, InvalidData, RemoteFileNotFound, path-not-found swallow, etc.).

Also extracts md5 content-rev computation duplicated between
LocalFileSyncBase._getLocalRev and WebdavApi._computeContentHash into a
shared file-based/content-rev.ts; both call sites preserve their own
error wrapping at the boundary.
2026-05-14 00:28:20 +02:00
johannesjo
b50fb7fc49 refactor(sync-core): drop unused encryption migration path
decryptWithMigration and DecryptResult had no host consumer; they
exposed a structural-migration entry point ("here is your ciphertext
re-encrypted under Argon2id") that nothing in the codebase reads. The
side-channel setLegacyKdfWarningHandler — which IS used — stays.

encryptWithDerivedKey/decryptWithDerivedKey lose their export keyword
and remain as module-internal helpers; encrypt/decrypt/encryptBatch/
decryptBatch still call them. Wire format and legacy-fallback semantics
are unchanged, so existing ciphertext continues to decrypt.

Test imports for compression and sync-file-prefix specs now go via
their source files instead of the trimmed barrel.
2026-05-14 00:22:27 +02:00
johannesjo
25410c4005 refactor(sync-core): prune 47 unused barrel exports
Removes exports with zero consumers outside the package. Source files
are unchanged; only the public barrel is trimmed. Covers compression
helper classes, sync-file-prefix error/config types, replay coordinator
internals, remote-apply result types, upload/download planning option
and plan types, ports misc, conflict-resolution helper types, and
sync-import-filter decision types.
2026-05-14 00:15:58 +02:00
johannesjo
6797e9eed2 refactor(sync): tighten extracted package surfaces
Combined polish from the post-extraction review:

- sync-core: strip NgRx-shaped types from EntityConfig/EntityRegistry;
  expose host extensions via generic param. Move StateSelector,
  PropsStateSelector, SelectByIdFactory, SelectById, EntityUpdateLike,
  EntityAdapterLike to a new app-side entity-registry-host.types.ts.
- sync-core: mark OpType.SyncImport/BackupImport/Repair as @deprecated;
  hosts should use createFullStateOpTypeHelpers().
- sync-providers: resolve provider.types.ts vs provider-types.ts
  duplication; inline implementation into the dashed canonical name.
- sync-providers: drop unused root barrel and "." export; consumers
  already use focused subpath barrels (/dropbox, /webdav, etc.).
- sync-providers: replace wildcard "@sp/sync-providers/*" tsconfig path
  alias with 11 explicit subpath entries matching package.json exports;
  deep-internal imports now fail at typecheck.
- sync-providers: move @sp/sync-core from dependencies to
  peerDependencies (kept in devDependencies for tests).
- both packages: add composite: true to enable project references;
  introduce tsconfig.build.json overlay so tsup DTS bundler still works.
- gitignore: ignore **/*.tsbuildinfo composite outputs.
2026-05-14 00:13:25 +02:00
Johannes Millan
0b21524e29 Merge branch 'feat/production-sync-server-is-full-of-these-b7ab49'
* feat/production-sync-server-is-full-of-these-b7ab49:
  fix(supersync): dedupe WS connections by clientId to evict stale sockets
2026-05-13 19:59:40 +02:00
Johannes Millan
ef4cbff44b Merge branch 'feat/do-a-complete-review-of-the-extration-66f89a'
* feat/do-a-complete-review-of-the-extration-66f89a:
  fix(sync): correct error class names, JSDoc, and missed cache-clear
2026-05-13 19:59:36 +02:00
Johannes Millan
70dd6f9eca build: ignore files 2026-05-13 19:59:02 +02:00
Johannes Millan
87f092bee3 fix(sync): correct error class names, JSDoc, and missed cache-clear
Findings from a multi-agent review of the recent sync extraction:

- Seven error classes in @sp/sync-providers shipped with a leading
  space in `name`, and `UploadRevToMatchMismatchAPIError` was further
  truncated to ' UploadRevToMatchMismatchAP'. Consumers use instanceof
  so runtime behavior was preserved, but stack traces, error envelopes,
  and structured-log meta carried the broken names. Added a regression
  test asserting instance.name matches ErrCtor.name for all 14 classes.
- @sp/sync-core decryptBatch JSDoc claimed Argon2 errors are never
  silently masked as legacy fallbacks, contradicting the actual
  catch-and-decryptLegacy path. The fallback is part of the public
  wire-format contract; rewrote the comment to match the implementation
  and reference the module-level wire-format spec.
- SuperSyncEncryptionToggleService.enable/disableEncryption promised
  "Clear cache on success" but never called clearSessionKeyCache().
  Added the call on the success path in both methods, matching the
  pattern used by EncryptionPasswordChangeService and
  FileBasedEncryptionService.
- Removed packages/sync-core/tests/ports.spec.ts — 57 LOC of
  vi.fn().mockResolvedValue type-assertion tests with no production
  code under test. Port shapes are enforced at the host via
  `implements` clauses with real behavioral coverage in sibling specs.
2026-05-13 19:49:26 +02:00
Johannes Millan
85c7543f49 fix(supersync): dedupe WS connections by clientId to evict stale sockets
A reconnecting device kept appending alongside its stale entry instead of
replacing it, so userSet would fill to MAX_CONNECTIONS_PER_USER and reject
legitimate reconnects with 4008 until the 30s+10s heartbeat cycle caught up.
addConnection now evicts any existing entry with the same clientId before
adding the new socket; the cap still applies to genuinely distinct clientIds.
2026-05-13 19:48:43 +02:00
Johannes Millan
a0309ac21b Merge branch 'feat/do-a-very-careful-review-of-all-our-d21bcb'
* feat/do-a-very-careful-review-of-all-our-d21bcb:
  fix(supersync): guard snapshot retry idempotency lookup by userId
2026-05-13 17:30:17 +02:00
Johannes Millan
3a56d9595b fix(supersync): guard snapshot retry idempotency lookup by userId
The DUPLICATE_OPERATION->success conversion in the snapshot route looked
up the existing op by primary key alone. `isSameDuplicateOperation`
already enforces ownership upstream, so this can't be tripped today, but
keeping the userId filter on the route's own lookup keeps the
idempotency conversion correct even if that upstream invariant ever
changes. Switches `findUnique({id})` to `findFirst({id, userId})`;
`id` is still the primary key so the plan is unchanged.
2026-05-13 17:29:43 +02:00
Johannes Millan
18cae275f5 fix(sync): harden encryption batching and SuperSync abort timer
- super-sync: keep the 75s abort timer alive across response.text() on
  non-OK responses so a stalled error body still triggers AbortError.
- decryptBatch: hold derived keys in a batch-local map. Previously
  Phase 3 read from the LRU session cache, which could evict entries
  mid-batch when the input contained more unique salts than the cache
  could hold (SESSION_DECRYPT_CACHE_MAX_SIZE = 100), crashing on the
  non-null assertion. Adds a 120-item regression test.
- session cache: replace the 32-bit djb2 password identifier with a
  length-prefixed full-password key (injective). A djb2 collision
  silently returned a key derived from a different password, producing
  undecryptable ciphertext on subsequent encrypts.
- docs: bless salt-per-session-per-password semantics explicitly in the
  encryption.ts JSDoc and the sync-core README so future readers and
  tests know IV uniqueness — not salt uniqueness — is the AES-GCM
  invariant being preserved.
2026-05-13 17:26:55 +02:00
Johannes Millan
4e1ace0786 refactor(sync): remove encryption test mocks 2026-05-13 17:26:55 +02:00
Johannes Millan
1d08cb9bc4 refactor(sync): split encryption primitives 2026-05-13 17:26:55 +02:00
Johannes Millan
4b856b3411 refactor(sync-core): extract encryption primitives 2026-05-13 17:26:55 +02:00
Johannes Millan
4bbcafbbab chore(supersync): warn when full-state aggregate scan approaches tx timeout
The aggregate now runs inside the 60s upload transaction. Typical histories
complete in <100ms, but pathological cases (millions of ops, long cleanup
retention) could approach the limit. Mirror the existing >5s slow-aggregate
warning from OperationDownloadService so production logs surface the issue
before it becomes a hard timeout.
2026-05-13 17:01:39 +02:00
Johannes Millan
40ebce57a8 fix(supersync): aggregate prior clocks for full-state ops and deterministic retry lookup
Two correctness fixes on top of the persisted-full-state-clock optimization.

1. Persist the aggregate of prior history (merged with the snapshot op's own
   clock) instead of just the snapshot op's clock. BACKUP_IMPORT uses a fresh
   `{ newClient: 1 }` clock by design, so storing it raw left downloading
   clients with a baseline that did not cover pre-import ops still alive in
   the server-side conflict-detection set; their first post-restore edit could
   be rejected as CONCURRENT. The aggregate scan moves from per-download to
   per-snapshot, which is a net win — full-state ops are rare.

2. Look up the SYNC_IMPORT idempotency target by exact opId first, then fall
   back to an ordered (`serverSeq DESC`) findFirst. Postgres ordering is
   undefined without ORDER BY, so the previous `findFirst` could non-
   deterministically return a different full-state op (e.g. a later
   BACKUP_IMPORT) and flip the idempotency check incorrectly.
2026-05-13 17:01:39 +02:00
Johannes Millan
4f15df8f40 feat(supersync): persist full-state vector clock and add snapshot retry idempotency
Two related improvements to snapshot/full-state handling:

- Persist the incoming op's vector clock as `user_sync_state.latest_full_state_vector_clock`
  so downloads can skip the expensive aggregate scan when the snapshot clock is
  still valid; falls back to the legacy aggregate when missing or invalid.
- Treat retried snapshot uploads (same opId) as idempotent successes instead of
  409/DUPLICATE_OPERATION so a dropped response doesn't push clients into the
  download-and-merge path. Per-namespace request dedup keeps ops and snapshot
  caches isolated.
2026-05-13 17:01:39 +02:00
Johannes Millan
25d2407e70 fix(supersync): mark for reconcile on first successful cleanup batch
Previously, the storage-counter reconcile marker was set only after the
per-user drain loop finished. With the new multi-batch flow, a transient
DB failure between batch 1 and batch 2 commits batch 1's deletes but
exits before the marker call — leaving the counter stale-high until the
next daily pass or process restart, instead of self-healing on the next
sync request.

Move the marker (and affectedUserIds push) to fire after the first
successful batch so any survivor deletes are reconciled. Cover with a
test that spies the private batch helper to throw on the second call
and asserts the user is still marked.
2026-05-13 14:11:15 +02:00
Johannes Millan
2123ede018 revert(supersync): drop CLEANUP_INITIAL_DELAY_MS env var and 30-min prod default
The throttle in deleteOldSyncedOpsForAllUsers now caps each cleanup pass to
short, bounded batches, so the original "shield warmup from DB-heavy retention
work" rationale no longer applies. A long initial delay only creates a
starvation risk on frequently-restarted deployments (rolling deploys or crash
loops could land inside the 30-min window every time and skip cleanup
entirely).

Revert to a fixed 10s warmup and remove the now-pointless knob from envs, the
compose file, and the Helm chart.
2026-05-13 14:11:15 +02:00
Johannes Millan
15a5b8d91e perf(supersync): throttle daily old-ops cleanup so it cannot stall the server
Daily retention cleanup previously issued one unbounded `deleteMany` per user
that covered every operation older than the retention cutoff. On large
backlogs this monopolized Postgres and made user-facing sync slow during the
window, which matched the production "sync is still slow" reports.

- Cap per-batch deletes (default 5k) and per-run total deletes (default 25k),
  with a select-then-delete-by-id pattern that keeps each statement short.
- Drain each user across batches before moving on so a fresh user with a tiny
  backlog isn't blocked behind a large one until next pass.
- Order candidates by `snapshotAt asc` so the stalest users win the budget
  when it's tight.
- Defer the initial cleanup pass after startup to 30 min in production (was
  10s) so retention work doesn't compete with deploy/restart warmup; call
  `unref()` on cleanup timers so they never hold the process open.
- Expose all knobs via env vars wired through docker-compose, env examples,
  and the Helm chart, with a strict positive-integer parser and clamped
  maxima so a misconfiguration cannot unwind the bound.
2026-05-13 14:11:15 +02:00
Johannes Millan
00098f52fb refactor(sync-providers): add tiered package exports 2026-05-13 14:11:15 +02:00
Johannes Millan
32ba4d144c perf(sync-providers): reduce webdav xml parser scans 2026-05-13 14:11:15 +02:00
Johannes Millan
8dd188b054 fix(sync): harden SuperSync downloads and websocket connects 2026-05-13 14:11:15 +02:00
Johannes Millan
f71fc0c543 fix(calendar): request verified Google OAuth scopes
Refs #7221
2026-05-13 14:11:15 +02:00
Johannes Millan
5fa8260bb8 refactor(sync): finish package extraction polish 2026-05-13 14:11:15 +02:00
Johannes Millan
e6788c7b31 Merge branch 'feat/i-am-getting-this-for-super-sync-6d324d'
* feat/i-am-getting-this-for-super-sync-6d324d:
  feat(supersync): retry transient web fetch failures and surface them as warnings
2026-05-13 11:53:09 +02:00
Johannes Millan
8475d79f3f Merge branch 'feat/look-for-additional-super-sync-server-dae67c'
* feat/look-for-additional-super-sync-server-dae67c:
  perf(super-sync): optimize status and conflict checks
2026-05-13 11:47:39 +02:00
Johannes Millan
47ce4b304c perf(super-sync): optimize status and conflict checks 2026-05-13 11:40:43 +02:00
Johannes Millan
e4f9a4e2e5 refactor(sync-providers): extract local file provider 2026-05-13 11:36:19 +02:00
Johannes Millan
8005b4ec52 feat(supersync): retry transient web fetch failures and surface them as warnings
Adds a 2-retry / 1s+2s backoff loop to the browser/Electron SuperSync
request path, mirroring the existing native retry. A new typed
NetworkUnavailableSPError replaces the previous string-shape contract
between provider and wrapper: the provider throws the typed error from
both web-retry exhaustion and native-failure handler, and the wrapper
matches via instanceof to show a transient WARNING snackbar
(F.SYNC.S.NETWORK_ERROR) without flipping into a hard ERROR state.

Drops the wrapper-side regex classifier (and its defensive
^HTTP\s+\d{3}\b filter), the now-unused isTransientNetworkError alias,
and dedupes the predicate call inside the web-retry catch. The single
remaining call site for the broad regex (operation-log-upload) now
imports isRetryableUploadError directly under its honest name.

Tests cover the 1s/2s cadence, retry-then-success, retry-exhaustion via
the typed error, and negative paths (AbortError, HTTP-status 5xx,
AuthFailSPError must NOT retry). The error class is added to the
cross-module identity safety net.
2026-05-13 11:20:02 +02:00
johannesjo
0b597a1f44 Merge branch 'task/on-production-server-8e6dbb' 2026-05-13 02:12:36 +02:00
johannesjo
ca660a0585 fix(supersync): harden production server deploy 2026-05-13 02:10:46 +02:00
johannesjo
e03a4d41bd fix(sync-providers): retry supersync rate limit uploads 2026-05-13 01:52:56 +02:00
johannesjo
3db24a7b50 fix(supersync): harden requestId, 429 surface, and native error logging
Four improvements landed on top of the slice:

1. **Encrypted payloads excluded from requestId fingerprint.**
   AES-GCM uses fresh IVs on every encryption, so the same logical
   op produces different ciphertext each retry. Hashing the raw
   payload would compute a different requestId on every retry,
   defeating server-side idempotency for encryption-enabled
   accounts. `_toRequestIdFingerprintOp` substitutes a fixed
   `'[encrypted-payload]'` sentinel when `op.isPayloadEncrypted`,
   so retries of the same logical op hash to the same id even
   after re-encryption. New spec call (#5/#6) verifies same
   `requestId` for the same op with different ciphertext.

2. **429 retry-delay extraction.** SuperSync's server returns
   Fastify's standard rate-limit shape (`{ statusCode: 429, code:
   'FST_TOO_MANY_REQUESTS', message: 'Rate limit exceeded, retry
   in 5 minutes' }`). `_extractRetryDelayReason` parses the
   `message` field and threads "retry in N minutes/seconds" into
   the thrown `HTTP 429 …` error message so users see actionable
   wait info. Spec updated to assert the full message.

3. **Native logger meta hardened.** `_handleNativeRequestError`
   and `_doWebFetch`'s catch previously logged `error:
   error.message`, which on iOS/Android TLS-cert errors could
   embed the resolved hostname into the exportable log. Replace
   with `_getSafeErrorLogMeta` (uses `toSyncLogError` from
   `@sp/sync-core`) which emits only `errorName` + optional
   `errorCode` — both safe primitives. TLS-cert spec extended to
   assert the logger meta also doesn't leak the hostname.

4. **`unable to connect` added to retryable patterns.** The
   user-facing native rewrite message ("Unable to connect to
   SuperSync server. …") is now recognized as retryable by the
   broad-pattern matcher, so upstream retry loops can retry the
   replacement message itself. Spec added.

Plus minor structural improvements: `SuperSyncHttpStatusError`
now carries `status: number`; the user-facing native message is a
named constant (`RETRYABLE_NATIVE_REQUEST_MESSAGE`); the
non-Error fallback throw at end of `_handleNativeRequestError`
scrubs via `toSyncLogError` too.

Package spec count: 277 (no count delta — existing tests extended).
Bundle: CJS 97.92 KB / ESM 95.12 KB (negligible delta).
2026-05-13 01:28:14 +02:00
johannesjo
b28ff1b571 Merge branch 'master' into task/handover-pr-5-next-slice-supersync-751252
Brings in master's `30da81ea5 fix supersync retry idempotency`,
which landed after this branch was created. The server-side files
merge cleanly: `super-sync-server/src/sync/sync.routes.ts` and
`sync.service.ts` now recognize the new `requestId` field for
idempotent upload replays, with two new server specs.

Two client-side conflicts, both resolved by keeping the branch
version because the slice already ports master's additions to the
package side:

- `src/app/op-log/sync-providers/super-sync/super-sync.ts`
  (content conflict). Master added the requestId helpers
  (`_createOpsUploadRequestId`, `_compactRequestIdPart`,
  `_hashRequestIdInput`, `_stableJsonStringify`,
  `_toStableJsonValue`) and the `OPS_UPLOAD_REQUEST_ID_PREFIX`
  constant to the old class-based file. The branch had replaced
  that file with the `createSuperSyncProvider()` factory shim and
  ported the helpers into the package class in commit fbefaa343.
  Resolution: keep the factory shim (ours).

- `src/app/op-log/sync-providers/super-sync/super-sync.spec.ts`
  (modify/delete). Master added two specs asserting the `requestId`
  format and stability. The branch deleted the file when the
  package's Vitest spec took over. The equivalent specs are already
  in `packages/sync-providers/tests/super-sync/super-sync.spec.ts`
  (committed in fbefaa343). Resolution: keep the deletion (ours).

Post-merge verification:
- `npm run sync-providers:test` → 277/277
- `npm run sync-providers:build` → CJS 97.92 KB / ESM 95.12 KB
- `npm run lint` → clean
- App imex/sync spec sweep: 381/381
2026-05-13 01:07:34 +02:00
johannesjo
a96e446479 refactor(supersync): tag native HTTP-status throws with an internal class
The post-review fix in 8d7e05cce used `error.message.startsWith('HTTP ')`
in `_handleNativeRequestError` to detect our own thrown HTTP errors
(so they propagate unchanged on the non-retryable native path).
The check is functionally correct but brittle — any foreign error
whose message coincidentally begins with "HTTP " would also pass
through, leaking whatever followed.

Replace with a file-local `SuperSyncHttpStatusError extends Error`
class. The web and native non-2xx branches throw this class; the
handler's instanceof check is now robust against foreign-message
collisions. The error class is intentionally NOT exported — its only
consumer is the catch block in `_doNativeFetch`.

Regression spec added: a foreign error with message
`"HTTP request from sync.example.com refused at 10.0.0.5"` gets
scrubbed to the name-only form. The hostname and IP don't leak to
the user-facing surface.

Bundle: CJS 97.92 KB / ESM 95.12 KB (negligible delta from the
class declaration). Package spec count: 277.
2026-05-13 01:02:40 +02:00
johannesjo
fbefaa3437 fix(supersync): port requestId idempotency missed in slice rebase
Master shipped commit 30da81ea5 ("fix supersync retry idempotency")
after this slice's worktree was created from 88961485f, so the
package class was missing the deterministic `requestId` on every
`uploadOps` payload. Without it the SuperSync server can't recognize
a retried upload (e.g. after a network drop between server commit
and client receipt) and rejects the replay as duplicate ops instead
of returning the cached response/piggybacked ops.

Port the four private helpers (`_createOpsUploadRequestId`,
`_compactRequestIdPart`, `_hashRequestIdInput`, `_stableJsonStringify`
+ `_toStableJsonValue`) and the `OPS_UPLOAD_REQUEST_ID_PREFIX`
constant into the package class, and add `requestId` to the
`uploadOps` JSON payload. All helpers are pure functions with no
external deps — no new ports required.

Port master's Vitest spec: assert the prefix/length on the existing
"uploads operations successfully" test, and add a dedicated
"stable requestId across retries and key-reordering" test that
exercises five batches and asserts the equivalence/divergence
classes (retry → same id, key reorder → same id, content change →
different id, batch resize → different id).

Bundle: CJS 97.79 KB / ESM 94.99 KB (+2.6 KB from the helpers).
Package spec count: 276 (was 275).

Caught by codex during post-slice review.
2026-05-13 00:50:17 +02:00
johannesjo
8d7e05cceb fix(supersync): plug residual gaps surfaced by post-slice review
Two post-merge findings from the careful review pass on the
SuperSync slice (PR 7b):

1. **Storage NaN bug.** The app-side `localStorage`-backed
   `SuperSyncStorage.getLastServerSeq` returned `NaN` for an empty-
   string value (`parseInt("", 10) === NaN`), which the package's
   `stored ?? 0` fallback could not catch (nullish coalescing only
   triggers on `null`/`undefined`). The original implementation
   used `stored ? parseInt(stored, 10) : 0` and returned `0` for
   any falsy string. Add an explicit `Number.isFinite` check in
   the adapter so the port contract (returns `number | null`) is
   actually honored. Doesn't happen in practice today — `setItem`
   always writes a stringified int — but a hypothetical regression
   that wrote `""` would silently break sync replay.

2. **`_handleNativeRequestError` residual hostname leak.** The
   non-retryable native path re-threw the original error
   unchanged. Foreign errors from the native HTTP executor (e.g.
   iOS TLS cert errors like "SSL certificate problem: hostname
   mismatch for sync.example.com") can embed the resolved
   hostname in `.message`, which surfaces in user-visible snack
   notifications via `super-sync-restore.service.ts` and
   similar. Scrub to a name-only form
   (`SuperSync native request failed: <Error.name>`) while
   preserving identity for our own thrown errors
   (`AuthFailSPError`, `MissingCredentialsSPError`,
   `HTTP <status>` form). The full message is still captured in
   the structured log meta for diagnostics.

Two new Vitest specs cover the scrub path and the
`AuthFailSPError` identity preservation through the non-retryable
native branch. Package spec count: 275 (was 273).
2026-05-13 00:45:41 +02:00
johannesjo
c40feef6d2 refactor(sync-providers): move SuperSync provider into package
Move SuperSync's provider class, model, and spec from
`src/app/op-log/sync-providers/super-sync/` into
`packages/sync-providers/src/super-sync/` behind the ports
established by slices 5-6 (logger, platform-info, web-fetch,
native-http-executor, credential-store) plus two new ports for this
slice: `SuperSyncStorage` (narrow, three methods for `lastServerSeq`
persistence) and `SuperSyncResponseValidators` (host-side wrapper
that keeps `@sp/shared-schema` out of the package boundary).

Privacy fixes applied inline per multi-review consensus:

- `AuthFailSPError(reason, body)` now only takes the extracted
  reason — body is no longer retained on `additionalLog`.
- `_handleNativeRequestError` drops the parenthesised
  `(${errorMessage})` interpolation from the user-facing message
  to avoid leaking hostname/URL on native-stack errors.
- Timeout `Error.message` drops the `path` interpolation
  (`excludeClient` clientId was leaking into the thrown message).
- `_extractServerErrorReason` caps the extracted server reason at
  80 chars to defend against future server contract drift.
- Thrown non-2xx errors use a fixed `HTTP <status> <statusText> —
<reason?>` form (web) or `HTTP <status> — <reason?>` (native)
  instead of embedding the raw response body in `Error.message`.

JSDoc invariants pinned on `getEncryptKey`, `getWebSocketParams`,
`_cachedServerSeqKey`, and `deleteAllData` response shape.

Compression switched to direct `@sp/sync-core` import (no
`CompressError` wrapping at SuperSync's call sites; the existing
app-side compression-handler still wraps for
`EncryptAndCompressHandlerService`).

Spec converted Jasmine → Vitest. `TestableSuperSyncProvider`
subclass gone — native-platform routing now tested via injected
`platformInfo` + `nativeHttpExecutor` mocks. Adds a new
`privacy regression: no user content in error/log surface`
describe block driven by an HTTP-error path with a body containing
plausible user content (taskId/title/accessToken).

App-side `super-sync.ts` collapses to a 50-line
`createSuperSyncProvider()` factory (no `extraPath` arg —
SuperSync explicitly ignored it). The `SyncProviderId.SuperSync`
enum stays app-side; `AssertSuperSyncId` conditional pins
enum-vs-constant drift at compile time. `super-sync.model.ts`
becomes a re-export shim. `sync-providers.factory.ts` updated.

`super-sync-restore.service.ts` narrows the package's
`RestorePoint<string>` return through the consensus "narrow at the
app shim" decision (single cast at the call boundary).

Bundle: CJS 95.15 KB / ESM 92.34 KB (up from 76.78 / 74.18). Right
in the performance reviewer's 95-100 KB estimate; under the
original 110 KB projection. Tiered barrel split remains a
documented deferral.

Package spec count: 273 (was 197, added 76 SuperSync specs).
All targeted app specs green: sync-wrapper (107), op-log (2513
across the directory), imex/sync (381).
2026-05-13 00:30:19 +02:00
johannesjo
30da81ea5a fix supersync retry idempotency 2026-05-13 00:14:35 +02:00
johannesjo
a2509cb8ac refactor(sync-providers): promote isRetryableUploadError helper
Move the broad-pattern transient-error matcher from
`src/app/op-log/sync/sync-error-utils.ts:96` into
`packages/sync-providers/src/http/retryable-upload-error.ts` as
`isRetryableUploadError`. Distinct from the package's existing
`isTransientNetworkError` at `http/native-http-retry.ts:136`, which
checks native-platform error codes for the CapacitorHttp retry
path; the two coexist because they answer different questions
(retry the native HTTP failure? vs retry the server-returned error
string?).

The name was renamed during promotion (per multi-review consensus)
to avoid the "discriminate on input shape" smell that the
input-typed names (`isTransientErrorMessage` vs
`isTransientNetworkError`) would have produced. The intent-anchored
form makes the call-site decision explicit.

App-side `sync-error-utils.ts` keeps its current export name
(`isTransientNetworkError`) as an alias re-export so
`operation-log-upload.service.ts:31, 166` continues to import
unchanged. No behavior change.

Sets up PR 7b — SuperSync (in the package) imports
`isRetryableUploadError` directly.
2026-05-13 00:07:09 +02:00
Johannes Millan
88961485fd fix(supersync): harden deploy database migration 2026-05-12 23:24:13 +02:00