Commit graph

196 commits

Author SHA1 Message Date
Johannes Millan
70414145a3
fix(sync): prevent magic link prefetch failures and improve auth resilience (#7175)
* fix(sync): prevent magic link prefetch failures and improve auth resilience

- Make /magic-login prefetch-immune: GET now renders a confirmation page
  instead of consuming the single-use token. A "Log In" button triggers
  POST /api/login/magic-link/verify which does the actual verification.
  This prevents email client link prefetchers (Outlook SafeLinks, Gmail)
  from consuming magic link tokens before users click them.

- Increase rate limits on auth endpoints for shared-IP scenarios (e.g.
  university NAT). Global: 100→500, register: 10→50, login: 5→30,
  verify: 10→50, magic-login page: 10→50.

- Add trustProxy to Fastify so req.ip uses X-Forwarded-For behind
  reverse proxies instead of collapsing all clients to one rate bucket.

- Clear stale SuperSync auth tokens after 3 consecutive AuthFailSPErrors
  instead of never clearing them. Prevents persistent failure loops where
  an invalid token keeps retrying indefinitely until app reinstall.

- Improve registration-to-login UX messaging to emphasize email
  verification is required before login links will work.

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* chore: update package-lock.json from npm install

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): address review feedback for auth resilience

- Remove conflicting style.css link from magic-login page
- Change trustProxy from true to 1 (trust single hop only)
- Increase /login/magic-link rate limit from 30 to 50 per 15min
- Delete dead magic-login-redirect.js (replaced by magic-login-confirm.js)
- Reset auth failure counter on non-auth errors (truly consecutive)
- Add test for counter reset on non-auth error between auth errors

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): increase passkey rate limits and fix CSP inline script violations

- Increase passkey endpoint rate limits to match magic-link equivalents (50/15min)
- Extract inline scripts from /reset-password and /recover-passkey pages
  into external JS files to comply with CSP scriptSrc: ["'self'"]
- Remove dead safeJsonForScript helper (no longer needed)
- Increase /recover-passkey page rate limit to 50/15min for consistency

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): update security tests for escapeHtml and fix recover/passkey rate limit

- Update server-security.spec.ts: tests now check for escapeHtml output
  (<, ") instead of safeJsonForScript output (\u003c), matching
  the new data-token attribute approach
- Increase /recover/passkey rate limit from 30 to 50 per 15min to match
  /login/magic-link (both send email with a secret link)

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): reset auth failure counter after successful download

Move the counter reset from the final InSync return to right after
downloadRemoteOps() succeeds. This ensures early returns (cancelled,
LWW pending, payload rejected) also break the consecutive-failure
chain, since a successful download confirms auth is working.

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-10 14:46:57 +02:00
Johannes Millan
d32f7037a3 fix(sync): preserve own vector clock counter across full-state op resets
When a full-state op (SYNC_IMPORT/BACKUP_IMPORT) arrived from another
client, mergeRemoteOpClocks reset the local clock to a "minimal" form
but only preserved the current client's counter from the incoming op's
clock. If the current client had issued ops (e.g. GLOBAL_CONFIG) not
reflected in the incoming full-state op's clock, its counter was dropped,
causing subsequent ops to reuse the same counter value. Downstream clients
then saw these ops as EQUAL (duplicate) and skipped them silently.

Fix: take max(mergedClock[clientId], currentClock[clientId]) when
rebuilding the clock after a full-state op reset.

Also add __SP_E2E_BLOCK_WS_DOWNLOAD flag to WsTriggeredDownloadService
to allow E2E tests to block automatic WS-triggered downloads during
concurrent conflict scenarios.

Fix archive conflict test by blocking WS downloads on Client A during
the concurrent edit phase so it doesn't auto-receive B's rename via
WebSocket before archiving (restoring the intended conflict scenario).

Fix LWW singleton test to assert convergence rather than specific winner.
Fix renameTask helper to avoid Playwright/Angular re-render races.
Fix shepherd.js import paths that broke the Angular dev server build.
2026-04-01 15:41:13 +02:00
Johannes Millan
821a87ba71 chore(deps): upgrade prisma from 5.22.0 to 7.6.0
Migrate super-sync-server to Prisma v7:
- Switch generator from prisma-client-js to prisma-client with local output
- Add prisma.config.ts for datasource URL configuration
- Use PrismaPg driver adapter instead of built-in Rust engine
- Add @prisma/adapter-pg and pg dependencies
- Update all imports from @prisma/client to generated client path
- Fix Buffer→Uint8Array for Bytes fields (Prisma v6 breaking change)
- Update Dockerfile: prisma CLI version, copy prisma.config.ts
- Add DATABASE_URL validation in db.ts
- Add src/generated/ to .gitignore
2026-03-31 20:05:03 +02:00
Thorsten Klein
7fa8f12132
feat(sync): add Helm chart and WebSocket push for SuperSync (#6971)
* feat(sync): add Helm chart for SuperSync Kubernetes deployment

Includes Deployment, StatefulSet (PostgreSQL), Service, Ingress,
ConfigMap, Secret, HPA, PDB, NetworkPolicy, and test templates.
Supports both bundled PostgreSQL and external database configurations.

* feat(sync): add WebSocket push notifications for near-realtime sync

Server: Fastify WebSocket plugin with connection manager, app-level
heartbeat (30s), debounced notifications, and per-user routing.
Client: WebSocket service with exponential backoff reconnection,
WS-triggered download service, and reduced polling when connected.

* fix(sync): improve WebSocket error handling and reactivity

Make syncInterval$ reactive to WS connection state, fix Set/Map
mutation during heartbeat iteration, add error handling to WS route
handler, separate JSON parsing from message handling, detect auth
failures in WS-triggered downloads, and add logging to all empty
catch blocks.

* fix(sync): address PR review findings for WebSocket and Helm

Fix race condition in WS-triggered download pipeline by moving
isSyncInProgress filter after debounce and adding guard in
_downloadOps. Add logging to remaining empty catch blocks, fix
missing $NODE_IP in Helm NOTES.txt, and correct inaccurate
comments in values.yaml and ws-triggered-download.service.ts.

* fix(sync): add rate limiting to WebSocket upgrade endpoint

Limit WS connection attempts to 10 per minute per IP to prevent
connection flooding, matching the rate-limit pattern used by other
sync endpoints.

* fix(sync): extract shared constants, fix debounce correctness, replace deprecated toPromise

Extract CLIENT_ID_REGEX and MAX_CLIENT_ID_LENGTH into sync.const.ts
to prevent drift between sync.routes.ts and websocket.routes.ts.

Fix debounce in notifyNewOps to accumulate excluded client IDs across
rapid calls from different clients, preventing self-notifications.

Replace deprecated toPromise() with firstValueFrom in connectWebSocket
and _sync methods. Add .catch() to reconnect path and logging to
silent early returns in _downloadOps.

* fix(build): restore correct package-lock.json and fix upstream lint errors

* revert: restore upstream HTML formatting to match CI prettier config

* fix(sync): use DownloadOutcome discriminated union in WsTriggeredDownloadService

* fix(sync): narrow TokenVerificationResult before accessing userId

* fix(sync): await async getProviderById in connectWebSocket

Missing await caused the Promise object to be cast to
SuperSyncProvider, making getWebSocketParams undefined.

* feat(sync): add SEED_USERS env var to create verified users on startup

For self-hosted single-user setups: set SEED_USERS=email1,email2 to
create verified users on boot and log their access tokens. Skips
existing users. Removes need for SMTP/magic link registration.

Also fix Dockerfile to use npm install instead of npm ci for lockfile
compatibility.

* fix(sync): address PR review feedback for Helm chart and server

Security:
- Remove seed-users.ts (logged full JWT tokens to stdout)
- Add fail assertions for missing jwtSecret and postgresql.password
- Add smtp.user/smtp.password values fields
- Add from: selector to NetworkPolicy ingress
- Add egress rule for external database when postgresql.enabled=false

Correctness:
- Fix PostgreSQL StatefulSet indentation for non-persistent mode
- Fix NOTES.txt panic on empty tls list (use len instead of index)
- Restore npm ci in Dockerfile by using node:24-alpine (ships npm 11)
- Add Recreate deployment strategy when using RWO PVC

Operational:
- Add fail guard preventing HPA maxReplicas > 1 (in-memory WS state)
- Fix PDB to use maxUnavailable instead of minAvailable
- Add WebSocket ingress timeout annotation examples
- Add Prisma db push init container for schema migrations
- Default serviceAccount.automount to false
- Add Chart.yaml maintainers, home, sources metadata

* feat(sync): add ALLOWED_EMAILS env var to restrict registration

Supports fully qualified emails (user@example.com) and domain
wildcards (*@example.com). When unset, all emails are allowed.
Applied to all three registration endpoints (passkey options,
passkey verify, magic link).

* fix(sync): exempt health endpoint from rate limiting

Kubernetes liveness/readiness probes hit /health every 5-15s,
exhausting the global rate limit (100 req/15min) and causing
429 responses that trigger container restarts.

* fix(sync): harden WebSocket, Helm chart, and sync services from multi-agent review

- Pin prisma@5.22.0 in init container and use migrate deploy instead of db push
- Add per-user WebSocket connection limit (max 10) to prevent resource exhaustion
- Add replicaCount > 1 fail guard in deployment template (in-memory WS state)
- Set maxPayload: 1024 on WebSocket plugin (only pong messages expected)
- Remove premature IN_SYNC status from download-only WS-triggered sync
- Fix double removeConnection on error+close events (let close handle cleanup)
- DRY debounce logic in notifyNewOps and store latestSeq on pending entry
- Remove dead fromClientId from NewOpsNotification and WsMessage interfaces
- Add takeUntilDestroyed to _wsProviderCleanup subscription
- Simplify email-allowlist.ts to eager init (eliminate mutable state)
- Guard connectWebSocket at call site for non-SuperSync providers
- Add distinctUntilChanged to syncInterval$ to prevent unnecessary resets
- Add container-level securityContext to PostgreSQL StatefulSet
- Fix HPA maxReplicas default to 1 (matches single-replica constraint)

* fix(sync): restore migration in Dockerfile CMD for non-Helm Docker users

Helm users get migration via init container (runs first, CMD becomes no-op).
Docker-compose/plain Docker users get automatic migration back on startup.

* fix(boards): add missing drag delay for touch and extract shared signal

Add cdkDragStartDelay to board-panel drag items, which was missing
entirely. Extract the repeated `isTouchActive() ? DRAG_DELAY_FOR_TOUCH : 0`
expression into a shared `dragDelayForTouch` computed signal and refactor
all 8 components to use it.

* test(sync): add comprehensive WebSocket test coverage

Add unit tests for the new WebSocket real-time sync notification pipeline:

- WebSocketConnectionService (server): connection lifecycle, max-per-user
  limits, notification debouncing, heartbeat, graceful shutdown (13 tests)
- WebSocket routes validation: token/clientId validation, close codes,
  error handling, validation ordering (18 tests)
- SuperSyncWebSocketService (frontend): reconnection with exponential
  backoff, heartbeat, disconnect cleanup, URL conversion (6 tests)
- WsTriggeredDownloadService: auth error handling, pipeline resilience,
  start() idempotency (3 tests)
- SyncWrapperService: connectWebSocket guards for non-SuperSync, null
  params, and already-connected cases (3 tests)
- E2E realtime push: verifies WS-triggered download between two clients

Quality fixes:
- Replace waitForTimeout with syncAndWait in E2E
- Add explanatory comment for microtask flushing in sync-wrapper spec
- Use getter for isSyncInProgress mock in ws-triggered-download spec

---------

Co-authored-by: Johannes Millan <johannes.millan@gmail.com>
2026-03-30 21:34:30 +02:00
Johannes Millan
6d29e6c02a fix(sync): transmit syncImportReason through sync pipeline
- Thread syncImportReason field from operation creation through compact
  codec, sync providers, server database, and back to receiving clients
- Constrain server Zod schema to z.enum() matching client SyncImportReason
  type for defense-in-depth validation
- Use !== undefined in codec for consistency with entityId/entityIds
- Rename piggybackedImport/incomingSyncImport to piggybackedFullStateOp/
  incomingFullStateOp for accuracy
2026-03-29 20:06:41 +02:00
Johannes Millan
f26732755f fix(sync): prevent transient server errors from wiping auth credentials
Server: verifyToken() catch-all was returning 401 "Invalid token" for ALL
errors including database failures. Now only JWT-specific errors
(JsonWebTokenError, TokenExpiredError) return 401. Database/infrastructure
errors propagate as 500s with sanitized logging.

Client: SuperSync AuthFailSPError no longer clears stored credentials —
the server rejection may be transient. The snackbar with "Configure"
button still shows so users can re-auth on genuine failures. Dropbox and
WebDAV still clear immediately so their OAuth/re-auth flows work.
2026-03-20 21:36:30 +01:00
Johannes Millan
2cc2760486 feat(i18n): complete translations and update locale files
Translate all missing keys to achieve full coverage (2086/2086)
for fr, pt-br, zh-tw, ro, ro-md, zh, es, de, and cs.
Also remove stale keys and include updated translations
for other locales. Remaining languages still have partial coverage.
2026-03-18 21:55:13 +01:00
Johannes Millan
1907df68d7 fix(sync-server): security hardening and GDPR log compliance
- Remove email addresses from all log messages (~18 locations) for GDPR
  compliance, replacing with userId where available
- Downgrade debug credential logs from info to debug level
- Remove options.user from passkey registration log (contained email in
  JSON payload)
- Add container security hardening to docker-compose.yml:
  - no-new-privileges on all containers
  - cap_drop: ALL on supersync and caddy
  - Memory limits (supersync 512m, postgres 1g, caddy 256m)
  - CPU limit on supersync (1.0)
- Add .health-alert/ to gitignore
- Add health-alert.sh cron script for container monitoring with
  OOM detection, disk checks, and email alerting
2026-03-18 20:48:51 +01:00
Johannes Millan
a01a511c25 feat(sync): add backup strategy with accounts-only dump and disaster recovery docs
- Add accounts-only pg_dump (users + passkeys) to backup script for
  lightweight disaster recovery when clients still have data
- Add pipefail to backup script to catch silent dump failures
- Add test endpoint to simulate partial server revert (ops-after/:serverSeq)
- Add 6 e2e tests covering all disaster recovery scenarios:
  complete data loss, partial revert, accounts-only restore (API + SQL),
  full dump restore + reset account, and all-clients-lost recovery
- Add backup-and-recovery.md with setup, recovery procedures, and
  decision tree
2026-03-18 20:15:45 +01:00
Johannes Millan
861425fd28 refactor(sync): improve vector clock implementation quality
- Replace O(N) Prisma findMany with PostgreSQL SQL aggregate
  (jsonb_each_text + GROUP BY + MAX) for snapshot vector clock
  computation, avoiding loading all ops' clocks into Node.js memory
- Add defensive regex filter (^[0-9]+$) for non-numeric JSONB values
- Fix || to ?? for nullish coalescing in vector clock value lookups
- Remove unreachable CONCURRENT check in compareVectorClocks
- Remove unused VectorClockMetrics interface and measureVectorClock
- Add 7 new unit tests for SQL aggregate path verification
- Add 10 integration tests running actual SQL against PostgreSQL
2026-03-17 13:59:40 +01:00
Johannes Millan
d401e6169e feat(tasks): add deadlines feature with reminders and planner integration
Add deadline support for tasks with date-only and time-specific deadlines.

Core:
- Add deadlineDay, deadlineWithTime, deadlineRemindAt fields to task model
- Add NgRx actions (setDeadline, removeDeadline, clearDeadlineReminder)
- Add meta-reducer with mutual exclusivity and input validation
- Register deadline actions in ActionType enum and op-log codes

UI:
- Create deadline dialog with calendar, time input, and reminder options
- Add deadline badge to task list row with overdue warning color
- Add deadline item to task detail panel with flag icon
- Add deadline options to task context menu
- Show deadlines in scheduled list page

Reminders:
- Add deadline reminder effects and selectors
- Show reminder dialog with grouped deadline/schedule sections
- Handle deadline reminder clearing on dismiss, done, add-to-today
- Cancel Android native deadline notifications on delete/archive

Planner:
- Show deadline tasks in planner day view and overdue section
- Create dedicated planner-deadline-task component
- Optimize deadline grouping with O(N) single-pass selector

Other:
- Add deadline-today banner effect for unplanned deadline tasks
- Add translation keys for all deadline UI strings
- Add E2E tests for deadline reminder flows
- Add unit tests for deadline reducer and overdue utility
- Extract shared isDeadlineOverdue utility function
- Guard app-state selectors against undefined during teardown
2026-03-15 18:48:00 +01:00
Johannes Millan
baf6124415 fix: address code review findings from yesterday's changes
- Add error handler to markdown paste promise in add-task-bar
- Fix misleading "try again later" in resend cap error message
  (cap is permanent, not time-based)
2026-03-13 11:07:22 +01:00
Johannes Millan
b5536adfe5 feat(sync): add verification resend cap of 20 attempts
Prevents email bombing by capping verificationResendCount at 20 for
both magic link and passkey registration flows. Returns a safe error
message when the cap is reached.
2026-03-12 13:50:50 +01:00
Johannes Millan
3ba5841421 feat(sync): add magic link registration to SuperSync server
Add email-only registration flow (no passkey required) with verification
email. For existing unverified users, email is sent before DB token
update to preserve the old verification link on failure.

- POST /api/register/magic-link endpoint with Zod validation and rate limiting
- registerWithMagicLink function in auth.ts with P2002 race condition handling
- Frontend "Register with Email" button and JS handler
- Export shared VERIFICATION_TOKEN_EXPIRY_MS constant (eliminates duplication)
- 8 unit tests covering all paths including email failure cleanup
2026-03-12 13:36:05 +01:00
Johannes Millan
cce9576946 test(e2e): fix supersync E2E tests for mandatory encryption flow
- Extract shared encryption warning helper and fix race conditions
- Restructure concurrent-import tests to import before enabling sync
- Fix encryption password preservation and change tests
- Add "Change Password" button detection in supersync page object
- Mark backup-import ID mismatch test as fixme (known limitation)
2026-03-06 16:36:38 +01:00
Johannes Millan
3dc86dd183
Fix/add rate limits (#6745)
* fix(build): fix tag resolution and add error handling in bump-android-version

The previous two-tag approach (`currentTag` → `prevTag`) would resolve the
wrong changelog range during `npm version` since the new tag doesn't exist
yet. Simplified to a single `lastTag...HEAD` range which correctly captures
all commits since the last release.

Added try-catch with fallback to last 20 commits when no tags exist, and
a fallback message for empty changelogs.

* fix(sync): differentiate auth error messages and clarify token revocation

- Rename server dashboard "Refresh Token" to "Revoke & Replace Token"
  with explicit warning that ALL devices will be disconnected
- Return rejection reason in 401 responses (revoked, expired, etc.)
  via discriminated union TokenVerificationResult type
- Show different client error messages for server-side token rejection
  vs missing local credentials to aid user diagnosis
- Extract server error reason from JSON response body in AuthFailSPError

Closes #6597

* refactor(sync): use generic auth reason and improve test readability

- Replace 'User not found' and 'Account not verified' with generic
  'Account unavailable' to avoid leaking account state in API responses
- Extract long reason strings to constants in middleware.spec.ts

* fix(sync-server): add per-route rate limits to silence CodeQL alert

Add explicit rate limits to routes that only had the global rate limit:
- GET /api/sync/status: 60/min
- DELETE /api/sync/data: 3/15min (destructive operation)
- GET /api/sync/restore-points: 30/min
- GET /api/sync/restore/:serverSeq: 10/5min (CPU-intensive)
- GET /reset-password: 20/15min
- GET /recover-passkey: 10/15min

These complement the global @fastify/rate-limit (100/15min) and silence
the CodeQL 'Missing rate limiting' alert on authenticated handlers.
2026-03-05 21:18:36 +01:00
Johannes Millan
7070fde51e
Pr 6741 (#6743)
* fix(build): fix tag resolution and add error handling in bump-android-version

The previous two-tag approach (`currentTag` → `prevTag`) would resolve the
wrong changelog range during `npm version` since the new tag doesn't exist
yet. Simplified to a single `lastTag...HEAD` range which correctly captures
all commits since the last release.

Added try-catch with fallback to last 20 commits when no tags exist, and
a fallback message for empty changelogs.

* fix(sync): differentiate auth error messages and clarify token revocation

- Rename server dashboard "Refresh Token" to "Revoke & Replace Token"
  with explicit warning that ALL devices will be disconnected
- Return rejection reason in 401 responses (revoked, expired, etc.)
  via discriminated union TokenVerificationResult type
- Show different client error messages for server-side token rejection
  vs missing local credentials to aid user diagnosis
- Extract server error reason from JSON response body in AuthFailSPError

Closes #6597

* refactor(sync): use generic auth reason and improve test readability

- Replace 'User not found' and 'Account not verified' with generic
  'Account unavailable' to avoid leaking account state in API responses
- Extract long reason strings to constants in middleware.spec.ts
2026-03-05 21:17:59 +01:00
Johannes Millan
da8e509972 refactor(sync): unify JWT expiry to 365 days for all auth methods
Replace separate JWT_EXPIRY_MAGIC_LINK (365d) and JWT_EXPIRY_PASSKEY (7d)
constants with a single JWT_EXPIRY (365d). The auth method only matters
during login — once a JWT is issued, it represents a verified session
regardless of how the user authenticated.
2026-02-16 18:44:27 +01:00
Johannes Millan
2bc73607ff
Add snapshotOpType parameter for correct operation typing (#6482)
* fix(sync): pass correct opType from client to server in snapshot upload

The server was hardcoding opType as 'SYNC_IMPORT' for all snapshot
uploads, losing the distinction between SYNC_IMPORT, BACKUP_IMPORT,
and REPAIR operations. Now the client sends snapshotOpType in the
request body, and the server uses it (with fallback to 'SYNC_IMPORT'
for backward compatibility with legacy clients).

https://claude.ai/code/session_01XnqdpGbFR6VRPdCAbZ2EB9

* test(sync): fix uploadSnapshot test expectations for 9th snapshotOpType arg

The uploadSnapshot call now passes snapshotOpType as a 9th argument.
Update all 5 failing test expectations to include the new argument
(SYNC_IMPORT, BACKUP_IMPORT, or REPAIR) and fix the argument count
check from 8 to 9.

https://claude.ai/code/session_01XnqdpGbFR6VRPdCAbZ2EB9

* fix(sync): add missing snapshotOpType param to file-based sync adapter

The file-based adapter's uploadSnapshot signature was missing the 9th
snapshotOpType parameter added in f328c35, making it inconsistent with
the provider interface. Add the parameter (unused, as file-based sync
has no server-side op log).

https://claude.ai/code/session_01XnqdpGbFR6VRPdCAbZ2EB9

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-13 16:06:39 +01:00
Johannes Millan
534f375a54 fix(sync): clear sync cycle cache on deleteAllData and extract ConflictType
Clear _syncCycleCache in _deleteAllData() to prevent stale cached data
from being served within the 30s TTL after a data wipe. Extract
ConflictType and ConflictResult into named types in sync.types.ts to
eliminate duplicated inline unions in sync.service.ts.
2026-02-12 16:27:59 +01:00
Johannes Millan
43b5808a3f fix(sync): harden vector clock sanitization and improve E2E test robustness
- Cap sanitizeVectorClock values at 100M instead of MAX_SAFE_INTEGER (prevents adversarial clock inflation)
- Add early-exit optimization in compareVectorClocks for CONCURRENT detection
- Add test.setTimeout(180000) to token-expiry and lastseq-preservation E2E tests
- Remove dead code (unreachable guard) in compaction E2E test
- Extract shared config helpers (navigateToMiscSettings, toggleSetting, etc.) into e2e/utils/config-helpers.ts
- Fix Client B cleanup in provider-switch test (move to finally block)
- Add clarifying comments for raw SQL schema coupling, CORS header, and mergeRemoteOpClocks caller contract
- Extract enforceStorageQuota helper in sync routes to reduce duplication
- Add conflictType to conflict detection for structured error handling
- Add MAX_CLEANUP_ITERATIONS guard to freeStorageForUpload
2026-02-12 16:27:56 +01:00
Johannes Millan
ab0d979e5e fix(sync): revert lastSeq preservation in deleteAllUserData
Unlike uploadOps clean-slate (which must preserve lastSeq to prevent
sequence reuse for other clients), deleteAllUserData should reset
lastSeq to 0. Account reset intentionally wipes everything and clients
detect the wipe via latestSeq=0, triggering a full state re-upload.
2026-02-12 16:27:56 +01:00
Johannes Millan
c70ced204e fix(sync): fix stale comments, doc refs, and preserve lastSeq on clean-slate
Remove references to deleted docs/ai/ files, update stale comments about
protectedClientIds and pruning-aware comparison to reflect current REPLACE
semantics, fix MAX=30→20 heading, client_0..29→19 comment, use ?? over ||,
preserve lastSeq in server clean-slate path to prevent sequence reuse, and
add clarifying comments for mock limitations and validation guards.
2026-02-12 16:27:56 +01:00
Johannes Millan
3f99e3773c fix(sync): fix replaceToken expiry, locale-independent sort, and stale comments
- Use JWT_EXPIRY_PASSKEY (7d) for replaceToken instead of 365d magic link
  expiry — token replacement is a security action, shorter lifetime is safer
- Replace localeCompare with locale-independent comparator in vector clock
  tie-breaking to ensure deterministic behavior across environments
- Fix 5 additional stale MAX=30 references in docs and tests (now 20)
- Update authentication.md to reflect dual JWT expiry tiers
- Clean up isLikelyPruningArtifact references in docs and LEGACY_MAX in tests
2026-02-12 16:27:55 +01:00
Johannes Millan
ef8ad4ded7 refactor(sync): address code review findings across server and client
- Consolidate getJwtSecret() and JWT_EXPIRY constants into auth.ts,
  eliminating 3x duplication with inconsistent validation (C1+C2)
- Add isVerified defense-in-depth check to verifyToken() (C3)
- Add Content-Transfer-Encoding to CORS allowedHeaders for
  Android/Capacitor clients (C4)
- Fix 9 stale comments referencing MAX_VECTOR_CLOCK_SIZE as 30
  when actual value is 20 (I1)
- Remove dead _syncVectorClockFromPfapi method and update
  referencing comments (S8)
- Replace in-memory payload size calculation with SQL aggregate
  in deleteOldestRestorePointAndOps to avoid loading large
  payloads into Node memory (I7)
2026-02-12 16:27:55 +01:00
Johannes Millan
0b73461690 fix(sync): fix vector clock comparison parity and validation consistency
- Fix client/server comparison divergence: remove isVectorClockEmpty
  short-circuit that caused {} vs {a:0} to return LESS_THAN on client
  but EQUAL on shared impl. Now delegates all comparisons to shared.
- Align server value cap with client: raise from 10M to MAX_SAFE_INTEGER
  to prevent silent clock entry stripping on long-term usage.
- Make client validation require integers (Number.isInteger) matching
  server-side validation.
- Add deterministic tie-breaking (localeCompare) to pruning sort for
  equal counter values.
- Fix stale comments: MAX references and CLAUDE.md DoS cap (5x→2.5x).
2026-02-12 16:27:55 +01:00
Johannes Millan
a335589dc6 refactor(sync): reduce vector clock DoS cap from 100 to 50 entries 2026-02-12 16:27:55 +01:00
Johannes Millan
f37280e082 refactor(sync): reduce MAX_VECTOR_CLOCK_SIZE from 30 to 20
Lower the cap to leave headroom for future increases and surface
size-related edge cases earlier. All pruning logic is MAX-agnostic
so this is a safe constant change with documentation updates.
2026-02-12 16:27:55 +01:00
Johannes Millan
77972bfe80 fix(sync): fix stale comments and string concatenation bug from vector clock simplification
Fix NaN in diagnostic log (unary + on string), update DoS cap comment
(50→150), remove dead normalization ternary, and update stale
"pruning-aware" references in comments.
2026-02-12 16:27:55 +01:00
Johannes Millan
29541951a3 refactor(sync): increase MAX_VECTOR_CLOCK_SIZE from 10 to 30 and remove defense layers
At MAX=10, pruning triggered frequently enough (11+ unique client IDs from
reinstalls/new browsers) to require 4 defense layers compensating for
information loss: pruning-aware comparison, protected client IDs with
migration, isLikelyPruningArtifact heuristic, and same-client check.

At MAX=30, pruning almost never triggers (needs 31+ unique client IDs).
A 30-entry clock is ~500 bytes — negligible bandwidth. This allows removing
most defense layers while keeping two cheap backward-compat checks for old
10-entry pruned data still on servers.

Removed:
- Pruning-aware mode in compareVectorClocks (standard comparison now)
- Protected client IDs mechanism (storage, migration, preservation)
- selectProtectedClientIds function
- Clock normalization in SyncImportFilterService

Kept temporarily (backward compat with old 10-entry data):
- isLikelyPruningArtifact with LEGACY_MAX=10
- Same-client check (always mathematically correct)
2026-02-12 16:27:55 +01:00
Johannes Millan
fc56aabec4 docs(sync): fix stale DoS cap references from 3x to 5x MAX_VECTOR_CLOCK_SIZE
The sanitizeVectorClock() DoS cap was changed to 5x MAX (50 entries) but
comments in CLAUDE.md, vector-clocks.md, sync.types.ts, and
validation.service.ts still referenced the old 3x MAX (30) value.
2026-02-10 20:38:32 +01:00
Johannes Millan
d4cd9a3575 fix: address 9 code review findings
- Change server-side pruning log from info to debug
- Increase vector clock sanitize limit from 3x to 5x MAX
- Add Formly hierarchy comment for WebDAV settings
- Add test documenting isLikelyPruningArtifact false positive limitation
- Replace `as any` with proper Partial<TaskRepeatCfgCopy> typing
- Add comment explaining `order` in SCHEDULE_AFFECTING_FIELDS
- Cache Intl.Collator instance, recreate only on locale change
- Document plugin management 30s timeout rationale
- Extract _isLikelyPruningArtifact as standalone pure function
2026-02-09 17:55:12 +01:00
Johannes Millan
a774c46700 fix(sync): remove client-side vector clock pruning from ConflictResolutionService and fix rejection counting
- Remove limitVectorClockSize from ConflictResolutionService (same
  infinite loop risk as SupersededOperationResolverService)
- Include retry-exceeded ops in permanentRejectionCount so sync status
  accurately reflects data loss
- Fix stale JSDoc (100 -> 30 entries) in sanitizeVectorClock
- Remove dead getProtectedClientIds mocks from spec files
- Extract _getEntityKey helper, remove redundant guards, optimize
  Object.keys calls in server pruning log
2026-02-09 17:55:12 +01:00
Johannes Millan
fdc942babb
fix(sync): prevent infinite loop when concurrent modification resolution keeps failing (#6434)
* fix(sync): prevent infinite loop when concurrent modification resolution keeps failing

When vector clock pruning makes it impossible to create a dominating clock
(e.g., entity clock has MAX entries and client ID isn't among them), the cycle
"upload → CONFLICT_CONCURRENT → merge clocks → upload → reject again" repeats
endlessly. This adds a per-entity retry counter (MAX_CONCURRENT_RESOLUTION_ATTEMPTS=3)
that permanently rejects ops after exceeding the limit, breaking the sync loop.

The counter resets when a sync cycle completes with no rejections (healthy state).

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

* fix(sync): move vector clock pruning after conflict detection to fix root cause

The infinite sync loop happens because it's mathematically impossible to build
a dominating clock with MAX_VECTOR_CLOCK_SIZE entries when the entity's clock
already has MAX entries and the client's ID isn't among them. The merged clock
needs MAX+1 entries (all entity clock IDs + client ID), but client-side pruning
drops one entity clock ID. The server's pruning-aware comparison then sees the
dropped key as non-shared and returns CONCURRENT instead of GREATER_THAN.

Fix: Move limitVectorClockSize from validation (before comparison) to
processOperation (after comparison, before storage). The full unpruned clock
is now used for conflict detection — all entity clock IDs are present so
bOnlyCount=0 → GREATER_THAN. Storage still gets the pruned clock.

Client-side: Stop pruning in SupersededOperationResolverService. The server
handles pruning after conflict detection.

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

* fix(sync): tighten vector clock sanitize limit from 100 to 3x MAX_VECTOR_CLOCK_SIZE

The old sanitize cap of 100 entries was unnecessarily wide. Since conflict
resolution clocks are at most ~12-15 entries (entity clock MAX=10 + client ID
+ a few merged), cap at 3x MAX (30) for DoS protection while leaving ample
room for legitimate clocks.

Also update server-side pruning tests to use realistic clock sizes (20 entries
instead of 50) to stay within the new sanitize limit.

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

* docs(sync): document vector clock pruning invariant and infinite loop fix

- Add "Pruning and the Pruning-Aware Comparison" section to vector-clocks.md
  explaining the critical invariant: server prunes AFTER comparison, not before
- Add rule 13 to CLAUDE.md to prevent future regressions
- Update conflict resolution key files table with rejected-ops-handler and
  superseded-operation-resolver services

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

* docs(sync): update last-updated date in conflict resolution docs

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

* test(sync): update pruning tests to reflect server-side pruning design

Client no longer prunes vector clocks during conflict resolution — the
server handles pruning after conflict detection. Tests now verify the
merged clock is sent unpruned with all keys preserved.

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

* test(sync): fix client-side pruning tests and add server-side regression test

- Update 7 SupersededOperationResolverService tests to verify client
  does NOT prune (server handles pruning after conflict detection)
- Remove redundant `newClock` alias in superseded-operation-resolver
- Add regression test: MAX+1 entry clock accepted as GREATER_THAN
  when it dominates a MAX entry entity clock (the core infinite loop fix)

https://claude.ai/code/session_016RAxms5dTouU98wFNQaAcv

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-09 12:01:43 +01:00
Johannes Millan
850f7248a7 chore(sync): extend JWT expiry to 365 days 2026-02-02 17:15:13 +01:00
Johannes Millan
cd2ae61ae0 Revert "feat(sync): add rolling JWT token refresh on sync requests"
This reverts commit eca323505d1923fcfc470d78fc8984251e825382.
2026-02-02 17:15:13 +01:00
Johannes Millan
717e4ba84d feat(sync): add rolling JWT token refresh on sync requests
On each successful sync request, the server now returns a fresh 14-day
JWT via X-Refreshed-Token response header. The client reads and stores
it automatically, preventing token expiry during active usage.

Server: export JWT_EXPIRY/getJwtSecret from auth, add createRefreshedToken(),
add onSend hook scoped to sync routes, expose header via CORS.
Client: store refreshed token in all 4 fetch methods, fix _getServerSeqKey
to hash only baseUrl (prevents full re-download on token change).
2026-02-02 17:15:13 +01:00
Johannes Millan
c5409bbd25 fix(sync): add server pruning log, harden test coverage for vector clocks
- Add Logger.warn() in ValidationService when oversized vector clocks
  are pruned server-side, improving observability for buggy clients
- Add tests for hasVectorClockChanges (previously zero coverage)
- Add symmetric LESS_THAN test for pruning-aware comparison mode
- Export MAX_LWW_REUPLOAD_RETRIES constant to eliminate magic number
  coupling between sync-wrapper service and its test
- Add test verifying uploading client ID is preserved during server-side
  clock pruning
- Add test for limitVectorClockSize when preserveClientIds exceeds MAX
2026-01-30 19:50:30 +01:00
Johannes Millan
495f0fe0d3 fix(sync): improve vector clock pruning awareness and harden tests
Make hasVectorClockChanges pruning-aware by checking clock size before
logging missing keys — downgrade to verbose when pruning is likely,
warn when corruption is likely. Add return value assertion to LWW retry
exhaustion test, asymmetric pruning test cases, server-side pruning
tradeoff documentation, and fix stale "max 50" in docs.
2026-01-30 19:35:51 +01:00
Johannes Millan
b113a2d7dd fix(sync): return CONCURRENT instead of EQUAL when pruned clocks have non-shared keys
When both vector clocks are at MAX size (pruning-aware mode) and shared
keys compare as equal, non-shared keys on both sides indicate genuinely
different causal histories. Returning EQUAL caused silent data loss by
skipping operations as duplicates. Returning CONCURRENT safely triggers
LWW conflict resolution instead.

Also fixes server snapshot pruning to preserve the requesting client's
ID and the snapshot author's client ID when limiting vector clock size.
2026-01-30 19:21:45 +01:00
Johannes Millan
c268b1a9a9 fix(sync): harden vector clock pruning and post-review sync fixes
Address issues found by code review agents after the LWW loop fix
(cb36c09538):

- Enforce MAX_VECTOR_CLOCK_SIZE on server upload to prevent oversized
  clocks from buggy/adversarial clients
- Limit aggregated snapshotVectorClock in download service to prevent
  unbounded growth with many clients
- Set UNKNOWN_OR_CHANGED status when LWW retry cap is exhausted instead
  of misleadingly reporting IN_SYNC
- Wrap startPostSyncCooldown() in try-catch so endApplyingRemoteOps()
  always runs even on failure
- Cap limitVectorClockSize output at MAX even when preserved IDs exceed
  the limit
- Update stale doc comments referencing MAX_VECTOR_CLOCK_SIZE=8 to 10
2026-01-30 18:40:54 +01:00
Johannes Millan
cb36c09538 fix(sync): prevent infinite LWW auto-resolve loop for TAG:TODAY
Three root causes addressed:

1. Pruning asymmetry: Different clients preserve their own clientId
   during vector clock pruning, producing different clock shapes.
   When compared, pruned-away keys default to 0, causing false
   CONCURRENT verdicts and infinite rejection loops. Fix: when both
   clocks are at MAX_VECTOR_CLOCK_SIZE, compare only shared keys
   (intersection) instead of the union.

2. Timing gap: Between endApplyingRemoteOps() and
   startPostSyncCooldown(), isInSyncWindow() returns false, allowing
   selector-based effects to fire and create TAG:TODAY operations.
   Fix: start cooldown BEFORE ending remote ops flag.

3. No retry limit: LWW re-upload had no cap, enabling infinite loops.
   Fix: cap at 3 retries, defer remaining to next sync cycle.

Also moves MAX_VECTOR_CLOCK_SIZE and limitVectorClockSize into
@sp/shared-schema so client and server share the same pruning
constant and algorithm.
2026-01-30 18:24:29 +01:00
Johannes Millan
361d0e605f refactor(sync): rename "stale" to "superseded" across sync/operation domain
Includes backward compatibility: client accepts both CONFLICT_SUPERSEDED
and CONFLICT_STALE from the server, and the server keeps a deprecated
CONFLICT_STALE alias. Remove after all deployments are updated.
2026-01-30 16:59:40 +01:00
Johannes Millan
100b8af5f2 fix(sync): return existingClock in conflict rejections to fix infinite loop
When encryption is enabled, clients could get stuck in infinite conflict
resolution loops because they couldn't create vector clocks that dominate
the server's state. The server already computed the existingClock during
conflict detection but didn't return it to the client.

This fix:
- Adds existingClock to UploadResult type (server)
- Returns conflict.existingClock in CONFLICT_CONCURRENT/CONFLICT_STALE rejections
- Passes existingClock through client upload service to rejected ops handler
- Merges entity clocks into extraClocks for stale operation resolution

Includes E2E tests for encryption + conflict resolution scenarios.
2026-01-25 18:10:08 +01:00
Johannes Millan
cc3176780d fix(sync): allow clean slate when isCleanSlate flag is set
Modified SYNC_IMPORT validation to skip duplicate check when isCleanSlate
is true. This allows password change operations to delete all existing
encrypted data and upload a fresh snapshot even when a SYNC_IMPORT
already exists on the server.

When changing encryption password:
- Client sends snapshot with isCleanSlate=true
- Server skips SYNC_IMPORT_EXISTS validation
- Server deletes ALL user data (operations, devices, state)
- Server accepts new SYNC_IMPORT with new encryption password

This completes the password change fix by ensuring old encrypted data
is removed before uploading data encrypted with the new password.
2026-01-24 21:14:57 +01:00
Johannes Millan
479ba906df fix(test): complete encryption enable/disable E2E tests with proper dialog handling
- Fixed disableEncryption() to handle 'Disable Encryption?' confirmation dialog
- Fixed enableEncryption() to fill password before checking checkbox (triggers dialog)
- Fixed overlay backdrop cleanup to prevent click interception
- Fixed test logic to use setupSuperSync() for secondary clients instead of triggering clean slates
- Added ensureOverlaysClosed() defensive checks in setupSuperSync()
- Fixed Advanced Config collapsible expansion in setupSuperSync()

Tests now properly verify:
C) Encryption disabled - other clients reconfigure to match
D) Encryption enabled - other clients require password
- Multiple encryption state changes
- Concurrent changes overwritten by clean slate

All fixes verified by sub-agents using systematic debugging.
2026-01-24 21:14:57 +01:00
Johannes Millan
e924b068ac fix(passkey): skip email verification in TEST_MODE for passkey registration
When TEST_MODE is enabled with autoVerifyUsers, passkey registration now
auto-verifies users immediately instead of sending verification emails.
This enables local development without SMTP configuration.
2026-01-24 21:14:57 +01:00
Johannes Millan
c576372d00 feat(sync): implement clean slate mechanism for encryption password changes
Adds clean slate approach for encryption password changes and full imports:
- Creates fresh SYNC_IMPORT with new client ID
- Server deletes all data atomically when isCleanSlate flag is set
- Prevents encrypted/unencrypted operation mixing
- Simpler error handling than previous recovery-based approach

Client changes:
- Add CleanSlateService for generating fresh sync baselines
- Add PreMigrationBackupService (placeholder for future backup functionality)
- Update EncryptionPasswordChangeService to use clean slate approach
- Add isCleanSlate flag to upload options and provider interface
- Update SuperSync provider to pass flag to API

Server changes:
- Add isCleanSlate parameter to upload operations endpoint
- Delete all operations, devices, and sync state when flag is true
- Clear caches after successful clean slate transaction

Tests:
- 24 unit tests for clean slate services (all passing)
- 9 unit tests for encryption password change service (8/9 passing)
- E2E tests verify encryption password change flow works correctly
2026-01-24 21:14:57 +01:00
Johannes Millan
737d11d8f3 feat(cors): normalize domain to lowercase and validate non-empty origins 2026-01-24 21:14:57 +01:00
Johannes Millan
ded17e96fc fix(supersync): parse wildcard patterns when universal wildcard present
Fixed logic bug where wildcard subdomain patterns (e.g., https://*.example.com)
were not being parsed to RegExp when the universal wildcard (*) was present in
CORS_ORIGINS. All origins were incorrectly stored as strings.

Before: CORS_ORIGINS="https://*.example.com,*,https://app.com"
  Result: ['https://*.example.com', '*', 'https://app.com'] (all strings)

After:
  Result: [RegExp, '*', 'https://app.com'] (wildcard pattern → RegExp)

Changes:
- Modified loadConfigFromEnv() to map over origins when * is present
- Keep * as-is, but parse other origins through parseCorsOrigin()
- Added test case verifying mixed origins with universal wildcard

All existing tests pass. New test validates RegExp matching works correctly.
2026-01-24 21:14:57 +01:00