Commit graph

323 commits

Author SHA1 Message Date
Johannes Millan
78f64508db feat(sync-server): add active-users-quick monitor command
Avoids the operations table entirely (sync_devices + users only) so it
returns in milliseconds even when operations grows past 1 GB. Useful when
the standard active-users command is too slow on large prod DBs.
2026-04-21 15:03:42 +02:00
Johannes Millan
0d5a562aba test(sync-server): freeze clock in 1ms-over-drift clamp test
The "just 1ms over max clock drift boundary" test captured Date.now()
once in the test and again inside uploadOps. If the second sample
advanced by >=1ms, the op was within the drift window from the
service's perspective, clamping never fired, and toBeLessThan failed
with equal values.

Freeze time with vi.useFakeTimers/setSystemTime so both samples
match, and tighten the assertion to the exact clamped value.
2026-04-21 15:03:41 +02:00
Johannes Millan
62f9f9cfd4 fix(sync-server): avoid per-user correlated scans in monitor usage
The showUsage query ran two correlated subqueries over the full
operations table once per user, each calling pg_column_size(payload)
on every row, then sorted all users before LIMIT 20. On a 1.85 GB
operations table with 3.6k users this effectively never returned.

Aggregate per-user size and count in a single CTE pass, then join.
One sequential scan instead of scans × users.
2026-04-20 18:51:12 +02:00
Johannes Millan
007c7024c7 chore(deps): bump @fastify/static to 9.1.1 in super-sync-server
Patches CVE-2026-6410 and CVE-2026-6414. Regenerates the root
package-lock.json so workspace resolution matches the bumped range.
2026-04-20 15:48:40 +02:00
dependabot[bot]
4b9bd23e56
chore(deps): bump the npm_and_yarn group across 1 directory with 9 updates (#7259)
Bumps the npm_and_yarn group with 9 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [fastify](https://github.com/fastify/fastify) | `5.8.4` | `5.8.5` |
| [nodemailer](https://github.com/nodemailer/nodemailer) | `8.0.4` | `8.0.5` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `6.4.1` | `6.4.2` |
| [@hono/node-server](https://github.com/honojs/node-server) | `1.19.11` | `1.19.14` |
| [axios](https://github.com/axios/axios) | `1.13.6` | `1.15.0` |
| [dompurify](https://github.com/cure53/DOMPurify) | `3.3.3` | `3.4.0` |
| [follow-redirects](https://github.com/follow-redirects/follow-redirects) | `1.15.11` | `1.16.0` |
| [hono](https://github.com/honojs/hono) | `4.12.9` | `4.12.14` |
| [lodash-es](https://github.com/lodash/lodash) | `4.17.23` | `4.18.1` |



Updates `fastify` from 5.8.4 to 5.8.5
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/compare/v5.8.4...v5.8.5)

Updates `nodemailer` from 8.0.4 to 8.0.5
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v8.0.4...v8.0.5)

Updates `vite` from 6.4.1 to 6.4.2
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.4.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.4.2/packages/vite)

Updates `@hono/node-server` from 1.19.11 to 1.19.14
- [Release notes](https://github.com/honojs/node-server/releases)
- [Commits](https://github.com/honojs/node-server/compare/v1.19.11...v1.19.14)

Updates `axios` from 1.13.6 to 1.15.0
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.13.6...v1.15.0)

Updates `dompurify` from 3.3.3 to 3.4.0
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](https://github.com/cure53/DOMPurify/compare/3.3.3...3.4.0)

Updates `follow-redirects` from 1.15.11 to 1.16.0
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.11...v1.16.0)

Updates `hono` from 4.12.9 to 4.12.14
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](https://github.com/honojs/hono/compare/v4.12.9...v4.12.14)

Updates `lodash-es` from 4.17.23 to 4.18.1
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.23...4.18.1)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.8.5
  dependency-type: direct:production
  dependency-group: npm_and_yarn
- dependency-name: nodemailer
  dependency-version: 8.0.5
  dependency-type: direct:production
  dependency-group: npm_and_yarn
- dependency-name: vite
  dependency-version: 6.4.2
  dependency-type: direct:development
  dependency-group: npm_and_yarn
- dependency-name: "@hono/node-server"
  dependency-version: 1.19.14
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: axios
  dependency-version: 1.15.0
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: dompurify
  dependency-version: 3.4.0
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: follow-redirects
  dependency-version: 1.16.0
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: hono
  dependency-version: 4.12.14
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: lodash-es
  dependency-version: 4.18.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 20:35:50 +02:00
Johannes Millan
0e9218bd68 fix(sync): handle data validation errors and improve client ID management
- Separate JsonParseError and SyncDataCorruptedError handlers in sync wrapper
- Handle corrupted remote data gracefully instead of throwing
- Add getOrGenerateClientId() as unified entry point, eliminating dual injection
  of ClientIdService + CLIENT_ID_PROVIDER in snapshot-upload, file-based-encryption,
  and sync-hydration services
- Use crypto.getRandomValues() instead of Math.random() for client ID generation
- Warn user when stored client ID is invalid and must be regenerated
- Fix flaky e2e supersync tests (premature waitForURL match on daily-summary URL)
2026-04-16 17:41:39 +02:00
Johannes Millan
f9cb0f9ef0
fix(sync): build @sp/shared-schema before running server tests (#7233)
The super-sync-server pretest script only ran `prisma generate` but
didn't build the @sp/shared-schema workspace dependency. This caused
15 test suites (and 20+ individual tests) to fail with "Failed to
resolve entry for package @sp/shared-schema" because the dist/ output
didn't exist.

https://claude.ai/code/session_01EsRx9RENAEdsNUqMQZqjGN

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-15 02:05:37 +02:00
dependabot[bot]
ad38eeb2ea
chore(deps): bump the npm_and_yarn group across 1 directory with 7 updates (#7178)
Bumps the npm_and_yarn group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [electron](https://github.com/electron/electron) | `37.10.3` | `41.2.0` |
| [nodemailer](https://github.com/nodemailer/nodemailer) | `8.0.4` | `8.0.5` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `6.4.1` | `6.4.2` |
| [@hono/node-server](https://github.com/honojs/node-server) | `1.19.11` | `1.19.13` |
| [axios](https://github.com/axios/axios) | `1.13.6` | `1.15.0` |
| [hono](https://github.com/honojs/hono) | `4.12.9` | `4.12.12` |
| [lodash-es](https://github.com/lodash/lodash) | `4.17.23` | `4.18.1` |



Updates `electron` from 37.10.3 to 41.2.0
- [Release notes](https://github.com/electron/electron/releases)
- [Commits](https://github.com/electron/electron/compare/v37.10.3...v41.2.0)

Updates `nodemailer` from 8.0.4 to 8.0.5
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v8.0.4...v8.0.5)

Updates `vite` from 6.4.1 to 6.4.2
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.4.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.4.2/packages/vite)

Updates `@hono/node-server` from 1.19.11 to 1.19.13
- [Release notes](https://github.com/honojs/node-server/releases)
- [Commits](https://github.com/honojs/node-server/compare/v1.19.11...v1.19.13)

Updates `axios` from 1.13.6 to 1.15.0
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.13.6...v1.15.0)

Updates `hono` from 4.12.9 to 4.12.12
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](https://github.com/honojs/hono/compare/v4.12.9...v4.12.12)

Updates `lodash-es` from 4.17.23 to 4.18.1
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.23...4.18.1)

---
updated-dependencies:
- dependency-name: electron
  dependency-version: 41.2.0
  dependency-type: direct:development
  dependency-group: npm_and_yarn
- dependency-name: nodemailer
  dependency-version: 8.0.5
  dependency-type: direct:production
  dependency-group: npm_and_yarn
- dependency-name: vite
  dependency-version: 6.4.2
  dependency-type: direct:development
  dependency-group: npm_and_yarn
- dependency-name: "@hono/node-server"
  dependency-version: 1.19.13
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: axios
  dependency-version: 1.15.0
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: hono
  dependency-version: 4.12.12
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: lodash-es
  dependency-version: 4.18.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 16:40:36 +02:00
Johannes Millan
70414145a3
fix(sync): prevent magic link prefetch failures and improve auth resilience (#7175)
* fix(sync): prevent magic link prefetch failures and improve auth resilience

- Make /magic-login prefetch-immune: GET now renders a confirmation page
  instead of consuming the single-use token. A "Log In" button triggers
  POST /api/login/magic-link/verify which does the actual verification.
  This prevents email client link prefetchers (Outlook SafeLinks, Gmail)
  from consuming magic link tokens before users click them.

- Increase rate limits on auth endpoints for shared-IP scenarios (e.g.
  university NAT). Global: 100→500, register: 10→50, login: 5→30,
  verify: 10→50, magic-login page: 10→50.

- Add trustProxy to Fastify so req.ip uses X-Forwarded-For behind
  reverse proxies instead of collapsing all clients to one rate bucket.

- Clear stale SuperSync auth tokens after 3 consecutive AuthFailSPErrors
  instead of never clearing them. Prevents persistent failure loops where
  an invalid token keeps retrying indefinitely until app reinstall.

- Improve registration-to-login UX messaging to emphasize email
  verification is required before login links will work.

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* chore: update package-lock.json from npm install

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): address review feedback for auth resilience

- Remove conflicting style.css link from magic-login page
- Change trustProxy from true to 1 (trust single hop only)
- Increase /login/magic-link rate limit from 30 to 50 per 15min
- Delete dead magic-login-redirect.js (replaced by magic-login-confirm.js)
- Reset auth failure counter on non-auth errors (truly consecutive)
- Add test for counter reset on non-auth error between auth errors

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): increase passkey rate limits and fix CSP inline script violations

- Increase passkey endpoint rate limits to match magic-link equivalents (50/15min)
- Extract inline scripts from /reset-password and /recover-passkey pages
  into external JS files to comply with CSP scriptSrc: ["'self'"]
- Remove dead safeJsonForScript helper (no longer needed)
- Increase /recover-passkey page rate limit to 50/15min for consistency

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): update security tests for escapeHtml and fix recover/passkey rate limit

- Update server-security.spec.ts: tests now check for escapeHtml output
  (&lt;, &quot;) instead of safeJsonForScript output (\u003c), matching
  the new data-token attribute approach
- Increase /recover/passkey rate limit from 30 to 50 per 15min to match
  /login/magic-link (both send email with a secret link)

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

* fix(sync): reset auth failure counter after successful download

Move the counter reset from the final InSync return to right after
downloadRemoteOps() succeeds. This ensures early returns (cancelled,
LWW pending, payload rejected) also break the consecutive-failure
chain, since a successful download confirms auth is working.

https://claude.ai/code/session_01WkbLzvhguQwRDtk7ht38aY

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-10 14:46:57 +02:00
Johannes Millan
d32f7037a3 fix(sync): preserve own vector clock counter across full-state op resets
When a full-state op (SYNC_IMPORT/BACKUP_IMPORT) arrived from another
client, mergeRemoteOpClocks reset the local clock to a "minimal" form
but only preserved the current client's counter from the incoming op's
clock. If the current client had issued ops (e.g. GLOBAL_CONFIG) not
reflected in the incoming full-state op's clock, its counter was dropped,
causing subsequent ops to reuse the same counter value. Downstream clients
then saw these ops as EQUAL (duplicate) and skipped them silently.

Fix: take max(mergedClock[clientId], currentClock[clientId]) when
rebuilding the clock after a full-state op reset.

Also add __SP_E2E_BLOCK_WS_DOWNLOAD flag to WsTriggeredDownloadService
to allow E2E tests to block automatic WS-triggered downloads during
concurrent conflict scenarios.

Fix archive conflict test by blocking WS downloads on Client A during
the concurrent edit phase so it doesn't auto-receive B's rename via
WebSocket before archiving (restoring the intended conflict scenario).

Fix LWW singleton test to assert convergence rather than specific winner.
Fix renameTask helper to avoid Playwright/Angular re-render races.
Fix shepherd.js import paths that broke the Angular dev server build.
2026-04-01 15:41:13 +02:00
Johannes Millan
821a87ba71 chore(deps): upgrade prisma from 5.22.0 to 7.6.0
Migrate super-sync-server to Prisma v7:
- Switch generator from prisma-client-js to prisma-client with local output
- Add prisma.config.ts for datasource URL configuration
- Use PrismaPg driver adapter instead of built-in Rust engine
- Add @prisma/adapter-pg and pg dependencies
- Update all imports from @prisma/client to generated client path
- Fix Buffer→Uint8Array for Bytes fields (Prisma v6 breaking change)
- Update Dockerfile: prisma CLI version, copy prisma.config.ts
- Add DATABASE_URL validation in db.ts
- Add src/generated/ to .gitignore
2026-03-31 20:05:03 +02:00
dependabot[bot]
d691883c5a
chore(deps): bump @types/supertest from 6.0.3 to 7.2.0 (#7042)
Bumps [@types/supertest](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/supertest) from 6.0.3 to 7.2.0.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/supertest)

---
updated-dependencies:
- dependency-name: "@types/supertest"
  dependency-version: 7.2.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-31 13:41:28 +02:00
dependabot[bot]
f9ba198302
chore(deps): bump vitest from 3.2.4 to 4.1.2 (#7036)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 3.2.4 to 4.1.2.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.2/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-version: 4.1.2
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-31 13:39:16 +02:00
Johannes Millan
627a6174c7 fix(sync): install prisma CLI in Docker image for reliable startup
Previously, `npx prisma@5.22.0 migrate deploy` in CMD downloaded prisma
from npm on every container start, making startup slow and dependent on
network availability. Now prisma is installed in the image at build time.

Also increases health check start_period from 10s to 30s to give
migrations time to complete before Docker starts counting failures.
2026-03-30 23:04:54 +02:00
Thorsten Klein
7fa8f12132
feat(sync): add Helm chart and WebSocket push for SuperSync (#6971)
* feat(sync): add Helm chart for SuperSync Kubernetes deployment

Includes Deployment, StatefulSet (PostgreSQL), Service, Ingress,
ConfigMap, Secret, HPA, PDB, NetworkPolicy, and test templates.
Supports both bundled PostgreSQL and external database configurations.

* feat(sync): add WebSocket push notifications for near-realtime sync

Server: Fastify WebSocket plugin with connection manager, app-level
heartbeat (30s), debounced notifications, and per-user routing.
Client: WebSocket service with exponential backoff reconnection,
WS-triggered download service, and reduced polling when connected.

* fix(sync): improve WebSocket error handling and reactivity

Make syncInterval$ reactive to WS connection state, fix Set/Map
mutation during heartbeat iteration, add error handling to WS route
handler, separate JSON parsing from message handling, detect auth
failures in WS-triggered downloads, and add logging to all empty
catch blocks.

* fix(sync): address PR review findings for WebSocket and Helm

Fix race condition in WS-triggered download pipeline by moving
isSyncInProgress filter after debounce and adding guard in
_downloadOps. Add logging to remaining empty catch blocks, fix
missing $NODE_IP in Helm NOTES.txt, and correct inaccurate
comments in values.yaml and ws-triggered-download.service.ts.

* fix(sync): add rate limiting to WebSocket upgrade endpoint

Limit WS connection attempts to 10 per minute per IP to prevent
connection flooding, matching the rate-limit pattern used by other
sync endpoints.

* fix(sync): extract shared constants, fix debounce correctness, replace deprecated toPromise

Extract CLIENT_ID_REGEX and MAX_CLIENT_ID_LENGTH into sync.const.ts
to prevent drift between sync.routes.ts and websocket.routes.ts.

Fix debounce in notifyNewOps to accumulate excluded client IDs across
rapid calls from different clients, preventing self-notifications.

Replace deprecated toPromise() with firstValueFrom in connectWebSocket
and _sync methods. Add .catch() to reconnect path and logging to
silent early returns in _downloadOps.

* fix(build): restore correct package-lock.json and fix upstream lint errors

* revert: restore upstream HTML formatting to match CI prettier config

* fix(sync): use DownloadOutcome discriminated union in WsTriggeredDownloadService

* fix(sync): narrow TokenVerificationResult before accessing userId

* fix(sync): await async getProviderById in connectWebSocket

Missing await caused the Promise object to be cast to
SuperSyncProvider, making getWebSocketParams undefined.

* feat(sync): add SEED_USERS env var to create verified users on startup

For self-hosted single-user setups: set SEED_USERS=email1,email2 to
create verified users on boot and log their access tokens. Skips
existing users. Removes need for SMTP/magic link registration.

Also fix Dockerfile to use npm install instead of npm ci for lockfile
compatibility.

* fix(sync): address PR review feedback for Helm chart and server

Security:
- Remove seed-users.ts (logged full JWT tokens to stdout)
- Add fail assertions for missing jwtSecret and postgresql.password
- Add smtp.user/smtp.password values fields
- Add from: selector to NetworkPolicy ingress
- Add egress rule for external database when postgresql.enabled=false

Correctness:
- Fix PostgreSQL StatefulSet indentation for non-persistent mode
- Fix NOTES.txt panic on empty tls list (use len instead of index)
- Restore npm ci in Dockerfile by using node:24-alpine (ships npm 11)
- Add Recreate deployment strategy when using RWO PVC

Operational:
- Add fail guard preventing HPA maxReplicas > 1 (in-memory WS state)
- Fix PDB to use maxUnavailable instead of minAvailable
- Add WebSocket ingress timeout annotation examples
- Add Prisma db push init container for schema migrations
- Default serviceAccount.automount to false
- Add Chart.yaml maintainers, home, sources metadata

* feat(sync): add ALLOWED_EMAILS env var to restrict registration

Supports fully qualified emails (user@example.com) and domain
wildcards (*@example.com). When unset, all emails are allowed.
Applied to all three registration endpoints (passkey options,
passkey verify, magic link).

* fix(sync): exempt health endpoint from rate limiting

Kubernetes liveness/readiness probes hit /health every 5-15s,
exhausting the global rate limit (100 req/15min) and causing
429 responses that trigger container restarts.

* fix(sync): harden WebSocket, Helm chart, and sync services from multi-agent review

- Pin prisma@5.22.0 in init container and use migrate deploy instead of db push
- Add per-user WebSocket connection limit (max 10) to prevent resource exhaustion
- Add replicaCount > 1 fail guard in deployment template (in-memory WS state)
- Set maxPayload: 1024 on WebSocket plugin (only pong messages expected)
- Remove premature IN_SYNC status from download-only WS-triggered sync
- Fix double removeConnection on error+close events (let close handle cleanup)
- DRY debounce logic in notifyNewOps and store latestSeq on pending entry
- Remove dead fromClientId from NewOpsNotification and WsMessage interfaces
- Add takeUntilDestroyed to _wsProviderCleanup subscription
- Simplify email-allowlist.ts to eager init (eliminate mutable state)
- Guard connectWebSocket at call site for non-SuperSync providers
- Add distinctUntilChanged to syncInterval$ to prevent unnecessary resets
- Add container-level securityContext to PostgreSQL StatefulSet
- Fix HPA maxReplicas default to 1 (matches single-replica constraint)

* fix(sync): restore migration in Dockerfile CMD for non-Helm Docker users

Helm users get migration via init container (runs first, CMD becomes no-op).
Docker-compose/plain Docker users get automatic migration back on startup.

* fix(boards): add missing drag delay for touch and extract shared signal

Add cdkDragStartDelay to board-panel drag items, which was missing
entirely. Extract the repeated `isTouchActive() ? DRAG_DELAY_FOR_TOUCH : 0`
expression into a shared `dragDelayForTouch` computed signal and refactor
all 8 components to use it.

* test(sync): add comprehensive WebSocket test coverage

Add unit tests for the new WebSocket real-time sync notification pipeline:

- WebSocketConnectionService (server): connection lifecycle, max-per-user
  limits, notification debouncing, heartbeat, graceful shutdown (13 tests)
- WebSocket routes validation: token/clientId validation, close codes,
  error handling, validation ordering (18 tests)
- SuperSyncWebSocketService (frontend): reconnection with exponential
  backoff, heartbeat, disconnect cleanup, URL conversion (6 tests)
- WsTriggeredDownloadService: auth error handling, pipeline resilience,
  start() idempotency (3 tests)
- SyncWrapperService: connectWebSocket guards for non-SuperSync, null
  params, and already-connected cases (3 tests)
- E2E realtime push: verifies WS-triggered download between two clients

Quality fixes:
- Replace waitForTimeout with syncAndWait in E2E
- Add explanatory comment for microtask flushing in sync-wrapper spec
- Use getter for isSyncInProgress mock in ws-triggered-download spec

---------

Co-authored-by: Johannes Millan <johannes.millan@gmail.com>
2026-03-30 21:34:30 +02:00
Johannes Millan
6d29e6c02a fix(sync): transmit syncImportReason through sync pipeline
- Thread syncImportReason field from operation creation through compact
  codec, sync providers, server database, and back to receiving clients
- Constrain server Zod schema to z.enum() matching client SyncImportReason
  type for defense-in-depth validation
- Use !== undefined in codec for consistency with entityId/entityIds
- Rename piggybackedImport/incomingSyncImport to piggybackedFullStateOp/
  incomingFullStateOp for accuracy
2026-03-29 20:06:41 +02:00
Johannes Millan
e2145fa2ef fix(sync-server): use compiled JS for monitoring npm scripts in Docker
The monitor:all, analyze-storage, and related npm scripts used tsx
which isn't available in the production Docker image. Switch to
node dist/scripts/*.js and add :dev variants for local development.
2026-03-27 18:05:30 +01:00
Johannes Millan
e6b82f973a fix(sync-server): remove jq dependency from deploy script
Use docker compose Go templates instead of JSON+jq for container
status output in the failure path.
2026-03-27 17:55:49 +01:00
Johannes Millan
4ee5dd4ed7 fix(sync-server): use compiled JS for monitoring scripts in Docker
The production container runs as non-root user `supersync` without tsx
(dev dependency excluded via --omit=dev). Analysis and monitor-all
commands failed because they tried to use tsx/npx tsx which couldn't
write to node_modules. All scripts are already compiled to dist/ during
the Docker build, so use those directly.
2026-03-27 17:33:38 +01:00
dependabot[bot]
3423193d82
chore(deps): bump nodemailer (#6974)
Bumps the npm_and_yarn group with 1 update in the / directory: [nodemailer](https://github.com/nodemailer/nodemailer).


Updates `nodemailer` from 7.0.13 to 8.0.4
- [Release notes](https://github.com/nodemailer/nodemailer/releases)
- [Changelog](https://github.com/nodemailer/nodemailer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodemailer/nodemailer/compare/v7.0.13...v8.0.4)

---
updated-dependencies:
- dependency-name: nodemailer
  dependency-version: 8.0.4
  dependency-type: direct:production
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-27 12:26:07 +01:00
Johannes Millan
602b8002e7 fix(sync-server): harden deploy with Caddyfile validation and container checks
- Pin Caddy image to 2.11-alpine to prevent breaking changes from
  floating tags
- Validate Caddyfile syntax before deploying to catch config errors
  early
- Check all container states after startup to detect crashes before
  waiting on the HTTPS health check
- Show logs from all services on failure, not just supersync
2026-03-23 13:34:05 +01:00
Johannes Millan
db30f92e23 fix(sync-server): remove invalid timeout subdirective from Caddyfile
Caddy v2 reverse_proxy does not support a top-level `timeout`
subdirective. The response_header_timeout in the transport block
already covers long-running operations.
2026-03-23 13:05:40 +01:00
Johannes Millan
c53c8bc38c fix(sync-server): remove no-new-privileges from Caddy container
Caddy Alpine images set file capabilities on the binary via setcap.
The no-new-privileges security option prevents the kernel from honoring
file capabilities during exec, causing "operation not permitted" on
startup. Container remains hardened with cap_drop: ALL + cap_add:
NET_BIND_SERVICE.
2026-03-23 13:03:54 +01:00
Johannes Millan
1eaa95c4fe fix(sync-server): add NET_BIND_SERVICE capability for Caddy container
Newer Caddy Alpine images set file capabilities on the binary, which
requires NET_BIND_SERVICE to bind to ports 80/443. Without it,
cap_drop: ALL prevents execution entirely.
2026-03-23 13:03:54 +01:00
Johannes Millan
5aff8d0aae fix(monitoring): use Prisma.sql for conditional SQL fragments in analyze-storage
Prisma's $queryRaw tagged template treats ${} as parameterized values, not
raw SQL. Conditional fragments like `${userId ? \`WHERE ...\` : ''}` were
sent as string parameters, causing PostgreSQL syntax error 42601.
2026-03-23 13:03:31 +01:00
Johannes Millan
2d7656c5c8 fix(monitoring): mask PII, add types, fix sort and validation issues
Mask user emails by default (--unmask to reveal). Replace all any[]
query results with typed interfaces. Fix showUsageHistory sort that
compared formatted byte strings. Add parseIntArg validation for CLI
flags. Fix unbounded recursion in analyzePayload. Replace execSync with
execFileSync for DATA_DIR to prevent shell injection. Add --no-save flag
for usage command.
2026-03-23 10:52:57 +01:00
Johannes Millan
fe65d81635
Add active users monitoring command with engagement metrics (#6921)
* feat(sync): add active-users command to monitor CLI

Adds a new `active-users` command to the SuperSync monitor script that reports:
- Total registered and verified user counts
- Active users by time period (24h, 7d, 30d, 90d) based on device and sync activity
- New registration counts
- Recently active users table with device count and ops
- Users who never synced

Usage: npm run monitor -- active-users

https://claude.ai/code/session_014Tc5vtXW4Z8QZFMDFWKErP

* feat(sync): add engaged users metric to active-users report

Shows users who were active on 3+ distinct days in the last 2 weeks
with new sync operations, giving a measure of genuine recurring usage.

https://claude.ai/code/session_014Tc5vtXW4Z8QZFMDFWKErP

* docs(sync): add active-users to docker monitoring docs

Add missing active-users command to docker-monitor.sh case statement,
help text, and DOCKER-MONITORING.md guide.

https://claude.ai/code/session_014Tc5vtXW4Z8QZFMDFWKErP

* refactor(sync): improve active-users command from review feedback

- Fix timezone-unsafe DATE(): use AT TIME ZONE 'UTC' with explicit
  double precision cast for consistent day boundaries
- Replace correlated subquery with LEFT JOIN for ops_7d count
- Show total active count when LIMIT truncates the table
- Add --threshold and --limit CLI flags for flexibility
- Combine device/ops metrics into single line (connected / syncing)
- Add skipInQuick to run-all-monitoring.ts
- Update docker-monitor.sh header, help text, and DOCKER-MONITORING.md
  with active-users command, flags, and performance notes
- Clarify "never synced" output label

https://claude.ai/code/session_014Tc5vtXW4Z8QZFMDFWKErP

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-22 23:15:14 +01:00
Johannes Millan
43e04bc407 fix(sync): fix supersync server unit test failures
Add pretest script to run prisma generate before tests, ensuring the
Prisma client is always generated in any environment. Fix verifyToken
mocks across all test files to include valid:true matching the
TokenVerificationResult discriminated union. Expand jsonwebtoken mock
to include JsonWebTokenError/TokenExpiredError stubs needed by auth.ts.
Add $queryRaw to transaction mock in sync-fixes tests.
2026-03-22 10:10:19 +01:00
Johannes Millan
f26732755f fix(sync): prevent transient server errors from wiping auth credentials
Server: verifyToken() catch-all was returning 401 "Invalid token" for ALL
errors including database failures. Now only JWT-specific errors
(JsonWebTokenError, TokenExpiredError) return 401. Database/infrastructure
errors propagate as 500s with sanitized logging.

Client: SuperSync AuthFailSPError no longer clears stored credentials —
the server rejection may be transient. The snackbar with "Configure"
button still shows so users can re-auth on genuine failures. Dropbox and
WebDAV still clear immediately so their OAuth/re-auth flows work.
2026-03-20 21:36:30 +01:00
Johannes Millan
2cc2760486 feat(i18n): complete translations and update locale files
Translate all missing keys to achieve full coverage (2086/2086)
for fr, pt-br, zh-tw, ro, ro-md, zh, es, de, and cs.
Also remove stale keys and include updated translations
for other locales. Remaining languages still have partial coverage.
2026-03-18 21:55:13 +01:00
Johannes Millan
7c0849e953 fix(sync-server): auto-pull latest code in deploy script
Adds git pull --ff-only before docker compose pull so scripts,
docker-compose.yml, and other repo files are updated on each deploy.
Falls back gracefully if pull fails.
2026-03-18 20:51:59 +01:00
Johannes Millan
1907df68d7 fix(sync-server): security hardening and GDPR log compliance
- Remove email addresses from all log messages (~18 locations) for GDPR
  compliance, replacing with userId where available
- Downgrade debug credential logs from info to debug level
- Remove options.user from passkey registration log (contained email in
  JSON payload)
- Add container security hardening to docker-compose.yml:
  - no-new-privileges on all containers
  - cap_drop: ALL on supersync and caddy
  - Memory limits (supersync 512m, postgres 1g, caddy 256m)
  - CPU limit on supersync (1.0)
- Add .health-alert/ to gitignore
- Add health-alert.sh cron script for container monitoring with
  OOM detection, disk checks, and email alerting
2026-03-18 20:48:51 +01:00
Johannes Millan
a01a511c25 feat(sync): add backup strategy with accounts-only dump and disaster recovery docs
- Add accounts-only pg_dump (users + passkeys) to backup script for
  lightweight disaster recovery when clients still have data
- Add pipefail to backup script to catch silent dump failures
- Add test endpoint to simulate partial server revert (ops-after/:serverSeq)
- Add 6 e2e tests covering all disaster recovery scenarios:
  complete data loss, partial revert, accounts-only restore (API + SQL),
  full dump restore + reset account, and all-clients-lost recovery
- Add backup-and-recovery.md with setup, recovery procedures, and
  decision tree
2026-03-18 20:15:45 +01:00
Johannes Millan
609d6a7217 chore(sync): exclude integration test from default run, scope integration config 2026-03-17 13:59:40 +01:00
Johannes Millan
861425fd28 refactor(sync): improve vector clock implementation quality
- Replace O(N) Prisma findMany with PostgreSQL SQL aggregate
  (jsonb_each_text + GROUP BY + MAX) for snapshot vector clock
  computation, avoiding loading all ops' clocks into Node.js memory
- Add defensive regex filter (^[0-9]+$) for non-numeric JSONB values
- Fix || to ?? for nullish coalescing in vector clock value lookups
- Remove unreachable CONCURRENT check in compareVectorClocks
- Remove unused VectorClockMetrics interface and measureVectorClock
- Add 7 new unit tests for SQL aggregate path verification
- Add 10 integration tests running actual SQL against PostgreSQL
2026-03-17 13:59:40 +01:00
Johannes Millan
d401e6169e feat(tasks): add deadlines feature with reminders and planner integration
Add deadline support for tasks with date-only and time-specific deadlines.

Core:
- Add deadlineDay, deadlineWithTime, deadlineRemindAt fields to task model
- Add NgRx actions (setDeadline, removeDeadline, clearDeadlineReminder)
- Add meta-reducer with mutual exclusivity and input validation
- Register deadline actions in ActionType enum and op-log codes

UI:
- Create deadline dialog with calendar, time input, and reminder options
- Add deadline badge to task list row with overdue warning color
- Add deadline item to task detail panel with flag icon
- Add deadline options to task context menu
- Show deadlines in scheduled list page

Reminders:
- Add deadline reminder effects and selectors
- Show reminder dialog with grouped deadline/schedule sections
- Handle deadline reminder clearing on dismiss, done, add-to-today
- Cancel Android native deadline notifications on delete/archive

Planner:
- Show deadline tasks in planner day view and overdue section
- Create dedicated planner-deadline-task component
- Optimize deadline grouping with O(N) single-pass selector

Other:
- Add deadline-today banner effect for unplanned deadline tasks
- Add translation keys for all deadline UI strings
- Add E2E tests for deadline reminder flows
- Add unit tests for deadline reducer and overdue utility
- Extract shared isDeadlineOverdue utility function
- Guard app-state selectors against undefined during teardown
2026-03-15 18:48:00 +01:00
Johannes Millan
baf6124415 fix: address code review findings from yesterday's changes
- Add error handler to markdown paste promise in add-task-bar
- Fix misleading "try again later" in resend cap error message
  (cap is permanent, not time-based)
2026-03-13 11:07:22 +01:00
Johannes Millan
b5536adfe5 feat(sync): add verification resend cap of 20 attempts
Prevents email bombing by capping verificationResendCount at 20 for
both magic link and passkey registration flows. Returns a safe error
message when the cap is reached.
2026-03-12 13:50:50 +01:00
Johannes Millan
3ba5841421 feat(sync): add magic link registration to SuperSync server
Add email-only registration flow (no passkey required) with verification
email. For existing unverified users, email is sent before DB token
update to preserve the old verification link on failure.

- POST /api/register/magic-link endpoint with Zod validation and rate limiting
- registerWithMagicLink function in auth.ts with P2002 race condition handling
- Frontend "Register with Email" button and JS handler
- Export shared VERIFICATION_TOKEN_EXPIRY_MS constant (eliminates duplication)
- 8 unit tests covering all paths including email failure cleanup
2026-03-12 13:36:05 +01:00
dependabot[bot]
620cd3dd52
chore(deps): bump fastify in the npm_and_yarn group across 1 directory (#6753)
Bumps the npm_and_yarn group with 1 update in the / directory: [fastify](https://github.com/fastify/fastify).


Updates `fastify` from 5.7.3 to 5.8.1
- [Release notes](https://github.com/fastify/fastify/releases)
- [Commits](https://github.com/fastify/fastify/compare/v5.7.3...v5.8.1)

---
updated-dependencies:
- dependency-name: fastify
  dependency-version: 5.8.1
  dependency-type: direct:production
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-07 11:02:30 +01:00
Johannes Millan
cce9576946 test(e2e): fix supersync E2E tests for mandatory encryption flow
- Extract shared encryption warning helper and fix race conditions
- Restructure concurrent-import tests to import before enabling sync
- Fix encryption password preservation and change tests
- Add "Change Password" button detection in supersync page object
- Mark backup-import ID mismatch test as fixme (known limitation)
2026-03-06 16:36:38 +01:00
Johannes Millan
3dc86dd183
Fix/add rate limits (#6745)
* fix(build): fix tag resolution and add error handling in bump-android-version

The previous two-tag approach (`currentTag` → `prevTag`) would resolve the
wrong changelog range during `npm version` since the new tag doesn't exist
yet. Simplified to a single `lastTag...HEAD` range which correctly captures
all commits since the last release.

Added try-catch with fallback to last 20 commits when no tags exist, and
a fallback message for empty changelogs.

* fix(sync): differentiate auth error messages and clarify token revocation

- Rename server dashboard "Refresh Token" to "Revoke & Replace Token"
  with explicit warning that ALL devices will be disconnected
- Return rejection reason in 401 responses (revoked, expired, etc.)
  via discriminated union TokenVerificationResult type
- Show different client error messages for server-side token rejection
  vs missing local credentials to aid user diagnosis
- Extract server error reason from JSON response body in AuthFailSPError

Closes #6597

* refactor(sync): use generic auth reason and improve test readability

- Replace 'User not found' and 'Account not verified' with generic
  'Account unavailable' to avoid leaking account state in API responses
- Extract long reason strings to constants in middleware.spec.ts

* fix(sync-server): add per-route rate limits to silence CodeQL alert

Add explicit rate limits to routes that only had the global rate limit:
- GET /api/sync/status: 60/min
- DELETE /api/sync/data: 3/15min (destructive operation)
- GET /api/sync/restore-points: 30/min
- GET /api/sync/restore/:serverSeq: 10/5min (CPU-intensive)
- GET /reset-password: 20/15min
- GET /recover-passkey: 10/15min

These complement the global @fastify/rate-limit (100/15min) and silence
the CodeQL 'Missing rate limiting' alert on authenticated handlers.
2026-03-05 21:18:36 +01:00
Johannes Millan
7070fde51e
Pr 6741 (#6743)
* fix(build): fix tag resolution and add error handling in bump-android-version

The previous two-tag approach (`currentTag` → `prevTag`) would resolve the
wrong changelog range during `npm version` since the new tag doesn't exist
yet. Simplified to a single `lastTag...HEAD` range which correctly captures
all commits since the last release.

Added try-catch with fallback to last 20 commits when no tags exist, and
a fallback message for empty changelogs.

* fix(sync): differentiate auth error messages and clarify token revocation

- Rename server dashboard "Refresh Token" to "Revoke & Replace Token"
  with explicit warning that ALL devices will be disconnected
- Return rejection reason in 401 responses (revoked, expired, etc.)
  via discriminated union TokenVerificationResult type
- Show different client error messages for server-side token rejection
  vs missing local credentials to aid user diagnosis
- Extract server error reason from JSON response body in AuthFailSPError

Closes #6597

* refactor(sync): use generic auth reason and improve test readability

- Replace 'User not found' and 'Account not verified' with generic
  'Account unavailable' to avoid leaking account state in API responses
- Extract long reason strings to constants in middleware.spec.ts
2026-03-05 21:17:59 +01:00
Johannes Millan
da8e509972 refactor(sync): unify JWT expiry to 365 days for all auth methods
Replace separate JWT_EXPIRY_MAGIC_LINK (365d) and JWT_EXPIRY_PASSKEY (7d)
constants with a single JWT_EXPIRY (365d). The auth method only matters
during login — once a JWT is issued, it represents a verified session
regardless of how the user authenticated.
2026-02-16 18:44:27 +01:00
Johannes Millan
2bc73607ff
Add snapshotOpType parameter for correct operation typing (#6482)
* fix(sync): pass correct opType from client to server in snapshot upload

The server was hardcoding opType as 'SYNC_IMPORT' for all snapshot
uploads, losing the distinction between SYNC_IMPORT, BACKUP_IMPORT,
and REPAIR operations. Now the client sends snapshotOpType in the
request body, and the server uses it (with fallback to 'SYNC_IMPORT'
for backward compatibility with legacy clients).

https://claude.ai/code/session_01XnqdpGbFR6VRPdCAbZ2EB9

* test(sync): fix uploadSnapshot test expectations for 9th snapshotOpType arg

The uploadSnapshot call now passes snapshotOpType as a 9th argument.
Update all 5 failing test expectations to include the new argument
(SYNC_IMPORT, BACKUP_IMPORT, or REPAIR) and fix the argument count
check from 8 to 9.

https://claude.ai/code/session_01XnqdpGbFR6VRPdCAbZ2EB9

* fix(sync): add missing snapshotOpType param to file-based sync adapter

The file-based adapter's uploadSnapshot signature was missing the 9th
snapshotOpType parameter added in f328c35, making it inconsistent with
the provider interface. Add the parameter (unused, as file-based sync
has no server-side op log).

https://claude.ai/code/session_01XnqdpGbFR6VRPdCAbZ2EB9

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-13 16:06:39 +01:00
Johannes Millan
454fefcb04
test(sync): add vector clock pruning edge case tests (#6506)
Add tests for previously untested vector clock pruning scenarios:

- Multi-preserve-ID pruning (two low-counter IDs both preserved)
- Overlapping/duplicate preserve IDs (Set deduplication)
- Preserve IDs missing from clock (silent skip, no crash)
- Snapshot vector clock aggregation + pruning integration
- Post-snapshot comparison correctness after pruning
- Server sanitizeVectorClock counter cap at 100M
- DoS cap at 2.5x MAX (reject, not prune)
- Invalid inputs (null, arrays, empty keys, long keys, negative/float values)

https://claude.ai/code/session_01GdsbKoo8eax394j2UyWUu1

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-13 12:42:36 +01:00
Johannes Millan
534f375a54 fix(sync): clear sync cycle cache on deleteAllData and extract ConflictType
Clear _syncCycleCache in _deleteAllData() to prevent stale cached data
from being served within the 30s TTL after a data wipe. Extract
ConflictType and ConflictResult into named types in sync.types.ts to
eliminate duplicated inline unions in sync.service.ts.
2026-02-12 16:27:59 +01:00
Johannes Millan
43b5808a3f fix(sync): harden vector clock sanitization and improve E2E test robustness
- Cap sanitizeVectorClock values at 100M instead of MAX_SAFE_INTEGER (prevents adversarial clock inflation)
- Add early-exit optimization in compareVectorClocks for CONCURRENT detection
- Add test.setTimeout(180000) to token-expiry and lastseq-preservation E2E tests
- Remove dead code (unreachable guard) in compaction E2E test
- Extract shared config helpers (navigateToMiscSettings, toggleSetting, etc.) into e2e/utils/config-helpers.ts
- Fix Client B cleanup in provider-switch test (move to finally block)
- Add clarifying comments for raw SQL schema coupling, CORS header, and mergeRemoteOpClocks caller contract
- Extract enforceStorageQuota helper in sync routes to reduce duplication
- Add conflictType to conflict detection for structured error handling
- Add MAX_CLEANUP_ITERATIONS guard to freeStorageForUpload
2026-02-12 16:27:56 +01:00
Johannes Millan
ab0d979e5e fix(sync): revert lastSeq preservation in deleteAllUserData
Unlike uploadOps clean-slate (which must preserve lastSeq to prevent
sequence reuse for other clients), deleteAllUserData should reset
lastSeq to 0. Account reset intentionally wipes everything and clients
detect the wipe via latestSeq=0, triggering a full state re-upload.
2026-02-12 16:27:56 +01:00
Johannes Millan
5b7c9bdfb7 test(sync): add comprehensive SuperSync test coverage
Add 7 E2E tests (lastSeq preservation, token expiry recovery, cascade
delete, compaction resilience, global config edge cases, provider switch,
multi-tab), 2 server unit tests (replaceToken expiry regression,
middleware auth), 2 client integration tests (IndexedDB error recovery,
schema version sync), and locale-independent sort edge cases for vector
clocks.
2026-02-12 16:27:56 +01:00