* feat(sync): add Helm chart for SuperSync Kubernetes deployment Includes Deployment, StatefulSet (PostgreSQL), Service, Ingress, ConfigMap, Secret, HPA, PDB, NetworkPolicy, and test templates. Supports both bundled PostgreSQL and external database configurations. * feat(sync): add WebSocket push notifications for near-realtime sync Server: Fastify WebSocket plugin with connection manager, app-level heartbeat (30s), debounced notifications, and per-user routing. Client: WebSocket service with exponential backoff reconnection, WS-triggered download service, and reduced polling when connected. * fix(sync): improve WebSocket error handling and reactivity Make syncInterval$ reactive to WS connection state, fix Set/Map mutation during heartbeat iteration, add error handling to WS route handler, separate JSON parsing from message handling, detect auth failures in WS-triggered downloads, and add logging to all empty catch blocks. * fix(sync): address PR review findings for WebSocket and Helm Fix race condition in WS-triggered download pipeline by moving isSyncInProgress filter after debounce and adding guard in _downloadOps. Add logging to remaining empty catch blocks, fix missing $NODE_IP in Helm NOTES.txt, and correct inaccurate comments in values.yaml and ws-triggered-download.service.ts. * fix(sync): add rate limiting to WebSocket upgrade endpoint Limit WS connection attempts to 10 per minute per IP to prevent connection flooding, matching the rate-limit pattern used by other sync endpoints. * fix(sync): extract shared constants, fix debounce correctness, replace deprecated toPromise Extract CLIENT_ID_REGEX and MAX_CLIENT_ID_LENGTH into sync.const.ts to prevent drift between sync.routes.ts and websocket.routes.ts. Fix debounce in notifyNewOps to accumulate excluded client IDs across rapid calls from different clients, preventing self-notifications. Replace deprecated toPromise() with firstValueFrom in connectWebSocket and _sync methods. Add .catch() to reconnect path and logging to silent early returns in _downloadOps. * fix(build): restore correct package-lock.json and fix upstream lint errors * revert: restore upstream HTML formatting to match CI prettier config * fix(sync): use DownloadOutcome discriminated union in WsTriggeredDownloadService * fix(sync): narrow TokenVerificationResult before accessing userId * fix(sync): await async getProviderById in connectWebSocket Missing await caused the Promise object to be cast to SuperSyncProvider, making getWebSocketParams undefined. * feat(sync): add SEED_USERS env var to create verified users on startup For self-hosted single-user setups: set SEED_USERS=email1,email2 to create verified users on boot and log their access tokens. Skips existing users. Removes need for SMTP/magic link registration. Also fix Dockerfile to use npm install instead of npm ci for lockfile compatibility. * fix(sync): address PR review feedback for Helm chart and server Security: - Remove seed-users.ts (logged full JWT tokens to stdout) - Add fail assertions for missing jwtSecret and postgresql.password - Add smtp.user/smtp.password values fields - Add from: selector to NetworkPolicy ingress - Add egress rule for external database when postgresql.enabled=false Correctness: - Fix PostgreSQL StatefulSet indentation for non-persistent mode - Fix NOTES.txt panic on empty tls list (use len instead of index) - Restore npm ci in Dockerfile by using node:24-alpine (ships npm 11) - Add Recreate deployment strategy when using RWO PVC Operational: - Add fail guard preventing HPA maxReplicas > 1 (in-memory WS state) - Fix PDB to use maxUnavailable instead of minAvailable - Add WebSocket ingress timeout annotation examples - Add Prisma db push init container for schema migrations - Default serviceAccount.automount to false - Add Chart.yaml maintainers, home, sources metadata * feat(sync): add ALLOWED_EMAILS env var to restrict registration Supports fully qualified emails (user@example.com) and domain wildcards (*@example.com). When unset, all emails are allowed. Applied to all three registration endpoints (passkey options, passkey verify, magic link). * fix(sync): exempt health endpoint from rate limiting Kubernetes liveness/readiness probes hit /health every 5-15s, exhausting the global rate limit (100 req/15min) and causing 429 responses that trigger container restarts. * fix(sync): harden WebSocket, Helm chart, and sync services from multi-agent review - Pin prisma@5.22.0 in init container and use migrate deploy instead of db push - Add per-user WebSocket connection limit (max 10) to prevent resource exhaustion - Add replicaCount > 1 fail guard in deployment template (in-memory WS state) - Set maxPayload: 1024 on WebSocket plugin (only pong messages expected) - Remove premature IN_SYNC status from download-only WS-triggered sync - Fix double removeConnection on error+close events (let close handle cleanup) - DRY debounce logic in notifyNewOps and store latestSeq on pending entry - Remove dead fromClientId from NewOpsNotification and WsMessage interfaces - Add takeUntilDestroyed to _wsProviderCleanup subscription - Simplify email-allowlist.ts to eager init (eliminate mutable state) - Guard connectWebSocket at call site for non-SuperSync providers - Add distinctUntilChanged to syncInterval$ to prevent unnecessary resets - Add container-level securityContext to PostgreSQL StatefulSet - Fix HPA maxReplicas default to 1 (matches single-replica constraint) * fix(sync): restore migration in Dockerfile CMD for non-Helm Docker users Helm users get migration via init container (runs first, CMD becomes no-op). Docker-compose/plain Docker users get automatic migration back on startup. * fix(boards): add missing drag delay for touch and extract shared signal Add cdkDragStartDelay to board-panel drag items, which was missing entirely. Extract the repeated `isTouchActive() ? DRAG_DELAY_FOR_TOUCH : 0` expression into a shared `dragDelayForTouch` computed signal and refactor all 8 components to use it. * test(sync): add comprehensive WebSocket test coverage Add unit tests for the new WebSocket real-time sync notification pipeline: - WebSocketConnectionService (server): connection lifecycle, max-per-user limits, notification debouncing, heartbeat, graceful shutdown (13 tests) - WebSocket routes validation: token/clientId validation, close codes, error handling, validation ordering (18 tests) - SuperSyncWebSocketService (frontend): reconnection with exponential backoff, heartbeat, disconnect cleanup, URL conversion (6 tests) - WsTriggeredDownloadService: auth error handling, pipeline resilience, start() idempotency (3 tests) - SyncWrapperService: connectWebSocket guards for non-SuperSync, null params, and already-connected cases (3 tests) - E2E realtime push: verifies WS-triggered download between two clients Quality fixes: - Replace waitForTimeout with syncAndWait in E2E - Add explanatory comment for microtask flushing in sync-wrapper spec - Use getter for isSyncInProgress mock in ws-triggered-download spec --------- Co-authored-by: Johannes Millan <johannes.millan@gmail.com> |
||
|---|---|---|
| .. | ||
| archive/encryption-attempts-openvz-incompatible | ||
| docs | ||
| helm/supersync | ||
| prisma | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| tools | ||
| .env.example | ||
| .gitignore | ||
| Caddyfile | ||
| docker-compose.build.yml | ||
| docker-compose.monitoring.yml | ||
| docker-compose.yml | ||
| DOCKER-MONITORING.md | ||
| Dockerfile | ||
| Dockerfile.test | ||
| env.example | ||
| package.json | ||
| privacy-policy-en.md | ||
| privacy-policy.md | ||
| README.md | ||
| sync-server-architecture-diagrams.md | ||
| terms-of-service-en.md | ||
| terms-of-service.md | ||
| tsconfig.json | ||
| vitest.config.ts | ||
| vitest.integration.config.ts | ||
SuperSync Server
A custom, high-performance synchronization server for Super Productivity.
Note: This server implements a custom operation-based synchronization protocol (Event Sourcing), not WebDAV. It is designed specifically for the Super Productivity client's efficient sync requirements.
Related Documentation:
- Authentication Architecture - Auth design decisions and security features
- Operation Log Architecture - Client-side architecture
- Server Architecture Diagrams - Visual diagrams
- Backup & Disaster Recovery - Backup setup and recovery procedures
Architecture
The server uses an Append-Only Log architecture backed by PostgreSQL (via Prisma):
- Operations: Clients upload atomic operations (Create, Update, Delete, Move).
- Sequence Numbers: The server assigns a strictly increasing
server_seqto each operation. - Synchronization: Clients request "all operations since sequence
X". - Snapshots: The server can regenerate the full state by replaying operations, optimizing initial syncs.
Key Design Principles
| Principle | Description |
|---|---|
| Server-Authoritative | Server assigns monotonic sequence numbers for total ordering |
| Client-Side Conflict Resolution | Server stores operations as-is; clients detect and resolve conflicts |
| E2E Encryption Support | Payloads can be encrypted client-side; server treats them as opaque blobs |
| Idempotent Uploads | Request ID deduplication prevents duplicate operations |
Quick Start
Docker (Recommended)
The easiest way to run the server is using the provided Docker Compose configuration.
# 1. Copy environment example
cp .env.example .env
# 2. Configure .env (Set JWT_SECRET, DOMAIN, POSTGRES_PASSWORD)
nano .env
# 3. Start the stack (Server + Postgres + Caddy)
docker-compose up -d
Manual Setup (Development)
# Install dependencies
npm install
# Generate Prisma Client
npx prisma generate
# Set up .env
cp .env.example .env
# Edit .env to point to your PostgreSQL instance (DATABASE_URL)
# Push schema to DB
npx prisma db push
# Start the server
npm run dev
# Or build and run
npm run build
npm start
Configuration
All configuration is done via environment variables.
| Variable | Default | Description |
|---|---|---|
PORT |
1900 |
Server port |
DATABASE_URL |
- | PostgreSQL connection string (e.g. postgresql://user:pass@localhost:5432/db) |
JWT_SECRET |
- | Required. Secret for signing JWTs (min 32 chars) |
PUBLIC_URL |
- | Required. Public URL used for email links (e.g. https://sync.example.com) |
CORS_ORIGINS |
https://app.super-productivity.com |
Allowed CORS origins |
SMTP_HOST |
- | SMTP Server for emails |
API Endpoints
Authentication
Register a new user
POST /api/register
Content-Type: application/json
{
"email": "user@example.com",
"password": "yourpassword"
}
Response:
{
"message": "User registered. Please verify your email.",
"id": 1,
"email": "user@example.com"
}
Login
POST /api/login
Content-Type: application/json
{
"email": "user@example.com",
"password": "yourpassword"
}
Response:
{
"token": "jwt-token",
"user": { "id": 1, "email": "user@example.com" }
}
Synchronization
All sync endpoints require Bearer authentication: Authorization: Bearer <jwt-token>
1. Upload Operations
Send new changes to the server.
POST /api/sync/ops
2. Download Operations
Get changes from other devices.
GET /api/sync/ops?sinceSeq=123
3. Get Snapshot
Get the full current state (optimized).
GET /api/sync/snapshot
4. Sync Status
Check pending operations and device status.
GET /api/sync/status
Client Configuration
In Super Productivity, configure the Custom Sync provider with:
- Base URL:
https://sync.your-domain.com(or your deployed URL) - Auth Token: JWT token from login
Maintenance
Scripts
The server includes scripts for administrative tasks. These use the configured database.
# Delete a user account
npm run delete-user -- user@example.com
# Clear sync data (preserves account)
npm run clear-data -- user@example.com
# Clear ALL sync data (dangerous)
npm run clear-data -- --all
API Details
Upload Operations (POST /api/sync/ops)
Request body:
{
"ops": [
{
"id": "uuid-v7",
"opType": "UPD",
"entityType": "TASK",
"entityId": "task-123",
"payload": { "changes": { "title": "New title" } },
"vectorClock": { "clientA": 5 },
"timestamp": 1701234567890,
"schemaVersion": 1
}
],
"clientId": "clientA",
"lastKnownSeq": 100
}
Response:
{
"results": [{ "opId": "uuid-v7", "accepted": true, "serverSeq": 101 }],
"newOps": [],
"latestSeq": 101
}
Download Operations (GET /api/sync/ops)
Query parameters:
sinceSeq(required): Server sequence number to start fromlimit(optional): Max operations to return (default: 500)
Upload Snapshot (POST /api/sync/snapshot)
Used for full-state operations (BackupImport, SyncImport, Repair):
{
"state": {
/* Full AppDataComplete */
},
"clientId": "clientA",
"reason": "initial",
"vectorClock": { "clientA": 10 },
"schemaVersion": 1
}
Security Features
| Feature | Implementation |
|---|---|
| Authentication | JWT Bearer tokens in Authorization header |
| Timing Attack Mitigation | Dummy hash comparison on invalid users |
| Input Validation | Operation ID, entity ID, schema version validated |
| Rate Limiting | Configurable per-user limits |
| Vector Clock Sanitization | Limited to 50 entries, 255 char keys |
| Entity Type Allowlist | Prevents injection of invalid entity types |
| Request Deduplication | Prevents duplicate operations on retry |
Multi-Instance Deployment Considerations
When deploying multiple server instances behind a load balancer, be aware of these limitations:
Passkey Challenge Storage
Issue: WebAuthn challenges are stored in an in-memory Map, which doesn't work across instances.
Symptom: Passkey registration/login fails if the challenge generation request hits instance A but verification hits instance B.
Solution for multi-instance:
- Implement Redis-backed challenge storage
- Or use sticky sessions (less ideal)
Current status: A warning is logged at startup in production if in-memory storage is used.
Snapshot Generation Locks
Issue: Concurrent snapshot generation prevention uses an in-memory Map.
Symptom: Same user may trigger duplicate snapshot computations across different instances.
Impact: Performance only (no data corruption) - snapshots are deterministic.
Solution for multi-instance:
- Implement Redis distributed lock (optional, only for performance)
Single-Instance Deployment
For single-instance deployments, these limitations do not apply. The current implementation is fully functional and well-tested for single-instance use.
Security Notes
- Set JWT_SECRET to a secure random value in production (min 32 characters).
- Use HTTPS in production. The Docker setup includes Caddy to handle this automatically.
- Restrict CORS origins in production.
- Database backups are recommended for production deployments.