Address findings from a full sync-system review (blockers + high/medium):
- USE_REMOTE ("Use Server Data") is now a true download-first rebuild:
fetches the complete server history (incl. own/already-known ops via a
raw-download mode) and validates it before any local mutation; aborts
untouched on download failure, empty remote, or newer-schema ops.
- Widen the incoming full-state conflict gate to treat all user work as
meaningful (not just TASK/PROJECT/TAG/NOTE CUD); fix the piggyback race
by judging against a pre-upload pending snapshot.
- Stop advancing the server cursor past ops blocked by schema/migration
failures so they are retried after an app update; block any op from a
newer schema version (no forward-compat band).
- Retry of failed remote ops runs archive side effects only, never
re-dispatching reducers whose effects already committed (fixes additive
double-apply of time-tracking/counter deltas across hydration+retry).
- Use local monotonic seq (not lexical UUIDv7) to decide which ops are
covered by an uploaded snapshot, preventing dropped unsynced ops under
clock rollback.
- Server: include entityIds in duplicate-operation equality.
- Capture buffer no longer silently drops accepted actions at 100 (warns
to reload); drop only past a 5000 pathological cap.
Adds regression coverage for each fix. sync-core 209/209,
super-sync-server 817/817, and all touched Angular specs pass.
|
||
|---|---|---|
| .. | ||
| src | ||
| tests | ||
| .gitignore | ||
| package.json | ||
| README.md | ||
| tsconfig.build.json | ||
| tsconfig.json | ||
| tsconfig.spec.json | ||
| tsup.config.ts | ||
| vitest.config.ts | ||
@sp/sync-core
Framework-agnostic primitives for the Super Productivity sync engine: operation-log types, vector clocks, conflict resolution, gzip compression, and end-to-end encryption. Consumed by the main app and the SuperSync server; no Angular/Electron/Capacitor dependencies.
Encryption
The encryption layer provides Argon2id key derivation and AES-256-GCM authenticated encryption, with a WebCrypto path and an @noble/ciphers fallback for environments where crypto.subtle is unavailable (notably Android Capacitor on http://localhost).
import {
encrypt,
decrypt,
encryptBatch,
decryptBatch,
clearSessionKeyCache,
setLegacyKdfWarningHandler,
} from '@sp/sync-core';
const cipher = await encrypt('hello', password);
const plain = await decrypt(cipher, password);
Wire format (public contract)
| Format | Bytes |
|---|---|
| Argon2id | [SALT (16)] [IV (12)] [AES-GCM ciphertext + auth tag (>= 16)] |
| Legacy | [IV (12)] [AES-GCM ciphertext + auth tag (>= 16)] |
All ciphertexts are base64-encoded for transport. The format is discriminated by length: < 28 bytes is invalid, < 44 bytes is unambiguously legacy, >= 44 bytes is treated as Argon2id with a legacy fallback on auth failure. Do not change this without a versioning migration.
Salt and IV semantics
- The IV (12 bytes) is freshly random per call. AES-GCM security under a fixed key reduces to IV uniqueness, which this guarantees.
- The salt (16 bytes) is derived once per
(process session, password)pair and reused across everyencrypt/encryptBatchcall in that session. This is intentional — it lets the session cache amortize the ~500 ms–2 s Argon2id derivation. Two encryptions of the same plaintext within a session therefore share the salt prefix and differ only in IV and ciphertext. Do not assert per-call salt uniqueness in tests.
Session key caching
encrypt/decrypt/encryptBatch/decryptBatch all share three in-memory caches (encrypt key, decrypt key by salt, legacy PBKDF2 key) that survive across sync cycles. Argon2id derivation is expensive (~500–2000 ms on mobile with the default 64 MiB / 3 iterations); the cache turns repeated syncs from minutes into seconds.
Call clearSessionKeyCache() whenever the user changes their password or logs out. Keys live in memory only and are never persisted.
Legacy-KDF migration
Old data was encrypted with PBKDF2 using the password as its own salt — cryptographically weak. decrypt() and decryptBatch() still read legacy ciphertexts so existing sync data remains accessible.
setLegacyKdfWarningHandler(fn) registers a callback fired on every successful legacy decrypt, regardless of which entry point was used. The host throttles user-facing messages (e.g. show a deprecation banner once per session).
Argon2id parameters
Defaults are OWASP 2023 mobile guidance (parallelism: 1, iterations: 3, memorySize: 64 MiB). Tests can weaken them via setArgon2ParamsForTesting({ ... }) — this throws when called with NODE_ENV === 'production' in Node bundles. Restore defaults by calling with no argument.
Other exports
OpType,Operation,VectorClockand friends — op-log primitive typescompareVectorClocks,mergeVectorClocks,limitVectorClockSize— clock algebraclassifyOpAgainstSyncImport— full-state-import op dispositioncreateSyncFilePrefixHelpers— host-configured file prefix codeccompressWithGzip,decompressGzipFromString— gzip helpersreplayOperationBatch,applyRemoteOperations— replay and apply coordinatorsplanRegularOpsAfterFullStateUpload,planSnapshotHydration, etc. — sync planning
See src/index.ts for the full barrel and the JSDoc on individual symbols for usage.
Tests
npm test # typecheck specs + vitest run, Node WebCrypto + @noble fallback
npm run test:watch # watch mode
npm run build # tsup -> ESM + CJS + .d.ts
Browser-context smoke coverage lives in the consuming app at src/app/op-log/encryption/encryption.browser.spec.ts (Karma + real Chrome).