refactor(sync-core): configure full-state op classification

This commit is contained in:
Johannes Millan 2026-05-11 20:13:11 +02:00
parent 00658ac67a
commit d2be63c9c4
11 changed files with 376 additions and 111 deletions

View file

@ -1,9 +1,9 @@
# `@sp/sync-core` Extraction Plan
> **Status: In progress - PR 1, PR 2 guardrails/logger adapter work, PR 3a
> vector-clock ownership, and the PR 3b pure helper slices are present on this
> branch. Remaining PR 2 cleanup is PR text alignment plus targeted future
> `SyncLogger` routing for files as they move.**
> vector-clock ownership, full-state op classification config, and the PR 3b
> pure helper slices are present on this branch. Remaining cleanup is PR text
> alignment plus targeted future `SyncLogger` routing for files as they move.**
**Goal:** Carve the sync engine out of `src/app/op-log/` into a reusable,
framework-agnostic, **domain-agnostic** `@sp/sync-core` package, plus a sibling
@ -103,15 +103,22 @@ groundwork:
- `eslint.config.js` applies `no-restricted-imports` and a dynamic-import ban to
`packages/sync-core/**/*.ts`.
- The package currently exports operation primitives, apply types, LWW helper
factory, entity-key helpers, `SyncStateCorruptedError`, entity-registry
contracts, and the privacy-aware logger port.
factory, full-state op-type helper factory, entity-key helpers,
`SyncStateCorruptedError`, entity-registry contracts, and the privacy-aware
logger port.
- The app registry now has `buildEntityRegistry()` and an `ENTITY_REGISTRY`
injection token. Existing helper functions still read the app-side
`ENTITY_CONFIGS` singleton for compatibility.
Current extraction state and remaining immediate debt:
- `FULL_STATE_OP_TYPES` still lives in `@sp/sync-core` as compatibility debt.
- Full-state operation classification is now host-configured via
`createFullStateOpTypeHelpers()`. The SP-facing
`src/app/op-log/core/operation.types.ts` shim instantiates its own
`FULL_STATE_OP_TYPES` and `isFullStateOpType`; the package root keeps
deprecated SP compatibility exports for existing consumers. `OpType.SyncImport`,
`OpType.BackupImport`, and `OpType.Repair` remain in `@sp/sync-core` only as
host-defined compatibility strings.
- Vector-clock compare/merge/prune now lives in `@sp/sync-core`, with
`@sp/shared-schema` re-exporting it for existing client/server imports.
- `SyncLogger` exists, but movable app code still mostly calls `OpLog` directly.
@ -137,14 +144,12 @@ Current extraction state and remaining immediate debt:
Suggested next order:
1. Finish PR 2 documentation and verification.
2. Add the app-side `SyncLogger` adapter before moving `OpLog`-using files.
2. Continue targeted `SyncLogger` routing for files as they become movable.
3. Treat the PR 3b pure conflict-resolution and sync-import slices as complete
for this round.
4. Make full-state operation classification configurable or explicitly mark
those op types as host-defined compatibility strings.
5. Clean up logger/config dependencies before moving compression, prefix, or
4. Clean up logger/config dependencies before moving compression, prefix, or
error helpers.
6. Defer port/orchestration work until the remaining pure/config boundaries are
5. Defer port/orchestration work until the remaining pure/config boundaries are
settled.
## PR 1 - Thin First Slice (#7546)
@ -176,8 +181,15 @@ Source: `packages/sync-core/src/`. All exports come through `index.ts`.
- `OperationLogEntry`, `EntityConflict`, `ConflictResult`, `EntityChange`,
`MultiEntityPayload`.
- `VectorClock = Record<string, number>`.
- `FULL_STATE_OP_TYPES`, `isFullStateOpType`, `isMultiEntityPayload`,
`extractActionPayload`.
- `isMultiEntityPayload`, `extractActionPayload`.
**Full-state op-type helper factory** (`full-state-op-types.ts`):
- `createFullStateOpTypeHelpers<TOpType>(fullStateOpTypes)` returns the
host-owned `FULL_STATE_OP_TYPES` set and `isFullStateOpType` predicate.
- The package keeps deprecated SP compatibility exports for
`FULL_STATE_OP_TYPES` / `isFullStateOpType`, but reusable hosts should
instantiate their own helper instead of using those defaults.
**LWW factory** (`lww-update-action-types.ts`):
@ -202,7 +214,8 @@ Each previously-public symbol path keeps working via thin shims:
- `src/app/op-log/core/operation.types.ts` re-exports generic symbols and
redeclares SP-narrowed `Operation`, `OperationLogEntry`, `EntityChange`,
`EntityConflict`, `ConflictResult`, and `MultiEntityPayload`.
`EntityConflict`, `ConflictResult`, and `MultiEntityPayload`. It also
instantiates `createFullStateOpTypeHelpers()` with SP's full-state op strings.
- `src/app/op-log/core/types/apply.types.ts` redeclares app-narrowed apply
result/options types.
- `src/app/op-log/core/lww-update-action-types.ts` instantiates the LWW helper
@ -222,8 +235,8 @@ Each previously-public symbol path keeps working via thin shims:
- Fix comments that imply `sync-core` depends on `shared-schema`. The build may
run after `shared-schema`, but the package dependency direction must remain
absent.
- Decide whether `FULL_STATE_OP_TYPES` is acceptable compatibility debt for PR 1
or whether it should already become app-configurable.
- Resolved in follow-up: `FULL_STATE_OP_TYPES` is now app-configured via
`createFullStateOpTypeHelpers()`.
### Verification
@ -277,6 +290,8 @@ Already present:
- `EncryptAndCompressHandlerService` now accepts a `SyncLogger` constructor
argument and uses the app adapter by default, proving the direct-constructor
path for package-level classes without changing sync behavior.
- `op-log/encryption/compression-handler.ts` now routes compression failures
through `SyncLogger` + `toSyncLogError()` and logs only safe length metadata.
- A deliberate bad-import check was run with a temporary
`packages/sync-core/src/__boundary-check__.ts` importing `@angular/core`;
`npm run lint:file -- packages/sync-core/src/__boundary-check__.ts` failed on
@ -391,9 +406,9 @@ Initial candidate-file audit:
- `op-log/encryption/encrypt-and-compress-handler.service.ts`: safe prefix and
flag metadata now goes through `SyncLogger`.
- `op-log/encryption/compression-handler.ts`: still logs raw errors directly;
before moving, route failures through `toSyncLogError()` and preserve only
safe counts such as compressed input length.
- `op-log/encryption/compression-handler.ts`: routes failures through
`toSyncLogError()` and preserves only safe counts such as input length. It
remains app-side until compression errors are split from app diagnostics.
- `op-log/core/errors/sync-errors.ts`: not move-ready. Several constructors log
validation params, raw samples, or additional objects; split generic error
classes from app-specific diagnostics before routing through `SyncLogger`.

View file

@ -1,4 +1,4 @@
import { Operation } from './operation.types';
import type { Operation } from './operation.types';
/**
* Result of applying a batch of operations.
@ -6,16 +6,16 @@ import { Operation } from './operation.types';
* Allows callers to handle partial success scenarios where some operations
* were applied before an error occurred.
*/
export interface ApplyOperationsResult {
export interface ApplyOperationsResult<TOperation extends Operation<string> = Operation> {
/** Operations that were successfully applied. */
appliedOps: Operation[];
appliedOps: TOperation[];
/**
* If an error occurred, this contains the failed operation and the error.
* Operations after this one in the batch were NOT applied.
*/
failedOp?: {
op: Operation;
op: TOperation;
error: Error;
};
}

View file

@ -24,7 +24,7 @@ export interface DeepEqualOptions {
}
export interface EntityFrontierContext {
localOpsForEntity: Pick<Operation, 'vectorClock'>[];
localOpsForEntity: Pick<Operation<string>, 'vectorClock'>[];
appliedFrontier: VectorClock | undefined;
snapshotVectorClock: VectorClock | undefined;
snapshotEntityKeys: ReadonlySet<string> | undefined;
@ -41,7 +41,7 @@ export interface ClockCorruptionAdjustmentOptions {
}
export interface LwwConflictResolutionPlan<
TConflict extends EntityConflict = EntityConflict,
TConflict extends EntityConflictLike<Operation<string>> = EntityConflict,
> {
conflict: TConflict;
winner: LwwConflictResolutionWinner;
@ -51,25 +51,29 @@ export interface LwwConflictResolutionPlan<
remoteMaxTimestamp?: number;
}
export interface LwwConflictResolutionPlanningOptions {
isArchiveAction: (op: Operation) => boolean;
export interface LwwConflictResolutionPlanningOptions<
TOperation extends Operation<string> = Operation,
> {
isArchiveAction: (op: TOperation) => boolean;
toEntityKey?: (entityType: string, entityId: string) => string;
}
export interface LocalDeleteRemoteUpdateConversionOptions {
export interface LocalDeleteRemoteUpdateConversionOptions<
TOperation extends Operation<string> = Operation,
> {
payloadKey: string | ((entityType: string) => string);
toLwwUpdateActionType: (entityType: string) => string;
isSingletonEntityId?: (entityId: string) => boolean;
onMissingBaseEntity?: (ctx: {
conflict: EntityConflict;
localDeleteOp: Operation | undefined;
remoteOp: Operation;
conflict: EntityConflictLike<TOperation>;
localDeleteOp: TOperation | undefined;
remoteOp: TOperation;
localDeletePayloadKeys: string[] | undefined;
}) => void;
}
export type EntityConflictLike<TOperation extends Operation> = Omit<
EntityConflict,
export type EntityConflictLike<TOperation extends Operation<string>> = Omit<
EntityConflict<Operation<string>>,
'localOps' | 'remoteOps'
> & {
localOps: TOperation[];
@ -77,7 +81,7 @@ export type EntityConflictLike<TOperation extends Operation> = Omit<
};
export interface LwwResolvedConflict<
TOperation extends Operation = Operation,
TOperation extends Operation<string> = Operation,
TConflict extends EntityConflictLike<TOperation> = EntityConflictLike<TOperation>,
> {
conflict: TConflict;
@ -86,14 +90,16 @@ export interface LwwResolvedConflict<
}
export interface LwwResolutionPartitionOptions<
TOperation extends Operation,
TOperation extends Operation<string>,
TConflict extends EntityConflictLike<TOperation> = EntityConflictLike<TOperation>,
> {
processRemoteWinnerOps?: (conflict: TConflict) => TOperation[];
toEntityKey?: (entityType: string, entityId: string) => string;
}
export interface LwwResolutionPartitions<TOperation extends Operation = Operation> {
export interface LwwResolutionPartitions<
TOperation extends Operation<string> = Operation,
> {
localWinsCount: number;
remoteWinsCount: number;
remoteWinsOps: TOperation[];
@ -106,7 +112,7 @@ export interface LwwResolutionPartitions<TOperation extends Operation = Operatio
const resolvePayloadKey = (
entityType: string,
payloadKey: LocalDeleteRemoteUpdateConversionOptions['payloadKey'],
payloadKey: LocalDeleteRemoteUpdateConversionOptions<Operation<string>>['payloadKey'],
): string => (typeof payloadKey === 'function' ? payloadKey(entityType) : payloadKey);
const getPayloadKeys = (payload: unknown): string[] | undefined => {
@ -166,10 +172,10 @@ export const extractUpdateChanges = (
* and singleton-id semantics so the core remains domain-agnostic.
*/
export const convertLocalDeleteRemoteUpdatesToLww = <
TOperation extends Operation = Operation,
TOperation extends Operation<string> = Operation,
>(
conflict: EntityConflictLike<TOperation>,
options: LocalDeleteRemoteUpdateConversionOptions,
options: LocalDeleteRemoteUpdateConversionOptions<TOperation>,
): TOperation[] => {
const localDeleteOp = conflict.localOps.find((op) => op.opType === OpType.Delete);
@ -281,7 +287,7 @@ export const deepEqual = (
* the same resulting state.
*/
export const isIdenticalConflict = (
conflict: EntityConflict,
conflict: EntityConflict<Operation<string>>,
logger: SyncLogger = NOOP_SYNC_LOGGER,
): boolean => {
const { localOps, remoteOps } = conflict;
@ -324,9 +330,9 @@ export const isIdenticalConflict = (
/**
* Suggests a conservative default conflict resolution for the UI/orchestrator.
*/
export const suggestConflictResolution = (
localOps: Operation[],
remoteOps: Operation[],
export const suggestConflictResolution = <TOperation extends Operation<string>>(
localOps: TOperation[],
remoteOps: TOperation[],
): ConflictResolutionSuggestion => {
if (localOps.length === 0) return 'remote';
if (remoteOps.length === 0) return 'local';
@ -363,10 +369,11 @@ export const suggestConflictResolution = (
* be created and which app-side factory should create it.
*/
export const planLwwConflictResolutions = <
TConflict extends EntityConflict = EntityConflict,
TOperation extends Operation<string> = Operation,
TConflict extends EntityConflictLike<TOperation> = EntityConflictLike<TOperation>,
>(
conflicts: TConflict[],
options: LwwConflictResolutionPlanningOptions,
options: LwwConflictResolutionPlanningOptions<TOperation>,
): Array<LwwConflictResolutionPlan<TConflict>> => {
const toEntityKey =
options.toEntityKey ??
@ -443,7 +450,7 @@ export const planLwwConflictResolutions = <
* pending local ops are superseded.
*/
export const partitionLwwResolutions = <
TOperation extends Operation = Operation,
TOperation extends Operation<string> = Operation,
TConflict extends EntityConflictLike<TOperation> = EntityConflictLike<TOperation>,
TResolution extends LwwResolvedConflict<TOperation, TConflict> = LwwResolvedConflict<
TOperation,

View file

@ -0,0 +1,43 @@
import { OpType } from './operation.types';
export interface FullStateOpTypeHelpers<TOpType extends string = string> {
FULL_STATE_OP_TYPES: ReadonlySet<TOpType>;
isFullStateOpType: (opType: string) => opType is TOpType;
}
/**
* Creates host-owned helpers for classifying operations that replace full state.
*
* Full-state operation names are application wire conventions, so sync-core
* only provides the classification helper and the host supplies the op strings.
*/
export const createFullStateOpTypeHelpers = <TOpType extends string>(
fullStateOpTypes: readonly TOpType[],
): FullStateOpTypeHelpers<TOpType> => {
const fullStateOpTypeSet: ReadonlySet<TOpType> = new Set(fullStateOpTypes);
return {
FULL_STATE_OP_TYPES: fullStateOpTypeSet,
isFullStateOpType: (opType: string): opType is TOpType =>
fullStateOpTypeSet.has(opType as TOpType),
};
};
const compatibilityFullStateOpTypeHelpers = createFullStateOpTypeHelpers<OpType>([
OpType.SyncImport,
OpType.BackupImport,
OpType.Repair,
]);
/**
* @deprecated Super Productivity compatibility export. Hosts should instantiate
* their own helper with `createFullStateOpTypeHelpers()`.
*/
export const FULL_STATE_OP_TYPES =
compatibilityFullStateOpTypeHelpers.FULL_STATE_OP_TYPES;
/**
* @deprecated Super Productivity compatibility export. Hosts should instantiate
* their own helper with `createFullStateOpTypeHelpers()`.
*/
export const isFullStateOpType = compatibilityFullStateOpTypeHelpers.isFullStateOpType;

View file

@ -1,11 +1,5 @@
// Operation log primitives — the generic, app-agnostic core of the sync engine.
export {
OpType,
FULL_STATE_OP_TYPES,
isFullStateOpType,
isMultiEntityPayload,
extractActionPayload,
} from './operation.types';
export { OpType, isMultiEntityPayload, extractActionPayload } from './operation.types';
export type {
VectorClock,
Operation,
@ -35,6 +29,14 @@ export type {
SyncImportFilterKeepReason,
} from './sync-import-filter';
// Full-state operation classification helper. Hosts supply their own op strings.
export {
FULL_STATE_OP_TYPES,
createFullStateOpTypeHelpers,
isFullStateOpType,
} from './full-state-op-types';
export type { FullStateOpTypeHelpers } from './full-state-op-types';
// LWW (Last-Writer-Wins) update action-type helpers — factory parameterized by
// the host application's entity-type list, so the lib stays domain-agnostic.
export { createLwwUpdateActionTypeHelpers } from './lww-update-action-types';

View file

@ -8,15 +8,24 @@ export enum OpType {
Delete = 'DEL',
Move = 'MOV', // For list reordering
Batch = 'BATCH', // For bulk operations (import, mass update)
/** Replaces entire app state from remote sync. All concurrent ops are discarded. */
/**
* Host-defined full-state compatibility string used by Super Productivity.
* Reusable core logic must receive full-state classification from the host.
*/
SyncImport = 'SYNC_IMPORT',
/** Replaces entire app state from backup file. All concurrent ops are discarded. */
/**
* Host-defined full-state compatibility string used by Super Productivity.
* Reusable core logic must receive full-state classification from the host.
*/
BackupImport = 'BACKUP_IMPORT',
/** Auto-repair operation containing full repaired state. */
/**
* Host-defined full-state compatibility string used by Super Productivity.
* Reusable core logic must receive full-state classification from the host.
*/
Repair = 'REPAIR',
}
export interface Operation {
export interface Operation<TOpType extends string = OpType> {
/**
* Unique identifier for the operation.
* Should be a UUID v7 (time-ordered) to allow for rough chronological sorting
@ -36,7 +45,7 @@ export interface Operation {
* High-level operation category (Create, Update, Delete, etc.).
* Used for broad logic handling like persistence strategies or conflict resolution rules.
*/
opType: OpType;
opType: TOpType;
// SCOPE
/**
@ -92,7 +101,7 @@ export interface Operation {
schemaVersion: number;
}
export interface OperationLogEntry {
export interface OperationLogEntry<TOperation extends Operation<string> = Operation> {
/**
* Local, monotonic auto-increment integer (IndexedDB primary key).
* Strictly orders operations as they arrived or were generated on THIS specific device.
@ -102,7 +111,7 @@ export interface OperationLogEntry {
/**
* The operation data itself (the synchronized part).
*/
op: Operation;
op: TOperation;
/**
* Local timestamp (epoch ms) indicating when this operation was written to the LOCAL database.
@ -141,39 +150,20 @@ export interface OperationLogEntry {
retryCount?: number;
}
export interface EntityConflict {
export interface EntityConflict<TOperation extends Operation<string> = Operation> {
entityType: string;
entityId: string;
localOps: Operation[];
remoteOps: Operation[];
localOps: TOperation[];
remoteOps: TOperation[];
suggestedResolution: 'local' | 'remote' | 'merge' | 'manual';
mergedPayload?: unknown;
}
export interface ConflictResult {
nonConflicting: Operation[];
conflicts: EntityConflict[];
export interface ConflictResult<TOperation extends Operation<string> = Operation> {
nonConflicting: TOperation[];
conflicts: EntityConflict<TOperation>[];
}
// =============================================================================
// FULL-STATE OPERATION HELPERS
// =============================================================================
/**
* OpTypes that contain full application state in their payload.
*/
export const FULL_STATE_OP_TYPES = new Set<OpType>([
OpType.SyncImport,
OpType.BackupImport,
OpType.Repair,
]);
/**
* Type guard to check if an operation is a full-state operation.
*/
export const isFullStateOpType = (opType: OpType | string): boolean =>
FULL_STATE_OP_TYPES.has(opType as OpType);
// =============================================================================
// MULTI-ENTITY OPERATIONS
// =============================================================================
@ -181,10 +171,10 @@ export const isFullStateOpType = (opType: OpType | string): boolean =>
/**
* Represents a single entity change within a multi-entity operation.
*/
export interface EntityChange {
export interface EntityChange<TOpType extends string = OpType> {
entityType: string;
entityId: string;
opType: OpType;
opType: TOpType;
/**
* The actual changes:
* - For Create: Full entity object
@ -198,9 +188,9 @@ export interface EntityChange {
* Payload wrapper for multi-entity operations.
* Contains the original action payload plus all entity changes computed from state diff.
*/
export interface MultiEntityPayload {
export interface MultiEntityPayload<TOpType extends string = OpType> {
actionPayload: Record<string, unknown>;
entityChanges: EntityChange[];
entityChanges: EntityChange<TOpType>[];
}
/**

View file

@ -0,0 +1,104 @@
import { describe, expect, it } from 'vitest';
import {
FULL_STATE_OP_TYPES,
createFullStateOpTypeHelpers,
isFullStateOpType,
} from '../src/full-state-op-types';
import {
partitionLwwResolutions,
planLwwConflictResolutions,
} from '../src/conflict-resolution';
import { OpType } from '../src/operation.types';
import type {
ConflictResult,
EntityConflict,
MultiEntityPayload,
Operation,
OperationLogEntry,
} from '../src/operation.types';
describe('createFullStateOpTypeHelpers', () => {
it('classifies only the host-supplied op types as full-state operations', () => {
const helpers = createFullStateOpTypeHelpers([
'SYNC_IMPORT',
'BACKUP_IMPORT',
'REPAIR',
] as const);
expect(helpers.FULL_STATE_OP_TYPES.has('SYNC_IMPORT')).toBe(true);
expect(helpers.FULL_STATE_OP_TYPES.has('BACKUP_IMPORT')).toBe(true);
expect(helpers.FULL_STATE_OP_TYPES.has('REPAIR')).toBe(true);
expect(helpers.isFullStateOpType('SYNC_IMPORT')).toBe(true);
expect(helpers.isFullStateOpType('CRT')).toBe(false);
});
it('does not hard-code Super Productivity full-state op names', () => {
const helpers = createFullStateOpTypeHelpers(['HOST_FULL_REPLACE'] as const);
expect(helpers.FULL_STATE_OP_TYPES.has('HOST_FULL_REPLACE')).toBe(true);
expect(helpers.isFullStateOpType('HOST_FULL_REPLACE')).toBe(true);
expect(helpers.isFullStateOpType('SYNC_IMPORT')).toBe(false);
});
it('keeps deprecated Super Productivity compatibility exports', () => {
expect(FULL_STATE_OP_TYPES.has(OpType.SyncImport)).toBe(true);
expect(FULL_STATE_OP_TYPES.has(OpType.BackupImport)).toBe(true);
expect(FULL_STATE_OP_TYPES.has(OpType.Repair)).toBe(true);
expect(isFullStateOpType(OpType.SyncImport)).toBe(true);
expect(isFullStateOpType(OpType.Create)).toBe(false);
});
it('allows host-specific operation opType strings across exported wrappers', () => {
type HostOpType = OpType | 'HOST_FULL_REPLACE';
type HostOperation = Operation<HostOpType>;
const hostOp: HostOperation = {
id: 'op-1',
actionType: '[Host] Replace State',
opType: 'HOST_FULL_REPLACE',
entityType: 'HOST_ENTITY',
payload: {},
clientId: 'client-1',
vectorClock: { client1: 1 },
timestamp: 1,
schemaVersion: 1,
};
const entry: OperationLogEntry<HostOperation> = {
seq: 1,
op: hostOp,
appliedAt: 1,
source: 'local',
};
const conflict: EntityConflict<HostOperation> = {
entityType: 'HOST_ENTITY',
entityId: 'entity-1',
localOps: [hostOp],
remoteOps: [hostOp],
suggestedResolution: 'manual',
};
const result: ConflictResult<HostOperation> = {
nonConflicting: [hostOp],
conflicts: [conflict],
};
const multiEntityPayload: MultiEntityPayload<HostOpType> = {
actionPayload: {},
entityChanges: [
{
entityType: 'HOST_ENTITY',
entityId: 'entity-1',
opType: 'HOST_FULL_REPLACE',
changes: {},
},
],
};
const plans = planLwwConflictResolutions<HostOperation>([conflict], {
isArchiveAction: (op: HostOperation): boolean => op.actionType === '[Host] Archive',
});
const partitions = partitionLwwResolutions<HostOperation>(plans);
expect(entry.op.opType).toBe('HOST_FULL_REPLACE');
expect(result.conflicts[0].localOps[0].opType).toBe('HOST_FULL_REPLACE');
expect(multiEntityPayload.entityChanges[0].opType).toBe('HOST_FULL_REPLACE');
expect(partitions.remoteWinsOps[0].opType).toBe('HOST_FULL_REPLACE');
});
});

View file

@ -14,18 +14,26 @@ import type {
} from '@sp/sync-core';
import type { EntityType as SharedEntityType } from '@sp/shared-schema';
import { ENTITY_TYPES } from '@sp/shared-schema';
import {
createFullStateOpTypeHelpers,
isMultiEntityPayload as libIsMultiEntityPayload,
OpType,
} from '@sp/sync-core';
import { ActionType } from './action-types.enum';
export {
OpType,
FULL_STATE_OP_TYPES,
isFullStateOpType,
extractActionPayload,
} from '@sp/sync-core';
import { isMultiEntityPayload as libIsMultiEntityPayload } from '@sp/sync-core';
export { OpType, extractActionPayload } from '@sp/sync-core';
export type { VectorClock };
export { ENTITY_TYPES, ActionType };
const fullStateOpTypeHelpers = createFullStateOpTypeHelpers<OpType>([
OpType.SyncImport,
OpType.BackupImport,
OpType.Repair,
]);
export const FULL_STATE_OP_TYPES = fullStateOpTypeHelpers.FULL_STATE_OP_TYPES;
export const isFullStateOpType = fullStateOpTypeHelpers.isFullStateOpType;
/**
* Entity type Super Productivity's domain set, sourced from `@sp/shared-schema`
* so client and server agree on the union.

View file

@ -1,3 +1,4 @@
import { NOOP_SYNC_LOGGER } from '@sp/sync-core';
import { OpLog } from '../../core/log';
import { DecompressError } from '../core/errors/sync-errors';
import {
@ -9,6 +10,7 @@ import {
describe('compression-handler', () => {
beforeEach(() => {
spyOn(OpLog, 'err').and.stub();
spyOn(OpLog, 'log').and.stub();
spyOn(console, 'log').and.stub();
});
@ -46,6 +48,45 @@ describe('compression-handler', () => {
);
});
it('logs only sanitized error metadata on decompression failure', async () => {
(OpLog.err as jasmine.Spy).calls.reset();
const invalidGzip = btoa('not gzip data');
await expectAsync(decompressGzipFromString(invalidGzip)).toBeRejectedWithError(
DecompressError,
);
expect(OpLog.err).toHaveBeenCalledOnceWith(
'[compression-handler] gzip decompression failed',
jasmine.objectContaining({ name: jasmine.any(String) }),
{
inputLength: invalidGzip.length,
sanitizedInputLength: invalidGzip.length,
},
);
expect((OpLog.err as jasmine.Spy).calls.mostRecent().args[1] instanceof Error)
.withContext('logged error identity must be sanitized')
.toBeFalse();
const additionalLogText = (OpLog.log as jasmine.Spy).calls
.allArgs()
.flat()
.join('\n');
expect(additionalLogText).not.toContain('not gzip data');
});
it('preserves DecompressError when the injected logger throws', async () => {
const throwingLogger = {
...NOOP_SYNC_LOGGER,
err: () => {
throw new Error('logger failed');
},
};
await expectAsync(
decompressGzipFromString(btoa('not gzip data'), throwingLogger),
).toBeRejectedWithError(DecompressError);
});
// Issue #6581: decompressGzipFromString should sanitize base64 input
// to handle data corruption from WebDAV servers, partial uploads, etc.

View file

@ -1,5 +1,11 @@
import { CompressError, DecompressError } from '../core/errors/sync-errors';
import { OpLog } from '../../core/log';
import type { SyncLogMeta, SyncLogger } from '@sp/sync-core';
import { toSyncLogError } from '@sp/sync-core';
import {
CompressError,
DecompressError,
extractErrorMessage,
} from '../core/errors/sync-errors';
import { OP_LOG_SYNC_LOGGER } from '../core/sync-logger.adapter';
/**
* Reads all bytes from a ReadableStream without using the Response constructor,
@ -29,12 +35,43 @@ const readAllBytes = async (
return result;
};
const logCompressionError = (
logger: SyncLogger,
message: string,
error: unknown,
meta: SyncLogMeta,
): void => {
try {
logger.err(message, toSyncLogError(error), meta);
} catch {
// Logging is best-effort and must not mask the original compression error.
}
};
const toSafeErrorForWrapper = (error: unknown): Error => {
const syncLogError = toSyncLogError(error);
const safeError = new Error(extractErrorMessage(error) ?? syncLogError.name);
safeError.name = syncLogError.name;
if (syncLogError.code !== undefined) {
Object.defineProperty(safeError, 'code', {
value: syncLogError.code,
enumerable: false,
});
}
return safeError;
};
/**
* Compresses a string using gzip and returns the raw bytes.
* Use this for binary transmission (e.g., HTTP with Content-Encoding: gzip).
*/
// eslint-disable-next-line prefer-arrow/prefer-arrow-functions
export async function compressWithGzip(input: string): Promise<Uint8Array> {
export async function compressWithGzip(
input: string,
logger: SyncLogger = OP_LOG_SYNC_LOGGER,
): Promise<Uint8Array> {
try {
const stream = new CompressionStream('gzip');
const writer = stream.writable.getWriter();
@ -47,8 +84,10 @@ export async function compressWithGzip(input: string): Promise<Uint8Array> {
const [, compressed] = await Promise.all([writePromise, readPromise]);
return compressed;
} catch (error) {
OpLog.err(error);
throw new CompressError(error);
logCompressionError(logger, '[compression-handler] gzip compression failed', error, {
inputLength: input.length,
});
throw new CompressError(toSafeErrorForWrapper(error));
}
}
@ -57,7 +96,10 @@ export async function compressWithGzip(input: string): Promise<Uint8Array> {
* Use this for JSON payloads where binary data needs string encoding.
*/
// eslint-disable-next-line prefer-arrow/prefer-arrow-functions
export async function compressWithGzipToString(input: string): Promise<string> {
export async function compressWithGzipToString(
input: string,
logger: SyncLogger = OP_LOG_SYNC_LOGGER,
): Promise<string> {
try {
const stream = new CompressionStream('gzip');
const writer = stream.writable.getWriter();
@ -79,8 +121,15 @@ export async function compressWithGzipToString(input: string): Promise<string> {
reader.readAsDataURL(new Blob([compressed as unknown as BlobPart]));
});
} catch (error) {
OpLog.err(error);
throw new CompressError(error);
logCompressionError(
logger,
'[compression-handler] gzip string compression failed',
error,
{
inputLength: input.length,
},
);
throw new CompressError(toSafeErrorForWrapper(error));
}
}
@ -108,6 +157,7 @@ const sanitizeBase64 = (input: string): string => {
// eslint-disable-next-line prefer-arrow/prefer-arrow-functions
export async function decompressGzipFromString(
compressedBase64: string,
logger: SyncLogger = OP_LOG_SYNC_LOGGER,
): Promise<string> {
try {
const sanitized = sanitizeBase64(compressedBase64);
@ -128,10 +178,15 @@ export async function decompressGzipFromString(
const decoded = new TextDecoder().decode(decompressed);
return decoded;
} catch (error) {
OpLog.err(error);
if (compressedBase64) {
OpLog.err('base64 input length:', compressedBase64.length);
}
throw new DecompressError(error);
logCompressionError(
logger,
'[compression-handler] gzip decompression failed',
error,
{
inputLength: compressedBase64.length,
sanitizedInputLength: sanitizeBase64(compressedBase64).length,
},
);
throw new DecompressError(toSafeErrorForWrapper(error));
}
}

View file

@ -86,7 +86,7 @@ export class EncryptAndCompressHandlerService {
);
let dataStr = JSON.stringify(data);
if (isCompress) {
dataStr = await compressWithGzipToString(dataStr);
dataStr = await compressWithGzipToString(dataStr, this._logger);
}
if (isEncrypt) {
if (!encryptKey || encryptKey.length === 0) {
@ -152,7 +152,7 @@ export class EncryptAndCompressHandlerService {
}
if (isCompressed) {
outStr = await decompressGzipFromString(outStr);
outStr = await decompressGzipFromString(outStr, this._logger);
}
let parsedData: T;