mirror of
https://github.com/ether/etherpad-lite.git
synced 2026-07-30 05:23:24 +00:00
986f139a61
* feat(metrics): expose 3 Prometheus counters for the scaling dive Per the spec section 6 of #7756: enables the load-test harness to attribute *where* time goes on the server, not just the gauge headline (CPU / event-loop / memory) the dive doc starts from. New /stats/prometheus rows: - etherpad_pad_users{padId} — gauge, derived from sessioninfos on each scrape. Lets the harness confirm the pad it points at actually has the expected concurrency. - etherpad_changeset_apply_duration_seconds — histogram observed inside handleUserChanges. Separates "apply path is slow" from "fan-out is slow" when latency rises. - etherpad_socket_emits_total{type} — counter at the broadcast emit sites (handleCustomObjectMessage, handleCustomMessage, sendChatMessageToPadClients) and inside the NEW_CHANGES per-socket loop in updatePadClients. Bucketed by message type so the harness can measure the amplification factor of each lever (especially the fan-out batching lever). Metric handles live in a new prom-instruments.ts module rather than in prometheus.ts itself, so PadMessageHandler can import the recording helpers without creating a circular dependency (prometheus.ts already requires PadMessageHandler). Tests: smoke test verifies recordSocketEmit + recordChangesetApply move the underlying counters/histogram. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(metrics): address Qodo review — flag-gate, scope histogram, bound label cardinality Three issues raised on the initial PR: 1. **Feature flag.** Per project compliance rule, new features must be behind a flag and disabled by default. Adds `settings.scalingDiveMetrics` (default `false`). When off, recordSocketEmit() / recordChangesetApply() short-circuit to no-ops and the metrics are never even registered with the Prometheus register. Enable only when running the ether/etherpad-load-test scaling-dive harness. 2. **Histogram scope.** Previously the etherpad_changeset_apply_duration_seconds timer wrapped the whole handleUserChanges() body — including `await exports.updatePadClients(pad)` — so the histogram measured apply+fan-out, defeating its stated purpose. Now stopped immediately after the apply work (`assert.equal(...rev, r)`), before the ACCEPT_COMMIT socket emit and the updatePadClients call. Failed applies deliberately don't observe so the success-path distribution stays clean. 3. **Label cardinality.** handleCustomMessage was passing the user-supplied msgString (an HTTP-API param) directly as the `type` label value. A misbehaving API caller could grow prom-client's internal label map until OOM. Now bucketed against a known-types allowlist; anything outside it lands in `other`. Tests updated: 5/5 — covers happy path, "other" bucketing of unknown/unsafe labels, and that the flag-disabled state is a true no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
67 lines
2.5 KiB
TypeScript
67 lines
2.5 KiB
TypeScript
// Prometheus instruments referenced from the hot path (PadMessageHandler).
|
|
//
|
|
// Defined in a separate file so that PadMessageHandler can import the
|
|
// recording helpers without creating a circular import with prometheus.ts
|
|
// (which already requires PadMessageHandler to read sessioninfos).
|
|
//
|
|
// The metrics themselves are added to the central Registry by prometheus.ts.
|
|
//
|
|
// Everything here is gated behind settings.scalingDiveMetrics (default false).
|
|
// When the flag is off the recording helpers short-circuit to no-ops and the
|
|
// metrics are never registered, so production deployments don't pay for
|
|
// instrumentation they don't use.
|
|
|
|
import client from 'prom-client';
|
|
import settings from './utils/Settings';
|
|
|
|
export const enabled = (): boolean => settings.scalingDiveMetrics === true;
|
|
|
|
export const changesetApplyDuration = new client.Histogram({
|
|
name: 'etherpad_changeset_apply_duration_seconds',
|
|
help: 'Time spent applying an incoming USER_CHANGES message on the server (apply path only, excludes fan-out to other clients)',
|
|
buckets: [0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 5],
|
|
});
|
|
|
|
export const socketEmitsTotal = new client.Counter({
|
|
name: 'etherpad_socket_emits_total',
|
|
help: 'Number of socket.io broadcast emits, bucketed by message type',
|
|
labelNames: ['type'],
|
|
});
|
|
|
|
export const padUsersGauge = new client.Gauge({
|
|
name: 'etherpad_pad_users',
|
|
help: 'Active users connected to a pad, keyed by padId',
|
|
labelNames: ['padId'],
|
|
});
|
|
|
|
// Allowlist of message-type label values. Anything outside this set is rolled
|
|
// into 'other' so a misbehaving plugin or HTTP-API caller passing a
|
|
// user-controlled msgString cannot explode prom-client's internal label-cardinality
|
|
// state.
|
|
const KNOWN_TYPES = new Set([
|
|
'NEW_CHANGES',
|
|
'ACCEPT_COMMIT',
|
|
'CHAT_MESSAGE',
|
|
'CLIENT_VARS',
|
|
'CLIENT_MESSAGE',
|
|
'CUSTOM',
|
|
'USER_NEWINFO',
|
|
'USERINFO_UPDATE',
|
|
'USER_LEAVE',
|
|
]);
|
|
|
|
/** Start a timer for the changeset apply path. Call the returned function when done.
|
|
* Returns a no-op stopper when the feature flag is off. */
|
|
export const recordChangesetApply = (): (() => void) => {
|
|
if (!enabled()) return () => {};
|
|
return changesetApplyDuration.startTimer();
|
|
};
|
|
|
|
/** Increment the socket-emit counter for the given message type.
|
|
* No-op when the feature flag is off. Unknown/missing types are bucketed as
|
|
* 'other' to keep label cardinality bounded. */
|
|
export const recordSocketEmit = (type: string | undefined): void => {
|
|
if (!enabled()) return;
|
|
const label = type && KNOWN_TYPES.has(type) ? type : 'other';
|
|
socketEmitsTotal.labels(label).inc();
|
|
};
|