etherpad-lite/src/node/prometheus.ts
John McLear 986f139a61
feat(metrics): 3 Prometheus counters for scaling dive (#7756) (#7762)
* feat(metrics): expose 3 Prometheus counters for the scaling dive

Per the spec section 6 of #7756: enables the load-test harness to
attribute *where* time goes on the server, not just the gauge headline
(CPU / event-loop / memory) the dive doc starts from.

New /stats/prometheus rows:

- etherpad_pad_users{padId} — gauge, derived from sessioninfos on
  each scrape. Lets the harness confirm the pad it points at actually
  has the expected concurrency.

- etherpad_changeset_apply_duration_seconds — histogram observed
  inside handleUserChanges. Separates "apply path is slow" from
  "fan-out is slow" when latency rises.

- etherpad_socket_emits_total{type} — counter at the broadcast
  emit sites (handleCustomObjectMessage, handleCustomMessage,
  sendChatMessageToPadClients) and inside the NEW_CHANGES per-socket
  loop in updatePadClients. Bucketed by message type so the harness
  can measure the amplification factor of each lever (especially the
  fan-out batching lever).

Metric handles live in a new prom-instruments.ts module rather than
in prometheus.ts itself, so PadMessageHandler can import the
recording helpers without creating a circular dependency
(prometheus.ts already requires PadMessageHandler).

Tests: smoke test verifies recordSocketEmit + recordChangesetApply
move the underlying counters/histogram.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metrics): address Qodo review — flag-gate, scope histogram, bound label cardinality

Three issues raised on the initial PR:

1. **Feature flag.** Per project compliance rule, new features must
   be behind a flag and disabled by default. Adds
   `settings.scalingDiveMetrics` (default `false`). When off,
   recordSocketEmit() / recordChangesetApply() short-circuit to
   no-ops and the metrics are never even registered with the
   Prometheus register. Enable only when running the
   ether/etherpad-load-test scaling-dive harness.

2. **Histogram scope.** Previously the
   etherpad_changeset_apply_duration_seconds timer wrapped the
   whole handleUserChanges() body — including
   `await exports.updatePadClients(pad)` — so the histogram
   measured apply+fan-out, defeating its stated purpose. Now
   stopped immediately after the apply work (`assert.equal(...rev,
   r)`), before the ACCEPT_COMMIT socket emit and the
   updatePadClients call. Failed applies deliberately don't observe
   so the success-path distribution stays clean.

3. **Label cardinality.** handleCustomMessage was passing the
   user-supplied msgString (an HTTP-API param) directly as the
   `type` label value. A misbehaving API caller could grow
   prom-client's internal label map until OOM. Now bucketed against
   a known-types allowlist; anything outside it lands in `other`.

Tests updated: 5/5 — covers happy path, "other" bucketing of
unknown/unsafe labels, and that the flag-disabled state is a true
no-op.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 19:54:42 +01:00

58 lines
2 KiB
TypeScript

import client from 'prom-client';
const db = require('./db/DB').db;
const PadMessageHandler = require('./handler/PadMessageHandler');
const register = new client.Registry();
const gaugeDB = new client.Gauge({
name: 'ueberdb_stats',
help: 'ueberdb stats',
labelNames: ['type'],
});
register.registerMetric(gaugeDB);
const totalUsersGauge = new client.Gauge({
name: 'etherpad_total_users',
help: 'Total number of users',
});
register.registerMetric(totalUsersGauge);
const activePadsGauge = new client.Gauge({
name: 'etherpad_active_pads',
help: 'Total number of active pads',
});
register.registerMetric(activePadsGauge);
// Added for the #7756 scaling dive: lets the load-test harness attribute
// where time goes (apply path vs. fan-out) and confirm per-pad concurrency.
// The metric handles live in prom-instruments.ts to avoid a circular import
// with PadMessageHandler (which records into them on the hot path).
// Gated behind settings.scalingDiveMetrics so production deployments don't
// pay for the instrumentation by default.
import {padUsersGauge, changesetApplyDuration, socketEmitsTotal, enabled as scalingDiveMetricsEnabled} from './prom-instruments';
if (scalingDiveMetricsEnabled()) {
register.registerMetric(padUsersGauge);
register.registerMetric(changesetApplyDuration);
register.registerMetric(socketEmitsTotal);
}
client.collectDefaultMetrics({register});
const monitor = async function () {
for (const [metric, value] of Object.entries(db.metrics)) {
if (typeof value !== 'number') continue;
gaugeDB.set({type: metric}, value);
}
activePadsGauge.set(PadMessageHandler.getActivePadCountFromSessionInfos());
totalUsersGauge.set(PadMessageHandler.getTotalActiveUsers());
if (scalingDiveMetricsEnabled()) {
// Per-pad concurrency: reset to avoid stale labels for pads that drained.
padUsersGauge.reset();
for (const [padId, count] of PadMessageHandler.getPadUsersMap()) {
padUsersGauge.set({padId}, count);
}
}
return register;
};
export default monitor;