fix: page sessionstorage cleanup to avoid OOM (#7830) (#7831)

* fix: page sessionstorage cleanup to avoid OOM (#7830)

SessionStore._cleanup() previously called `findKeys('sessionstorage:*',
null)`, materialising every session key into a single array. On decade-
old MariaDB installs with millions of sessions this OOMs the node
process within ~15 minutes — see #7830.

Switch to ueberdb2 6.1.0's findKeysPaged with a 500-key page size, and
yield to the event loop between pages so the DB driver can release each
page's buffered rows and request handlers can interleave.

The break is now driven by `page.length === 0` rather than `page.length
< CLEANUP_PAGE_SIZE` so a stubbed/throttled paged source still iterates
the full keyspace.

Adds a regression test that seeds 50 sessionstorage rows, monkey-patches
`DB.findKeysPaged` to use a 4-key page, runs cleanup, and asserts every
expired row is removed plus every valid row preserved across page
boundaries.

Closes #7830

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address Qodo review on #7831

Four follow-ups raised by Qodo on the session cleanup paging fix:

- DB.ts: fail-fast at init() if any required wrapper method (incl.
  findKeysPaged) is missing, so a stale ueberdb2 pin surfaces at boot
  rather than crashing the first cleanup run an hour later.
- SessionStore: bound a single _cleanup() run to 10 minutes. Under
  sustained session creation the keyspace can grow faster than cleanup
  drains it; without a budget the next scheduled run would never fire.
  When the budget hits, log a warning and let the next run continue.
- SessionStore: log the defensive `page[0] <= after` cursor-stall break.
  Previously the loop exited silently, leaving expired rows behind with
  no operator-visible signal of the backend regression.
- Tests: the paged-cleanup regression test now removes both expiredSids
  AND validSids in finally, so a failed assertion doesn't leak rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: note paged session cleanup in CHANGELOG + settings template

CHANGELOG.md picks up an entry under 3.1.0 Notable fixes describing the
OOM cause, the paged iteration, the 10-minute per-run budget, the
cursor-stall logging, and the fail-fast init guard.

settings.json.template's sessionCleanup comment adds the page-size,
budget, and pointer to #7830 so admins can reason about the new
behaviour from the template alone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate lockfile against ueberdb2 6.1.2

Now that ether/ueberDB#983 unblocked the publish workflow (OIDC trusted
publishing), ueberdb2 6.1.2 is live on npm and the `^6.1.0` pin in
src/package.json resolves cleanly. Resolves the ERR_PNPM_OUTDATED_LOCKFILE
that was blocking CI on this PR.

29 SessionStore backend tests still green against the published tarball.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
John McLear 2026-05-22 10:56:35 +01:00 committed by GitHub
parent 0d40f2d049
commit d0180033e4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 204 additions and 24 deletions

View file

@ -634,6 +634,12 @@
/*
* Whether to periodically clean up expired and stale sessions from the
* database. Set to false to disable. Default: true.
*
* Cleanup runs hourly and walks the sessionstorage:* keyspace in 500-key
* pages so very large session tables don't pin the keys in memory all at
* once (see https://github.com/ether/etherpad/issues/7830). A single run
* is capped at 10 minutes; if your DB is so large the budget is reached,
* a warning is logged and the next scheduled run continues.
*/
"sessionCleanup": true
},