feat(updater): tier 3 — auto update with grace window (#7607) (#7720)

* feat(updater): scheduled execution state + graceStartTag dedupe field (#7607)

Preparation for Tier 3 of the auto-update subsystem:
- ExecutionStatus gains `scheduled` (targetTag, scheduledFor, startedAt).
- EmailSendLog gains `graceStartTag` for one-shot grace-start email dedupe.
- state validator accepts the new shape, requires per-status fields,
  and backfills graceStartTag=null on a Tier 1/2 state file.

Plus the implementation plan at
docs/superpowers/plans/2026-05-11-auto-update-pr3-tier3-auto.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(updater): decideSchedule pure decision function (#7607)

Adds src/node/updater/Scheduler.ts with the Tier 3 pure decision logic:
- schedules when canAuto + idle/verified/terminal-cleared
- reschedules when a newer tag appears mid-grace
- emits a grace-start email (once per tag) when adminEmail is set
- cancels a stale schedule when policy flips canAuto off
- no-ops during in-flight / terminal states
- clamps preApplyGraceMinutes to [0, 7 days]

Also extends Notifier's EmailKind union with 'grace-start' so the
decision result types correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(updater): scheduler timer runner with arm/cancel (#7607)

Adds createSchedulerRunner to Scheduler.ts:
- arm(): clears any prior timer, sets a fresh one for scheduledFor
- cancel(): clears the pending timer, idempotent
- past scheduledFor → fires with delay=0 (rehydrate after restart-in-grace)
- single-fire-per-arm semantics; armedFor cleared on fire

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(updater): extract apply pipeline shared by HTTP + scheduler (#7607)

Lifts the preflight → drain → execute orchestration out of the
/admin/update/apply HTTP handler into src/node/updater/applyPipeline.ts.
The HTTP handler keeps its 4xx status mapping; the pipeline owns the
state transitions, lock release, drain coordination, and rollback hand-
off. The new ApplyPipelineDeps interface accepts an onAccepted callback
so the HTTP path can still 202 mid-flow while the Tier 3 scheduler path
(next commit) can no-op.

Adds `scheduled` to the apply allowed-entry list so an admin can "Apply
now" during the Tier 3 grace window.

13 vitest cases cover happy / preflight-failed / cancelled / busy /
lock-held / scheduled-entry / rollback / lock-release. Existing 12
mocha integration tests still pass without change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(updater): wire Tier 3 scheduler into boot + performCheck (#7607)

- expressCreateServer instantiates the scheduler runner and rehydrates
  the timer when a prior boot left state.execution = scheduled
- performCheck evaluates decideSchedule after the notifier pass:
  schedule transitions state + sends grace-start email + arms timer;
  cancel-schedule resets to idle + cancels timer
- shutdown cancels the timer
- exposes cancelScheduler() so the cancel endpoint (next commit) can
  drop the pending schedule
- buildSchedulerApplyDeps() supplies the full production-wired pipeline
  deps (preflight, executor, rollback) for the scheduler-triggered apply

Adds tests/backend/specs/updater-scheduler-integration.ts covering
boot-rehydrate fire-on-past and the decision-to-state round-trip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(updater): cancel handler supports Tier 3 scheduled state (#7607)

POST /admin/update/cancel now accepts execution.status === 'scheduled'
in addition to preflight/draining. The handler calls cancelScheduler()
to drop the pending in-process timer, then transitions state to idle
with lastResult.outcome = 'cancelled' (mirroring the existing pattern).

Adds a Tier 3 integration test that seeds a scheduled state, calls
/admin/update/cancel, and asserts the state machine landed correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(admin): countdown + cancel UI for Tier 3 scheduled updates (#7607)

- store.ts: extend Execution union with the scheduled variant
- UpdatePage.tsx: render countdown panel during scheduled; Apply button
  is relabelled "Apply now" so the admin can skip the remaining grace;
  Cancel button accepts scheduled state
- UpdateBanner.tsx: dedicated scheduled banner with live remaining time
- en.json: new i18n keys (execution.scheduled, banner.scheduled,
  page.scheduled.{title,countdown,apply_now})

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(updater): playwright spec for Tier 3 scheduled UI (#7607)

Three cases against a mocked /admin/update/status:
- countdown panel + Apply now + Cancel render when execution is scheduled
- Cancel button posts /admin/update/cancel and triggers re-fetch
- /admin (banner) shows "Auto-update to <tag> scheduled" copy

Mirrors the existing update-page-actions.spec.ts mock pattern (page.route).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(updater): document Tier 3 auto with grace window (#7607)

- doc/admin/updates.md: flip Tier 3 from "designed, not yet implemented"
  to current; expand preApplyGraceMinutes table row; add a Tier 3
  section explaining schedule / cancel / Apply now / restart-in-grace
  and the grace-start email
- settings.json.template: clarify the preApplyGraceMinutes comment
- CHANGELOG.md: Unreleased entry for Tier 3
- runbook §11: full Tier 3 smoke (happy, cancel, apply-now, restart-in-
  grace, email) plus the additional sign-off checkboxes

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(admin): UpdatePage handles missing execution field; scope spec locator (#7607)

Two CI fixes for PR #7720:

1. UpdatePage.tsx — optional-chain us.execution.status. Integration test
   stubs (update-banner.spec.ts) ship payloads without the Tier 2/3
   execution / lastResult / lockHeld fields; without optional chaining
   on the new scheduled-derivation line the whole page crashed before
   the h1 rendered, breaking the unrelated "renders current version"
   test.

2. update-scheduled.spec.ts — scope the v2.7.2 assertion to the
   .update-scheduled section. The regex was matching three elements
   (banner, countdown panel, changelog link) and tripped Playwright's
   strict-mode locator check.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(updater): address Qodo review (Tier 3 race conditions + tier-off bypass) (#7607)

Four fixes for bugs flagged by Qodo's review of PR #7720:

1. **Tier=off bypasses scheduler** (correctness). expressCreateServer
   used to instantiate the scheduler and rehydrate any persisted
   `scheduled` state regardless of `updates.tier`. A user who set
   `tier: "off"` after a schedule had been persisted would still see
   the timer fire after restart. The boot path now skips scheduler
   creation when tier is off and explicitly clears a stale scheduled
   state to idle (logged so the admin sees what happened).

2. **Timer fire skips state recheck** (reliability). The scheduler's
   timer callback called applyUpdate() directly. Race: admin clicks
   Cancel at the same instant the timer fires, or the tier flips
   during the grace window. Now schedulerTriggerApply re-loads state
   and re-evaluates policy via a new pure decideTriggerApply() helper
   in Scheduler.ts. If state is no longer scheduled (or scheduled for a
   different tag), aborts. If policy now denies auto, persists state
   back to idle and aborts.

3. **Apply-now leaves scheduler timer armed** (correctness). The apply
   endpoint accepts `scheduled` as an entry status but didn't cancel
   the in-process scheduler timer. After the admin clicks Apply now,
   the still-armed timer could later fire and attempt another apply
   (especially if the manual one finishes in preflight-failed, which
   is also an allowed-entry status). Apply handler now calls
   cancelScheduler() when entering from `scheduled`.

4. **scheduledFor not validated as timestamp** (reliability). State
   validator only required scheduledFor / startedAt etc. to be
   non-empty strings; a hand-edited "scheduledFor": "garbage" would
   pass validation and yield NaN delay → immediate fire. The
   validator now requires known timestamp fields to be parseable
   via Date.parse().

Tests: 6 new decideTriggerApply cases + 3 new state.ts validation
cases. 189 vitest pass / 29 mocha integration pass / ts-check clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
John McLear 2026-05-12 20:50:06 +01:00 committed by GitHub
parent 197f007b4b
commit cbe551b432
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
22 changed files with 3470 additions and 152 deletions

File diff suppressed because it is too large Load diff

View file

@ -200,3 +200,47 @@ Tick every line before approving the release that introduces this code:
- [ ] `var/log/update.log` is rotated when it crosses 10 MB (force this by writing >10 MB into the file and triggering an Apply).
If any line is unticked, do not ship the release.
## 11. Tier 3 — grace window, scheduled apply, cancel, restart-in-grace
Configure the VM for tier 3:
```jsonc
{
"updates": {
"tier": "auto",
"preApplyGraceMinutes": 2, // short for smoke
"drainSeconds": 15,
"checkIntervalHours": 1
}
}
```
1. As in §3, `git checkout v2.7.2`. Restart Etherpad. Wait for the immediate first version check (~5s after boot).
2. Confirm a schedule was created:
```bash
curl -fsSL http://localhost:9001/admin/update/status | jq '.execution'
# Expect: {"status":"scheduled","targetTag":"v...","scheduledFor":"...","startedAt":"..."}
```
3. Visit `/admin/update`. Confirm:
- A countdown panel renders with the localised "Etherpad will start updating to vX.Y.Z in Nm Ms." copy.
- Two buttons: **Cancel** and **Apply now**.
4. **Happy path:** wait for the timer to fire. The same flow as §4 (drain, executor, exit 75) runs. State lands on `verified` after the supervisor restarts on the new version.
5. **Cancel path:** repeat steps 13 in a fresh setup. Click **Cancel** during the countdown. Expected:
- State transitions to `idle`; `lastResult.outcome: "cancelled"`.
- `journalctl -u etherpad` shows `cancelled pending schedule (admin-cancellation)`.
- The next version check (within `checkIntervalHours`) re-schedules the same release — this is correct: the schedule was cancelled but the policy still wants the update. To opt out completely, set `updates.tier: "notify"` instead.
6. **Apply-now path:** repeat steps 13. Click **Apply now**. The regular Tier 2 pipeline starts immediately; the previously-armed timer is harmlessly stale (the executor takes over before it fires).
7. **Restart-in-grace path:** repeat steps 13, then `sudo systemctl restart etherpad` mid-countdown. On boot, the journal logs `updater: rehydrating Tier 3 schedule for vX.Y.Z at ...`. `/admin/update` resumes the countdown from the persisted `scheduledFor` (no re-arming, no re-emailing).
8. **Email path** (if `adminEmail` is set): the journal logs `(would send email) ... [Etherpad] Auto-update scheduled for ...` exactly once per scheduled tag. Re-arming for the same tag does not re-email. A different tag arming over the top does.
If any step diverges, capture `var/log/update.log` and stop. Add to the §10 sign-off checklist:
- [ ] Tier 3 schedule transitions execution → `scheduled` after the version check.
- [ ] Countdown panel renders the localised string (not the i18n key).
- [ ] Cancel during scheduled returns to `idle`.
- [ ] Apply now during scheduled runs the Tier 2 pipeline immediately.
- [ ] Restart-in-grace rehydrates the timer.
- [ ] `grace-start` email fires once per tag when `adminEmail` is set.