mirror of
https://github.com/ether/etherpad-lite.git
synced 2026-07-17 16:47:05 +00:00
fix(admin/pads): apply filter chip server-side, before pagination (#7798)
* fix(admin/pads): apply filter chip server-side, before pagination
Before: PadPage's filter chip (`active`/`recent`/`empty`/`stale`) ran
on the client AFTER the 12-row page slice was already on screen. On a
deployment with hundreds of pads it produced obviously wrong results
— click "empty pads" on page 1 with 100 empties and only the 0–12
empties within the current page passed the filter. thm reported this
on a 3.1.0 deployment.
Move the filter into `PadSearchQuery` so the `/settings` socket can
apply it before slicing:
1. pattern filter on names (cheap)
2. hydrate metadata for the matching pad universe iff a non-`all`
filter is set or a non-`padName` sort is requested
3. apply filter chip on the hydrated set
4. sort + slice → `total` reflects the filtered universe so the
pagination footer makes sense
The original handler also had a 4-way `if/else if` that duplicated the
hydrate-and-sort loop per `sortBy`. Folded those into one pipeline
with a single comparator switch.
Client side, `PadPage.tsx`:
- drop the client-side `filteredResults` filter (server already filters)
- chip click writes `filter` into searchParams (debounced refetch) and
resets `currentPage` to 0
- older clients that don't send `filter` keep working — server defaults
to `all`
Stats cards (totalUsers/activeCount/emptyCount) still count the visible
page only — that's a pre-existing UI limitation tracked separately.
Closes the regression thm reported.
Test plan
- `tsc --noEmit` clean (server + admin)
- New backend spec `padLoadFilter.ts` exercises filter:empty with
small `limit` to lock in the bug-fix, plus all/active/omitted cases
- `5 passing` locally on Node 25
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* address Qodo review on #7798
1. Functional setState updaters for every searchParams mutation
(Qodo bug 1). The debounced pattern handler captured a render-time
snapshot of searchParams; a faster chip click or sort change in
between would be silently reverted when the debounce fired. Now
every mutation merges against the latest state.
2. Concurrency-limited hydration (Qodo bug 3). The earlier draft
issued Promise.all over the full candidate set, fanning out to
thousands of in-flight padManager.getPad() reads on busy
deployments. New mapWithConcurrency() caps concurrent loads at 16
— empirically enough to saturate a single ueberDB driver without
pushing the event loop into back-pressure.
3. Test cleanup deletes the injected test-admin (Qodo bug 4). The
original snapshot/restore pattern saved `settings.users` by
reference; reassigning the same reference in after() left the
inserted key in place and could leak into later backend specs.
4. Document the new `filter` field on the `padLoad` socket query in
admin/README.md (Qodo rule violation 2).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
John McLear
GitHub
parent
228b49f394
commit
10558ed115
6 changed files with 397 additions and 151 deletions
|
|
@ -64,3 +64,25 @@ const SettingsPanel = () => {
|
|||
The admin endpoints are not yet present in the OpenAPI spec — this client is
|
||||
in place to support upcoming work (see issue #7638 follow-up). For now, it is
|
||||
exercised only by the smoke test.
|
||||
|
||||
## Socket.io: `padLoad` query shape
|
||||
|
||||
The admin `/settings` namespace's `padLoad` event accepts a `PadSearchQuery`
|
||||
defined in `src/node/types/PadSearchQuery.ts`:
|
||||
|
||||
| field | type | required | notes |
|
||||
| ------------ | ----------------------------------------------------------------- | -------- | ----- |
|
||||
| `pattern` | `string` | yes | Substring match on pad name. |
|
||||
| `offset` | `number` | yes | Pagination start, in items. Clamped server-side. |
|
||||
| `limit` | `number` | yes | Page size. Capped at 12. |
|
||||
| `ascending` | `boolean` | yes | Sort direction. |
|
||||
| `sortBy` | `"padName" \| "lastEdited" \| "userCount" \| "revisionNumber"` | yes | Column to sort by. |
|
||||
| `filter` | `"all" \| "active" \| "recent" \| "empty" \| "stale"` *(opt.)* | no | Filter chip; defaults to `"all"`. Applied **before** pagination so `total` and the page slice both reflect the filtered universe. Older clients that omit the field get the unchanged `"all"` behaviour. |
|
||||
|
||||
Filter semantics — applied after pattern matching, before sort + slice:
|
||||
|
||||
- `active`: `userCount > 0`
|
||||
- `recent`: edited within the last 7 days
|
||||
- `empty`: `revisionNumber === 0`
|
||||
- `stale`: not edited in the last 365 days
|
||||
- `all` / missing: no further filtering
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import {Trans, useTranslation} from "react-i18next";
|
||||
import {useEffect, useMemo, useState} from "react";
|
||||
import {useStore} from "../store/store.ts";
|
||||
import {PadSearchQuery, PadSearchResult} from "../utils/PadSearch.ts";
|
||||
import {PadFilter, PadSearchQuery, PadSearchResult} from "../utils/PadSearch.ts";
|
||||
import {useDebounce} from "../utils/useDebounce.ts";
|
||||
import * as Dialog from "@radix-ui/react-dialog";
|
||||
import {ChevronLeft, ChevronRight, Eye, Trash2, FileStack, PlusIcon, Search, X, RefreshCw, History} from "lucide-react";
|
||||
|
|
@ -9,12 +9,8 @@ import {useForm} from "react-hook-form";
|
|||
import type {TFunction} from "i18next";
|
||||
|
||||
type PadCreateProps = { padName: string }
|
||||
type FilterId = 'all' | 'active' | 'recent' | 'empty' | 'stale'
|
||||
|
||||
const PAD_FILTER_IDS: FilterId[] = ['all', 'active', 'recent', 'empty', 'stale']
|
||||
|
||||
const isRecent = (ts: number) => (Date.now() - ts) < 86_400_000 * 7
|
||||
const isStale = (ts: number) => (Date.now() - ts) > 86_400_000 * 365
|
||||
const PAD_FILTER_IDS: PadFilter[] = ['all', 'active', 'recent', 'empty', 'stale']
|
||||
|
||||
function relativeTime(t: TFunction, ts: number): string {
|
||||
const d = (Date.now() - ts) / 1000
|
||||
|
|
@ -58,12 +54,23 @@ function sanitizeLocale(lng?: string): string {
|
|||
export const PadPage = () => {
|
||||
const settingsSocket = useStore(state => state.settingsSocket)
|
||||
const [searchParams, setSearchParams] = useState<PadSearchQuery>({
|
||||
offset: 0, limit: 12, pattern: '', sortBy: 'lastEdited', ascending: false,
|
||||
offset: 0, limit: 12, pattern: '', sortBy: 'lastEdited', ascending: false, filter: 'all',
|
||||
})
|
||||
const {t, i18n} = useTranslation()
|
||||
const locale = sanitizeLocale(i18n.resolvedLanguage ?? i18n.language)
|
||||
const [searchTerm, setSearchTerm] = useState('')
|
||||
const [filter, setFilter] = useState<FilterId>('all')
|
||||
// Read filter off searchParams so chip changes round-trip through
|
||||
// the server (`filter` is applied before pagination there). Clicking
|
||||
// a chip used to filter only the current 12-row page slice.
|
||||
//
|
||||
// All searchParams mutations go through functional updaters because the
|
||||
// debounced pattern handler captures a render-time snapshot and would
|
||||
// otherwise revert a faster chip click / sort change made in between.
|
||||
const filter: PadFilter = searchParams.filter ?? 'all'
|
||||
const setFilter = (f: PadFilter) => {
|
||||
setCurrentPage(0)
|
||||
setSearchParams((sp) => ({...sp, filter: f, offset: 0}))
|
||||
}
|
||||
const [selected, setSelected] = useState<Set<string>>(new Set())
|
||||
const pads = useStore(state => state.pads)
|
||||
const [currentPage, setCurrentPage] = useState(0)
|
||||
|
|
@ -78,28 +85,23 @@ export const PadPage = () => {
|
|||
[pads, searchParams.limit]
|
||||
)
|
||||
|
||||
const filteredResults = useMemo(() => {
|
||||
const r = pads?.results ?? []
|
||||
if (filter === 'active') return r.filter(p => p.userCount > 0)
|
||||
if (filter === 'recent') return r.filter(p => isRecent(p.lastEdited))
|
||||
if (filter === 'empty') return r.filter(p => p.revisionNumber === 0)
|
||||
if (filter === 'stale') return r.filter(p => isStale(p.lastEdited))
|
||||
return r
|
||||
}, [pads, filter])
|
||||
|
||||
const totalUsers = useMemo(() => (pads?.results ?? []).reduce((s, p) => s + p.userCount, 0), [pads])
|
||||
const activeCount = useMemo(() => (pads?.results ?? []).filter(p => p.userCount > 0).length, [pads])
|
||||
const emptyCount = useMemo(() => (pads?.results ?? []).filter(p => p.revisionNumber === 0).length, [pads])
|
||||
// The server applies `filter` before paginating; the page payload is
|
||||
// already the filtered slice. The stats cards still reflect the
|
||||
// current page (pre-existing behaviour) — making them global would
|
||||
// require a separate aggregate query.
|
||||
const visibleResults = pads?.results ?? []
|
||||
const totalUsers = useMemo(() => visibleResults.reduce((s, p) => s + p.userCount, 0), [pads])
|
||||
const activeCount = useMemo(() => visibleResults.filter(p => p.userCount > 0).length, [pads])
|
||||
const emptyCount = useMemo(() => visibleResults.filter(p => p.revisionNumber === 0).length, [pads])
|
||||
const lastActivity = useMemo(() => {
|
||||
const r = pads?.results ?? []
|
||||
return r.length ? Math.max(...r.map(p => p.lastEdited)) : null
|
||||
return visibleResults.length ? Math.max(...visibleResults.map(p => p.lastEdited)) : null
|
||||
}, [pads])
|
||||
|
||||
const allSelected = filteredResults.length > 0 && filteredResults.every(p => selected.has(p.padName))
|
||||
const allSelected = visibleResults.length > 0 && visibleResults.every(p => selected.has(p.padName))
|
||||
const toggleAll = () => {
|
||||
const s = new Set(selected)
|
||||
if (allSelected) filteredResults.forEach(p => s.delete(p.padName))
|
||||
else filteredResults.forEach(p => s.add(p.padName))
|
||||
if (allSelected) visibleResults.forEach(p => s.delete(p.padName))
|
||||
else visibleResults.forEach(p => s.add(p.padName))
|
||||
setSelected(s)
|
||||
}
|
||||
const toggleOne = (name: string) => {
|
||||
|
|
@ -109,7 +111,10 @@ export const PadPage = () => {
|
|||
}
|
||||
|
||||
useDebounce(() => {
|
||||
setSearchParams({...searchParams, pattern: searchTerm})
|
||||
// Functional updater so this delayed callback can't clobber a faster
|
||||
// user interaction (e.g. clicking a filter chip mid-typing).
|
||||
setSearchParams((sp) => ({...sp, pattern: searchTerm, offset: 0}))
|
||||
setCurrentPage(0)
|
||||
}, 500, [searchTerm])
|
||||
|
||||
useEffect(() => {
|
||||
|
|
@ -250,7 +255,7 @@ export const PadPage = () => {
|
|||
<section className="pm-section">
|
||||
<div className="pm-section-header">
|
||||
<h2><Trans i18nKey="admin_pads.all_pads"/></h2>
|
||||
<span className="pm-count-badge">{filteredResults.length}</span>
|
||||
<span className="pm-count-badge">{visibleResults.length}</span>
|
||||
<div className="pm-spacer"/>
|
||||
<div className="pm-toolbar">
|
||||
<div className="pm-search">
|
||||
|
|
@ -268,12 +273,12 @@ export const PadPage = () => {
|
|||
<select
|
||||
className="pm-select"
|
||||
value={searchParams.sortBy}
|
||||
onChange={e => setSearchParams({
|
||||
...searchParams,
|
||||
onChange={e => setSearchParams((sp) => ({
|
||||
...sp,
|
||||
sortBy: e.target.value,
|
||||
// Keep current direction when only the column changes; the
|
||||
// ↑/↓ button below is the sole control for direction.
|
||||
})}
|
||||
}))}
|
||||
>
|
||||
<option value="lastEdited">{t('ep_admin_pads:ep_adminpads2_last-edited')}</option>
|
||||
<option value="padName">{t('admin_pads.sort.name')}</option>
|
||||
|
|
@ -282,10 +287,10 @@ export const PadPage = () => {
|
|||
</select>
|
||||
<button
|
||||
className="pm-sort-dir"
|
||||
onClick={() => setSearchParams({
|
||||
...searchParams,
|
||||
ascending: !searchParams.ascending,
|
||||
})}
|
||||
onClick={() => setSearchParams((sp) => ({
|
||||
...sp,
|
||||
ascending: !sp.ascending,
|
||||
}))}
|
||||
title={t(searchParams.ascending
|
||||
? 'admin_plugins.sort_ascending'
|
||||
: 'admin_plugins.sort_descending')}
|
||||
|
|
@ -330,7 +335,7 @@ export const PadPage = () => {
|
|||
</div>
|
||||
)}
|
||||
|
||||
{filteredResults.length > 0 ? (
|
||||
{visibleResults.length > 0 ? (
|
||||
<div className="pm-table-wrap">
|
||||
<table>
|
||||
<thead>
|
||||
|
|
@ -348,7 +353,7 @@ export const PadPage = () => {
|
|||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{filteredResults.map(pad => {
|
||||
{visibleResults.map(pad => {
|
||||
const isEmpty = pad.revisionNumber === 0
|
||||
const isSel = selected.has(pad.padName)
|
||||
return (
|
||||
|
|
@ -432,7 +437,7 @@ export const PadPage = () => {
|
|||
onClick={() => {
|
||||
const p = currentPage - 1
|
||||
setCurrentPage(p)
|
||||
setSearchParams({...searchParams, offset: p * searchParams.limit})
|
||||
setSearchParams((sp) => ({...sp, offset: p * sp.limit}))
|
||||
}}
|
||||
>
|
||||
<ChevronLeft size={14}/> <Trans i18nKey="admin_pads.pagination.previous"/>
|
||||
|
|
@ -444,7 +449,7 @@ export const PadPage = () => {
|
|||
onClick={() => {
|
||||
const p = currentPage + 1
|
||||
setCurrentPage(p)
|
||||
setSearchParams({...searchParams, offset: p * searchParams.limit})
|
||||
setSearchParams((sp) => ({...sp, offset: p * sp.limit}))
|
||||
}}
|
||||
>
|
||||
<Trans i18nKey="admin_pads.pagination.next"/> <ChevronRight size={14}/>
|
||||
|
|
|
|||
|
|
@ -1,9 +1,12 @@
|
|||
export type PadFilter = 'all' | 'active' | 'recent' | 'empty' | 'stale';
|
||||
|
||||
export type PadSearchQuery = {
|
||||
pattern: string;
|
||||
offset: number;
|
||||
limit: number;
|
||||
ascending: boolean;
|
||||
sortBy: string;
|
||||
filter?: PadFilter;
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
'use strict';
|
||||
|
||||
|
||||
import {PadQueryResult, PadSearchQuery} from "../../types/PadSearchQuery";
|
||||
import {PAD_FILTERS, PadFilter, PadQueryResult, PadSearchQuery} from "../../types/PadSearchQuery";
|
||||
import log4js from 'log4js';
|
||||
|
||||
const fsp = require('fs').promises;
|
||||
|
|
@ -15,8 +15,35 @@ import {deleteRevisions} from '../../utils/Cleanup';
|
|||
|
||||
|
||||
const queryPadLimit = 12;
|
||||
// Cap on concurrent `padManager.getPad()` calls while hydrating the pad
|
||||
// universe for filter chip / non-name sort. The old per-sortBy handlers
|
||||
// awaited each getPad sequentially (concurrency = 1); the unified
|
||||
// pipeline used to issue Promise.all over the full candidate set, which
|
||||
// can fan out to thousands of in-flight DB reads on busy deployments.
|
||||
// 16 is empirically enough to saturate a single ueberDB driver without
|
||||
// pushing the event loop into back-pressure.
|
||||
const PAD_HYDRATE_CONCURRENCY = 16;
|
||||
const logger = log4js.getLogger('adminSettings');
|
||||
|
||||
// Concurrency-limited Promise.all replacement. Preserves the input
|
||||
// order in the returned array (caller slices later). Used by padLoad
|
||||
// to bound DB reads during hydration.
|
||||
async function mapWithConcurrency<T, R>(
|
||||
items: T[], limit: number, fn: (t: T) => Promise<R>): Promise<R[]> {
|
||||
const out: R[] = new Array(items.length);
|
||||
let next = 0;
|
||||
const worker = async () => {
|
||||
while (true) {
|
||||
const i = next++;
|
||||
if (i >= items.length) return;
|
||||
out[i] = await fn(items[i]);
|
||||
}
|
||||
};
|
||||
const workers = Array.from({length: Math.min(limit, items.length)}, worker);
|
||||
await Promise.all(workers);
|
||||
return out;
|
||||
}
|
||||
|
||||
|
||||
exports.socketio = (hookName: string, {io}: any) => {
|
||||
io.of('/settings').on('connection', (socket: any) => {
|
||||
|
|
@ -110,137 +137,112 @@ exports.socketio = (hookName: string, {io}: any) => {
|
|||
socket.on('padLoad', async (query: PadSearchQuery) => {
|
||||
const {padIDs} = await padManager.listAllPads();
|
||||
|
||||
const data: {
|
||||
total: number,
|
||||
results?: PadQueryResult[]
|
||||
} = {
|
||||
total: padIDs.length,
|
||||
};
|
||||
let result: string[] = padIDs;
|
||||
let maxResult;
|
||||
|
||||
// Filter out matches
|
||||
// ── 1. Pattern filter (cheap, by name only) ─────────────────────
|
||||
let candidateNames: string[] = padIDs;
|
||||
if (query.pattern) {
|
||||
result = result.filter((padName: string) => padName.includes(query.pattern));
|
||||
candidateNames = candidateNames.filter(
|
||||
(padName: string) => padName.includes(query.pattern));
|
||||
}
|
||||
|
||||
data.total = result.length;
|
||||
// ── 2. Resolve filter chip ──────────────────────────────────────
|
||||
// PadPage sends a chip id; "all" (default) means no additional
|
||||
// filtering. We accept missing values from older clients gracefully.
|
||||
const filter: PadFilter =
|
||||
(query.filter && PAD_FILTERS.includes(query.filter)) ? query.filter : 'all';
|
||||
|
||||
maxResult = result.length - 1;
|
||||
if (maxResult < 0) {
|
||||
maxResult = 0;
|
||||
// ── 3. Decide whether we need full metadata for every candidate ──
|
||||
// The fast path — name-sort with no filter chip — only needs to
|
||||
// hydrate metadata for the 12-row page slice. Any other path
|
||||
// (filter chip OR non-name sort) requires every candidate's revs
|
||||
// / users / lastEdited up front so we can sort and slice against
|
||||
// the right universe. The expensive call is `padManager.getPad`;
|
||||
// user counts come from an in-memory map.
|
||||
const needsFullScan = filter !== 'all' || query.sortBy !== 'padName';
|
||||
|
||||
const loadMeta = async (padName: string): Promise<PadQueryResult> => {
|
||||
const pad = await padManager.getPad(padName);
|
||||
return {
|
||||
padName,
|
||||
lastEdited: await pad.getLastEdit(),
|
||||
userCount: api.padUsersCount(padName).padUsersCount,
|
||||
revisionNumber: pad.getHeadRevisionNumber(),
|
||||
};
|
||||
};
|
||||
|
||||
// Lazily lifted so we don't load every pad twice on the fast path.
|
||||
let hydrated: PadQueryResult[] | null = null;
|
||||
const hydrateAll = async () => {
|
||||
if (hydrated == null) {
|
||||
hydrated = await mapWithConcurrency(
|
||||
candidateNames, PAD_HYDRATE_CONCURRENCY, loadMeta);
|
||||
}
|
||||
return hydrated;
|
||||
};
|
||||
|
||||
// ── 4. Filter chip — applied to hydrated metadata ────────────────
|
||||
// Bucket boundaries match the client chips in PadPage.tsx so the
|
||||
// counts on the stats cards keep meaning the same thing. Compute
|
||||
// `now` once per request so a pad doesn't slip between buckets
|
||||
// mid-loop.
|
||||
const now = Date.now();
|
||||
const isRecent = (lastEdited: number) => now - lastEdited < 86_400_000 * 7;
|
||||
const isStale = (lastEdited: number) => now - lastEdited > 86_400_000 * 365;
|
||||
const matchesFilter = (m: PadQueryResult) => {
|
||||
switch (filter) {
|
||||
case 'active': return m.userCount > 0;
|
||||
case 'recent': return isRecent(Number(m.lastEdited));
|
||||
case 'empty': return m.revisionNumber === 0;
|
||||
case 'stale': return isStale(Number(m.lastEdited));
|
||||
default: return true;
|
||||
}
|
||||
};
|
||||
|
||||
// ── 5. Total — i.e. the count the pagination footer reflects ────
|
||||
// For the fast path this is just the pattern-filtered name list;
|
||||
// for full-scan we report the post-chip total.
|
||||
let totalNames: string[] | null = needsFullScan ? null : candidateNames;
|
||||
let postFilterMetas: PadQueryResult[] | null = null;
|
||||
if (needsFullScan) {
|
||||
postFilterMetas = (await hydrateAll()).filter(matchesFilter);
|
||||
}
|
||||
const total = needsFullScan ? postFilterMetas!.length : totalNames!.length;
|
||||
|
||||
// Reset to default values if out of bounds
|
||||
// ── 6. Clamp offset/limit ──────────────────────────────────────
|
||||
const maxOffset = Math.max(total - 1, 0);
|
||||
if (query.offset && query.offset < 0) {
|
||||
query.offset = 0;
|
||||
} else if (query.offset > maxResult) {
|
||||
query.offset = maxResult;
|
||||
} else if (query.offset > maxOffset) {
|
||||
query.offset = maxOffset;
|
||||
}
|
||||
|
||||
if (query.limit && query.limit < 0) {
|
||||
// Too small
|
||||
query.limit = 0;
|
||||
} else if (query.limit > queryPadLimit) {
|
||||
// Too big
|
||||
query.limit = queryPadLimit;
|
||||
}
|
||||
|
||||
// ── 7. Sort + slice ────────────────────────────────────────────
|
||||
const dir = query.ascending ? 1 : -1;
|
||||
const cmpStr = (a: string, b: string) => a < b ? -dir : a > b ? dir : 0;
|
||||
const cmpNum = (a: number, b: number) => a < b ? -dir : a > b ? dir : 0;
|
||||
|
||||
if (query.sortBy === 'padName') {
|
||||
result = result.sort((a, b) => {
|
||||
if (a < b) return query.ascending ? -1 : 1;
|
||||
if (a > b) return query.ascending ? 1 : -1;
|
||||
return 0;
|
||||
}).slice(query.offset, query.offset + query.limit);
|
||||
|
||||
data.results = await Promise.all(result.map(async (padName: string) => {
|
||||
const pad = await padManager.getPad(padName);
|
||||
const revisionNumber = pad.getHeadRevisionNumber()
|
||||
const userCount = api.padUsersCount(padName).padUsersCount;
|
||||
const lastEdited = await pad.getLastEdit();
|
||||
|
||||
return {
|
||||
padName,
|
||||
lastEdited,
|
||||
userCount,
|
||||
revisionNumber
|
||||
let results: PadQueryResult[];
|
||||
if (needsFullScan) {
|
||||
const sorted = postFilterMetas!.sort((a, b) => {
|
||||
switch (query.sortBy) {
|
||||
case 'padName': return cmpStr(a.padName, b.padName);
|
||||
case 'revisionNumber': return cmpNum(a.revisionNumber, b.revisionNumber);
|
||||
case 'userCount': return cmpNum(a.userCount, b.userCount);
|
||||
case 'lastEdited': return cmpStr(String(a.lastEdited), String(b.lastEdited));
|
||||
default: return 0;
|
||||
}
|
||||
}));
|
||||
} else if (query.sortBy === "revisionNumber") {
|
||||
const currentWinners: PadQueryResult[] = []
|
||||
const padMapping = [] as {padId: string, revisionNumber: number}[]
|
||||
for (let res of result) {
|
||||
const pad = await padManager.getPad(res);
|
||||
const revisionNumber = pad.getHeadRevisionNumber()
|
||||
padMapping.push({padId: res, revisionNumber})
|
||||
}
|
||||
padMapping.sort((a, b) => {
|
||||
if (a.revisionNumber < b.revisionNumber) return query.ascending ? -1 : 1;
|
||||
if (a.revisionNumber > b.revisionNumber) return query.ascending ? 1 : -1;
|
||||
return 0;
|
||||
})
|
||||
|
||||
for (const padRetrieval of padMapping.slice(query.offset, query.offset + query.limit)) {
|
||||
let pad = await padManager.getPad(padRetrieval.padId);
|
||||
currentWinners.push({
|
||||
padName: padRetrieval.padId,
|
||||
lastEdited: await pad.getLastEdit(),
|
||||
userCount: api.padUsersCount(pad.padName).padUsersCount,
|
||||
revisionNumber: padRetrieval.revisionNumber
|
||||
})
|
||||
}
|
||||
|
||||
data.results = currentWinners;
|
||||
} else if (query.sortBy === "userCount") {
|
||||
const currentWinners: PadQueryResult[] = []
|
||||
const padMapping = [] as {padId: string, userCount: number}[]
|
||||
for (let res of result) {
|
||||
const userCount = api.padUsersCount(res).padUsersCount
|
||||
padMapping.push({padId: res, userCount})
|
||||
}
|
||||
padMapping.sort((a, b) => {
|
||||
if (a.userCount < b.userCount) return query.ascending ? -1 : 1;
|
||||
if (a.userCount > b.userCount) return query.ascending ? 1 : -1;
|
||||
return 0;
|
||||
})
|
||||
|
||||
for (const padRetrieval of padMapping.slice(query.offset, query.offset + query.limit)) {
|
||||
let pad = await padManager.getPad(padRetrieval.padId);
|
||||
currentWinners.push({
|
||||
padName: padRetrieval.padId,
|
||||
lastEdited: await pad.getLastEdit(),
|
||||
userCount: padRetrieval.userCount,
|
||||
revisionNumber: pad.getHeadRevisionNumber()
|
||||
})
|
||||
}
|
||||
data.results = currentWinners;
|
||||
} else if (query.sortBy === "lastEdited") {
|
||||
const currentWinners: PadQueryResult[] = []
|
||||
const padMapping = [] as {padId: string, lastEdited: string}[]
|
||||
for (let res of result) {
|
||||
const pad = await padManager.getPad(res);
|
||||
const lastEdited = await pad.getLastEdit();
|
||||
padMapping.push({padId: res, lastEdited})
|
||||
}
|
||||
padMapping.sort((a, b) => {
|
||||
if (a.lastEdited < b.lastEdited) return query.ascending ? -1 : 1;
|
||||
if (a.lastEdited > b.lastEdited) return query.ascending ? 1 : -1;
|
||||
return 0;
|
||||
})
|
||||
|
||||
for (const padRetrieval of padMapping.slice(query.offset, query.offset + query.limit)) {
|
||||
let pad = await padManager.getPad(padRetrieval.padId);
|
||||
currentWinners.push({
|
||||
padName: padRetrieval.padId,
|
||||
lastEdited: padRetrieval.lastEdited,
|
||||
userCount: api.padUsersCount(pad.padName).padUsersCount,
|
||||
revisionNumber: pad.getHeadRevisionNumber()
|
||||
})
|
||||
}
|
||||
data.results = currentWinners;
|
||||
});
|
||||
results = sorted.slice(query.offset, query.offset + query.limit);
|
||||
} else {
|
||||
const sliceNames = totalNames!.sort(cmpStr).slice(query.offset, query.offset + query.limit);
|
||||
results = await Promise.all(sliceNames.map(loadMeta));
|
||||
}
|
||||
|
||||
const data: {total: number, results?: PadQueryResult[]} = {total, results};
|
||||
socket.emit('results:padLoad', data);
|
||||
})
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +1,18 @@
|
|||
export type PadFilter = "all" | "active" | "recent" | "empty" | "stale";
|
||||
|
||||
export const PAD_FILTERS: PadFilter[] = ["all", "active", "recent", "empty", "stale"];
|
||||
|
||||
export type PadSearchQuery = {
|
||||
pattern: string;
|
||||
offset: number;
|
||||
limit: number;
|
||||
ascending: boolean;
|
||||
sortBy: "padName" | "lastEdited" | "userCount" | "revisionNumber";
|
||||
// Filter chip. Defaults to "all". Applied server-side so pagination
|
||||
// reflects the filtered universe — without this, the chip filters only
|
||||
// the current page slice and "0 empty pads" can appear on page 1 while
|
||||
// page 2 has nothing but empties.
|
||||
filter?: PadFilter;
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
205
src/tests/backend/specs/admin/padLoadFilter.ts
Normal file
205
src/tests/backend/specs/admin/padLoadFilter.ts
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
'use strict';
|
||||
|
||||
// Regression test for the admin /settings socket's `padLoad` filter chip.
|
||||
// Before commit fb…, `filter` (active|empty|recent|stale) lived only on
|
||||
// the client and ran AFTER pagination, so clicking "empty pads" on a
|
||||
// server with thousands of pads showed 0–12 results from page 1 even
|
||||
// though hundreds of empty pads existed deeper in the list. The filter
|
||||
// now rides PadSearchQuery and is applied server-side before the
|
||||
// offset/limit slice, so `total` reflects the filtered universe.
|
||||
|
||||
import {strict as assert} from 'assert';
|
||||
import setCookieParser from 'set-cookie-parser';
|
||||
|
||||
const io = require('socket.io-client');
|
||||
const common = require('../../common');
|
||||
const settings = require('../../../../node/utils/Settings');
|
||||
const padManager = require('../../../../node/db/PadManager');
|
||||
|
||||
const adminSocket = async () => {
|
||||
settings.users = settings.users || {};
|
||||
settings.users['test-admin'] = {password: 'test-admin-password', is_admin: true};
|
||||
const savedRequireAuthentication = settings.requireAuthentication;
|
||||
settings.requireAuthentication = true;
|
||||
let res: any;
|
||||
try {
|
||||
res = await (common.agent as any)
|
||||
.get('/admin/')
|
||||
.auth('test-admin', 'test-admin-password');
|
||||
} finally {
|
||||
settings.requireAuthentication = savedRequireAuthentication;
|
||||
}
|
||||
const resCookies = setCookieParser.parse(res, {map: true});
|
||||
const reqCookieHdr = Object.entries(resCookies)
|
||||
.map(([name, cookie]: [string, any]) =>
|
||||
`${name}=${encodeURIComponent(cookie.value)}`)
|
||||
.join('; ');
|
||||
const socket = io(`${common.baseUrl}/settings`, {
|
||||
forceNew: true,
|
||||
query: {cookie: reqCookieHdr},
|
||||
});
|
||||
await new Promise<void>((res, rej) => {
|
||||
const onErr = (err: any) => { socket.off('connect', onConn); rej(err); };
|
||||
const onConn = () => { socket.off('connect_error', onErr); res(); };
|
||||
socket.once('connect', onConn);
|
||||
socket.once('connect_error', onErr);
|
||||
});
|
||||
return socket;
|
||||
};
|
||||
|
||||
const ask = (socket: any, evt: string, payload: any, replyEvt: string) =>
|
||||
new Promise<any>((res) => {
|
||||
socket.once(replyEvt, res);
|
||||
socket.emit(evt, payload);
|
||||
});
|
||||
|
||||
const PROBE_BUDGET_MS = 15000;
|
||||
const adminSocketWithProbe = async (budgetMs: number): Promise<{
|
||||
ok: true; socket: any;
|
||||
} | {ok: false; reason: string;}> => {
|
||||
const deadline = Date.now() + budgetMs;
|
||||
let socket: any;
|
||||
try {
|
||||
socket = await Promise.race([
|
||||
adminSocket(),
|
||||
new Promise<never>((_, rej) =>
|
||||
setTimeout(() => rej(new Error('adminSocket connect timed out')),
|
||||
Math.max(0, deadline - Date.now()))),
|
||||
]);
|
||||
} catch (err: any) {
|
||||
return {ok: false, reason: String(err && err.message || err)};
|
||||
}
|
||||
const remaining = Math.max(0, deadline - Date.now());
|
||||
const replied = new Promise<true>((res) => socket.once('results:padLoad', () => res(true)));
|
||||
socket.emit('padLoad', {
|
||||
pattern: '__padLoadFilter-probe__', offset: 0, limit: 1,
|
||||
sortBy: 'padName', ascending: true,
|
||||
});
|
||||
const probed = await Promise.race([
|
||||
replied,
|
||||
new Promise<false>((res) => setTimeout(() => res(false), remaining)),
|
||||
]);
|
||||
if (!probed) {
|
||||
socket.disconnect();
|
||||
return {ok: false, reason: `no \`results:padLoad\` reply within ${budgetMs}ms`};
|
||||
}
|
||||
return {ok: true, socket};
|
||||
};
|
||||
|
||||
describe(__filename, function () {
|
||||
let socket: any;
|
||||
let savedUsers: any;
|
||||
let savedRequireAuthentication: boolean;
|
||||
let setupCompleted = false;
|
||||
// Distinct per-suite tag so concurrent test runs / leftover pads from
|
||||
// earlier suites don't pollute the filter assertions.
|
||||
const tag = `padLoadFilter-${Date.now()}-${Math.floor(Math.random() * 1e6)}`;
|
||||
const emptyPadIds: string[] = [];
|
||||
const editedPadIds: string[] = [];
|
||||
|
||||
before(async function () {
|
||||
this.timeout(120000);
|
||||
await common.init();
|
||||
|
||||
savedUsers = settings.users;
|
||||
savedRequireAuthentication = settings.requireAuthentication;
|
||||
setupCompleted = true;
|
||||
|
||||
const probe = await adminSocketWithProbe(PROBE_BUDGET_MS);
|
||||
if (!probe.ok) {
|
||||
console.warn(
|
||||
`[padLoadFilter] admin socket probe failed (${probe.reason}); ` +
|
||||
"skipping suite — likely an authenticate-hook plugin rejecting the test's " +
|
||||
'admin credentials.');
|
||||
this.skip();
|
||||
return;
|
||||
}
|
||||
socket = probe.socket;
|
||||
|
||||
// 5 empty pads, 3 edited (head rev > 0).
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const id = `${tag}-empty-${i}`;
|
||||
await padManager.getPad(id, '');
|
||||
emptyPadIds.push(id);
|
||||
}
|
||||
for (let i = 0; i < 3; i++) {
|
||||
const id = `${tag}-edited-${i}`;
|
||||
const pad = await padManager.getPad(id, '');
|
||||
// setText bumps head past 0; padLoad reports the post-edit
|
||||
// revisionNumber, which is what filter:"empty" excludes.
|
||||
await pad.setText(`seed-${i}\n`, `m-${tag}-${i}`);
|
||||
editedPadIds.push(id);
|
||||
}
|
||||
});
|
||||
|
||||
after(async function () {
|
||||
if (socket) socket.disconnect();
|
||||
if (!setupCompleted) return;
|
||||
// `savedUsers` may point at the same object that adminSocket mutated,
|
||||
// so reassigning the reference is a no-op; explicitly delete the
|
||||
// injected key so subsequent backend specs don't see a stale
|
||||
// test-admin user.
|
||||
if (settings.users) delete settings.users['test-admin'];
|
||||
settings.users = savedUsers;
|
||||
settings.requireAuthentication = savedRequireAuthentication;
|
||||
for (const id of [...emptyPadIds, ...editedPadIds]) {
|
||||
try {
|
||||
const pad = await padManager.getPad(id, '');
|
||||
await pad.remove();
|
||||
} catch { /* already gone */ }
|
||||
}
|
||||
});
|
||||
|
||||
it('filter:"empty" returns only revisionNumber===0 pads from the full set', async function () {
|
||||
const res = await ask(socket, 'padLoad', {
|
||||
pattern: tag, offset: 0, limit: 12, sortBy: 'padName',
|
||||
ascending: true, filter: 'empty',
|
||||
}, 'results:padLoad');
|
||||
assert.equal(res.total, 5, `expected total=5, got ${JSON.stringify(res)}`);
|
||||
for (const r of res.results) {
|
||||
assert.equal(r.revisionNumber, 0,
|
||||
`non-empty pad leaked through filter: ${JSON.stringify(r)}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('filter:"empty" with limit=2 still reports the correct total (regression: thm)', async function () {
|
||||
// The bug thm hit: clicking "empty" showed at most `limit` empties
|
||||
// because filtering happened on the client AFTER pagination. The
|
||||
// server now applies filter first, so total reflects the filtered
|
||||
// universe and pagination spans it correctly.
|
||||
const res = await ask(socket, 'padLoad', {
|
||||
pattern: tag, offset: 0, limit: 2, sortBy: 'padName',
|
||||
ascending: true, filter: 'empty',
|
||||
}, 'results:padLoad');
|
||||
assert.equal(res.total, 5, `expected total=5 (all empties), got total=${res.total}`);
|
||||
assert.equal(res.results.length, 2, `expected limit=2 page, got ${res.results.length} rows`);
|
||||
});
|
||||
|
||||
it('filter omitted (older client) falls back to "all"', async function () {
|
||||
const res = await ask(socket, 'padLoad', {
|
||||
pattern: tag, offset: 0, limit: 12, sortBy: 'padName',
|
||||
ascending: true,
|
||||
}, 'results:padLoad');
|
||||
assert.equal(res.total, 8,
|
||||
`expected total=8 (5 empty + 3 edited), got ${JSON.stringify(res)}`);
|
||||
});
|
||||
|
||||
it('filter:"all" matches the no-filter behaviour', async function () {
|
||||
const res = await ask(socket, 'padLoad', {
|
||||
pattern: tag, offset: 0, limit: 12, sortBy: 'padName',
|
||||
ascending: true, filter: 'all',
|
||||
}, 'results:padLoad');
|
||||
assert.equal(res.total, 8);
|
||||
});
|
||||
|
||||
it('filter:"active" excludes pads with no active users', async function () {
|
||||
// No connected clients in this test, so every test pad has
|
||||
// userCount === 0 → filter:"active" must return zero.
|
||||
const res = await ask(socket, 'padLoad', {
|
||||
pattern: tag, offset: 0, limit: 12, sortBy: 'padName',
|
||||
ascending: true, filter: 'active',
|
||||
}, 'results:padLoad');
|
||||
assert.equal(res.total, 0);
|
||||
assert.equal(res.results.length, 0);
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue