From 8640dee6352edef0b47e2e1a33c0579de0cc3a51 Mon Sep 17 00:00:00 2001 From: Michael Mayer Date: Thu, 9 Apr 2026 19:28:14 +0200 Subject: [PATCH] Docs: Split AGENTS guides into a hierarchical structure Signed-off-by: Michael Mayer --- AGENTS.md | 614 ++++++----------------------- frontend/AGENTS.md | 51 +++ internal/AGENTS.md | 43 ++ internal/api/AGENTS.md | 31 ++ internal/commands/AGENTS.md | 45 +++ internal/config/AGENTS.md | 22 ++ internal/photoprism/AGENTS.md | 25 ++ internal/service/cluster/AGENTS.md | 45 +++ pkg/AGENTS.md | 31 ++ 9 files changed, 412 insertions(+), 495 deletions(-) create mode 100644 frontend/AGENTS.md create mode 100644 internal/AGENTS.md create mode 100644 internal/api/AGENTS.md create mode 100644 internal/commands/AGENTS.md create mode 100644 internal/config/AGENTS.md create mode 100644 internal/photoprism/AGENTS.md create mode 100644 internal/service/cluster/AGENTS.md create mode 100644 pkg/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md index 25d306461..f2be6f1be 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,523 +1,147 @@ -# PhotoPrism® — Repository Guidelines +# PhotoPrism Repository Guidelines -**Last Updated:** March 8, 2026 +**Last Updated:** April 9, 2026 ## Purpose -This file tells automated coding agents (and humans) where to find the single sources of truth for building, testing, and contributing to this repository. Visit https://agents.md/ to learn more. +Entry point for agents and humans. ## Sources of Truth -- Makefile targets (always prefer existing targets): https://github.com/photoprism/photoprism/blob/develop/Makefile -- Developer Guide – Setup: https://docs.photoprism.app/developer-guide/setup/ -- Developer Guide – Tests: https://docs.photoprism.app/developer-guide/tests/ +- Makefile: https://github.com/photoprism/photoprism/blob/develop/Makefile +- Setup guide: https://docs.photoprism.app/developer-guide/setup/ +- Test guide: https://docs.photoprism.app/developer-guide/tests/ - Contributing: https://github.com/photoprism/photoprism/blob/develop/CONTRIBUTING.md - Security: https://github.com/photoprism/photoprism/blob/develop/SECURITY.md -- REST API: https://docs.photoprism.dev/ (Swagger), https://docs.photoprism.app/developer-guide/api/ (Docs) -- Code Maps: [`CODEMAP.md`](CODEMAP.md) (Backend/Go), [`frontend/CODEMAP.md`](frontend/CODEMAP.md) (Frontend/JS) -- Packages: `README.md` files under `internal/`, `pkg/`, and `frontend/src/`, e.g. [`internal/photoprism/README.md`](internal/photoprism/README.md), [`internal/photoprism/batch/README.md`](internal/photoprism/batch/README.md), [`internal/config/README.md`](internal/config/README.md), [`internal/server/README.md`](internal/server/README.md), [`internal/api/README.md`](internal/api/README.md), [`internal/service/webdav/README.md`](internal/service/webdav/README.md), [`internal/thumb/README.md`](internal/thumb/README.md), [`internal/ffmpeg/README.md`](internal/ffmpeg/README.md), and [`frontend/src/common/README.md`](frontend/src/common/README.md). -- Face Detection & Embeddings: [`internal/ai/face/README.md`](internal/ai/face/README.md) -- Vision Config & Engines: [`internal/ai/vision/README.md`](internal/ai/vision/README.md), [`internal/ai/vision/openai/README.md`](internal/ai/vision/openai/README.md), [`internal/ai/vision/ollama/README.md`](internal/ai/vision/ollama/README.md) -- Terminology Glossary: [`GLOSSARY.md`](GLOSSARY.md) (single source for term definitions across specs/docs) -- Regenerate `NOTICE` files with `make notice` when dependencies change (e.g., updates to `go.mod`, `go.sum`, `package-lock.json`, or other lockfiles). Do not edit `NOTICE` or `frontend/NOTICE` manually. +- REST API: https://docs.photoprism.dev/ and https://docs.photoprism.app/developer-guide/api/ +- Code maps: [`CODEMAP.md`](CODEMAP.md), [`frontend/CODEMAP.md`](frontend/CODEMAP.md) +- Package docs: `README.md` files under `internal/`, `pkg/`, and `frontend/src/` +- AI/Vision docs: [`internal/ai/face/README.md`](internal/ai/face/README.md), [`internal/ai/vision/README.md`](internal/ai/vision/README.md), [`internal/ai/vision/openai/README.md`](internal/ai/vision/openai/README.md), [`internal/ai/vision/ollama/README.md`](internal/ai/vision/ollama/README.md) +- Glossary: [`GLOSSARY.md`](GLOSSARY.md) +- When dependencies change, regenerate `NOTICE` files with `make notice`; do not edit `NOTICE` or `frontend/NOTICE` manually. -> Quick Tip: to inspect GitHub issue details without leaving the terminal, run `curl -s https://api.github.com/repos/photoprism/photoprism/issues/`; if `gh` is set up, you MAY also run `gh issue view -R photoprism/photoprism`. +## Subtree Guides -### Local Agent Progress +- [`internal/AGENTS.md`](internal/AGENTS.md): internal Go rules. +- [`internal/api/AGENTS.md`](internal/api/AGENTS.md): API rules. +- [`internal/config/AGENTS.md`](internal/config/AGENTS.md): config rules. +- [`internal/commands/AGENTS.md`](internal/commands/AGENTS.md): CLI rules. +- [`internal/photoprism/AGENTS.md`](internal/photoprism/AGENTS.md): import and index rules. +- [`internal/service/cluster/AGENTS.md`](internal/service/cluster/AGENTS.md): cluster rules. +- [`frontend/AGENTS.md`](frontend/AGENTS.md): frontend rules. +- [`pkg/AGENTS.md`](pkg/AGENTS.md): `pkg/*` security and test rules. -- Use the following task files to track progress and handoff notes across sessions/environments: - - `.agents/TODO.md` for actionable tasks. - - `.agents/DONE.md` for completed tasks. -- These files are local workflow aids and may not exist yet in a given workspace. +Optional nested repositories such as `plus/`, `pro/`, `portal/`, and `specs/` may contain their own `AGENTS.md` files. When present, treat those files as additional directory-local guidance. + +## Local Agent Progress + +- Use `.agents/TODO.md` for actionable tasks and `.agents/DONE.md` for completed work. +- These files are local workflow aids and may not exist in every workspace. ## Style Notes ### Commit Messages -Use concise, imperative subjects with a one-word prefix indicating the scope or topic: - -- `Config: Add tests for "darktable-cli" path detection` - -If the commit relates to specific issues or pull requests, reference their IDs in the message: - -- `Docker: Use two stage build to reduce image size #123 #5632` - -Commit messages must not exceed 80 characters in length. +- Use concise imperative subjects with a one-word prefix, for example `Config: Add tests for "darktable-cli" path detection`. +- Append issue or PR IDs when relevant. +- Commit messages must not exceed 80 characters. ### GitHub Issues -Issue titles MUST be concise, use the imperative mood, and start with a single capitalized prefix followed by a colon and a space, e.g. `Search: Add filter for RAW image formats`. - -Issue descriptions MUST begin with a one-sentence **User Story** where the sentence itself is fully bold in the format: `**As a , I want , so that .**` -Follow the User Story with a clear summary of the expected behavior, rationale, technical considerations, and constraints. - -Descriptions MUST conclude with a checklist of **Acceptance Criteria**: -- Use GitHub checklist formatting: `- [ ]` -- Criteria MUST be clear, testable, and unambiguous. -- Each item MUST use one of the following priority keywords: - - `MUST` — required for the issue to be considered complete - - `SHOULD` — strongly recommended but not strictly required - - `MAY` — optional enhancement - -Additional details MAY be included as needed, such as related issues, references, screenshots, or external resources. - -> Agents MUST create, edit, close, reopen, relabel, or otherwise modify GitHub issues only when explicitly requested by the user. +- Titles must be concise, imperative, and start with one capitalized prefix plus `: `, for example `Search: Add filter for RAW image formats`. +- Descriptions must begin with a one-sentence bold user story: `**As a , I want , so that .**` +- Follow with behavior, rationale, technical considerations, and constraints. +- End with `- [ ]` checklist items for the acceptance criteria, each using `MUST`, `SHOULD`, or `MAY`. +- Agents may create, edit, close, reopen, relabel, or otherwise modify GitHub issues only when explicitly requested by the user. ### Specifications & Documentation -- Document headings must use **Title Case** (in APA or AP style) across Markdown files to keep generated navigation and changelogs consistent. Always spell the product name as `PhotoPrism`; this proper noun is an exception to generic naming rules. -- When writing CLI examples or scripts, place option flags before positional arguments unless the command requires a different order. -- Use RFC 3339 UTC timestamps in request and response examples, and valid ID, UID and UUID examples in docs and tests. -- Technical specifications in the nested `specs/` subrepository may not be present in every clone or environment. Do not add `Makefile` targets in the main project that depend on `specs/` paths. When `specs/` is available, you MAY run its tools manually (e.g., `bash specs/scripts/lint-status.sh`), but the main repo must remain buildable without `specs/`. - - Testing Guides: `specs/dev/backend-testing.md` (Backend/Go), `specs/dev/frontend-testing.md` (Frontend/JS) - - Auto-generated configuration and command references live under `specs/generated/`. Agents MUST NOT read, analyze, or modify anything in this directory; refer humans to `specs/generated/README.md` if regeneration is required. - - Nested Git repositories may appear to be ignored; if so, change directories before staging or committing updates. +- Markdown headings must use Title Case. Always spell the product name as `PhotoPrism`. +- Put option flags before positional arguments unless the command requires another order. +- Use RFC 3339 UTC timestamps and valid ID, UID, and UUID examples in docs and tests. +- The nested `specs/` repository may be absent. Do not add main-repo `Makefile` targets that depend on it; when present, you may run its tools manually. +- Testing guides live at `specs/dev/backend-testing.md` and `specs/dev/frontend-testing.md`. +- Do not read, analyze, or modify `specs/generated/`; refer humans to `specs/generated/README.md` when regeneration is needed. +- Refresh `**Last Updated:**` when you change document contents, but leave it unchanged for whitespace-only or formatting-only edits. +- Nested Git repositories may appear ignored; change into them before staging or committing updates. -> **Title Case** rules (APA/AP implementation): -> - Capitalize the first word of a title/heading and the first word of a subtitle. -> - Capitalize the first word after a colon, an em dash, or end punctuation. -> - Capitalize major words, including the second part of hyphenated major words. -> - Capitalize all words of four letters or more. -> - Lowercase only minor words of three letters or fewer (articles, short conjunctions, short prepositions), except when they are in one of the positions above. -> - In headings, prefer `&` where needed; do not use `And` or `Or` in titles. - -> Refresh the `**Last Updated:**` date at the top of documents whenever you make changes to their contents, using the format `January 20, 2026` (without time); leave it as-is for simple formatting or whitespace-only edits. +Title Case rules: +- Capitalize the first word, the first word after a colon, dash, or end punctuation, and all major words, including the second part of a hyphenated major word. +- Lowercase only articles, short conjunctions, and short prepositions of three letters or fewer when they are not in one of those positions. +- Prefer `&` where needed; do not use `And` or `Or` in titles. ## Safety & Data -- If `git status` shows unexpected changes, assume a human might be editing; if you think you caused them, ask for permission before using reset commands like `git checkout` or `git reset`. -- Do not run `git config` (global or repo-level); changing Git configuration is prohibited for agents. -- Do not run destructive commands against production data. Prefer ephemeral volumes and test fixtures for acceptance tests. -- Never commit secrets, local configurations, or cache files. Use environment variables or a local `.env`. +- If `git status` shows unexpected changes, assume a human may be editing; ask before using reset-style commands. +- Do not run `git config` at either the global or repository level. +- Do not run destructive commands against production data; prefer ephemeral volumes and test fixtures for acceptance tests. +- Never commit secrets, local configurations, or cache files; use environment variables or a local `.env`. - Ensure `.env`, `.config`, `.local`, `.codex`, and `.gocache` are ignored in `.gitignore` and `.dockerignore`. -- Prefer using existing caches, workers, and batching strategies referenced in code and `Makefile`. -- Consider memory/CPU impact of changes; only suggest benchmarks or profiling when justified. - -> If anything in this file conflicts with the `Makefile` or Sources of Truth, **ask** for clarification before proceeding. - -## Project Structure & Languages - -- Backend: Go (`internal/`, `pkg/`, `cmd/`) + MariaDB/SQLite - - Package boundaries: Code in `pkg/*` MUST NOT import from `internal/*`. - - If you need access to config/entity/DB, put new code in a package under `internal/` instead of `pkg/`. -- GORM field naming: When adding struct fields that include uppercase abbreviations (e.g., `LabelNSFW`, `UserID`, `URLHash`), set an explicit `gorm:"column:"` tag so column names stay consistent (`label_nsfw`, `user_id`, `url_hash` instead of split-letter variants). -- Frontend: Vue 3 + Vuetify 3 (`frontend/`) -- Docker/compose for dev/CI; Traefik is used for local TLS (`*.localssl.dev`) - -> Nested Git repositories may appear to be ignored; if so, change directories before staging or committing updates. - -### Web Templates & Shared Assets - -- HTML entrypoints live under `assets/templates/`; key files are `index.gohtml`, `app.gohtml`, `app.js.gohtml`, and `splash.gohtml`. The browser check logic resides in `assets/static/js/browser-check.js` and is included via `app.js.gohtml`; it performs capability checks (Promise, fetch, AbortController, `script.noModule`, etc.) before the main bundle executes. -- OIDC login completion for the web UI is bridged through `assets/templates/auth.gohtml`, which writes the session into namespaced browser storage and must stay aligned with `frontend/src/common/session.js`, `frontend/src/common/storage.js`, and the login form toggle in `frontend/src/page/auth/login.vue` (`Stay signed in on this device`). Preserve the `session` storage-preference flag across the OIDC callback so ephemeral `sessionStorage` logins still restore after redirect. -- When touching frontend session bootstrap, verify that `frontend/src/common/session.js` resolves `storageNamespace` from the real client config shape (`window.__CONFIG__` / `config.values`), not only from simplified mocks. Include a focused test that would fail if session restore fell back to the `pp:root:` namespace. -- To preserve the fallback messaging, keep the script order in `app.js.gohtml` so `browser-check.js` loads before the bundle script (`{{ .config.JsUri }}`). Do not add `defer` or `async` to the bundle tag unless you reintroduce a guarded loader. -- The same loader partial is reused in private packages (`pro/assets/templates/index.gohtml`, `plus/assets/templates/index.gohtml`, `portal/assets/templates/index.gohtml`). Whenever you touch `app.js.gohtml` or change how we load the bundle, mirror the update by running commands such as `cd pro && sed -n '1,160p' assets/templates/index.gohtml` (and similarly for `plus` and `portal`) to confirm they include the shared partial instead of hard-coding the bundle tag. -- Splash styles are defined in `frontend/src/css/splash.css`. Add new splash elements (for example `.splash-warning`) there so both public and private editions remain visually consistent. -- Browser baseline: PhotoPrism requires Safari 13 / iOS 13 or current Chrome, Edge, or Firefox. Update the message in `assets/templates/app.js.gohtml` (and the matching CSS) if support changes. - -### Frontend Translations - -- Frontend translation extraction source of truth is root `make gettext-extract` (runs `scripts/gettext-extract.sh`), which scans `frontend/src` plus available private overlays in `plus/frontend`, `pro/frontend`, and `portal/frontend`. Subrepo compatibility targets (`make -C plus gettext-extract`, `make -C pro gettext-extract`, `make -C portal gettext-extract`) delegate to this root target. -- Avoid punctuation-only gettext keys (for example `$gettext("—")`), as they create noisy/unhelpful entries in `frontend/src/locales/translations.pot`. - -## Agent Runtime (Host vs Container) - -Agents MAY run either: - -- **Inside the Development Environment container** (recommended for least privilege). -- **On the host** (outside Docker), in which case the agent MAY start/stop the Dev Environment as needed. - -### Detecting the environment (agent logic) - -Agents SHOULD detect the runtime and choose commands accordingly: - -- **Inside container if** `/.dockerenv` exists (authoritative signal). -- Path hint: when the project path is `/go/src/github.com/photoprism/photoprism` *and* `/.dockerenv` is absent, assume you are on the host with a bind mount; treat it as host mode and prefer host-side Docker commands. - -#### Examples - -Bash: - -```bash -if [ -f "/.dockerenv" ]; then - echo "container" -else - echo "host" -fi -``` - -Node.js: - -```js -const fs = require("fs"); -const inContainer = fs.existsSync("/.dockerenv"); -console.log(inContainer ? "container" : "host"); -``` - -### Agent installation and invocation - -- **Inside container**: Prefer running agents via `npm exec` (no global install), for example: - - `npm exec --yes -- --help` - - Or use `npx ...` - - If the agent is distributed via npm and must be global, install inside the container only: - - `npm install -g ` - - Replace `` / `` with the names from the agent’s official docs. - -- **On host**: Use the vendor’s recommended install for your OS. Ensure your agent runs from the repository root so it can discover `AGENTS.md` and project files. - -## Build & Run (local) - -- Run `make help` to see common targets (or open the `Makefile`). - -- **Host mode** (agent runs on the host; agent MAY manage Docker lifecycle): - - Build local dev image (once): `make docker-build` - - Start services: `docker compose up` (add `-d` to start in the background) - - Follow live app logs: `docker compose logs -f --tail=100 photoprism` (Ctrl+C to stop) - - All services: `docker compose logs -f --tail=100` - - Last 10 minutes only: `docker compose logs -f --since=10m photoprism` - - Plain output (easier to copy): `docker compose logs -f --no-log-prefix --no-color photoprism` - - Execute a single command in the app container: `docker compose exec photoprism ` - - Example: `docker compose exec photoprism ./photoprism help` - - Why `./photoprism`? It runs the locally built binary in the project directory. - - Run as non-root to avoid root-owned files on bind mounts: - `docker compose exec -u "$(id -u):$(id -g)" photoprism ` - - Durable alternative: set the service user or `PHOTOPRISM_UID`/`PHOTOPRISM_GID` in `compose.yaml`; if you hit issues, run `make fix-permissions`. - - Open a terminal session in the app container: `make terminal` - - Stop everything when done: `docker compose --profile=all down --remove-orphans` (`make down` does the same) - -- **Container mode** (agent runs inside the app container): - - Install deps: `make dep` - - Build frontend/backend: `make build-js` and `make build-go` - - Watch frontend changes (auto-rebuild): `make watch-js` - - Or run directly: `cd frontend && npm run watch` - - Tips: refresh the browser to see changes; running the watcher outside the container can be faster on non-Linux hosts; stop with Ctrl+C - - Start the PhotoPrism server: `./photoprism start` - - Open http://localhost:2342/ (HTTP) - - Or https://app.localssl.dev/ (HTTPS via Traefik reverse proxy) - - Only if Traefik is running and the dev compose labels are active - - Labels for `*.localssl.dev` are defined in the dev compose files, e.g. https://github.com/photoprism/photoprism/blob/develop/compose.yaml - - Admin Login: Local compose files set `PHOTOPRISM_ADMIN_USER=admin` and `PHOTOPRISM_ADMIN_PASSWORD=photoprism`; if the credentials differ, inspect `compose.yaml` (or the active environment) for these variables before logging in. - - Do not use the Docker CLI inside the container; starting/stopping services requires host Docker access. If you need to manage compose while inside the dev container, switch to host mode (or ask a human) instead of running `docker compose` there. - -Note: Across our public documentation, official images, and in production, the command-line interface (CLI) name is `photoprism`. Other PhotoPrism binary names are only used in development builds for side-by-side comparisons of the Community Edition (CE) with PhotoPrism Plus (`photoprism-plus`), PhotoPrism Pro (`photoprism-pro`), and PhotoPrism Portal (`photoprism-portal`). - -### Operating Systems & Architectures - -- Our guides and command examples generally assume the use of a Linux/Unix shell on a 64-bit AMD64 or ARM64 system. -- For Windows-specifics, see the Developer Guide FAQ: https://docs.photoprism.app/developer-guide/faq/#can-your-development-environment-be-used-under-windows - -## Code Style & Lint - -- Go: run `make fmt-go swag-fmt` to reformat the backend code + Swagger annotations (see `Makefile` for additional targets) - - Run `make lint-go` (golangci-lint) after Go changes; prefer `golangci-lint run ./internal//...` for focused edits. - - Doc comments for packages and exported identifiers must be complete sentences that begin with the name of the thing being described and end with a period. - - All newly added functions, including unexported helpers, must have a concise doc comment that explains their behavior. - - For short examples inside comments, indent code rather than using backticks; godoc treats indented blocks as preformatted. -- Branding: Always spell the product name as `PhotoPrism`; this proper noun is an exception to generic naming rules. -- Every Go package must contain a `.go` file in its root (for example, `internal/auth/jwt/jwt.go`) with the standard license header and a short package description comment explaining its purpose. -- JS/Vue: use the lint/format scripts in `frontend/package.json` (ESLint + Prettier) -- All added code and tests **must** be formatted according to our standards. - -## Tests - -- From within the Development Environment: - - Full unit test suite: `make test` (runs backend and frontend tests) - - Test frontend/backend: `make test-js` and `make test-go` - - Linting: `make lint` (all), `make lint-go` (golangci-lint with `.golangci.yml`, prints findings without failing due to `--issues-exit-code 0`), `make lint-js` (ESLint/Prettier) - - Go packages: `go test` (all tests) or `go test -run ` (specific tests only) -- Do not run multiple test commands in parallel. Suites share fixture files, temporary assets, and database state, so concurrent runs can trigger false failures, readonly database errors, or fixture conflicts. -- Need to inspect the MariaDB data while iterating? Connect directly inside the dev shell with `mariadb -D photoprism` and run SQL without rebuilding Go code. -- Go tests live beside sources: for `path/to/pkg/.go`, add tests in `path/to/pkg/_test.go` (create if missing). For the same function, group related cases as `t.Run(...)` sub-tests (table-driven where helpful) and use **PascalCase** for subtest names (for example, `t.Run("Success", ...)`). -- Frontend unit tests use **Vitest**; see scripts in `frontend/package.json`. - - Vitest watch/coverage: `make vitest-watch` and `make vitest-coverage` -- Acceptance tests: use the `acceptance-*` targets in the `Makefile` - - For one-off checks, run a single TestCafe case by `testID` and keep startup/cleanup in the repo root: - ```bash - make storage/acceptance - make acceptance-sqlite-restart - make wait-2 - (cd frontend && npm run testcafe -- "chrome --headless=new --use-gl=angle --use-angle=swiftshader --disable-features=LocalNetworkAccessChecks" --config-file ./testcaferc.json --test-meta mode=public,type=short,testID=components-001 "tests/acceptance") - make acceptance-sqlite-stop - ``` - - If your command temporarily changes into `frontend/`, run `make acceptance-sqlite-stop` after returning to the repository root; running that target from `frontend/` fails with "No rule to make target". -- Portal proxy URI validation: use the Portal test environment with `NODES=2` and verify both instance routes when changing `PHOTOPRISM_PORTAL_PROXY_URI` (Portal) and matching node `PHOTOPRISM_SITE_URL` prefixes; use `PORTAL_TEST_ENV_ARGS=--proxy-uri=/instance/` to regenerate consistent `.env` values. -- Portal test environment default: run a full rebuild via `make -C portal test-env NODES=2` before `make -C portal test-start`; avoid `--no-build` partial refreshes unless you intentionally validate env-only changes, as mixed/stale staged assets can load the wrong frontend edition. - -### Playwright MCP Usage - -- **Endpoint & Navigation** — Playwright MCP is preconfigured to reach the dev server at `http://localhost:2342/`. - Use `playwright__browser_navigate` to open the login route under the configured frontend URI (default `/library/login` for CE/Plus/Pro, `/portal/admin/login` for Portal), sign in, and then call `playwright__browser_take_screenshot` to capture the page state. -- **Viewport Defaults** — Desktop sessions open with a `1280×900` viewport by default. - Use `playwright__browser_resize` if the viewport is not preconfigured or you need to adjust it mid-run. -- **Mobile Workflows** — When testing responsive layouts, use the `playwright_mobile` server (for example, `playwright_mobile__browser_navigate`). - It launches with a `375×667` viewport, matching a typical smartphone display, so you can capture mobile layouts without manual resizing. -- **Authentication** — Default admin credentials are `admin` / `photoprism`: - - If login fails, check your active Compose file or container environment for `PHOTOPRISM_ADMIN_USER` and `PHOTOPRISM_ADMIN_PASSWORD`. - - Tip: if your MCP supports it, persist a storage state after login and reuse it in later steps to skip re-authentication. -- **Sidebar Navigation** — The sidebar nests items such as `Library → Errors`: - - Expand a parent entry by clicking its chevron before selecting links inside. -- **Session Cleanup** — After scripted interactions, close the browser tab with `playwright__browser_close` (or `playwright_mobile__browser_close`) to keep the MCP session tidy for subsequent runs. -- **Stability / Waiting** — Prefer robust waits over sleeps: - - After navigation: `waitUntil: 'networkidle'` (or wait for a key locator). - - Before clicking: ensure the locator is `visible` and `enabled`. - - Use role/label/text selectors over brittle XPaths. -- **Screenshot Format & Size** — Keep artifacts small and reproducible: - - Prefer **JPEG** with quality (e.g., `quality: 80`) instead of PNG. - - Limit to the visible viewport (`fullPage: false`), unless explicitly required. - - Name files deterministically, e.g., `.local/screenshots//__.jpg` (create the folder if it doesn’t exist). - - Avoid embedding large screenshots in chat history—reference the file path instead. - - **Desktop example** (if your MCP tool exposes Playwright options 1:1): - ```json - { - "path": ".local/screenshots/fix-event-leaks/login__desktop.jpg", - "type": "jpeg", - "quality": 80, - "fullPage": false - } - ``` -- **Non-interactive runs** — If `npx` is fetching the MCP server at runtime, add `--yes` to its args (or preinstall and use `--no-install`) to avoid prompts in CI. - -### FFmpeg Tests & Hardware Gating - -- By default, do not run GPU/HW encoder integrations in CI. Gate with `PHOTOPRISM_FFMPEG_ENCODER` (one of: `vaapi`, `intel`, `nvidia`). -- Negative-path tests should remain fast and always run: - - Missing ffmpeg binary → immediate exec error. - - Unwritable destination → command fails without creating files. -- Prefer command-string assertions when hardware is unavailable; enable HW runs locally only when a device is configured. - -### Fast, Focused Test Recipes - -- Filesystem + archives (fast): `go test ./pkg/fs -run 'Copy|Move|Unzip' -count=1` -- Media helpers (fast): `go test ./pkg/media/... -count=1` -- Thumbnails (libvips, moderate): `go test ./internal/thumb/... -count=1` -- FFmpeg command builders (moderate): `go test ./internal/ffmpeg -run 'Remux|Transcode|Extract' -count=1` - -### CLI Testing Gotchas (Go) - -- Exit codes and `os.Exit`: - - `urfave/cli` calls `os.Exit(code)` when a command returns `cli.Exit(...)`, which will terminate `go test` abruptly (often after logs like `http 401:`). - - Use the test helper `RunWithTestContext` (in `internal/commands/commands_test.go`) which temporarily overrides `cli.OsExiter` so the process doesn’t exit; you still receive the error to assert `ExitCoder`. - - If you only need to assert the exit code and don’t need printed output, you can invoke `cmd.Action(ctx)` directly and check `err.(cli.ExitCoder).ExitCode()`. -- Non‑interactive mode: set `PHOTOPRISM_CLI=noninteractive` and/or pass `--yes` to avoid prompts that block tests and CI. -- SQLite DSN in tests: - - `config.NewTestConfig("")` defaults to SQLite with a per‑suite DSN like `..db`. Don’t assert an empty DSN for SQLite. - - Clean up any per‑suite SQLite files in tests with `t.Cleanup(func(){ _ = os.Remove(dsn) })` if you capture the DSN. - -### Frontend Focus Management - -- Dialogs must follow the shared focus pattern documented in `frontend/src/common/README.md`. -- Always expose `ref="dialog"` on `` overlays, call `$view.enter/leave` in `@after-enter` / `@after-leave`, and avoid positive `tabindex` values. -- Persistent dialogs (those with the `persistent` prop) must handle Escape via `@keydown.esc.exact` so Vuetify’s default rejection animation is suppressed; keep other shortcuts on `@keyup` so inner inputs can cancel them first. -- Global shortcuts run through `onShortCut(ev)` in `common/view.js`; it only forwards Escape and `ctrl`/`meta` combinations, so do not rely on it for arbitrary keys. -- When a dialog opens nested menus (for example, combobox suggestion lists), ensure they work with the global trap; see the README for troubleshooting tips. - -### Filesystem Permissions & io/fs Aliasing (Go) - -- Always use our shared permission variables from `pkg/fs` when creating files/directories: - - Directories: `fs.ModeDir` (0o755 with umask) - - Regular files: `fs.ModeFile` (0o644 with umask) - - Config files: `fs.ModeConfigFile` (default 0o664) - - Secrets/tokens: `fs.ModeSecretFile` (default 0o600) - - Backups: `fs.ModeBackupFile` (default 0o600) -- Do not pass stdlib `io/fs` flags (e.g., `fs.ModeDir`) to functions expecting permission bits. - - When importing the stdlib package, alias it to avoid collisions: `iofs "io/fs"` or `gofs "io/fs"`. - - Our package is `github.com/photoprism/photoprism/pkg/fs` and provides the only approved permission constants for `os.MkdirAll`, `os.WriteFile`, `os.OpenFile`, and `os.Chmod`. -- Prefer `filepath.Join` for filesystem paths; reserve `path.Join` for URL paths. -- For slash-based logical paths stored in DB/config/API payloads (for example folder album paths), normalize with `clean.SlashPath(...)` instead of repeating ad-hoc `strings.ReplaceAll(..., "\\", "/")` + trim logic. - -### File I/O — Overwrite Policy (force semantics) - -- Default is safety-first: callers must not overwrite non-empty destination files unless they opt-in with a `force` flag. -- Replacing empty destination files is allowed without `force=true` (useful for placeholder files). -- Open destinations with `O_WRONLY|O_CREATE|O_TRUNC` to avoid trailing bytes when overwriting; use `O_EXCL` when the caller must detect collisions. -- Where this lives: - - App-level helpers: `internal/photoprism/mediafile.go` (`MediaFile.Copy/Move`). - - Reusable utils: `pkg/fs/copy.go`, `pkg/fs/move.go`. -- When to set `force=true`: - - Explicit “replace” actions or admin tools where the user confirmed overwrite. - - Not for import/index flows; Originals must not be clobbered. - -### Archive Extraction — Security Checklist - -- Always validate ZIP entry names with a safe join; reject: - - absolute paths (e.g., `/etc/passwd`). - - Windows drive/volume paths (e.g., `C:\\…` or `C:/…`). - - any entry that escapes the target directory after cleaning (path traversal via `..`). -- ZIP entry names use slash semantics, not host OS semantics: - - Validate in ZIP-name space with `path.Clean` / `path.IsAbs`, reject backslashes (`\`), and use `path.Base` for hidden-name checks. - - Convert to OS paths only at write time with `filepath.FromSlash(...)`. - - Enforce destination containment with `filepath.Rel(...)` rather than string-prefix checks. -- Enforce per-file and total size budgets to prevent resource exhaustion. -- Skip OS metadata directories (e.g., `__MACOSX`) and reject suspicious names. -- Where this lives: `pkg/fs/zip.go` (`Unzip`, `UnzipFile`, `safeJoin`). -- Tests to keep: - - Absolute/volume paths rejected (Windows-specific backslash path covered on Windows). - - `..` traversal skipped; `__MACOSX` skipped. - - Per-file and total size limits enforced; directory entries created; nested paths extracted safely. - -### HTTP Download — Security Checklist - -- Use the shared safe HTTP helper instead of ad‑hoc `net/http` code: - - Package: `pkg/http/safe` → `safe.Download(destPath, url, *safe.Options)`. - - Default policy in this repo: allow only `http/https`, enforce timeouts and max size, write to a `0600` temp file then rename. -- SSRF protection (mandatory unless explicitly needed for tests): - - Set `AllowPrivate=false` to block private/loopback/multicast/link‑local ranges. - - All redirect targets are validated; the final connected peer IP is also checked. - - Prefer an image‑focused `Accept` header for image downloads: `"image/jpeg, image/png, */*;q=0.1"`. -- Avatars and small images: use the thin wrapper in `internal/thumb/avatar.SafeDownload` which applies stricter defaults (15s timeout, 10 MiB, `AllowPrivate=false`). -- Tests using `httptest.Server` on 127.0.0.1 must pass `AllowPrivate=true` explicitly to succeed. -- Keep per‑resource size budgets small; rely on `io.LimitReader` + `Content-Length` prechecks. - -## Agent Quick Tips (Do This) - -### Testing & Fixtures - -- Go tests live next to their sources (`path/to/pkg/_test.go`); group related cases as `t.Run(...)` sub-tests to keep table-driven coverage readable, and name each subtest with a PascalCase string. -- Keep Go scratch work inside `internal/...`; Go refuses to import `internal/` packages from directories like `/tmp`, so create temporary helpers under a throwaway folder such as `internal/tmp/` instead of using external paths. -- Prefer focused `go test` runs for speed (`go test ./internal/ -run -count=1`, `go test ./internal/commands -run -count=1`) and avoid `./...` unless you need the entire suite. -- Heavy packages such as `internal/entity` and `internal/photoprism` run migrations and fixtures; expect 30–120s on first run and narrow with `-run` to keep iterations low. -- For CLI-driven tests, wrap commands with `RunWithTestContext(cmd, args)` so `urfave/cli` cannot exit the process, and assert CLI output with `assert.Contains`/regex because `show` reports quote strings. -- In `internal/photoprism` tests, rely on `photoprism.Config()` for runtime-accurate behavior; only build a new config if you replace it via `photoprism.SetConfig`. -- Generate identifiers with `rnd.GenerateUID(entity.ClientUID)` for OAuth client IDs and `rnd.UUIDv7()` for node UUIDs; treat `node.uuid` as required in responses. -- When creating or editing shell scripts, run `shellcheck ` (or the relevant `make` target) and resolve warnings before exiting the task. -- When adding persistent fixtures (photos, files, labels, etc.), always obtain new IDs via `rnd.GenerateUID(...)` with the matching prefix (`entity.PhotoUID`, `entity.FileUID`, `entity.LabelUID`, …) instead of inventing manual strings so the search helpers recognize them. -- For database updates, prefer the `entity.Values` type alias over raw `map[string]interface{}` so helpers stay type-safe and consistent with existing code. -- Reach for `config.NewMinimalTestConfig(t.TempDir())` when a test only needs filesystem/config scaffolding, and use `config.NewMinimalTestConfigWithDb("", t.TempDir())` when you need a fresh SQLite schema without the cached fixture snapshot. -- Config test helpers now auto-discover the repo `assets/` directory; you should not set `PHOTOPRISM_ASSETS_PATH` manually in package `init()` functions unless you have a non-standard layout. -- Hub API traffic is disabled in tests by default via `hub.ApplyTestConfig()`; opt back in with `PHOTOPRISM_TEST_HUB=test`. -- Avoid `config.TestConfig()` in new tests unless you truly need the fully seeded fixture set: it shares a singleton instance that runs `InitializeTestData()` and wipes `storage/testdata`. Tests that write to Originals/Import (e.g. WebDAV helpers) should instead call `config.NewMinimalTestConfig(t.TempDir())` (or the DB variant) and follow up with `conf.CreateDirectories()` so they operate on an isolated sandbox. -- Shared fixtures live under `storage/testdata`; `NewTestConfig("")` already calls `InitializeTestData()`, but call `c.InitializeTestData()` (and optionally `c.AssertTestData(t)`) when you construct custom configs so originals/import/cache/temp exist. `InitializeTestData()` clears old data, downloads fixtures if needed, then calls `CreateDirectories()`. -- `PhotoFixtures.Get()` and similar helpers return value copies; when a test needs the database-backed row (with associations preloaded), re-query by UID/ID using helpers like `entity.FindPhoto(fixture)` so updates observe persisted IDs and in-memory caches stay coherent. -- For slimmer tests that only need config objects, prefer the new helpers in `internal/config/test.go`: `NewMinimalTestConfig(t.TempDir())` when no database is needed, or `NewMinimalTestConfigWithDb("", t.TempDir())` to spin up an isolated SQLite schema without seeding all fixtures. -- When you need illustrative credentials (join tokens, client IDs/secrets, etc.), reuse the shared `Example*` constants (see `internal/service/cluster/examples.go`) so tests, docs, and examples stay consistent. -- Hidden error UI checks for the hidden route under the frontend URI (default `/library/hidden` for CE/Plus/Pro, `/portal/admin/hidden` for Portal) require both `files.file_error` and `photos.photo_quality = -1`; hidden searches are quality-gated, so setting only `file_error` will not surface the row in Hidden results. - -### Roles & ACL - -- Map roles via the shared tables: users through `acl.ParseRole(s)` / `acl.UserRoles[...]`, clients through `acl.ClientRoles[...]`. -- Treat `RoleAliasNone` ("none") and an empty string as `RoleNone`; no caller-specific overrides. -- Default unknown client roles to `RoleClient`; `acl.ParseRole` already handles `0/false/nil` as none for users. -- Build CLI role help from `Roles.CliUsageString()` (e.g., `acl.ClientRoles.CliUsageString()`); never hand-maintain role lists. -- When checking JWT/client scopes, use the shared helpers (`acl.ScopePermits` / `acl.ScopeAttrPermits`) instead of hand-written parsing. - -### Import/Index - -- ImportWorker may skip files if an identical file already exists (duplicate detection). Use unique copies or assert DB rows after ensuring a non‑duplicate destination. -- Mixed roots: when testing related files, keep `SamplesPath()/ImportPath()/OriginalsPath()` consistent so `RelatedFiles` and `AllowExt` behave as expected. -- `IndexOptions*` helpers now require a `*config.Config`; pass the active config (or `config.NewMinimalTestConfig(t.TempDir())` in unit tests) so face/label/NSFW scheduling matches the current run. -- Folder albums use path-first lookup/update (`album_path`) to avoid slug collisions for emoji child paths; re-indexing can repair stale collision titles when a child folder incorrectly shows the parent name, while preserving user-custom titles. -- Label and label-search logic should reuse `entity.FindLabels(...)`, `entity.FindLabelIDs(...)`, and `entity.LabelSlugs(...)` so homophone-aware exact-name matching stays aligned across `internal/entity` and `internal/entity/search`; avoid adding ad-hoc slug SQL in search code. - -### CLI Usage & Assertions - -- Prefer the shared helpers like `DryRunFlag(...)` and `YesFlag()` when adding new CLI flags so behaviour stays consistent across commands. -- Wrap CLI tests in `RunWithTestContext(cmd, args)` so `urfave/cli` cannot exit the process; assert quoted `show` output with `assert.Contains`/regex for the trailing ", or " rule. -- Prefer `--json` responses for automation. `photoprism show commands --json [--nested]` exposes the tree view (add `--all` for hidden entries). -- Use `internal/commands/catalog` to inspect commands/flags without running the binary; when validating large JSON docs, marshal DTOs via `catalog.BuildFlat/BuildNode` instead of parsing CLI stdout. -- Expect `show` commands to return arrays of snake_case rows, except `photoprism show config`, which yields `{ sections: [...] }`, and the `config-options`/`config-yaml` variants, which flatten to a top-level array. - -### API & Config Changes - -- Respect precedence: `options.yml` overrides CLI/env values, which override defaults. When adding a new option, update `internal/config/options.go` (yaml/flag tags), register it in `internal/config/flags.go`, expose a getter, surface it in `*config.Report()`, and write generated values back to `options.yml` by setting `c.options.OptionsYaml` before persisting. Use `CliTestContext` in `internal/config/test.go` to exercise new flags. -- For `options.yml` writes in Go code, prefer config-owned persistence helpers over ad-hoc YAML handling: use `Config.SaveOptionsPatch(...)` for generic merges and `Config.SaveClusterOptionsUpdate(...)` for cluster-managed metadata updates. -- Use `pkg/fs.ConfigFilePath` when you need a config filename so existing `.yml` files remain valid and new installs can adopt `.yaml` transparently (the helper also covers other paired extensions such as `.toml`/`.tml`). -- When touching configuration in Go code, use the public accessors on `*config.Config` (e.g. `Config.JWKSUrl()`, `Config.SetJWKSUrl()`, `Config.ClusterUUID()`) instead of mutating `Config.Options()` directly; reserve raw option tweaks for test fixtures only. -- When introducing new metadata sources (e.g., `SrcOllama`, `SrcOpenAI`), define them in both `internal/entity/src.go` and the frontend lookup tables (`frontend/src/common/util.js`) so UI badges and server priorities stay aligned. -- Vision worker scheduling is controlled via `VisionSchedule` / `VisionFilter` and the `Run` property set in `vision.yml`. Utilities like `vision.FilterModels` and `entity.Photo.ShouldGenerateLabels/Caption` help decide when work is required before loading media files. -- Logging: use the shared logger (`event.Log`) via the package-level `log` variable (see `internal/auth/jwt/logger.go`) instead of direct `fmt.Print*` or ad-hoc loggers. -- Logging terminology: in human-readable log text, prefer canonical runtime terms (`instance`, `service`) and reserve `node` for contract-bound names (`/cluster/nodes`, `Node*`, `PHOTOPRISM_NODE_*`). -- Audit outcomes: import `github.com/photoprism/photoprism/pkg/log/status` and end every `event.Audit*` slice with a single outcome token such as `status.Succeeded`, `status.Failed`, `status.Denied`, or other constants defined there (no additional segments afterwards). -- Error outcomes: when a sanitized error string should be the outcome, call `status.Error(err)` instead of adding a placeholder and passing `clean.Error(err)` manually. -- Cluster registry tests (`internal/service/cluster/registry`) currently rely on a full test config because they persist `entity.Client` rows. They run migrations and seed the SQLite DB, so they are intentionally slow. If you refactor them, consider sharing a single `config.TestConfig()` across subtests or building a lightweight schema harness; do not swap to the minimal config helper unless the tests stop touching the database. -- Favor explicit CLI flags: check `c.cliCtx.IsSet("")` before overriding user-supplied values, and follow the `ClusterUUID` pattern (`options.yml` → CLI/env → generated UUIDv4 persisted). -- Database helpers: reuse `conf.Db()` / `conf.Database*()`, avoid GORM `WithContext`, quote MySQL identifiers, and reject unsupported drivers early. -- Handler conventions: reuse limiter stacks (`limiter.Auth`, `limiter.Login`) and `limiter.AbortJSON` for 429s, lean on `api.ClientIP`, `header.BearerToken`, and `Abort*` helpers, compare secrets with constant time checks, set `Cache-Control: no-store` on sensitive responses, and register routes in `internal/server/routes.go`. For new list endpoints default `count=100` (max 1000) and `offset≥0`, document parameters explicitly, and set portal mode via `PHOTOPRISM_NODE_ROLE=portal` plus `PHOTOPRISM_JOIN_TOKEN` when needed. -- Swagger & docs: annotate only routed handlers in `internal/api/*.go`, use full `/api/v1/...` paths, skip helpers, and regenerate docs with `make fmt-go swag-fmt swag` or `make swag-json` (which also strips duplicate `time.Duration` enums). When iterating, target packages with `go test ./internal/api -run Cluster -count=1` or similarly scoped runs. -- Testing helpers: isolate config paths with `t.TempDir()`, reuse `NewConfig`, `CliTestContext`, and `NewApiTest()` harnesses, authenticate via `AuthenticateAdmin`, `AuthenticateUser`, or `OAuthToken`, toggle auth with `conf.SetAuthMode(config.AuthModePasswd)`, and prefer OAuth client tokens over non-admin fixtures for negative permission checks. -- Registry data and secrets: store portal/node registry files under `conf.PortalConfigPath()/nodes/` with mode `0600`, keep secrets out of logs, and only return them on creation/rotation flows. - -### Formatting (Go) - -- Go is formatted by `gofmt` and uses tabs. Do not hand-format indentation. -- Always run after edits: `make fmt-go` (gofmt + goimports). - -### API Shape Checklist - -- When renaming or adding fields: - - Update DTOs in `internal/service/cluster/response.go` and any mappers. - - Update handlers and regenerate Swagger: `make fmt-go swag-fmt swag`. - - Update tests (search/replace old field names) and examples in `specs/`. - - Quick grep: `rg -n 'oldField|newField' -S` across code, tests, and specs. - -### API/CLI Tests: Known Pitfalls - -- Gin routes: Register `CreateSession(router)` once per test router; reusing it twice panics on duplicate route. -- CLI commands: Some commands defer `conf.Shutdown()` or emit signals that close the DB. The harness re‑opens DB before each run, but avoid invoking `start` or emitting signals in unit tests. -- Signals: `internal/commands/start.go` waits on `process.Signal`; calling `process.Shutdown()/Restart()` can close DB. Prefer not to trigger signals in tests. - -### Download CLI Workbench (yt-dlp, remux, importer) - -- Code anchors - - CLI flags and examples: `internal/commands/download.go` - - Core implementation (testable): `internal/commands/download_impl.go` - - yt-dlp helpers and arg wiring: `internal/photoprism/dl/*` (`options.go`, `info.go`, `file.go`, `meta.go`) - - Importer entry point: `internal/photoprism/get/import.go`; options: `internal/photoprism/import_options.go` - -- Quick test runs (fast feedback) - - yt-dlp package: `go test ./internal/photoprism/dl -run 'Options|Created|PostprocessorArgs' -count=1` - - CLI command: `go test ./internal/commands -run 'DownloadImpl|HelpFlags' -count=1` - -- FFmpeg-less tests - - In tests: set `c.Options().FFmpegBin = "/bin/false"` and `c.Settings().Index.Convert = false` to avoid ffmpeg dependencies when not validating remux. - -- Stubbing yt-dlp (no network) - - Use a tiny shell script that: - - prints minimal JSON for `--dump-single-json` - - creates a file and prints its path when `--print` is requested - - Harness env vars (supported by our tests): - - `YTDLP_ARGS_LOG` — append final args for assertion - - `YTDLP_OUTPUT_FILE` — absolute file path to create for `--print` - - `YTDLP_DUMMY_CONTENT` — file contents to avoid importer duplicate detection between tests - -- Remux policy and metadata - - Pipe method: PhotoPrism remux (ffmpeg) always embeds title/description/created. - - File method: yt‑dlp writes files; we pass `--postprocessor-args 'ffmpeg:-metadata creation_time='` so imports get `Created` even without local remux (fallback from `upload_date`/`release_date`). - - Default remux policy: `auto`; use `always` for the most complete metadata (chapters, extended tags). - - CLI defaults: `photoprism dl` now defaults to `--method pipe` and `--impersonate firefox`; pass `-i none` to disable impersonation. Pipe mode streams raw media and PhotoPrism handles the final FFmpeg remux so metadata (title, description, author, creation time) still comes from `RemuxOptionsFromInfo`. - -- Testing workflow: lean on the focused commands above; if importer dedupe kicks in, vary bytes with `YTDLP_DUMMY_CONTENT` or adjust `dest`, and remember `internal/photoprism` is heavy so validate downstream packages first. - -### Sessions & Redaction (building sessions in tests) - -- Admin session (full view): `AuthenticateAdmin(app, router)`. -- User session: Create a non‑admin test user (role=guest), set a password, then `AuthenticateUser`. -- Client session (redacted internal fields; `SiteUrl` visible): - ```go - s, _ := entity.AddClientSession("test-client", conf.SessionMaxAge(), "cluster", authn.GrantClientCredentials, nil) - token := s.AuthToken() - r := AuthenticatedRequest(app, http.MethodGet, "/api/v1/cluster/nodes", token) - ``` - Admins see `AdvertiseUrl` and `Database`; client/user sessions don’t. `SiteUrl` is safe to show to all roles. Client config also includes `storageNamespace` (SHA-256 of `SiteUrl`) for browser storage scoping and is safe to expose. - -### Preflight Checklist - -- `go build ./...` -- `make fmt-go swag-fmt swag` -- `go test ./internal/service/cluster/registry -count=1` -- `go test ./internal/api -run 'Cluster' -count=1` -- `go test ./internal/commands -run 'ClusterRegister|ClusterNodesRotate' -count=1` -- Tooling constraints: `make swag` may fetch modules, so confirm network access before running it. - -### Cluster Operations - -- Keep bootstrap code decoupled: avoid importing `internal/service/cluster/node/*` from `internal/config` or the cluster root, let nodes talk to the Portal over HTTP(S), and rely on constants from `internal/service/cluster/const.go`. -- Bootstrap refreshes node OAuth credentials on 401/403 responses (rotate secret + retry) and logs the refresh at info level; if the secret file cannot be written, the value stays cached in memory so the current process can continue. -- Portal validation now accepts HTTP advertise URLs only for loopback hosts or cluster-internal domains (`*.svc`, `*.cluster.local`, `*.internal`); everything else must use HTTPS. -- Config init order: load `options.yml` (`c.initSettings()`), run `EarlyExt().InitEarly(c)`, connect/register the DB, then invoke `Ext().Init(c)`. -- Theme endpoint: `GET /api/v1/cluster/theme` streams a zip from `conf.ThemePath()`; only reinstall when `app.js` is missing and always use the header helpers in `pkg/http/header`. -- Registration flow: send `rotate=true` only for MySQL/MariaDB nodes without credentials, treat 401/403/404 as terminal, include `ClientID` + `ClientSecret` when renaming an existing node, and persist only newly generated secrets or DB settings. -- Registry & DTOs: use the client-backed registry (`NewClientRegistryWithConfig`)—the file-backed version is legacy—and treat migration as complete only after swapping callsites, building, and running focused API/CLI tests. Nodes are keyed by UUID v7 (`/api/v1/cluster/nodes/{uuid}`), the registry interface stays UUID-first (`Get`, `FindByNodeUUID`, `FindByClientID`, `RotateSecret`, `DeleteAllByUUID`), CLI lookups resolve `uuid → ClientID → name`, and DTOs normalize `Database.{Name,User,Driver,RotatedAt}` while exposing `ClientSecret` only during creation/rotation. `nodes rm --all-ids` cleans duplicate client rows, admin responses may include `AdvertiseUrl`/`Database`, client/user sessions stay redacted, registry files live under `conf.PortalConfigPath()/nodes/` (mode 0600), and `ClientData` no longer stores `NodeUUID`. -- Provisioner & DSN: database/user names use UUID-based HMACs (`d`, `u` where the prefix defaults to `cluster_` but may be overridden via the portal-only `database-provision-prefix` flag); `BuildDSN` accepts a `driver` but falls back to MySQL format with a warning when unsupported. -- If we add Postgres provisioning support, extend `BuildDSN` and `provisioner.DatabaseDriver` handling, add validations, and return `driver=postgres` consistently in API/CLI. -- Testing: exercise Portal endpoints with `httptest`, guard extraction paths with `pkg/fs.Unzip` size caps, and expect admin-only fields to disappear when authenticated as a client/user session. +- Prefer existing caches, workers, and batching strategies already referenced by the code and `Makefile`. +- Consider CPU and memory impact; only suggest profiling or benchmarks when justified. +- If anything here conflicts with the `Makefile` or the sources of truth, ask for clarification before proceeding. + +## Project Layout & Shared Rules + +- Backend: Go in `internal/`, `pkg/`, and `cmd/`, backed by MariaDB or SQLite. +- Frontend: Vue 3 plus Vuetify 3 under `frontend/`. +- Local dev and CI use Docker Compose; Traefik provides local TLS via `*.localssl.dev`. +- Code in `pkg/*` must not import from `internal/*`. If you need config, entity, or DB access, add code under `internal/`. +- Shared Go filesystem rules: + - Use `pkg/fs` permission constants: `fs.ModeDir`, `fs.ModeFile`, `fs.ModeConfigFile`, `fs.ModeSecretFile`, and `fs.ModeBackupFile`. + - When importing the stdlib `io/fs`, alias it to avoid collisions, for example `iofs "io/fs"` or `gofs "io/fs"`. + - Do not pass stdlib `io/fs` mode flags where permission bits are expected. + - Prefer `filepath.Join` for filesystem paths and `path.Join` only for URL paths. + - Normalize slash-based logical paths stored in DB, config, or API payloads with `clean.SlashPath(...)`. +- Shared Go style rules: + - After Go edits, run `make fmt-go` and keep `gofmt` tab indentation. + - Doc comments for packages and exported identifiers must be complete sentences that begin with the described name and end with a period. + - Every new function, including unexported helpers, needs a concise doc comment. + - Every new Go function, including unexported helpers, must have focused test coverage in the corresponding `*_test.go` files; update existing tests or add new ones as needed. + - For short examples in comments, indent code instead of using backticks. + - Every Go package must contain a root `.go` file with the standard license header and a short package description comment. +- Shared JS/Vue testing rules: + - New JavaScript functions, including helpers, should be tested whenever practical; update existing tests or add new ones as needed. + - New Vue components should have component-test coverage, and existing component tests should be updated as needed when behavior changes. +- When adding a metadata source such as `SrcOllama` or `SrcOpenAI`, update both `internal/entity/src.go` and `frontend/src/common/util.js` so backend and UI stay aligned. + +## Agent Runtime + +- Detect container mode by checking for `/.dockerenv`. +- If the repo path is `/go/src/github.com/photoprism/photoprism` and `/.dockerenv` is absent, treat the environment as host mode with a bind mount and prefer host-side Docker commands. +- Bash check: `[ -f "/.dockerenv" ] && echo container || echo host` +- Node.js check: `require("fs").existsSync("/.dockerenv")` +- Inside the container, prefer `npm exec --yes -- --help` or `npx ...`; if a global npm install is unavoidable, install it only inside the container. +- On the host, use the vendor-recommended install method and run from the repository root so agent discovery sees this file. + +## Build, Format & Test + +- Run `make help` to see supported targets. +- Host mode: + - `make docker-build` + - `docker compose up` or `docker compose up -d` + - `docker compose logs -f --tail=100 photoprism` + - `docker compose exec photoprism ./photoprism help` + - `docker compose exec -u "$(id -u):$(id -g)" photoprism ` to avoid root-owned files + - `make terminal` + - `docker compose --profile=all down --remove-orphans` or `make down` +- Container mode: + - `make dep` + - `make build-js` and `make build-go` + - `make watch-js` or `cd frontend && npm run watch` + - `./photoprism start` + - Local URLs: `http://localhost:2342/` and, with Traefik, `https://app.localssl.dev/` + - Local compose defaults to `admin` / `photoprism`; inspect `compose.yaml` if they differ. + - Do not use the Docker CLI inside the container; manage Compose from the host instead. +- The public CLI name is always `photoprism`; development-only side-by-side binaries may use edition-specific names. +- Our command examples assume a Linux or Unix shell on 64-bit AMD64 or ARM64; see the Developer Guide FAQ for Windows-specific notes. + +Formatting and test entry points: +- Full suite: `make test`, `make lint` +- Go-specific lint, format, and package-test rules live in [`internal/AGENTS.md`](internal/AGENTS.md). +- Frontend lint, Vitest, acceptance, and Playwright rules live in [`frontend/AGENTS.md`](frontend/AGENTS.md). +- Go tests live next to their sources; use PascalCase `t.Run(...)` names for related subtests. +- Do not run multiple test commands in parallel; suites share fixtures, assets, and database state. +- Prefer focused test runs such as `go test ./path/to/pkg -run Name -count=1` while iterating. +- Use `mariadb -D photoprism` inside the dev shell when you need to inspect MariaDB state directly. +- Run `shellcheck ` on edited shell scripts, or use the corresponding `make` target. diff --git a/frontend/AGENTS.md b/frontend/AGENTS.md new file mode 100644 index 000000000..c6a54729a --- /dev/null +++ b/frontend/AGENTS.md @@ -0,0 +1,51 @@ +# Frontend Guidelines + +**Last Updated:** April 9, 2026 + +## Frontend Linting & Test Entry Points + +- Use the lint and format scripts declared in `frontend/package.json`; all added JS, Vue, and frontend tests must follow those standards. +- Frontend unit tests use Vitest. Common entry points are `make test-js`, `make vitest-watch`, and `make vitest-coverage`. +- New JavaScript functions, including helpers, should be tested whenever practical; update existing tests or add new ones as needed. +- New Vue components should have component-test coverage, and existing component tests should be updated as needed when component behavior changes. +- Acceptance tests use the `acceptance-*` targets in the root `Makefile`. +- For one-off TestCafe checks, keep startup and cleanup in the repository root: `make storage/acceptance`, `make acceptance-sqlite-restart`, `make wait-2`, then `(cd frontend && npm run testcafe -- "chrome --headless=new --use-gl=angle --use-angle=swiftshader --disable-features=LocalNetworkAccessChecks" --config-file ./testcaferc.json --test-meta mode=public,type=short,testID=components-001 "tests/acceptance")`, then `make acceptance-sqlite-stop`. +- If a command temporarily changes into `frontend/`, return to the repository root before running `make acceptance-sqlite-stop`. + +## Templates, Session Bootstrap & Browser Baseline + +- HTML entry points live under `assets/templates/`; the key files are `index.gohtml`, `app.gohtml`, `app.js.gohtml`, and `splash.gohtml`. +- Browser checks live in `assets/static/js/browser-check.js` and must load before the main bundle from `app.js.gohtml`. Do not add `defer` or `async` unless you restore guarded loading. +- OIDC completion is bridged through `assets/templates/auth.gohtml` and must stay aligned with `frontend/src/common/session.js`, `frontend/src/common/storage.js`, and `frontend/src/page/auth/login.vue`. Preserve the `session` storage preference across the callback so `sessionStorage` logins survive redirect. +- When touching frontend session bootstrap, verify that `frontend/src/common/session.js` resolves `storageNamespace` from the real client config shape (`window.__CONFIG__` or `config.values`), not only from simplified mocks. Include a focused test that would fail if restore fell back to `pp:root:`. +- The loader partial is reused in `pro/assets/templates/index.gohtml`, `plus/assets/templates/index.gohtml`, and `portal/assets/templates/index.gohtml`; whenever you change `app.js.gohtml` or bundle loading, verify those files still include the shared partial. +- Splash styles live in `frontend/src/css/splash.css`; add new splash elements there so public and private editions stay aligned. +- Browser baseline: PhotoPrism supports Safari 13 and iOS 13 or current Chrome, Edge, and Firefox. Update the message in `assets/templates/app.js.gohtml` and matching CSS if support changes. + +## Translations + +- Translation extraction source of truth is the root `make gettext-extract`, which runs `scripts/gettext-extract.sh` across `frontend/src` and any available `plus`, `pro`, or `portal` overlays. +- Compatibility targets such as `make -C plus gettext-extract` delegate to the root target. +- Avoid punctuation-only gettext keys such as `$gettext("—")`; they create noisy entries in `frontend/src/locales/translations.pot`. + +## Focus Management + +- Dialogs must follow the shared focus pattern documented in `frontend/src/common/README.md`. +- Always expose `ref="dialog"` on `` overlays, call `$view.enter` and `$view.leave` in `@after-enter` and `@after-leave`, and avoid positive `tabindex` values. +- Persistent dialogs must handle Escape via `@keydown.esc.exact` so Vuetify's rejection animation is suppressed; keep other shortcuts on `@keyup` so inner inputs can cancel them first. +- Global shortcuts flow through `onShortCut(ev)` in `common/view.js`; it forwards only Escape and `ctrl` or `meta` combinations. +- When a dialog opens nested menus such as combobox suggestion lists, verify they still cooperate with the global trap. + +## Playwright MCP Usage + +- Default endpoint is `http://localhost:2342/`; default login routes are `/library/login` for CE, Plus, and Pro, and `/portal/admin/login` for Portal. +- Use the local compose admin credentials; if login fails, inspect the active compose environment. +- Desktop sessions default to `1280x900`; mobile sessions should use the mobile Playwright server with `375x667`. +- Close the browser tab after scripted interactions. +- Prefer waits over sleeps, click only visible and enabled elements, and use role, label, or text selectors instead of brittle XPath selectors. +- Keep screenshots small and reproducible: prefer JPEG, visible viewport, deterministic `.local/screenshots//__.jpg` names, and no large inline screenshots. +- If `npx` fetches an MCP server at runtime, add `--yes` or preinstall it to avoid prompts. + +## Frontend Test Gotchas + +- Hidden-route UI checks under `/library/hidden` or `/portal/admin/hidden` require both `files.file_error` and `photos.photo_quality = -1`; `file_error` alone will not surface the row. diff --git a/internal/AGENTS.md b/internal/AGENTS.md new file mode 100644 index 000000000..757ae9077 --- /dev/null +++ b/internal/AGENTS.md @@ -0,0 +1,43 @@ +# Internal Go Guidelines + +**Last Updated:** April 9, 2026 + +This file applies to `internal/` and defers subtree-specific rules to the narrower guides under `internal/api/`, `internal/config/`, `internal/commands/`, `internal/photoprism/`, and `internal/service/cluster/`. + +## Internal Linting + +- Run `make lint-go` after Go changes; for focused work, prefer `golangci-lint run ./internal//...`. + +## Logging, Naming & Status + +- When adding GORM struct fields with uppercase abbreviations such as `LabelNSFW`, `UserID`, or `URLHash`, set an explicit `gorm:"column:"` tag so column names stay stable. +- Use the shared logger via the package-level `log` variable backed by `event.Log`; avoid `fmt.Print*` and ad-hoc loggers. +- In human-readable log text, prefer `instance` and `service`; reserve `node` for contract-bound names such as `/cluster/nodes`, `Node*`, and `PHOTOPRISM_NODE_*`. +- End every `event.Audit*` slice with exactly one status token from `pkg/log/status`, such as `status.Succeeded`, `status.Failed`, or `status.Denied`. +- When the outcome should be a sanitized error string, use `status.Error(err)` instead of hand-building it. + +## Internal Tests & Fixtures + +- Keep Go scratch work inside `internal/...`; Go rejects imports from `internal/` when helpers live under paths such as `/tmp`. +- Heavy packages such as `internal/entity` and `internal/photoprism` run migrations and fixtures; expect slower first runs and narrow them with `-run`. +- For database updates, prefer `entity.Values` over raw `map[string]interface{}`. +- When adding persistent fixtures, generate IDs with `rnd.GenerateUID(...)` and the matching prefix instead of inventing manual strings. +- Prefer `config.NewMinimalTestConfig(t.TempDir())` for filesystem or config scaffolding and `config.NewMinimalTestConfigWithDb("", t.TempDir())` for isolated SQLite schemas. +- `internal/config` test helpers now auto-discover the repository `assets/` directory; do not set `PHOTOPRISM_ASSETS_PATH` manually in `init()` unless you truly have a non-standard layout. +- Hub API traffic is disabled in tests by default via `hub.ApplyTestConfig()`; opt back in with `PHOTOPRISM_TEST_HUB=test`. +- Avoid `config.TestConfig()` in new tests unless you need the fully seeded singleton fixture set; tests that write to Originals or Import should use isolated minimal configs plus `conf.CreateDirectories()`. +- `config.NewTestConfig("")` defaults to SQLite with a per-suite DSN such as `..db`; do not assert an empty DSN, and clean up captured DSNs with `t.Cleanup(...)` if needed. +- `NewTestConfig("")` already calls `InitializeTestData()`. If you build a custom config, call `c.InitializeTestData()` and optionally `c.AssertTestData(t)` so Originals, Import, cache, and temp exist. +- `PhotoFixtures.Get()` and similar helpers return value copies; re-query with helpers such as `entity.FindPhoto(...)` when a test needs the persisted row with associations. +- Reuse shared `Example*` constants for illustrative credentials in tests and docs. + +## Focused Internal Test Runs + +- Thumbnails: `go test ./internal/thumb/... -count=1` +- FFmpeg command builders: `go test ./internal/ffmpeg -run 'Remux|Transcode|Extract' -count=1` + +## FFmpeg Hardware Gating + +- Do not run GPU or hardware encoder integrations in CI by default; gate them with `PHOTOPRISM_FFMPEG_ENCODER` set to `vaapi`, `intel`, or `nvidia`. +- Keep negative-path ffmpeg tests fast and always runnable: missing ffmpeg should fail immediately, and unwritable destinations should fail without creating files. +- When hardware is unavailable, prefer command-string assertions; enable full hardware runs locally only when a device is configured. diff --git a/internal/api/AGENTS.md b/internal/api/AGENTS.md new file mode 100644 index 000000000..ef02daba3 --- /dev/null +++ b/internal/api/AGENTS.md @@ -0,0 +1,31 @@ +# API Guidelines + +**Last Updated:** April 9, 2026 + +## Handler Conventions + +- Reuse limiter stacks such as `limiter.Auth` and `limiter.Login`, and use `limiter.AbortJSON` for `429` responses. +- Lean on `api.ClientIP`, `header.BearerToken`, and the shared `Abort*` helpers instead of duplicating request parsing. +- Compare secrets in constant time and set `Cache-Control: no-store` on sensitive responses. +- Register routes in `internal/server/routes.go`. +- New list endpoints should default to `count=100`, cap at `1000`, require `offset >= 0`, and document their parameters explicitly. +- When a test or fixture needs portal mode, set `PHOTOPRISM_NODE_ROLE=portal` together with `PHOTOPRISM_JOIN_TOKEN`. + +## Roles, Scopes & Sessions + +- Map user roles through `acl.ParseRole(s)` and `acl.UserRoles[...]`; map client roles through `acl.ClientRoles[...]`. +- Treat `RoleAliasNone` (`none`) and the empty string as `RoleNone`; default unknown client roles to `RoleClient`. +- When checking JWT or client scopes, use `acl.ScopePermits` and `acl.ScopeAttrPermits` instead of handwritten parsing. +- Use `AuthenticateAdmin`, `AuthenticateUser`, or `OAuthToken` helpers in tests rather than building auth flows manually. +- Build client-session requests with `entity.AddClientSession(..., authn.GrantClientCredentials, nil)` and `AuthenticatedRequest(...)`. +- Admin sessions may see `AdvertiseUrl` and `Database`; client and user sessions must not. `SiteUrl` and the client `storageNamespace` derived from it are safe to expose. + +## Swagger & API Tests + +- Annotate only routed handlers in `internal/api/*.go`; use full `/api/v1/...` paths and skip helper functions. +- Regenerate API docs with `make fmt-go swag-fmt swag` or `make swag-json`; `make swag` may fetch modules, so confirm network access first. +- While iterating, prefer focused runs such as `go test ./internal/api -run Cluster -count=1`. +- Isolate config paths with `t.TempDir()` and reuse `NewConfig`, `CliTestContext`, and `NewApiTest()` helpers. +- For negative permission checks, prefer OAuth client tokens over non-admin user fixtures. +- Register `CreateSession(router)` only once per test router; a second registration panics on duplicate routes. + diff --git a/internal/commands/AGENTS.md b/internal/commands/AGENTS.md new file mode 100644 index 000000000..27f2921aa --- /dev/null +++ b/internal/commands/AGENTS.md @@ -0,0 +1,45 @@ +# Command-Line Guidelines + +**Last Updated:** April 9, 2026 + +## CLI Conventions + +- Prefer shared helpers such as `DryRunFlag(...)` and `YesFlag()` when adding command flags. +- Build CLI role help from `Roles.CliUsageString()` such as `acl.ClientRoles.CliUsageString()`; never hand-maintain role lists. +- Prefer `--json` for automation. `photoprism show commands --json [--nested]` exposes the command tree; add `--all` for hidden entries. +- Use `internal/commands/catalog` to inspect commands and flags without running the binary; when validating large JSON docs, marshal DTOs with `catalog.BuildFlat` or `catalog.BuildNode`. +- Expect `show` commands to return arrays of snake_case rows, except `photoprism show config`, which returns `{ sections: [...] }`, and `config-options` or `config-yaml`, which flatten to a top-level array. + +## CLI Tests + +- `urfave/cli` calls `os.Exit(code)` when a command returns `cli.Exit(...)`. Wrap command tests with `RunWithTestContext(cmd, args)` so `cli.OsExiter` is overridden and `go test` keeps running. +- If you only need the exit code, call `cmd.Action(ctx)` directly and assert `err.(cli.ExitCoder).ExitCode()`. +- Set `PHOTOPRISM_CLI=noninteractive` and or pass `--yes` to avoid prompts in tests and CI. +- Some commands defer `conf.Shutdown()` or emit signals that close the DB. Avoid invoking `start` or sending process signals in unit tests. +- `internal/commands/start.go` waits on `process.Signal`; avoid `process.Shutdown()` and `process.Restart()` in unit tests. + +## Download CLI Workbench + +Code anchors: +- CLI flags and examples: `internal/commands/download.go` +- Core implementation: `internal/commands/download_impl.go` +- yt-dlp helpers: `internal/photoprism/dl/*` +- Importer entry point: `internal/photoprism/get/import.go` + +Focused runs: +- `go test ./internal/commands -run 'DownloadImpl|HelpFlags' -count=1` +- `go test ./internal/photoprism/dl -run 'Options|Created|PostprocessorArgs' -count=1` + +FFmpeg-less tests: +- Set `c.Options().FFmpegBin = "/bin/false"` and `c.Settings().Index.Convert = false` when the test is not validating remux behavior. + +Stubbing yt-dlp without network: +- Use a small shell script that prints minimal JSON for `--dump-single-json` and creates a file when `--print` is requested. +- Supported harness env vars: `YTDLP_ARGS_LOG`, `YTDLP_OUTPUT_FILE`, and `YTDLP_DUMMY_CONTENT`. + +Remux and defaults: +- Pipe mode always remuxes through PhotoPrism ffmpeg and embeds title, description, and created time. +- File mode relies on yt-dlp output and passes `--postprocessor-args 'ffmpeg:-metadata creation_time='` so imports still get `Created`. +- Default remux policy is `auto`; use `always` when you need the most complete metadata. +- `photoprism dl` defaults to `--method pipe` and `--impersonate firefox`; pass `-i none` to disable impersonation. + diff --git a/internal/config/AGENTS.md b/internal/config/AGENTS.md new file mode 100644 index 000000000..6211f7d5b --- /dev/null +++ b/internal/config/AGENTS.md @@ -0,0 +1,22 @@ +# Config Guidelines + +**Last Updated:** April 9, 2026 + +## Option Precedence & Wiring + +- Respect config precedence: `options.yml` overrides CLI and environment values, which override defaults. +- When adding a new option, update `internal/config/options.go` for YAML and flag tags, register the flag in `internal/config/flags.go`, expose a getter, surface it in `*config.Report()`, and persist generated values by setting `c.options.OptionsYaml` before writing `options.yml`. +- Exercise new flags with `CliTestContext` from `internal/config/test.go`. + +## Config Persistence + +- For `options.yml` writes, use config-owned helpers instead of ad-hoc YAML handling: `Config.SaveOptionsPatch(...)` for generic merges and `Config.SaveClusterOptionsUpdate(...)` for cluster-managed updates. +- Use `pkg/fs.ConfigFilePath` when you need a config filename so existing `.yml` files stay valid while new installs may adopt `.yaml`. +- In Go code, use public `*config.Config` accessors such as `Config.JWKSUrl()`, `Config.SetJWKSUrl()`, and `Config.ClusterUUID()` instead of mutating `Config.Options()` directly; reserve raw option mutation for test fixtures. + +## CLI Override Rules & DB Helpers + +- Favor explicit CLI flags: check `c.cliCtx.IsSet("")` before overriding user-supplied values. +- Follow the `ClusterUUID` pattern for generated values: `options.yml`, then CLI or environment overrides, then a generated value persisted back to disk. +- Reuse `conf.Db()` and `conf.Database*()` helpers, avoid GORM `WithContext`, quote MySQL identifiers, and reject unsupported drivers early. + diff --git a/internal/photoprism/AGENTS.md b/internal/photoprism/AGENTS.md new file mode 100644 index 000000000..1fd5d64dc --- /dev/null +++ b/internal/photoprism/AGENTS.md @@ -0,0 +1,25 @@ +# PhotoPrism Core Import & Index Guidelines + +**Last Updated:** April 9, 2026 + +## Import & Index Behavior + +- `ImportWorker` may skip a file when an identical file already exists. Use unique copies or assert DB rows only after ensuring the destination is not a duplicate. +- Keep `SamplesPath()`, `ImportPath()`, and `OriginalsPath()` consistent when testing related files so `RelatedFiles` and `AllowExt` behave correctly. +- `IndexOptions*` helpers now require a `*config.Config`; pass the active config, or `config.NewMinimalTestConfig(t.TempDir())` in unit tests, so face, label, and NSFW scheduling matches the current run. +- Vision scheduling is controlled by `VisionSchedule`, `VisionFilter`, and the `Run` property in `vision.yml`. Use helpers such as `vision.FilterModels` and `entity.Photo.ShouldGenerateLabels` or `ShouldGenerateCaption` before loading media. +- Folder albums use path-first lookup and update via `album_path` to avoid slug collisions for emoji child paths; re-indexing may repair stale collision titles while preserving user-custom titles. +- Reuse `entity.FindLabels(...)`, `entity.FindLabelIDs(...)`, and `entity.LabelSlugs(...)` for label lookup so homophone-aware exact-name matching stays aligned across entity and search packages. + +## File I/O Overwrite Policy + +- Default to safety: callers must not overwrite non-empty destination files unless they opt in with `force=true`. +- Replacing an empty destination file is allowed without `force=true`. +- Open overwriting destinations with `O_WRONLY|O_CREATE|O_TRUNC` so stale trailing bytes cannot survive; use `O_EXCL` when the caller must detect collisions. +- App-level overwrite helpers live in `internal/photoprism/mediafile.go`; reusable helpers live in `pkg/fs/copy.go` and `pkg/fs/move.go`. +- Set `force=true` only for explicit replace flows or admin tools with confirmed overwrite, not for import or index flows that touch Originals. + +## Tests + +- In `internal/photoprism` tests, rely on `photoprism.Config()` for runtime-accurate behavior; only build a new config if you replace it with `photoprism.SetConfig`. + diff --git a/internal/service/cluster/AGENTS.md b/internal/service/cluster/AGENTS.md new file mode 100644 index 000000000..f4f5bfd95 --- /dev/null +++ b/internal/service/cluster/AGENTS.md @@ -0,0 +1,45 @@ +# Cluster Guidelines + +**Last Updated:** April 9, 2026 + +## Bootstrap & Registration + +- Keep bootstrap code decoupled: do not import `internal/service/cluster/node/*` from `internal/config` or the cluster root; nodes talk to the Portal over HTTP(S) and use `internal/service/cluster/const.go`. +- On `401` or `403`, bootstrap refreshes node OAuth credentials by rotating the secret and retrying; log that at info level. If the secret file cannot be written, keep the rotated value in memory. +- Portal validation may accept HTTP advertise URLs only for loopback or cluster-internal domains such as `*.svc`, `*.cluster.local`, and `*.internal`; all other advertise URLs must use HTTPS. +- Registration flow: send `rotate=true` only for MySQL or MariaDB nodes without credentials, treat `401`, `403`, and `404` as terminal, include `ClientID` plus `ClientSecret` when renaming an existing node, and persist only newly generated secrets or DB settings. +- Config init order for cluster-aware startup is: load `options.yml` with `c.initSettings()`, run `EarlyExt().InitEarly(c)`, connect or register the DB, then invoke `Ext().Init(c)`. + +## Registry, DTOs & Provisioning + +- Use `NewClientRegistryWithConfig`; the file-backed registry is legacy. +- Nodes are keyed by UUID v7 at `/api/v1/cluster/nodes/{uuid}`. Keep the registry interface UUID-first: `Get`, `FindByNodeUUID`, `FindByClientID`, `RotateSecret`, and `DeleteAllByUUID`. +- CLI lookups should resolve `uuid -> ClientID -> name`. +- DTOs normalize `Database.{Name,User,Driver,RotatedAt}` and expose `ClientSecret` only during creation or rotation. +- `nodes rm --all-ids` must clean duplicate client rows. +- Registry files live under `conf.PortalConfigPath()/nodes/` with mode `0600`, and `ClientData` no longer stores `NodeUUID`. +- Database and user names use UUID-based HMACs in `d` and `u` form; the prefix defaults to `cluster_` and may be overridden only by the portal-only `database-provision-prefix` flag. +- `BuildDSN` accepts a `driver` but falls back to MySQL format with a warning when the driver is unsupported. +- If Postgres provisioning is added, extend both `BuildDSN` and `provisioner.DatabaseDriver`, add validations, and return `driver=postgres` consistently in API and CLI output. + +## Cluster API & Theme Changes + +- When renaming or adding cluster response fields, update DTOs in `internal/service/cluster/response.go`, handlers, Swagger, tests, specs, and grep for old and new field names. +- The theme endpoint `GET /api/v1/cluster/theme` streams a zip from `conf.ThemePath()`. Reinstall only when `app.js` is missing and use the shared helpers in `pkg/http/header`. +- Admin responses may include `AdvertiseUrl` and `Database`; client and user sessions must remain redacted. + +## Cluster Tests + +- Generate OAuth client IDs with `rnd.GenerateUID(entity.ClientUID)` and node UUIDs with `rnd.UUIDv7()`; treat `node.uuid` as required in responses. +- Cluster registry tests under `internal/service/cluster/registry` intentionally use a full `config.TestConfig()` because they persist `entity.Client` rows. Do not switch them to minimal config helpers unless the tests stop touching the database. +- Exercise Portal endpoints with `httptest`, guard extraction paths with `pkg/fs.Unzip` size caps, and confirm admin-only fields disappear for client or user sessions. +- Portal proxy URI validation must use the Portal test environment with `NODES=2` and verify both instance routes when changing `PHOTOPRISM_PORTAL_PROXY_URI` or matching node `PHOTOPRISM_SITE_URL` prefixes; use `PORTAL_TEST_ENV_ARGS=--proxy-uri=/instance/` to regenerate consistent `.env` values. +- Before `make -C portal test-start`, run a full rebuild with `make -C portal test-env NODES=2`; avoid `--no-build` refreshes unless you are intentionally validating env-only changes. + +## Cluster Preflight + +- `go build ./...` +- `make fmt-go swag-fmt swag` +- `go test ./internal/service/cluster/registry -count=1` +- `go test ./internal/api -run 'Cluster' -count=1` +- `go test ./internal/commands -run 'ClusterRegister|ClusterNodesRotate' -count=1` diff --git a/pkg/AGENTS.md b/pkg/AGENTS.md new file mode 100644 index 000000000..739389d0f --- /dev/null +++ b/pkg/AGENTS.md @@ -0,0 +1,31 @@ +# Package Security & Test Guidelines + +**Last Updated:** April 9, 2026 + +## Archive Extraction Security + +- Always validate ZIP entry names with a safe join. Reject absolute paths, Windows drive or volume paths, and any entry that escapes the target directory after cleaning. +- ZIP entry names use slash semantics, not host OS semantics: validate with `path.Clean` and `path.IsAbs`, reject backslashes, and use `path.Base` for hidden-name checks. +- Convert ZIP names to OS paths only at write time with `filepath.FromSlash(...)`. +- Enforce destination containment with `filepath.Rel(...)` rather than string-prefix checks. +- Enforce per-file and total-size budgets to prevent resource exhaustion. +- Skip OS metadata directories such as `__MACOSX` and reject suspicious names. +- Keep tests for absolute and volume path rejection, traversal skipping, `__MACOSX` skipping, size limits, directory creation, and safe nested extraction. +- The current implementation lives in `pkg/fs/zip.go` via `Unzip`, `UnzipFile`, and `safeJoin`. + +## HTTP Download Security + +- Use `pkg/http/safe` and `safe.Download(destPath, url, *safe.Options)` instead of ad-hoc `net/http` download code. +- Default policy allows only `http` and `https`, enforces timeouts and max size, writes to a `0600` temp file, then renames into place. +- For SSRF protection, set `AllowPrivate=false` unless a test explicitly needs private or loopback addresses. +- Validate redirect targets and the final connected peer IP. +- Prefer an image-focused `Accept` header for image downloads: `"image/jpeg, image/png, */*;q=0.1"`. +- Use `internal/thumb/avatar.SafeDownload` for avatars and other small images; it applies a 15-second timeout, a 10 MiB cap, and `AllowPrivate=false`. +- Tests using `httptest.Server` on `127.0.0.1` must set `AllowPrivate=true`. +- Keep size budgets small and rely on `io.LimitReader` plus `Content-Length` prechecks. + +## Focused Package Test Runs + +- Filesystem copy, move, and unzip helpers: `go test ./pkg/fs -run 'Copy|Move|Unzip' -count=1` +- Media helpers: `go test ./pkg/media/... -count=1` +