mirror of
https://github.com/Dispatcharr/Dispatcharr.git
synced 2026-07-17 16:50:53 +00:00
Enhancement: add caching for plugin detail responses and implement cache invalidation on repo refresh.
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
Base Image Build / prepare (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
Base Image Build / prepare (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
Bug Fix: The plugin detail endpoint called GPG via `subprocess.Popen` to verify per-plugin manifest signatures. Replaced with a `_gpg_run()` helper that uses `os.posix_spawn`, matching the pattern used by the ffmpeg and script-handler fixes.
This commit is contained in:
parent
2f66b7c183
commit
7e0ea5eae0
3 changed files with 222 additions and 72 deletions
|
|
@ -105,6 +105,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
- Dependency updates:
|
||||
- `Django` 6.0.4 → 6.0.5 (security patch; see Security section)
|
||||
|
||||
### Removed
|
||||
|
||||
- **`python-gnupg` dependency dropped.** GPG manifest signature verification now calls the `gpg` binary directly via `os.posix_spawn` (see Fixed below). The `python-gnupg` Python library was the only consumer and has been removed from `pyproject.toml`. The `gpg` binary itself is still required on the host (it was always required since `python-gnupg` is just a wrapper around it).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **DVR settings form no longer flashes back to old values during save.** The comskip mode and hardware acceleration selects briefly showed stale values while the save was in flight because the Zustand settings store update (triggered by the API response) fired the `useEffect([settings])` re-hydration hook mid-save. An `isSavingRef` guard now suppresses the reactive re-hydration while a save is in progress; after a successful save the form is explicitly synced from the freshly-updated store state instead.
|
||||
|
|
@ -116,6 +120,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
- `input/http_streamer.py`: set `O_NONBLOCK` on the HTTP-to-pipe relay write-end with an EAGAIN retry loop for the same reason.
|
||||
- `core/views.py` (`stream_view`): replaced `subprocess.Popen` with `os.posix_spawn`; also fixed a pre-existing indentation bug where the `return StreamingHttpResponse(...)` was accidentally nested inside `stream_generator` (making every successful response return `None` and raise a Django error). Also corrected two `NameError` references to the undefined `stream_id` variable in log messages.
|
||||
- `apps/connect/handlers/script.py` (`ScriptHandler`): replaced `subprocess.run` with a `_posix_run` helper that uses `os.posix_spawn` + cooperative `select.select` reads + non-blocking `waitpid` polling. Without this fix, any script-type connect integration configured for events fired from a uWSGI worker (e.g. `client_connect`) would deadlock the serving greenlet. Note: `cwd` is no longer set to the script's directory during execution (it inherits the worker's cwd); this was a minor convenience, not a documented guarantee.
|
||||
- **`POST /api/plugins/repos/plugin-detail/` hung for up to 105 seconds under gevent+uWSGI.** The plugin detail endpoint called GPG via `subprocess.Popen` to verify per-plugin manifest signatures, triggering the same `fork()` atfork deadlock described above. Replaced with a `_gpg_run()` helper that uses `os.posix_spawn`, matching the pattern used by the ffmpeg and script-handler fixes. `select.select()` drains stdout/stderr cooperatively (gevent-patched) and `os.waitpid(WNOHANG)` with `time.sleep(0.01)` reaps the child without blocking the hub. Results are cached in Redis for 5 minutes per manifest URL so repeat detail fetches skip GPG entirely. The cache is also invalidated per-plugin when the owning repo's hub manifest is refreshed, so a newly released version is visible immediately after a manual hub refresh.
|
||||
- **XC server sub-path URLs now work correctly.** When a provider serves its XC API from a sub-path (e.g. `http://server/Pluto/gb/player_api.php`), Dispatcharr was stripping the path entirely and hitting the root (`/player_api.php`) instead. `_normalize_url` now preserves sub-path components and only strips any trailing `.php` segment (covering `player_api.php`, `get.php`, `xmltv.php`, and any future endpoint without a maintained list). The same fix is applied to `get_transformed_credentials` in the M3U profile transformation path. (Fixes #1218)
|
||||
- **M3U filter delete confirmation showed wrong field name and had a typo.** The confirmation dialog for deleting an M3U filter read `filter.type` (always `undefined`) instead of `filter.filter_type`, leaving the "Type:" line blank, and displayed "Patter:" instead of "Pattern:". Both are corrected. — Thanks [@nick4810](https://github.com/nick4810)
|
||||
- **M3U form FileInput expanded the modal width on long filenames.** Uploading a local M3U file with a long name caused the `FileInput` to expand beyond the modal's layout bounds. The input now clips overflow with `textOverflow: ellipsis`. — Thanks [@nick4810](https://github.com/nick4810)
|
||||
|
|
|
|||
126
Plugin_repo.md
126
Plugin_repo.md
|
|
@ -81,6 +81,7 @@ This is the simplest valid repo manifest - one plugin with enough info to show i
|
|||
Dispatcharr accepts two top-level shapes:
|
||||
|
||||
**Wrapped (supports signing):**
|
||||
|
||||
```json
|
||||
{
|
||||
"manifest": { "plugins": [...], ... },
|
||||
|
|
@ -89,6 +90,7 @@ Dispatcharr accepts two top-level shapes:
|
|||
```
|
||||
|
||||
**Flat (no signing):**
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": [...],
|
||||
|
|
@ -118,36 +120,36 @@ If the name contains any of these, the repo will be rejected on add and skipped
|
|||
|
||||
### Top-Level Metadata
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `registry_name` | **Yes** | Display name for the repo. Must not contain words like "official" or "dispatcharr" that could be mistaken for an official repo (see [Name Restrictions](#name-restrictions)). |
|
||||
| `registry_url` | No | URL to the repo's home page (e.g. GitHub). Used as a fallback for generating icon URLs. |
|
||||
| `root_url` | No | Base URL for resolving relative URLs in plugin entries. Trailing slashes are stripped. |
|
||||
| `plugins` | **Yes** | Array of plugin entry objects. |
|
||||
| Field | Required | Description |
|
||||
| --------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `registry_name` | **Yes** | Display name for the repo. Must not contain words like "official" or "dispatcharr" that could be mistaken for an official repo (see [Name Restrictions](#name-restrictions)). |
|
||||
| `registry_url` | No | URL to the repo's home page (e.g. GitHub). Used as a fallback for generating icon URLs. |
|
||||
| `root_url` | No | Base URL for resolving relative URLs in plugin entries. Trailing slashes are stripped. |
|
||||
| `plugins` | **Yes** | Array of plugin entry objects. |
|
||||
|
||||
### Plugin Entry Fields
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `slug` | **Yes** | Unique identifier. Alphanumeric, dashes, and underscores. Used as the install directory name (lowercased, dashes converted to underscores). |
|
||||
| `name` | **Yes** | Human-readable display name. |
|
||||
| `description` | No | Short description shown on the plugin card. |
|
||||
| `author` | No | Author or organization name. |
|
||||
| `maintainers` | No | Array of maintainer GitHub usernames (e.g. `["alice", "bob"]`). Shown in the detail view. |
|
||||
| `license` | No | SPDX license identifier (e.g. `MIT`, `GPL-3.0`). Displayed as a link to the SPDX license page. |
|
||||
| `deprecated` | No | Boolean. When `true`, marks the plugin as deprecated in the store UI. Omit or set to `false` for active plugins. |
|
||||
| `repo_url` | No | URL to the plugin's source code repository (e.g. GitHub). |
|
||||
| `discord_thread` | No | URL to a Discord thread or channel for plugin support. Must start with `http://` or `https://`. |
|
||||
| `latest_version` | No | Current latest version string (semver: `1.2.3` or `v1.2.3`). Drives update detection. |
|
||||
| `last_updated` | No | ISO 8601 timestamp of the latest release. Shown as "Built" date in the detail view. |
|
||||
| `manifest_url` | No | URL (or relative path) to the per-plugin manifest with full version history. See [Per-Plugin Manifest](#per-plugin-manifest). |
|
||||
| `latest_url` | No | Direct download URL (or relative path) to the latest release zip. |
|
||||
| `latest_sha256` | No | SHA256 checksum of the latest release zip (lowercase hex, 64 chars). |
|
||||
| `latest_md5` | No | MD5 checksum of the latest release zip. Informational only - not validated by Dispatcharr. |
|
||||
| `latest_size` | No | Size of the latest release zip in kilobytes. Informational only. |
|
||||
| `icon_url` | No | URL (or relative path) to a logo image (PNG recommended). |
|
||||
| `min_dispatcharr_version` | No | Minimum Dispatcharr version required. Install is blocked if the running version is older. |
|
||||
| `max_dispatcharr_version` | No | Maximum Dispatcharr version supported. Install is blocked if the running version is newer. |
|
||||
| Field | Required | Description |
|
||||
| ------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `slug` | **Yes** | Unique identifier. Alphanumeric, dashes, and underscores. Used as the install directory name (lowercased, dashes converted to underscores). |
|
||||
| `name` | **Yes** | Human-readable display name. |
|
||||
| `description` | No | Short description shown on the plugin card. |
|
||||
| `author` | No | Author or organization name. |
|
||||
| `maintainers` | No | Array of maintainer GitHub usernames (e.g. `["alice", "bob"]`). Shown in the detail view. |
|
||||
| `license` | No | SPDX license identifier (e.g. `MIT`, `GPL-3.0`). Displayed as a link to the SPDX license page. |
|
||||
| `deprecated` | No | Boolean. When `true`, marks the plugin as deprecated in the store UI. Omit or set to `false` for active plugins. |
|
||||
| `repo_url` | No | URL to the plugin's source code repository (e.g. GitHub). |
|
||||
| `discord_thread` | No | URL to a Discord thread or channel for plugin support. Must start with `http://` or `https://`. |
|
||||
| `latest_version` | No | Current latest version string (semver: `1.2.3` or `v1.2.3`). Drives update detection. |
|
||||
| `last_updated` | No | ISO 8601 timestamp of the latest release. Shown as "Built" date in the detail view. |
|
||||
| `manifest_url` | No | URL (or relative path) to the per-plugin manifest with full version history. See [Per-Plugin Manifest](#per-plugin-manifest). |
|
||||
| `latest_url` | No | Direct download URL (or relative path) to the latest release zip. |
|
||||
| `latest_sha256` | No | SHA256 checksum of the latest release zip (lowercase hex, 64 chars). |
|
||||
| `latest_md5` | No | MD5 checksum of the latest release zip. Informational only - not validated by Dispatcharr. |
|
||||
| `latest_size` | No | Size of the latest release zip in kilobytes. Informational only. |
|
||||
| `icon_url` | No | URL (or relative path) to a logo image (PNG recommended). |
|
||||
| `min_dispatcharr_version` | No | Minimum Dispatcharr version required. Install is blocked if the running version is older. |
|
||||
| `max_dispatcharr_version` | No | Maximum Dispatcharr version supported. Install is blocked if the running version is newer. |
|
||||
|
||||
Extra fields in a plugin entry are passed through to the frontend as-is, so you can include custom metadata (e.g. `homepage`, `tags`) without breaking anything.
|
||||
|
||||
|
|
@ -160,6 +162,7 @@ If `root_url` is set and a URL field (`manifest_url`, `latest_url`, `icon_url`)
|
|||
```
|
||||
|
||||
This lets you keep plugin entries compact:
|
||||
|
||||
```json
|
||||
{
|
||||
"root_url": "https://raw.githubusercontent.com/myorg/my-plugins/releases",
|
||||
|
|
@ -175,6 +178,7 @@ This lets you keep plugin entries compact:
|
|||
```
|
||||
|
||||
**Icon fallback:** If `icon_url` is missing and `registry_url` is set, Dispatcharr generates a fallback URL by converting the GitHub URL to a raw content URL:
|
||||
|
||||
```
|
||||
{registry_url => raw.githubusercontent.com}/refs/heads/main/plugins/{slug}/logo.png
|
||||
```
|
||||
|
|
@ -186,6 +190,7 @@ This lets you keep plugin entries compact:
|
|||
The per-plugin manifest provides full version history. It is fetched on-demand when a user clicks "More Info" on a plugin card. It is **not required** - if `manifest_url` is absent, the UI builds a detail view from the repo-level fields instead.
|
||||
|
||||
Include a per-plugin manifest if you want to:
|
||||
|
||||
- Offer multiple downloadable versions
|
||||
- Show per-version compatibility ranges
|
||||
- Display build timestamps and commit links for each version
|
||||
|
|
@ -196,6 +201,7 @@ Include a per-plugin manifest if you want to:
|
|||
Same as the root manifest - both flat and wrapped formats are accepted:
|
||||
|
||||
**Flat (no signing):**
|
||||
|
||||
```json
|
||||
{
|
||||
"slug": "...",
|
||||
|
|
@ -204,6 +210,7 @@ Same as the root manifest - both flat and wrapped formats are accepted:
|
|||
```
|
||||
|
||||
**Wrapped (supports signing):**
|
||||
|
||||
```json
|
||||
{
|
||||
"manifest": {
|
||||
|
|
@ -272,33 +279,33 @@ Use the wrapped format if you want to GPG-sign the per-plugin manifest.
|
|||
|
||||
### Per-Plugin Manifest Fields
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `slug` | No | Plugin identifier (should match the repo entry). |
|
||||
| `name` | No | Display name. |
|
||||
| `description` | No | Full description shown in the detail modal. |
|
||||
| `author` | No | Author/org name shown in the detail modal. |
|
||||
| `license` | No | SPDX license identifier. |
|
||||
| `latest_version` | No | Latest version string. |
|
||||
| `registry_name` | No | Registry name inherited from the parent repo manifest. Injected automatically by the official publish tooling. |
|
||||
| `registry_url` | No | Registry URL inherited from the parent repo manifest. Used by the store to build commit links. Injected automatically by the official publish tooling. |
|
||||
| `versions` | No | Array of version objects (newest first recommended). |
|
||||
| `latest` | No | Object mirroring the latest version entry for quick access. Accepts all the same fields as a version object. Additionally, `latest_url` may appear here pointing to a stable symlink (e.g. `plugin-latest.zip`) that always resolves to the newest release. |
|
||||
| Field | Required | Description |
|
||||
| ---------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `slug` | No | Plugin identifier (should match the repo entry). |
|
||||
| `name` | No | Display name. |
|
||||
| `description` | No | Full description shown in the detail modal. |
|
||||
| `author` | No | Author/org name shown in the detail modal. |
|
||||
| `license` | No | SPDX license identifier. |
|
||||
| `latest_version` | No | Latest version string. |
|
||||
| `registry_name` | No | Registry name inherited from the parent repo manifest. Injected automatically by the official publish tooling. |
|
||||
| `registry_url` | No | Registry URL inherited from the parent repo manifest. Used by the store to build commit links. Injected automatically by the official publish tooling. |
|
||||
| `versions` | No | Array of version objects (newest first recommended). |
|
||||
| `latest` | No | Object mirroring the latest version entry for quick access. Accepts all the same fields as a version object. Additionally, `latest_url` may appear here pointing to a stable symlink (e.g. `plugin-latest.zip`) that always resolves to the newest release. |
|
||||
|
||||
### Version Object Fields
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `version` | **Yes** | Version string (`1.2.3` or `v1.2.3`). |
|
||||
| `url` | **Yes** | Download URL for the zip. Relative URLs are resolved against the repo's `root_url`. |
|
||||
| `checksum_sha256` | No | SHA256 hex checksum. **Strongly recommended.** Validated on install - mismatch blocks the install. |
|
||||
| `prerelease` | No | Boolean. When `true`, marks this version as a pre-release (alpha, beta, RC, etc.). If the installed version is a prerelease, Dispatcharr will not suggest updating to the latest stable version - the user must install a new version manually. The latest version in the root manifest is always assumed to be stable, so this field only needs to appear in the per-plugin manifest. Omit or set to `false` for stable releases. |
|
||||
| `build_timestamp` | No | ISO 8601 build timestamp. Shown as "Built" in the version detail. |
|
||||
| `commit_sha` | No | Full Git commit SHA. Used to build a commit link if `registry_url` is set. |
|
||||
| `commit_sha_short` | No | Abbreviated commit SHA. Displayed in the version detail table as a clickable link. |
|
||||
| `size` | No | Size of this version's zip in kilobytes. Informational only. |
|
||||
| `min_dispatcharr_version` | No | Minimum compatible Dispatcharr version. |
|
||||
| `max_dispatcharr_version` | No | Maximum compatible Dispatcharr version. |
|
||||
| Field | Required | Description |
|
||||
| ------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `version` | **Yes** | Version string (`1.2.3` or `v1.2.3`). |
|
||||
| `url` | **Yes** | Download URL for the zip. Relative URLs are resolved against the repo's `root_url`. |
|
||||
| `checksum_sha256` | No | SHA256 hex checksum. **Strongly recommended.** Validated on install - mismatch blocks the install. |
|
||||
| `prerelease` | No | Boolean. When `true`, marks this version as a pre-release (alpha, beta, RC, etc.). If the installed version is a prerelease, Dispatcharr will not suggest updating to the latest stable version - the user must install a new version manually. The latest version in the root manifest is always assumed to be stable, so this field only needs to appear in the per-plugin manifest. Omit or set to `false` for stable releases. |
|
||||
| `build_timestamp` | No | ISO 8601 build timestamp. Shown as "Built" in the version detail. |
|
||||
| `commit_sha` | No | Full Git commit SHA. Used to build a commit link if `registry_url` is set. |
|
||||
| `commit_sha_short` | No | Abbreviated commit SHA. Displayed in the version detail table as a clickable link. |
|
||||
| `size` | No | Size of this version's zip in kilobytes. Informational only. |
|
||||
| `min_dispatcharr_version` | No | Minimum compatible Dispatcharr version. |
|
||||
| `max_dispatcharr_version` | No | Maximum compatible Dispatcharr version. |
|
||||
|
||||
Relative `url` values in versions are resolved the same way as repo-level URLs: `{root_url}/{url}`.
|
||||
|
||||
|
|
@ -348,6 +355,7 @@ jq -c '.manifest' manifest.json | gpg --armor --detach-sign
|
|||
```
|
||||
|
||||
In code terms:
|
||||
|
||||
```python
|
||||
import json
|
||||
canonical = json.dumps(manifest_obj, separators=(",", ":")) + "\n"
|
||||
|
|
@ -371,11 +379,11 @@ Use the wrapped format so the signature sits alongside the manifest:
|
|||
|
||||
### Verification Results
|
||||
|
||||
| Result | Meaning | UI Badge |
|
||||
|--------|---------|----------|
|
||||
| `true` | Valid signature | Green checkmark |
|
||||
| `false` | Invalid signature or verification error | Red X |
|
||||
| `null` | Not attempted (no signature, no key, or `python-gnupg` not installed) | Gray/neutral |
|
||||
| Result | Meaning | UI Badge |
|
||||
| ------- | ------------------------------------------------------------------- | --------------- |
|
||||
| `true` | Valid signature | Green checkmark |
|
||||
| `false` | Invalid signature or verification error | Red X |
|
||||
| `null` | Not attempted (no signature, no key, or `gpg` binary not installed) | Gray/neutral |
|
||||
|
||||
### Signing Workflow Example
|
||||
|
||||
|
|
@ -446,6 +454,7 @@ my_plugin-1.0.0.zip
|
|||
```
|
||||
|
||||
Or with a subdirectory:
|
||||
|
||||
```
|
||||
my_plugin-1.0.0.zip
|
||||
my_plugin/
|
||||
|
|
@ -477,6 +486,7 @@ The plugin is installed **disabled** by default. The user can enable it from the
|
|||
Dispatcharr detects updates by comparing `installed_version` (stored in the database) against `latest_version` from the repo manifest. This uses repo-level fields only - per-plugin manifests are not needed for update detection.
|
||||
|
||||
A plugin shows "Update Available" when:
|
||||
|
||||
- It is managed (installed from a repo)
|
||||
- Its `installed_version` differs from `latest_version`
|
||||
- It was installed from the same repo
|
||||
|
|
@ -488,7 +498,9 @@ A plugin shows "Update Available" when:
|
|||
A plugin repo manifest is just a JSON file served over HTTPS. Some options:
|
||||
|
||||
### GitHub Pages / Raw Content
|
||||
|
||||
Host your manifest and release zips in a GitHub repo. Use raw.githubusercontent.com URLs:
|
||||
|
||||
```
|
||||
https://raw.githubusercontent.com/myorg/my-plugins/main/manifest.json
|
||||
```
|
||||
|
|
@ -496,9 +508,11 @@ https://raw.githubusercontent.com/myorg/my-plugins/main/manifest.json
|
|||
Use `root_url` pointing to your releases branch/path so version URLs stay relative.
|
||||
|
||||
### Static File Server
|
||||
|
||||
Any web server that serves JSON works. Dispatcharr fetches manifests server-side, so CORS is not needed.
|
||||
|
||||
### GitHub Releases
|
||||
|
||||
You can host release zips as GitHub Release assets and reference them with absolute URLs in your manifest. The manifest itself can live in the repo's default branch.
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
import hashlib
|
||||
import ipaddress
|
||||
import logging
|
||||
import io
|
||||
import json
|
||||
import re
|
||||
import socket
|
||||
|
|
@ -9,6 +8,7 @@ from rest_framework.views import APIView
|
|||
from rest_framework.response import Response
|
||||
from rest_framework import status, serializers
|
||||
from drf_spectacular.utils import extend_schema, inline_serializer
|
||||
from django.core.cache import cache
|
||||
from django.conf import settings
|
||||
from django.core.files.uploadedfile import UploadedFile
|
||||
from django.http import FileResponse
|
||||
|
|
@ -335,6 +335,21 @@ def _save_fetched_manifest_to_repo(repo, data, verified):
|
|||
return None
|
||||
|
||||
|
||||
def _invalidate_plugin_detail_cache(repo_id, manifest_data):
|
||||
manifest = manifest_data.get("manifest", manifest_data)
|
||||
root_url = manifest.get("root_url", "").rstrip("/")
|
||||
keys = []
|
||||
for p in manifest.get("plugins", []):
|
||||
url = p.get("manifest_url", "")
|
||||
if not url:
|
||||
continue
|
||||
if root_url and not url.startswith(("http://", "https://")):
|
||||
url = f"{root_url}/{url}"
|
||||
keys.append(f"plugin_detail:{repo_id}:{hashlib.md5(url.encode()).hexdigest()}")
|
||||
if keys:
|
||||
cache.delete_many(keys)
|
||||
|
||||
|
||||
def _unmanage_dropped_slugs(repo, new_manifest_data):
|
||||
"""After a manifest refresh, clear source_repo on any installed plugins
|
||||
whose slug is no longer listed in the repo's manifest. Also syncs the
|
||||
|
|
@ -571,6 +586,7 @@ class PluginDeleteAPIView(PluginAuthMixin, APIView):
|
|||
# ---------------------------------------------------------------------------
|
||||
|
||||
MANIFEST_FETCH_TIMEOUT = 15
|
||||
PLUGIN_DETAIL_CACHE_TTL = 300 # seconds
|
||||
|
||||
OFFICIAL_KEY_PATH = os.path.join(
|
||||
os.path.dirname(__file__), "keys", "dispatcharr-plugins.pub"
|
||||
|
|
@ -589,6 +605,101 @@ def _normalize_pgp_key(text):
|
|||
return text
|
||||
|
||||
|
||||
def _gpg_run(cmd, input_data=None, timeout=30):
|
||||
"""
|
||||
Run a GPG command using os.posix_spawn.
|
||||
|
||||
os.posix_spawn skips pthread_atfork handlers, avoiding the indefinite hang
|
||||
that fork()-based approaches suffer under gevent+uWSGI. select.select()
|
||||
and time.sleep() are gevent-patched so reads and the waitpid poll yield to
|
||||
the hub cooperatively.
|
||||
|
||||
Returns (returncode, stdout_bytes, stderr_bytes).
|
||||
"""
|
||||
import select as _select
|
||||
import signal as _signal
|
||||
import time as _time
|
||||
|
||||
stdin_r, stdin_w = os.pipe()
|
||||
stdout_r, stdout_w = os.pipe()
|
||||
stderr_r, stderr_w = os.pipe()
|
||||
|
||||
try:
|
||||
executable = shutil.which(cmd[0]) or cmd[0]
|
||||
pid = os.posix_spawn(
|
||||
executable, cmd, os.environ,
|
||||
file_actions=[
|
||||
(os.POSIX_SPAWN_DUP2, stdin_r, 0),
|
||||
(os.POSIX_SPAWN_DUP2, stdout_w, 1),
|
||||
(os.POSIX_SPAWN_DUP2, stderr_w, 2),
|
||||
(os.POSIX_SPAWN_CLOSE, stdin_r),
|
||||
(os.POSIX_SPAWN_CLOSE, stdin_w),
|
||||
(os.POSIX_SPAWN_CLOSE, stdout_w),
|
||||
(os.POSIX_SPAWN_CLOSE, stderr_w),
|
||||
(os.POSIX_SPAWN_CLOSE, stdout_r),
|
||||
(os.POSIX_SPAWN_CLOSE, stderr_r),
|
||||
],
|
||||
)
|
||||
except Exception:
|
||||
for fd in (stdin_r, stdin_w, stdout_r, stdout_w, stderr_r, stderr_w):
|
||||
try:
|
||||
os.close(fd)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
for fd in (stdin_r, stdout_w, stderr_w):
|
||||
os.close(fd)
|
||||
|
||||
try:
|
||||
if input_data:
|
||||
os.write(stdin_w, input_data)
|
||||
finally:
|
||||
os.close(stdin_w)
|
||||
|
||||
out, err = [], []
|
||||
done = set()
|
||||
deadline = _time.monotonic() + timeout
|
||||
try:
|
||||
while len(done) < 2:
|
||||
remaining = deadline - _time.monotonic()
|
||||
if remaining <= 0:
|
||||
try:
|
||||
os.kill(pid, _signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
break
|
||||
fds = [fd for fd in (stdout_r, stderr_r) if fd not in done]
|
||||
readable, _, _ = _select.select(fds, [], [], min(remaining, 0.5))
|
||||
for fd in readable:
|
||||
data = os.read(fd, 8192)
|
||||
if data:
|
||||
(out if fd == stdout_r else err).append(data)
|
||||
else:
|
||||
done.add(fd)
|
||||
finally:
|
||||
for fd in (stdout_r, stderr_r):
|
||||
try:
|
||||
os.close(fd)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
deadline = _time.monotonic() + 5.0
|
||||
while _time.monotonic() < deadline:
|
||||
try:
|
||||
wpid, st = os.waitpid(pid, os.WNOHANG)
|
||||
except ChildProcessError:
|
||||
return -1, b"".join(out), b"".join(err)
|
||||
if wpid == pid:
|
||||
if os.WIFEXITED(st):
|
||||
return os.WEXITSTATUS(st), b"".join(out), b"".join(err)
|
||||
if os.WIFSIGNALED(st):
|
||||
return -os.WTERMSIG(st), b"".join(out), b"".join(err)
|
||||
return -1, b"".join(out), b"".join(err)
|
||||
_time.sleep(0.01)
|
||||
return -1, b"".join(out), b"".join(err)
|
||||
|
||||
|
||||
def _verify_manifest_signature(manifest_obj, signature_armored, public_key_text=None):
|
||||
"""Verify a detached GPG signature over the canonical manifest JSON.
|
||||
|
||||
|
|
@ -597,7 +708,7 @@ def _verify_manifest_signature(manifest_obj, signature_armored, public_key_text=
|
|||
repos). When *None* the bundled official key is used instead.
|
||||
|
||||
Returns True if valid, False if invalid/error, None if verification
|
||||
could not be attempted (no signature, no key, gnupg missing, etc.).
|
||||
could not be attempted (no signature, no key, gpg binary missing, etc.).
|
||||
"""
|
||||
if not signature_armored:
|
||||
return None
|
||||
|
|
@ -614,18 +725,19 @@ def _verify_manifest_signature(manifest_obj, signature_armored, public_key_text=
|
|||
logger.debug("No GPG public key available; skipping verification")
|
||||
return None
|
||||
|
||||
try:
|
||||
import gnupg
|
||||
except ImportError:
|
||||
logger.debug("python-gnupg not installed; skipping signature verification")
|
||||
return None
|
||||
|
||||
tmp_home = tempfile.mkdtemp(prefix="gpg_verify_")
|
||||
try:
|
||||
gpg = gnupg.GPG(gnupghome=tmp_home)
|
||||
import_result = gpg.import_keys(key_text)
|
||||
if not import_result.fingerprints:
|
||||
logger.warning("Failed to import GPG public key")
|
||||
key_bytes = key_text.encode("utf-8") if isinstance(key_text, str) else key_text
|
||||
rc, _, import_stderr = _gpg_run(
|
||||
["gpg", "--batch", "--no-tty", "--status-fd", "2",
|
||||
"--homedir", tmp_home, "--import"],
|
||||
input_data=key_bytes,
|
||||
)
|
||||
if logger.isEnabledFor(logging.DEBUG):
|
||||
for line in import_stderr.decode("utf-8", errors="replace").splitlines():
|
||||
logger.debug("gpg import: %s", line)
|
||||
if rc != 0:
|
||||
logger.warning("GPG key import failed (rc=%d)", rc)
|
||||
return None
|
||||
|
||||
# Must match what the signing script produces: jq -c '.manifest'
|
||||
|
|
@ -639,8 +751,18 @@ def _verify_manifest_signature(manifest_obj, signature_armored, public_key_text=
|
|||
with open(sig_path, "w") as sf:
|
||||
sf.write(signature_armored)
|
||||
|
||||
verified = gpg.verify_data(sig_path, manifest_bytes)
|
||||
return bool(verified)
|
||||
rc, _, verify_stderr = _gpg_run(
|
||||
["gpg", "--batch", "--no-tty", "--status-fd", "2",
|
||||
"--homedir", tmp_home, "--verify", sig_path, "-"],
|
||||
input_data=manifest_bytes,
|
||||
)
|
||||
if logger.isEnabledFor(logging.DEBUG):
|
||||
for line in verify_stderr.decode("utf-8", errors="replace").splitlines():
|
||||
logger.debug("gpg verify: %s", line)
|
||||
return rc == 0 and b"VALIDSIG" in verify_stderr
|
||||
except FileNotFoundError:
|
||||
logger.debug("gpg binary not found; skipping signature verification")
|
||||
return None
|
||||
except Exception:
|
||||
logger.exception("GPG signature verification error")
|
||||
return False
|
||||
|
|
@ -881,6 +1003,7 @@ class PluginRepoRefreshAPIView(PluginAuthMixin, APIView):
|
|||
if err:
|
||||
return Response({"error": err}, status=status.HTTP_400_BAD_REQUEST)
|
||||
_unmanage_dropped_slugs(repo, data)
|
||||
_invalidate_plugin_detail_cache(repo.id, data)
|
||||
return Response(PluginRepoSerializer(repo).data)
|
||||
|
||||
|
||||
|
|
@ -1017,6 +1140,12 @@ class PluginDetailManifestAPIView(PluginAuthMixin, APIView):
|
|||
_validate_fetch_url(manifest_url)
|
||||
except ValueError as e:
|
||||
return Response({"error": str(e)}, status=status.HTTP_400_BAD_REQUEST)
|
||||
|
||||
cache_key = f"plugin_detail:{repo_id}:{hashlib.md5(manifest_url.encode()).hexdigest()}"
|
||||
cached = cache.get(cache_key)
|
||||
if cached is not None:
|
||||
return Response(cached)
|
||||
|
||||
try:
|
||||
resp = http_requests.get(manifest_url, timeout=MANIFEST_FETCH_TIMEOUT)
|
||||
resp.raise_for_status()
|
||||
|
|
@ -1045,10 +1174,12 @@ class PluginDetailManifestAPIView(PluginAuthMixin, APIView):
|
|||
if url_val and not url_val.startswith(("http://", "https://")):
|
||||
manifest_obj["latest"][url_field] = f"{root_url}/{url_val}"
|
||||
|
||||
return Response({
|
||||
result = {
|
||||
"manifest": manifest_obj,
|
||||
"signature_verified": verified,
|
||||
})
|
||||
}
|
||||
cache.set(cache_key, result, PLUGIN_DETAIL_CACHE_TTL)
|
||||
return Response(result)
|
||||
except Exception as e:
|
||||
logger.exception("Failed to fetch plugin manifest from %s", manifest_url)
|
||||
return Response(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue