Mirrors/Dispatcharr
1
0
Fork 1
mirror of https://github.com/Dispatcharr/Dispatcharr.git synced 2026-07-18 09:06:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Merge branch 'Dispatcharr:main' into feature-819-UserTableUpdate

Browse source
This commit is contained in:
Damien 2026-04-18 11:45:47 +01:00 • committed by GitHub
commit f403b5c97a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
175 changed files with 24103 additions and 6627 deletions
Download patch file Download diff file Expand all files Collapse all files

166
CHANGELOG.md
View file

@ -7,6 +7,172 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.23.0] - 2026-04-17
### Security
- Set `DEFAULT_PERMISSION_CLASSES` to `IsAdmin` in the DRF configuration. All viewsets and function-based views that require non-admin or unauthenticated access were explicitly annotated: proxy streaming endpoints (`stream_ts`, `stream_xc`, `stream_vod`, `head_vod`, `stream_xc_movie`, `stream_xc_episode`) use `@permission_classes([AllowAny])` (access is controlled by the per-stream-type network allow-list inside the view body); the `UserAgentViewSet`, `StreamProfileViewSet`, `CoreSettingsViewSet`, and `ProxySettingsViewSet` gained `get_permissions()` methods mapping read actions to `IsStandardUser` and write actions to `IsAdmin`; and `AuthViewSet.logout` was updated to return `[Authenticated()]`.
- Fixed missing `network_access_allowed` checks in the VOD proxy. `stream_vod`, `head_vod`, `stream_xc_movie`, and `stream_xc_episode` were not checking the `STREAMS` network policy, unlike the equivalent TS proxy endpoints.
- Explicitly marked the HDHomeRun discovery endpoints (`DiscoverAPIView`, `LineupAPIView`, `LineupStatusAPIView`, `HDHRDeviceXMLAPIView`) and the version endpoint with `permission_classes = [AllowAny]` to document their intentionally public access now that the global default is `IsAdmin`.
- Fixed path traversal vulnerability in file uploads. The M3U account upload (`apps/m3u/api_views.py`), logo upload (`apps/channels/api_views.py`), and backup upload (`apps/backups/api_views.py`) all used the uploaded filename directly without sanitization. `os.path.join()` discards all preceding components when it encounters an absolute path segment, and `pathlib`'s `/` operator behaves identically; a relative `../` sequence also escapes via OS path resolution at `open()` time. All three upload paths now strip directory components via `Path(name).name` and validate the resolved path remains within the intended upload directory. Exploiting any of these required admin credentials.
- Prevented users from setting `xc_password` (and other admin-managed keys) on their own account via the `PATCH /api/accounts/users/me/` endpoint.
- Hardened the HLS proxy `change_stream` endpoint by converting it from a plain Django view to a DRF `@api_view` with `@permission_classes([IsAdmin])`, ensuring the endpoint actually enforces admin-only access. The previous decorator arrangement (`@csrf_exempt` + `@permission_classes`) had no effect on a plain Django view.
- Added rate limiting to the login endpoint (`POST /api/accounts/token/`) using DRF's built-in throttling. A `LoginRateThrottle` (3 requests/minute per IP, sliding window) is applied to the `TokenObtainPairView`. Repeated failed attempts from the same IP receive `429 Too Many Requests`.
- Extended rate limiting to the session-auth login alias (`POST /api/accounts/auth/login/`). It now delegates entirely to `TokenObtainPairView`, inheriting its throttle, network access check, and audit logging, and returns JWT tokens instead of a session cookie (the session-based response was unusable since `SessionAuthentication` is not in `DEFAULT_AUTHENTICATION_CLASSES`). Both endpoints share the same `"login"` throttle scope, so attempts across either path count against the same per-IP limit.
- Removed `CORS_ALLOW_CREDENTIALS = True` from CORS configuration. Dispatcharr authenticates via JWT `Authorization` headers and API keys — not cookies — so credentials are never sent cross-origin by browsers. The setting was also redundant: browsers reject `Access-Control-Allow-Credentials: true` when `Access-Control-Allow-Origin` is a wildcard (`*`), so it had no effect in practice.
- Updated frontend npm dependencies to resolve 6 audit vulnerabilities (6 high):
- Updated `@xmldom/xmldom` 0.8.11 → 0.8.12, resolving **high** XML injection via unsafe CDATA serialization allowing attacker-controlled markup insertion ([GHSA-wh4c-j3r5-mjhp](https://github.com/advisories/GHSA-wh4c-j3r5-mjhp))
- Updated `lodash` 4.17.23 → 4.18.1, resolving **high** Code Injection via `_.template` imports key names ([GHSA-r5fr-rjxr-66jc](https://github.com/advisories/GHSA-r5fr-rjxr-66jc)) and **high** Prototype Pollution via array path bypass in `_.unset` and `_.omit` ([GHSA-f23m-r3pf-42rh](https://github.com/advisories/GHSA-f23m-r3pf-42rh))
- Updated `vite` 7.3.1 → 7.3.2, resolving **high** Path Traversal in optimized deps `.map` handling ([GHSA-4w7w-66w2-5vf9](https://github.com/advisories/GHSA-4w7w-66w2-5vf9)), **high** `server.fs.deny` bypass with queries ([GHSA-v2wj-q39q-566r](https://github.com/advisories/GHSA-v2wj-q39q-566r)), and **high** Arbitrary File Read via dev server WebSocket ([GHSA-p9ff-h696-f583](https://github.com/advisories/GHSA-p9ff-h696-f583))
- Updated `Django` 6.0.3 → 6.0.4, resolving the following CVEs:
- **CVE-2026-33033**: Potential DoS via `MultiPartParser` through crafted multipart uploads.
- **CVE-2026-33034**: SGI requests with a missing or understated `Content-Length` header could bypass the `DATA_UPLOAD_MAX_MEMORY_SIZE` limit.
- **CVE-2026-4292**: Privilege abuse in `ModelAdmin.list_editable`.
- **CVE-2026-3902**: ASGI header spoofing via underscore/hyphen conflation.
- **CVE-2026-4277**: Privilege abuse in `GenericInlineModelAdmin`.
### Added
- **EPG historical data window**: the EPG XML output and XC EPG API now support a `prev_days` URL parameter (e.g. `&prev_days=3`) to include past programs in the EPG response. This allows third-party players that request historical program schedules to receive the data they need. The EPG URL builder in the Channels page exposes "Days forward" and "Days back" controls. Per-user defaults for both values (`epg_days` / `epg_prev_days`) can be configured in the User settings modal and are applied automatically when no URL parameter is present. (Closes #1154)
- **Plugin Hub**: administrators can now browse, install, and update plugins directly from remote repositories via a new Plugin Hub page in Settings. (Closes #393) — Thanks [@sethwv](https://github.com/sethwv)
- Install plugins directly from the hub: the release zip is downloaded, SHA256 integrity is verified, and the plugin is installed atomically.
- Update managed plugins when a newer version is available from their source repo. Version compatibility constraints (`min_dispatcharr_version` / `max_dispatcharr_version`) are enforced at install time.
- Browse available plugins from all enabled repos with name, description, version, author, and icon.
- Plugins installed from a repo are tracked as "managed": source repo, slug, installed version, prerelease flag, and deprecated status are all persisted and surfaced in the UI.
- Add plugin repositories by manifest URL. The official Dispatcharr Plugins repository is pre-configured; third-party repos are supported by supplying an optional GPG public key.
- Manifest signatures are verified via GPG; the official repo uses a bundled public key. Signature status is displayed per-repo.
- Preview a repository URL before adding it - validates the manifest and reports plugin count and signature status without saving anything.
- Configurable automatic manifest refresh interval (in hours; 0 to disable) runs as a Celery background task.
### Removed
- Removed dead `VODConnectionManager` class (`apps/proxy/vod_proxy/connection_manager.py`) and its associated helpers, which had been superseded by `MultiWorkerVODConnectionManager`. All active code already used the multi-worker implementation. Removed the unused `VODConnectionManager` import from `vod_proxy/views.py`, the unscheduled `cleanup_vod_connections` task from `apps/proxy/tasks.py`, and the unscheduled `cleanup_vod_persistent_connections` task from `core/tasks.py`.
- Removed dead VOD URL routes: `VODPlaylistView` (playlist generation), `VODPositionView` (position tracking), and the class-based `VODStatsView` (replaced by the existing function-based `vod_stats` view).
- Removed dead `updateVODPosition()` API method from `frontend/src/api.js`, which called the now-removed position tracking endpoint.
### Fixed
- Fixed TV Guide "Record One" always scheduling the recording on the first channel that matched the program's `tvg_id`, rather than the channel the user actually selected. When multiple channels share the same EPG source, the intended channel was silently ignored. The selected channel object is now passed explicitly through the click handler chain to `recordOne`, bypassing the `findChannelByTvgId` fallback lookup entirely. (Fixes #1140) — Thanks [@fezster](https://github.com/fezster)
- Graceful container shutdown: `docker stop` no longer results in exit 137 (SIGKILL). The entrypoint now explicitly stops all child processes — including uWSGI workers, Celery, Daphne, and Redis, which are spawned as uWSGI `attach-daemon` children and were previously invisible to the signal handler. A polling loop replaces the old fixed `sleep`, exiting as soon as all processes have stopped (up to an 8-second ceiling before force-stopping). PostgreSQL is stopped using `pg_ctl stop -m immediate` as a fallback rather than SIGKILL to avoid data corruption. Process names are now recorded at startup and displayed correctly in crash diagnostics. The unexpected-exit diagnostic block is now suppressed on normal `docker stop` shutdowns. — Thanks [@Shokkstokk](https://github.com/Shokkstokk) for the initial fix!
- Fixed two race conditions in the VOD proxy that caused the `profile_connections` counter to go permanently negative, allowing connections beyond the configured profile limit. (1) `_decrement_profile_connections()` used a GET-before-DECR guard: two concurrent decrements could both read the same positive value, both pass the guard, and both fire, driving the counter below zero. Replaced with an unconditional `DECR` followed by a clamp-to-zero if the result is negative. (2) The `stream_generator` decremented `active_streams` and then checked `has_active_streams()` in two separate Redis round-trips without locking. A concurrent generator on another worker could read `active_streams=0` in the window between those two calls and also decrement the profile counter, producing a double-decrement. A new `decrement_active_streams_and_check()` method performs both operations under a single distributed lock, and a `profile_decremented` flag guards all four call sites in the generator so the profile counter is only ever decremented once per stream. (Closes #1125) — Thanks [@firestaerter3](https://github.com/firestaerter3)
- Fixed a provider TCP connection leak in the VOD proxy `stream_generator`. When a stream ended via an unhandled exception path that reached the `finally` block without any of the three exception handlers having run (e.g. an error raised before the first `yield`), the `finally` block decremented counters but never called `redis_connection.cleanup()`. The upstream `requests.Response` and `requests.Session` were left open until garbage collection. The `finally` block now starts a `delayed_cleanup` daemon thread (matching the 1-second delay used by the normal-completion and `GeneratorExit` paths) so that seeking clients have time to reconnect and increment `active_streams` before `cleanup()` checks whether it is safe to close the connection.
- Fixed manual stream selection from the Stats page not enforcing M3U profile connection limits in multi-worker deployments. When a non-owning worker handled the `change_stream` request it correctly packaged `stream_id` and `m3u_profile_id` into the Redis pubsub message, but the owning worker's pubsub handler only consumed `url` and `user_agent` silently dropping both IDs before calling `stream_manager.update_url()`. Because `update_url` only calls `update_stream_profile()` when a `stream_id` is provided, the `profile_connections` counter was never updated after the switch, causing subsequent capacity checks to see incorrect counts and bypass the full-profile guard. The handler now extracts `stream_id` and `m3u_profile_id` from the event and forwards them to `update_url()`. The bug did not affect single-worker / dev-mode deployments because the owning worker handles those requests directly without pubsub.
- Fixed the `next_stream` rotation endpoint applying the same class of bug: `get_stream_info_for_switch()` was called and returned `m3u_profile_id`, but the result was dropped when forwarding to `ChannelService.change_stream_url()`, so `update_stream_profile()` was never called and `profile_connections` counters were not updated after an automatic stream rotation.
- Fixed stream switch metadata (`url`, `user_agent`, `stream_id`, `m3u_profile`) being written to Redis before the switch was confirmed to succeed. If the switch failed, URL unchanged or exception during teardown, Redis described a URL not actually in use. Metadata is now written only after `update_url()` returns `True`; on failure the owner writes `stream_manager.url` back as the ground truth. The non-owner no longer pre-writes metadata at all, all needed info is carried in the pubsub payload and written by the owner after confirmation.
- Fixed the Stats page "Active Stream" dropdown not updating when a stream switch occurs. The card was matching the active stream by comparing the URL stored in Redis against stream URLs from the database, which failed silently when the stored URL was a transformed/rewritten value that didn't substring-match the original. The dropdown now matches by `stream_id` (the authoritative value already present in the stats payload) and re-runs only when `stream_id` changes, so the normal polling interval drives updates with no extra renders.
- Fixed the XC Password field in the User modal being editable by standard users despite the backend (`PATCH /api/accounts/users/me/`) stripping `xc_password` from `custom_properties` for non-admin users, causing the change to silently revert on save. The field and its generate button are now disabled with an explanatory description when the current user is not an administrator.
- Fixed live stream hiccups caused by nginx buffering TS proxy data to disk. The `/proxy/` location block used `proxy_buffering off` and `proxy_read/send_timeout` directives, which are silently ignored when the upstream is `uwsgi_pass` (a different directive family). nginx was therefore defaulting to `uwsgi_buffering on`, spooling stream data through temp files on disk. Replaced with the correct `uwsgi_buffering off`, `uwsgi_read_timeout 300s`, and `uwsgi_send_timeout 300s` directives so stream data flows directly from uWSGI to the client socket without intermediate disk I/O.
- Fixed the logo cache endpoint (`/api/channels/logos/{id}/cache/`) holding a uWSGI greenlet indefinitely when fetching from a slow or dripping remote server. The previous implementation used `StreamingHttpResponse(iter_content())` with only a per-chunk read timeout; a server that drips data just fast enough to reset the per-read timer could hold the greenlet open forever. Replaced with an eager read loop enforcing a hard total-download deadline (10 s) and a size cap (5 MB). Also fixed a race condition in the existing negative-cache logic: the failure entry for a URL was cleared immediately upon receiving HTTP 200, before the body was read. A concurrent greenlet seeing no failure entry during a slow download that ultimately timed out would also attempt the fetch, defeating the cache. The entry is now cleared only after the full body has been successfully received.
- Fixed uploading a local M3U file with no expiration date set sending the string `"null"` as the `exp_date` field in the `FormData` request, causing a 400 validation error from the API. Null/undefined values are now skipped when building the `FormData` body, matching the behaviour already present in the update path.
- Fixed `PATCH /api/channels/channels/edit/bulk/` returning a 500 error when the request body included a `streams` list. The bulk edit handler was iterating `validated_data` directly and calling `setattr(channel, "streams", value)`, which Django prohibits on ManyToMany fields. Also added an `@extend_schema` decorator so the Swagger UI correctly documents the endpoint as accepting a JSON array and shows the `streams` field. (Fixes #883)
- Fixed several incorrect or incomplete OpenAPI (`@extend_schema`) schemas across the API:
- `POST /api/epg/import/` — request body was undocumented; now correctly shows the `id` field. Description updated from "import" to "refresh" to match frontend and backend terminology.
- `DELETE /api/channels/logos/bulk-delete/` — `delete_files` boolean was missing from the documented request body.
- `POST /api/channels/channels/batch-set-epg/` — `epg_data_id` inside each association object was not marked `allow_null`/`required=False`, even though passing `null` is the correct way to remove an EPG link.
- `PUT /api/connect/integrations/{id}/subscriptions/set/` — endpoint had no `@extend_schema` at all; now documents that the request body is a JSON array of subscription objects.
### Changed
- **Output bitrate DB persistence**: the `ffmpeg_output_bitrate` stat is no longer written to the database on every FFmpeg stats tick (~2/second). Instead, a local exponential moving average (EMA, α=0.1) accumulates readings continuously. The first 10 samples (~5 seconds) are discarded as warmup to avoid polluting the average with FFmpeg's unstable ramp-up values. After warmup, the smoothed value is flushed to the database at most once every 30 seconds, and a final flush occurs when the stream stops but only if the EMA has been seeded (i.e. the stream ran past warmup). Streams that stop during warmup leave the existing database value untouched, preserving previously accurate measurements when channel-hopping.
- Performance: `generate_m3u`, `generate_epg`, and `xc_get_live_streams` now use `select_related('channel_group', 'logo')` (or `select_related('logo')` for EPG) on every Channel queryset in `apps/output/views.py`. Previously each channel in the loop triggered a separate database query for its `logo` and `channel_group` foreign keys; with the JOIN-based prefetch this is reduced to a single query per request. On deployments with ~2 000 channels, `xc_get_live_streams` response time drops from ~2.5–4 s to ~250–450 ms. (Closes #1127) — Thanks [@xBOBxSAGETx](https://github.com/xBOBxSAGETx)
- Performance: `generate_epg` now uses `select_related('epg_data__epg_source')` on all EPG channel querysets, eliminating N+1 database queries for `EPGSource` traversal per channel (~15 s improvement on ~2000-channel deployments; total EPG generation time dropped from ~87 s to ~72 s in benchmarks).
- Performance: `xc_get_epg` now uses `select_related('epg_data__epg_source')` on all three channel fetch paths. Previously each request triggered 2 additional queries to resolve `channel.epg_data` and `channel.epg_data.epg_source`.
- Performance: `generate_m3u` now uses `prefetch_related` for streams when `?direct=true` is requested, eliminating N+1 stream queries (one per channel) on that code path.
- Performance: `EPGGridAPIView` (`apps/epg/api_views.py`) now uses `select_related('epg_data__epg_source')` on the `channels_with_custom_dummy` queryset, eliminating 2 extra queries per channel (for `epg_data` and `epg_source`) in the dummy EPG generation loop.
- Performance: `generate_epg` now issues a single cross-channel `ProgramData` bulk query. `.values()` returns plain dicts, bypassing per-row Django model instantiation. Results are consumed in independent 5000-row keyset-paginated chunks. Combined with the `select_related` improvements above, EPG generation time on large deployments is significantly reduced.
- Performance: `xc_get_live_streams` no longer calls `ChannelGroup.objects.get_or_create(name="Default Group")` once per null-group channel; replaced with a lazy-initialised closure that executes at most one query regardless of how many ungrouped channels are present.
- AIO containers now connect to the internal PostgreSQL instance via a Unix domain socket instead of TCP loopback. Users who have `POSTGRES_HOST` explicitly set to `localhost` or `127.0.0.1` in their compose file are automatically migrated to the socket path; any other explicit value (external host/IP) is left untouched. — Thanks [@JCBird1012](https://github.com/JCBird1012)
- Improved the EPG response cache key. Previously it was based on the raw query string and username, meaning a user default of `epg_days=7` and an explicit `&days=7` URL parameter produced different cache entries for identical output. The key is now built from all resolved effective parameter values (`days`, `prev_days`, `cachedlogos`, `tvg_id_source`) so semantically equivalent requests always share the same cache entry.
- Improved the HDHR, M3U, and EPG URL builder popovers in the Channels table: each popover now opens with a brief intro sentence describing its purpose. Toggle switches were refactored to use Mantine's native `label` and `description` props (replacing the previous manual `Group`/`Stack`/`Text` layout), giving each switch a properly styled description line beneath its label. Switch alignment was also corrected. Toggles now appear on the left with the label and description stacked to the right, consistent with standard Mantine form layout.
- Redesigned the User settings modal with a tabbed layout: **Account** (username, email, name, password), **Permissions** (user level, stream limit, channel profiles, mature content filter - admin only), **EPG Defaults** (days forward/back), and **API & XC** (XC password, API key management). Fields are now logically grouped rather than split across two ad-hoc columns.
- EPG channel scanning now automatically removes stale `EPGData` entries. tvg-ids that were present in a previous scan but are no longer found in the upstream source, provided they are not mapped to any channel. This prevents unbounded database bloat over time. Entries mapped to at least one channel are always preserved.
- Rewrote the M3U line parser as an `iter_m3u_entries` generator that owns the full per-entry state machine. Intermediate directive lines between `#EXTINF` and the stream URL are now handled correctly rather than corrupting the pending entry or being silently misassigned. A `#EXTINF` with no following URL is discarded with a warning instead of carrying over a `url`-less entry into batch processing. Attribute keys are normalised to lowercase during parsing (provider attribute names remain case-insensitive end-to-end). The `#EXTINF` attribute regex is pre-compiled at module load, and attribute lookups use O(1) `dict.get()` instead of linear scans — approximately 10% faster parsing on large M3U files.
- Added support for the `#EXTGRP` directive in M3U files. When a `group-title` attribute is absent from the `#EXTINF` line, the value from a following `#EXTGRP:` line is used as the group. An explicit `group-title` attribute always takes priority. (Closes #1088)
- Added accumulation of `#EXTVLCOPT` directives per entry. Options are stored as a list under `vlc_opts` inside the stream's `custom_properties`, available for downstream use (e.g. passing VLC-specific options to the player). This is for a planned future enhancement and can also be utlized with the API.
- M3U stream name parsing now uses the comma text (the canonical display title per the base `#EXTINF` spec) as the primary stream name, falling back to `tvc-guide-title`, then `tvg-name`, rather than preferring `tvg-name` first. Providers that use `tvg-name` as an EPG key and put the human-readable title after the comma will now display the correct name. Providers that duplicate the same value in both fields are unaffected. (Fixes #1081)
- FloatingVideo player: the native video controls (timeline, play/pause, volume) are now hidden by default when a live stream starts and only appear when the user hovers over the player.
- Enhanced Swagger UI authorization dialog: registered a custom `OpenApiAuthenticationExtension` for `ApiKeyAuthentication` so drf-spectacular now generates an `ApiKeyAuth (apiKey)` entry alongside `jwtAuth`. Both entries include descriptive text linking to the relevant endpoints (`/api/accounts/token/`, `/api/accounts/api-keys/generate/`, `/api/accounts/api-keys/revoke/`).
- Refactored frontend form components (`AccountInfoModal`, `AssignChannelNumbers`, `Channel`, `ChannelBatch`, `ChannelGroup`, `Connection`, `CronBuilder`, `DummyEPG`, and `EPG`) to extract business logic into dedicated utility modules under `src/utils/forms/`. Each extracted module is covered by unit tests. Mantine compound component references (`Table.Tbody`, `Popover.Target`, `Accordion.Item`, etc.) have been updated to use flat named imports. — Thanks [@nick4810](https://github.com/nick4810)
- Improved the EPG BOM fix from v0.22.1: replaced the `lstrip(b'\xef\xbb\xbf')` / `startswith` approach with `start.find(b'<?xml')`, which locates the XML declaration regardless of any leading bytes BOM, whitespace, or other encoding markers without needing to know what those bytes are.
- Dependency updates:
- `Django` 6.0.3 → 6.0.4 (security patch; see Security section)
- `djangorestframework` 3.16.1 → 3.17.1
- `requests` 2.33.0 → 2.33.1
- `gevent` 25.9.1 → 26.4.0
- `rapidfuzz` 3.14.3 → 3.14.5
- `sentence-transformers` 5.3.0 → 5.4.0
- `lxml` 6.0.2 → 6.0.3
- Added `python-gnupg` for GPG signature verification of official and third-party plugin repository manifests.
## [0.22.1] - 2026-04-05
### Fixed
- Fixed EPG sources that emit a UTF-8 BOM (e.g. ErsatzTV, EPGShare, WebGrab+Plus) parsing 0 channels and 0 programmes after the HTML entity fix introduced in v0.22.0. `bytes.lstrip()` only strips ASCII whitespace, leaving the three BOM bytes (`EF BB BF`) in place, so `stripped.startswith(b'<?xml')` returned `False`. The function fell through to the no-declaration branch and prepended the HTML entity DOCTYPE block _before_ the BOM and XML declaration, producing invalid XML that lxml silently discarded under `recover=True`. Fixed by stripping the BOM explicitly before the whitespace strip: `start.lstrip(b'\xef\xbb\xbf').lstrip()`. BOM-free files are unaffected. (Closes #1173) — Thanks [@dwot](https://github.com/dwot) for the fix!
## [0.22.0] - 2026-04-01
### Security
- Updated `requests` 2.32.5 → 2.33.0, resolving the following CVE:
- **CVE-2026-25645** (moderate): Insecure temp file reuse in `extract_zipped_paths()` utility function.
- Updated frontend npm dependencies to resolve 4 audit vulnerabilities (2 moderate, 2 high):
- Updated `brace-expansion` 5.0.2 → 5.0.5, resolving **moderate** zero-step sequence causing process hang and memory exhaustion ([GHSA-f886-m6hf-6m8v](https://github.com/advisories/GHSA-f886-m6hf-6m8v))
- Updated `flatted` 3.4.1 → 3.4.2, resolving **high** Prototype Pollution via `parse()` in NodeJS flatted ([GHSA-rf6f-7fwh-wjgh](https://github.com/advisories/GHSA-rf6f-7fwh-wjgh))
- Updated `picomatch` 4.0.3 → 4.0.4, resolving **high** method injection in POSIX character classes causing incorrect glob matching ([GHSA-3v7f-55p6-f55p](https://github.com/advisories/GHSA-3v7f-55p6-f55p)) and a ReDoS vulnerability via extglob quantifiers ([GHSA-c2c7-rcm5-vvqj](https://github.com/advisories/GHSA-c2c7-rcm5-vvqj))
- Updated `yaml` 1.10.2 → 1.10.3, resolving **moderate** stack overflow via deeply nested YAML collections ([GHSA-48c2-rrv3-qjmp](https://github.com/advisories/GHSA-48c2-rrv3-qjmp))
### Added
- Connection cards on the Stats page now show the **username** of the connected user. For live channel connections a new User column appears between IP Address and Connected; for VOD connections the username is shown inline next to the IP address in the Client summary row. The username is resolved from the user store using the `user_id` stored in Redis client metadata. (Closes #766, Closes #586)
- `ip_address` and `user_id` were not included in the client info returned by `get_detailed_channel_info()` despite being available in the Redis hash. Both fields are now extracted and returned. `user_id` is now also included in the VOD stats response.
- Web UI stream preview now sends an `Authorization: Bearer` header with each mpegts.js request, identifying the logged-in user. Live channel previews initiated from the web UI now appear on the Stats page with the correct username rather than as unknown user.
- `client_connect` and `client_disconnect` system events now include the **username** of the connected user. The username is stored alongside the client metadata in Redis and included in the event payload for `log_system_event` calls (making it available to webhook and script integrations).
- Donate button added to the sidebar footer. A heart icon links to the project's Open Collective page, visible in both expanded and collapsed states. Hovering shows a "Support Dispatcharr" tooltip. The version string is also now clickable to copy it to the clipboard.
- User stream limits: administrators can now set a maximum number of concurrent streams per user account. When a user reaches their limit, the system can automatically terminate an existing stream to free a slot based on configurable rules. Limit enforcement applies to both live channels and VOD. (Closes #544)
- Each user account has a new **Stream Limit** field (0 = unlimited) configurable from the user edit form in Settings → Users.
- Global enforcement behaviour is configurable in Settings → User Limits:
- **Terminate on Limit Exceeded**: automatically stop an existing stream when the user's limit is reached (vs. rejecting the new connection).
- **Terminate Oldest**: prefer terminating the oldest stream when freeing a slot; disable to prefer the newest.
- **Prioritize Single-Client Channels**: prefer terminating streams on channels that only this user is watching.
- **Ignore Same-Channel Connections**: count multiple connections to the same live channel as one stream toward the limit. Same-channel reconnects are always allowed through. When this is enabled and a channel must be freed, all connections to the chosen channel are terminated together so that the unique-channel count actually decreases. VOD is explicitly excluded from this bypass since VOD connections are not shared upstream.
- TLS and mutual TLS (mTLS) support for Redis and PostgreSQL connections in modular deployments. Supports encrypted connections, server certificate verification (Redis: on/off; PostgreSQL: verify-full, verify-ca, require), CA certificate configuration, and client certificate authentication. Configured via environment variables in the docker compose file. Includes startup validation for certificate paths and TLS/URL scheme conflicts, and a read-only Connection Security panel in System Settings. (Closes #950) — Thanks [@CodeBormen](https://github.com/CodeBormen)
- Status filter for M3U group and VOD category filter modals: A new **All / Enabled / Disabled** segmented control is now shown alongside the text search input in the Live, VOD - Movies, and VOD - Series tabs of the M3U Group Filter modal. The status filter works in combination with the text search and also scopes the "Select Visible" / "Deselect Visible" buttons so they only act on the currently visible subset. (Closes #312)
### Changed
- M3U Profile form (XC accounts): added a **Simple / Advanced** mode toggle for credential-based URL rewriting. In Simple mode users enter just a new username and password; the search and replace patterns are built automatically from the account's current credentials. In Advanced mode the full regex fields are shown as before. The selected mode is saved to `custom_properties.xcMode` and auto-detected on existing profiles (a profile whose search pattern matches the account's current `username/password` is recognised as Simple automatically). The Live Regex Demonstration panel is hidden in Simple mode.
- XtreamCodes VOD endpoints (`/movie/` and `/series/`) no longer redirect clients to a UUID-based proxy URL. Requests are now handled directly in the proxy layer via `stream_xc_movie` and `stream_xc_episode`, which call `stream_vod()` internally. The original XC path is preserved for the client throughout the stream.
- `CustomTable` column layout now supports flexible (`grow`) columns alongside fixed-width ones:
- Column definitions accept a `grow` property (boolean or number) to opt into flex layout. A numeric value sets the flex-grow weight, allowing relative sizing between grow columns (e.g. `grow: 2` gives a column twice the share of spare space as `grow: 1`).
- `maxSize` is now respected on grow columns, capping how wide they expand via `maxWidth`.
- The wrapper's `minWidth` calculation now uses `minSize` (not TanStack's 150px default) for grow columns, preventing the table from overflowing its container when columns would otherwise be sized larger than available space.
- Dependency updates:
- `requests` 2.32.5 → 2.33.0 (security patch; see Security section)
- `celery` 5.6.2 → 5.6.3
- `torch` 2.10.0+cpu → 2.11.0+cpu
- `sentence-transformers` 5.2.3 → 5.3.0
- `yt-dlp` 2026.3.13 → 2026.3.17
- Docker base image cleanup: removed `python-is-python3`, `python3-pip`, and `streamlink` from the apt package list in `DispatcharrBase`. `python3-pip` and `streamlink` were pulling outdated system Python packages (e.g. `requests 2.31.0`, `cryptography 41.0.7`, `lxml 5.2.1`) into the system Python's site-packages despite the app running entirely in the uv-managed venv at `/dispatcharrpy`. `streamlink` is already installed in the venv via `pyproject.toml`. `python-is-python3` is unnecessary as `PATH` resolves bare `python` to the venv binary.
- M3U table **Max Streams** column now reflects the combined limit across all active profiles. When a playlist has multiple active profiles, the column displays their summed total (or ∞ if any profile is unlimited) and a hover tooltip lists each profile's individual limit by name. (Closes #816)
- Toggling an M3U profile's active state now immediately updates the playlist store (including the `playlists` array), so the **Max Streams** total in the M3U table reflects the change without a page reload.
- M3U account form: **Max Streams** field changed from a plain text input to a number input with increment/decrement controls, consistent with other integer fields.
- M3U account form: removed unused `useMantineTheme` import and `theme` variable.
- Moved `guideUtils.js` from `frontend/src/pages/` to `frontend/src/utils/` to be consistent with other utility modules (e.g. `networkUtils.js`). Updated all imports across `GuideRow.jsx`, `HourTimeline.jsx`, `ProgramDetailModal.jsx`, `RecordingCardUtils.js`, `Guide.jsx`, and related test files.
- Frontend cleanup: removed unused imports from `M3UGroupFilter`, `LiveGroupFilter`, and `VODCategoryFilter` (`Yup`, `M3UProfiles`, several unused Mantine components, dead `OptionWithTooltip` component, duplicate lucide-react imports, and `Divider` in `VODCategoryFilter`). No behaviour changes.
- Network Access settings: leaving a field blank no longer shows a validation error. The default CIDR range for that field is saved automatically and a "Defaults Restored" warning is displayed listing which fields were reset. (Closes #726)
### Fixed
- M3U profile URL rewriting now uses the `regex` module instead of `re` across all URL transform code paths (`url_utils.transform_url`, `core/views.py`, `vod_proxy/_transform_url`, `tasks.get_transformed_credentials`, and the WebSocket live-preview handler in `consumers.py`). The `regex` module natively accepts JavaScript/PCRE-style named capture groups (`(?<name>...)`) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a `re.error`. As a further improvement, `regex` also supports variable-length lookbehind assertions (e.g. `(?<=a+)`), which `re` rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling `regex.sub`: `$<name>` → `\g<name>` and `$1`/`$2`/… → `\1`/`\2`/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
- Web UI stream preview (`FloatingVideo`) was calling `mpegts.createPlayer()` with all `Config` options (e.g. `enableWorker`, `liveSync`, `headers`) merged into the first `MediaDataSource` argument. mpegts.js only reads `Config` from the optional second argument; unrecognised fields in the first are silently ignored. As a result all player configuration was effectively the library defaults — worker offloading was disabled, latency management had no effect, and the `Authorization: Bearer` header (required for user identification) was never sent. Fixed by splitting into the correct two-argument call. Both `liveBufferLatencyChasing` and `liveSync` have been disabled, eliminating playback-rate fluctuations that caused audible stuttering on live streams. SourceBuffer cleanup thresholds were also relaxed from 10s/5s to 120s/60s to prevent frequent SourceBuffer pauses.
- HTML named entities in XMLTV EPG files are now correctly preserved during lxml parsing. Some EPG providers (particularly French and other European sources) use HTML named entities like `&eacute;`, `&icirc;`, `&uuml;` in channel names, program titles, and metadata. These are not valid XML entities — lxml 6.0.2 with `recover=True` silently drops them, causing characters to go missing (e.g., "Chaîne Télé" becomes "Chane Tl"). This is now fixed by injecting an XML `<!DOCTYPE tv [...]>` internal subset declaring all 252 HTML 4 named entities directly into the byte stream that lxml reads, using a lightweight in-memory wrapper (`_PrependStream`) with zero disk I/O. libxml2 resolves the entities during its normal C-level parse pass — no Python-level preprocessing or temporary files are involved. The DOCTYPE block (~8 KB) is built once at module load from Python's stdlib `html.entities.name2codepoint` and reused for every parse. Files that already declare their own `<!DOCTYPE>` are passed through unchanged. (Closes #1095) — Thanks [@CodeBormen](https://github.com/CodeBormen) for helping with this!
- Duplicate recordings created when EPG sources refresh and re-evaluate series rules (Fixes #940) — Thanks [@CodeBormen](https://github.com/CodeBormen):
- **Program ID instability**: `parse_programs_for_source()` deletes and recreates all `ProgramData` rows with new auto-increment IDs on every EPG refresh. The dedup set used these IDs, so it never matched after a refresh. Deduplication now uses a stable `(tvg_id, start_time, end_time)` composite key sourced from `Recording.custom_properties.program`.
- **Secondary guard using wrong times**: The DB guard compared unadjusted program times against offset-adjusted `Recording.start_time`/`end_time`, so it never matched when any DVR pre/post offset was configured. It now queries `custom_properties__program__start_time/end_time` (the original, unadjusted program times stored at recording creation).
- **No concurrency guard**: Each EPG source refresh fired `evaluate_series_rules.delay()` independently. Concurrent tasks loaded the dedup set before others committed, allowing races. Evaluation is now serialized with `acquire_task_lock` (reusing the existing EPG task pattern). Gracefully degrades if Redis is unavailable — the primary and secondary dedup guards still protect.
- EPG refresh tasks (`refresh_epg_data`) were being killed mid-transaction on large EPG sources. The `soft_time_limit=1700s` introduced in v0.21.0 raised `SoftTimeLimitExceeded`, a subclass of `Exception`, which was swallowed by the existing `except Exception` handler in `parse_programs_for_source`, leaving the database in a partial state with no logged error. `soft_time_limit` has been removed from `refresh_epg_data` and `time_limit` raised to 14400s (4 hours) as a true last-resort ceiling; the existing `TaskLockRenewer` daemon thread continues to renew the Redis lock every 120s for legitimately long-running tasks.
## [0.21.1] - 2026-03-18
### Fixed

586
Plugin_repo.md Normal file
View file

@ -0,0 +1,586 @@
# Dispatcharr Plugin Repository Specification
How to create and host a plugin repository that Dispatcharr can consume.
For writing plugins themselves, see [Plugins.md](Plugins.md).
---
## Overview
Dispatcharr discovers plugins from remote repositories using a two-level manifest system:
1. **Repo manifest** - a JSON file listing all plugins in the repo with basic metadata.
2. **Per-plugin manifest** (optional) - a JSON file per plugin with full version history, checksums, and compatibility info.
Users add a repo by its manifest URL. Dispatcharr fetches and caches the repo manifest periodically (default: every 6 hours, configurable). The UI displays all plugins from enabled repos in a browsable store.
---
## Repo Manifest
The repo manifest is the entry point. Dispatcharr fetches this URL and caches the response.
### Minimal Example (no signing)
```json
{
"registry_name": "My Plugin Repo",
"plugins": [
{
"slug": "my_plugin",
"name": "My Plugin",
"description": "Does something useful",
"author": "Your Name",
"latest_version": "1.0.0",
"latest_url": "https://example.com/releases/my_plugin-1.0.0.zip"
}
]
}
```
This is the simplest valid repo manifest - one plugin with enough info to show in the store and install.
### Full Example (with signing)
```json
{
"manifest": {
"registry_name": "My Plugin Repo",
"registry_url": "https://github.com/myorg/my-plugins",
"root_url": "https://raw.githubusercontent.com/myorg/my-plugins/releases",
"plugins": [
{
"slug": "weather_display",
"name": "Weather Display",
"description": "Shows weather info on the dashboard",
"author": "Acme Labs",
"license": "MIT",
"latest_version": "1.2.5",
"last_updated": "2025-01-20T15:30:00Z",
"manifest_url": "plugins/weather_display/manifest.json",
"latest_url": "plugins/weather_display/releases/weather_display-1.2.5.zip",
"latest_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"icon_url": "plugins/weather_display/logo.png",
"min_dispatcharr_version": "2.5.0",
"max_dispatcharr_version": null
}
]
},
"signature": "-----BEGIN PGP SIGNATURE-----\n..."
}
```
### Accepted Formats
Dispatcharr accepts two top-level shapes:
**Wrapped (supports signing):**
```json
{
"manifest": { "plugins": [...], ... },
"signature": "..."
}
```
**Flat (no signing):**
```json
{
"plugins": [...],
"registry_name": "...",
"root_url": "..."
}
```
The wrapped format is required for signing. If you don't need signing, the flat format works and is simpler.
### Name Restrictions
`registry_name` is required. Dispatcharr rejects repos that are missing it.
Third-party repos must not use names that could be confused with an official Dispatcharr repo. The following words are blocked in `registry_name` (case-insensitive):
- "official"
- "dispatcharr plugins"
- "dispatcharr repo"
- "dispatcharr official"
If the name contains any of these, the repo will be rejected on add and skipped during refresh.
---
## Repo Manifest Fields
### Top-Level Metadata
| Field | Required | Description |
|-------|----------|-------------|
| `registry_name` | **Yes** | Display name for the repo. Must not contain words like "official" or "dispatcharr" that could be mistaken for an official repo (see [Name Restrictions](#name-restrictions)). |
| `registry_url` | No | URL to the repo's home page (e.g. GitHub). Used as a fallback for generating icon URLs. |
| `root_url` | No | Base URL for resolving relative URLs in plugin entries. Trailing slashes are stripped. |
| `plugins` | **Yes** | Array of plugin entry objects. |
### Plugin Entry Fields
| Field | Required | Description |
|-------|----------|-------------|
| `slug` | **Yes** | Unique identifier. Alphanumeric, dashes, and underscores. Used as the install directory name (lowercased, dashes converted to underscores). |
| `name` | **Yes** | Human-readable display name. |
| `description` | No | Short description shown on the plugin card. |
| `author` | No | Author or organization name. |
| `license` | No | SPDX license identifier (e.g. `MIT`, `GPL-3.0`). Displayed as a link to the SPDX license page. |
| `latest_version` | No | Current latest version string (semver: `1.2.3` or `v1.2.3`). Drives update detection. |
| `last_updated` | No | ISO 8601 timestamp of the latest release. Shown as "Built" date in the detail view. |
| `manifest_url` | No | URL (or relative path) to the per-plugin manifest with full version history. See [Per-Plugin Manifest](#per-plugin-manifest). |
| `latest_url` | No | Direct download URL (or relative path) to the latest release zip. |
| `latest_sha256` | No | SHA256 checksum of the latest release zip (lowercase hex, 64 chars). |
| `latest_md5` | No | MD5 checksum of the latest release zip. Informational only - not validated by Dispatcharr. |
| `icon_url` | No | URL (or relative path) to a logo image (PNG recommended). |
| `min_dispatcharr_version` | No | Minimum Dispatcharr version required. Install is blocked if the running version is older. |
| `max_dispatcharr_version` | No | Maximum Dispatcharr version supported. Install is blocked if the running version is newer. |
Extra fields in a plugin entry are passed through to the frontend as-is, so you can include custom metadata (e.g. `homepage`, `tags`) without breaking anything.
### URL Resolution
If `root_url` is set and a URL field (`manifest_url`, `latest_url`, `icon_url`) does not start with `http://` or `https://`, it is treated as relative and resolved as:
```
{root_url}/{field_value}
```
This lets you keep plugin entries compact:
```json
{
"root_url": "https://raw.githubusercontent.com/myorg/my-plugins/releases",
"plugins": [
{
"slug": "my_plugin",
"latest_url": "plugins/my_plugin/my_plugin-1.0.0.zip",
"icon_url": "plugins/my_plugin/logo.png",
"manifest_url": "plugins/my_plugin/manifest.json"
}
]
}
```
**Icon fallback:** If `icon_url` is missing and `registry_url` is set, Dispatcharr generates a fallback URL by converting the GitHub URL to a raw content URL:
```
{registry_url => raw.githubusercontent.com}/refs/heads/main/plugins/{slug}/logo.png
```
---
## Per-Plugin Manifest (Optional)
The per-plugin manifest provides full version history. It is fetched on-demand when a user clicks "More Info" on a plugin card. It is **not required** - if `manifest_url` is absent, the UI builds a detail view from the repo-level fields instead.
Include a per-plugin manifest if you want to:
- Offer multiple downloadable versions
- Show per-version compatibility ranges
- Display build timestamps and commit links for each version
- Provide detailed author/license info beyond what's in the repo manifest
### Accepted Formats
Same as the root manifest - both flat and wrapped formats are accepted:
**Flat (no signing):**
```json
{
"slug": "...",
"versions": [...]
}
```
**Wrapped (supports signing):**
```json
{
"manifest": {
"slug": "...",
"versions": [...]
},
"signature": "-----BEGIN PGP SIGNATURE-----\n..."
}
```
Use the wrapped format if you want to GPG-sign the per-plugin manifest.
### Example
```json
{
"slug": "weather_display",
"name": "Weather Display",
"description": "Shows weather information on the Dispatcharr dashboard",
"author": "Acme Labs",
"license": "MIT",
"latest_version": "1.2.5",
"versions": [
{
"version": "1.2.5",
"url": "releases/weather_display-1.2.5.zip",
"checksum_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"build_timestamp": "2025-01-20T15:30:00Z",
"commit_sha": "4e8f1b108c1e84f60520710d13e54eb2fb519648",
"commit_sha_short": "4e8f1b1",
"min_dispatcharr_version": "2.5.0",
"max_dispatcharr_version": null
},
{
"version": "1.2.5-rc.1",
"url": "releases/weather_display-1.2.5-rc.1.zip",
"checksum_sha256": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2",
"prerelease": true,
"build_timestamp": "2025-01-18T09:00:00Z",
"min_dispatcharr_version": "2.5.0"
},
{
"version": "1.2.4",
"url": "releases/weather_display-1.2.4.zip",
"checksum_sha256": "d4d967a67a4947e55183308cece206b30dda3e1b4fe00aae60f45a49c83b7ed6",
"build_timestamp": "2025-01-15T10:00:00Z",
"min_dispatcharr_version": "2.4.0"
}
],
"latest": {
"version": "1.2.5",
"url": "releases/weather_display-1.2.5.zip",
"checksum_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"build_timestamp": "2025-01-20T15:30:00Z",
"min_dispatcharr_version": "2.5.0"
}
}
```
### Per-Plugin Manifest Fields
| Field | Required | Description |
|-------|----------|-------------|
| `slug` | No | Plugin identifier (should match the repo entry). |
| `name` | No | Display name. |
| `description` | No | Full description shown in the detail modal. |
| `author` | No | Author/org name shown in the detail modal. |
| `license` | No | SPDX license identifier. |
| `latest_version` | No | Latest version string. |
| `versions` | No | Array of version objects (newest first recommended). |
| `latest` | No | Object mirroring the latest version entry for quick access. |
### Version Object Fields
| Field | Required | Description |
|-------|----------|-------------|
| `version` | **Yes** | Version string (`1.2.3` or `v1.2.3`). |
| `url` | **Yes** | Download URL for the zip. Relative URLs are resolved against the repo's `root_url`. |
| `checksum_sha256` | No | SHA256 hex checksum. **Strongly recommended.** Validated on install - mismatch blocks the install. |
| `prerelease` | No | Boolean. When `true`, marks this version as a pre-release (alpha, beta, RC, etc.). If the installed version is a prerelease, Dispatcharr will not suggest updating to the latest stable version - the user must install a new version manually. The latest version in the root manifest is always assumed to be stable, so this field only needs to appear in the per-plugin manifest. Omit or set to `false` for stable releases. |
| `build_timestamp` | No | ISO 8601 build timestamp. Shown as "Built" in the version detail. |
| `commit_sha` | No | Full Git commit SHA. Used to build a commit link if `registry_url` is set. |
| `commit_sha_short` | No | Abbreviated commit SHA. Displayed in the version detail table as a clickable link. |
| `min_dispatcharr_version` | No | Minimum compatible Dispatcharr version. |
| `max_dispatcharr_version` | No | Maximum compatible Dispatcharr version. |
Relative `url` values in versions are resolved the same way as repo-level URLs: `{root_url}/{url}`.
---
## Without a Per-Plugin Manifest
If you omit `manifest_url` from a plugin entry, the store still works. When a user clicks "More Info", the UI builds a detail view from the repo-level fields:
- `description`, `author`, `license` from the plugin entry
- A single version entry built from `latest_version`, `latest_url`, `latest_sha256`, `min_dispatcharr_version`, `max_dispatcharr_version`, and `last_updated`
This is the simplest path for third-party repos that only publish one version at a time. You lose version history and per-version release dates, but install, update detection, and everything else works the same.
---
## Signing
Signing your repo manifest lets Dispatcharr verify it hasn't been tampered with. Signing is **optional** - unsigned repos work fine but show an "unverified" badge in the UI.
### How It Works
1. You generate a GPG keypair.
2. You sign the manifest JSON and include the detached signature in the response.
3. When adding the repo in Dispatcharr, the user pastes your public key.
4. Dispatcharr verifies the signature on every manifest fetch.
### Key Format
Standard PGP/GPG armored keys:
```
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBG...
...
-----END PGP PUBLIC KEY BLOCK-----
```
### Signing Convention
The signature is computed over the **canonical JSON** representation of the `manifest` object (not the entire response), plus a trailing newline:
```bash
# Canonical format: compact JSON (no spaces) + trailing newline
jq -c '.manifest' manifest.json | gpg --armor --detach-sign
```
In code terms:
```python
import json
canonical = json.dumps(manifest_obj, separators=(",", ":")) + "\n"
```
> **Important:** The signing input must be `json.dumps(obj, separators=(",", ":")) + "\n"` - compact JSON with no whitespace, followed by exactly one newline. Any difference (pretty-printing, trailing spaces, key ordering changes) will cause verification to fail.
### Manifest Structure for Signing
Use the wrapped format so the signature sits alongside the manifest:
```json
{
"manifest": {
"registry_name": "...",
"plugins": [...]
},
"signature": "-----BEGIN PGP SIGNATURE-----\n...\n-----END PGP SIGNATURE-----"
}
```
### Verification Results
| Result | Meaning | UI Badge |
|--------|---------|----------|
| `true` | Valid signature | Green checkmark |
| `false` | Invalid signature or verification error | Red X |
| `null` | Not attempted (no signature, no key, or `python-gnupg` not installed) | Gray/neutral |
### Signing Workflow Example
```bash
# Generate a keypair (one-time)
gpg --gen-key
# Export your public key (give this to repo users)
gpg --armor --export "your@email.com" > my-repo.pub
# Build your manifest
cat > manifest.json << 'EOF'
{
"manifest": {
"registry_name": "My Repo",
"root_url": "https://example.com/releases",
"plugins": [
{
"slug": "my_plugin",
"name": "My Plugin",
"latest_version": "1.0.0",
"latest_url": "plugins/my_plugin/my_plugin-1.0.0.zip"
}
]
}
}
EOF
# Sign the manifest object (canonical JSON + newline)
jq -c '.manifest' manifest.json | gpg --armor --detach-sign > manifest.sig
# Combine into final output
jq --arg sig "$(cat manifest.sig)" '.signature = $sig' manifest.json > signed_manifest.json
```
### Third-Party Key Management
When a user adds your repo URL, they can paste your public key. Dispatcharr stores the key per-repo and uses it for verification. Users can update the key at any time from the repo management UI.
If you don't provide a key and the repo is not the official Dispatcharr repo, signature verification is skipped (result: `null`).
---
## Release Zip Format
Each plugin release is a `.zip` archive.
### Requirements
- Must contain a `plugin.py` with a `Plugin` class, **or** a Python package with `__init__.py` exporting a `Plugin` class.
- Files can be at the top level of the zip or inside a single subdirectory.
- Optionally include `plugin.json` for metadata discovery without code execution.
- Optionally include `logo.png` for the plugin icon.
### Size Limits
- Maximum 2000 files per archive.
- Maximum total size: 200 MB (configurable via `MAX_PLUGIN_IMPORT_BYTES` setting).
### Recommended Structure
```
my_plugin-1.0.0.zip
plugin.py
plugin.json
logo.png
(any other files your plugin needs)
```
Or with a subdirectory:
```
my_plugin-1.0.0.zip
my_plugin/
plugin.py
plugin.json
logo.png
utils.py
```
---
## Install Flow
When a user installs a plugin from the store:
1. **Version compatibility check** - if `min_dispatcharr_version` or `max_dispatcharr_version` is set, the running Dispatcharr version is compared. Install is blocked if out of range.
2. **Download** - the zip is streamed from `download_url` (max 200 MB).
3. **SHA256 integrity check** - if `sha256` was provided, the download is hashed and compared. Mismatch blocks the install.
4. **Extraction** - the zip is extracted to a temp directory, validated, then moved to `/data/plugins/{plugin_key}/`. If the plugin already exists, the old version is backed up and restored on failure (atomic rollback).
5. **Registration** - a `PluginConfig` record is created or updated, linking the plugin to its source repo and slug.
6. **Discovery reload** - the plugin loader re-scans all plugin directories.
The plugin is installed **disabled** by default. The user can enable it from the post-install dialog or the My Plugins page.
---
## Update Detection
Dispatcharr detects updates by comparing `installed_version` (stored in the database) against `latest_version` from the repo manifest. This uses repo-level fields only - per-plugin manifests are not needed for update detection.
A plugin shows "Update Available" when:
- It is managed (installed from a repo)
- Its `installed_version` differs from `latest_version`
- It was installed from the same repo
---
## Hosting Options
A plugin repo manifest is just a JSON file served over HTTPS. Some options:
### GitHub Pages / Raw Content
Host your manifest and release zips in a GitHub repo. Use raw.githubusercontent.com URLs:
```
https://raw.githubusercontent.com/myorg/my-plugins/main/manifest.json
```
Use `root_url` pointing to your releases branch/path so version URLs stay relative.
### Static File Server
Any web server that serves JSON works. Dispatcharr fetches manifests server-side, so CORS is not needed.
### GitHub Releases
You can host release zips as GitHub Release assets and reference them with absolute URLs in your manifest. The manifest itself can live in the repo's default branch.
---
## Refresh Behavior
- Manifests are refreshed automatically at a configurable interval (default: 6 hours, setting: `refresh_interval_hours`, 0 = disabled).
- Users can force a refresh from the repo management UI.
- A new repo is refreshed immediately when added.
- On refresh, if a plugin's `slug` disappears from the manifest, its `PluginConfig` is unlinked from the repo (becomes "unmanaged") but the installed files are not deleted.
---
## Checklist: Publishing a Plugin Repo
### Minimum Viable Repo
- [ ] Host a JSON file at a stable, public URL
- [ ] Set `registry_name` (required, must not sound official)
- [ ] Include at least one plugin entry with `slug`, `name`, and `latest_version`
- [ ] Host a downloadable `.zip` for each plugin and set `latest_url`
- [ ] Share the manifest URL with users
### Recommended
- [ ] Set `root_url` so plugin URLs can be relative
- [ ] Include `description`, `author`, and `icon_url` per plugin
- [ ] Include `latest_sha256` for integrity verification
- [ ] Include `license` (SPDX identifier)
- [ ] Include `last_updated` timestamps
- [ ] Add a per-plugin `manifest_url` with version history
- [ ] Include `sha256` in every version object
- [ ] Include `min_dispatcharr_version` where applicable
- [ ] Include `plugin.json` in each release zip
### Optional
- [ ] Sign your manifest with GPG and publish your public key
- [ ] Set `registry_url` to enable automatic icon fallback
- [ ] Set `max_dispatcharr_version` if a plugin is incompatible with newer releases
---
## Quick Reference: Repo Manifest Schema
```json
{
"manifest": {
"registry_name": "string (required)",
"registry_url": "string (optional)",
"root_url": "string (optional)",
"plugins": [
{
"slug": "string (required)",
"name": "string (required)",
"description": "string",
"author": "string",
"license": "string (SPDX)",
"latest_version": "string (semver)",
"last_updated": "string (ISO 8601)",
"manifest_url": "string (URL or relative path)",
"latest_url": "string (URL or relative path)",
"latest_sha256": "string (64-char hex)",
"icon_url": "string (URL or relative path)",
"min_dispatcharr_version": "string (semver)",
"max_dispatcharr_version": "string (semver) or null"
}
]
},
"signature": "string (armored PGP signature, optional)"
}
```
## Quick Reference: Per-Plugin Manifest Schema
```json
{
"slug": "string",
"name": "string",
"description": "string",
"author": "string",
"license": "string (SPDX)",
"latest_version": "string (semver)",
"versions": [
{
"version": "string (required)",
"url": "string (required, URL or relative path)",
"checksum_sha256": "string (64-char hex)",
"build_timestamp": "string (ISO 8601)",
"commit_sha": "string",
"commit_sha_short": "string",
"min_dispatcharr_version": "string (semver)",
"max_dispatcharr_version": "string (semver) or null"
}
],
"latest": {
"version": "string",
"url": "string",
"checksum_sha256": "string",
"build_timestamp": "string",
"min_dispatcharr_version": "string",
"max_dispatcharr_version": "string or null"
}
}
```

83
apps/accounts/api_views.py
View file

@ -6,6 +6,7 @@ from django.views.decorators.csrf import csrf_exempt
from rest_framework.decorators import api_view, permission_classes, action
from rest_framework.response import Response
from rest_framework import viewsets, status, serializers
from rest_framework.throttling import AnonRateThrottle
from drf_spectacular.utils import extend_schema, OpenApiParameter, inline_serializer
from drf_spectacular.types import OpenApiTypes
import json
@ -20,9 +21,14 @@ from rest_framework_simplejwt.views import TokenObtainPairView, TokenRefreshView
logger = logging.getLogger(__name__)
class LoginRateThrottle(AnonRateThrottle):
scope = "login"
class TokenObtainPairView(TokenObtainPairView):
throttle_classes = [LoginRateThrottle]
def post(self, request, *args, **kwargs):
# Custom logic here
if not network_access_allowed(request, "UI"):
# Log blocked login attempt due to network restrictions
from core.utils import log_system_event
@ -153,12 +159,11 @@ class AuthViewSet(viewsets.ViewSet):
Login doesn't require auth, but logout does
"""
if self.action == 'logout':
from rest_framework.permissions import IsAuthenticated
return [IsAuthenticated()]
return [Authenticated()]
return []
@extend_schema(
description="Authenticate and log in a user",
description="Alias for POST /api/accounts/token/ — returns JWT access and refresh tokens.",
request=inline_serializer(
name="LoginRequest",
fields={
@ -168,55 +173,10 @@ class AuthViewSet(viewsets.ViewSet):
),
)
def login(self, request):
"""Logs in a user and returns user details"""
username = request.data.get("username")
password = request.data.get("password")
user = authenticate(request, username=username, password=password)
# Get client info for logging
from core.utils import log_system_event
client_ip = request.META.get('REMOTE_ADDR', 'unknown')
user_agent = request.META.get('HTTP_USER_AGENT', 'unknown')
logger.debug(f"Login attempt via session: user={username} ip={client_ip}")
if user:
login(request, user)
# Update last_login timestamp
from django.utils import timezone
user.last_login = timezone.now()
user.save(update_fields=['last_login'])
# Log successful login
log_system_event(
event_type='login_success',
user=username,
client_ip=client_ip,
user_agent=user_agent,
)
logger.info(f"Login success via session: user={username} ip={client_ip}")
return Response(
{
"message": "Login successful",
"user": {
"id": user.id,
"username": user.username,
"email": user.email,
"groups": list(user.groups.values_list("name", flat=True)),
},
}
)
# Log failed login attempt
log_system_event(
event_type='login_failed',
user=username or 'unknown',
client_ip=client_ip,
user_agent=user_agent,
reason='Invalid credentials',
)
logger.info(f"Login failed via session: user={username} ip={client_ip}")
return Response({"error": "Invalid credentials"}, status=400)
"""Delegates to TokenObtainPairView (JWT login). Throttling, logging, and
network access checks are handled there."""
view = TokenObtainPairView.as_view()
return view(request._request)
@extend_schema(
description="Log out the current user",
@ -287,11 +247,18 @@ class UserViewSet(viewsets.ModelViewSet):
if request.method == "PATCH":
ALLOWED_FIELDS = {"custom_properties", "first_name", "last_name", "email", "password"}
disallowed = set(request.data.keys()) - ALLOWED_FIELDS
if disallowed:
return Response(
{"detail": f"Fields not allowed for self-update: {', '.join(disallowed)}"},
status=400,
)
for key in disallowed:
request.data.pop(key, None)
# Strip admin-managed keys from custom_properties so users cannot
# set their own XC credentials via this endpoint.
ADMIN_ONLY_PROPS = {"xc_password"}
cp = request.data.get("custom_properties")
if isinstance(cp, dict):
for key in ADMIN_ONLY_PROPS:
cp.pop(key, None)
serializer = UserSerializer(user, data=request.data, partial=True)
serializer.is_valid(raise_exception=True)
serializer.save()

37
apps/accounts/authentication.py
View file

@ -1,9 +1,46 @@
from rest_framework import authentication
from rest_framework import exceptions
from django.conf import settings
from drf_spectacular.extensions import OpenApiAuthenticationExtension
from .models import User
class JWTAuthenticationScheme(OpenApiAuthenticationExtension):
target_class = "rest_framework_simplejwt.authentication.JWTAuthentication"
name = "jwtAuth"
def get_security_definition(self, auto_schema):
return {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": (
"JWT Bearer authentication.\n\n"
"Obtain a token pair via `POST /api/accounts/token/` using your username and password, "
"then paste the **access token** here — Swagger adds the `Bearer ` prefix automatically.\n\n"
"Access tokens expire after 30 minutes. Refresh using `POST /api/accounts/token/refresh/`."
),
}
class ApiKeyAuthenticationScheme(OpenApiAuthenticationExtension):
target_class = "apps.accounts.authentication.ApiKeyAuthentication"
name = "ApiKeyAuth"
def get_security_definition(self, auto_schema):
return {
"type": "apiKey",
"in": "header",
"name": "X-API-Key",
"description": (
"API key authentication.\n\n"
"Pass your personal API key in the `X-API-Key` request header. "
"Keys can be generated via `POST /api/accounts/api-keys/generate/` "
"and revoked via `POST /api/accounts/api-keys/revoke/`."
),
}
class ApiKeyAuthentication(authentication.BaseAuthentication):
"""
Accepts header `Authorization: ApiKey <key>` or `X-API-Key: <key>`.

18
apps/accounts/migrations/0006_user_stream_limit.py Normal file
View file

@ -0,0 +1,18 @@
# Generated by Django 5.2.11 on 2026-03-19 13:46
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0005_alter_user_managers'),
]
operations = [
migrations.AddField(
model_name='user',
name='stream_limit',
field=models.IntegerField(default=0),
),
]

1
apps/accounts/models.py
View file

@ -30,6 +30,7 @@ class User(AbstractUser):
user_level = models.IntegerField(default=UserLevel.STREAMER)
custom_properties = models.JSONField(default=dict, blank=True, null=True)
api_key = models.CharField(max_length=200, blank=True, null=True, db_index=True)
stream_limit = models.IntegerField(default=0)
def __str__(self):
return self.username

1
apps/accounts/serializers.py
View file

@ -65,6 +65,7 @@ class UserSerializer(serializers.ModelSerializer):
"channel_profiles",
"custom_properties",
"avatar_config",
"stream_limit",
"is_staff",
"is_superuser",
"last_login",

13
apps/backups/api_views.py
View file

@ -13,6 +13,7 @@ from rest_framework.permissions import AllowAny
from apps.accounts.permissions import IsAdmin
from rest_framework.parsers import MultiPartParser, FormParser
from rest_framework.response import Response
from core.utils import safe_upload_path
from . import services
from .tasks import create_backup_task, restore_backup_task
@ -267,10 +268,18 @@ def upload_backup(request):
try:
backup_dir = services.get_backup_dir()
filename = uploaded.name or "uploaded-backup.zip"
# Sanitize filename: strip directory components to prevent path traversal
filename = Path(uploaded.name or "uploaded-backup.zip").name
if not filename:
filename = "uploaded-backup.zip"
try:
safe_upload_path(filename, str(backup_dir))
except ValueError:
return Response({"detail": "Invalid filename."}, status=status.HTTP_400_BAD_REQUEST)
# Ensure unique filename
backup_file = backup_dir / filename
backup_file = (backup_dir / filename).resolve()
counter = 1
while backup_file.exists():
name_parts = filename.rsplit(".", 1)

30
apps/backups/services.py
View file

@ -28,10 +28,36 @@ def _is_postgresql() -> bool:
def _get_pg_env() -> dict:
"""Get environment variables for PostgreSQL commands."""
"""Get environment variables for PostgreSQL commands.
Includes PGPASSWORD for password auth and PGSSL* variables for TLS.
Reads TLS config from DATABASES['default']['OPTIONS'], which is
populated by settings.py when POSTGRES_SSL=true.
"""
db_config = settings.DATABASES["default"]
env = os.environ.copy()
env["PGPASSWORD"] = db_config.get("PASSWORD", "")
password = db_config.get("PASSWORD", "")
if password:
env["PGPASSWORD"] = password
else:
env.pop("PGPASSWORD", None)
# Propagate TLS configuration from Django OPTIONS to libpq env vars.
options = db_config.get("OPTIONS", {})
_ssl_env_map = {
"sslmode": "PGSSLMODE",
"sslrootcert": "PGSSLROOTCERT",
"sslcert": "PGSSLCERT",
"sslkey": "PGSSLKEY",
}
# Always strip inherited PGSSL* vars first, then set only what is explicitly configured
for opt_key, env_key in _ssl_env_map.items():
env.pop(env_key, None)
value = options.get(opt_key)
if value:
env[env_key] = value
return env

90
apps/backups/tests.py
View file

@ -15,6 +15,96 @@ from . import services
User = get_user_model()
class PgEnvTlsTestCase(TestCase):
"""Test that _get_pg_env includes TLS and password env vars correctly."""
databases = []
@patch('apps.backups.services.settings')
def test_pg_env_includes_ssl_vars_when_tls_enabled(self, mock_settings):
mock_settings.DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "testdb",
"USER": "testuser",
"PASSWORD": "testpass",
"HOST": "localhost",
"PORT": 5432,
"OPTIONS": {
"sslmode": "verify-full",
"sslrootcert": "/certs/ca.crt",
"sslcert": "/certs/client.crt",
"sslkey": "/certs/client.key",
},
}
}
env = services._get_pg_env()
self.assertEqual(env["PGSSLMODE"], "verify-full")
self.assertEqual(env["PGSSLROOTCERT"], "/certs/ca.crt")
self.assertEqual(env["PGSSLCERT"], "/certs/client.crt")
self.assertEqual(env["PGSSLKEY"], "/certs/client.key")
self.assertEqual(env["PGPASSWORD"], "testpass")
@patch('apps.backups.services.settings')
def test_pg_env_no_ssl_vars_when_tls_disabled(self, mock_settings):
mock_settings.DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "testdb",
"USER": "testuser",
"PASSWORD": "testpass",
"HOST": "localhost",
"PORT": 5432,
}
}
env = services._get_pg_env()
self.assertNotIn("PGSSLMODE", env)
self.assertNotIn("PGSSLROOTCERT", env)
self.assertNotIn("PGSSLCERT", env)
self.assertNotIn("PGSSLKEY", env)
self.assertEqual(env["PGPASSWORD"], "testpass")
@patch('apps.backups.services.settings')
def test_pg_env_no_password_when_empty(self, mock_settings):
"""Cert-only auth: PGPASSWORD must not be set when password is empty."""
mock_settings.DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "testdb",
"USER": "testuser",
"PASSWORD": "",
"HOST": "localhost",
"PORT": 5432,
"OPTIONS": {"sslmode": "verify-full"},
}
}
env = services._get_pg_env()
self.assertNotIn("PGPASSWORD", env)
self.assertEqual(env["PGSSLMODE"], "verify-full")
@patch('apps.backups.services.settings')
def test_pg_env_partial_ssl_options(self, mock_settings):
"""Server-only TLS: only sslmode and CA cert, no client cert/key."""
mock_settings.DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "testdb",
"USER": "testuser",
"PASSWORD": "pass",
"HOST": "localhost",
"PORT": 5432,
"OPTIONS": {
"sslmode": "verify-ca",
"sslrootcert": "/certs/ca.crt",
},
}
}
env = services._get_pg_env()
self.assertEqual(env["PGSSLMODE"], "verify-ca")
self.assertEqual(env["PGSSLROOTCERT"], "/certs/ca.crt")
self.assertNotIn("PGSSLCERT", env)
self.assertNotIn("PGSSLKEY", env)
class BackupServicesTestCase(TestCase):
"""Test cases for backup services"""

194
apps/channels/api_views.py
View file

@ -24,12 +24,13 @@ from apps.accounts.permissions import (
)
from core.models import UserAgent, CoreSettings
from core.utils import RedisClient
from core.utils import RedisClient, safe_upload_path
from .models import (
Stream,
Channel,
ChannelGroup,
ChannelStream,
Logo,
ChannelProfile,
ChannelProfileMembership,
@ -61,7 +62,7 @@ from rest_framework.filters import SearchFilter, OrderingFilter
from apps.epg.models import EPGData
from apps.vod.models import Movie, Series
from django.db.models import Q
from django.http import StreamingHttpResponse, FileResponse, Http404
from django.http import HttpResponse, StreamingHttpResponse, FileResponse, Http404
from django.utils import timezone
import mimetypes
from django.conf import settings
@ -630,6 +631,53 @@ class ChannelViewSet(viewsets.ModelViewSet):
context["include_streams"] = include_streams
return context
@extend_schema(
methods=["PATCH"],
description=(
"Bulk edit multiple channels in a single request. "
"Accepts a JSON array of channel update objects. Each object must include `id` (the channel's primary key). "
"All other fields are optional and support partial updates. "
"The `streams` field accepts a list of stream IDs and will replace the channel's current stream assignments. "
"All updates are validated before any changes are applied and executed in a single database transaction."
),
request=inline_serializer(
name="ChannelBulkEditRequest",
fields={
"id": serializers.IntegerField(help_text="ID of the channel to update (required)."),
"name": serializers.CharField(required=False),
"channel_number": serializers.FloatField(required=False),
"channel_group_id": serializers.IntegerField(required=False, allow_null=True),
"streams": serializers.ListField(
child=serializers.IntegerField(),
required=False,
help_text="List of stream IDs to assign to this channel (replaces existing assignments).",
),
"stream_profile_id": serializers.IntegerField(required=False, allow_null=True),
"logo_id": serializers.IntegerField(required=False, allow_null=True),
"tvg_id": serializers.CharField(required=False, allow_blank=True),
"tvc_guide_stationid": serializers.CharField(required=False, allow_blank=True),
"epg_data_id": serializers.IntegerField(required=False, allow_null=True),
"user_level": serializers.IntegerField(required=False),
"is_adult": serializers.BooleanField(required=False),
},
many=True,
),
responses={
200: inline_serializer(
name="ChannelBulkEditResponse",
fields={
"message": serializers.CharField(),
"channels": ChannelSerializer(many=True),
},
),
400: inline_serializer(
name="ChannelBulkEditErrorResponse",
fields={
"errors": serializers.ListField(child=serializers.DictField()),
},
),
},
)
@action(detail=False, methods=["patch"], url_path="edit/bulk")
def edit_bulk(self, request):
"""
@ -709,19 +757,24 @@ class ChannelViewSet(viewsets.ModelViewSet):
# Apply all updates in a transaction
with transaction.atomic():
streams_updates = []
for channel, validated_data in validated_updates:
# Pop streams before setattr loop — M2M fields can't be set via setattr
streams = validated_data.pop("streams", None)
if streams is not None:
streams_updates.append((channel, streams))
for key, value in validated_data.items():
setattr(channel, key, value)
# Single bulk_update query instead of individual saves
channels_to_update = [channel for channel, _ in validated_updates]
if channels_to_update:
# Collect all unique field names from all updates
# Collect all unique field names from all updates (streams already popped)
all_fields = set()
for _, validated_data in validated_updates:
all_fields.update(validated_data.keys())
# Only call bulk_update if there are fields to update
# Only call bulk_update if there are non-M2M fields to update
if all_fields:
Channel.objects.bulk_update(
channels_to_update,
@ -729,6 +782,32 @@ class ChannelViewSet(viewsets.ModelViewSet):
batch_size=100
)
# Handle streams M2M updates separately
for channel, streams in streams_updates:
normalized_ids = [
stream.id if hasattr(stream, "id") else stream for stream in streams
]
current_links = {
cs.stream_id: cs for cs in channel.channelstream_set.all()
}
existing_ids = set(current_links.keys())
new_ids = set(normalized_ids)
to_remove = existing_ids - new_ids
if to_remove:
channel.channelstream_set.filter(stream_id__in=to_remove).delete()
for order, stream_id in enumerate(normalized_ids):
if stream_id in current_links:
cs = current_links[stream_id]
if cs.order != order:
cs.order = order
cs.save(update_fields=["order"])
else:
ChannelStream.objects.create(
channel=channel, stream_id=stream_id, order=order
)
# Return the updated objects (already in memory)
serialized_channels = ChannelSerializer(
[channel for channel, _ in validated_updates],
@ -1486,7 +1565,11 @@ class ChannelViewSet(viewsets.ModelViewSet):
name="EpgAssociation",
fields={
"channel_id": serializers.IntegerField(),
"epg_data_id": serializers.IntegerField(),
"epg_data_id": serializers.IntegerField(
required=False,
allow_null=True,
help_text="EPG data ID to link. Pass null to remove EPG linkage.",
),
},
),
)
@ -1662,7 +1745,12 @@ class BulkDeleteLogosAPIView(APIView):
"logo_ids": serializers.ListField(
child=serializers.IntegerField(),
help_text="Logo IDs to delete",
)
),
"delete_files": serializers.BooleanField(
required=False,
default=False,
help_text="Whether to also delete local logo files from disk.",
),
},
),
)
@ -1901,10 +1989,13 @@ class LogoViewSet(viewsets.ModelViewSet):
{"error": str(e)}, status=status.HTTP_400_BAD_REQUEST
)
file_name = file.name
file_path = os.path.join("/data/logos", file_name)
# Sanitize filename: strip directory components to prevent path traversal
try:
file_path = safe_upload_path(file.name, "/data/logos")
except ValueError:
return Response({"error": "Invalid filename."}, status=status.HTTP_400_BAD_REQUEST)
os.makedirs(os.path.dirname(file_path), exist_ok=True)
os.makedirs("/data/logos", exist_ok=True)
with open(file_path, "wb+") as destination:
for chunk in file.chunks():
destination.write(chunk)
@ -1924,7 +2015,7 @@ class LogoViewSet(viewsets.ModelViewSet):
# Get custom name from request data, fallback to filename
custom_name = request.data.get('name', '').strip()
logo_name = custom_name if custom_name else file_name
logo_name = custom_name if custom_name else os.path.basename(file_path)
logo, _ = Logo.objects.get_or_create(
url=file_path,
@ -1966,7 +2057,7 @@ class LogoViewSet(viewsets.ModelViewSet):
return response
else: # Remote image
# Skip URLs that recently failed to avoid blocking Daphne workers
# Skip URLs that recently failed to avoid blocking workers
# on unreachable hosts (e.g., dead CDNs referenced by old recordings).
fail_expiry = _logo_fetch_failures.get(logo_url)
if fail_expiry and time.monotonic() < fail_expiry:
@ -1982,15 +2073,37 @@ class LogoViewSet(viewsets.ModelViewSet):
# Fallback to hardcoded if default not found
user_agent = 'Dispatcharr/1.0'
# Add proper timeouts to prevent hanging
# Hard total timeout (connect + full download) prevents a slow
# server dripping bytes from holding a greenlet indefinitely.
_LOGO_TOTAL_TIMEOUT = 10 # seconds
_LOGO_MAX_BYTES = 5 * 1024 * 1024 # 5 MB
remote_response = requests.get(
logo_url,
stream=True,
timeout=(3, 5), # (connect_timeout, read_timeout)
timeout=(3, 5), # (connect_timeout, read_timeout per chunk)
headers={'User-Agent': user_agent}
)
if remote_response.status_code == 200:
# Success — clear any previous failure entry
# Eagerly read the full image with a total time + size cap
# so the greenlet is released quickly.
chunks = []
total = 0
deadline = time.monotonic() + _LOGO_TOTAL_TIMEOUT
for chunk in remote_response.iter_content(chunk_size=8192):
total += len(chunk)
if total > _LOGO_MAX_BYTES:
remote_response.close()
raise Http404("Remote image too large")
if time.monotonic() > deadline:
remote_response.close()
now = time.monotonic()
_logo_fetch_failures[logo_url] = now + _LOGO_FAIL_TTL
raise Http404("Remote image fetch timed out")
chunks.append(chunk)
body = b"".join(chunks)
# Full read succeeded, clear any previous failure entry
_logo_fetch_failures.pop(logo_url, None)
# Try to get content type from response headers first
@ -2004,13 +2117,14 @@ class LogoViewSet(viewsets.ModelViewSet):
if not content_type:
content_type = "image/jpeg"
response = StreamingHttpResponse(
remote_response.iter_content(chunk_size=8192),
response = HttpResponse(
body,
content_type=content_type,
)
if(remote_response.headers.get("Cache-Control")):
response["Content-Length"] = str(len(body))
if remote_response.headers.get("Cache-Control"):
response["Cache-Control"] = remote_response.headers.get("Cache-Control")
if(remote_response.headers.get("Last-Modified")):
if remote_response.headers.get("Last-Modified"):
response["Last-Modified"] = remote_response.headers.get("Last-Modified")
response["Content-Disposition"] = 'inline; filename="{}"'.format(
os.path.basename(logo_url)
@ -2674,7 +2788,49 @@ class RecordingViewSet(viewsets.ModelViewSet):
recording_id = instance.pk
channel_name = instance.channel.name
# Capture state before the DB row is deleted
# Attempt to close the DVR client connection for this channel if active
try:
channel_uuid = str(instance.channel.uuid)
# Lazy imports to avoid module overhead if proxy isn't used
from core.utils import RedisClient
from apps.proxy.ts_proxy.redis_keys import RedisKeys
from apps.proxy.ts_proxy.services.channel_service import ChannelService
r = RedisClient.get_client()
if r:
client_set_key = RedisKeys.clients(channel_uuid)
client_ids = r.smembers(client_set_key) or []
stopped = 0
for cid in client_ids:
try:
meta_key = RedisKeys.client_metadata(channel_uuid, cid)
ua = r.hget(meta_key, "user_agent")
# Identify DVR recording client by its user agent
if ua and "Dispatcharr-DVR" in ua:
try:
ChannelService.stop_client(channel_uuid, cid)
stopped += 1
except Exception as inner_e:
logger.debug(f"Failed to stop DVR client {cid} for channel {channel_uuid}: {inner_e}")
except Exception as inner:
logger.debug(f"Error while checking client metadata: {inner}")
if stopped:
logger.info(f"Stopped {stopped} DVR client(s) for channel {channel_uuid} due to recording cancellation")
# If no clients remain after stopping DVR clients, proactively stop the channel
try:
remaining = r.scard(client_set_key) or 0
except Exception:
remaining = 0
if remaining == 0:
try:
ChannelService.stop_channel(channel_uuid)
logger.info(f"Stopped channel {channel_uuid} (no clients remain)")
except Exception as sc_e:
logger.debug(f"Unable to stop channel {channel_uuid}: {sc_e}")
except Exception as e:
logger.debug(f"Unable to stop DVR clients for cancelled recording: {e}")
# Capture paths before deletion
cp = instance.custom_properties or {}
rec_status = cp.get("status", "")
file_path = cp.get("file_path")

153
apps/channels/models.py
View file

@ -204,7 +204,7 @@ class Stream(models.Model):
return stream_profile
def get_stream(self):
def get_stream(self, requester=None):
"""
Finds an available profile for this stream and reserves a connection slot.
@ -400,6 +400,144 @@ class Channel(models.Model):
return stream_profile
def _pick_channel_to_preempt(
self,
profile_id,
requester_level,
redis_client,
exclude_channel_ids=None,
cooldown_seconds=30,
):
"""
Pick the lowest-impact channel to terminate on the given profile.
Returns: Optional[int] channel_id to preempt
"""
exclude_channel_ids = set(exclude_channel_ids or [])
candidates = []
# 1) Try to get active channel IDs for this profile from an index set if available
ch_set_key = f"ts_proxy:profile:{profile_id}:channels"
try:
ch_ids = { (int(x) if not isinstance(x, int) else x) for x in (redis_client.smembers(ch_set_key) or set()) }
except Exception:
ch_ids = set()
logger.debug("Candidate channels for preemption:")
logger.debug(ch_ids)
# 2) Fallback: scan metadata keys and filter by m3u_profile == profile_id
if not ch_ids:
cursor = 0
pattern = "ts_proxy:channel:*:metadata"
while True:
cursor, keys = redis_client.scan(cursor=cursor, match=pattern, count=500)
if keys:
# Prefer HGET m3u_profile if metadata is a hash
pipe = redis_client.pipeline()
for k in keys:
pipe.hget(k, "m3u_profile")
prof_vals = pipe.execute()
for k, prof_val in zip(keys, prof_vals):
try:
pid = int(prof_val) if prof_val is not None else None
except Exception:
pid = None
if pid == profile_id:
parts = k.split(":") # ts_proxy:channel:{id}:metadata
if len(parts) >= 4:
try:
ch_ids.add(int(parts[2]))
except Exception:
pass
if cursor == 0:
break
logger.debug("Candidate channels for preemption:")
logger.debug(ch_ids)
if not ch_ids:
return None
# 3) Score candidates
for ch_id in ch_ids:
if ch_id in exclude_channel_ids:
continue
# Skip if recently preempted
last_preempt_key = f"ts_proxy:channel:{ch_id}:last_preempt"
try:
last_preempt = float(redis_client.get(last_preempt_key) or 0.0)
except Exception:
last_preempt = 0.0
if last_preempt and (time.time() - last_preempt) < cooldown_seconds:
continue
# Clients and their levels
clients_key = f"ts_proxy:channel:{ch_id}:clients"
member_ids = list(redis_client.smembers(clients_key) or [])
viewer_count = len(member_ids)
max_viewer_level = 0
if viewer_count:
pipe = redis_client.pipeline()
for cid in member_ids:
pipe.hget(f"ts_proxy:channel:{ch_id}:clients:{cid}", "user_level")
levels_raw = pipe.execute()
levels = []
for lv in levels_raw:
try:
levels.append(int(lv or 0))
except Exception:
levels.append(0)
max_viewer_level = max(levels or [0])
# Only preempt if requester strictly outranks this channel's viewers
if requester_level <= max_viewer_level:
continue
# Metadata (protected/recording/started_at_ts)
meta_key = f"ts_proxy:channel:{ch_id}:metadata"
try:
protected, recording, started_at_ts = redis_client.hmget(
meta_key, "protected", "recording", "started_at_ts"
)
except Exception:
protected = recording = started_at_ts = None
protected = str(protected or "0") in ("1", "true", "True")
recording = str(recording or "0") in ("1", "true", "True")
if protected or recording:
continue
try:
started_at_ts = float(started_at_ts) if started_at_ts is not None else None
except Exception:
started_at_ts = None
if started_at_ts is None:
started_at_ts = time.time() # treat unknown as newest
# Score: lower is safer to terminate
has_viewers = 1 if viewer_count > 0 else 0
score = (has_viewers, max_viewer_level, viewer_count, started_at_ts)
candidates.append((score, ch_id))
logger.debug("Candidate channels after scoring:")
logger.debug(candidates)
if not candidates:
return None
candidates.sort(key=lambda x: x[0])
victim_id = candidates[0][1]
# Mark preempt timestamp to avoid thrashing
try:
redis_client.set(f"ts_proxy:channel:{victim_id}:last_preempt", str(time.time()), ex=3600)
except Exception:
pass
return victim_id
def _check_and_reserve_profile_slot(self, profile, redis_client):
"""
Atomically check and reserve a connection slot for the given profile.
@ -432,7 +570,7 @@ class Channel(models.Model):
redis_client.decr(profile_connections_key)
return (False, new_count - 1)
def get_stream(self):
def get_stream(self, requester=None):
"""
Finds an available stream for the requested channel and returns the selected stream and profile.
@ -513,6 +651,17 @@ class Channel(models.Model):
None,
) # Return newly assigned stream and matched profile
else:
# At capacity: try to preempt a lower-impact channel on this profile
victim_channel_id = self._pick_channel_to_preempt(
profile_id=profile.id,
requester_level=requester.user_level if requester else 100,
redis_client=redis_client,
exclude_channel_ids=None,
)
if victim_channel_id:
logger.info(f"Preempting channel {victim_channel_id} for new stream on profile {profile.id}")
# return self.id, profile.id, victim_channel_id
# This profile is at max connections
has_streams_but_maxed_out = True
logger.debug(

82
apps/channels/tasks.py
View file

@ -19,6 +19,7 @@ from rapidfuzz import fuzz
from apps.channels.models import Channel
from apps.epg.models import EPGData
from core.models import CoreSettings
from core.utils import acquire_task_lock, release_task_lock
from django.db import OperationalError, close_old_connections
from channels.layers import get_channel_layer
@ -1070,12 +1071,39 @@ def match_single_channel_epg(channel_id):
def evaluate_series_rules_impl(tvg_id: str | None = None):
"""Synchronous implementation of series rule evaluation; returns details for debugging."""
result = {"scheduled": 0, "details": []}
# Serialize all invocations to prevent concurrent evaluations from
# racing to create duplicate recordings (e.g. multiple EPG sources
# refreshing simultaneously each firing evaluate_series_rules.delay()).
# If Redis is unavailable, proceed without lock — the primary and
# secondary dedup guards still prevent duplicates.
lock_acquired = False
try:
lock_acquired = acquire_task_lock('evaluate_series_rules', 'all')
if not lock_acquired:
result["details"].append({"status": "skipped", "reason": "concurrent evaluation in progress"})
return result
except (ConnectionError, OSError, AttributeError):
logger.warning("Could not acquire series rule evaluation lock (Redis unavailable), proceeding without lock")
try:
return _evaluate_series_rules_locked(tvg_id, result)
finally:
if lock_acquired:
try:
release_task_lock('evaluate_series_rules', 'all')
except (ConnectionError, OSError, AttributeError):
logger.warning("Could not release series rule evaluation lock")
def _evaluate_series_rules_locked(tvg_id, result):
"""Inner implementation of series rule evaluation, called under lock."""
from django.utils import timezone
from apps.channels.models import Recording, Channel
from apps.epg.models import EPGData, ProgramData
rules = CoreSettings.get_dvr_series_rules()
result = {"scheduled": 0, "details": []}
if not isinstance(rules, list) or not rules:
return result
@ -1089,14 +1117,23 @@ def evaluate_series_rules_impl(tvg_id: str | None = None):
now = timezone.now()
horizon = now + timedelta(days=7)
# Preload existing recordings' program ids to avoid duplicates
existing_program_ids = set()
for rec in Recording.objects.all().only("custom_properties"):
# Preload existing recordings keyed by stable program attributes that
# survive EPG refreshes (tvg_id + original start/end times stored in
# custom_properties). ProgramData.id changes on every EPG refresh so
# it cannot be used for deduplication. Only load future recordings
# to bound the set size — past recordings cannot collide with newly
# scheduled future programs.
existing_program_keys = set()
for cp in Recording.objects.filter(
end_time__gte=now,
).values_list("custom_properties", flat=True):
try:
pid = rec.custom_properties.get("program", {}).get("id") if rec.custom_properties else None
if pid is not None:
# Normalize to string for consistent comparisons
existing_program_ids.add(str(pid))
prog_data = (cp or {}).get("program", {})
tvg_id_val = prog_data.get("tvg_id")
st = prog_data.get("start_time")
et = prog_data.get("end_time")
if tvg_id_val and st and et:
existing_program_keys.add((str(tvg_id_val), str(st), str(et)))
except Exception:
continue
@ -1191,17 +1228,21 @@ def evaluate_series_rules_impl(tvg_id: str | None = None):
created_here = 0
for prog in unique_programs:
try:
# Skip if already scheduled by program id
if str(prog.id) in existing_program_ids:
# Skip if a recording already exists for this exact airing
# (keyed by tvg_id + original program times, which are stable
# across EPG refreshes unlike ProgramData.id).
prog_key = (str(prog.tvg_id), prog.start_time.isoformat(), prog.end_time.isoformat())
if prog_key in existing_program_keys:
continue
# Extra guard: skip if a recording exists for the same channel + timeslot
# Extra guard: DB query using the same stable attributes
# stored in custom_properties (unadjusted program times,
# not offset-adjusted Recording.start_time/end_time).
try:
from django.db.models import Q
if Recording.objects.filter(
channel=channel,
start_time=prog.start_time,
end_time=prog.end_time,
).filter(Q(custom_properties__program__id=prog.id) | Q(custom_properties__program__title=prog.title)).exists():
custom_properties__program__tvg_id=prog.tvg_id,
custom_properties__program__start_time=prog.start_time.isoformat(),
custom_properties__program__end_time=prog.end_time.isoformat(),
).exists():
continue
except Exception:
continue # already scheduled/recorded
@ -1245,7 +1286,7 @@ def evaluate_series_rules_impl(tvg_id: str | None = None):
}
},
)
existing_program_ids.add(str(prog.id))
existing_program_keys.add(prog_key)
created_here += 1
try:
prefetch_recording_artwork.apply_async(args=[rec.id], countdown=1)
@ -2452,15 +2493,12 @@ def run_recording(recording_id, channel_id, start_time_str, end_time_str):
metadata_key = RedisKeys.channel_metadata(str(channel.uuid))
md = r.hgetall(metadata_key)
if md:
def _gv(bkey):
return md.get(bkey.encode('utf-8'))
def _d(bkey, cast=str):
v = _gv(bkey)
v = md.get(bkey)
try:
if v is None:
return None
s = v.decode('utf-8')
s = v
return cast(s) if cast is not str else s
except Exception:
return None

718
apps/channels/tests/test_series_rule_dedup.py Normal file
View file

@ -0,0 +1,718 @@
"""Tests for series rule evaluation deduplication.
Unit tests verify the dedup logic in evaluate_series_rules_impl.
Integration tests exercise the full path: EPG refresh → series rule
evaluation → Recording creation → post_save signal chain.
"""
from datetime import timedelta
from unittest.mock import patch, MagicMock
from django.test import TestCase
from django.utils import timezone
from apps.channels.models import Channel, Recording
from apps.epg.models import EPGSource, EPGData, ProgramData
from core.models import CoreSettings
def _set_series_rules(rules):
"""Helper to store series rules in CoreSettings."""
CoreSettings.set_dvr_series_rules(rules)
def _set_dvr_offsets(pre_min=0, post_min=0):
"""Helper to store DVR pre/post offsets."""
CoreSettings._update_group("dvr_settings", "DVR Settings", {
"pre_offset_minutes": pre_min,
"post_offset_minutes": post_min,
})
class SeriesRuleDedupBaseTestCase(TestCase):
"""Shared setup for series rule dedup tests."""
def setUp(self):
self.now = timezone.now()
self.epg_source = EPGSource.objects.create(
name="Test EPG", source_type="xmltv"
)
self.epg = EPGData.objects.create(
tvg_id="test.channel.1",
name="Test Channel EPG",
epg_source=self.epg_source,
)
self.channel = Channel.objects.create(
channel_number=1, name="Test Channel", epg_data=self.epg
)
_set_series_rules([{
"tvg_id": "test.channel.1",
"mode": "all",
"title": "Test Show",
}])
_set_dvr_offsets(pre_min=0, post_min=0)
def _create_program(self, hours_from_now=1, title="Test Show",
sub_title="Episode 1", tvg_id="test.channel.1"):
"""Create a ProgramData at the given offset."""
start = self.now + timedelta(hours=hours_from_now)
end = start + timedelta(hours=1)
return ProgramData.objects.create(
epg=self.epg,
tvg_id=tvg_id,
start_time=start,
end_time=end,
title=title,
sub_title=sub_title,
)
def _simulate_epg_refresh(self, programs_data):
"""Delete all ProgramData and recreate with new IDs (simulates EPG refresh)."""
ProgramData.objects.filter(epg=self.epg).delete()
new_programs = []
for data in programs_data:
prog = ProgramData.objects.create(epg=self.epg, **data)
new_programs.append(prog)
return new_programs
def _program_data_for_refresh(self, prog):
"""Build the dict needed by _simulate_epg_refresh from a ProgramData."""
return {
"tvg_id": prog.tvg_id,
"start_time": prog.start_time,
"end_time": prog.end_time,
"title": prog.title,
"sub_title": prog.sub_title,
}
# ---------------------------------------------------------------------------
# Unit tests: dedup logic in evaluate_series_rules_impl
# ---------------------------------------------------------------------------
@patch("apps.channels.tasks.prefetch_recording_artwork")
@patch("apps.channels.signals.schedule_recording_task", return_value="mock-task-id")
class ProgramIdStabilityTests(SeriesRuleDedupBaseTestCase):
"""Verify dedup works after EPG refresh changes ProgramData IDs."""
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_no_duplicate_after_epg_refresh(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Same program should not be recorded twice after EPG refresh."""
from apps.channels.tasks import evaluate_series_rules_impl
prog = self._create_program(hours_from_now=2)
old_id = prog.id
result1 = evaluate_series_rules_impl()
self.assertEqual(result1["scheduled"], 1)
self.assertEqual(Recording.objects.count(), 1)
new_programs = self._simulate_epg_refresh(
[self._program_data_for_refresh(prog)]
)
self.assertNotEqual(old_id, new_programs[0].id)
result2 = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
self.assertEqual(result2["scheduled"], 0)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_no_duplicate_with_offsets_after_refresh(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Dedup works when DVR offsets shift Recording times away from program times."""
from apps.channels.tasks import evaluate_series_rules_impl
_set_dvr_offsets(pre_min=5, post_min=5)
prog = self._create_program(hours_from_now=2)
result1 = evaluate_series_rules_impl()
self.assertEqual(result1["scheduled"], 1)
rec = Recording.objects.first()
self.assertEqual(rec.start_time, prog.start_time - timedelta(minutes=5))
self.assertEqual(rec.end_time, prog.end_time + timedelta(minutes=5))
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
result2 = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_different_episodes_still_recorded(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Different episodes on the same channel should each get a recording."""
from apps.channels.tasks import evaluate_series_rules_impl
self._create_program(hours_from_now=2, sub_title="Episode 1")
self._create_program(hours_from_now=4, sub_title="Episode 2")
result = evaluate_series_rules_impl()
self.assertEqual(result["scheduled"], 2)
self.assertEqual(Recording.objects.count(), 2)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_new_episode_after_refresh_is_recorded(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""A genuinely new episode appearing after EPG refresh should be recorded."""
from apps.channels.tasks import evaluate_series_rules_impl
prog = self._create_program(hours_from_now=2, sub_title="Episode 1")
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
self._simulate_epg_refresh([
self._program_data_for_refresh(prog),
{
"tvg_id": "test.channel.1",
"start_time": prog.end_time,
"end_time": prog.end_time + timedelta(hours=1),
"title": "Test Show",
"sub_title": "Episode 2",
},
])
result2 = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 2)
self.assertEqual(result2["scheduled"], 1)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_multiple_epg_refreshes_no_duplicates(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Multiple consecutive EPG refreshes should not accumulate duplicates."""
from apps.channels.tasks import evaluate_series_rules_impl
prog = self._create_program(hours_from_now=2)
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
for _ in range(5):
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
@patch("apps.channels.tasks.prefetch_recording_artwork")
@patch("apps.channels.signals.schedule_recording_task", return_value="mock-task-id")
class ConcurrencyGuardTests(SeriesRuleDedupBaseTestCase):
"""Verify the task lock prevents concurrent evaluation."""
def test_lock_acquired_and_released(self, mock_schedule, mock_artwork):
"""evaluate_series_rules_impl acquires and releases the task lock."""
from apps.channels.tasks import evaluate_series_rules_impl
self._create_program(hours_from_now=2)
with patch("apps.channels.tasks.acquire_task_lock", return_value=True) as mock_lock, \
patch("apps.channels.tasks.release_task_lock") as mock_release:
evaluate_series_rules_impl()
mock_lock.assert_called_once_with('evaluate_series_rules', 'all')
mock_release.assert_called_once_with('evaluate_series_rules', 'all')
def test_skips_when_lock_held(self, mock_schedule, mock_artwork):
"""Returns early with skip reason when lock is already held."""
from apps.channels.tasks import evaluate_series_rules_impl
self._create_program(hours_from_now=2)
with patch("apps.channels.tasks.acquire_task_lock", return_value=False):
result = evaluate_series_rules_impl()
self.assertEqual(result["scheduled"], 0)
self.assertTrue(
any(d.get("reason") == "concurrent evaluation in progress"
for d in result["details"]),
)
self.assertEqual(Recording.objects.count(), 0)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_lock_released_on_exception(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Lock is released even if the inner implementation raises."""
from apps.channels.tasks import evaluate_series_rules_impl
with patch("apps.channels.tasks._evaluate_series_rules_locked",
side_effect=RuntimeError("test error")):
with self.assertRaises(RuntimeError):
evaluate_series_rules_impl()
mock_release.assert_called_once_with('evaluate_series_rules', 'all')
@patch("apps.channels.tasks.prefetch_recording_artwork")
@patch("apps.channels.signals.schedule_recording_task", return_value="mock-task-id")
class SecondaryGuardTests(SeriesRuleDedupBaseTestCase):
"""Verify the secondary DB guard uses stable program attributes."""
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_secondary_guard_catches_duplicate_with_offsets(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Secondary guard works with stale program IDs and DVR offsets."""
from apps.channels.tasks import evaluate_series_rules_impl
_set_dvr_offsets(pre_min=10, post_min=10)
prog = self._create_program(hours_from_now=2)
# Pre-existing recording with a stale program ID (from previous EPG refresh)
Recording.objects.create(
channel=self.channel,
start_time=prog.start_time - timedelta(minutes=10),
end_time=prog.end_time + timedelta(minutes=10),
custom_properties={
"program": {
"id": 99999,
"tvg_id": prog.tvg_id,
"title": prog.title,
"start_time": prog.start_time.isoformat(),
"end_time": prog.end_time.isoformat(),
}
},
)
result = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
self.assertEqual(result["scheduled"], 0)
# ---------------------------------------------------------------------------
# Integration tests: full path from EPG refresh through recording creation
# ---------------------------------------------------------------------------
@patch("apps.channels.tasks.prefetch_recording_artwork")
@patch("apps.channels.signals.schedule_recording_task", return_value="mock-task-id")
class IntegrationEPGRefreshTests(SeriesRuleDedupBaseTestCase):
"""End-to-end tests simulating the EPG refresh → evaluate → record flow.
These exercise the full signal chain: evaluate_series_rules_impl creates
a Recording, the post_save signal fires schedule_recording_task, and
subsequent evaluations (after EPG refresh) must not create duplicates.
"""
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_single_episode_no_duplicates(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Simulate: create rule → evaluate → EPG refresh → re-evaluate.
The full recording lifecycle must result in exactly 1 recording.
"""
from apps.channels.tasks import evaluate_series_rules_impl
# Initial EPG data
prog = self._create_program(hours_from_now=2, sub_title="Pilot")
# First evaluation creates the recording
result1 = evaluate_series_rules_impl()
self.assertEqual(result1["scheduled"], 1)
self.assertEqual(Recording.objects.count(), 1)
# Verify the recording was created with correct program metadata
rec = Recording.objects.first()
self.assertEqual(rec.custom_properties["program"]["tvg_id"], "test.channel.1")
self.assertEqual(rec.custom_properties["program"]["title"], "Test Show")
self.assertEqual(
rec.custom_properties["program"]["start_time"],
prog.start_time.isoformat()
)
# Verify the post_save signal scheduled a task
mock_schedule.assert_called()
initial_schedule_count = mock_schedule.call_count
# Simulate EPG refresh (programs get new DB IDs)
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
# Re-evaluate after refresh (this is what EPG refresh triggers)
result2 = evaluate_series_rules_impl()
self.assertEqual(result2["scheduled"], 0)
self.assertEqual(Recording.objects.count(), 1)
# No additional task scheduling should have occurred
self.assertEqual(mock_schedule.call_count, initial_schedule_count)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_with_offsets_no_duplicates(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Full flow with DVR offsets: recording times differ from program times."""
from apps.channels.tasks import evaluate_series_rules_impl
_set_dvr_offsets(pre_min=5, post_min=10)
prog = self._create_program(hours_from_now=3, sub_title="Episode 1")
result1 = evaluate_series_rules_impl()
self.assertEqual(result1["scheduled"], 1)
rec = Recording.objects.first()
# Verify offset-adjusted recording times
self.assertEqual(rec.start_time, prog.start_time - timedelta(minutes=5))
self.assertEqual(rec.end_time, prog.end_time + timedelta(minutes=10))
# Verify original (unadjusted) program times in custom_properties
self.assertEqual(
rec.custom_properties["program"]["start_time"],
prog.start_time.isoformat()
)
self.assertEqual(
rec.custom_properties["program"]["end_time"],
prog.end_time.isoformat()
)
# EPG refresh + re-evaluate
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
result2 = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
self.assertEqual(result2["scheduled"], 0)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_multiple_episodes_across_refreshes(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""New episodes appear across multiple EPG refreshes; each recorded once."""
from apps.channels.tasks import evaluate_series_rules_impl
ep1 = self._create_program(hours_from_now=2, sub_title="Episode 1")
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
# EPG refresh adds episode 2 alongside episode 1
ep1_data = self._program_data_for_refresh(ep1)
ep2_start = ep1.end_time
ep2_data = {
"tvg_id": "test.channel.1",
"start_time": ep2_start,
"end_time": ep2_start + timedelta(hours=1),
"title": "Test Show",
"sub_title": "Episode 2",
}
self._simulate_epg_refresh([ep1_data, ep2_data])
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 2)
# Another EPG refresh adds episode 3
ep3_start = ep2_start + timedelta(hours=1)
ep3_data = {
"tvg_id": "test.channel.1",
"start_time": ep3_start,
"end_time": ep3_start + timedelta(hours=1),
"title": "Test Show",
"sub_title": "Episode 3",
}
self._simulate_epg_refresh([ep1_data, ep2_data, ep3_data])
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 3)
# Final EPG refresh with no new episodes — count must stay at 3
self._simulate_epg_refresh([ep1_data, ep2_data, ep3_data])
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 3)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_multiple_series_rules(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Multiple series rules on different channels, each evaluated correctly."""
from apps.channels.tasks import evaluate_series_rules_impl
# Second channel with its own EPG
epg2 = EPGData.objects.create(
tvg_id="test.channel.2",
name="Channel 2 EPG",
epg_source=self.epg_source,
)
channel2 = Channel.objects.create(
channel_number=2, name="Test Channel 2", epg_data=epg2
)
_set_series_rules([
{"tvg_id": "test.channel.1", "mode": "all", "title": "Show A"},
{"tvg_id": "test.channel.2", "mode": "all", "title": "Show B"},
])
# Programs on both channels
start1 = self.now + timedelta(hours=2)
prog1 = ProgramData.objects.create(
epg=self.epg, tvg_id="test.channel.1",
start_time=start1, end_time=start1 + timedelta(hours=1),
title="Show A", sub_title="Episode 1",
)
start2 = self.now + timedelta(hours=3)
prog2 = ProgramData.objects.create(
epg=epg2, tvg_id="test.channel.2",
start_time=start2, end_time=start2 + timedelta(hours=1),
title="Show B", sub_title="Episode 1",
)
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 2)
self.assertEqual(Recording.objects.filter(channel=self.channel).count(), 1)
self.assertEqual(Recording.objects.filter(channel=channel2).count(), 1)
# EPG refresh for both channels
ProgramData.objects.filter(epg=self.epg).delete()
ProgramData.objects.filter(epg=epg2).delete()
ProgramData.objects.create(
epg=self.epg, tvg_id="test.channel.1",
start_time=start1, end_time=start1 + timedelta(hours=1),
title="Show A", sub_title="Episode 1",
)
ProgramData.objects.create(
epg=epg2, tvg_id="test.channel.2",
start_time=start2, end_time=start2 + timedelta(hours=1),
title="Show B", sub_title="Episode 1",
)
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 2,
"No duplicates across multiple series rules after EPG refresh")
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_rapid_epg_refreshes_simulate_user_report(
self, mock_release, mock_lock, mock_schedule, mock_artwork
):
"""Reproduce the user-reported scenario: series rule + multiple EPG refreshes
causing count to balloon from 6 to 25 and 5 simultaneous recordings.
Simulates 6 episodes with 5 EPG refreshes (each assigning new ProgramData IDs).
"""
from apps.channels.tasks import evaluate_series_rules_impl
# Create 6 episodes (the user had "next of 6")
episodes = []
for i in range(6):
start = self.now + timedelta(hours=2 + i * 2)
episodes.append({
"tvg_id": "test.channel.1",
"start_time": start,
"end_time": start + timedelta(hours=1),
"title": "Test Show",
"sub_title": f"Episode {i + 1}",
})
# Create initial ProgramData
for ep in episodes:
ProgramData.objects.create(epg=self.epg, **ep)
# First evaluation: should create exactly 6 recordings
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 6)
# Simulate 5 EPG refreshes (the user saw count balloon to 25)
for refresh_num in range(5):
self._simulate_epg_refresh(episodes)
result = evaluate_series_rules_impl()
self.assertEqual(
Recording.objects.count(), 6,
f"After EPG refresh #{refresh_num + 1}, expected 6 recordings "
f"but got {Recording.objects.count()}"
)
self.assertEqual(result["scheduled"], 0)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_recording_survives_program_removal_and_readd(
self, mock_release, mock_lock, mock_schedule, mock_artwork
):
"""Program temporarily disappears from EPG then reappears — no duplicate."""
from apps.channels.tasks import evaluate_series_rules_impl
prog = self._create_program(hours_from_now=2, sub_title="Episode 1")
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
# EPG refresh removes the program entirely
self._simulate_epg_refresh([])
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1,
"Existing recording preserved when program disappears from EPG")
# EPG refresh adds the program back (new ID)
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1,
"No duplicate when program reappears with new ID")
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_celery_task_wrapper_calls_impl(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""The @shared_task evaluate_series_rules delegates to _impl correctly."""
from apps.channels.tasks import evaluate_series_rules
self._create_program(hours_from_now=2)
result = evaluate_series_rules()
self.assertEqual(result["scheduled"], 1)
self.assertEqual(Recording.objects.count(), 1)
# Call again (simulating a second EPG refresh trigger)
result2 = evaluate_series_rules()
self.assertEqual(result2["scheduled"], 0)
self.assertEqual(Recording.objects.count(), 1)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_tvg_id_scoped_evaluation(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Scoped evaluation (tvg_id parameter) still prevents duplicates."""
from apps.channels.tasks import evaluate_series_rules_impl
prog = self._create_program(hours_from_now=2)
result1 = evaluate_series_rules_impl(tvg_id="test.channel.1")
self.assertEqual(result1["scheduled"], 1)
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
result2 = evaluate_series_rules_impl(tvg_id="test.channel.1")
self.assertEqual(result2["scheduled"], 0)
self.assertEqual(Recording.objects.count(), 1)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_full_flow_offset_change_between_refreshes(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Changing DVR offsets between EPG refreshes doesn't create duplicates.
Even though Recording.start_time/end_time change when offsets change,
the dedup key uses the original program times from custom_properties.
"""
from apps.channels.tasks import evaluate_series_rules_impl
_set_dvr_offsets(pre_min=5, post_min=5)
prog = self._create_program(hours_from_now=2)
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
rec = Recording.objects.first()
original_start = rec.start_time
original_end = rec.end_time
# Change offsets
_set_dvr_offsets(pre_min=10, post_min=15)
# EPG refresh
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
result = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1,
"Changing offsets between refreshes should not create duplicates")
self.assertEqual(result["scheduled"], 0)
# ---------------------------------------------------------------------------
# Edge case tests: Redis unavailability, non-series recordings, robustness
# ---------------------------------------------------------------------------
@patch("apps.channels.tasks.prefetch_recording_artwork")
@patch("apps.channels.signals.schedule_recording_task", return_value="mock-task-id")
class RedisUnavailabilityTests(SeriesRuleDedupBaseTestCase):
"""Verify evaluation works when Redis is unavailable (lock cannot be acquired)."""
def test_proceeds_when_redis_down(self, mock_schedule, mock_artwork):
"""Evaluation succeeds (with dedup guards) when Redis raises on lock acquire."""
from apps.channels.tasks import evaluate_series_rules_impl
self._create_program(hours_from_now=2)
with patch("apps.channels.tasks.acquire_task_lock",
side_effect=ConnectionError("Redis unavailable")):
result = evaluate_series_rules_impl()
self.assertEqual(result["scheduled"], 1)
self.assertEqual(Recording.objects.count(), 1)
def test_dedup_still_works_without_lock(self, mock_schedule, mock_artwork):
"""Dedup guards prevent duplicates even when the lock is unavailable."""
from apps.channels.tasks import evaluate_series_rules_impl
prog = self._create_program(hours_from_now=2)
# First call: Redis down, proceeds without lock
with patch("apps.channels.tasks.acquire_task_lock",
side_effect=ConnectionError("Redis unavailable")):
evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1)
# EPG refresh
self._simulate_epg_refresh([self._program_data_for_refresh(prog)])
# Second call: Redis still down
with patch("apps.channels.tasks.acquire_task_lock",
side_effect=ConnectionError("Redis unavailable")):
result = evaluate_series_rules_impl()
self.assertEqual(Recording.objects.count(), 1,
"Dedup guards prevent duplicates even without lock")
self.assertEqual(result["scheduled"], 0)
def test_lock_not_released_when_not_acquired(self, mock_schedule, mock_artwork):
"""release_task_lock is not called if acquire raised an exception."""
from apps.channels.tasks import evaluate_series_rules_impl
self._create_program(hours_from_now=2)
with patch("apps.channels.tasks.acquire_task_lock",
side_effect=ConnectionError("Redis unavailable")), \
patch("apps.channels.tasks.release_task_lock") as mock_release:
evaluate_series_rules_impl()
mock_release.assert_not_called()
@patch("apps.channels.tasks.prefetch_recording_artwork")
@patch("apps.channels.signals.schedule_recording_task", return_value="mock-task-id")
class NonSeriesRecordingTests(SeriesRuleDedupBaseTestCase):
"""Verify non-series recordings don't interfere with series rule dedup."""
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_manual_recording_without_program_data_ignored(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Recordings without custom_properties.program are skipped by dedup key builder."""
from apps.channels.tasks import evaluate_series_rules_impl
# Manual recording with no program metadata
Recording.objects.create(
channel=self.channel,
start_time=self.now + timedelta(hours=2),
end_time=self.now + timedelta(hours=3),
custom_properties={},
)
prog = self._create_program(hours_from_now=2)
result = evaluate_series_rules_impl()
self.assertEqual(result["scheduled"], 1)
self.assertEqual(Recording.objects.count(), 2)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_recurring_rule_recording_does_not_interfere(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Recordings from recurring rules (custom_properties.rule) don't block series rules."""
from apps.channels.tasks import evaluate_series_rules_impl
Recording.objects.create(
channel=self.channel,
start_time=self.now + timedelta(hours=2),
end_time=self.now + timedelta(hours=3),
custom_properties={"rule": {"id": 1, "name": "Daily News"}},
)
prog = self._create_program(hours_from_now=2)
result = evaluate_series_rules_impl()
self.assertEqual(result["scheduled"], 1)
self.assertEqual(Recording.objects.count(), 2)
@patch("apps.channels.tasks.acquire_task_lock", return_value=True)
@patch("apps.channels.tasks.release_task_lock")
def test_recording_with_null_custom_properties_ignored(self, mock_release, mock_lock,
mock_schedule, mock_artwork):
"""Recordings with None custom_properties don't crash the dedup key builder."""
from apps.channels.tasks import evaluate_series_rules_impl
Recording.objects.create(
channel=self.channel,
start_time=self.now + timedelta(hours=2),
end_time=self.now + timedelta(hours=3),
custom_properties=None,
)
prog = self._create_program(hours_from_now=2)
result = evaluate_series_rules_impl()
self.assertEqual(result["scheduled"], 1)

30
apps/connect/api_views.py
View file

@ -1,9 +1,10 @@
from rest_framework import viewsets, status
from rest_framework import viewsets, status, serializers
from rest_framework.pagination import PageNumberPagination
from django_filters.rest_framework import DjangoFilterBackend
from rest_framework.response import Response
from rest_framework.decorators import action
from django.utils import timezone
from drf_spectacular.utils import extend_schema, inline_serializer
from .models import Integration, EventSubscription, DeliveryLog
from .serializers import (
IntegrationSerializer,
@ -37,6 +38,33 @@ class IntegrationViewSet(viewsets.ModelViewSet):
serializer = EventSubscriptionSerializer(qs, many=True)
return Response(serializer.data)
@extend_schema(
methods=["PUT"],
description=(
"Replace the integration's event subscriptions with the provided list. "
"Accepts a JSON array of subscription objects. "
"Existing subscriptions not in the list will be deleted. "
"The 'payload_template' field is only relevant for webhook integrations."
),
request=inline_serializer(
name="SetSubscriptionsRequest",
fields={
"event": serializers.CharField(help_text="Event name (e.g. 'channel_start')."),
"enabled": serializers.BooleanField(required=False, default=True),
"payload_template": serializers.CharField(required=False, allow_blank=True, allow_null=True, help_text="Custom payload template (webhook integrations only)."),
},
many=True,
),
responses={200: inline_serializer(
name="SetSubscriptionsResponse",
fields={
"event": serializers.CharField(),
"enabled": serializers.BooleanField(),
"payload_template": serializers.CharField(allow_null=True),
},
many=True,
)},
)
@action(detail=True, methods=["put"], url_path=r"subscriptions/set")
def set_subscriptions(self, request, pk=None):
"""

12
apps/epg/api_views.py
View file

@ -175,7 +175,7 @@ class EPGGridAPIView(APIView):
# Get channels with custom dummy EPG sources (generate on-demand with patterns)
channels_with_custom_dummy = Channel.objects.filter(
epg_data__epg_source__source_type='dummy'
).distinct()
).select_related('epg_data__epg_source').distinct()
# Log what we found
without_count = channels_without_epg.count()
@ -427,7 +427,13 @@ class EPGImportAPIView(APIView):
return [Authenticated()]
@extend_schema(
description="Triggers an EPG data import",
description="Triggers an EPG data refresh for the given source.",
request=inline_serializer(
name="EPGImportRequest",
fields={
"id": serializers.IntegerField(help_text="ID of the EPG source to refresh."),
},
),
)
def post(self, request, format=None):
logger.info("EPGImportAPIView: Received request to import EPG data.")
@ -449,7 +455,7 @@ class EPGImportAPIView(APIView):
refresh_epg_data.delay(epg_id) # Trigger Celery task
logger.info("EPGImportAPIView: Task dispatched to refresh EPG data.")
return Response(
{"success": True, "message": "EPG data import initiated."},
{"success": True, "message": "EPG data refresh initiated."},
status=status.HTTP_202_ACCEPTED,
)

189
apps/epg/tasks.py
View file

@ -2,6 +2,7 @@
import logging
import gzip
import html.entities
import os
import uuid
import requests
@ -15,7 +16,7 @@ import zipfile
from celery import shared_task
from django.conf import settings
from django.db import transaction
from django.db import connection, transaction
from django.utils import timezone
from apps.channels.models import Channel
from core.models import UserAgent, CoreSettings
@ -28,6 +29,103 @@ from core.utils import acquire_task_lock, release_task_lock, TaskLockRenewer, se
logger = logging.getLogger(__name__)
# DOCTYPE internal subset for XMLTV files. Declares all 252 HTML 4 named
# entities so lxml/libxml2 can resolve references like &eacute; correctly
# instead of silently dropping them in recovery mode.
# The 5 XML-predefined entities (amp, lt, gt, quot, apos) are always
# recognised by the XML spec and must not be redeclared.
_XML_ENTITIES = frozenset({'amp', 'lt', 'gt', 'quot', 'apos'})
def _build_html_entity_doctype() -> bytes:
"""Build a DOCTYPE internal subset declaring all HTML 4 named entities."""
lines = [b'<!DOCTYPE tv [\n']
for name, codepoint in sorted(html.entities.name2codepoint.items()):
if name not in _XML_ENTITIES:
# Numeric character references are always valid XML regardless of codepoint.
lines.append(f'<!ENTITY {name} "&#x{codepoint:X};">\n'.encode('ascii'))
lines.append(b']>\n')
return b''.join(lines)
_HTML_ENTITY_DOCTYPE = _build_html_entity_doctype()
class _PrependStream:
"""Wraps an open binary file and prepends a bytes prefix to its content.
Used by _open_xmltv_file to inject a DOCTYPE entity block before the
file content reaches lxml's iterparse, with zero disk I/O.
"""
__slots__ = ('_prefix', '_prefix_pos', '_file')
def __init__(self, prefix: bytes, file_obj):
self._prefix = prefix
self._prefix_pos = 0
self._file = file_obj
def read(self, size=-1):
prefix_len = len(self._prefix)
if self._prefix_pos >= prefix_len:
return self._file.read(size)
remaining = prefix_len - self._prefix_pos
if size < 0:
chunk = self._prefix[self._prefix_pos:] + self._file.read()
self._prefix_pos = prefix_len
return chunk
if size <= remaining:
chunk = self._prefix[self._prefix_pos:self._prefix_pos + size]
self._prefix_pos += size
return chunk
chunk = self._prefix[self._prefix_pos:]
self._prefix_pos = prefix_len
return chunk + self._file.read(size - remaining)
def close(self):
self._file.close()
def __enter__(self):
return self
def __exit__(self, *_):
self.close()
def _open_xmltv_file(file_path: str):
"""Open an XMLTV file for lxml iterparse, injecting an HTML entity DOCTYPE.
Prepends a <!DOCTYPE tv [...]> block that declares all 252 HTML 4 named
entities so lxml/libxml2 resolves references like &eacute; correctly
instead of silently dropping them in recovery mode. This involves zero
disk I/O — the DOCTYPE is streamed in-memory before the file content.
If the file already contains a <!DOCTYPE> declaration the file is returned
unchanged; a second DOCTYPE would be invalid XML.
The caller is responsible for closing the returned object.
"""
f = open(file_path, 'rb')
start = f.read(512)
# Do not inject if the file already declares a DOCTYPE.
if b'<!DOCTYPE' in start or b'<!doctype' in start.lower():
f.seek(0)
return f
# Insert the DOCTYPE after the XML declaration if one is present.
xml_pos = start.find(b'<?xml')
if xml_pos >= 0:
decl_end = start.find(b'?>', xml_pos)
if decl_end >= 0:
xml_decl = start[:decl_end + 2]
f.seek(decl_end + 2)
return _PrependStream(xml_decl + b'\n' + _HTML_ENTITY_DOCTYPE, f)
# No XML declaration — insert DOCTYPE at the very start of the file.
f.seek(0)
return _PrependStream(_HTML_ENTITY_DOCTYPE, f)
def validate_icon_url_fast(icon_url, max_length=None):
"""
@ -146,7 +244,7 @@ def refresh_all_epg_data():
return "EPG data refreshed."
@shared_task(time_limit=1800, soft_time_limit=1700)
@shared_task(time_limit=14400)
def refresh_epg_data(source_id):
if not acquire_task_lock('refresh_epg_data', source_id):
logger.debug(f"EPG refresh for {source_id} already running")
@ -397,42 +495,41 @@ def fetch_xmltv(source):
# Download to temporary file
with open(temp_download_path, 'wb') as f:
for chunk in response.iter_content(chunk_size=16384): # Increased chunk size for better performance
if chunk:
f.write(chunk)
for chunk in response.iter_content(chunk_size=16384):
f.write(chunk)
downloaded += len(chunk)
elapsed_time = time.time() - start_time
downloaded += len(chunk)
elapsed_time = time.time() - start_time
# Calculate download speed in KB/s
speed = downloaded / elapsed_time / 1024 if elapsed_time > 0 else 0
# Calculate download speed in KB/s
speed = downloaded / elapsed_time / 1024 if elapsed_time > 0 else 0
# Calculate progress percentage
if total_size and total_size > 0:
progress = min(100, int((downloaded / total_size) * 100))
else:
# If no content length header, estimate progress
progress = min(95, int((downloaded / (10 * 1024 * 1024)) * 100)) # Assume 10MB if unknown
# Calculate progress percentage
if total_size and total_size > 0:
progress = min(100, int((downloaded / total_size) * 100))
else:
# If no content length header, estimate progress
progress = min(95, int((downloaded / (10 * 1024 * 1024)) * 100)) # Assume 10MB if unknown
# Time remaining (in seconds)
time_remaining = (total_size - downloaded) / (speed * 1024) if speed > 0 and total_size > 0 else 0
# Time remaining (in seconds)
time_remaining = (total_size - downloaded) / (speed * 1024) if speed > 0 and total_size > 0 else 0
# Only send updates at specified intervals to avoid flooding
current_time = time.time()
if current_time - last_update_time >= update_interval and progress > 0:
last_update_time = current_time
send_epg_update(
source.id,
"downloading",
progress,
speed=round(speed, 2),
elapsed_time=round(elapsed_time, 1),
time_remaining=round(time_remaining, 1),
downloaded=f"{downloaded / (1024 * 1024):.2f} MB"
)
# Only send updates at specified intervals to avoid flooding
current_time = time.time()
if current_time - last_update_time >= update_interval and progress > 0:
last_update_time = current_time
send_epg_update(
source.id,
"downloading",
progress,
speed=round(speed, 2),
elapsed_time=round(elapsed_time, 1),
time_remaining=round(time_remaining, 1),
downloaded=f"{downloaded / (1024 * 1024):.2f} MB"
)
# Explicitly delete the chunk to free memory immediately
del chunk
# Explicitly delete the chunk to free memory immediately
del chunk
# Send completion notification
send_epg_update(source.id, "downloading", 100)
@ -526,6 +623,7 @@ def fetch_xmltv(source):
source.save(update_fields=['status'])
logger.info(f"Cached EPG file saved to {source.file_path}")
return True
except requests.exceptions.HTTPError as e:
@ -840,7 +938,8 @@ def parse_channels_only(source):
# Replace full dictionary load with more efficient lookup set
existing_tvg_ids = set()
existing_epgs = {} # Initialize the dictionary that will lazily load objects
existing_epgs = {}
scanned_tvg_ids = set() # Track tvg_ids seen in the current scan for stale cleanup
last_id = 0
chunk_size = 5000
@ -888,7 +987,7 @@ def parse_channels_only(source):
# Open the file - no need to check file type since it's always XML now
logger.debug(f"Opening file for channel parsing: {file_path}")
source_file = open(file_path, 'rb')
source_file = _open_xmltv_file(file_path)
if process:
logger.debug(f"[parse_channels_only] Memory after opening file: {process.memory_info().rss / 1024 / 1024:.2f} MB")
@ -908,6 +1007,7 @@ def parse_channels_only(source):
channel_count += 1
tvg_id = elem.get('id', '').strip()
if tvg_id:
scanned_tvg_ids.add(tvg_id)
display_name = None
icon_url = None
for child in elem:
@ -1055,6 +1155,16 @@ def parse_channels_only(source):
if epgs_to_update:
EPGData.objects.bulk_update(epgs_to_update, ["name", "icon_url"])
logger.debug(f"[parse_channels_only] Updated final batch of {len(epgs_to_update)} EPG entries")
# Clean up stale EPGData: entries that existed before the scan but weren't seen, and aren't mapped to any channel.
# Use existing_tvg_ids - scanned_tvg_ids to avoid a full-table scan with a large EXCLUDE list.
potentially_stale = existing_tvg_ids - scanned_tvg_ids
if potentially_stale:
stale_qs = EPGData.objects.filter(epg_source=source, tvg_id__in=potentially_stale, channels__isnull=True)
deleted_count, _ = stale_qs.delete()
if deleted_count:
logger.info(f"[parse_channels_only] Cleaned up {deleted_count} stale EPG entries not in current scan and unmapped to any channel")
if process:
logger.debug(f"[parse_channels_only] Memory after final batch creation: {process.memory_info().rss / 1024 / 1024:.2f} MB")
@ -1121,6 +1231,9 @@ def parse_channels_only(source):
existing_epgs = None
epgs_to_create = None
epgs_to_update = None
if 'scanned_tvg_ids' in locals() and scanned_tvg_ids is not None:
scanned_tvg_ids.clear()
scanned_tvg_ids = None
cleanup_memory(log_usage=should_log_memory, force_collection=True)
except Exception as e:
logger.warning(f"Cleanup error: {e}")
@ -1271,7 +1384,7 @@ def parse_programs_for_tvg_id(epg_id):
try:
# Open the file directly - no need to check compression
logger.debug(f"Opening file for parsing: {file_path}")
source_file = open(file_path, 'rb')
source_file = _open_xmltv_file(file_path)
# Stream parse the file using lxml's iterparse
program_parser = etree.iterparse(source_file, events=('end',), tag='programme', remove_blank_text=True, recover=True)
@ -1544,7 +1657,7 @@ def parse_programs_for_source(epg_source, tvg_id=None):
try:
logger.debug(f"Opening file for single-pass parsing: {file_path}")
source_file = open(file_path, 'rb')
source_file = _open_xmltv_file(file_path)
# Stream parse the file using lxml's iterparse
program_parser = etree.iterparse(source_file, events=('end',), tag='programme', remove_blank_text=True, recover=True)
@ -1657,6 +1770,10 @@ def parse_programs_for_source(epg_source, tvg_id=None):
batch_size = 1000
try:
with transaction.atomic():
# Kill any individual statement that hangs longer than 10 minutes.
# SET LOCAL automatically resets when this transaction ends (commit or rollback).
with connection.cursor() as cursor:
cursor.execute("SET LOCAL statement_timeout = '10min'")
# Delete existing programs for mapped EPGs
deleted_count = ProgramData.objects.filter(epg_id__in=mapped_epg_ids).delete()[0]
logger.debug(f"Deleted {deleted_count} existing programs")

195
apps/epg/tests/test_entity_resolution.py Normal file
View file

@ -0,0 +1,195 @@
import os
import tempfile
from django.test import TestCase
from apps.epg.tasks import (
_NAMED_ENTITY_RE,
_detect_xml_encoding,
_replace_html_entity,
_resolve_html_entities,
)
class ReplaceHtmlEntityTests(TestCase):
"""Tests for the regex callback that resolves individual HTML entities."""
def _sub(self, text):
return _NAMED_ENTITY_RE.sub(_replace_html_entity, text)
def test_french_accented(self):
self.assertEqual(self._sub("Cha&icirc;ne T&eacute;l&eacute;"), "Chaîne Télé")
def test_german_umlauts(self):
self.assertEqual(self._sub("M&uuml;nchen &Uuml;bersicht &szlig;"), "München Übersicht ß")
def test_spanish(self):
self.assertEqual(self._sub("Espa&ntilde;a &iquest;Qu&eacute;?"), "España ¿Qué?")
def test_portuguese(self):
self.assertEqual(self._sub("Comunica&ccedil;&atilde;o"), "Comunicação")
def test_scandinavian(self):
self.assertEqual(self._sub("Norsk &oslash; &aring; &aelig;"), "Norsk ø å æ")
def test_greek_letters(self):
self.assertEqual(self._sub("&alpha;&beta;&gamma;"), "αβγ")
def test_currency_and_symbols(self):
self.assertEqual(self._sub("&copy; &euro; &pound; &yen;"), "© € £ ¥")
def test_preserves_xml_amp(self):
self.assertEqual(self._sub("A &amp; B"), "A &amp; B")
def test_preserves_xml_lt_gt(self):
self.assertEqual(self._sub("&lt;tag&gt;"), "&lt;tag&gt;")
def test_preserves_xml_quot_apos(self):
self.assertEqual(self._sub("&quot;hello&apos;"), "&quot;hello&apos;")
def test_preserves_uppercase_xml_entities(self):
"""&AMP;, &LT;, &GT;, &QUOT; resolve to XML-special chars; must not be replaced."""
self.assertEqual(self._sub("&AMP;"), "&AMP;")
self.assertEqual(self._sub("&LT;"), "&LT;")
self.assertEqual(self._sub("&GT;"), "&GT;")
self.assertEqual(self._sub("&QUOT;"), "&QUOT;")
def test_partial_entity_match_preserved(self):
"""html.unescape can partially match &amp inside &ampersand; — must not corrupt."""
self.assertEqual(self._sub("&ampersand;"), "&ampersand;")
def test_mixed_html_and_xml_entities(self):
self.assertEqual(
self._sub("R&eacute;sum&eacute; &amp; Co &lt;test&gt;"),
"Résumé &amp; Co &lt;test&gt;",
)
def test_plain_ascii_unchanged(self):
self.assertEqual(self._sub("Plain ASCII text"), "Plain ASCII text")
def test_direct_utf8_unchanged(self):
self.assertEqual(self._sub("日本語テレビ"), "日本語テレビ")
def test_unknown_entity_preserved(self):
self.assertEqual(self._sub("&zzfakeentity;"), "&zzfakeentity;")
class ResolveHtmlEntitiesFileTests(TestCase):
"""Tests for the file-level preprocessing function."""
def _make_file(self, content):
fd, path = tempfile.mkstemp(suffix=".xml")
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(content)
self.addCleanup(lambda: os.unlink(path) if os.path.exists(path) else None)
return path
def test_resolves_entities_in_file(self):
path = self._make_file(
'<?xml version="1.0"?>\n<tv><channel><display-name>T&eacute;l&eacute;</display-name></channel></tv>'
)
_resolve_html_entities(path)
with open(path, "r", encoding="utf-8") as f:
content = f.read()
self.assertIn("Télé", content)
self.assertNotIn("&eacute;", content)
def test_preserves_xml_entities_in_file(self):
path = self._make_file("<tv><desc>A &amp; B &lt;C&gt;</desc></tv>")
_resolve_html_entities(path)
with open(path, "r", encoding="utf-8") as f:
content = f.read()
self.assertIn("&amp;", content)
self.assertIn("&lt;", content)
self.assertIn("&gt;", content)
def test_no_temp_file_left_on_success(self):
path = self._make_file("<tv>test</tv>")
_resolve_html_entities(path)
self.assertFalse(os.path.exists(path + ".entity_tmp"))
def test_plain_file_unchanged(self):
original = '<?xml version="1.0"?>\n<tv><channel><display-name>Plain</display-name></channel></tv>'
path = self._make_file(original)
_resolve_html_entities(path)
with open(path, "r", encoding="utf-8") as f:
content = f.read()
self.assertEqual(content, original)
def test_utf8_content_preserved(self):
original = "<tv><channel><display-name>日本語テレビ</display-name></channel></tv>"
path = self._make_file(original)
_resolve_html_entities(path)
with open(path, "r", encoding="utf-8") as f:
content = f.read()
self.assertIn("日本語テレビ", content)
def test_iso_8859_1_encoding(self):
"""Files declaring ISO-8859-1 should be read in that encoding."""
xml = '<?xml version="1.0" encoding="ISO-8859-1"?>\n<tv><channel><display-name>Cha&icirc;ne</display-name></channel></tv>'
fd, path = tempfile.mkstemp(suffix=".xml")
with os.fdopen(fd, "wb") as f:
f.write(xml.encode("iso-8859-1"))
self.addCleanup(lambda: os.unlink(path) if os.path.exists(path) else None)
_resolve_html_entities(path)
with open(path, "r", encoding="iso-8859-1") as f:
content = f.read()
self.assertIn("Cha\u00eene", content)
self.assertNotIn("&icirc;", content)
def test_detect_encoding_utf8_default(self):
"""Headers without an encoding declaration default to UTF-8."""
self.assertEqual(_detect_xml_encoding(b'<?xml version="1.0"?>'), "utf-8")
def test_detect_encoding_iso_8859_1(self):
"""Encoding is read from the XML declaration."""
self.assertEqual(
_detect_xml_encoding(b'<?xml version="1.0" encoding="ISO-8859-1"?>'),
"ISO-8859-1",
)
def test_detect_encoding_single_quotes(self):
"""Encoding detection works with single-quoted attributes."""
self.assertEqual(
_detect_xml_encoding(b"<?xml version='1.0' encoding='windows-1252'?>"),
"windows-1252",
)
def test_detect_encoding_unknown_falls_back(self):
"""Unrecognized encoding falls back to UTF-8."""
self.assertEqual(
_detect_xml_encoding(b'<?xml version="1.0" encoding="x-fake-codec"?>'),
"utf-8",
)
def test_iso_8859_1_with_entities_roundtrip(self):
"""ISO-8859-1 file with entities: resolved without corrupting existing accented chars."""
# Mix of direct ISO-8859-1 chars and HTML entities
xml_str = '<?xml version="1.0" encoding="ISO-8859-1"?>\n<tv><channel><display-name>D\xe9j\xe0 &eacute;mission</display-name></channel></tv>'
fd, path = tempfile.mkstemp(suffix=".xml")
with os.fdopen(fd, "wb") as f:
f.write(xml_str.encode("iso-8859-1"))
self.addCleanup(lambda: os.unlink(path) if os.path.exists(path) else None)
_resolve_html_entities(path)
with open(path, "r", encoding="iso-8859-1") as f:
content = f.read()
self.assertIn("D\xe9j\xe0", content, "Existing accented chars should be preserved")
self.assertIn("\xe9mission", content, "Entity should be resolved")
self.assertNotIn("&eacute;", content)
def test_mismatched_encoding_leaves_file_untouched(self):
"""File declaring UTF-8 but containing Latin-1 bytes is left alone."""
# \xe9 is valid ISO-8859-1 but invalid as a standalone UTF-8 byte
raw = b'<?xml version="1.0" encoding="UTF-8"?>\n<tv><channel><display-name>\xe9</display-name></channel></tv>'
fd, path = tempfile.mkstemp(suffix=".xml")
with os.fdopen(fd, "wb") as f:
f.write(raw)
self.addCleanup(lambda: os.unlink(path) if os.path.exists(path) else None)
original_bytes = raw # save for comparison
_resolve_html_entities(path)
with open(path, "rb") as f:
result_bytes = f.read()
self.assertEqual(result_bytes, original_bytes, "File should be untouched on decode error")

5
apps/hdhr/api_views.py
View file

@ -1,6 +1,7 @@
from rest_framework import viewsets, status
from rest_framework.response import Response
from rest_framework.views import APIView
from rest_framework.permissions import AllowAny
from apps.accounts.permissions import Authenticated, permission_classes_by_action
from django.http import JsonResponse, HttpResponseForbidden, HttpResponse
import logging
@ -46,6 +47,7 @@ class HDHRDeviceViewSet(viewsets.ModelViewSet):
# 🔹 2) Discover API
class DiscoverAPIView(APIView):
"""Returns device discovery information"""
permission_classes = [AllowAny]
@extend_schema(
description="Retrieve HDHomeRun device discovery information",
@ -98,6 +100,7 @@ class DiscoverAPIView(APIView):
# 🔹 3) Lineup API
class LineupAPIView(APIView):
"""Returns available channel lineup"""
permission_classes = [AllowAny]
@extend_schema(
description="Retrieve the available channel lineup",
@ -138,6 +141,7 @@ class LineupAPIView(APIView):
# 🔹 4) Lineup Status API
class LineupStatusAPIView(APIView):
"""Returns the current status of the HDHR lineup"""
permission_classes = [AllowAny]
@extend_schema(
description="Retrieve the HDHomeRun lineup status",
@ -155,6 +159,7 @@ class LineupStatusAPIView(APIView):
# 🔹 5) Device XML API
class HDHRDeviceXMLAPIView(APIView):
"""Returns HDHomeRun device configuration in XML"""
permission_classes = [AllowAny]
@extend_schema(
description="Retrieve the HDHomeRun device XML configuration",

17
apps/m3u/api_views.py
View file

@ -20,6 +20,7 @@ import json
from .models import M3UAccount, M3UFilter, ServerGroup, M3UAccountProfile
from core.models import UserAgent
from core.utils import safe_upload_path
from apps.channels.models import ChannelGroupM3UAccount
from core.serializers import UserAgentSerializer
from apps.vod.models import M3UVODCategoryRelation
@ -54,10 +55,12 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
file_path = None
if "file" in request.FILES:
file = request.FILES["file"]
file_name = file.name
file_path = os.path.join("/data/uploads/m3us", file_name)
try:
file_path = safe_upload_path(file.name, "/data/uploads/m3us")
except ValueError:
return Response({"detail": "Invalid filename."}, status=status.HTTP_400_BAD_REQUEST)
os.makedirs(os.path.dirname(file_path), exist_ok=True)
os.makedirs("/data/uploads/m3us", exist_ok=True)
with open(file_path, "wb+") as destination:
for chunk in file.chunks():
destination.write(chunk)
@ -117,10 +120,12 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
file_path = None
if "file" in request.FILES:
file = request.FILES["file"]
file_name = file.name
file_path = os.path.join("/data/uploads/m3us", file_name)
try:
file_path = safe_upload_path(file.name, "/data/uploads/m3us")
except ValueError:
return Response({"detail": "Invalid filename."}, status=status.HTTP_400_BAD_REQUEST)
os.makedirs(os.path.dirname(file_path), exist_ok=True)
os.makedirs("/data/uploads/m3us", exist_ok=True)
with open(file_path, "wb+") as destination:
for chunk in file.chunks():
destination.write(chunk)

175
apps/m3u/tasks.py
View file

@ -1,6 +1,7 @@
# apps/m3u/tasks.py
import logging
import re
import regex
import requests
import os
import gc
@ -38,6 +39,8 @@ logger = logging.getLogger(__name__)
BATCH_SIZE = 1500 # Optimized batch size for threading
m3u_dir = os.path.join(settings.MEDIA_ROOT, "cached_m3u")
_EXTINF_ATTR_RE = re.compile(r'([^\s=]+)\s*=\s*(["\'])(.*?)\2')
def fetch_m3u_lines(account, use_cache=False):
os.makedirs(m3u_dir, exist_ok=True)
@ -484,18 +487,13 @@ def parse_extinf_line(line: str) -> dict:
return None
content = line[len("#EXTINF:") :].strip()
# Single pass: extract all attributes AND track the last attribute position
# This regex matches both key="value" and key='value' patterns
# Single pass: extract all attributes AND track the last attribute position.
# Keys are normalised to lowercase so downstream code can use plain dict.get()
attrs = {}
last_attr_end = 0
# Use a single regex that handles both quote types.
# Keys must stop at '=' so values like base64-padded URLs ending with '=='
# don't get folded into the preceding attribute name.
for match in re.finditer(r'([^\s=]+)\s*=\s*(["\'])(.*?)\2', content):
key = match.group(1)
value = match.group(3)
attrs[key] = value
for match in _EXTINF_ATTR_RE.finditer(content):
attrs[match.group(1).lower()] = match.group(3)
last_attr_end = match.end()
# Everything after the last attribute (skipping leading comma and whitespace) is the display name
@ -513,15 +511,80 @@ def parse_extinf_line(line: str) -> dict:
else:
display_name = content.strip()
# Use tvg-name attribute if available; otherwise try tvc-guide-title, then fall back to display name.
name = get_case_insensitive_attr(attrs, "tvg-name", None)
if not name:
name = get_case_insensitive_attr(attrs, "tvc-guide-title", None)
if not name:
name = display_name
# Per the base #EXTINF spec, the comma text is the canonical human-readable title.
# Fall back to tvc-guide-title, then tvg-name (which some providers use as an EPG key,
# not a display label), and finally the raw content if everything else is empty.
name = display_name or attrs.get("tvc-guide-title") or attrs.get("tvg-name") or content.strip()
return {"attributes": attrs, "display_name": display_name, "name": name}
def iter_m3u_entries(lines):
"""
Generator that yields fully-assembled M3U stream entries from raw lines.
Each yielded dict is guaranteed to contain a ``url`` key in addition to the
fields produced by :func:`parse_extinf_line` (``attributes``, ``display_name``,
``name``). Recognised extended-tag lines that appear *between* an ``#EXTINF``
and its URL are accumulated into the pending entry so they are available for
downstream processing:
- ``#EXTGRP`` — sets ``attributes["group-title"]`` when no ``group-title``
attribute was present on the ``#EXTINF`` line (explicit attribute wins).
- ``#EXTVLCOPT`` — stored as a list under the ``vlc_opts`` key.
Unknown directives (``#KODIPROP``, etc.) and blank lines are
silently skipped while keeping the pending entry intact. A second ``#EXTINF``
before a URL discards the first entry with a warning. A trailing ``#EXTINF``
at end-of-file with no URL is also discarded.
Adding support for a new directive requires only a new ``elif`` branch here;
no other code needs to change.
"""
pending = None
pending_line_num = None
for line_num, raw_line in enumerate(lines, 1):
line = raw_line.strip()
if not line:
continue
if line.startswith("#EXTINF"):
if pending is not None:
logger.warning(
f"Line {pending_line_num}: #EXTINF had no URL (next #EXTINF at line {line_num}); "
f"discarding entry: {list(pending['attributes'].items())[:3]}"
)
parsed = parse_extinf_line(line)
if parsed is None:
logger.warning(f"Line {line_num}: Failed to parse #EXTINF: {line[:200]}")
pending = parsed # None if malformed; URL branch guards on `pending is not None`
pending_line_num = line_num
elif line.startswith("#EXTGRP:"):
# Only apply when group-title is absent — explicit attribute wins.
if pending is not None and "group-title" not in pending["attributes"]:
pending["attributes"]["group-title"] = line[len("#EXTGRP:"):].strip()
# else: #EXTGRP outside an entry, or group-title already set — silently skip
elif line.startswith("#EXTVLCOPT:"):
if pending is not None:
pending.setdefault("vlc_opts", []).append(line[len("#EXTVLCOPT:"):])
# else: #EXTVLCOPT outside an entry — silently skip
elif pending is not None and line.startswith(("http", "rtsp", "rtp", "udp")):
pending["url"] = normalize_stream_url(line) if line.startswith("udp") else line
yield pending
pending = None
pending_line_num = None
# else: unknown directive or bare content — skip, keeping pending intact
if pending is not None:
logger.warning(
f"Line {pending_line_num}: #EXTINF at end of file had no URL; "
f"discarding entry: {list(pending['attributes'].items())[:3]}"
)
@shared_task
def refresh_m3u_accounts():
"""Queue background parse for all active M3UAccounts."""
@ -1114,7 +1177,7 @@ def process_m3u_batch_direct(account_id, batch, groups, hash_keys):
"m3u_account": account,
"channel_group_id": int(groups.get(group_title)),
"stream_hash": stream_hash,
"custom_properties": stream_info["attributes"],
"custom_properties": {**stream_info["attributes"], "vlc_opts": stream_info["vlc_opts"]} if "vlc_opts" in stream_info else stream_info["attributes"],
"is_adult": parse_is_adult(stream_info["attributes"].get("is_adult", 0)),
"is_stale": False,
"stream_id": provider_stream_id,
@ -1482,7 +1545,6 @@ def refresh_m3u_groups(account_id, use_cache=False, full_refresh=False, scan_sta
release_task_lock("refresh_m3u_account_groups", account_id)
return error_msg, None
else:
# Here's the key change - use the success flag from fetch_m3u_lines
lines, success = fetch_m3u_lines(account, use_cache)
if not success:
# If fetch failed, don't continue processing
@ -1493,71 +1555,20 @@ def refresh_m3u_groups(account_id, use_cache=False, full_refresh=False, scan_sta
# Log basic file structure for debugging
logger.debug(f"Processing {len(lines)} lines from M3U file")
line_count = 0
extinf_count = 0
url_count = 0
valid_stream_count = 0
problematic_lines = []
for line_index, line in enumerate(lines):
line_count += 1
line = line.strip()
for entry in iter_m3u_entries(lines):
valid_stream_count += 1
group_title_attr = get_case_insensitive_attr(entry["attributes"], "group-title", "")
if group_title_attr and group_title_attr not in groups:
logger.debug(f"Found new group for M3U account {account_id}: '{group_title_attr}'")
groups[group_title_attr] = {}
extinf_data.append(entry)
if line.startswith("#EXTINF"):
extinf_count += 1
parsed = parse_extinf_line(line)
if parsed:
group_title_attr = get_case_insensitive_attr(
parsed["attributes"], "group-title", ""
)
if group_title_attr:
group_name = group_title_attr
# Log new groups as they're discovered
if group_name not in groups:
logger.debug(
f"Found new group for M3U account {account_id}: '{group_name}'"
)
groups[group_name] = {}
if valid_stream_count % 1000 == 0:
logger.debug(f"Processed {valid_stream_count} valid streams so far for M3U account: {account_id}")
extinf_data.append(parsed)
else:
# Log problematic EXTINF lines
logger.warning(
f"Failed to parse EXTINF at line {line_index+1}: {line[:200]}"
)
problematic_lines.append((line_index + 1, line[:200]))
elif extinf_data and (line.startswith("http") or line.startswith("rtsp") or line.startswith("rtp") or line.startswith("udp")):
url_count += 1
# Normalize UDP URLs only (e.g., remove VLC-specific @ prefix)
normalized_url = normalize_stream_url(line) if line.startswith("udp") else line
# Associate URL with the last EXTINF line
extinf_data[-1]["url"] = normalized_url
valid_stream_count += 1
# Periodically log progress for large files
if valid_stream_count % 1000 == 0:
logger.debug(
f"Processed {valid_stream_count} valid streams so far for M3U account: {account_id}"
)
# Log summary statistics
logger.info(
f"M3U parsing complete - Lines: {line_count}, EXTINF: {extinf_count}, URLs: {url_count}, Valid streams: {valid_stream_count}"
)
if problematic_lines:
logger.warning(
f"Found {len(problematic_lines)} problematic lines during parsing"
)
for i, (line_num, content) in enumerate(
problematic_lines[:10]
): # Log max 10 examples
logger.warning(f"Problematic line #{i+1} at line {line_num}: {content}")
if len(problematic_lines) > 10:
logger.warning(
f"... and {len(problematic_lines) - 10} more problematic lines"
)
logger.info(f"M3U parsing complete - Valid streams: {valid_stream_count}")
# Log group statistics
logger.info(
@ -2399,11 +2410,13 @@ def get_transformed_credentials(account, profile=None):
# Apply profile-specific transformations if profile is provided
if profile and profile.search_pattern and profile.replace_pattern:
try:
# Handle backreferences in the replacement pattern
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', profile.replace_pattern)
# Handle backreferences: convert JS-style $<name> -> \g<name>, $1 -> \1
# regex module accepts JS-style (?<name>...) named groups natively
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', profile.replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
# Apply transformation to the complete URL
transformed_complete_url = re.sub(profile.search_pattern, safe_replace_pattern, complete_url)
transformed_complete_url = regex.sub(profile.search_pattern, safe_replace_pattern, complete_url)
logger.info(f"Transformed complete URL: {complete_url} -> {transformed_complete_url}")
# Extract components from the transformed URL

799
apps/output/views.py
View file

File diff suppressed because it is too large Load diff

17
apps/plugins/api_urls.py
View file

@ -8,6 +8,14 @@ from .api_views import (
PluginImportAPIView,
PluginDeleteAPIView,
PluginLogoAPIView,
PluginRepoListCreateAPIView,
PluginRepoPreviewAPIView,
PluginRepoDetailAPIView,
PluginRepoRefreshAPIView,
AvailablePluginsAPIView,
PluginDetailManifestAPIView,
PluginInstallFromRepoAPIView,
PluginRepoSettingsAPIView,
)
app_name = "plugins"
@ -21,4 +29,13 @@ urlpatterns = [
path("plugins/<str:key>/run/", PluginRunAPIView.as_view(), name="run"),
path("plugins/<str:key>/enabled/", PluginEnabledAPIView.as_view(), name="enabled"),
path("plugins/<str:key>/logo/", PluginLogoAPIView.as_view(), name="logo"),
# Plugin repos (hub / store) - static paths first, then parametric
path("repos/", PluginRepoListCreateAPIView.as_view(), name="repo-list"),
path("repos/available/", AvailablePluginsAPIView.as_view(), name="available-plugins"),
path("repos/plugin-detail/", PluginDetailManifestAPIView.as_view(), name="plugin-detail-manifest"),
path("repos/install/", PluginInstallFromRepoAPIView.as_view(), name="repo-install"),
path("repos/settings/", PluginRepoSettingsAPIView.as_view(), name="repo-settings"),
path("repos/preview/", PluginRepoPreviewAPIView.as_view(), name="repo-preview"),
path("repos/<int:pk>/", PluginRepoDetailAPIView.as_view(), name="repo-detail"),
path("repos/<int:pk>/refresh/", PluginRepoRefreshAPIView.as_view(), name="repo-refresh"),
]

1220
apps/plugins/api_views.py
View file

File diff suppressed because it is too large Load diff

50
apps/plugins/apps.py
View file

@ -52,3 +52,53 @@ class PluginsConfig(AppConfig):
import logging
logging.getLogger(__name__).exception("Plugin discovery wiring failed during app ready")
# Register periodic task for refreshing plugin repo manifests
self._setup_repo_refresh_schedule()
# Refresh repo manifests once at startup so the UI always has current data
self._enqueue_startup_refresh()
def _enqueue_startup_refresh(self):
from dispatcharr.app_initialization import should_skip_initialization
if should_skip_initialization():
return
try:
from .tasks import refresh_plugin_repos
refresh_plugin_repos.apply_async(countdown=10)
except Exception:
import logging
logging.getLogger(__name__).debug(
"Could not enqueue startup plugin repo refresh (Celery may not be ready yet)"
)
def _setup_repo_refresh_schedule(self):
from dispatcharr.app_initialization import should_skip_initialization
if should_skip_initialization():
return
try:
from core.scheduling import create_or_update_periodic_task, delete_periodic_task
from core.models import CoreSettings
from .tasks import PLUGIN_REPO_REFRESH_TASK_NAME
interval = 6
try:
obj = CoreSettings.objects.get(key="plugin_repo_settings")
interval = obj.value.get("refresh_interval_hours", 6)
except CoreSettings.DoesNotExist:
pass
if interval == 0:
delete_periodic_task(PLUGIN_REPO_REFRESH_TASK_NAME)
else:
create_or_update_periodic_task(
task_name=PLUGIN_REPO_REFRESH_TASK_NAME,
celery_task_path="apps.plugins.tasks.refresh_plugin_repos",
interval_hours=interval,
enabled=True,
)
except Exception:
import logging
logging.getLogger(__name__).debug(
"Could not set up plugin repo refresh schedule (migrations may not have run yet)"
)

11
apps/plugins/keys/dispatcharr-plugins.pub Normal file
View file

@ -0,0 +1,11 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mDMEacgfABYJKwYBBAHaRw8BAQdAh1MuVNBxk+CExQPjOVDvAGvIk6BdGS2ce9/h
zB7lYtW0TERpc3BhdGNoYXJyIFBsdWdpbiBSZXBvIChkaXNwYXRjaGFyci1hdXRv
Z2VuZXJhdGVkKSA8cGx1Z2luc0BkaXNwYXRjaGFyci50dj6IrwQTFgoAVxYhBEap
MFaOD7nKg0zX+H7AOmtMIjTOBQJpyB8AGxSAAAAAAAQADm1hbnUyLDIuNSsxLjEy
LDAsMwIbAwULCQgHAgIiAgYVCgkICwIEFgIDAQIeBwIXgAAKCRB+wDprTCI0zvNZ
AP9r3TpMpiI8BCNo9B5M9lJ+QLRo9ihPWIcqBzJ9eFCoSQEAgguiZsNy6aJzKjIb
yDvGuoZi3I2/GNM/f2qVzFtgPQk=
=Zf/y
-----END PGP PUBLIC KEY BLOCK-----

52
apps/plugins/loader.py
View file

@ -367,21 +367,35 @@ class PluginManager:
obj.save()
def list_plugins(self) -> List[Dict[str, Any]]:
from .models import PluginConfig
from .models import PluginConfig, PluginRepo
plugins: List[Dict[str, Any]] = []
with self._lock:
registry_snapshot = dict(self._registry)
try:
configs = {c.key: c for c in PluginConfig.objects.all()}
configs = {c.key: c for c in PluginConfig.objects.select_related("source_repo").all()}
except Exception as e:
# Database might not be migrated yet; fall back to registry only
logger.warning("PluginConfig table unavailable; listing registry only: %s", e)
configs = {}
# Build repo latest-version lookup from cached manifests
repo_latest = {} # slug -> latest_version
try:
for repo in PluginRepo.objects.filter(enabled=True):
manifest_data = repo.cached_manifest or {}
manifest = manifest_data.get("manifest", manifest_data)
for rp in manifest.get("plugins", []):
s = rp.get("slug", "")
if s:
repo_latest[s] = rp.get("latest_version", "")
except Exception:
pass
# First, include all discovered plugins
for key, lp in registry_snapshot.items():
conf = configs.get(key)
conf_slug = conf.slug if conf else ""
trusted = bool(conf and (conf.ever_enabled or conf.enabled))
logo_url = self._get_logo_url(key, path=lp.path)
plugins.append(
@ -393,7 +407,7 @@ class PluginManager:
"author": getattr(lp, "author", "") or "",
"help_url": getattr(lp, "help_url", "") or "",
"enabled": conf.enabled if conf else False,
"ever_enabled": getattr(conf, "ever_enabled", False) if conf else False,
"ever_enabled": conf.ever_enabled if conf else False,
"fields": lp.fields or [],
"settings": (conf.settings if conf else {}),
"actions": lp.actions or [],
@ -402,6 +416,22 @@ class PluginManager:
"loaded": bool(lp.loaded),
"legacy": bool(getattr(lp, "legacy", False)),
"logo_url": logo_url,
"source_repo": conf.source_repo_id if conf else None,
"source_repo_name": conf.source_repo.name if conf and conf.source_repo else None,
"is_official_repo": bool(conf and conf.source_repo and conf.source_repo.is_official),
"slug": conf_slug,
"is_managed": bool(conf and conf.source_repo_id),
"installed_version_is_prerelease": bool(
conf and conf.installed_version_is_prerelease
),
"update_available": bool(
conf_slug and conf and conf.source_repo_id
and not (conf and conf.installed_version_is_prerelease)
and repo_latest.get(conf_slug)
and lp.version != repo_latest.get(conf_slug)
),
"latest_version": repo_latest.get(conf_slug, ""),
"deprecated": conf.deprecated if conf else False,
}
)
@ -428,6 +458,22 @@ class PluginManager:
"loaded": False,
"legacy": False,
"logo_url": self._get_logo_url(key),
"source_repo": conf.source_repo_id,
"source_repo_name": conf.source_repo.name if conf.source_repo else None,
"is_official_repo": bool(conf.source_repo and conf.source_repo.is_official),
"slug": conf.slug,
"is_managed": bool(conf.source_repo_id),
"installed_version_is_prerelease": bool(
conf.installed_version_is_prerelease
),
"update_available": bool(
conf.slug and conf.source_repo_id
and not conf.installed_version_is_prerelease
and repo_latest.get(conf.slug)
and conf.version != repo_latest.get(conf.slug)
),
"latest_version": repo_latest.get(conf.slug or "", ""),
"deprecated": conf.deprecated,
}
)

84
apps/plugins/migrations/0002_pluginrepo.py Normal file
View file

@ -0,0 +1,84 @@
import django.db.models.deletion
from django.db import migrations, models
def seed_official_repo(apps, schema_editor):
PluginRepo = apps.get_model("plugins", "PluginRepo")
PluginRepo.objects.get_or_create(
url="https://raw.githubusercontent.com/Dispatcharr/Plugins/releases/manifest.json",
defaults={
"name": "Dispatcharr Official",
"is_official": True,
"enabled": True,
},
)
def unseed_official_repo(apps, schema_editor):
PluginRepo = apps.get_model("plugins", "PluginRepo")
PluginRepo.objects.filter(is_official=True).delete()
class Migration(migrations.Migration):
dependencies = [
("plugins", "0001_initial"),
]
operations = [
migrations.CreateModel(
name="PluginRepo",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("name", models.CharField(max_length=255)),
("url", models.URLField(unique=True)),
("is_official", models.BooleanField(default=False)),
("enabled", models.BooleanField(default=True)),
("cached_manifest", models.JSONField(blank=True, default=dict)),
("last_fetched", models.DateTimeField(blank=True, null=True)),
("public_key", models.TextField(blank=True, default="")),
("signature_verified", models.BooleanField(blank=True, default=None, null=True)),
("last_fetch_status", models.CharField(blank=True, default="", max_length=255)),
("created_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
],
options={
"ordering": ["-is_official", "name"],
},
),
migrations.RunPython(seed_official_repo, unseed_official_repo),
migrations.AddField(
model_name="pluginconfig",
name="source_repo",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="installed_plugins",
to="plugins.pluginrepo",
),
),
migrations.AddField(
model_name="pluginconfig",
name="slug",
field=models.CharField(blank=True, default="", max_length=128),
),
migrations.AddField(
model_name="pluginconfig",
name="installed_version_is_prerelease",
field=models.BooleanField(default=False),
),
migrations.AddField(
model_name="pluginconfig",
name="deprecated",
field=models.BooleanField(default=False),
),
]

44
apps/plugins/models.py
View file

@ -12,8 +12,52 @@ class PluginConfig(models.Model):
# Tracks whether this plugin has ever been enabled at least once
ever_enabled = models.BooleanField(default=False)
settings = models.JSONField(default=dict, blank=True)
# Managed plugin fields (populated when installed from a repo)
source_repo = models.ForeignKey(
"PluginRepo",
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name="installed_plugins",
)
slug = models.CharField(max_length=128, blank=True, default="")
installed_version_is_prerelease = models.BooleanField(default=False)
deprecated = models.BooleanField(default=False)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
@property
def is_managed(self):
return bool(self.source_repo_id)
def __str__(self) -> str:
return f"{self.name} ({self.key})"
OFFICIAL_REPO_URL = (
"https://raw.githubusercontent.com/Dispatcharr/Plugins/releases/manifest.json"
)
class PluginRepo(models.Model):
"""A remote plugin repository manifest URL."""
name = models.CharField(max_length=255)
url = models.URLField(unique=True)
is_official = models.BooleanField(default=False)
enabled = models.BooleanField(default=True)
cached_manifest = models.JSONField(default=dict, blank=True)
public_key = models.TextField(blank=True, default="")
signature_verified = models.BooleanField(null=True, blank=True, default=None)
last_fetched = models.DateTimeField(null=True, blank=True)
last_fetch_status = models.CharField(max_length=255, blank=True, default="")
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
ordering = ["-is_official", "name"]
def __str__(self) -> str:
return self.name

38
apps/plugins/serializers.py
View file

@ -1,4 +1,5 @@
from rest_framework import serializers
from .models import PluginRepo
class PluginActionSerializer(serializers.Serializer):
@ -46,3 +47,40 @@ class PluginSerializer(serializers.Serializer):
fields = PluginFieldSerializer(many=True)
settings = serializers.JSONField()
actions = PluginActionSerializer(many=True)
source_repo = serializers.IntegerField(required=False, allow_null=True)
slug = serializers.CharField(required=False, allow_blank=True)
is_managed = serializers.BooleanField(required=False)
deprecated = serializers.BooleanField(required=False)
class PluginRepoSerializer(serializers.ModelSerializer):
registry_url = serializers.SerializerMethodField()
plugin_count = serializers.SerializerMethodField()
class Meta:
model = PluginRepo
fields = [
"id",
"name",
"url",
"is_official",
"enabled",
"public_key",
"signature_verified",
"registry_url",
"plugin_count",
"last_fetched",
"last_fetch_status",
"created_at",
"updated_at",
]
read_only_fields = ["id", "name", "is_official", "signature_verified", "registry_url", "plugin_count", "last_fetched", "last_fetch_status", "created_at", "updated_at"]
def get_registry_url(self, obj):
manifest = (obj.cached_manifest or {}).get("manifest", obj.cached_manifest or {})
return manifest.get("registry_url", "") or ""
def get_plugin_count(self, obj):
manifest = (obj.cached_manifest or {}).get("manifest", obj.cached_manifest or {})
plugins = manifest.get("plugins", [])
return len(plugins) if isinstance(plugins, list) else 0

33
apps/plugins/tasks.py Normal file
View file

@ -0,0 +1,33 @@
import logging
from celery import shared_task
logger = logging.getLogger(__name__)
PLUGIN_REPO_REFRESH_TASK_NAME = "plugin-repo-refresh-task"
@shared_task
def refresh_plugin_repos():
"""Refresh cached manifests for all enabled plugin repos."""
from .models import PluginRepo
from .api_views import _fetch_manifest, _save_fetched_manifest_to_repo, _unmanage_dropped_slugs
from django.utils import timezone
repos = PluginRepo.objects.filter(enabled=True)
for repo in repos:
try:
key_text = repo.public_key if not repo.is_official else None
data, verified = _fetch_manifest(repo.url, public_key_text=key_text)
err = _save_fetched_manifest_to_repo(repo, data, verified)
if err:
logger.warning("Skipping repo '%s': %s", repo.name, err)
continue
_unmanage_dropped_slugs(repo, data)
logger.info("Refreshed plugin repo '%s'", repo.name)
except Exception as e:
resp = getattr(e, 'response', None)
status_str = str(resp.status_code) if resp is not None and hasattr(resp, 'status_code') else type(e).__name__
repo.last_fetch_status = status_str[:255]
repo.last_fetched = timezone.now()
repo.save(update_fields=["last_fetch_status", "last_fetched", "updated_at"])
logger.warning("Failed to refresh plugin repo '%s': %s", repo.name, e)

20
apps/proxy/hls_proxy/views.py
View file

@ -4,6 +4,8 @@ import logging
from django.http import StreamingHttpResponse, JsonResponse, HttpResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_http_methods
from rest_framework.decorators import api_view, permission_classes
from apps.accounts.permissions import IsAdmin
from .server import ProxyServer, Config
logger = logging.getLogger(__name__)
@ -15,7 +17,7 @@ def stream_endpoint(request, channel_id):
"""Handle HLS manifest requests"""
if channel_id not in proxy_server.stream_managers:
return JsonResponse({'error': 'Channel not found'}, status=404)
response = proxy_server.stream_endpoint(channel_id)
return StreamingHttpResponse(
response[0],
@ -30,10 +32,10 @@ def get_segment(request, segment_name):
try:
segment_num = int(segment_name.split('.')[0])
buffer = proxy_server.stream_buffers.get(segment_num)
if not buffer:
return JsonResponse({'error': 'Segment not found'}, status=404)
return StreamingHttpResponse(
buffer,
content_type='video/MP2T'
@ -44,19 +46,19 @@ def get_segment(request, segment_name):
logger.error(f"Error serving segment: {e}")
return JsonResponse({'error': str(e)}, status=500)
@csrf_exempt
@require_http_methods(["POST"])
@api_view(["POST"])
@permission_classes([IsAdmin])
def change_stream(request, channel_id):
"""Change stream URL for existing channel"""
try:
if channel_id not in proxy_server.stream_managers:
return JsonResponse({'error': 'Channel not found'}, status=404)
data = json.loads(request.body)
new_url = data.get('url')
if not new_url:
return JsonResponse({'error': 'No URL provided'}, status=400)
manager = proxy_server.stream_managers[channel_id]
if manager.update_url(new_url):
return JsonResponse({
@ -64,7 +66,7 @@ def change_stream(request, channel_id):
'channel': channel_id,
'url': new_url
})
return JsonResponse({
'message': 'URL unchanged',
'channel': channel_id,
@ -85,7 +87,7 @@ def initialize_stream(request, channel_id):
url = data.get('url')
if not url:
return JsonResponse({'error': 'No URL provided'}, status=400)
proxy_server.initialize_channel(url, channel_id)
return JsonResponse({
'message': 'Stream initialized',

19
apps/proxy/tasks.py
View file

@ -1,16 +1,11 @@
# yourapp/tasks.py
from celery import shared_task
from channels.layers import get_channel_layer
from asgiref.sync import async_to_sync
import redis
import json
import logging
import re
import gc # Add import for garbage collection
import gc
from core.utils import RedisClient
from apps.proxy.ts_proxy.channel_status import ChannelStatus
from core.utils import send_websocket_update
from apps.proxy.vod_proxy.connection_manager import get_connection_manager
logger = logging.getLogger(__name__)
@ -31,7 +26,7 @@ def fetch_channel_stats():
while True:
cursor, keys = redis_client.scan(cursor, match=channel_pattern)
for key in keys:
channel_id_match = re.search(r"ts_proxy:channel:(.*):metadata", key.decode('utf-8'))
channel_id_match = re.search(r"ts_proxy:channel:(.*):metadata", key)
if channel_id_match:
ch_id = channel_id_match.group(1)
channel_info = ChannelStatus.get_basic_channel_info(ch_id)
@ -61,12 +56,4 @@ def fetch_channel_stats():
all_channels = None
gc.collect()
@shared_task
def cleanup_vod_connections():
"""Clean up stale VOD connections"""
try:
connection_manager = get_connection_manager()
connection_manager.cleanup_stale_connections(max_age_seconds=3600) # 1 hour
logger.info("VOD connection cleanup completed")
except Exception as e:
logger.error(f"Error in VOD connection cleanup: {e}", exc_info=True)

208
apps/proxy/ts_proxy/channel_status.py
View file

@ -39,19 +39,19 @@ class ChannelStatus:
info = {
'channel_id': channel_id,
'state': metadata.get(ChannelMetadataField.STATE.encode('utf-8'), b'unknown').decode('utf-8'),
'url': metadata.get(ChannelMetadataField.URL.encode('utf-8'), b'').decode('utf-8'),
'stream_profile': metadata.get(ChannelMetadataField.STREAM_PROFILE.encode('utf-8'), b'').decode('utf-8'),
'started_at': metadata.get(ChannelMetadataField.INIT_TIME.encode('utf-8'), b'0').decode('utf-8'),
'owner': metadata.get(ChannelMetadataField.OWNER.encode('utf-8'), b'unknown').decode('utf-8'),
'buffer_index': int(buffer_index_value.decode('utf-8')) if buffer_index_value else 0,
'state': metadata.get(ChannelMetadataField.STATE, 'unknown'),
'url': metadata.get(ChannelMetadataField.URL, ''),
'stream_profile': metadata.get(ChannelMetadataField.STREAM_PROFILE, ''),
'started_at': metadata.get(ChannelMetadataField.INIT_TIME, '0'),
'owner': metadata.get(ChannelMetadataField.OWNER, 'unknown'),
'buffer_index': int(buffer_index_value) if buffer_index_value else 0,
}
# Add stream ID and name information
stream_id_bytes = metadata.get(ChannelMetadataField.STREAM_ID.encode('utf-8'))
stream_id_bytes = metadata.get(ChannelMetadataField.STREAM_ID)
if stream_id_bytes:
try:
stream_id = int(stream_id_bytes.decode('utf-8'))
stream_id = int(stream_id_bytes)
info['stream_id'] = stream_id
# Look up stream name from database
@ -66,10 +66,10 @@ class ChannelStatus:
logger.warning(f"Invalid stream_id format in Redis: {stream_id_bytes}")
# Add M3U profile information
m3u_profile_id_bytes = metadata.get(ChannelMetadataField.M3U_PROFILE.encode('utf-8'))
m3u_profile_id_bytes = metadata.get(ChannelMetadataField.M3U_PROFILE)
if m3u_profile_id_bytes:
try:
m3u_profile_id = int(m3u_profile_id_bytes.decode('utf-8'))
m3u_profile_id = int(m3u_profile_id_bytes)
info['m3u_profile_id'] = m3u_profile_id
# Look up M3U profile name from database
@ -84,22 +84,22 @@ class ChannelStatus:
logger.warning(f"Invalid m3u_profile_id format in Redis: {m3u_profile_id_bytes}")
# Add timing information
state_changed_field = ChannelMetadataField.STATE_CHANGED_AT.encode('utf-8')
state_changed_field = ChannelMetadataField.STATE_CHANGED_AT
if state_changed_field in metadata:
state_changed_at = float(metadata[state_changed_field].decode('utf-8'))
state_changed_at = float(metadata[state_changed_field])
info['state_changed_at'] = state_changed_at
info['state_duration'] = time.time() - state_changed_at
init_time_field = ChannelMetadataField.INIT_TIME.encode('utf-8')
init_time_field = ChannelMetadataField.INIT_TIME
if init_time_field in metadata:
created_at = float(metadata[init_time_field].decode('utf-8'))
created_at = float(metadata[init_time_field])
info['started_at'] = created_at
info['uptime'] = time.time() - created_at
# Add data throughput information
total_bytes_field = ChannelMetadataField.TOTAL_BYTES.encode('utf-8')
total_bytes_field = ChannelMetadataField.TOTAL_BYTES
if total_bytes_field in metadata:
total_bytes = int(metadata[total_bytes_field].decode('utf-8'))
total_bytes = int(metadata[total_bytes_field])
info['total_bytes'] = total_bytes
# Format total bytes in human-readable form
@ -130,7 +130,7 @@ class ChannelStatus:
stale_client_ids = []
for client_id in client_ids:
client_id_str = client_id.decode('utf-8')
client_id_str = client_id
client_key = RedisKeys.client_metadata(channel_id, client_id_str)
client_data = proxy_server.redis_client.hgetall(client_key)
@ -141,33 +141,35 @@ class ChannelStatus:
client_info = {
'client_id': client_id_str,
'user_agent': client_data.get(b'user_agent', b'unknown').decode('utf-8'),
'worker_id': client_data.get(b'worker_id', b'unknown').decode('utf-8'),
'user_agent': client_data.get('user_agent', 'unknown'),
'worker_id': client_data.get('worker_id', 'unknown'),
'ip_address': client_data.get('ip_address', 'unknown'),
'user_id': client_data.get('user_id', '0'),
}
if b'connected_at' in client_data:
connected_at = float(client_data[b'connected_at'].decode('utf-8'))
if 'connected_at' in client_data:
connected_at = float(client_data['connected_at'])
client_info['connected_at'] = connected_at
client_info['connection_duration'] = time.time() - connected_at
if b'last_active' in client_data:
last_active = float(client_data[b'last_active'].decode('utf-8'))
if 'last_active' in client_data:
last_active = float(client_data['last_active'])
client_info['last_active'] = last_active
client_info['last_active_ago'] = time.time() - last_active
# Add transfer rate statistics
if b'bytes_sent' in client_data:
client_info['bytes_sent'] = int(client_data[b'bytes_sent'].decode('utf-8'))
if 'bytes_sent' in client_data:
client_info['bytes_sent'] = int(client_data['bytes_sent'])
# Add average transfer rate
if b'avg_rate_KBps' in client_data:
client_info['avg_rate_KBps'] = float(client_data[b'avg_rate_KBps'].decode('utf-8'))
elif b'transfer_rate_KBps' in client_data: # For backward compatibility
client_info['avg_rate_KBps'] = float(client_data[b'transfer_rate_KBps'].decode('utf-8'))
if 'avg_rate_KBps' in client_data:
client_info['avg_rate_KBps'] = float(client_data['avg_rate_KBps'])
elif 'transfer_rate_KBps' in client_data: # For backward compatibility
client_info['avg_rate_KBps'] = float(client_data['transfer_rate_KBps'])
# Add current transfer rate
if b'current_rate_KBps' in client_data:
client_info['current_rate_KBps'] = float(client_data[b'current_rate_KBps'].decode('utf-8'))
if 'current_rate_KBps' in client_data:
client_info['current_rate_KBps'] = float(client_data['current_rate_KBps'])
clients.append(client_info)
@ -249,7 +251,7 @@ class ChannelStatus:
while True:
cursor, keys = proxy_server.redis_client.scan(cursor, match=buffer_key_pattern, count=100)
if keys:
all_buffer_keys.extend([k.decode('utf-8') for k in keys])
all_buffer_keys.extend([k for k in keys])
if cursor == 0 or len(all_buffer_keys) >= 20: # Limit to 20 keys
break
@ -279,61 +281,64 @@ class ChannelStatus:
}
# Add FFmpeg stream information
video_codec = metadata.get(ChannelMetadataField.VIDEO_CODEC.encode('utf-8'))
video_codec = metadata.get(ChannelMetadataField.VIDEO_CODEC)
if video_codec:
info['video_codec'] = video_codec.decode('utf-8')
info['video_codec'] = video_codec
resolution = metadata.get(ChannelMetadataField.RESOLUTION.encode('utf-8'))
resolution = metadata.get(ChannelMetadataField.RESOLUTION)
if resolution:
info['resolution'] = resolution.decode('utf-8')
info['resolution'] = resolution
source_fps = metadata.get(ChannelMetadataField.SOURCE_FPS.encode('utf-8'))
source_fps = metadata.get(ChannelMetadataField.SOURCE_FPS)
if source_fps:
info['source_fps'] = float(source_fps.decode('utf-8'))
info['source_fps'] = source_fps
pixel_format = metadata.get(ChannelMetadataField.PIXEL_FORMAT.encode('utf-8'))
pixel_format = metadata.get(ChannelMetadataField.PIXEL_FORMAT)
if pixel_format:
info['pixel_format'] = pixel_format.decode('utf-8')
info['pixel_format'] = pixel_format
source_bitrate = metadata.get(ChannelMetadataField.SOURCE_BITRATE.encode('utf-8'))
source_bitrate = metadata.get(ChannelMetadataField.SOURCE_BITRATE)
if source_bitrate:
info['source_bitrate'] = float(source_bitrate.decode('utf-8'))
info['source_bitrate'] = source_bitrate
audio_codec = metadata.get(ChannelMetadataField.AUDIO_CODEC.encode('utf-8'))
audio_codec = metadata.get(ChannelMetadataField.AUDIO_CODEC)
if audio_codec:
info['audio_codec'] = audio_codec.decode('utf-8')
info['audio_codec'] = audio_codec
sample_rate = metadata.get(ChannelMetadataField.SAMPLE_RATE.encode('utf-8'))
sample_rate = metadata.get(ChannelMetadataField.SAMPLE_RATE)
if sample_rate:
info['sample_rate'] = int(sample_rate.decode('utf-8'))
info['sample_rate'] = sample_rate
audio_channels = metadata.get(ChannelMetadataField.AUDIO_CHANNELS.encode('utf-8'))
audio_channels = metadata.get(ChannelMetadataField.AUDIO_CHANNELS)
if audio_channels:
info['audio_channels'] = audio_channels.decode('utf-8')
info['audio_channels'] = audio_channels
audio_bitrate = metadata.get(ChannelMetadataField.AUDIO_BITRATE.encode('utf-8'))
audio_bitrate = metadata.get(ChannelMetadataField.AUDIO_BITRATE)
if audio_bitrate:
info['audio_bitrate'] = float(audio_bitrate.decode('utf-8'))
info['audio_bitrate'] = audio_bitrate
# Add FFmpeg performance stats
ffmpeg_speed = metadata.get(ChannelMetadataField.FFMPEG_SPEED.encode('utf-8'))
ffmpeg_speed = metadata.get(ChannelMetadataField.FFMPEG_SPEED)
if ffmpeg_speed:
info['ffmpeg_speed'] = float(ffmpeg_speed.decode('utf-8'))
info['ffmpeg_speed'] = ffmpeg_speed
ffmpeg_fps = metadata.get(ChannelMetadataField.FFMPEG_FPS.encode('utf-8'))
ffmpeg_fps = metadata.get(ChannelMetadataField.FFMPEG_FPS)
if ffmpeg_fps:
info['ffmpeg_fps'] = float(ffmpeg_fps.decode('utf-8'))
info['ffmpeg_fps'] = ffmpeg_fps
actual_fps = metadata.get(ChannelMetadataField.ACTUAL_FPS.encode('utf-8'))
actual_fps = metadata.get(ChannelMetadataField.ACTUAL_FPS)
if actual_fps:
info['actual_fps'] = float(actual_fps.decode('utf-8'))
info['actual_fps'] = actual_fps
ffmpeg_bitrate = metadata.get(ChannelMetadataField.FFMPEG_BITRATE.encode('utf-8'))
ffmpeg_bitrate = metadata.get(ChannelMetadataField.FFMPEG_BITRATE)
if ffmpeg_bitrate:
info['ffmpeg_bitrate'] = float(ffmpeg_bitrate.decode('utf-8'))
stream_type = metadata.get(ChannelMetadataField.STREAM_TYPE.encode('utf-8'))
info['ffmpeg_bitrate'] = ffmpeg_bitrate
stream_type = metadata.get(ChannelMetadataField.STREAM_TYPE)
if stream_type:
info['stream_type'] = stream_type.decode('utf-8')
info['stream_type'] = stream_type
return info
@ -378,33 +383,27 @@ class ChannelStatus:
client_count = proxy_server.redis_client.scard(client_set_key) or 0
# Calculate uptime
init_time_bytes = metadata.get(ChannelMetadataField.INIT_TIME.encode('utf-8'), b'0')
created_at = float(init_time_bytes.decode('utf-8'))
init_time_bytes = metadata.get(ChannelMetadataField.INIT_TIME, '0')
created_at = float(init_time_bytes)
uptime = time.time() - created_at if created_at > 0 else 0
# Safely decode bytes or use defaults
def safe_decode(bytes_value, default="unknown"):
if bytes_value is None:
return default
return bytes_value.decode('utf-8')
# Simplified info
info = {
'channel_id': channel_id,
'state': safe_decode(metadata.get(ChannelMetadataField.STATE.encode('utf-8'))),
'url': safe_decode(metadata.get(ChannelMetadataField.URL.encode('utf-8')), ""),
'stream_profile': safe_decode(metadata.get(ChannelMetadataField.STREAM_PROFILE.encode('utf-8')), ""),
'owner': safe_decode(metadata.get(ChannelMetadataField.OWNER.encode('utf-8'))),
'buffer_index': int(buffer_index_value.decode('utf-8')) if buffer_index_value else 0,
'state': metadata.get(ChannelMetadataField.STATE),
'url': metadata.get(ChannelMetadataField.URL, ""),
'stream_profile': metadata.get(ChannelMetadataField.STREAM_PROFILE, ""),
'owner': metadata.get(ChannelMetadataField.OWNER),
'buffer_index': int(buffer_index_value) if buffer_index_value else 0,
'client_count': client_count,
'uptime': uptime
}
# Add stream ID and name information
stream_id_bytes = metadata.get(ChannelMetadataField.STREAM_ID.encode('utf-8'))
stream_id_bytes = metadata.get(ChannelMetadataField.STREAM_ID)
if stream_id_bytes:
try:
stream_id = int(stream_id_bytes.decode('utf-8'))
stream_id = int(stream_id_bytes)
info['stream_id'] = stream_id
# Look up stream name from database
@ -419,9 +418,9 @@ class ChannelStatus:
logger.warning(f"Invalid stream_id format in Redis: {stream_id_bytes}")
# Add data throughput information to basic info
total_bytes_bytes = proxy_server.redis_client.hget(metadata_key, ChannelMetadataField.TOTAL_BYTES.encode('utf-8'))
total_bytes_bytes = proxy_server.redis_client.hget(metadata_key, ChannelMetadataField.TOTAL_BYTES)
if total_bytes_bytes:
total_bytes = int(total_bytes_bytes.decode('utf-8'))
total_bytes = int(total_bytes_bytes)
info['total_bytes'] = total_bytes
# Calculate and add bitrate
@ -458,25 +457,28 @@ class ChannelStatus:
if client_id in stale_client_ids:
continue
client_id_str = client_id.decode('utf-8')
client_key = RedisKeys.client_metadata(channel_id, client_id_str)
client_key = RedisKeys.client_metadata(channel_id, client_id)
client_info = {
'client_id': client_id_str,
'client_id': client_id,
}
user_agent_bytes = proxy_server.redis_client.hget(client_key, 'user_agent')
client_info['user_agent'] = safe_decode(user_agent_bytes)
client_info['user_agent'] = user_agent_bytes
ip_address_bytes = proxy_server.redis_client.hget(client_key, 'ip_address')
if ip_address_bytes:
client_info['ip_address'] = safe_decode(ip_address_bytes)
client_info['ip_address'] = ip_address_bytes
connected_at_bytes = proxy_server.redis_client.hget(client_key, 'connected_at')
if connected_at_bytes:
connected_at = float(connected_at_bytes.decode('utf-8'))
connected_at = float(connected_at_bytes)
client_info['connected_since'] = time.time() - connected_at
user_id_bytes = proxy_server.redis_client.hget(client_key, 'user_id')
if user_id_bytes:
client_info['user_id'] = user_id_bytes
clients.append(client_info)
# Add clients to info
@ -484,10 +486,10 @@ class ChannelStatus:
info['client_count'] = client_count
# Add M3U profile information
m3u_profile_id_bytes = metadata.get(ChannelMetadataField.M3U_PROFILE.encode('utf-8'))
if m3u_profile_id_bytes:
m3u_profile_id = metadata.get(ChannelMetadataField.M3U_PROFILE)
if m3u_profile_id:
try:
m3u_profile_id = int(m3u_profile_id_bytes.decode('utf-8'))
m3u_profile_id = int(m3u_profile_id)
info['m3u_profile_id'] = m3u_profile_id
# Look up M3U profile name from database
@ -499,32 +501,36 @@ class ChannelStatus:
except (ImportError, DatabaseError) as e:
logger.warning(f"Failed to get M3U profile name for ID {m3u_profile_id}: {e}")
except ValueError:
logger.warning(f"Invalid m3u_profile_id format in Redis: {m3u_profile_id_bytes}")
logger.warning(f"Invalid m3u_profile_id format in Redis: {m3u_profile_id}")
# Add stream info to basic info as well
video_codec = metadata.get(ChannelMetadataField.VIDEO_CODEC.encode('utf-8'))
video_codec = metadata.get(ChannelMetadataField.VIDEO_CODEC)
if video_codec:
info['video_codec'] = video_codec.decode('utf-8')
info['video_codec'] = video_codec
resolution = metadata.get(ChannelMetadataField.RESOLUTION.encode('utf-8'))
resolution = metadata.get(ChannelMetadataField.RESOLUTION)
if resolution:
info['resolution'] = resolution.decode('utf-8')
info['resolution'] = resolution
source_fps = metadata.get(ChannelMetadataField.SOURCE_FPS.encode('utf-8'))
source_fps = metadata.get(ChannelMetadataField.SOURCE_FPS)
if source_fps:
info['source_fps'] = float(source_fps.decode('utf-8'))
ffmpeg_speed = metadata.get(ChannelMetadataField.FFMPEG_SPEED.encode('utf-8'))
info['source_fps'] = float(source_fps)
ffmpeg_speed = metadata.get(ChannelMetadataField.FFMPEG_SPEED)
if ffmpeg_speed:
info['ffmpeg_speed'] = float(ffmpeg_speed.decode('utf-8'))
audio_codec = metadata.get(ChannelMetadataField.AUDIO_CODEC.encode('utf-8'))
info['ffmpeg_speed'] = float(ffmpeg_speed)
audio_codec = metadata.get(ChannelMetadataField.AUDIO_CODEC)
if audio_codec:
info['audio_codec'] = audio_codec.decode('utf-8')
audio_channels = metadata.get(ChannelMetadataField.AUDIO_CHANNELS.encode('utf-8'))
info['audio_codec'] = audio_codec
audio_channels = metadata.get(ChannelMetadataField.AUDIO_CHANNELS)
if audio_channels:
info['audio_channels'] = audio_channels.decode('utf-8')
stream_type = metadata.get(ChannelMetadataField.STREAM_TYPE.encode('utf-8'))
info['audio_channels'] = audio_channels
stream_type = metadata.get(ChannelMetadataField.STREAM_TYPE)
if stream_type:
info['stream_type'] = stream_type.decode('utf-8')
info['stream_type'] = stream_type
return info
except Exception as e:

34
apps/proxy/ts_proxy/client_manager.py
View file

@ -1,10 +1,8 @@
"""Client connection management for TS streams"""
import threading
import logging
import time
import json
import gevent
from typing import Set, Optional
from apps.proxy.config import TSConfig as Config
from redis.exceptions import ConnectionError, TimeoutError
@ -58,7 +56,8 @@ class ClientManager:
from django.conf import settings
redis_url = getattr(settings, 'REDIS_URL', 'redis://localhost:6379/0')
redis_client = redis.Redis.from_url(redis_url, decode_responses=True)
ssl_params = getattr(settings, 'REDIS_SSL_PARAMS', {})
redis_client = redis.Redis.from_url(redis_url, decode_responses=True, **ssl_params)
all_channels = []
cursor = 0
@ -129,7 +128,7 @@ class ClientManager:
# Check for stale activity using last_active field
last_active = self.redis_client.hget(client_key, "last_active")
if last_active:
last_active_time = float(last_active.decode('utf-8'))
last_active_time = float(last_active)
ghost_timeout = self.heartbeat_interval * getattr(Config, 'GHOST_CLIENT_MULTIPLIER', 5.0)
if current_time - last_active_time > ghost_timeout:
@ -229,7 +228,7 @@ class ClientManager:
except Exception as e:
logger.error(f"Error notifying owner of client activity: {e}")
def add_client(self, client_id, client_ip, user_agent=None):
def add_client(self, client_id, client_ip, user_agent=None, user=None):
"""Add a client with duplicate prevention"""
if client_id in self._registered_clients:
logger.debug(f"Client {client_id} already registered, skipping")
@ -247,7 +246,9 @@ class ClientManager:
"ip_address": client_ip,
"connected_at": current_time,
"last_active": current_time,
"worker_id": self.worker_id or "unknown"
"worker_id": self.worker_id or "unknown",
"user_id": str(user.id) if user is not None else "0",
# "user_level": user.user_level if user is not None else 100, # default to a high value since no user means the non-user specific M3U/HDHR
}
try:
@ -277,7 +278,8 @@ class ClientManager:
"channel_id": self.channel_id,
"client_id": client_id,
"worker_id": self.worker_id or "unknown",
"timestamp": time.time()
"timestamp": time.time(),
"username": user.username if user is not None else "unknown"
}
if user_agent:
@ -308,8 +310,6 @@ class ClientManager:
def remove_client(self, client_id):
"""Remove a client from this channel and Redis"""
client_ip = None
with self.lock:
if client_id in self.clients:
self.clients.remove(client_id)
@ -320,13 +320,11 @@ class ClientManager:
self.last_active_time = time.time()
if self.redis_client:
# Get client IP before removing the data
# Get client data before removing the data
client_key = f"ts_proxy:channel:{self.channel_id}:clients:{client_id}"
client_data = self.redis_client.hgetall(client_key)
if client_data and b'ip_address' in client_data:
client_ip = client_data[b'ip_address'].decode('utf-8')
elif client_data and 'ip_address' in client_data:
client_ip = client_data['ip_address']
client_username = self.redis_client.hget(client_key, "username") or "unknown"
if isinstance(client_username, bytes):
client_username = client_username.decode("utf-8")
# Remove from channel's client set
self.redis_client.srem(self.client_set_key, client_id)
@ -367,7 +365,8 @@ class ClientManager:
"client_id": client_id,
"worker_id": self.worker_id or "unknown",
"timestamp": time.time(),
"remaining_clients": remaining
"remaining_clients": remaining,
"username": client_username
})
self.redis_client.publish(RedisKeys.events_channel(self.channel_id), event_data)
@ -434,8 +433,7 @@ class ClientManager:
client_id_list = list(client_ids)
pipe = redis_client.pipeline()
for cid in client_id_list:
cid_str = cid.decode('utf-8')
pipe.exists(RedisKeys.client_metadata(channel_id, cid_str))
pipe.exists(RedisKeys.client_metadata(channel_id, cid))
results = pipe.execute()
stale_ids = [

124
apps/proxy/ts_proxy/server.py
View file

@ -166,6 +166,7 @@ class ProxyServer:
redis_password = os.environ.get("REDIS_PASSWORD", getattr(settings, 'REDIS_PASSWORD', ''))
redis_user = os.environ.get("REDIS_USER", getattr(settings, 'REDIS_USER', ''))
ssl_params = getattr(settings, 'REDIS_SSL_PARAMS', {})
pubsub_client = redis.Redis(
host=redis_host,
port=redis_port,
@ -175,7 +176,9 @@ class ProxyServer:
socket_timeout=60,
socket_connect_timeout=10,
socket_keepalive=True,
health_check_interval=30
health_check_interval=30,
decode_responses=True,
**ssl_params
)
logger.info("Created fallback Redis PubSub client for event listener")
@ -196,8 +199,8 @@ class ProxyServer:
continue
try:
channel = message["channel"].decode("utf-8")
data = json.loads(message["data"].decode("utf-8"))
channel = message["channel"]
data = json.loads(message["data"])
event_type = data.get("event")
channel_id = data.get("channel_id")
@ -224,26 +227,29 @@ class ProxyServer:
# Handle stream switch request
new_url = data.get("url")
user_agent = data.get("user_agent")
event_stream_id = data.get("stream_id")
event_m3u_profile_id = data.get("m3u_profile_id")
if new_url and channel_id in self.stream_managers:
# Update metadata in Redis
# Mark the switch as in-progress in Redis so other workers know to wait
if self.redis_client:
metadata_key = RedisKeys.channel_metadata(channel_id)
self.redis_client.hset(metadata_key, "url", new_url)
if user_agent:
self.redis_client.hset(metadata_key, "user_agent", user_agent)
# Set switch status
status_key = RedisKeys.switch_status(channel_id)
self.redis_client.set(status_key, "switching")
# Perform the stream switch
# Perform the stream switch, forwarding stream_id and m3u_profile_id
stream_manager = self.stream_managers[channel_id]
success = stream_manager.update_url(new_url)
success = stream_manager.update_url(new_url, event_stream_id, event_m3u_profile_id)
if success:
logger.info(f"Stream switch initiated for channel {channel_id}")
# Confirm the URL in metadata now that the switch happened
if self.redis_client:
metadata_key = RedisKeys.channel_metadata(channel_id)
self.redis_client.hset(metadata_key, "url", new_url)
if user_agent:
self.redis_client.hset(metadata_key, "user_agent", user_agent)
# Publish confirmation
switch_result = {
"event": EventType.STREAM_SWITCHED, # Use constant instead of string
@ -263,6 +269,14 @@ class ProxyServer:
else:
logger.error(f"Failed to switch stream for channel {channel_id}")
# Roll back the URL in metadata to what the manager will
# actually reconnect to. The non-owner may have pre-written
# the desired URL; use stream_manager.url (the ground truth)
# so Redis is consistent with the live stream.
if self.redis_client:
metadata_key = RedisKeys.channel_metadata(channel_id)
self.redis_client.hset(metadata_key, "url", stream_manager.url)
# Publish failure
switch_result = {
"event": EventType.STREAM_SWITCHED,
@ -373,7 +387,7 @@ class ProxyServer:
if result is None:
return None
try:
return result.decode('utf-8')
return result
except (AttributeError, UnicodeDecodeError) as e:
logger.error(f"Error decoding channel owner for {channel_id}: {e}, raw={result!r}")
return None
@ -412,7 +426,7 @@ class ProxyServer:
current_owner = self._execute_redis_command(
lambda: self.redis_client.get(lock_key)
)
if current_owner and current_owner.decode('utf-8') == self.worker_id:
if current_owner and current_owner == self.worker_id:
# Refresh TTL
self._execute_redis_command(
lambda: self.redis_client.expire(lock_key, ttl)
@ -437,7 +451,7 @@ class ProxyServer:
# Only delete if we're the current owner to prevent race conditions
current = self.redis_client.get(lock_key)
if current and current.decode('utf-8') == self.worker_id:
if current and current == self.worker_id:
self.redis_client.delete(lock_key)
logger.info(f"Released ownership of channel {channel_id}")
@ -471,7 +485,7 @@ class ProxyServer:
return False
return False
if current.decode('utf-8') == self.worker_id:
if current == self.worker_id:
self.redis_client.expire(lock_key, ttl)
return True
@ -488,15 +502,15 @@ class ProxyServer:
metadata_key = RedisKeys.channel_metadata(channel_id)
if self.redis_client.exists(metadata_key):
metadata = self.redis_client.hgetall(metadata_key)
if b'state' in metadata:
state = metadata[b'state'].decode('utf-8')
if 'state' in metadata:
state = metadata['state']
active_states = [ChannelState.INITIALIZING, ChannelState.CONNECTING,
ChannelState.WAITING_FOR_CLIENTS, ChannelState.ACTIVE, ChannelState.BUFFERING]
if state in active_states:
logger.info(f"Channel {channel_id} already being initialized with state {state}")
# Create buffer and client manager only if we don't have them
if channel_id not in self.stream_buffers:
self.stream_buffers[channel_id] = StreamBuffer(channel_id, redis_client=self.redis_client)
self.stream_buffers[channel_id] = StreamBuffer(channel_id, redis_client=RedisClient.get_buffer())
if channel_id not in self.client_managers:
self.client_managers[channel_id] = ClientManager(
channel_id,
@ -507,7 +521,7 @@ class ProxyServer:
# Create buffer and client manager instances (or reuse if they exist)
if channel_id not in self.stream_buffers:
buffer = StreamBuffer(channel_id, redis_client=self.redis_client)
buffer = StreamBuffer(channel_id, redis_client=RedisClient.get_buffer())
self.stream_buffers[channel_id] = buffer
if channel_id not in self.client_managers:
@ -546,18 +560,18 @@ class ProxyServer:
# If no url was passed, try to get from Redis
if not url and existing_metadata:
url_bytes = existing_metadata.get(b'url')
url_bytes = existing_metadata.get('url')
if url_bytes:
channel_url = url_bytes.decode('utf-8')
channel_url = url_bytes
ua_bytes = existing_metadata.get(b'user_agent')
ua_bytes = existing_metadata.get('user_agent')
if ua_bytes:
channel_user_agent = ua_bytes.decode('utf-8')
channel_user_agent = ua_bytes
# Get stream ID from metadata if not provided
if not channel_stream_id and b'stream_id' in existing_metadata:
if not channel_stream_id and 'stream_id' in existing_metadata:
try:
channel_stream_id = int(existing_metadata[b'stream_id'].decode('utf-8'))
channel_stream_id = int(existing_metadata['stream_id'])
logger.debug(f"Found stream_id {channel_stream_id} in metadata for channel {channel_id}")
except (ValueError, TypeError) as e:
logger.debug(f"Could not parse stream_id from metadata: {e}")
@ -572,7 +586,7 @@ class ProxyServer:
# Create buffer but not stream manager (only if not already exists)
if channel_id not in self.stream_buffers:
buffer = StreamBuffer(channel_id=channel_id, redis_client=self.redis_client)
buffer = StreamBuffer(channel_id=channel_id, redis_client=RedisClient.get_buffer())
self.stream_buffers[channel_id] = buffer
# Create client manager with channel_id and redis_client (only if not already exists)
@ -595,7 +609,7 @@ class ProxyServer:
# Create buffer but not stream manager (only if not already exists)
if channel_id not in self.stream_buffers:
buffer = StreamBuffer(channel_id=channel_id, redis_client=self.redis_client)
buffer = StreamBuffer(channel_id=channel_id, redis_client=RedisClient.get_buffer())
self.stream_buffers[channel_id] = buffer
# Create client manager with channel_id and redis_client (only if not already exists)
@ -634,12 +648,12 @@ class ProxyServer:
# Verify the stream_id was set correctly in Redis
stream_id_value = self.redis_client.hget(metadata_key, "stream_id")
if stream_id_value:
logger.info(f"Verified stream_id {stream_id_value.decode('utf-8')} is set in Redis for channel {channel_id}")
logger.info(f"Verified stream_id {stream_id_value} is set in Redis for channel {channel_id}")
else:
logger.warning(f"Failed to set stream_id in Redis for channel {channel_id}")
# Create stream buffer
buffer = StreamBuffer(channel_id=channel_id, redis_client=self.redis_client)
buffer = StreamBuffer(channel_id=channel_id, redis_client=RedisClient.get_buffer())
logger.debug(f"Created StreamBuffer for channel {channel_id}")
self.stream_buffers[channel_id] = buffer
@ -733,8 +747,8 @@ class ProxyServer:
metadata = self.redis_client.hgetall(metadata_key)
# Get channel state and owner
state = metadata.get(b'state', b'unknown').decode('utf-8')
owner = metadata.get(b'owner', b'').decode('utf-8')
state = metadata.get('state', 'unknown')
owner = metadata.get('owner', '')
# States that indicate the channel is running properly or shutting down
valid_states = [ChannelState.ACTIVE, ChannelState.WAITING_FOR_CLIENTS,
@ -772,8 +786,8 @@ class ProxyServer:
return False
else:
# Unknown or initializing state, check how long it's been in this state
if b'state_changed_at' in metadata:
state_changed_at = float(metadata[b'state_changed_at'].decode('utf-8'))
if 'state_changed_at' in metadata:
state_changed_at = float(metadata['state_changed_at'])
state_age = time.time() - state_changed_at
# If in initializing state for too long, consider it stale
@ -811,8 +825,8 @@ class ProxyServer:
# If we have metadata, log details for debugging
if metadata:
state = metadata.get(b'state', b'unknown').decode('utf-8')
owner = metadata.get(b'owner', b'unknown').decode('utf-8')
state = metadata.get('state', 'unknown')
owner = metadata.get('owner', 'unknown')
logger.info(f"Zombie channel details - state: {state}, owner: {owner}")
# Clean up Redis keys
@ -937,16 +951,16 @@ class ProxyServer:
metadata = self.redis_client.hgetall(metadata_key)
if metadata:
# Calculate runtime from init_time
if b'init_time' in metadata:
if 'init_time' in metadata:
try:
init_time = float(metadata[b'init_time'].decode('utf-8'))
init_time = float(metadata['init_time'])
runtime = round(time.time() - init_time, 2)
except Exception:
pass
# Get total bytes transferred
if b'total_bytes' in metadata:
if 'total_bytes' in metadata:
try:
total_bytes = int(metadata[b'total_bytes'].decode('utf-8'))
total_bytes = int(metadata['total_bytes'])
except Exception:
pass
@ -1057,8 +1071,8 @@ class ProxyServer:
if self.redis_client:
metadata_key = RedisKeys.channel_metadata(channel_id)
metadata = self.redis_client.hgetall(metadata_key)
if metadata and b'state' in metadata:
channel_state = metadata[b'state'].decode('utf-8')
if metadata and 'state' in metadata:
channel_state = metadata['state']
# Check if channel has any clients left
total_clients = 0
@ -1090,9 +1104,9 @@ class ProxyServer:
# Get connection_ready_time from metadata (indicates if channel reached ready state)
connection_ready_time = None
if metadata and b'connection_ready_time' in metadata:
if metadata and 'connection_ready_time' in metadata:
try:
connection_ready_time = float(metadata[b'connection_ready_time'].decode('utf-8'))
connection_ready_time = float(metadata['connection_ready_time'])
except (ValueError, TypeError):
pass
@ -1104,15 +1118,15 @@ class ProxyServer:
attempt_value = self.redis_client.get(attempt_key)
if attempt_value:
try:
connection_attempt_time = float(attempt_value.decode('utf-8'))
connection_attempt_time = float(attempt_value)
except (ValueError, TypeError):
pass
# Also get init time as a fallback
init_time = None
if metadata and b'init_time' in metadata:
if metadata and 'init_time' in metadata:
try:
init_time = float(metadata[b'init_time'].decode('utf-8'))
init_time = float(metadata['init_time'])
except (ValueError, TypeError):
pass
@ -1183,7 +1197,7 @@ class ProxyServer:
disconnect_value = self.redis_client.get(disconnect_key)
if disconnect_value:
try:
disconnect_time = float(disconnect_value.decode('utf-8'))
disconnect_time = float(disconnect_value)
except (ValueError, TypeError) as e:
logger.error(f"Invalid disconnect time for channel {channel_id}: {e}")
@ -1304,7 +1318,7 @@ class ProxyServer:
for key in channel_keys:
try:
channel_id = key.decode('utf-8').split(':')[2]
channel_id = key.split(':')[2]
# Check if this channel has an owner
owner = self.get_channel_owner(channel_id)
@ -1349,7 +1363,7 @@ class ProxyServer:
for key in channel_keys:
try:
channel_id = key.decode('utf-8').split(':')[2]
channel_id = key.split(':')[2]
# Get metadata first
metadata = self.redis_client.hgetall(key)
@ -1364,7 +1378,7 @@ class ProxyServer:
continue
# Get owner
owner = metadata.get(b'owner', b'').decode('utf-8') if b'owner' in metadata else ''
owner = metadata.get('owner', '') if 'owner' in metadata else ''
# Check if owner is still alive
owner_alive = False
@ -1378,7 +1392,7 @@ class ProxyServer:
# If no owner and no clients, clean it up
if not owner_alive and client_count == 0:
state = metadata.get(b'state', b'unknown').decode('utf-8') if b'state' in metadata else 'unknown'
state = metadata.get('state', 'unknown')
logger.warning(f"Found orphaned metadata for channel {channel_id} (state: {state}, owner: {owner}, clients: {client_count}) - cleaning up")
# If we have it locally, stop it properly to clean up transcode/proxy processes
@ -1397,7 +1411,7 @@ class ProxyServer:
real_count = max(0, client_count - len(stale_ids))
if real_count <= 0:
# No real clients remain — safe to clean up.
state = metadata.get(b'state', b'unknown').decode('utf-8') if b'state' in metadata else 'unknown'
state = metadata.get('state', 'unknown')
logger.warning(
f"Orphaned channel {channel_id} (state: {state}, "
f"owner: {owner}) had {client_count} ghost client(s) "
@ -1492,8 +1506,8 @@ class ProxyServer:
# Get current state for logging
current_state = None
metadata = self.redis_client.hgetall(metadata_key)
if metadata and b'state' in metadata:
current_state = metadata[b'state'].decode('utf-8')
if metadata and 'state' in metadata:
current_state = metadata['state']
# Only update if state is actually changing
if current_state == new_state:

52
apps/proxy/ts_proxy/services/channel_service.py
View file

@ -61,7 +61,7 @@ class ChannelService:
# Verify the stream_id was set
stream_id_value = proxy_server.redis_client.hget(metadata_key, ChannelMetadataField.STREAM_ID)
if stream_id_value:
logger.debug(f"Verified stream_id {stream_id_value.decode('utf-8')} is now set in Redis")
logger.debug(f"Verified stream_id {stream_id_value} is now set in Redis")
else:
logger.error(f"Failed to set stream_id {stream_id} in Redis before initialization")
@ -131,7 +131,7 @@ class ChannelService:
try:
# This is inefficient but used for diagnostics - in production would use more targeted checks
redis_keys = proxy_server.redis_client.keys(f"ts_proxy:*:{channel_id}*")
redis_keys = [k.decode('utf-8') for k in redis_keys] if redis_keys else []
redis_keys = [k for k in redis_keys] if redis_keys else []
except Exception as e:
logger.error(f"Error checking Redis keys: {e}")
@ -168,15 +168,6 @@ class ChannelService:
else:
result = {'status': 'success'}
# Update metadata in Redis regardless of ownership
if proxy_server.redis_client:
try:
ChannelService._update_channel_metadata(channel_id, new_url, user_agent, stream_id, m3u_profile_id)
result['metadata_updated'] = True
except Exception as e:
logger.error(f"Error updating Redis metadata: {e}", exc_info=True)
result['metadata_updated'] = False
# If we're the owner, update directly
if proxy_server.am_i_owner(channel_id) and channel_id in proxy_server.stream_managers:
logger.info(f"This worker is the owner, changing stream URL for channel {channel_id}")
@ -187,14 +178,33 @@ class ChannelService:
success = manager.update_url(new_url, stream_id, m3u_profile_id)
logger.info(f"Stream URL changed from {old_url} to {new_url}, result: {success}")
# Update Redis metadata based on the actual outcome.
# On success, write the new values. On failure, restore whatever URL
# the manager will actually reconnect to (may be old_url if the
# exception happened before self.url was reassigned, or new_url if it
# happened after) so Redis never describes a URL that isn't in use.
if proxy_server.redis_client:
try:
if success:
ChannelService._update_channel_metadata(channel_id, new_url, user_agent, stream_id, m3u_profile_id)
else:
ChannelService._update_channel_metadata(channel_id, manager.url, user_agent)
result['metadata_updated'] = True
except Exception as e:
logger.error(f"Error updating Redis metadata: {e}", exc_info=True)
result['metadata_updated'] = False
result.update({
'direct_update': True,
'success': success,
'worker_id': proxy_server.worker_id
})
else:
# If we're not the owner, publish an event for the owner to pick up
logger.info(f"Not the owner, requesting URL change via Redis PubSub")
# Not the owner: publish the switch event. The owner will update metadata
# after the actual switch attempt succeeds (or roll back on failure).
# All needed info (url, user_agent, stream_id, m3u_profile_id) is carried
# in the pubsub message, so there is no reason to pre-write metadata here.
logger.debug(f"This worker is not the owner, publishing stream switch event for channel {channel_id}")
if proxy_server.redis_client:
ChannelService._publish_stream_switch_event(channel_id, new_url, user_agent, stream_id, m3u_profile_id)
result.update({
@ -236,8 +246,8 @@ class ChannelService:
metadata_key = RedisKeys.channel_metadata(channel_id)
try:
metadata = proxy_server.redis_client.hgetall(metadata_key)
if metadata and b'state' in metadata:
state = metadata[b'state'].decode('utf-8')
if metadata and 'state' in metadata:
state = metadata['state']
channel_info = {"state": state}
# Immediately mark as stopping in metadata so clients detect it faster
@ -382,8 +392,8 @@ class ChannelService:
metadata = proxy_server.redis_client.hgetall(metadata_key)
# Extract state and owner
state = metadata.get(ChannelMetadataField.STATE.encode(), b'unknown').decode('utf-8')
owner = metadata.get(ChannelMetadataField.OWNER.encode(), b'unknown').decode('utf-8')
state = metadata.get(ChannelMetadataField.STATE, 'unknown')
owner = metadata.get(ChannelMetadataField.OWNER, 'unknown')
# Valid states indicate channel is running properly
valid_states = [ChannelState.ACTIVE, ChannelState.WAITING_FOR_CLIENTS, ChannelState.CONNECTING]
@ -409,7 +419,7 @@ class ChannelService:
}
if last_data:
last_data_time = float(last_data.decode('utf-8'))
last_data_time = float(last_data)
data_age = time.time() - last_data_time
details["last_data_age"] = data_age
@ -432,13 +442,13 @@ class ChannelService:
try:
# Use factory to parse the line based on stream type
parsed_data = LogParserFactory.parse(stream_type, stream_info_line)
if not parsed_data:
return
# Update Redis and database with parsed data
ChannelService._update_stream_info_in_redis(
channel_id,
channel_id,
parsed_data.get('video_codec'),
parsed_data.get('resolution'),
parsed_data.get('width'),
@ -579,7 +589,7 @@ class ChannelService:
metadata_key = RedisKeys.channel_metadata(channel_id)
# First check if the key exists and what type it is
key_type = proxy_server.redis_client.type(metadata_key).decode('utf-8')
key_type = proxy_server.redis_client.type(metadata_key)
logger.debug(f"Redis key {metadata_key} is of type: {key_type}")
# Build metadata update dict

30
apps/proxy/ts_proxy/stream_generator.py
View file

@ -25,7 +25,7 @@ class StreamGenerator:
data delivery, and cleanup.
"""
def __init__(self, channel_id, client_id, client_ip, client_user_agent, channel_initializing=False):
def __init__(self, channel_id, client_id, client_ip, client_user_agent, channel_initializing=False, user=None):
"""
Initialize the stream generator with client and channel details.
@ -35,12 +35,14 @@ class StreamGenerator:
client_ip: Client's IP address
client_user_agent: User agent string from client
channel_initializing: Whether the channel is still initializing
user: Authenticated user making the request
"""
self.channel_id = channel_id
self.client_id = client_id
self.client_ip = client_ip
self.client_user_agent = client_user_agent
self.channel_initializing = channel_initializing
self.user = user
# Performance and state tracking
self.stream_start_time = time.time()
@ -112,7 +114,8 @@ class StreamGenerator:
channel_name=channel_obj.name,
client_ip=self.client_ip,
client_id=self.client_id,
user_agent=self.client_user_agent[:100] if self.client_user_agent else None
user_agent=self.client_user_agent[:100] if self.client_user_agent else None,
username=self.user.username if self.user else None
)
except Exception as e:
logger.error(f"Could not log client connect event: {e}")
@ -141,13 +144,13 @@ class StreamGenerator:
metadata_key = RedisKeys.channel_metadata(self.channel_id)
metadata = proxy_server.redis_client.hgetall(metadata_key)
if metadata and b'state' in metadata:
state = metadata[b'state'].decode('utf-8')
if metadata and 'state' in metadata:
state = metadata['state']
if state in ['waiting_for_clients', 'active']:
logger.info(f"[{self.client_id}] Channel {self.channel_id} now ready (state={state})")
return True
elif state in ['error', 'stopped', 'stopping']: # Added 'stopping' to error states
error_message = metadata.get(b'error_message', b'Unknown error').decode('utf-8')
error_message = metadata.get('error_message', 'Unknown error')
logger.error(f"[{self.client_id}] Channel {self.channel_id} in error state: {state}, message: {error_message}")
# Send error packet before giving up
yield create_ts_packet('error', f"Error: {error_message}")
@ -155,9 +158,9 @@ class StreamGenerator:
else:
# Improved logging to track initialization progress
init_time = "unknown"
if b'init_time' in metadata:
if 'init_time' in metadata:
try:
init_time_float = float(metadata[b'init_time'].decode('utf-8'))
init_time_float = float(metadata['init_time'])
init_duration = time.time() - init_time_float
init_time = f"{init_duration:.1f}s ago"
except:
@ -390,8 +393,8 @@ class StreamGenerator:
# Channel state in metadata
metadata_key = RedisKeys.channel_metadata(self.channel_id)
metadata = proxy_server.redis_client.hgetall(metadata_key)
if metadata and b'state' in metadata:
state = metadata[b'state'].decode('utf-8')
if metadata and 'state' in metadata:
state = metadata['state']
if state in ['error', 'stopped', 'stopping']:
logger.info(f"[{self.client_id}] Channel in {state} state, terminating stream")
return False
@ -555,8 +558,6 @@ class StreamGenerator:
if metadata:
stream_id_bytes = proxy_server.redis_client.hget(metadata_key, ChannelMetadataField.STREAM_ID)
if stream_id_bytes:
stream_id = int(stream_id_bytes.decode('utf-8'))
# Check if we're the last client
if self.channel_id in proxy_server.client_managers:
client_count = proxy_server.client_managers[self.channel_id].get_total_client_count()
@ -595,7 +596,8 @@ class StreamGenerator:
client_id=self.client_id,
user_agent=self.client_user_agent[:100] if self.client_user_agent else None,
duration=round(elapsed, 2),
bytes_sent=self.bytes_sent
bytes_sent=self.bytes_sent,
username=self.user.username if self.user else None
)
except Exception as e:
logger.error(f"Could not log client disconnect event: {e}")
@ -630,10 +632,10 @@ class StreamGenerator:
gevent.spawn(delayed_shutdown)
def create_stream_generator(channel_id, client_id, client_ip, client_user_agent, channel_initializing=False):
def create_stream_generator(channel_id, client_id, client_ip, client_user_agent, channel_initializing=False, user=None):
"""
Factory function to create a new stream generator.
Returns a function that can be passed to StreamingHttpResponse.
"""
generator = StreamGenerator(channel_id, client_id, client_ip, client_user_agent, channel_initializing)
generator = StreamGenerator(channel_id, client_id, client_ip, client_user_agent, channel_initializing, user=user)
return generator.generate

67
apps/proxy/ts_proxy/stream_manager.py
View file

@ -90,7 +90,7 @@ class StreamManager:
# Try to get stream_id specifically
stream_id_bytes = buffer.redis_client.hget(metadata_key, "stream_id")
if stream_id_bytes:
self.current_stream_id = int(stream_id_bytes.decode('utf-8'))
self.current_stream_id = int(stream_id_bytes)
self.tried_stream_ids.add(self.current_stream_id)
logger.info(f"Loaded stream ID {self.current_stream_id} from Redis for channel {buffer.channel_id}")
else:
@ -123,6 +123,12 @@ class StreamManager:
# Add HTTP reader thread property
self.http_reader = None
# Output bitrate smoothing / throttled DB persistence
self._smoothed_output_bitrate = None
self._last_bitrate_db_save_time = 0
self._bitrate_db_save_interval = 30 # seconds between DB writes
self._bitrate_warmup_samples = 10 # discard first N samples while EMA stabilizes (~5s)
def _create_session(self):
"""Create and configure requests session with optimal settings"""
session = requests.Session()
@ -413,7 +419,7 @@ class StreamManager:
is_owner = (
current_owner
and self.worker_id
and current_owner.decode('utf-8') == self.worker_id
and current_owner == self.worker_id
)
no_owner = current_owner is None
@ -423,7 +429,7 @@ class StreamManager:
metadata_key, ChannelMetadataField.STATE
)
current_state = (
current_state_bytes.decode('utf-8')
current_state_bytes
if current_state_bytes else None
)
should_update = current_state in ChannelState.PRE_ACTIVE
@ -773,13 +779,25 @@ class StreamManager:
if any(x is not None for x in [ffmpeg_speed, ffmpeg_fps, actual_fps, ffmpeg_output_bitrate]):
self._update_ffmpeg_stats_in_redis(ffmpeg_speed, ffmpeg_fps, actual_fps, ffmpeg_output_bitrate)
# Also save ffmpeg_output_bitrate to database if we have stream_id
# Update local EMA and periodically flush to database
if ffmpeg_output_bitrate is not None and self.current_stream_id:
from .services.channel_service import ChannelService
ChannelService._update_stream_stats_in_db(
self.current_stream_id,
ffmpeg_output_bitrate=ffmpeg_output_bitrate
)
if self._bitrate_warmup_samples > 0:
# Discard early samples from the EMA
self._bitrate_warmup_samples -= 1
else:
if self._smoothed_output_bitrate is None:
self._smoothed_output_bitrate = ffmpeg_output_bitrate
else:
self._smoothed_output_bitrate = 0.9 * self._smoothed_output_bitrate + 0.1 * ffmpeg_output_bitrate
now = time.time()
if now - self._last_bitrate_db_save_time >= self._bitrate_db_save_interval:
from .services.channel_service import ChannelService
ChannelService._update_stream_stats_in_db(
self.current_stream_id,
ffmpeg_output_bitrate=round(self._smoothed_output_bitrate, 1)
)
self._last_bitrate_db_save_time = now
# Fix the f-string formatting
actual_fps_str = f"{actual_fps:.1f}" if actual_fps is not None else "N/A"
@ -1057,6 +1075,20 @@ class StreamManager:
# Set running to false to ensure thread exits
self.running = False
# Flush the final bitrate to DB on stop only if warmup completed and we have
# a meaningful EMA. Short previews / channel hops that die during warmup do NOT
# write anything, preserving any previously correct value in the database.
if self._smoothed_output_bitrate is not None and self.current_stream_id:
final_bitrate = self._smoothed_output_bitrate
try:
from .services.channel_service import ChannelService
ChannelService._update_stream_stats_in_db(
self.current_stream_id,
ffmpeg_output_bitrate=round(final_bitrate, 1)
)
except Exception as e:
logger.debug(f"Error flushing final bitrate to DB for channel {self.channel_id}: {e}")
def update_url(self, new_url, stream_id=None, m3u_profile_id=None):
"""Update stream URL and reconnect with proper cleanup for both HTTP and transcode sessions"""
if new_url == self.url:
@ -1114,6 +1146,11 @@ class StreamManager:
self.url = new_url
self.connected = False
# Reset bitrate EMA on every URL change so stale data never carries over
self._smoothed_output_bitrate = None
self._last_bitrate_db_save_time = 0
self._bitrate_warmup_samples = 10
# Update stream ID if provided
if stream_id:
old_stream_id = self.current_stream_id
@ -1151,7 +1188,7 @@ class StreamManager:
logger.error(f"Error during URL update for channel {self.channel_id}: {e}", exc_info=True)
return False
finally:
# CRITICAL FIX: Always reset the URL switching flag when done, whether successful or not
# Always reset the URL switching flag when done, whether successful or not
self.url_switching = False
logger.info(f"Stream switch completed for channel {self.channel_id}")
@ -1503,9 +1540,9 @@ class StreamManager:
current_state = None
try:
metadata = redis_client.hgetall(metadata_key)
state_field = ChannelMetadataField.STATE.encode('utf-8')
state_field = ChannelMetadataField.STATE
if metadata and state_field in metadata:
current_state = metadata[state_field].decode('utf-8')
current_state = metadata[state_field]
except Exception as e:
logger.error(f"Error checking current state: {e}")
@ -1656,7 +1693,7 @@ class StreamManager:
new_user_agent = stream_info['user_agent']
new_transcode = stream_info['transcode']
# CRITICAL FIX: Check if the new URL is the same as current URL
# Check if the new URL is the same as current URL
# This can happen when current_stream_id is None and we accidentally select the same stream
if new_url == self.url:
logger.warning(f"Stream ID {stream_id} generates the same URL as current stream ({new_url}). "
@ -1665,7 +1702,7 @@ class StreamManager:
logger.info(f"Switching from URL {self.url} to {new_url} for channel {self.channel_id}")
# IMPORTANT: Just update the URL, don't stop the channel or release resources
# Just update the URL, don't stop the channel or release resources
switch_result = self.update_url(new_url, stream_id, profile_id)
if not switch_result:
logger.error(f"Failed to update URL for stream ID {stream_id} for channel {self.channel_id}")
@ -1710,4 +1747,4 @@ class StreamManager:
"""Safely reset the URL switching state if it gets stuck"""
self.url_switching = False
self.url_switch_start_time = 0
logger.info(f"Reset URL switching state for channel {self.channel_id}")
logger.info(f"Reset URL switching state for channel {self.channel_id}")

19
apps/proxy/ts_proxy/url_utils.py
View file

@ -3,7 +3,7 @@ Utilities for handling stream URLs and transformations.
"""
import logging
import re
import regex
from typing import Optional, Tuple, List
from django.shortcuts import get_object_or_404
from apps.channels.models import Channel, Stream
@ -146,13 +146,14 @@ def transform_url(input_url: str, search_pattern: str, replace_pattern: str) ->
logger.debug(f" base URL: {input_url}")
logger.debug(f" search: {search_pattern}")
# Handle backreferences in the replacement pattern
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', replace_pattern)
# Convert JS-style backreferences in replace pattern: $<name> -> \g<name>, $1 -> \1
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
logger.debug(f" replace: {replace_pattern}")
logger.debug(f" safe replace: {safe_replace_pattern}")
# Apply the transformation
stream_url = re.sub(search_pattern, safe_replace_pattern, input_url)
# Apply the transformation (regex module accepts JS-style (?<name>...) natively)
stream_url = regex.sub(search_pattern, safe_replace_pattern, input_url)
logger.info(f"Generated stream url: {stream_url}")
return stream_url
@ -211,9 +212,9 @@ def get_stream_info_for_switch(channel_id: str, target_stream_id: Optional[int]
existing_stream_id = redis_client.get(f"channel_stream:{channel.id}")
if existing_stream_id:
# Decode bytes to string/int for proper Redis key lookup
existing_stream_id = existing_stream_id.decode('utf-8')
existing_stream_id = existing_stream_id
existing_profile_id = redis_client.get(f"stream_profile:{existing_stream_id}")
if existing_profile_id and int(existing_profile_id.decode('utf-8')) == profile.id:
if existing_profile_id and int(existing_profile_id) == profile.id:
channel_using_profile = True
logger.debug(f"Channel {channel.id} already using profile {profile.id}")
@ -349,9 +350,9 @@ def get_alternate_streams(channel_id: str, current_stream_id: Optional[int] = No
existing_stream_id = redis_client.get(f"channel_stream:{channel.id}")
if existing_stream_id:
# Decode bytes to string/int for proper Redis key lookup
existing_stream_id = existing_stream_id.decode('utf-8')
existing_stream_id = existing_stream_id
existing_profile_id = redis_client.get(f"stream_profile:{existing_stream_id}")
if existing_profile_id and int(existing_profile_id.decode('utf-8')) == profile.id:
if existing_profile_id and int(existing_profile_id) == profile.id:
channel_using_profile = True
logger.debug(f"Channel {channel.id} already using profile {profile.id}")

4
apps/proxy/ts_proxy/utils.py
View file

@ -83,7 +83,7 @@ def create_ts_packet(packet_type='null', message=None):
# Add message to payload if provided
if message:
msg_bytes = message.encode('utf-8')
msg_bytes = message
packet[4:4+min(len(msg_bytes), 180)] = msg_bytes[:180]
return bytes(packet)
@ -113,4 +113,4 @@ def get_logger(component_name=None):
# Default if detection fails
logger_name = "ts_proxy"
return logging.getLogger(logger_name)
return logging.getLogger(logger_name)

48
apps/proxy/ts_proxy/views.py
View file

@ -19,6 +19,7 @@ from apps.m3u.models import M3UAccount, M3UAccountProfile
from apps.accounts.models import User
from core.models import UserAgent, CoreSettings, PROXY_PROFILE_NAME
from rest_framework.decorators import api_view, permission_classes
from rest_framework.permissions import AllowAny
from rest_framework.response import Response
from apps.accounts.permissions import (
IsAdmin,
@ -40,16 +41,21 @@ from .utils import get_logger
from uuid import UUID
import gevent
from dispatcharr.utils import network_access_allowed
from apps.proxy.utils import check_user_stream_limits
logger = get_logger()
@api_view(["GET"])
def stream_ts(request, channel_id):
@permission_classes([AllowAny])
def stream_ts(request, channel_id, user=None):
if not network_access_allowed(request, "STREAMS"):
return JsonResponse({"error": "Forbidden"}, status=403)
"""Stream TS data to client with immediate response and keep-alive packets during initialization"""
if user is None and hasattr(request, 'user') and request.user.is_authenticated:
user = request.user
channel = get_stream_object(channel_id)
client_user_agent = None
@ -71,6 +77,13 @@ def stream_ts(request, channel_id):
)
break
if user:
if not check_user_stream_limits(user, client_id, media_id=channel_id):
return JsonResponse(
{"error": f"Stream limit exceeded ({user.stream_limit} concurrent streams allowed)"},
status=429
)
# Check if we need to reinitialize the channel
needs_initialization = True
channel_state = None
@ -81,9 +94,9 @@ def stream_ts(request, channel_id):
metadata_key = RedisKeys.channel_metadata(channel_id)
if proxy_server.redis_client.exists(metadata_key):
metadata = proxy_server.redis_client.hgetall(metadata_key)
state_field = ChannelMetadataField.STATE.encode("utf-8")
state_field = ChannelMetadataField.STATE
if state_field in metadata:
channel_state = metadata[state_field].decode("utf-8")
channel_state = metadata[state_field]
# Active/running states - channel is operational, don't reinitialize
if channel_state in [
@ -119,9 +132,9 @@ def stream_ts(request, channel_id):
)
# Unknown/empty state - check if owner is alive
else:
owner_field = ChannelMetadataField.OWNER.encode("utf-8")
owner_field = ChannelMetadataField.OWNER
if owner_field in metadata:
owner = metadata[owner_field].decode("utf-8")
owner = metadata[owner_field]
owner_heartbeat_key = f"ts_proxy:worker:{owner}:heartbeat"
if proxy_server.redis_client.exists(owner_heartbeat_key):
# Owner is still active with unknown state - don't reinitialize
@ -399,7 +412,7 @@ def stream_ts(request, channel_id):
metadata_key, ChannelMetadataField.STATE
)
if state_bytes:
current_state = state_bytes.decode("utf-8")
current_state = state_bytes
logger.debug(
f"[{client_id}] Current state of channel {channel_id}: {current_state}"
)
@ -475,12 +488,12 @@ def stream_ts(request, channel_id):
)
if url_bytes:
url = url_bytes.decode("utf-8")
url = url_bytes
if ua_bytes:
stream_user_agent = ua_bytes.decode("utf-8")
stream_user_agent = ua_bytes
# Extract transcode setting from Redis
if profile_bytes:
profile_str = profile_bytes.decode("utf-8")
profile_str = profile_bytes
use_transcode = (
profile_str == PROXY_PROFILE_NAME or profile_str == "None"
)
@ -516,12 +529,12 @@ def stream_ts(request, channel_id):
# Register client
buffer = proxy_server.stream_buffers[channel_id]
client_manager = proxy_server.client_managers[channel_id]
client_manager.add_client(client_id, client_ip, client_user_agent)
client_manager.add_client(client_id, client_ip, client_user_agent, user)
logger.info(f"[{client_id}] Client registered with channel {channel_id}")
# Create a stream generator for this client
generate = create_stream_generator(
channel_id, client_id, client_ip, client_user_agent, channel_initializing
channel_id, client_id, client_ip, client_user_agent, channel_initializing, user=user
)
# Return the StreamingHttpResponse from the main function
@ -543,6 +556,7 @@ def stream_ts(request, channel_id):
@api_view(["GET"])
@permission_classes([AllowAny])
def stream_xc(request, username, password, channel_id):
user = get_object_or_404(User, username=username)
@ -557,7 +571,6 @@ def stream_xc(request, username, password, channel_id):
if custom_properties["xc_password"] != password:
return Response({"error": "Invalid credentials"}, status=401)
print(f"Fetchin channel with ID: {channel_id}")
if user.user_level < 10:
user_profile_count = user.channel_profiles.count()
@ -585,7 +598,7 @@ def stream_xc(request, username, password, channel_id):
channel = get_object_or_404(Channel, id=channel_id)
# @TODO: we've got the file 'type' via extension, support this when we support multiple outputs
return stream_ts(request._request, str(channel.uuid))
return stream_ts(request._request, str(channel.uuid), user)
@csrf_exempt
@ -713,7 +726,7 @@ def channel_status(request, channel_id=None):
)
for key in keys:
channel_id_match = re.search(
r"ts_proxy:channel:(.*):metadata", key.decode("utf-8")
r"ts_proxy:channel:(.*):metadata", key
)
if channel_id_match:
ch_id = channel_id_match.group(1)
@ -834,7 +847,7 @@ def next_stream(request, channel_id):
metadata_key, ChannelMetadataField.STREAM_ID
)
if stream_id_bytes:
current_stream_id = int(stream_id_bytes.decode("utf-8"))
current_stream_id = int(stream_id_bytes)
logger.info(
f"Found current stream ID {current_stream_id} in Redis for channel {channel_id}"
)
@ -844,7 +857,7 @@ def next_stream(request, channel_id):
metadata_key, ChannelMetadataField.M3U_PROFILE
)
if profile_id_bytes:
profile_id = int(profile_id_bytes.decode("utf-8"))
profile_id = int(profile_id_bytes)
logger.info(
f"Found M3U profile ID {profile_id} in Redis for channel {channel_id}"
)
@ -916,7 +929,8 @@ def next_stream(request, channel_id):
channel_id,
stream_info["url"],
stream_info["user_agent"],
next_stream_id, # Pass the stream_id to be stored in Redis
next_stream_id,
stream_info.get("m3u_profile_id"),
)
if result.get("status") == "error":

2
apps/proxy/urls.py
View file

@ -6,4 +6,4 @@ urlpatterns = [
path('ts/', include('apps.proxy.ts_proxy.urls')),
path('hls/', include('apps.proxy.hls_proxy.urls')),
path('vod/', include('apps.proxy.vod_proxy.urls')),
]
]

185
apps/proxy/utils.py Normal file
View file

@ -0,0 +1,185 @@
import logging
from core.utils import RedisClient
from apps.proxy.vod_proxy.multi_worker_connection_manager import MultiWorkerVODConnectionManager, get_vod_client_stop_key
from core.models import CoreSettings
from apps.proxy.ts_proxy.services.channel_service import ChannelService
logger = logging.getLogger("proxy")
def attempt_stream_termination(user_id, requesting_client_id, active_connections):
try:
logger.info("[stream limits]" f"[{requesting_client_id}] User {user_id} has {len(active_connections)} active connections, checking termination candidates")
user_limit_settings = CoreSettings.get_user_limits_settings()
terminate_oldest = user_limit_settings.get("terminate_oldest", True)
prioritize_single = user_limit_settings.get("prioritize_single_client_channels", True)
ignore_same_channel = user_limit_settings.get("ignore_same_channel_connections", False)
channel_counts = {}
for connection in active_connections:
media_id = connection['media_id']
channel_counts[media_id] = channel_counts.get(media_id, 0) + 1
def prioritize(connection):
is_multi = channel_counts[connection['media_id']] > 1
# if we're ignoring same-channel connections, put them at the end
same_ch_key = 1 if (ignore_same_channel and is_multi) else 0
# key for prioritizing single-client channels
single_key = 0 if (prioritize_single and not is_multi) else 1
# sort by age setting
time_key = connection['connected_at'] if terminate_oldest else -connection['connected_at']
return (same_ch_key, single_key, time_key)
termination_candidates = sorted(active_connections, key=prioritize)
if not termination_candidates:
logger.warning("[stream limits]" f"[{requesting_client_id}] No termination candidates found for user {user_id}")
return False
target = termination_candidates[0]
logger.info("[stream limits]"
f"[{requesting_client_id}] Terminating client {target['client_id']} "
f"on media {target['media_id']} (connected_at={target['connected_at']})"
)
# When counting by unique channel, freeing one connection from a multi-connection
# channel doesn't free a slot — terminate all connections to that channel so the
# unique-channel count actually drops by one.
targets = (
[c for c in active_connections if c['media_id'] == target['media_id']]
if ignore_same_channel
else [target]
)
for t in targets:
if t['type'] == 'live':
result = ChannelService.stop_client(t['media_id'], t['client_id'])
if result.get("status") == "error":
logger.warning(f"[stream limits][{requesting_client_id}] Failed to stop client {t['client_id']} on channel {t['media_id']}")
else:
connection_manager = MultiWorkerVODConnectionManager.get_instance()
redis_client = connection_manager.redis_client
if not redis_client:
return False
connection_key = f"vod_persistent_connection:{t['client_id']}"
connection_data = redis_client.hgetall(connection_key)
if not connection_data:
logger.warning(f"VOD connection not found: {t['client_id']}")
continue
stop_key = get_vod_client_stop_key(t['client_id'])
redis_client.setex(stop_key, 60, "true") # 60 second TTL
return True
except Exception as e:
logger.error("[stream limits]" f"[{requesting_client_id}] Error during stream termination for user {user_id}: {e}")
return False
def get_user_active_connections(user_id):
redis_client = RedisClient.get_client()
connections = []
try:
# Grab live streams
for key in redis_client.scan_iter(match="ts_proxy:channel:*:clients:*", count=1000):
parts = key.split(':')
if len(parts) >= 5:
channel_id = parts[2]
client_id = parts[4]
client_user_id = redis_client.hget(key, 'user_id')
connected_at = redis_client.hget(key, 'connected_at')
logger.debug(f"[stream limits] user_id = {user_id}")
logger.debug(f"[stream limits] channel_id = {channel_id}")
logger.debug(f"[stream limits] client_id = {client_id}")
if client_user_id and int(client_user_id) == user_id:
try:
logger.debug(f"[stream limits] Found LIVE connection for user {user_id} on channel {channel_id} with client ID {client_id}")
connected_at = float(connected_at) if connected_at else 0
connections.append({
'media_id': channel_id,
'client_id': client_id,
'connected_at': connected_at,
'type': 'live',
})
except (ValueError, TypeError):
pass
# Grab VOD
for key in redis_client.scan_iter(match="vod_persistent_connection:*", count=1000):
parts = key.split(':')
if len(parts) >= 2:
client_id = parts[1]
client_user_id = redis_client.hget(key, 'user_id')
connected_at = redis_client.hget(key, 'created_at')
content_uuid = redis_client.hget(key, 'content_uuid')
logger.debug(f"[stream limits] user_id = {user_id}")
logger.debug(f"[stream limits] client_id = {client_id}")
if client_user_id and int(client_user_id) == user_id:
try:
logger.debug(f"[stream limits] Found VOD connection for user {user_id} on content {content_uuid} with client ID {client_id}")
connected_at = float(connected_at) if connected_at else 0
connections.append({
'media_id': content_uuid or client_id,
'client_id': client_id,
'connected_at': connected_at,
'type': 'vod',
})
except (ValueError, TypeError):
pass
return connections
except Exception as e:
logger.warning(f"Error getting active channel details for user {user_id}: {e}")
return []
def check_user_stream_limits(user, client_id, media_id=None):
# Check user stream limits
if user and user.stream_limit > 0:
logger.debug("[stream limits]" f"[{client_id}] User {user.username} (ID: {user.id}) is requesting a stream (stream_limit: {user.stream_limit})")
user_limit_settings = CoreSettings.get_user_limits_settings()
ignore_same_channel = user_limit_settings.get("ignore_same_channel_connections", False)
active_connections = get_user_active_connections(user.id)
unique_channel_count = set([conn['media_id'] for conn in active_connections])
user_stream_count = len(unique_channel_count) if ignore_same_channel else len(active_connections)
logger.debug(f"[stream limits]" f"[{client_id}] User {user.username} currently has {len(active_connections)} active connections across {len(unique_channel_count)} unique channels (counting method: {'unique channels' if ignore_same_channel else 'total connections'})")
# If ignore_same_channel is enabled and this request is for a live channel the user
# is already watching, allow it through without counting against the limit.
# VOD is excluded: connections aren't shared so multiple VOD connections to the
# same content would mean multiple upstream connections.
live_channel_ids = {str(conn['media_id']) for conn in active_connections if conn['type'] == 'live'}
if ignore_same_channel and media_id and str(media_id) in live_channel_ids:
logger.debug(f"[stream limits][{client_id}] Same-channel reconnect for {media_id} allowed (ignore_same_channel=True)")
return True
if user_stream_count >= user.stream_limit:
if user_limit_settings.get("terminate_on_limit_exceeded", True) == False:
return False
if user_stream_count >= user.stream_limit:
logger.warning("[stream limits]"
f"[{client_id}] User {user.username} (ID: {user.id}) has reached stream limit "
f"({user_stream_count}/{user.stream_limit} streams), attempting to free up slot"
)
if not attempt_stream_termination(user.id, client_id, active_connections):
return False
return True

1505
apps/proxy/vod_proxy/connection_manager.py
View file

File diff suppressed because it is too large Load diff

160
apps/proxy/vod_proxy/multi_worker_connection_manager.py
View file

@ -93,7 +93,7 @@ class SerializableConnectionState:
content_name: str = None, client_ip: str = None,
client_user_agent: str = None, utc_start: str = None,
utc_end: str = None, offset: str = None,
worker_id: str = None, connection_type: str = "redis_backed"):
worker_id: str = None, connection_type: str = "redis_backed", user_id: str = "unknown"):
self.session_id = session_id
self.stream_url = stream_url
self.headers = headers
@ -104,6 +104,7 @@ class SerializableConnectionState:
self.last_activity = time.time()
self.request_count = 0
self.active_streams = 0
self.user_id = user_id
# Session metadata (consolidated from vod_session key)
self.content_obj_type = content_obj_type
@ -160,7 +161,8 @@ class SerializableConnectionState:
'last_seek_byte': str(self.last_seek_byte),
'last_seek_percentage': str(self.last_seek_percentage),
'total_content_size': str(self.total_content_size),
'last_seek_timestamp': str(self.last_seek_timestamp)
'last_seek_timestamp': str(self.last_seek_timestamp),
'user_id': str(self.user_id),
}
@classmethod
@ -184,7 +186,8 @@ class SerializableConnectionState:
utc_end=data.get('utc_end') or '',
offset=data.get('offset') or '',
worker_id=data.get('worker_id') or None,
connection_type=data.get('connection_type', 'redis_backed')
connection_type=data.get('connection_type', 'redis_backed'),
user_id=data.get('user_id', 'unknown')
)
obj.last_activity = float(data.get('last_activity', time.time()))
obj.request_count = int(data.get('request_count', 0))
@ -224,7 +227,7 @@ class RedisBackedVODConnection:
# Convert bytes keys/values to strings if needed
if isinstance(list(data.keys())[0], bytes):
data = {k.decode('utf-8'): v.decode('utf-8') for k, v in data.items()}
data = {k: v for k, v in data.items()}
return SerializableConnectionState.from_dict(data)
except Exception as e:
@ -281,7 +284,7 @@ class RedisBackedVODConnection:
content_name: str = None, client_ip: str = None,
client_user_agent: str = None, utc_start: str = None,
utc_end: str = None, offset: str = None,
worker_id: str = None) -> bool:
worker_id: str = None, user=None) -> bool:
"""Create a new connection state in Redis with consolidated session metadata"""
if not self._acquire_lock():
logger.warning(f"[{self.session_id}] Could not acquire lock for connection creation")
@ -309,7 +312,8 @@ class RedisBackedVODConnection:
utc_start=utc_start,
utc_end=utc_end,
offset=offset,
worker_id=worker_id
worker_id=worker_id,
user_id=user.id if user else "unknown"
)
success = self._save_connection_state(state)
@ -365,6 +369,24 @@ class RedisBackedVODConnection:
timeout=(10, 10),
allow_redirects=allow_redirects
)
# If the cached final_url returned an error (e.g. an ephemeral dispatcharr session
# that has since expired), clear it and retry from the original stream_url.
if response.status_code >= 400 and state.final_url:
logger.warning(
f"[{self.session_id}] Cached final_url returned {response.status_code}, "
f"clearing and retrying from stream_url"
)
response.close()
state.final_url = None
response = self.local_session.get(
state.stream_url,
headers=headers,
stream=True,
timeout=(10, 10),
allow_redirects=True
)
response.raise_for_status()
# Update state with response info on first request
@ -522,6 +544,38 @@ class RedisBackedVODConnection:
finally:
self._release_lock()
def decrement_active_streams_and_check(self):
"""Atomically decrement active streams and return (success, has_remaining_streams).
Combines decrement + check under a single lock to eliminate the race window
between separate decrement_active_streams() and has_active_streams() calls.
Returns:
(True, False) - decremented successfully, no streams remain
(True, True) - decremented successfully, other streams still active
(False, True) - lock contention, assume streams remain (safe default)
"""
if not self._acquire_lock():
logger.warning(f"[{self.session_id}] DECR-AS-CHECK failed: could not acquire lock")
return False, True # Assume remaining to avoid skipping profile decrement
try:
state = self._get_connection_state()
if state and state.active_streams > 0:
old = state.active_streams
state.active_streams -= 1
state.last_activity = time.time()
self._save_connection_state(state)
logger.debug(f"[{self.session_id}] DECR-AS {old} -> {state.active_streams}")
return True, state.active_streams > 0
if not state:
logger.warning(f"[{self.session_id}] DECR-AS-CHECK failed: no state")
return False, False
logger.warning(f"[{self.session_id}] DECR-AS-CHECK failed: active_streams already {state.active_streams}")
return False, False
finally:
self._release_lock()
def has_active_streams(self) -> bool:
"""Check if connection has any active streams"""
state = self._get_connection_state()
@ -744,24 +798,29 @@ class MultiWorkerVODConnectionManager:
return None
def _decrement_profile_connections(self, m3u_profile_id: int):
"""Decrement profile connection count"""
"""Decrement profile connection count.
Uses a single atomic DECR (no GET-before-DECR) to avoid the race condition
where two concurrent decrements both pass a >0 guard and both fire, sending
the counter negative. If the counter would go below zero it is clamped to 0.
"""
try:
profile_connections_key = self._get_profile_connections_key(m3u_profile_id)
current_count = int(self.redis_client.get(profile_connections_key) or 0)
if current_count > 0:
new_count = self.redis_client.decr(profile_connections_key)
logger.info(f"[PROFILE-DECR] Profile {m3u_profile_id} connections: {new_count}")
return new_count
new_count = self.redis_client.decr(profile_connections_key)
if new_count < 0:
self.redis_client.set(profile_connections_key, 0)
new_count = 0
logger.warning(f"[PROFILE-DECR] Profile {m3u_profile_id} counter went negative, clamped to 0")
else:
logger.warning(f"[PROFILE-DECR] Profile {m3u_profile_id} already at 0 connections")
return 0
logger.info(f"[PROFILE-DECR] Profile {m3u_profile_id} connections: {new_count}")
return new_count
except Exception as e:
logger.error(f"Error decrementing profile connections: {e}")
return None
def stream_content_with_session(self, session_id, content_obj, stream_url, m3u_profile,
client_ip, client_user_agent, request,
utc_start=None, utc_end=None, offset=None, range_header=None):
utc_start=None, utc_end=None, offset=None, range_header=None, user=None):
"""Stream content with Redis-backed persistent connection"""
# Generate client ID
@ -858,7 +917,8 @@ class MultiWorkerVODConnectionManager:
utc_start=utc_start,
utc_end=utc_end,
offset=str(offset) if offset else None,
worker_id=self.worker_id
worker_id=self.worker_id,
user=user
):
logger.error(f"[{client_id}] Worker {self.worker_id} - Failed to create Redis connection")
# Roll back the profile slot reservation since connection failed
@ -933,7 +993,8 @@ class MultiWorkerVODConnectionManager:
# Create streaming generator
def stream_generator():
decremented = False
stream_decremented = False
profile_decremented = False
stop_signal_detected = False
try:
logger.info(f"[{client_id}] Worker {self.worker_id} - Starting Redis-backed stream")
@ -986,16 +1047,16 @@ class MultiWorkerVODConnectionManager:
logger.info(f"[{client_id}] Worker {self.worker_id} - Stream stopped by signal: {bytes_sent} bytes sent")
else:
logger.info(f"[{client_id}] Worker {self.worker_id} - Redis-backed stream completed: {bytes_sent} bytes sent")
redis_connection.decrement_active_streams()
decremented = True
stream_decremented, has_remaining = redis_connection.decrement_active_streams_and_check()
# Schedule smart cleanup if no active streams after normal completion
if not redis_connection.has_active_streams():
if stream_decremented and not has_remaining and not profile_decremented:
# Decrement profile counter immediately — don't defer to daemon thread
state = redis_connection._get_connection_state()
profile_id = state.m3u_profile_id if state else m3u_profile.id
if profile_id:
self._decrement_profile_connections(profile_id)
profile_decremented = True
logger.info(f"[{client_id}] Profile counter decremented for profile {profile_id} on normal completion")
def delayed_cleanup():
@ -1012,17 +1073,19 @@ class MultiWorkerVODConnectionManager:
except GeneratorExit:
logger.info(f"[{client_id}] Worker {self.worker_id} - Client disconnected from Redis-backed stream")
if not decremented:
redis_connection.decrement_active_streams()
decremented = True
if not stream_decremented:
stream_decremented, has_remaining = redis_connection.decrement_active_streams_and_check()
else:
has_remaining = redis_connection.has_active_streams()
# Schedule smart cleanup if no active streams
if not redis_connection.has_active_streams():
if not has_remaining and not profile_decremented:
# Decrement profile counter immediately — don't defer to daemon thread
state = redis_connection._get_connection_state()
profile_id = state.m3u_profile_id if state else m3u_profile.id
if profile_id:
self._decrement_profile_connections(profile_id)
profile_decremented = True
logger.info(f"[{client_id}] Profile counter decremented for profile {profile_id} on client disconnect")
def delayed_cleanup():
@ -1039,16 +1102,18 @@ class MultiWorkerVODConnectionManager:
except Exception as e:
logger.error(f"[{client_id}] Worker {self.worker_id} - Error in Redis-backed stream: {e}")
if not decremented:
redis_connection.decrement_active_streams()
decremented = True
if not stream_decremented:
stream_decremented, has_remaining = redis_connection.decrement_active_streams_and_check()
else:
has_remaining = redis_connection.has_active_streams()
# Decrement profile counter immediately if no other active streams
if not redis_connection.has_active_streams():
if not has_remaining and not profile_decremented:
state = redis_connection._get_connection_state()
profile_id = state.m3u_profile_id if state else m3u_profile.id
if profile_id:
self._decrement_profile_connections(profile_id)
profile_decremented = True
logger.info(f"[{client_id}] Profile counter decremented for profile {profile_id} on stream error")
# Smart cleanup on error - immediate cleanup since we're in error state
# No connection_manager — profile already decremented above
@ -1056,8 +1121,31 @@ class MultiWorkerVODConnectionManager:
yield b"Error: Stream interrupted"
finally:
if not decremented:
redis_connection.decrement_active_streams()
if not stream_decremented:
stream_decremented, has_remaining = redis_connection.decrement_active_streams_and_check()
if stream_decremented and not has_remaining and not profile_decremented:
state = redis_connection._get_connection_state()
profile_id = state.m3u_profile_id if state else m3u_profile.id
if profile_id:
self._decrement_profile_connections(profile_id)
profile_decremented = True
logger.info(f"[{client_id}] Profile counter decremented for profile {profile_id} in finally block")
# Delayed cleanup: wait 1s for seeking clients to reconnect
# before closing the provider connection and Redis keys.
# cleanup() re-checks active_streams under lock, so a
# reconnecting client that increments active_streams in
# time will prevent Redis key deletion.
def delayed_cleanup():
time.sleep(1)
logger.info(f"[{client_id}] Worker {self.worker_id} - Checking for smart cleanup in finally block")
# No connection_manager — profile already decremented above
redis_connection.cleanup(current_worker_id=self.worker_id)
import threading
cleanup_thread = threading.Thread(target=delayed_cleanup)
cleanup_thread.daemon = True
cleanup_thread.start()
# Create streaming response
response = StreamingHttpResponse(
@ -1272,14 +1360,14 @@ class MultiWorkerVODConnectionManager:
# Convert bytes to strings if needed
if isinstance(list(data.keys())[0], bytes):
data = {k.decode('utf-8'): v.decode('utf-8') for k, v in data.items()}
data = {k: v for k, v in data.items()}
last_activity = float(data.get('last_activity', 0))
active_streams = int(data.get('active_streams', 0))
# Clean up if stale and no active streams
if (current_time - last_activity > max_age_seconds) and active_streams == 0:
session_id = key.decode('utf-8').replace('vod_persistent_connection:', '')
session_id = key.replace('vod_persistent_connection:', '')
logger.info(f"Cleaning up stale connection: {session_id}")
# Clean up connection and related keys
@ -1376,7 +1464,7 @@ class MultiWorkerVODConnectionManager:
if connection_data:
# Convert bytes to strings if needed
if isinstance(list(connection_data.keys())[0], bytes):
connection_data = {k.decode('utf-8'): v.decode('utf-8') for k, v in connection_data.items()}
connection_data = {k: v for k, v in connection_data.items()}
profile_id = connection_data.get('m3u_profile_id')
if profile_id:
@ -1438,7 +1526,7 @@ class MultiWorkerVODConnectionManager:
# Convert bytes keys/values to strings if needed
if isinstance(list(connection_data.keys())[0], bytes):
connection_data = {k.decode('utf-8'): v.decode('utf-8') for k, v in connection_data.items()}
connection_data = {k: v for k, v in connection_data.items()}
# Check if content matches (using consolidated data)
stored_content_type = connection_data.get('content_obj_type', '')
@ -1448,7 +1536,7 @@ class MultiWorkerVODConnectionManager:
continue
# Extract session ID
session_id = key.decode('utf-8').replace('vod_persistent_connection:', '')
session_id = key.replace('vod_persistent_connection:', '')
# Check if Redis-backed connection exists and has no active streams
redis_connection = RedisBackedVODConnection(session_id, self.redis_client)
@ -1526,4 +1614,4 @@ class MultiWorkerVODConnectionManager:
return redis_connection.get_session_metadata()
except Exception as e:
logger.error(f"Error getting session info for {session_id}: {e}")
return None
return None

0
apps/proxy/vod_proxy/tests/__init__.py Normal file
View file

253
apps/proxy/vod_proxy/tests/test_profile_connections.py Normal file
View file

@ -0,0 +1,253 @@
"""
Tests for VOD proxy profile connection counter fixes.
Covers three race conditions in multi_worker_connection_manager:
1. decrement_active_streams() return value was ignored — counter stuck on lock contention
2. Non-atomic GET-then-DECR in _decrement_profile_connections() — counter could go negative
3. has_active_streams() read without lock — race between decrement and check
"""
from unittest.mock import MagicMock, patch, call
from django.test import TestCase
class FakeRedis:
"""Minimal in-memory Redis stand-in for counter tests."""
def __init__(self):
self._data = {}
def get(self, key):
val = self._data.get(key)
return str(val).encode() if val is not None else None
def set(self, key, value, ex=None):
self._data[key] = int(value)
def incr(self, key):
self._data[key] = self._data.get(key, 0) + 1
return self._data[key]
def decr(self, key):
self._data[key] = self._data.get(key, 0) - 1
return self._data[key]
def delete(self, key):
self._data.pop(key, None)
def exists(self, key):
return key in self._data
def pipeline(self):
return FakePipeline(self)
class FakePipeline:
def __init__(self, redis):
self._redis = redis
self._cmds = []
def incr(self, key):
self._cmds.append(('incr', key))
return self
def decr(self, key):
self._cmds.append(('decr', key))
return self
def execute(self):
results = []
for cmd, key in self._cmds:
results.append(getattr(self._redis, cmd)(key))
self._cmds = []
return results
class MultiWorkerManagerImportMixin:
"""Mixin to import the manager class with patched Django/Redis deps."""
@classmethod
def get_manager_class(cls):
import importlib
import sys
# Stub out heavy Django deps so we can import the module standalone
for mod in ['apps.vod.models', 'apps.m3u.models', 'core.utils']:
if mod not in sys.modules:
sys.modules[mod] = MagicMock()
from apps.proxy.vod_proxy.multi_worker_connection_manager import (
MultiWorkerVODConnectionManager,
RedisBackedVODConnection,
)
return MultiWorkerVODConnectionManager, RedisBackedVODConnection
class TestDecrementProfileConnectionsAtomic(TestCase):
"""Bug 2: _decrement_profile_connections must be atomic (no GET-then-DECR)."""
def _make_manager(self, redis):
_, _ = MultiWorkerManagerImportMixin.get_manager_class()
from apps.proxy.vod_proxy.multi_worker_connection_manager import MultiWorkerVODConnectionManager
mgr = MultiWorkerVODConnectionManager.__new__(MultiWorkerVODConnectionManager)
mgr.redis_client = redis
mgr.worker_id = 'test-worker'
return mgr
def test_decrement_does_not_go_negative(self):
"""Counter must be clamped to 0, never go negative."""
redis = FakeRedis()
redis.set('profile_connections:1', 0)
mgr = self._make_manager(redis)
result = mgr._decrement_profile_connections(1)
self.assertEqual(result, 0)
self.assertEqual(int(redis._data.get('profile_connections:1', 0)), 0)
def test_decrement_from_one_reaches_zero(self):
"""Normal single decrement should reach 0."""
redis = FakeRedis()
redis.set('profile_connections:1', 1)
mgr = self._make_manager(redis)
result = mgr._decrement_profile_connections(1)
self.assertEqual(result, 0)
def test_concurrent_decrements_clamp_to_zero(self):
"""Two concurrent decrements of a counter at 1 must not leave it at -1."""
redis = FakeRedis()
redis.set('profile_connections:1', 1)
mgr = self._make_manager(redis)
# Simulate two concurrent decrements (both fire before either reads back)
mgr._decrement_profile_connections(1)
mgr._decrement_profile_connections(1)
final = int(redis._data.get('profile_connections:1', 0))
self.assertGreaterEqual(final, 0, "Counter must not go negative after concurrent decrements")
class TestDecrementActiveStreamsAndCheck(TestCase):
"""Bug 1 & 3: decrement_active_streams_and_check() must be atomic."""
def _make_connection(self, redis, session_id='test-session'):
from apps.proxy.vod_proxy.multi_worker_connection_manager import RedisBackedVODConnection
conn = RedisBackedVODConnection.__new__(RedisBackedVODConnection)
conn.session_id = session_id
conn.redis_client = redis
conn.connection_key = f'vod_connection:{session_id}'
conn.lock_key = f'vod_lock:{session_id}'
conn.local_session = None
conn._lock_acquired = False
return conn
def _make_state(self, active_streams=1, profile_id=7):
from apps.proxy.vod_proxy.multi_worker_connection_manager import SerializableConnectionState
state = SerializableConnectionState.__new__(SerializableConnectionState)
state.session_id = 'test-session'
state.stream_url = 'http://example.com/stream.mkv'
state.headers = {}
state.m3u_profile_id = profile_id
state.active_streams = active_streams
state.last_activity = 0
state.worker_id = 'test-worker'
state.content_type = None
state.content_length = None
state.final_url = None
state.request_count = 0
state.bytes_sent = 0
state.content_obj_type = None
state.content_uuid = None
state.content_name = None
state.client_ip = None
state.client_user_agent = None
state.utc_start = None
state.utc_end = None
state.offset = None
state.connection_type = 'redis'
state.created_at = 0
return state
def test_returns_success_and_no_remaining_when_last_stream(self):
"""When active_streams goes 1->0, should return (True, False)."""
from apps.proxy.vod_proxy.multi_worker_connection_manager import RedisBackedVODConnection
conn = MagicMock(spec=RedisBackedVODConnection)
conn.session_id = 'test'
state = MagicMock()
state.active_streams = 1
conn._acquire_lock.return_value = True
conn._get_connection_state.return_value = state
conn._save_connection_state.return_value = True
conn._release_lock.return_value = None
# Call the real method on the mock instance
result = RedisBackedVODConnection.decrement_active_streams_and_check(conn)
self.assertEqual(result, (True, False))
self.assertEqual(state.active_streams, 0)
def test_returns_success_and_remaining_when_other_streams_active(self):
"""When active_streams goes 2->1, should return (True, True)."""
from apps.proxy.vod_proxy.multi_worker_connection_manager import RedisBackedVODConnection
conn = MagicMock(spec=RedisBackedVODConnection)
conn.session_id = 'test'
state = MagicMock()
state.active_streams = 2
conn._acquire_lock.return_value = True
conn._get_connection_state.return_value = state
conn._save_connection_state.return_value = True
conn._release_lock.return_value = None
result = RedisBackedVODConnection.decrement_active_streams_and_check(conn)
self.assertEqual(result, (True, True))
self.assertEqual(state.active_streams, 1)
def test_returns_failure_and_assumes_remaining_on_lock_contention(self):
"""Lock contention must return (False, True) — assume streams remain to be safe."""
from apps.proxy.vod_proxy.multi_worker_connection_manager import RedisBackedVODConnection
conn = MagicMock(spec=RedisBackedVODConnection)
conn.session_id = 'test'
conn._acquire_lock.return_value = False
result = RedisBackedVODConnection.decrement_active_streams_and_check(conn)
self.assertEqual(result, (False, True))
conn._get_connection_state.assert_not_called()
def test_returns_failure_when_already_at_zero(self):
"""When active_streams is already 0, should return (False, False)."""
from apps.proxy.vod_proxy.multi_worker_connection_manager import RedisBackedVODConnection
conn = MagicMock(spec=RedisBackedVODConnection)
conn.session_id = 'test'
state = MagicMock()
state.active_streams = 0
conn._acquire_lock.return_value = True
conn._get_connection_state.return_value = state
conn._release_lock.return_value = None
result = RedisBackedVODConnection.decrement_active_streams_and_check(conn)
self.assertEqual(result, (False, False))
conn._save_connection_state.assert_not_called()
def test_lock_always_released_even_on_exception(self):
"""Lock must be released even if an exception occurs inside."""
from apps.proxy.vod_proxy.multi_worker_connection_manager import RedisBackedVODConnection
conn = MagicMock(spec=RedisBackedVODConnection)
conn.session_id = 'test'
conn._acquire_lock.return_value = True
conn._get_connection_state.side_effect = RuntimeError("Redis exploded")
with self.assertRaises(RuntimeError):
RedisBackedVODConnection.decrement_active_streams_and_check(conn)
conn._release_lock.assert_called_once()

18
apps/proxy/vod_proxy/urls.py
View file

@ -1,26 +1,20 @@
from django.urls import path
from . import views
from .views import stream_vod
app_name = 'vod_proxy'
urlpatterns = [
# Generic VOD streaming with session ID in path (for compatibility)
path('<str:content_type>/<uuid:content_id>/<str:session_id>', views.VODStreamView.as_view(), name='vod_stream_with_session'),
path('<str:content_type>/<uuid:content_id>/<str:session_id>/<int:profile_id>/', views.VODStreamView.as_view(), name='vod_stream_with_session_and_profile'),
path('<str:content_type>/<uuid:content_id>/<str:session_id>', stream_vod, name='vod_stream_with_session'),
path('<str:content_type>/<uuid:content_id>/<str:session_id>/<int:profile_id>/', stream_vod, name='vod_stream_with_session_and_profile'),
# Generic VOD streaming (supports movies, episodes, series) - legacy patterns
path('<str:content_type>/<uuid:content_id>', views.VODStreamView.as_view(), name='vod_stream'),
path('<str:content_type>/<uuid:content_id>/<int:profile_id>/', views.VODStreamView.as_view(), name='vod_stream_with_profile'),
# VOD playlist generation
path('playlist/', views.VODPlaylistView.as_view(), name='vod_playlist'),
path('playlist/<int:profile_id>/', views.VODPlaylistView.as_view(), name='vod_playlist_with_profile'),
# Position tracking
path('position/<uuid:content_id>/', views.VODPositionView.as_view(), name='vod_position'),
path('<str:content_type>/<uuid:content_id>', stream_vod, name='vod_stream'),
path('<str:content_type>/<uuid:content_id>/<int:profile_id>/', stream_vod, name='vod_stream_with_profile'),
# VOD Stats
path('stats/', views.VODStatsView.as_view(), name='vod_stats'),
path('stats/', views.vod_stats, name='vod_stats'),
# Stop VOD client connection
path('stop_client/', views.stop_vod_client, name='stop_vod_client'),

1824
apps/proxy/vod_proxy/views.py
View file

File diff suppressed because it is too large Load diff

83
apps/vod/api_views.py
View file

@ -11,6 +11,7 @@ from django.db.models import Q
import django_filters
import logging
import os
import time
import requests
from apps.accounts.permissions import (
Authenticated,
@ -36,6 +37,11 @@ from datetime import timedelta
logger = logging.getLogger(__name__)
# Negative cache for remote VOD logo URLs that failed to fetch.
# Prevents repeated blocking requests to unreachable hosts.
_vod_logo_fetch_failures = {}
_VOD_LOGO_FAIL_TTL = 300 # seconds
class VODPagination(PageNumberPagination):
page_size = 20 # Default page size to match frontend default
@ -578,13 +584,15 @@ class UnifiedContentViewSet(viewsets.ReadOnlyModelViewSet):
"series.id IN (SELECT DISTINCT series_id FROM vod_m3useriesrelation msr JOIN m3u_m3uaccount ma ON msr.m3u_account_id = ma.id WHERE ma.is_active = true)"
]
params = []
movie_params = []
series_params = []
if search:
where_conditions[0] += " AND LOWER(movies.name) LIKE %s"
where_conditions[1] += " AND LOWER(series.name) LIKE %s"
search_param = f"%{search.lower()}%"
params.extend([search_param, search_param])
movie_params.append(search_param)
series_params.append(search_param)
if category:
if '|' in category:
@ -592,15 +600,20 @@ class UnifiedContentViewSet(viewsets.ReadOnlyModelViewSet):
if cat_type == 'movie':
where_conditions[0] += " AND movies.id IN (SELECT movie_id FROM vod_m3umovierelation mmr JOIN vod_vodcategory c ON mmr.category_id = c.id WHERE c.name = %s)"
where_conditions[1] = "1=0" # Exclude series
params.append(cat_name)
movie_params.append(cat_name)
series_params = [] # no params needed for "1=0"
elif cat_type == 'series':
where_conditions[1] += " AND series.id IN (SELECT series_id FROM vod_m3useriesrelation msr JOIN vod_vodcategory c ON msr.category_id = c.id WHERE c.name = %s)"
where_conditions[0] = "1=0" # Exclude movies
params.append(cat_name)
series_params.append(cat_name)
movie_params = [] # no params needed for "1=0"
else:
where_conditions[0] += " AND movies.id IN (SELECT movie_id FROM vod_m3umovierelation mmr JOIN vod_vodcategory c ON mmr.category_id = c.id WHERE c.name = %s)"
where_conditions[1] += " AND series.id IN (SELECT series_id FROM vod_m3useriesrelation msr JOIN vod_vodcategory c ON msr.category_id = c.id WHERE c.name = %s)"
params.extend([category, category])
movie_params.append(category)
series_params.append(category)
params = movie_params + series_params
# Use UNION ALL with ORDER BY and LIMIT/OFFSET for true unified pagination
# This is much more efficient than Python sorting
@ -823,17 +836,62 @@ class VODLogoViewSet(viewsets.ModelViewSet):
return HttpResponse(status=500)
else:
# It's a remote URL - proxy it
# Skip URLs that recently failed to avoid blocking workers
fail_expiry = _vod_logo_fetch_failures.get(logo.url)
if fail_expiry and time.monotonic() < fail_expiry:
return HttpResponse(status=404)
try:
response = requests.get(logo.url, stream=True, timeout=10)
response.raise_for_status()
_LOGO_TOTAL_TIMEOUT = 10 # seconds
_LOGO_MAX_BYTES = 5 * 1024 * 1024 # 5 MB
content_type = response.headers.get('Content-Type', 'image/png')
return StreamingHttpResponse(
response.iter_content(chunk_size=8192),
content_type=content_type
remote_response = requests.get(
logo.url,
stream=True,
timeout=(3, 5), # (connect_timeout, read_timeout per chunk)
)
if remote_response.status_code != 200:
now = time.monotonic()
_vod_logo_fetch_failures[logo.url] = now + _VOD_LOGO_FAIL_TTL
return HttpResponse(status=404)
# Eagerly read the full image with a total time + size cap
# so the greenlet is released quickly.
chunks = []
total = 0
deadline = time.monotonic() + _LOGO_TOTAL_TIMEOUT
for chunk in remote_response.iter_content(chunk_size=8192):
total += len(chunk)
if total > _LOGO_MAX_BYTES:
remote_response.close()
return HttpResponse(status=404)
if time.monotonic() > deadline:
remote_response.close()
now = time.monotonic()
_vod_logo_fetch_failures[logo.url] = now + _VOD_LOGO_FAIL_TTL
return HttpResponse(status=404)
chunks.append(chunk)
body = b"".join(chunks)
# Full read succeeded, clear any previous failure entry
_vod_logo_fetch_failures.pop(logo.url, None)
content_type = remote_response.headers.get('Content-Type', 'image/png')
response = HttpResponse(body, content_type=content_type)
response["Content-Length"] = str(len(body))
if remote_response.headers.get("Cache-Control"):
response["Cache-Control"] = remote_response.headers.get("Cache-Control")
if remote_response.headers.get("Last-Modified"):
response["Last-Modified"] = remote_response.headers.get("Last-Modified")
response["Content-Disposition"] = 'inline; filename="{}"'.format(
os.path.basename(logo.url)
)
return response
except requests.exceptions.RequestException as e:
now = time.monotonic()
_vod_logo_fetch_failures[logo.url] = now + _VOD_LOGO_FAIL_TTL
logger.error(f"Error fetching remote VOD logo {logo.url}: {str(e)}")
return HttpResponse(status=404)
@ -896,4 +954,3 @@ class VODLogoViewSet(viewsets.ModelViewSet):
{"error": str(e)},
status=status.HTTP_500_INTERNAL_SERVER_ERROR
)

47
core/api_views.py
View file

@ -3,12 +3,13 @@
import json
import ipaddress
import logging
from django.conf import settings as django_settings
from django.db import models
from rest_framework import viewsets, status
from rest_framework.response import Response
from rest_framework.views import APIView
from django.shortcuts import get_object_or_404
from rest_framework.permissions import IsAuthenticated
from rest_framework.permissions import IsAuthenticated, AllowAny
from rest_framework.decorators import api_view, permission_classes, action
from drf_spectacular.utils import extend_schema, OpenApiParameter
from drf_spectacular.types import OpenApiTypes
@ -34,6 +35,9 @@ import os
from core.tasks import rehash_streams
from apps.accounts.permissions import (
Authenticated,
IsAdmin,
IsStandardUser,
permission_classes_by_action,
)
from dispatcharr.utils import get_client_ip
@ -49,6 +53,12 @@ class UserAgentViewSet(viewsets.ModelViewSet):
queryset = UserAgent.objects.all()
serializer_class = UserAgentSerializer
def get_permissions(self):
try:
return [perm() for perm in permission_classes_by_action[self.action]]
except KeyError:
return [Authenticated()]
class StreamProfileViewSet(viewsets.ModelViewSet):
"""
@ -58,6 +68,12 @@ class StreamProfileViewSet(viewsets.ModelViewSet):
queryset = StreamProfile.objects.all()
serializer_class = StreamProfileSerializer
def get_permissions(self):
try:
return [perm() for perm in permission_classes_by_action[self.action]]
except KeyError:
return [Authenticated()]
class CoreSettingsViewSet(viewsets.ModelViewSet):
"""
@ -68,6 +84,12 @@ class CoreSettingsViewSet(viewsets.ModelViewSet):
queryset = CoreSettings.objects.all()
serializer_class = CoreSettingsSerializer
def get_permissions(self):
try:
return [perm() for perm in permission_classes_by_action[self.action]]
except KeyError:
return [Authenticated()]
def update(self, request, *args, **kwargs):
instance = self.get_object()
old_value = instance.value
@ -170,6 +192,11 @@ class ProxySettingsViewSet(viewsets.ViewSet):
"""
serializer_class = ProxySettingsSerializer
def get_permissions(self):
if self.action in ('list', 'retrieve'):
return [IsStandardUser()]
return [IsAdmin()]
def _get_or_create_settings(self):
"""Get or create the proxy settings CoreSettings entry"""
try:
@ -301,7 +328,9 @@ def environment(request):
country_code = None
country_name = None
# 4) Get environment mode from system environment variable
# 4) Get environment mode and TLS status from settings
postgres_ssl = getattr(django_settings, "POSTGRES_SSL", False)
return Response(
{
"authenticated": True,
@ -309,7 +338,17 @@ def environment(request):
"local_ip": local_ip,
"country_code": country_code,
"country_name": country_name,
"env_mode": "dev" if os.getenv("DISPATCHARR_ENV") == "dev" else "prod",
"env_mode": os.getenv("DISPATCHARR_ENV", "aio"),
"redis_tls": {
"enabled": getattr(django_settings, "REDIS_SSL", False),
"verify": getattr(django_settings, "REDIS_SSL_VERIFY", True),
"mtls": bool(getattr(django_settings, "REDIS_SSL_CERT", "") and getattr(django_settings, "REDIS_SSL_KEY", "")),
},
"postgres_tls": {
"enabled": postgres_ssl,
"ssl_mode": getattr(django_settings, "POSTGRES_SSL_MODE", "verify-full") if postgres_ssl else None,
"mtls": bool(getattr(django_settings, "POSTGRES_SSL_CERT", "") and getattr(django_settings, "POSTGRES_SSL_KEY", "")),
},
}
)
@ -317,8 +356,8 @@ def environment(request):
@extend_schema(
description="Get application version information",
)
@api_view(["GET"])
@permission_classes([AllowAny])
def version(request):
# Import version information
from version import __version__, __timestamp__

14
core/management/commands/dropdb.py
View file

@ -1,5 +1,6 @@
import sys
import psycopg2
from psycopg2 import sql
from django.core.management.base import BaseCommand
from django.conf import settings
from django.db import connection
@ -15,6 +16,13 @@ class Command(BaseCommand):
host = db_settings.get('HOST', 'localhost')
port = db_settings.get('PORT', 5432)
# Read TLS parameters from Django OPTIONS (populated when POSTGRES_SSL=true)
db_options = db_settings.get('OPTIONS', {})
ssl_kwargs = {}
for key in ('sslmode', 'sslrootcert', 'sslcert', 'sslkey'):
if key in db_options:
ssl_kwargs[key] = db_options[key]
self.stdout.write(self.style.WARNING(
f"WARNING: This will irreversibly drop the entire database '{db_name}'!"
))
@ -30,13 +38,13 @@ class Command(BaseCommand):
maintenance_db = 'postgres'
try:
self.stdout.write("Connecting to maintenance database...")
conn = psycopg2.connect(dbname=maintenance_db, user=user, password=password, host=host, port=port)
conn = psycopg2.connect(dbname=maintenance_db, user=user, password=password, host=host, port=port, **ssl_kwargs)
conn.autocommit = True
cur = conn.cursor()
self.stdout.write(f"Dropping database '{db_name}'...")
cur.execute(f"DROP DATABASE IF EXISTS {db_name};")
cur.execute(sql.SQL("DROP DATABASE IF EXISTS {}").format(sql.Identifier(db_name)))
self.stdout.write(f"Creating database '{db_name}'...")
cur.execute(f"CREATE DATABASE {db_name};")
cur.execute(sql.SQL("CREATE DATABASE {}").format(sql.Identifier(db_name)))
cur.close()
conn.close()
self.stdout.write(self.style.SUCCESS(f"Database '{db_name}' has been dropped and recreated."))

24
core/migrations/022_default_user_limit_settings.py Normal file
View file

@ -0,0 +1,24 @@
# Generated by Django 5.1.6 on 2025-03-01 14:01
from django.db import migrations
from django.utils.text import slugify
def preload_user_limit_settings(apps, schema_editor):
CoreSettings = apps.get_model("core", "CoreSettings")
CoreSettings.objects.create(
key="user_limit_settings",
name="User Limit Settings",
value={},
)
class Migration(migrations.Migration):
dependencies = [
("core", "0021_systemnotification_notificationdismissal"),
]
operations = [
migrations.RunPython(preload_user_limit_settings),
]

10
core/models.py
View file

@ -156,6 +156,7 @@ PROXY_SETTINGS_KEY = "proxy_settings"
NETWORK_ACCESS_KEY = "network_access"
SYSTEM_SETTINGS_KEY = "system_settings"
EPG_SETTINGS_KEY = "epg_settings"
USER_LIMITS_SETTINGS_KEY = "user_limit_settings"
class CoreSettings(models.Model):
@ -362,6 +363,15 @@ class CoreSettings(models.Model):
cls._update_group(SYSTEM_SETTINGS_KEY, "System Settings", {"time_zone": value})
return value
@classmethod
def get_user_limits_settings(cls):
return cls._get_group(USER_LIMITS_SETTINGS_KEY, {
"terminate_on_limit_exceeded": True,
"prioritize_single_client_channels": True,
"ignore_same_channel_connections": False,
"terminate_oldest": True,
})
class SystemEvent(models.Model):
"""

4
core/redis_pubsub.py
View file

@ -201,10 +201,6 @@ class RedisPubSubManager:
channel = message.get('channel')
if channel:
# Decode binary channel name if needed
if isinstance(channel, bytes):
channel = channel.decode('utf-8')
# Find and call the appropriate handler
handler = self.message_handlers.get(channel)
if handler:

18
core/tasks.py
View file

@ -13,7 +13,7 @@ from apps.epg.models import EPGSource
from apps.m3u.tasks import refresh_single_m3u_account
from apps.epg.tasks import refresh_epg_data
from .models import CoreSettings
from apps.channels.models import Stream, ChannelStream
from apps.channels.models import ChannelStream
from django.db import transaction
logger = logging.getLogger(__name__)
@ -404,7 +404,7 @@ def fetch_channel_stats():
while True:
cursor, keys = redis_client.scan(cursor, match=channel_pattern)
for key in keys:
channel_id_match = re.search(r"ts_proxy:channel:(.*):metadata", key.decode('utf-8'))
channel_id_match = re.search(r"ts_proxy:channel:(.*):metadata", key)
if channel_id_match:
ch_id = channel_id_match.group(1)
channel_info = ChannelStatus.get_basic_channel_info(ch_id)
@ -753,20 +753,6 @@ def _determine_stream_to_keep(stream_a, stream_b):
return (stream_b, stream_a)
@shared_task
def cleanup_vod_persistent_connections():
"""Clean up stale VOD persistent connections"""
try:
from apps.proxy.vod_proxy.connection_manager import VODConnectionManager
# Clean up connections older than 30 minutes
VODConnectionManager.cleanup_stale_persistent_connections(max_age_seconds=1800)
logger.info("VOD persistent connection cleanup completed")
except Exception as e:
logger.error(f"Error during VOD persistent connection cleanup: {e}")
@shared_task
def check_for_version_update():
"""

73
core/tests.py
View file

@ -1,3 +1,5 @@
from unittest.mock import patch, MagicMock
from django.test import TestCase
from core.models import CoreSettings, DVR_SETTINGS_KEY, EPG_SETTINGS_KEY
@ -148,3 +150,74 @@ class EpgIgnoreListsTest(TestCase):
]:
self._set_epg_field_raw(field, "not a list")
self.assertEqual(getter(), [])
class DropDBCommandTlsTest(TestCase):
"""Verify dropdb management command passes TLS parameters to psycopg2."""
databases = []
_DB_WITH_TLS = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'testdb',
'USER': 'testuser',
'PASSWORD': 'testpass',
'HOST': 'localhost',
'PORT': 5432,
'OPTIONS': {
'sslmode': 'verify-full',
'sslrootcert': '/certs/ca.crt',
'sslcert': '/certs/client.crt',
'sslkey': '/certs/client.key',
},
}
}
_DB_NO_TLS = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'testdb',
'USER': 'testuser',
'PASSWORD': 'testpass',
'HOST': 'localhost',
'PORT': 5432,
}
}
@patch('core.management.commands.dropdb.psycopg2.connect')
@patch('core.management.commands.dropdb.connection')
@patch('builtins.input', return_value='yes')
def test_dropdb_passes_ssl_kwargs_when_tls_enabled(self, _inp, _conn, mock_connect):
mock_pg = MagicMock()
mock_connect.return_value = mock_pg
mock_pg.cursor.return_value = MagicMock()
with self.settings(DATABASES=self._DB_WITH_TLS):
from django.core.management import call_command
call_command('dropdb')
mock_connect.assert_called_once_with(
dbname='postgres', user='testuser', password='testpass',
host='localhost', port=5432,
sslmode='verify-full',
sslrootcert='/certs/ca.crt',
sslcert='/certs/client.crt',
sslkey='/certs/client.key',
)
@patch('core.management.commands.dropdb.psycopg2.connect')
@patch('core.management.commands.dropdb.connection')
@patch('builtins.input', return_value='yes')
def test_dropdb_no_ssl_kwargs_when_tls_disabled(self, _inp, _conn, mock_connect):
mock_pg = MagicMock()
mock_connect.return_value = mock_pg
mock_pg.cursor.return_value = MagicMock()
with self.settings(DATABASES=self._DB_NO_TLS):
from django.core.management import call_command
call_command('dropdb')
mock_connect.assert_called_once_with(
dbname='postgres', user='testuser', password='testpass',
host='localhost', port=5432,
)

212
core/utils.py
View file

@ -3,6 +3,7 @@ import logging
import time
import os
import threading
from pathlib import Path
import re
from django.conf import settings
from redis.exceptions import ConnectionError, TimeoutError
@ -13,6 +14,8 @@ from django.core.validators import URLValidator
from django.core.exceptions import ValidationError
import gc
_REDIS_TLS_HINT = " (TLS is enabled — verify certificate paths and that Redis is configured for TLS)"
logger = logging.getLogger(__name__)
# Import the command detector
@ -43,103 +46,116 @@ def natural_sort_key(text):
class RedisClient:
_client = None
_buffer = None
_pubsub_client = None
@classmethod
def get_client(cls, max_retries=5, retry_interval=1):
if cls._client is None:
retry_count = 0
while retry_count < max_retries:
def _init_client(cls, decode_responses=True, max_retries=5, retry_interval=1):
retry_count = 0
while retry_count < max_retries:
try:
# Get connection parameters from settings or environment
redis_host = os.environ.get("REDIS_HOST", getattr(settings, 'REDIS_HOST', 'localhost'))
redis_port = int(os.environ.get("REDIS_PORT", getattr(settings, 'REDIS_PORT', 6379)))
redis_db = int(os.environ.get("REDIS_DB", getattr(settings, 'REDIS_DB', 0)))
redis_password = os.environ.get("REDIS_PASSWORD", getattr(settings, 'REDIS_PASSWORD', ''))
redis_user = os.environ.get("REDIS_USER", getattr(settings, 'REDIS_USER', ''))
# Use standardized settings
socket_timeout = getattr(settings, 'REDIS_SOCKET_TIMEOUT', 5)
socket_connect_timeout = getattr(settings, 'REDIS_SOCKET_CONNECT_TIMEOUT', 5)
health_check_interval = getattr(settings, 'REDIS_HEALTH_CHECK_INTERVAL', 30)
socket_keepalive = getattr(settings, 'REDIS_SOCKET_KEEPALIVE', True)
retry_on_timeout = getattr(settings, 'REDIS_RETRY_ON_TIMEOUT', True)
# TLS params from settings (empty dict when TLS is disabled)
ssl_params = getattr(settings, 'REDIS_SSL_PARAMS', {})
# Create Redis client with better defaults
client = redis.Redis(
host=redis_host,
port=redis_port,
db=redis_db,
password=redis_password if redis_password else None,
username=redis_user if redis_user else None,
socket_timeout=socket_timeout,
socket_connect_timeout=socket_connect_timeout,
socket_keepalive=socket_keepalive,
health_check_interval=health_check_interval,
retry_on_timeout=retry_on_timeout,
decode_responses=decode_responses,
**ssl_params
)
# Validate connection with ping
client.ping()
# Disable persistence on first connection - improves performance
# Only try to disable if not in a read-only environment
try:
# Get connection parameters from settings or environment
redis_host = os.environ.get("REDIS_HOST", getattr(settings, 'REDIS_HOST', 'localhost'))
redis_port = int(os.environ.get("REDIS_PORT", getattr(settings, 'REDIS_PORT', 6379)))
redis_db = int(os.environ.get("REDIS_DB", getattr(settings, 'REDIS_DB', 0)))
redis_password = os.environ.get("REDIS_PASSWORD", getattr(settings, 'REDIS_PASSWORD', ''))
redis_user = os.environ.get("REDIS_USER", getattr(settings, 'REDIS_USER', ''))
client.config_set('save', '') # Disable RDB snapshots
client.config_set('appendonly', 'no') # Disable AOF logging
# Use standardized settings
socket_timeout = getattr(settings, 'REDIS_SOCKET_TIMEOUT', 5)
socket_connect_timeout = getattr(settings, 'REDIS_SOCKET_CONNECT_TIMEOUT', 5)
health_check_interval = getattr(settings, 'REDIS_HEALTH_CHECK_INTERVAL', 30)
socket_keepalive = getattr(settings, 'REDIS_SOCKET_KEEPALIVE', True)
retry_on_timeout = getattr(settings, 'REDIS_RETRY_ON_TIMEOUT', True)
# Disable protected mode when in debug mode
if os.environ.get('DISPATCHARR_DEBUG', '').lower() == 'true':
client.config_set('protected-mode', 'no') # Disable protected mode in debug
logger.warning("Redis protected mode disabled for debug environment")
# Create Redis client with better defaults
client = redis.Redis(
host=redis_host,
port=redis_port,
db=redis_db,
password=redis_password if redis_password else None,
username=redis_user if redis_user else None,
socket_timeout=socket_timeout,
socket_connect_timeout=socket_connect_timeout,
socket_keepalive=socket_keepalive,
health_check_interval=health_check_interval,
retry_on_timeout=retry_on_timeout
)
# Validate connection with ping
client.ping()
# Disable persistence on first connection - improves performance
# Only try to disable if not in a read-only environment
try:
client.config_set('save', '') # Disable RDB snapshots
client.config_set('appendonly', 'no') # Disable AOF logging
# Set optimal memory settings with environment variable support
# Get max memory from environment or use a larger default (512MB instead of 256MB)
#max_memory = os.environ.get('REDIS_MAX_MEMORY', '512mb')
#eviction_policy = os.environ.get('REDIS_EVICTION_POLICY', 'allkeys-lru')
# Apply memory settings
#client.config_set('maxmemory-policy', eviction_policy)
#client.config_set('maxmemory', max_memory)
#logger.info(f"Redis configured with maxmemory={max_memory}, policy={eviction_policy}")
# Disable protected mode when in debug mode
if os.environ.get('DISPATCHARR_DEBUG', '').lower() == 'true':
client.config_set('protected-mode', 'no') # Disable protected mode in debug
logger.warning("Redis protected mode disabled for debug environment")
logger.trace("Redis persistence disabled for better performance")
except redis.exceptions.ResponseError as e:
# Improve error handling for Redis configuration errors
if "OOM" in str(e):
logger.error(f"Redis OOM during configuration: {e}")
# Try to increase maxmemory as an emergency measure
try:
client.config_set('maxmemory', '768mb')
logger.warning("Applied emergency Redis memory increase to 768MB")
except:
pass
else:
logger.error(f"Redis configuration error: {e}")
logger.info(f"Connected to Redis at {redis_host}:{redis_port}/{redis_db}")
cls._client = client
break
except (ConnectionError, TimeoutError) as e:
retry_count += 1
if retry_count >= max_retries:
logger.error(f"Failed to connect to Redis after {max_retries} attempts: {e}")
return None
logger.trace("Redis persistence disabled for better performance")
except redis.exceptions.ResponseError as e:
# Improve error handling for Redis configuration errors
if "OOM" in str(e):
logger.error(f"Redis OOM during configuration: {e}")
# Try to increase maxmemory as an emergency measure
try:
client.config_set('maxmemory', '768mb')
logger.warning("Applied emergency Redis memory increase to 768MB")
except:
pass
else:
# Use exponential backoff for retries
wait_time = retry_interval * (2 ** (retry_count - 1))
logger.warning(f"Redis connection failed. Retrying in {wait_time}s... ({retry_count}/{max_retries})")
time.sleep(wait_time)
logger.error(f"Redis configuration error: {e}")
except Exception as e:
logger.error(f"Unexpected error connecting to Redis: {e}")
logger.info(f"Connected to Redis at {redis_host}:{redis_port}/{redis_db}")
return client
except (ConnectionError, TimeoutError) as e:
retry_count += 1
_tls_hint = _REDIS_TLS_HINT if ssl_params else ""
if retry_count >= max_retries:
logger.error(f"Failed to connect to Redis after {max_retries} attempts: {e}{_tls_hint}")
return None
else:
# Use exponential backoff for retries
wait_time = retry_interval * (2 ** (retry_count - 1))
logger.warning(f"Redis connection failed. Retrying in {wait_time}s... ({retry_count}/{max_retries})")
time.sleep(wait_time)
except Exception as e:
_tls_hint = ""
try:
_tls_hint = _REDIS_TLS_HINT if ssl_params else ""
except NameError:
pass
logger.error(f"Unexpected error connecting to Redis: {e}{_tls_hint}")
return None
return None
@classmethod
def get_client(cls, max_retries=5, retry_interval=1):
"""Get Redis client optimized for non-binary data (decoded responses)"""
if cls._client is None:
cls._client = cls._init_client(decode_responses=True, max_retries=max_retries, retry_interval=retry_interval)
return cls._client
@classmethod
def get_buffer(cls, max_retries=5, retry_interval=1):
"""Get Redis client optimized for binary data (no decoding)"""
if cls._buffer is None:
cls._buffer = cls._init_client(decode_responses=False, max_retries=max_retries, retry_interval=retry_interval)
return cls._buffer
@classmethod
def get_pubsub_client(cls, max_retries=5, retry_interval=1):
"""Get Redis client optimized for PubSub operations"""
@ -161,6 +177,8 @@ class RedisClient:
health_check_interval = getattr(settings, 'REDIS_HEALTH_CHECK_INTERVAL', 30)
retry_on_timeout = getattr(settings, 'REDIS_RETRY_ON_TIMEOUT', True)
ssl_params = getattr(settings, 'REDIS_SSL_PARAMS', {})
# Create Redis client with PubSub-optimized settings - no timeout
client = redis.Redis(
host=redis_host,
@ -172,7 +190,9 @@ class RedisClient:
socket_connect_timeout=socket_connect_timeout,
socket_keepalive=socket_keepalive,
health_check_interval=health_check_interval,
retry_on_timeout=retry_on_timeout
retry_on_timeout=retry_on_timeout,
decode_responses=True,
**ssl_params
)
# Validate connection with ping
@ -185,8 +205,9 @@ class RedisClient:
except (ConnectionError, TimeoutError) as e:
retry_count += 1
_tls_hint = _REDIS_TLS_HINT if ssl_params else ""
if retry_count >= max_retries:
logger.error(f"Failed to connect to Redis for PubSub after {max_retries} attempts: {e}")
logger.error(f"Failed to connect to Redis for PubSub after {max_retries} attempts: {e}{_tls_hint}")
return None
else:
# Use exponential backoff for retries
@ -195,7 +216,8 @@ class RedisClient:
time.sleep(wait_time)
except Exception as e:
logger.error(f"Unexpected error connecting to Redis for PubSub: {e}")
_tls_hint = _REDIS_TLS_HINT if ssl_params else ""
logger.error(f"Unexpected error connecting to Redis for PubSub: {e}{_tls_hint}")
return None
return cls._pubsub_client
@ -410,6 +432,20 @@ def cleanup_memory(log_usage=False, force_collection=True):
pass
logger.trace("Memory cleanup complete for django")
def safe_upload_path(filename: str, base_dir) -> str:
"""Return a safe absolute path for an uploaded file within base_dir.
Strips all directory components from *filename* and verifies the resolved
path stays inside *base_dir*. Raises ValueError on path traversal attempts.
"""
safe_name = Path(filename).name
base = Path(base_dir).resolve()
file_path = (base / safe_name).resolve()
if not file_path.is_relative_to(base):
raise ValueError("Invalid filename.")
return str(file_path)
def is_protected_path(file_path):
"""
Determine if a file path is in a protected directory that shouldn't be deleted.

13
core/views.py
View file

@ -4,7 +4,7 @@ from shlex import split as shlex_split
import sys
import subprocess
import logging
import re
import regex
import redis
from django.conf import settings
@ -42,12 +42,14 @@ def stream_view(request, channel_uuid):
redis_db = int(getattr(settings, "REDIS_DB", "0"))
redis_password = getattr(settings, "REDIS_PASSWORD", "")
redis_user = getattr(settings, "REDIS_USER", "")
ssl_params = getattr(settings, "REDIS_SSL_PARAMS", {})
redis_client = redis.Redis(
host=redis_host,
port=redis_port,
db=redis_db,
password=redis_password if redis_password else None,
username=redis_user if redis_user else None
username=redis_user if redis_user else None,
**ssl_params
)
# Retrieve the channel by the provided stream_id.
@ -130,10 +132,13 @@ def stream_view(request, channel_uuid):
# Prepare the pattern replacement.
logger.debug("Executing the following pattern replacement:")
logger.debug(f" search: {active_profile.search_pattern}")
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', active_profile.replace_pattern)
# Convert JS-style backreferences in replace: $<name> -> \g<name>, $1 -> \1
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', active_profile.replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
logger.debug(f" replace: {active_profile.replace_pattern}")
logger.debug(f" safe replace: {safe_replace_pattern}")
stream_url = re.sub(active_profile.search_pattern, safe_replace_pattern, input_url)
# regex module accepts JS-style (?<name>...) named groups natively
stream_url = regex.sub(active_profile.search_pattern, safe_replace_pattern, input_url)
logger.debug(f"Generated stream url: {stream_url}")
# Get the stream profile set on the channel.

2
core/xtream_codes.py
View file

@ -62,7 +62,7 @@ class Client:
url = f"{self.server_url}/{endpoint}"
logger.debug(f"XC API Request: {url} with params: {params}")
response = self.session.get(url, params=params, timeout=30)
response = self.session.get(url, params=params, timeout=60)
response.raise_for_status()
# Check if response is empty

6
dispatcharr/consumers.py
View file

@ -1,6 +1,6 @@
import json
from channels.generic.websocket import AsyncWebsocketConsumer
import re, logging
import regex, logging
logger = logging.getLogger(__name__)
@ -54,9 +54,9 @@ class MyWebSocketConsumer(AsyncWebsocketConsumer):
# Apply the transformation using the replace_with_mark function
try:
search_preview = re.sub(data["search"], replace_with_mark, data["url"])
search_preview = regex.sub(data["search"], replace_with_mark, data["url"])
except Exception as e:
search_preview = data["search"]
search_preview = data["url"]
logger.error(f"Failed to generate replace preview: {e}")
result = transform_url(data["url"], data["search"], data["replace"])

26
dispatcharr/persistent_lock.py
View file

@ -48,7 +48,7 @@ class PersistentLock:
Returns True if the expiration was successfully extended.
"""
current_value = self.redis_client.get(self.lock_key)
if current_value and current_value.decode("utf-8") == self.lock_token:
if current_value and current_value == self.lock_token:
self.redis_client.expire(self.lock_key, self.lock_timeout)
self.has_lock = False
return True
@ -74,18 +74,40 @@ class PersistentLock:
# Example usage (for testing purposes only):
if __name__ == "__main__":
import os
import sys
# Connect to Redis using environment variables; adjust connection parameters as needed.
redis_host = os.environ.get("REDIS_HOST", "localhost")
redis_port = int(os.environ.get("REDIS_PORT", 6379))
redis_db = int(os.environ.get("REDIS_DB", 0))
redis_password = os.environ.get("REDIS_PASSWORD", "")
redis_user = os.environ.get("REDIS_USER", "")
ssl_kwargs = {}
if os.environ.get("REDIS_SSL", "false").lower() == "true":
import ssl as _ssl
ssl_kwargs["ssl"] = True
ssl_kwargs["ssl_cert_reqs"] = (
_ssl.CERT_REQUIRED if os.environ.get("REDIS_SSL_VERIFY", "true").lower() == "true"
else _ssl.CERT_NONE
)
for env_var, key in [
("REDIS_SSL_CA_CERT", "ssl_ca_certs"),
("REDIS_SSL_CERT", "ssl_certfile"),
("REDIS_SSL_KEY", "ssl_keyfile"),
]:
path = os.environ.get(env_var, "")
if path:
if not os.path.isfile(path):
print(f"Redis TLS: {env_var}={path!r} — file not found.")
sys.exit(1)
ssl_kwargs[key] = path
client = redis.Redis(
host=redis_host,
port=redis_port,
db=redis_db,
password=redis_password if redis_password else None,
username=redis_user if redis_user else None
username=redis_user if redis_user else None,
**ssl_kwargs
)
lock = PersistentLock(client, "lock:example_account", lock_timeout=120)

195
dispatcharr/settings.py
View file

@ -1,7 +1,23 @@
import os
import ssl
from pathlib import Path
from datetime import timedelta
from urllib.parse import quote_plus
from django.core.exceptions import ImproperlyConfigured
def _validate_tls_cert_paths(paths, service_name):
"""Validate that configured TLS certificate file paths exist on disk.
Raises ImproperlyConfigured with a clear message identifying the
service and missing file so operators can fix their environment.
"""
for env_var, file_path in paths:
if file_path and not Path(file_path).is_file():
raise ImproperlyConfigured(
f"{service_name} TLS: {env_var}={file_path!r} — file not found. "
f"Check that the certificate file exists and the volume is mounted correctly."
)
BASE_DIR = Path(__file__).resolve().parent.parent
@ -12,6 +28,37 @@ REDIS_DB = os.environ.get("REDIS_DB", "0")
REDIS_USER = os.environ.get("REDIS_USER", "")
REDIS_PASSWORD = os.environ.get("REDIS_PASSWORD", "")
# Redis TLS configuration
REDIS_SSL = os.environ.get("REDIS_SSL", "false").lower() == "true"
REDIS_SSL_VERIFY = os.environ.get("REDIS_SSL_VERIFY", "true").lower() == "true"
REDIS_SSL_CA_CERT = os.environ.get("REDIS_SSL_CA_CERT", "")
REDIS_SSL_CERT = os.environ.get("REDIS_SSL_CERT", "")
REDIS_SSL_KEY = os.environ.get("REDIS_SSL_KEY", "")
# Reusable dict of SSL kwargs for redis.Redis() constructors
REDIS_SSL_PARAMS = {}
if REDIS_SSL:
_validate_tls_cert_paths([
("REDIS_SSL_CA_CERT", REDIS_SSL_CA_CERT),
("REDIS_SSL_CERT", REDIS_SSL_CERT),
("REDIS_SSL_KEY", REDIS_SSL_KEY),
], "Redis")
REDIS_SSL_PARAMS["ssl"] = True
REDIS_SSL_PARAMS["ssl_cert_reqs"] = ssl.CERT_REQUIRED if REDIS_SSL_VERIFY else ssl.CERT_NONE
if REDIS_SSL_CA_CERT:
REDIS_SSL_PARAMS["ssl_ca_certs"] = REDIS_SSL_CA_CERT
if REDIS_SSL_CERT:
REDIS_SSL_PARAMS["ssl_certfile"] = REDIS_SSL_CERT
if REDIS_SSL_KEY:
REDIS_SSL_PARAMS["ssl_keyfile"] = REDIS_SSL_KEY
_mtls = "enabled" if REDIS_SSL_CERT and REDIS_SSL_KEY else "disabled"
_verify = "on" if REDIS_SSL_VERIFY else "off"
print(f"Redis TLS: enabled (verify={_verify}, mTLS={_mtls})")
else:
print("Redis TLS: disabled")
# Set DEBUG to True for development, False for production
if os.environ.get("DISPATCHARR_DEBUG", "False").lower() == "true":
DEBUG = True
@ -120,20 +167,46 @@ TEMPLATES = [
WSGI_APPLICATION = "dispatcharr.wsgi.application"
ASGI_APPLICATION = "dispatcharr.asgi.application"
_redis_scheme = "rediss" if REDIS_SSL else "redis"
# URL-encoded auth string shared by CHANNEL_LAYERS and Celery broker URLs
if REDIS_PASSWORD:
_encoded_password = quote_plus(REDIS_PASSWORD)
if REDIS_USER:
_redis_auth = f"{quote_plus(REDIS_USER)}:{_encoded_password}@"
else:
_redis_auth = f":{_encoded_password}@"
else:
_redis_auth = ""
_channels_redis_url = f"{_redis_scheme}://{_redis_auth}{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}"
# channels_redis accepts either a URL string or a dict with "address" + kwargs.
# When TLS is enabled, pass SSL params alongside the URL so the connection pool
# uses the correct CA cert and verification settings.
if REDIS_SSL:
# Filter out "ssl" key — the rediss:// scheme already enables SSL.
# Passing ssl=True as a kwarg to aioredis from_url causes an error.
_channels_ssl = {k: v for k, v in REDIS_SSL_PARAMS.items() if k != "ssl"}
_channels_host = {"address": _channels_redis_url, **_channels_ssl}
else:
_channels_host = _channels_redis_url
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": ["redis://{redis_auth}{host}:{port}/{db}".format(
redis_auth=f"{quote_plus(REDIS_USER)}:{quote_plus(REDIS_PASSWORD)}@" if REDIS_PASSWORD and REDIS_USER else f":{quote_plus(REDIS_PASSWORD)}@" if REDIS_PASSWORD else "",
host=REDIS_HOST,
port=REDIS_PORT,
db=REDIS_DB
)], # URL format supports authentication
"hosts": [_channels_host],
},
},
}
# PostgreSQL TLS configuration (defined before DATABASES for module-level access)
POSTGRES_SSL = os.environ.get("POSTGRES_SSL", "false").lower() == "true"
POSTGRES_SSL_MODE = os.environ.get("POSTGRES_SSL_MODE", "verify-full")
POSTGRES_SSL_CA_CERT = os.environ.get("POSTGRES_SSL_CA_CERT", "")
POSTGRES_SSL_CERT = os.environ.get("POSTGRES_SSL_CERT", "")
POSTGRES_SSL_KEY = os.environ.get("POSTGRES_SSL_KEY", "")
if os.getenv("DB_ENGINE", None) == "sqlite":
DATABASES = {
"default": {
@ -154,6 +227,28 @@ else:
}
}
if POSTGRES_SSL:
_validate_tls_cert_paths([
("POSTGRES_SSL_CA_CERT", POSTGRES_SSL_CA_CERT),
("POSTGRES_SSL_CERT", POSTGRES_SSL_CERT),
("POSTGRES_SSL_KEY", POSTGRES_SSL_KEY),
], "PostgreSQL")
DATABASES["default"]["OPTIONS"] = {
"sslmode": POSTGRES_SSL_MODE,
}
if POSTGRES_SSL_CA_CERT:
DATABASES["default"]["OPTIONS"]["sslrootcert"] = POSTGRES_SSL_CA_CERT
if POSTGRES_SSL_CERT:
DATABASES["default"]["OPTIONS"]["sslcert"] = POSTGRES_SSL_CERT
if POSTGRES_SSL_KEY:
DATABASES["default"]["OPTIONS"]["sslkey"] = POSTGRES_SSL_KEY
_mtls = "enabled" if POSTGRES_SSL_CERT and POSTGRES_SSL_KEY else "disabled"
print(f"PostgreSQL TLS: enabled (sslmode={POSTGRES_SSL_MODE}, mTLS={_mtls})")
else:
print("PostgreSQL TLS: disabled")
AUTH_PASSWORD_VALIDATORS = [
{
"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
@ -170,7 +265,14 @@ REST_FRAMEWORK = {
"rest_framework_simplejwt.authentication.JWTAuthentication",
"apps.accounts.authentication.ApiKeyAuthentication",
],
"DEFAULT_PERMISSION_CLASSES": [
"apps.accounts.permissions.IsAdmin",
],
"DEFAULT_FILTER_BACKENDS": ["django_filters.rest_framework.DjangoFilterBackend"],
"DEFAULT_THROTTLE_CLASSES": [],
"DEFAULT_THROTTLE_RATES": {
"login": "3/minute",
},
}
SPECTACULAR_SETTINGS = {
@ -178,17 +280,6 @@ SPECTACULAR_SETTINGS = {
"DESCRIPTION": "API documentation for Dispatcharr",
"VERSION": "1.0.0",
"SERVE_INCLUDE_SCHEMA": False,
"SECURITY": [{"BearerAuth": []}],
"COMPONENTS": {
"securitySchemes": {
"BearerAuth": {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": "Enter your JWT access token. The 'Bearer ' prefix is added automatically.",
}
}
},
}
LANGUAGE_CODE = "en-us"
@ -208,22 +299,52 @@ STATICFILES_DIRS = [
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
AUTH_USER_MODEL = "accounts.User"
# Build default Redis URL from components for Celery with optional authentication
# Build auth string conditionally with URL encoding for special characters
if REDIS_PASSWORD:
encoded_password = quote_plus(REDIS_PASSWORD)
if REDIS_USER:
encoded_user = quote_plus(REDIS_USER)
redis_auth = f"{encoded_user}:{encoded_password}@"
else:
redis_auth = f":{encoded_password}@"
_default_redis_url = f"{_redis_scheme}://{_redis_auth}{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}"
# Celery/Kombu require SSL parameters in the URL query string because
# internal URL parsing can overwrite the CELERY_BROKER_USE_SSL dict.
if REDIS_SSL:
_celery_ssl_params = [
f"ssl_cert_reqs={'CERT_REQUIRED' if REDIS_SSL_VERIFY else 'CERT_NONE'}",
]
if REDIS_SSL_CA_CERT:
_celery_ssl_params.append(f"ssl_ca_certs={REDIS_SSL_CA_CERT}")
if REDIS_SSL_CERT:
_celery_ssl_params.append(f"ssl_certfile={REDIS_SSL_CERT}")
if REDIS_SSL_KEY:
_celery_ssl_params.append(f"ssl_keyfile={REDIS_SSL_KEY}")
_default_celery_url = f"{_default_redis_url}?{'&'.join(_celery_ssl_params)}"
else:
redis_auth = ""
_default_redis_url = f"redis://{redis_auth}{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}"
CELERY_BROKER_URL = os.environ.get("CELERY_BROKER_URL", _default_redis_url)
_default_celery_url = _default_redis_url
CELERY_BROKER_URL = os.environ.get("CELERY_BROKER_URL", _default_celery_url)
CELERY_RESULT_BACKEND = os.environ.get("CELERY_RESULT_BACKEND", CELERY_BROKER_URL)
# Validate that URL overrides don't conflict with TLS settings
for _url_var, _url_val in [
("CELERY_BROKER_URL", CELERY_BROKER_URL),
("CELERY_RESULT_BACKEND", CELERY_RESULT_BACKEND),
]:
_is_override = os.environ.get(_url_var) is not None
if not _is_override:
continue
_url_is_ssl = _url_val.startswith("rediss://")
if REDIS_SSL and not _url_is_ssl:
raise ImproperlyConfigured(
f"REDIS_SSL is enabled but {_url_var} uses redis:// (plaintext). "
f"Change the URL scheme to rediss:// or remove the {_url_var} override."
)
if not REDIS_SSL and _url_is_ssl:
raise ImproperlyConfigured(
f"{_url_var} uses rediss:// (TLS) but REDIS_SSL is not enabled. "
f"Set REDIS_SSL=true and configure the TLS certificate settings."
)
# Celery TLS configuration — required in addition to the rediss:// URL scheme.
# Uses the same cert params as REDIS_SSL_PARAMS, minus the "ssl" key that
# redis-py needs but Celery/Kombu does not.
if REDIS_SSL:
CELERY_BROKER_USE_SSL = {k: v for k, v in REDIS_SSL_PARAMS.items() if k != "ssl"}
CELERY_RESULT_BACKEND_USE_SSL = CELERY_BROKER_USE_SSL
# Configure Redis key prefix
CELERY_RESULT_BACKEND_TRANSPORT_OPTIONS = {
"global_keyprefix": "celery-tasks:", # Set the Redis key prefix for Celery
@ -288,7 +409,6 @@ BACKUP_DATA_DIRS = [
SERVER_IP = "127.0.0.1"
CORS_ALLOW_ALL_ORIGINS = True
CORS_ALLOW_CREDENTIALS = True
CSRF_TRUSTED_ORIGINS = ["http://*", "https://*"]
APPEND_SLASH = True
@ -299,8 +419,19 @@ SIMPLE_JWT = {
"BLACKLIST_AFTER_ROTATION": True, # Optional: Whether to blacklist refresh tokens
}
# Redis connection settings
# Redis connection settings — _default_redis_url uses rediss:// when REDIS_SSL is enabled
REDIS_URL = os.environ.get("REDIS_URL", _default_redis_url)
if os.environ.get("REDIS_URL") is not None:
if REDIS_SSL and not REDIS_URL.startswith("rediss://"):
raise ImproperlyConfigured(
"REDIS_SSL is enabled but REDIS_URL uses redis:// (plaintext). "
"Change the URL scheme to rediss:// or remove the REDIS_URL override."
)
if not REDIS_SSL and REDIS_URL.startswith("rediss://"):
raise ImproperlyConfigured(
"REDIS_URL uses rediss:// (TLS) but REDIS_SSL is not enabled. "
"Set REDIS_SSL=true and configure the TLS certificate settings."
)
REDIS_SOCKET_TIMEOUT = 60 # Socket timeout in seconds
REDIS_SOCKET_CONNECT_TIMEOUT = 5 # Connection timeout in seconds
REDIS_HEALTH_CHECK_INTERVAL = 15 # Health check every 15 seconds

10
dispatcharr/urls.py
View file

@ -6,7 +6,7 @@ from django.views.generic import TemplateView, RedirectView
from .routing import websocket_urlpatterns
from apps.output.views import xc_player_api, xc_panel_api, xc_get, xc_xmltv
from apps.proxy.ts_proxy.views import stream_xc
from apps.output.views import xc_movie_stream, xc_series_stream
from apps.proxy.vod_proxy.views import stream_xc_movie, stream_xc_episode
urlpatterns = [
# API Routes
@ -44,13 +44,13 @@ urlpatterns = [
# XC VOD endpoints
path(
"movie/<str:username>/<str:password>/<str:stream_id>.<str:extension>",
xc_movie_stream,
name="xc_movie_stream",
stream_xc_movie,
name="stream_xc_movie",
),
path(
"series/<str:username>/<str:password>/<str:stream_id>.<str:extension>",
xc_series_stream,
name="xc_series_stream",
stream_xc_episode,
name="stream_xc_episode",
),
# Admin
path("admin", RedirectView.as_view(url="/admin/", permanent=True)),

8
docker/Dockerfile
View file

@ -12,7 +12,7 @@ COPY ./frontend /app/frontend
# remove any node_modules that may have been copied from the host (x86)
RUN rm -rf node_modules || true; \
npm install --no-audit --progress=false;
RUN npm run build; \
RUN npm run build && \
rm -rf node_modules .cache
# --- Redeclare build arguments for the next stage ---
@ -32,9 +32,9 @@ COPY . /app
COPY ./docker/nginx.conf /etc/nginx/sites-enabled/default
# Fix line endings and make entrypoint scripts executable
RUN for f in /app/docker/entrypoint*.sh; do \
if [ -f "$f" ]; then \
sed -i 's/\r$//' "$f" && chmod +x "$f"; \
fi; \
if [ -f "$f" ]; then \
sed -i 's/\r$//' "$f" && chmod +x "$f"; \
fi; \
done
# Clean out existing frontend folder
RUN rm -rf /app/frontend

36
docker/docker-compose.yml
View file

@ -14,6 +14,7 @@ services:
- 9191:9191
volumes:
- ./data:/data
#- ./certs:/certs:ro # TLS certificates (optional)
depends_on:
db:
condition: service_healthy
@ -33,15 +34,26 @@ services:
- POSTGRES_DB=dispatcharr
- POSTGRES_USER=dispatch
- POSTGRES_PASSWORD=secret
# PostgreSQL TLS (optional) — mount certs via the volume above
#- POSTGRES_SSL=true # required to enable TLS
#- POSTGRES_SSL_MODE=verify-full # optional: verify-full (default) | verify-ca | require
#- POSTGRES_SSL_CA_CERT=/certs/postgres/ca.crt # optional: CA cert to verify the server
#- POSTGRES_SSL_CERT=/certs/postgres/client.crt # optional: client cert (only if server requires client auth)
#- POSTGRES_SSL_KEY=/certs/postgres/client.key # optional: client key (only if server requires client auth)
# Redis Connection
- REDIS_HOST=redis
- REDIS_PORT=6379
# Redis Authentication (Optional)
# Uncomment and set if your Redis requires authentication:
#- REDIS_PASSWORD=your_strong_redis_password
#- REDIS_USER=your_redis_username # For Redis 6+ ACL - see Redis service below
# Redis TLS (optional) — mount certs via the volume above
#- REDIS_SSL=true # required to enable TLS
#- REDIS_SSL_VERIFY=true # optional: set false for self-signed certs without a CA
#- REDIS_SSL_CA_CERT=/certs/redis/ca.crt # optional: CA cert to verify the server
#- REDIS_SSL_CERT=/certs/redis/client.crt # optional: client cert (only if server requires client auth)
#- REDIS_SSL_KEY=/certs/redis/client.key # optional: client key (only if server requires client auth)
# Logging
- DISPATCHARR_LOG_LEVEL=info
@ -95,6 +107,7 @@ services:
condition: service_started
volumes:
- ./data:/data
#- ./certs:/certs:ro # TLS certificates (optional)
extra_hosts:
- "host.docker.internal:host-gateway"
entrypoint: ["/app/docker/entrypoint.celery.sh"]
@ -108,21 +121,30 @@ services:
# Must match the web service port for DVR recording and internal API calls
- DISPATCHARR_PORT=9191
# PostgreSQL Connection
# PostgreSQL — must match web service settings
- POSTGRES_HOST=db
- POSTGRES_PORT=5432
- POSTGRES_DB=dispatcharr
- POSTGRES_USER=dispatch
- POSTGRES_PASSWORD=secret
# PostgreSQL TLS — must match web service
#- POSTGRES_SSL=true
#- POSTGRES_SSL_MODE=verify-full
#- POSTGRES_SSL_CA_CERT=/certs/postgres/ca.crt
#- POSTGRES_SSL_CERT=/certs/postgres/client.crt
#- POSTGRES_SSL_KEY=/certs/postgres/client.key
# Redis Connection
# Redis — must match web service settings
- REDIS_HOST=redis
- REDIS_PORT=6379
# Redis Authentication (Optional)
# Uncomment and set if your Redis requires authentication:
#- REDIS_PASSWORD=your_strong_redis_password
#- REDIS_USER=your_redis_username # For Redis 6+ ACL - see Redis service below
#- REDIS_USER=your_redis_username
# Redis TLS — must match web service
#- REDIS_SSL=true
#- REDIS_SSL_VERIFY=true
#- REDIS_SSL_CA_CERT=/certs/redis/ca.crt
#- REDIS_SSL_CERT=/certs/redis/client.crt
#- REDIS_SSL_KEY=/certs/redis/client.key
# Logging
- DISPATCHARR_LOG_LEVEL=info

4
docker/entrypoint.celery.sh
View file

@ -36,6 +36,10 @@ if [ "$USE_LEGACY_NUMPY" = "true" ]; then
fi
fi
# Fix TLS client key permissions/ownership for PostgreSQL.
FIXED_KEY_PATH="/data/.pg-client-celery.key"
. /app/docker/init/00-fix-pg-ssl-key.sh
# Wait for migrations to complete
# Uses 'migrate --check' which exits 0 only when all migrations are applied,
# and exits 1 on unapplied migrations OR connection errors (safe either way)

162
docker/entrypoint.sh
View file

@ -2,9 +2,27 @@
set -e # Exit immediately if a command exits with a non-zero status
# Guard flag to prevent cleanup running twice (trap + explicit call)
_cleanup_done=false
# Function to clean up only running processes
cleanup() {
if $_cleanup_done; then return; fi
_cleanup_done=true
set +e # Disable exit-on-error so cleanup always runs fully
echo "🔥 Cleanup triggered! Stopping services..."
# Explicitly stop uwsgi workers - children of 'su' wrapper, not tracked in pids[]
echo "⛔ Stopping uwsgi workers..."
pkill -TERM -f uwsgi 2>/dev/null || true
# Stop celery, daphne, redis - also not tracked in pids[]
echo "⛔ Stopping celery, daphne, redis..."
pkill -TERM -f "celery" 2>/dev/null || true
pkill -TERM -f "daphne" 2>/dev/null || true
pkill -TERM -f "redis-server" 2>/dev/null || true
# Stop tracked processes (postgres, nginx, su/uwsgi wrapper)
for pid in "${pids[@]}"; do
if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then
echo "⛔ Stopping process (PID: $pid)..."
@ -13,14 +31,38 @@ cleanup() {
echo "✅ Process (PID: $pid) already stopped."
fi
done
# Wait up to 8 s for graceful shutdown, exit early once all are gone
# (leaves headroom within Docker's default 10 s stop_grace_period)
_shutdown_timeout=8
_shutdown_elapsed=0
while [ "$_shutdown_elapsed" -lt "$_shutdown_timeout" ]; do
pgrep -f "uwsgi|celery|daphne|redis-server|postgres" >/dev/null 2>&1 || break
sleep 1
_shutdown_elapsed=$((_shutdown_elapsed + 1))
done
# Force kill anything still lingering
pkill -KILL -f uwsgi 2>/dev/null || true
pkill -KILL -f "celery" 2>/dev/null || true
pkill -KILL -f "daphne" 2>/dev/null || true
pkill -KILL -f "redis-server" 2>/dev/null || true
# Use pg_ctl immediate stop rather than SIGKILL. Avoids data corruption
# while still forcing a fast exit (crash recovery runs on next startup)
if pgrep -f "postgres" >/dev/null 2>&1; then
su - "$POSTGRES_USER" -c "$PG_BINDIR/pg_ctl -D ${POSTGRES_DIR} stop -m immediate" 2>/dev/null || true
fi
wait
echo "✅ All processes stopped cleanly."
}
# Catch termination signals (CTRL+C, Docker Stop, etc.)
trap cleanup TERM INT
# Initialize an array to store PIDs
# Initialize an array to store PIDs and a map of PID->name
pids=()
declare -A pid_names
# Function to echo with timestamp
echo_with_timestamp() {
@ -30,8 +72,23 @@ echo_with_timestamp() {
# Set PostgreSQL environment variables
export POSTGRES_DB=${POSTGRES_DB:-dispatcharr}
export POSTGRES_USER=${POSTGRES_USER:-dispatch}
export POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-secret}
export POSTGRES_HOST=${POSTGRES_HOST:-localhost}
# AIO mode: default to 'secret' for internal DB.
# Modular mode + TLS: no default — cert-only auth (mTLS) uses no password.
# Modular mode + no TLS: preserve 'secret' default for backward compatibility.
if [[ "${DISPATCHARR_ENV:-}" == "modular" && "${POSTGRES_SSL:-}" == "true" ]]; then
export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-}"
else
export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-secret}"
fi
export DISPATCHARR_ENV=${DISPATCHARR_ENV:-aio}
if [[ "$DISPATCHARR_ENV" == "aio" ]]; then
# Use Unix socket for loopback values (unset, localhost, 127.0.0.1)
if [[ -z "$POSTGRES_HOST" || "$POSTGRES_HOST" == "localhost" || "$POSTGRES_HOST" == "127.0.0.1" ]]; then
export POSTGRES_HOST=/var/run/postgresql
fi
else
export POSTGRES_HOST=${POSTGRES_HOST:-localhost}
fi
export POSTGRES_PORT=${POSTGRES_PORT:-5432}
export PG_VERSION=$(ls /usr/lib/postgresql/ | sort -V | tail -n 1)
export PG_BINDIR="/usr/lib/postgresql/${PG_VERSION}/bin"
@ -96,6 +153,20 @@ echo "Environment DISPATCHARR_LOG_LEVEL set to: '${DISPATCHARR_LOG_LEVEL}'"
# Also make the log level available in /etc/environment for all login shells
#grep -q "DISPATCHARR_LOG_LEVEL" /etc/environment || echo "DISPATCHARR_LOG_LEVEL=${DISPATCHARR_LOG_LEVEL}" >> /etc/environment
# Translate Dispatcharr POSTGRES_SSL_* env vars into libpq-recognized PGSSL*
# env vars. Called once before any external PostgreSQL connection; all child
# processes (psql, pg_dump, pg_isready, createdb, dropdb) inherit these
# automatically. No-op when POSTGRES_SSL is not "true".
setup_pg_ssl_env() {
if [ "${POSTGRES_SSL:-false}" != "true" ]; then
return 0
fi
export PGSSLMODE="${POSTGRES_SSL_MODE:-verify-full}"
if [ -n "${POSTGRES_SSL_CA_CERT:-}" ]; then export PGSSLROOTCERT="$POSTGRES_SSL_CA_CERT"; fi
if [ -n "${POSTGRES_SSL_CERT:-}" ]; then export PGSSLCERT="$POSTGRES_SSL_CERT"; fi
if [ -n "${POSTGRES_SSL_KEY:-}" ]; then export PGSSLKEY="$POSTGRES_SSL_KEY"; fi
}
# READ-ONLY - don't let users change these
export POSTGRES_DIR=/data/db
@ -112,6 +183,14 @@ variables=(
CELERY_NICE_LEVEL UWSGI_NICE_LEVEL DJANGO_SECRET_KEY
)
# TLS variables are optional — only propagate when set to avoid noisy warnings
for _tls_var in POSTGRES_SSL POSTGRES_SSL_MODE POSTGRES_SSL_CA_CERT POSTGRES_SSL_CERT POSTGRES_SSL_KEY \
REDIS_SSL REDIS_SSL_VERIFY REDIS_SSL_CA_CERT REDIS_SSL_CERT REDIS_SSL_KEY; do
if [ -n "${!_tls_var+x}" ]; then
variables+=("$_tls_var")
fi
done
# Truncate files before rewriting
> /etc/profile.d/dispatcharr.sh
@ -146,6 +225,22 @@ fi
echo "Starting user setup..."
. /app/docker/init/01-user-setup.sh
# Fix TLS client key permissions/ownership BEFORE any external PG connections.
# Must run after 01-user-setup.sh (user exists for chown) and before
# 02-postgres.sh / pg_isready (which make the first external PG connections).
FIXED_KEY_PATH="/data/.pg-client.key"
. /app/docker/init/00-fix-pg-ssl-key.sh
# Propagate the fixed path to login shells (su - strips env vars)
if [ "${POSTGRES_SSL_KEY:-}" = "$FIXED_KEY_PATH" ]; then
sed -i "/^POSTGRES_SSL_KEY=/d" /etc/environment
echo "POSTGRES_SSL_KEY='$FIXED_KEY_PATH'" >> /etc/environment
sed -i "s|export POSTGRES_SSL_KEY=.*|export POSTGRES_SSL_KEY='$FIXED_KEY_PATH'|" /etc/profile.d/dispatcharr.sh
fi
# Export libpq TLS env vars so all subsequent psql/pg_dump/pg_isready calls
# (in 02-postgres.sh, modular-mode checks, etc.) use TLS automatically.
setup_pg_ssl_env
# Initialize PostgreSQL (script handles modular vs internal mode internally)
echo "Setting up PostgreSQL..."
. /app/docker/init/02-postgres.sh
@ -165,7 +260,7 @@ if [[ "$DISPATCHARR_ENV" != "modular" ]]; then
done
postgres_pid=$(su - "$POSTGRES_USER" -c "$PG_BINDIR/pg_ctl -D ${POSTGRES_DIR} status" | sed -n 's/.*PID: \([0-9]\+\).*/\1/p')
echo "✅ Postgres started with PID $postgres_pid"
pids+=("$postgres_pid")
if [ -n "$postgres_pid" ]; then pids+=("$postgres_pid"); pid_names[$postgres_pid]="postgres"; fi
# Unconditional startup guarantees — run on every AIO startup.
# Each is idempotent and handles all scenarios (fresh, upgrade, restart).
@ -207,13 +302,13 @@ if [[ "$DISPATCHARR_ENV" = "dev" ]]; then
su - "$POSTGRES_USER" -c "cd /app/frontend && npm run dev &"
npm_pid=$(pgrep vite | sort | head -n1)
echo "✅ vite started with PID $npm_pid"
pids+=("$npm_pid")
if [ -n "$npm_pid" ]; then pids+=("$npm_pid"); pid_names[$npm_pid]="vite"; fi
else
echo "🚀 Starting nginx..."
nginx
nginx_pid=$(pgrep nginx | sort | head -n1)
nginx_pid=$(pgrep nginx | sort | head -n1)
echo "✅ nginx started with PID $nginx_pid"
pids+=("$nginx_pid")
if [ -n "$nginx_pid" ]; then pids+=("$nginx_pid"); pid_names[$nginx_pid]="nginx"; fi
fi
@ -262,39 +357,7 @@ fi
# This preserves both the nice value and environment variables
nice -n "$UWSGI_NICE_LEVEL" su - "$POSTGRES_USER" -c "cd /app && exec $VIRTUAL_ENV/bin/uwsgi $uwsgi_args" & uwsgi_pid=$!
echo "✅ uwsgi started with PID $uwsgi_pid (nice $UWSGI_NICE_LEVEL)"
pids+=("$uwsgi_pid")
# sed -i 's/protected-mode yes/protected-mode no/g' /etc/redis/redis.conf
# su - "$POSTGRES_USER" -c "redis-server --protected-mode no &"
# redis_pid=$(pgrep redis)
# echo "✅ redis started with PID $redis_pid"
# pids+=("$redis_pid")
# echo "🚀 Starting gunicorn..."
# su - "$POSTGRES_USER" -c "cd /app && gunicorn dispatcharr.asgi:application \
# --bind 0.0.0.0:5656 \
# --worker-class uvicorn.workers.UvicornWorker \
# --workers 2 \
# --threads 1 \
# --timeout 0 \
# --keep-alive 30 \
# --access-logfile - \
# --error-logfile - &"
# gunicorn_pid=$(pgrep gunicorn | sort | head -n1)
# echo "✅ gunicorn started with PID $gunicorn_pid"
# pids+=("$gunicorn_pid")
# echo "Starting celery and beat..."
# su - "$POSTGRES_USER" -c "cd /app && celery -A dispatcharr worker -l info --autoscale=8,2 &"
# celery_pid=$(pgrep celery | sort | head -n1)
# echo "✅ celery started with PID $celery_pid"
# pids+=("$celery_pid")
# su - "$POSTGRES_USER" -c "cd /app && celery -A dispatcharr beat -l info &"
# beat_pid=$(pgrep beat | sort | head -n1)
# echo "✅ celery beat started with PID $beat_pid"
# pids+=("$beat_pid")
pids+=("$uwsgi_pid"); pid_names[$uwsgi_pid]="uwsgi"
# Wait for services to fully initialize before checking hardware
echo "⏳ Waiting for services to fully initialize before hardware check..."
@ -307,18 +370,23 @@ echo "🔍 Running hardware acceleration check..."
# Wait for at least one process to exit and log the process that exited first
if [ ${#pids[@]} -gt 0 ]; then
echo "⏳ Dispatcharr is running. Monitoring processes..."
set +e
while kill -0 "${pids[@]}" 2>/dev/null; do
sleep 1 # Wait for a second before checking again
done
echo "🚨 One of the processes exited! Checking which one..."
# Only report unexpected exits — skip if cleanup was already triggered by
# the trap (i.e. docker stop sent SIGTERM and we shut down intentionally)
if ! $_cleanup_done; then
echo "🚨 One of the processes exited unexpectedly! Checking which one..."
for pid in "${pids[@]}"; do
if ! kill -0 "$pid" 2>/dev/null; then
process_name=$(ps -p "$pid" -o comm=)
echo "❌ Process $process_name (PID: $pid) has exited!"
fi
done
for pid in "${pids[@]}"; do
if ! kill -0 "$pid" 2>/dev/null; then
process_name=${pid_names[$pid]:-unknown}
echo "❌ Process $process_name (PID: $pid) has exited!"
fi
done
fi
else
echo "❌ No processes started. Exiting."
exit 1

44
docker/init/00-fix-pg-ssl-key.sh Normal file
View file

@ -0,0 +1,44 @@
#!/bin/bash
#
# Fix TLS client key permissions and ownership for PostgreSQL.
# libpq requires the client key to be 0600 or stricter.
#
# Triggers on:
# - Permissions too open (Docker Desktop mounts files as 0777)
# - Wrong ownership (Kubernetes secrets / Docker volumes mount as root;
# the application user can't read a root-owned 0600 key)
# - Read-only source (volume mounted :ro — can't chmod in place)
#
# Usage: source this script with FIXED_KEY_PATH set to the destination.
# FIXED_KEY_PATH="/data/.pg-client.key"
# . /app/docker/init/00-fix-pg-ssl-key.sh
#
# After sourcing, POSTGRES_SSL_KEY is updated to the fixed path if a copy
# was needed. The caller is responsible for propagating the new value to
# /etc/environment or profile.d if required.
: "${FIXED_KEY_PATH:?FIXED_KEY_PATH must be set before sourcing fix-pg-ssl-key.sh}"
if [ -n "${POSTGRES_SSL_KEY:-}" ] && [ -f "$POSTGRES_SSL_KEY" ]; then
_key_perms=$(stat -c '%a' "$POSTGRES_SSL_KEY" 2>/dev/null)
_key_owner=$(stat -c '%u' "$POSTGRES_SSL_KEY" 2>/dev/null)
_needs_fix=false
if [ "$_key_perms" != "600" ] && [ "$_key_perms" != "640" ]; then
_needs_fix=true
elif [ "$(id -u)" = "0" ] && [ -n "${PUID:-}" ] && [ "$_key_owner" != "$PUID" ]; then
_needs_fix=true
fi
if [ "$_needs_fix" = true ]; then
cp "$POSTGRES_SSL_KEY" "$FIXED_KEY_PATH"
chmod 600 "$FIXED_KEY_PATH"
if [ "$(id -u)" = "0" ] && [ -n "${PUID:-}" ]; then
chown "${PUID}:${PGID:-$PUID}" "$FIXED_KEY_PATH"
fi
export POSTGRES_SSL_KEY="$FIXED_KEY_PATH"
echo "Fixed PostgreSQL client key (perms: ${_key_perms}, owner: ${_key_owner} → ${PUID:-root}:600)"
fi
unset _key_perms _key_owner _needs_fix
fi

50
docker/init/99-init-dev.sh
View file

@ -1,25 +1,33 @@
#!/bin/bash
echo "🚀 Development Mode - Setting up Frontend..."
if [ ! -e "/tmp/init" ]; then
echo "🚀 Development Mode - Setting up Frontend..."
# Install Node.js
if ! command -v node 2>&1 >/dev/null
then
echo "=== setting up nodejs ==="
curl -sL https://deb.nodesource.com/setup_23.x -o /tmp/nodesource_setup.sh
bash /tmp/nodesource_setup.sh
apt-get update
apt-get install -y --no-install-recommends \
nodejs
fi
# Install frontend dependencies
cd /app/frontend && npm install
# Install Python dependencies using UV
cd /app && uv sync --python $UV_PROJECT_ENVIRONMENT/bin/python --no-install-project --no-dev
# Install debugpy for remote debugging
if [ "$DISPATCHARR_DEBUG" = "true" ]; then
echo "=== setting up debugpy ==="
uv pip install --python $UV_PROJECT_ENVIRONMENT/bin/python debugpy
# Install Node.js
if ! command -v node 2>&1 >/dev/null
then
echo "=== setting up nodejs ==="
curl -sL https://deb.nodesource.com/setup_23.x -o /tmp/nodesource_setup.sh
bash /tmp/nodesource_setup.sh
apt-get update
apt-get install -y --no-install-recommends \
nodejs
fi
# Install frontend dependencies
cd /app/frontend && npm install
# Install Python dependencies using UV
cd /app && uv sync --python $UV_PROJECT_ENVIRONMENT/bin/python --no-install-project --no-dev
# Install debugpy for remote debugging
if [ "$DISPATCHARR_DEBUG" = "true" ]; then
echo "=== setting up debugpy ==="
uv pip install --python $UV_PROJECT_ENVIRONMENT/bin/python debugpy
fi
if [[ "$DISPATCHARR_ENV" = "dev" ]]; then
touch /tmp/init
fi
else
echo "Development mode initialization already done. Skipping dev setup."
fi

13
docker/nginx.conf
View file

@ -42,7 +42,7 @@ server {
alias /data/backups/;
}
location /api/logos/(?<logo_id>\d+)/cache/ {
location ~ ^/api/channels/logos/(?<logo_id>\d+)/cache/ {
proxy_pass http://127.0.0.1:5656;
proxy_cache logo_cache;
proxy_cache_key "$scheme$request_uri"; # Cache per logo URL
@ -50,7 +50,7 @@ server {
proxy_cache_use_stale error timeout updating; # Serve stale if Django is slow
}
location ~ ^/api/channels/logos/(?<logo_id>\d+)/cache/ {
location ~ ^/api/vod/vodlogos/(?<logo_id>\d+)/cache/ {
proxy_pass http://127.0.0.1:5656;
proxy_cache logo_cache;
proxy_cache_key "$scheme$request_uri"; # Cache per logo URL
@ -91,12 +91,9 @@ server {
location /proxy/ {
include uwsgi_params;
uwsgi_pass unix:/app/uwsgi.sock;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
uwsgi_buffering off;
uwsgi_read_timeout 300s;
uwsgi_send_timeout 300s;
client_max_body_size 0;
}
}

892
docker/tests/test-tls-postgres.sh Normal file
View file

@ -0,0 +1,892 @@
#!/bin/bash
#
# Integration test suite for TLS/mTLS in modular mode.
# Validates that Dispatcharr connects correctly to external PostgreSQL and
# Redis services using various TLS configurations.
#
# Prerequisites:
# - Docker Desktop (or Docker Engine) running
# - Internet access (pulls postgres:17, redis:latest)
# - ~10-15 minutes for a full run
#
# Usage:
# cd <repo_root>
# bash docker/tests/test-tls-postgres.sh [--skip-build] [--keep-on-fail] [scenario_name]
#
# Options:
# --skip-build Skip Docker image build (use existing dispatcharr:tls-test image)
# --keep-on-fail Don't clean up containers/volumes on failure (for debugging)
# scenario_name Run only the named scenario
#
# Scenarios:
# modular_mtls_no_password PG mTLS cert-only auth, no password
# modular_mtls_with_password PG mTLS + password auth combined
# modular_tls_server_only PG server-side TLS only (no client cert)
# modular_tls_key_permission PG mTLS with 0777 client key (Docker Desktop scenario)
# modular_no_tls_regression Non-TLS modular mode still works
# modular_pg_verify_full PG mTLS with verify-full (CN must match hostname)
# modular_redis_tls Redis with TLS (server-side verification)
# modular_full_tls_celery PG mTLS + Redis TLS with separate Celery container
#
# Exit codes:
# 0 All tests passed
# 1 One or more tests failed (or build failed)
set -uo pipefail
# Prevent Git Bash (MINGW) from converting Unix paths
export MSYS_NO_PATHCONV=1
###############################################################################
# Configuration
###############################################################################
IMAGE_NAME="dispatcharr:tls-test"
TEST_PREFIX="tls_test"
STARTUP_TIMEOUT=120
SKIP_BUILD=false
KEEP_ON_FAIL=false
SINGLE_SCENARIO=""
PASS=0
FAIL=0
SKIP=0
ERRORS=()
CERT_DIR=""
# Colors (disabled if not a terminal)
if [ -t 1 ]; then
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
else
RED=''; GREEN=''; YELLOW=''; CYAN=''; BOLD=''; NC=''
fi
###############################################################################
# Parse arguments
###############################################################################
for arg in "$@"; do
case "$arg" in
--skip-build) SKIP_BUILD=true ;;
--keep-on-fail) KEEP_ON_FAIL=true ;;
-*) echo "Unknown option: $arg"; exit 1 ;;
*) SINGLE_SCENARIO="$arg" ;;
esac
done
###############################################################################
# Helpers
###############################################################################
CURRENT_SCENARIO=""
CLEANUP_ITEMS=()
log_pass() { echo -e " ${GREEN}✅ $1${NC}"; PASS=$((PASS + 1)); }
log_fail() { echo -e " ${RED}❌ $1${NC}"; FAIL=$((FAIL + 1)); ERRORS+=("[$CURRENT_SCENARIO] $1"); }
log_skip() { echo -e " ${YELLOW}⏭️ $1${NC}"; SKIP=$((SKIP + 1)); }
log_info() { echo -e " ${CYAN}ℹ️ $1${NC}"; }
section() { echo -e "\n${BOLD}━━━ $1 ━━━${NC}"; SCENARIO_FAIL_BEFORE=$FAIL; }
track_container() { CLEANUP_ITEMS+=("container:$1"); }
track_volume() { CLEANUP_ITEMS+=("volume:$1"); }
track_network() { CLEANUP_ITEMS+=("network:$1"); }
fresh_volume() {
local vol="$1"
docker rm -f $(docker ps -aq --filter "volume=${vol}") 2>/dev/null || true
docker volume rm "$vol" 2>/dev/null || true
docker volume create "$vol" >/dev/null
track_volume "$vol"
}
cleanup_scenario() {
if [ "$KEEP_ON_FAIL" = true ] && [ "$FAIL" -gt "${SCENARIO_FAIL_BEFORE:-0}" ]; then
log_info "Keeping resources for debugging (--keep-on-fail)"
CLEANUP_ITEMS=()
return
fi
for item in "${CLEANUP_ITEMS[@]}"; do
local type="${item%%:*}"
local name="${item#*:}"
case "$type" in
container) docker stop "$name" 2>/dev/null; docker rm -f "$name" 2>/dev/null ;;
volume) docker volume rm "$name" 2>/dev/null ;;
network) docker network rm "$name" 2>/dev/null ;;
esac
done
CLEANUP_ITEMS=()
}
trap 'cleanup_scenario; [ -n "$CERT_DIR" ] && rm -rf "$CERT_DIR"' EXIT
wait_for_ready() {
local name="$1"
local timeout="${2:-$STARTUP_TIMEOUT}"
local elapsed=0
while [ $elapsed -lt $timeout ]; do
if ! docker ps -q -f "name=^${name}$" 2>/dev/null | grep -q .; then
echo " Container $name exited unexpectedly"
return 1
fi
if docker logs "$name" 2>&1 | grep -q "uwsgi started with PID"; then
return 0
fi
sleep 3
((elapsed+=3))
done
echo " Timeout (${timeout}s) waiting for $name"
return 1
}
_capture_logs() {
local container="$1" logfile="$2"
docker logs "$container" > "$logfile" 2>&1
}
check_log_contains() {
local container="$1" pattern="$2" description="$3"
local tmplog; tmplog=$(mktemp)
_capture_logs "$container" "$tmplog"
if grep -q "$pattern" "$tmplog"; then
log_pass "$description"
else
log_fail "$description (pattern not found: $pattern)"
fi
rm -f "$tmplog"
}
check_log_absent() {
local container="$1" pattern="$2" description="$3"
local tmplog; tmplog=$(mktemp)
_capture_logs "$container" "$tmplog"
if grep -q "$pattern" "$tmplog"; then
log_fail "$description (unexpected pattern found: $pattern)"
else
log_pass "$description"
fi
rm -f "$tmplog"
}
check_migrations_done() {
local container="$1"
local tmplog; tmplog=$(mktemp)
_capture_logs "$container" "$tmplog"
if grep -qE "Running migrations|No migrations to apply|Operations to perform|Applying .+\.\.\. OK" "$tmplog"; then
log_pass "Django migrations completed"
elif grep -q "uwsgi started with PID" "$tmplog"; then
log_pass "Django migrations completed (confirmed via uwsgi startup)"
else
log_fail "Django migrations did not complete"
fi
rm -f "$tmplog"
}
check_no_permission_errors() {
local container="$1"
local tmplog; tmplog=$(mktemp)
_capture_logs "$container" "$tmplog"
local errors
errors=$(grep -iE "permission denied|operation not permitted" "$tmplog" \
| grep -v "GPU acceleration" | grep -v "Warning:" | head -5)
rm -f "$tmplog"
if [ -n "$errors" ]; then
log_fail "Permission errors in logs:"
echo "$errors" | while read -r line; do echo " $line"; done
else
log_pass "No permission errors in logs"
fi
}
dump_logs_on_fail() {
local container="$1"
if [ $FAIL -gt ${SCENARIO_FAIL_BEFORE:-0} ]; then
echo -e " ${YELLOW}--- Container logs ($container) ---${NC}"
docker logs "$container" 2>&1 | tail -30 | sed 's/^/ /'
echo -e " ${YELLOW}--- End logs ---${NC}"
fi
}
###############################################################################
# Certificate generation
###############################################################################
generate_test_certs() {
CERT_DIR=$(mktemp -d)
log_info "Generating test certificates in $CERT_DIR"
# Generate certs inside a container for cross-platform compatibility.
# Shared CA for both PG and Redis. CN of server certs must match their
# Docker container hostnames for verify-full mode.
docker run --rm --entrypoint sh \
-v "$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR"):/certs" \
-w /certs alpine/openssl -c '
# Shared CA
openssl req -new -x509 -days 1 -nodes \
-keyout ca.key -out ca.crt -subj "/CN=Test CA" 2>/dev/null &&
# PostgreSQL server cert (CN = PG container hostname)
openssl req -new -nodes \
-keyout pg-server.key -out pg-server.csr -subj "/CN='"${TEST_PREFIX}"'_pg" 2>/dev/null &&
openssl x509 -req -days 1 -in pg-server.csr \
-CA ca.crt -CAkey ca.key -CAcreateserial -out pg-server.crt 2>/dev/null &&
# PostgreSQL client cert (CN = POSTGRES_USER)
openssl req -new -nodes \
-keyout pg-client.key -out pg-client.csr -subj "/CN=dispatch" 2>/dev/null &&
openssl x509 -req -days 1 -in pg-client.csr \
-CA ca.crt -CAkey ca.key -CAcreateserial -out pg-client.crt 2>/dev/null &&
# Redis server cert (CN = Redis container hostname)
openssl req -new -nodes \
-keyout redis-server.key -out redis-server.csr -subj "/CN='"${TEST_PREFIX}"'_redis" 2>/dev/null &&
openssl x509 -req -days 1 -in redis-server.csr \
-CA ca.crt -CAkey ca.key -CAcreateserial -out redis-server.crt 2>/dev/null &&
# Backwards-compat aliases (existing PG-only scenarios use these names)
cp pg-server.crt server.crt && cp pg-server.key server.key &&
cp pg-client.crt client.crt && cp pg-client.key client.key &&
chmod 600 pg-server.key pg-client.key redis-server.key server.key client.key
' || { log_fail "Certificate generation failed"; return 1; }
log_pass "Test certificates generated"
}
###############################################################################
# Start a TLS-enabled Redis container
###############################################################################
start_tls_redis() {
local name="$1" net="$2"
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
# Redis needs certs owned by redis user (uid 999 in the official image).
# Mount certs, copy to a writable location, fix ownership, then start
# with TLS flags.
docker run -d --name "$name" --network "$net" \
-v "${cert_mount}:/certs:ro" \
redis:latest \
sh -c '
cp /certs/redis-server.crt /certs/redis-server.key /certs/ca.crt /tmp/ &&
chmod 600 /tmp/redis-server.key &&
chown redis:redis /tmp/redis-server.crt /tmp/redis-server.key /tmp/ca.crt &&
exec redis-server \
--tls-port 6379 --port 0 \
--tls-cert-file /tmp/redis-server.crt \
--tls-key-file /tmp/redis-server.key \
--tls-ca-cert-file /tmp/ca.crt \
--tls-auth-clients no
' >/dev/null
# Wait for Redis TLS to be ready
local elapsed=0
while [ $elapsed -lt 20 ]; do
if docker exec "$name" redis-cli --tls \
--cert /certs/redis-server.crt --key /certs/redis-server.key --cacert /certs/ca.crt \
ping 2>/dev/null | grep -q "PONG"; then
break
fi
sleep 2; elapsed=$((elapsed + 2))
done
}
###############################################################################
# Start a TLS-enabled PostgreSQL container
###############################################################################
start_tls_postgres() {
local name="$1" net="$2" hba_auth="$3"
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
docker run -d --name "$name" --network "$net" \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=tempsetup \
-e POSTGRES_DB=dispatcharr \
-v "${cert_mount}:/certs:ro" \
postgres:17 >/dev/null
# Wait for PG to initialize
local elapsed=0
while [ $elapsed -lt 30 ]; do
if docker exec "$name" su postgres -c "/usr/lib/postgresql/17/bin/pg_isready" 2>/dev/null | grep -q "accepting"; then
break
fi
sleep 2; ((elapsed+=2))
done
# Configure SSL and pg_hba.conf
docker exec "$name" bash -c "
cp /certs/server.crt /certs/server.key /certs/ca.crt /var/lib/postgresql/
chown postgres:postgres /var/lib/postgresql/server.crt /var/lib/postgresql/server.key /var/lib/postgresql/ca.crt
chmod 600 /var/lib/postgresql/server.key
echo \"ssl = on\" >> /var/lib/postgresql/data/postgresql.conf
echo \"ssl_cert_file = '/var/lib/postgresql/server.crt'\" >> /var/lib/postgresql/data/postgresql.conf
echo \"ssl_key_file = '/var/lib/postgresql/server.key'\" >> /var/lib/postgresql/data/postgresql.conf
echo \"ssl_ca_file = '/var/lib/postgresql/ca.crt'\" >> /var/lib/postgresql/data/postgresql.conf
cat > /var/lib/postgresql/data/pg_hba.conf << HBA
local all all trust
hostssl all all 0.0.0.0/0 ${hba_auth}
hostssl all all ::0/0 ${hba_auth}
HBA
su postgres -c '/usr/lib/postgresql/17/bin/pg_ctl reload -D /var/lib/postgresql/data'
" >/dev/null 2>&1
sleep 1
}
###############################################################################
# Test scenarios
###############################################################################
test_modular_mtls_no_password() {
CURRENT_SCENARIO="modular_mtls_no_password"
section "Modular mode — mTLS cert-only auth (no password)"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
start_tls_postgres "$pg_name" "$net" "cert"
docker run -d --name "$redis_name" --network "$net" redis:latest >/dev/null
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
# No POSTGRES_PASSWORD — cert-only auth
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-e POSTGRES_SSL=true \
-e POSTGRES_SSL_MODE=verify-ca \
-e POSTGRES_SSL_CA_CERT=/certs/ca.crt \
-e POSTGRES_SSL_CERT=/certs/client.crt \
-e POSTGRES_SSL_KEY=/certs/client.key \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started with mTLS cert-only auth"
check_log_contains "$name" "PostgreSQL version check passed" \
"Version check passed with mTLS"
check_log_contains "$name" "PostgreSQL TLS: enabled" \
"Django sees TLS enabled"
check_migrations_done "$name"
check_no_permission_errors "$name"
else
log_fail "Container failed to start with mTLS cert-only auth"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
test_modular_mtls_with_password() {
CURRENT_SCENARIO="modular_mtls_with_password"
section "Modular mode — mTLS + password auth"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
# cert + md5 password
start_tls_postgres "$pg_name" "$net" "cert"
docker run -d --name "$redis_name" --network "$net" redis:latest >/dev/null
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=tempsetup \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-e POSTGRES_SSL=true \
-e POSTGRES_SSL_MODE=verify-ca \
-e POSTGRES_SSL_CA_CERT=/certs/ca.crt \
-e POSTGRES_SSL_CERT=/certs/client.crt \
-e POSTGRES_SSL_KEY=/certs/client.key \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started with mTLS + password"
check_log_contains "$name" "PostgreSQL version check passed" \
"Version check passed with mTLS + password"
check_migrations_done "$name"
else
log_fail "Container failed to start with mTLS + password"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
test_modular_tls_server_only() {
CURRENT_SCENARIO="modular_tls_server_only"
section "Modular mode — server-only TLS (no client cert)"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
# md5 auth over TLS (no client cert required)
start_tls_postgres "$pg_name" "$net" "md5"
docker run -d --name "$redis_name" --network "$net" redis:latest >/dev/null
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=tempsetup \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-e POSTGRES_SSL=true \
-e POSTGRES_SSL_MODE=verify-ca \
-e POSTGRES_SSL_CA_CERT=/certs/ca.crt \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started with server-only TLS"
check_log_contains "$name" "PostgreSQL version check passed" \
"Version check passed with server-only TLS"
check_migrations_done "$name"
else
log_fail "Container failed to start with server-only TLS"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
test_modular_tls_key_permission() {
CURRENT_SCENARIO="modular_tls_key_permission"
section "Modular mode — mTLS with 0777 client key (Docker Desktop scenario)"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
start_tls_postgres "$pg_name" "$net" "cert"
docker run -d --name "$redis_name" --network "$net" redis:latest >/dev/null
# Create a copy of certs with 0777 key permissions
local bad_perms_dir
bad_perms_dir=$(mktemp -d)
cp "$CERT_DIR"/ca.crt "$CERT_DIR"/client.crt "$CERT_DIR"/client.key "$bad_perms_dir/"
chmod 777 "$bad_perms_dir/client.key"
local cert_mount
cert_mount="$(cygpath -w "$bad_perms_dir" 2>/dev/null || echo "$bad_perms_dir")"
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-e POSTGRES_SSL=true \
-e POSTGRES_SSL_MODE=verify-ca \
-e POSTGRES_SSL_CA_CERT=/certs/ca.crt \
-e POSTGRES_SSL_CERT=/certs/client.crt \
-e POSTGRES_SSL_KEY=/certs/client.key \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started with 0777 client key"
check_log_contains "$name" "Fixed PostgreSQL client key" \
"Key permission fix triggered"
check_log_contains "$name" "PostgreSQL version check passed" \
"Version check passed after key fix"
check_migrations_done "$name"
else
log_fail "Container failed to start with 0777 client key"
fi
dump_logs_on_fail "$name"
rm -rf "$bad_perms_dir"
cleanup_scenario
}
test_modular_no_tls_regression() {
CURRENT_SCENARIO="modular_no_tls_regression"
section "Modular mode — no TLS (regression check)"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
# Plain PostgreSQL — no TLS
docker run -d --name "$pg_name" --network "$net" \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=secret \
-e POSTGRES_DB=dispatcharr \
postgres:17 >/dev/null
local elapsed=0
while [ $elapsed -lt 30 ]; do
if docker exec "$pg_name" su postgres -c "/usr/lib/postgresql/17/bin/pg_isready" 2>/dev/null | grep -q "accepting"; then
break
fi
sleep 2; ((elapsed+=2))
done
docker run -d --name "$redis_name" --network "$net" redis:latest >/dev/null
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=secret \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started without TLS (regression check)"
check_log_contains "$name" "PostgreSQL version check passed" \
"Version check passed without TLS"
check_log_absent "$name" "Fixed PostgreSQL client key" \
"No key fix when TLS disabled"
check_migrations_done "$name"
else
log_fail "Container failed to start without TLS"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
test_modular_pg_verify_full() {
CURRENT_SCENARIO="modular_pg_verify_full"
section "Modular mode — PG mTLS with verify-full (CN must match hostname)"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
start_tls_postgres "$pg_name" "$net" "cert"
docker run -d --name "$redis_name" --network "$net" redis:latest >/dev/null
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
# verify-full requires server cert CN to match the hostname used to connect.
# Our PG server cert CN is "${TEST_PREFIX}_pg", which matches the container name
# used in POSTGRES_HOST.
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-e POSTGRES_SSL=true \
-e POSTGRES_SSL_MODE=verify-full \
-e POSTGRES_SSL_CA_CERT=/certs/ca.crt \
-e POSTGRES_SSL_CERT=/certs/client.crt \
-e POSTGRES_SSL_KEY=/certs/client.key \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started with verify-full"
check_log_contains "$name" "PostgreSQL version check passed" \
"Version check passed with verify-full"
check_log_contains "$name" "sslmode=verify-full" \
"Django reports verify-full mode"
check_migrations_done "$name"
else
log_fail "Container failed to start with verify-full"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
test_modular_redis_tls() {
CURRENT_SCENARIO="modular_redis_tls"
section "Modular mode — Redis with TLS"
local name="${TEST_PREFIX}_app"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"; track_container "$name"
# Plain PG (no TLS) — isolate Redis TLS testing
docker run -d --name "$pg_name" --network "$net" \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=secret \
-e POSTGRES_DB=dispatcharr \
postgres:17 >/dev/null
local elapsed=0
while [ $elapsed -lt 30 ]; do
if docker exec "$pg_name" su postgres -c "/usr/lib/postgresql/17/bin/pg_isready" 2>/dev/null | grep -q "accepting"; then
break
fi
sleep 2; elapsed=$((elapsed + 2))
done
start_tls_redis "$redis_name" "$net"
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
docker run -d --name "$name" --network "$net" \
-e DISPATCHARR_ENV=modular \
-e POSTGRES_HOST="$pg_name" \
-e POSTGRES_PORT=5432 \
-e POSTGRES_USER=dispatch \
-e POSTGRES_PASSWORD=secret \
-e POSTGRES_DB=dispatcharr \
-e REDIS_HOST="$redis_name" \
-e REDIS_SSL=true \
-e REDIS_SSL_VERIFY=false \
-e REDIS_SSL_CA_CERT=/certs/ca.crt \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if wait_for_ready "$name"; then
log_pass "Container started with Redis TLS"
check_log_contains "$name" "Redis TLS: enabled" \
"Django reports Redis TLS enabled"
check_log_contains "$name" "Redis at ${redis_name}" \
"Redis connected via TLS"
check_migrations_done "$name"
else
log_fail "Container failed to start with Redis TLS"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
test_modular_full_tls_celery() {
CURRENT_SCENARIO="modular_full_tls_celery"
section "Modular mode — PG mTLS + Redis TLS with Celery container"
local name="${TEST_PREFIX}_app"
local celery_name="${TEST_PREFIX}_celery"
local pg_name="${TEST_PREFIX}_pg"
local redis_name="${TEST_PREFIX}_redis"
local net="${TEST_PREFIX}_net"
local vol="${name}_data"
cleanup_scenario
docker network create "$net" >/dev/null 2>&1
fresh_volume "$vol"
track_network "$net"
track_container "$pg_name"; track_container "$redis_name"
track_container "$name"; track_container "$celery_name"
start_tls_postgres "$pg_name" "$net" "cert"
start_tls_redis "$redis_name" "$net"
local cert_mount
cert_mount="$(cygpath -w "$CERT_DIR" 2>/dev/null || echo "$CERT_DIR")"
# Shared env vars for both web and celery containers
local -a tls_env=(
-e DISPATCHARR_ENV=modular
-e POSTGRES_HOST="$pg_name"
-e POSTGRES_PORT=5432
-e POSTGRES_USER=dispatch
-e POSTGRES_DB=dispatcharr
-e REDIS_HOST="$redis_name"
-e POSTGRES_SSL=true
-e POSTGRES_SSL_MODE=verify-ca
-e POSTGRES_SSL_CA_CERT=/certs/ca.crt
-e POSTGRES_SSL_CERT=/certs/client.crt
-e POSTGRES_SSL_KEY=/certs/client.key
-e REDIS_SSL=true
-e REDIS_SSL_VERIFY=false
-e REDIS_SSL_CA_CERT=/certs/ca.crt
)
# Start web container first (generates JWT, runs migrations)
docker run -d --name "$name" --network "$net" \
"${tls_env[@]}" \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
"$IMAGE_NAME" >/dev/null
if ! wait_for_ready "$name"; then
log_fail "Web container failed to start with full TLS"
dump_logs_on_fail "$name"
cleanup_scenario
return
fi
log_pass "Web container started with PG mTLS + Redis TLS"
# Start Celery container (shares /data volume for JWT, waits for migrations)
docker run -d --name "$celery_name" --network "$net" \
"${tls_env[@]}" \
-e DJANGO_SETTINGS_MODULE=dispatcharr.settings \
-e PYTHONUNBUFFERED=1 \
-v "${cert_mount}:/certs:ro" \
-v "${vol}:/data" \
--entrypoint /app/docker/entrypoint.celery.sh \
"$IMAGE_NAME" >/dev/null
# Wait for Celery to start (look for "starting Celery" message)
local elapsed=0
local celery_ok=false
while [ $elapsed -lt 90 ]; do
if ! docker ps -q -f "name=^${celery_name}$" 2>/dev/null | grep -q .; then
echo " Celery container exited unexpectedly"
break
fi
if docker logs "$celery_name" 2>&1 | grep -q "starting Celery"; then
celery_ok=true
break
fi
sleep 3; elapsed=$((elapsed + 3))
done
if [ "$celery_ok" = true ]; then
log_pass "Celery container started with PG mTLS + Redis TLS"
check_log_contains "$celery_name" "Migrations complete" \
"Celery confirmed migrations complete via TLS"
check_log_contains "$celery_name" "PostgreSQL TLS: enabled" \
"Celery sees PostgreSQL TLS enabled"
check_log_contains "$celery_name" "Redis TLS: enabled" \
"Celery sees Redis TLS enabled"
else
log_fail "Celery container failed to start with full TLS"
echo -e " ${YELLOW}--- Celery logs ---${NC}"
docker logs "$celery_name" 2>&1 | tail -20 | sed 's/^/ /'
echo -e " ${YELLOW}--- End logs ---${NC}"
fi
dump_logs_on_fail "$name"
cleanup_scenario
}
###############################################################################
# Main
###############################################################################
echo -e "${BOLD}╔═══════════════════════════════════════════════════════════╗${NC}"
echo -e "${BOLD}║ Dispatcharr — TLS Integration Tests ║${NC}"
echo -e "${BOLD}╚═══════════════════════════════════════════════════════════╝${NC}"
# Build image
if [ "$SKIP_BUILD" = false ]; then
echo -e "\n${BOLD}Building test image...${NC}"
if ! docker build -t "$IMAGE_NAME" -f docker/Dockerfile . 2>&1 | tail -5; then
echo -e "${RED}Build failed${NC}"
exit 1
fi
echo -e "${GREEN}Build complete${NC}"
else
echo -e "\n${YELLOW}Skipping build (--skip-build)${NC}"
fi
# Generate certificates
generate_test_certs || exit 1
# Run scenarios
SCENARIOS=(
modular_mtls_no_password
modular_mtls_with_password
modular_tls_server_only
modular_tls_key_permission
modular_no_tls_regression
modular_pg_verify_full
modular_redis_tls
modular_full_tls_celery
)
for scenario in "${SCENARIOS[@]}"; do
if [ -n "$SINGLE_SCENARIO" ] && [ "$scenario" != "$SINGLE_SCENARIO" ]; then
continue
fi
"test_${scenario}"
done
# Clean up certs
rm -rf "$CERT_DIR"
# Summary
echo -e "\n${BOLD}═══════════════════════════════════════════════════════════${NC}"
echo -e " ${GREEN}Passed: $PASS${NC} ${RED}Failed: $FAIL${NC} ${YELLOW}Skipped: $SKIP${NC}"
if [ ${#ERRORS[@]} -gt 0 ]; then
echo -e "\n ${RED}Failures:${NC}"
for err in "${ERRORS[@]}"; do
echo -e " ${RED}• $err${NC}"
done
fi
echo -e "${BOLD}═══════════════════════════════════════════════════════════${NC}"
[ $FAIL -eq 0 ]

4
docker/uwsgi.dev.ini
View file

@ -8,7 +8,7 @@
exec-pre = python /app/scripts/wait_for_redis.py
; Start Redis first
attach-daemon = redis-server
attach-daemon = redis-server --protected-mode no
; Then start other services with configurable nice level (default: 5 for low priority)
; Users can override via CELERY_NICE_LEVEL environment variable in docker-compose
attach-daemon = nice -n $(CELERY_NICE_LEVEL) celery -A dispatcharr worker --autoscale=6,1
@ -57,4 +57,4 @@ logformat-strftime = true
log-date = %%Y-%%m-%%d %%H:%%M:%%S,000
# Use formatted time with environment variable for log level
log-format = %(ftime) $(DISPATCHARR_LOG_LEVEL) uwsgi.requests Worker ID: %(wid) %(method) %(status) %(uri) %(msecs)ms
log-buffering = 1024 # Add buffer size limit for logging
log-buffering = 1024 # Add buffer size limit for logging

62
frontend/package-lock.json generated
View file

@ -2518,9 +2518,9 @@
}
},
"node_modules/@xmldom/xmldom": {
"version": "0.8.11",
"resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.11.tgz",
"integrity": "sha512-cQzWCtO6C8TQiYl1ruKNn2U6Ao4o4WBBcbL61yJl84x+j5sOWWFU9X7DpND8XZG3daDppSsigMdfAIl2upQBRw==",
"version": "0.8.12",
"resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.12.tgz",
"integrity": "sha512-9k/gHF6n/pAi/9tqr3m3aqkuiNosYTurLLUtc7xQ9sxB/wm7WPygCv8GYa6mS0fLJEHhqMC1ATYhz++U/lRHqg==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
@ -2704,16 +2704,16 @@
}
},
"node_modules/brace-expansion": {
"version": "5.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.2.tgz",
"integrity": "sha512-Pdk8c9poy+YhOgVWw1JNN22/HcivgKWwpxKq04M/jTmHyCZn12WPJebZxdjSa5TmBqISrUSgNYU3eRORljfCCw==",
"version": "5.0.5",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.5.tgz",
"integrity": "sha512-VZznLgtwhn+Mact9tfiwx64fA9erHH/MCXEUfB/0bX/6Fz6ny5EGTXYltMocqg4xFAQZtnO3DHWWXi8RiuN7cQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^4.0.2"
},
"engines": {
"node": "20 || >=22"
"node": "18 || 20 || >=22"
}
},
"node_modules/cac": {
@ -2850,9 +2850,9 @@
}
},
"node_modules/cosmiconfig/node_modules/yaml": {
"version": "1.10.2",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-1.10.2.tgz",
"integrity": "sha512-r3vXyErRCYJ7wg28yvBY5VSoAF8ZvlcW9/BwUzEtUsjvX/DKs24dIkuwjtuprwJJHsbyUbLApepYTR1BN4uHrg==",
"version": "1.10.3",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-1.10.3.tgz",
"integrity": "sha512-vIYeF1u3CjlhAFekPPAk2h/Kv4T3mAkMox5OymRiJQB0spDP10LHvt+K7G9Ny6NuuMAb25/6n1qyUjAcGNf/AA==",
"license": "ISC",
"engines": {
"node": ">= 6"
@ -3554,9 +3554,9 @@
}
},
"node_modules/flatted": {
"version": "3.4.1",
"resolved": "https://registry.npmjs.org/flatted/-/flatted-3.4.1.tgz",
"integrity": "sha512-IxfVbRFVlV8V/yRaGzk0UVIcsKKHMSfYw66T/u4nTwlWteQePsxe//LjudR1AMX4tZW3WFCh3Zqa/sjlqpbURQ==",
"version": "3.4.2",
"resolved": "https://registry.npmjs.org/flatted/-/flatted-3.4.2.tgz",
"integrity": "sha512-PjDse7RzhcPkIJwy5t7KPWQSZ9cAbzQXcafsetQoD7sOJRQlGikNbx7yZp2OotDnJyrDcbyRq3Ttb18iYOqkxA==",
"dev": true,
"license": "ISC"
},
@ -3986,9 +3986,9 @@
}
},
"node_modules/lodash": {
"version": "4.17.23",
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.23.tgz",
"integrity": "sha512-LgVTMpQtIopCi79SJeDiP0TfWi5CNEc/L/aRdTh3yIvmZXTnheWpKjSZhnvMl8iXbC1tFg9gdHHDMLoV7CnG+w==",
"version": "4.18.1",
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.18.1.tgz",
"integrity": "sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q==",
"license": "MIT"
},
"node_modules/lodash.clamp": {
@ -4362,9 +4362,9 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.4.tgz",
"integrity": "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==",
"dev": true,
"license": "MIT",
"engines": {
@ -5490,9 +5490,9 @@
}
},
"node_modules/vite": {
"version": "7.3.1",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.1.tgz",
"integrity": "sha512-w+N7Hifpc3gRjZ63vYBXA56dvvRlNWRczTdmCBBa+CotUzAPf5b7YMdMR/8CQoeYE5LX3W4wj6RYTgonm1b9DA==",
"version": "7.3.2",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.2.tgz",
"integrity": "sha512-Bby3NOsna2jsjfLVOHKes8sGwgl4TT0E6vvpYgnAYDIF/tie7MRaFthmKuHx1NSXjiTueXH3do80FMQgvEktRg==",
"dev": true,
"license": "MIT",
"dependencies": {
@ -5794,6 +5794,24 @@
"dev": true,
"license": "MIT"
},
"node_modules/yaml": {
"version": "2.8.3",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.3.tgz",
"integrity": "sha512-AvbaCLOO2Otw/lW5bmh9d/WEdcDFdQp2Z2ZUH3pX9U2ihyUY0nvLv7J6TrWowklRGPYbB/IuIMfYgxaCPg5Bpg==",
"dev": true,
"license": "ISC",
"optional": true,
"peer": true,
"bin": {
"yaml": "bin.mjs"
},
"engines": {
"node": ">= 14.6"
},
"funding": {
"url": "https://github.com/sponsors/eemeli"
}
},
"node_modules/yocto-queue": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/yocto-queue/-/yocto-queue-0.1.0.tgz",

5
frontend/src/App.jsx
View file

@ -14,6 +14,7 @@ import Stats from './pages/Stats';
import DVR from './pages/DVR';
import Settings from './pages/Settings';
import PluginsPage from './pages/Plugins';
import PluginBrowsePage from './pages/PluginBrowse';
import ConnectPage from './pages/Connect';
import ConnectLogsPage from './pages/ConnectLogs';
import Users from './pages/Users';
@ -153,6 +154,10 @@ const App = () => {
<Route path="/guide" element={<Guide />} />
<Route path="/dvr" element={<DVR />} />
<Route path="/stats" element={<Stats />} />
<Route
path="/plugins/browse"
element={<PluginBrowsePage />}
/>
<Route path="/plugins" element={<PluginsPage />} />
<Route path="/connect" element={<ConnectPage />} />
<Route

181
frontend/src/api.js
View file

@ -1193,6 +1193,7 @@ export default class API {
if (values.file) {
body = new FormData();
for (const prop in values) {
if (values[prop] === null || values[prop] === undefined) continue;
body.append(prop, values[prop]);
}
} else {
@ -1601,9 +1602,7 @@ export default class API {
});
const playlist = await API.getPlaylist(accountId);
usePlaylistsStore
.getState()
.updateProfiles(playlist.id, playlist.profiles);
usePlaylistsStore.getState().updatePlaylist(playlist);
} catch (e) {
errorNotification(`Failed to update profile for account ${accountId}`, e);
}
@ -1910,26 +1909,28 @@ export default class API {
}
}
static async importPlugin(file) {
static async importPlugin(file, overwrite = false, silent = false) {
try {
const form = new FormData();
form.append('file', file);
if (overwrite) form.append('overwrite', 'true');
const response = await request(`${host}/api/plugins/plugins/import/`, {
method: 'POST',
body: form,
});
return response;
} catch (e) {
// Show only the concise error message for plugin import
const msg =
(e?.body && (e.body.error || e.body.detail)) ||
e?.message ||
'Failed to import plugin';
notifications.show({
title: 'Import failed',
message: msg,
color: 'red',
});
if (!silent) {
const msg =
(e?.body && (e.body.error || e.body.detail)) ||
e?.message ||
'Failed to import plugin';
notifications.show({
title: 'Import failed',
message: msg,
color: 'red',
});
}
throw e;
}
}
@ -1994,6 +1995,130 @@ export default class API {
}
}
// Plugin Repos API
static async getPluginRepos() {
try {
return await request(`${host}/api/plugins/repos/`);
} catch (e) {
errorNotification('Failed to retrieve plugin repos', e);
return [];
}
}
static async addPluginRepo(data) {
try {
return await request(`${host}/api/plugins/repos/`, {
method: 'POST',
body: data,
});
} catch (e) {
errorNotification('Failed to add plugin repo', e);
throw e;
}
}
static async deletePluginRepo(id) {
try {
return await request(`${host}/api/plugins/repos/${id}/`, {
method: 'DELETE',
});
} catch (e) {
errorNotification('Failed to delete plugin repo', e);
throw e;
}
}
static async updatePluginRepo(id, data) {
try {
return await request(`${host}/api/plugins/repos/${id}/`, {
method: 'PUT',
body: data,
});
} catch (e) {
errorNotification('Failed to update plugin repo', e);
}
}
static async refreshPluginRepo(id) {
try {
return await request(`${host}/api/plugins/repos/${id}/refresh/`, {
method: 'POST',
});
} catch (e) {
errorNotification('Failed to refresh plugin repo', e);
}
}
static async getAvailablePlugins() {
try {
const response = await request(`${host}/api/plugins/repos/available/`);
return response.plugins || [];
} catch (e) {
errorNotification('Failed to retrieve available plugins', e);
return [];
}
}
static async getPluginDetailManifest(repoId, manifestUrl) {
try {
const response = await request(
`${host}/api/plugins/repos/plugin-detail/`,
{
method: 'POST',
body: { repo_id: repoId, manifest_url: manifestUrl },
}
);
return response;
} catch (e) {
errorNotification('Failed to retrieve plugin details', e);
return null;
}
}
static async getPluginRepoSettings() {
try {
return await request(`${host}/api/plugins/repos/settings/`);
} catch (e) {
errorNotification('Failed to retrieve repo settings', e);
return null;
}
}
static async updatePluginRepoSettings(data) {
try {
return await request(`${host}/api/plugins/repos/settings/`, {
method: 'PUT',
body: data,
});
} catch (e) {
errorNotification('Failed to update repo settings', e);
return null;
}
}
static async installPluginFromRepo(data) {
try {
return await request(`${host}/api/plugins/repos/install/`, {
method: 'POST',
body: data,
});
} catch (e) {
errorNotification('Failed to install plugin', e);
return null;
}
}
static async previewPluginRepo(url, publicKey) {
try {
return await request(`${host}/api/plugins/repos/preview/`, {
method: 'POST',
body: { url, public_key: publicKey || '' },
});
} catch {
return null;
}
}
static async checkSetting(values) {
const { id, ...payload } = values;
@ -3000,12 +3125,15 @@ export default class API {
}
}
static async updateUser(id, body) {
static async updateUser(id, body, self = false) {
try {
const response = await request(`${host}/api/accounts/users/${id}/`, {
method: 'PATCH',
body,
});
const response = await request(
`${host}/api/accounts/users/${self ? 'me' : id}/`,
{
method: 'PATCH',
body,
}
);
useUsersStore.getState().updateUser(response);
@ -3211,21 +3339,6 @@ export default class API {
}
}
static async updateVODPosition(vodUuid, clientId, position) {
try {
const response = await request(
`${host}/proxy/vod/stream/${vodUuid}/position/`,
{
method: 'POST',
body: { client_id: clientId, position },
}
);
return response;
} catch (e) {
errorNotification('Failed to update playback position', e);
}
}
static async getSystemEvents(limit = 100, offset = 0, eventType = null) {
try {
const params = new URLSearchParams();

59
frontend/src/components/FloatingVideo.jsx
View file

@ -2,6 +2,7 @@
import React, { useCallback, useEffect, useRef, useState } from 'react';
import Draggable from 'react-draggable';
import useVideoStore from '../store/useVideoStore';
import useAuthStore from '../store/auth';
import mpegts from 'mpegts.js';
import { CloseButton, Flex, Loader, Text, Box } from '@mantine/core';
import {
@ -117,6 +118,7 @@ export default function FloatingVideo() {
const contentType = useVideoStore((s) => s.contentType);
const metadata = useVideoStore((s) => s.metadata);
const hideVideo = useVideoStore((s) => s.hideVideo);
const accessToken = useAuthStore((s) => s.accessToken);
const videoRef = useRef(null);
const playerRef = useRef(null);
@ -133,7 +135,8 @@ export default function FloatingVideo() {
const [isLoading, setIsLoading] = useState(false);
const [loadError, setLoadError] = useState(null);
const [showOverlay, setShowOverlay] = useState(true);
const [showOverlay, setShowOverlay] = useState(false);
const [showControls, setShowControls] = useState(false);
const [videoSize, setVideoSize] = useState(() => {
const prefs = getPlayerPrefs();
const saved = prefs.size;
@ -223,7 +226,8 @@ export default function FloatingVideo() {
setIsLoading(true);
setLoadError(null);
setShowOverlay(true); // Show overlay initially
setShowOverlay(false);
setShowControls(false);
console.log('Initializing VOD player for:', streamUrl);
@ -246,7 +250,8 @@ export default function FloatingVideo() {
console.log('Auto-play prevented:', e);
setLoadError('Auto-play was prevented. Click play to start.');
});
// Start overlay timer when video is ready
// Show overlay briefly when video is ready, then auto-hide
setShowOverlay(true);
startOverlayTimer();
};
const handleError = (e) => {
@ -298,8 +303,7 @@ export default function FloatingVideo() {
setIsLoading(true);
setLoadError(null);
console.log('Initializing live stream player for:', streamUrl);
setShowControls(false);
try {
if (!mpegts.getFeatureList().mseLivePlayback) {
@ -310,20 +314,34 @@ export default function FloatingVideo() {
return;
}
const player = mpegts.createPlayer({
type: 'mpegts',
url: streamUrl,
isLive: true,
enableWorker: true,
enableStashBuffer: false,
liveBufferLatencyChasing: true,
liveSync: true,
cors: true,
autoCleanupSourceBuffer: true,
autoCleanupMaxBackwardDuration: 10,
autoCleanupMinBackwardDuration: 5,
reuseRedirectedURL: true,
});
// mpegts.js workers run in WorkerGlobalScope where relative URLs are
// not resolved against the page origin. Always pass an absolute URL.
const absoluteStreamUrl =
streamUrl.startsWith('/') && typeof window !== 'undefined'
? `${window.location.origin}${streamUrl}`
: streamUrl;
const player = mpegts.createPlayer(
{
type: 'mpegts',
url: absoluteStreamUrl,
isLive: true,
cors: true,
},
{
enableWorker: true,
enableStashBuffer: false,
liveBufferLatencyChasing: false,
liveSync: false,
autoCleanupSourceBuffer: true,
autoCleanupMaxBackwardDuration: 120,
autoCleanupMinBackwardDuration: 60,
reuseRedirectedURL: true,
headers: accessToken
? { Authorization: `Bearer ${accessToken}` }
: undefined,
}
);
player.attachMediaElement(videoRef.current);
@ -834,6 +852,7 @@ export default function FloatingVideo() {
<Box
style={{ position: 'relative' }}
onMouseEnter={() => {
setShowControls(true);
if (contentType === 'vod' && !isLoading) {
setShowOverlay(true);
if (overlayTimeoutRef.current) {
@ -850,7 +869,7 @@ export default function FloatingVideo() {
{/* Enhanced video element with better controls for VOD */}
<video
ref={videoRef}
controls
controls={showControls}
className="floating-video-no-drag"
style={{
width: '100%',

2
frontend/src/components/GuideRow.jsx
View file

@ -5,7 +5,7 @@ import {
MINUTE_BLOCK_WIDTH,
MINUTE_INCREMENT,
PROGRAM_HEIGHT,
} from '../pages/guideUtils.js';
} from '../utils/guideUtils.js';
import { Box, Flex, Text, Tooltip } from '@mantine/core';
import { Play } from 'lucide-react';
import logo from '../images/logo.png';

2
frontend/src/components/HourTimeline.jsx
View file

@ -1,7 +1,7 @@
import React from 'react';
import { Box, Text } from '@mantine/core';
import { format } from '../utils/dateTimeUtils.js';
import { HOUR_WIDTH } from '../pages/guideUtils.js';
import { HOUR_WIDTH } from '../utils/guideUtils.js';
const HourBlock = React.memo(
({ hourData, timeFormat, formatDayLabel, handleTimeClick }) => {

430
frontend/src/components/PluginDetailPanel.jsx Normal file
View file

@ -0,0 +1,430 @@
import React, { useState } from 'react';
import {
ActionIcon,
Alert,
Badge,
Button,
Group,
Loader,
Select,
Stack,
Table,
Text,
Tooltip,
} from '@mantine/core';
import { AlertTriangle, Ban, Check, Download, RefreshCw, ShieldAlert, ShieldCheck, Trash2 } from 'lucide-react';
import { compareVersions } from './pluginUtils.js';
export const GitHubIcon = ({ size = 16 }) => (
<svg width={size} height={size} viewBox="0 0 24 24" fill="currentColor">
<path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" />
</svg>
);
export const DiscordIcon = ({ size = 16 }) => (
<svg width={size} height={size} viewBox="0 0 24 24" fill="currentColor">
<path d="M20.317 4.37a19.791 19.791 0 00-4.885-1.515.074.074 0 00-.079.037c-.21.375-.444.864-.608 1.25a18.27 18.27 0 00-5.487 0 12.64 12.64 0 00-.617-1.25.077.077 0 00-.079-.037A19.736 19.736 0 003.677 4.37a.07.07 0 00-.032.027C.533 9.046-.32 13.58.099 18.057a.082.082 0 00.031.057 19.9 19.9 0 005.993 3.03.078.078 0 00.084-.028c.462-.63.874-1.295 1.226-1.994a.076.076 0 00-.041-.106 13.107 13.107 0 01-1.872-.892.077.077 0 01-.008-.128 10.2 10.2 0 00.372-.292.074.074 0 01.077-.01c3.928 1.793 8.18 1.793 12.062 0a.074.074 0 01.078.01c.12.098.246.198.373.292a.077.077 0 01-.006.127 12.299 12.299 0 01-1.873.892.077.077 0 00-.041.107c.36.698.772 1.362 1.225 1.993a.076.076 0 00.084.028 19.839 19.839 0 006.002-3.03.077.077 0 00.032-.054c.5-5.177-.838-9.674-3.549-13.66a.061.061 0 00-.031-.03zM8.02 15.33c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.956-2.419 2.157-2.419 1.21 0 2.176 1.096 2.157 2.42 0 1.333-.956 2.418-2.157 2.418zm7.975 0c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.956-2.419 2.157-2.419 1.21 0 2.176 1.096 2.157 2.42 0 1.333-.947 2.418-2.157 2.418z" />
</svg>
);
/**
* Shared plugin detail panel used in both PluginCard and AvailablePluginCard modals.
*
* Props:
* - detail manifest detail object { manifest: { ... }, signature_verified }
* - detailLoading boolean
* - selectedVersion string | null
* - onVersionChange (version) => void
* - installedVersion string | null currently installed version
* - appVersion string current app version for compat checks
* - installing boolean
* - uninstalling boolean
* - onInstall (params) => void called with { version, url, sha256, min/max }
* - onUninstall () => void called when uninstall button clicked
* - installStatus string | null 'unmanaged' | 'different_repo' | 'installed' | 'update_available' | 'not_installed'
* - installedSourceRepoName string for different_repo tooltip
* - installedVersionIsPrerelease boolean
* - repoId number
* - slug string
*/
const PluginDetailPanel = ({
detail,
detailLoading,
selectedVersion,
onVersionChange,
installedVersion,
installedVersionIsPrerelease = false,
appVersion,
installing = false,
uninstalling = false,
onInstall,
onUninstall,
installStatus,
installedSourceRepoName,
repoId,
slug,
}) => {
if (detailLoading) {
return (
<Stack align="center" py="xl">
<Loader size="sm" />
<Text size="sm" c="dimmed">Loading plugin details…</Text>
</Stack>
);
}
if (!detail?.manifest) {
return <Text size="sm" c="dimmed">Failed to load plugin details.</Text>;
}
const manifest = detail.manifest;
const selectedVersionData = manifest.versions?.find(
(v) => v.version === selectedVersion
);
const isSelSame = installedVersion && selectedVersion &&
compareVersions(selectedVersion, installedVersion) === 0;
const isSelDowngrade = installedVersion && selectedVersion &&
compareVersions(selectedVersion, installedVersion) < 0;
const isInstalled = !!installedVersion;
const selMeetsMin = !selectedVersionData?.min_dispatcharr_version ||
compareVersions(appVersion, selectedVersionData.min_dispatcharr_version) >= 0;
const selMeetsMax = !selectedVersionData?.max_dispatcharr_version ||
compareVersions(appVersion, selectedVersionData.max_dispatcharr_version) <= 0;
const selCompatible = selMeetsMin && selMeetsMax;
const isOverwrite = installStatus === 'unmanaged' || installStatus === 'different_repo';
const handleInstallClick = () => {
if (isSelSame && onUninstall) {
onUninstall();
return;
}
if (!selectedVersionData?.url || !onInstall) return;
const params = {
repo_id: repoId,
slug,
version: selectedVersion,
download_url: selectedVersionData.url,
sha256: selectedVersionData.checksum_sha256,
min_dispatcharr_version: selectedVersionData.min_dispatcharr_version,
max_dispatcharr_version: selectedVersionData.max_dispatcharr_version,
prerelease: selectedVersionData.prerelease === true,
};
onInstall(params);
};
const getButtonProps = () => {
if (isOverwrite) {
return {
label: installing ? 'Installing…' : 'Overwrite',
color: 'orange',
icon: installing ? <Loader size={14} /> : <Download size={14} />,
variant: 'filled',
tooltip: installStatus === 'unmanaged'
? 'Installed manually – installing will take over management'
: `Managed by ${installedSourceRepoName || 'another repo'} – installing will transfer management to this repo`,
};
}
if (isSelSame) {
return {
label: uninstalling ? 'Uninstalling…' : 'Uninstall',
color: 'red',
icon: uninstalling ? <Loader size={14} /> : <Trash2 size={14} />,
variant: 'light',
};
}
if (!selCompatible) {
return {
label: 'Incompatible',
color: 'gray',
icon: <AlertTriangle size={14} />,
variant: 'filled',
};
}
if (isSelDowngrade) {
return {
label: installing ? 'Downgrading…' : 'Downgrade',
color: 'orange',
icon: installing ? <Loader size={14} /> : <AlertTriangle size={14} />,
variant: 'filled',
};
}
if (isInstalled && !installedVersionIsPrerelease) {
return {
label: installing ? 'Updating…' : 'Update',
color: 'yellow',
icon: installing ? <Loader size={14} /> : <RefreshCw size={14} />,
variant: 'filled',
};
}
return {
label: installing ? 'Installing…' : 'Install',
color: undefined,
icon: installing ? <Loader size={14} /> : <Download size={14} />,
variant: 'filled',
};
};
const btnProps = getButtonProps();
const btnDisabled = (isSelSame ? uninstalling : (!selCompatible || installing || !selectedVersionData?.url));
return (
<Stack gap="md">
{manifest.description && (
<Text size="sm">{manifest.description}</Text>
)}
<Group gap="xs" wrap="wrap">
{manifest.author && (
<Badge size="sm" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>AUTHOR</span>
{manifest.author}
</Badge>
)}
{manifest.license && (
<Badge
size="sm"
variant="default"
component="a"
href={`https://spdx.org/licenses/${encodeURIComponent(manifest.license)}.html`}
target="_blank"
rel="noopener noreferrer"
style={{ cursor: 'pointer' }}
>
<span style={{ opacity: 0.5, marginRight: 4 }}>LICENSE</span>
{manifest.license}
</Badge>
)}
{detail.signature_verified != null && (
detail.signature_verified ? (
<Badge
size="sm"
variant="default"
leftSection={<ShieldCheck size={10} />}
>
Verified Signature
</Badge>
) : (
<Tooltip label="Invalid Signature">
<Badge
size="sm"
variant="filled"
color="red"
leftSection={<ShieldAlert size={10} />}
>
Unverified
</Badge>
</Tooltip>
)
)}
{manifest.repo_url && (
<Tooltip label="Source Repository">
<ActionIcon
variant="subtle"
color="gray"
size="sm"
component="a"
href={manifest.repo_url}
target="_blank"
rel="noopener noreferrer"
>
<GitHubIcon size={16} />
</ActionIcon>
</Tooltip>
)}
{manifest.discord_thread && (() => {
const isDiscordChannel = /^https:\/\/discord\.com\/channels\//.test(manifest.discord_thread);
return (
<Tooltip label="Discord Discussion">
<ActionIcon
variant="subtle"
color="gray"
size="sm"
component="a"
href={isDiscordChannel
? manifest.discord_thread.replace('https://', 'discord://')
: manifest.discord_thread}
{...(!isDiscordChannel && { target: '_blank', rel: 'noopener noreferrer' })}
>
<DiscordIcon size={16} />
</ActionIcon>
</Tooltip>
);
})()}
</Group>
{manifest.deprecated && (
<Alert
icon={<Ban size={16} />}
color="red"
variant="light"
title="Deprecated Plugin"
>
This plugin has been marked as deprecated by its maintainer. It may no longer receive
updates or fixes, and could stop working with future versions of Dispatcharr.
Consider looking for an alternative.
</Alert>
)}
{manifest.versions?.length > 0 && (() => {
const installedMissing = installedVersion &&
!manifest.versions.some((v) => compareVersions(v.version, installedVersion) === 0);
const buildLabel = (v) =>
`v${v.version}${v.prerelease ? ' (prerelease)' : ''}${v.version === manifest.latest?.version ? ' (latest)' : ''}${installedVersion && compareVersions(v.version, installedVersion) === 0 ? ' (installed)' : ''}`;
let versions = [...manifest.versions];
if (installedVersionIsPrerelease) {
const prereleases = versions.filter((v) => v.prerelease);
const stable = versions.filter((v) => !v.prerelease);
versions = [...prereleases, ...stable];
}
const versionItems = versions.map((v) => ({
value: v.version,
label: buildLabel(v),
disabled: false,
}));
if (installedMissing) {
const ghostItem = {
value: installedVersion,
label: `v${installedVersion} (installed)`,
disabled: true,
};
// Insert in sorted position (newest first, matching manifest order convention)
const idx = versionItems.findIndex(
(item) => compareVersions(installedVersion, item.value) > 0
);
if (idx === -1) {
versionItems.push(ghostItem);
} else {
versionItems.splice(idx, 0, ghostItem);
}
}
return (
<>
<Group gap="xs" align="flex-end">
<Select
label="Version"
size="xs"
allowDeselect={false}
value={selectedVersion}
onChange={onVersionChange}
data={versionItems}
style={{ maxWidth: 240 }}
/>
<Group gap="xs" align="center">
{btnProps.tooltip ? (
<Tooltip label={btnProps.tooltip}>
<Button
size="xs"
variant={btnProps.variant}
color={btnProps.color}
leftSection={btnProps.icon}
disabled={btnDisabled}
onClick={handleInstallClick}
>
{btnProps.label}
</Button>
</Tooltip>
) : (
<Button
size="xs"
variant={btnProps.variant}
color={btnProps.color}
leftSection={btnProps.icon}
disabled={btnDisabled}
onClick={handleInstallClick}
>
{btnProps.label}
</Button>
)}
{!selCompatible && selectedVersionData && !isSelSame && (() => {
const parts = [];
if (!selMeetsMin) parts.push(`${selectedVersionData.min_dispatcharr_version} or newer`);
if (!selMeetsMax) parts.push(`${selectedVersionData.max_dispatcharr_version} or older`);
const label = !selMeetsMin
? `Min ${selectedVersionData.min_dispatcharr_version}`
: `Max ${selectedVersionData.max_dispatcharr_version}`;
return (
<Tooltip label={`Incompatible: requires Dispatcharr ${parts.join(' and ')} (you have v${appVersion})`}>
<Group gap={4} align="center" wrap="nowrap">
<AlertTriangle size={14} color="var(--mantine-color-yellow-6)" />
<Text size="xs" c="yellow">{label}</Text>
</Group>
</Tooltip>
);
})()}
</Group>
</Group>
{selectedVersionData && (
<Table fontSize="xs" striped highlightOnHover style={{ tableLayout: 'auto' }}>
<Table.Tbody>
{selectedVersionData.build_timestamp && (
<Table.Tr>
<Table.Td fw={500} style={{ whiteSpace: 'nowrap' }}>Built</Table.Td>
<Table.Td>{new Date(selectedVersionData.build_timestamp).toLocaleString()}</Table.Td>
</Table.Tr>
)}
{selectedVersionData.min_dispatcharr_version && (
<Table.Tr>
<Table.Td fw={500} style={{ whiteSpace: 'nowrap' }}>Min Version</Table.Td>
<Table.Td>{selectedVersionData.min_dispatcharr_version}</Table.Td>
</Table.Tr>
)}
{selectedVersionData.max_dispatcharr_version && (
<Table.Tr>
<Table.Td fw={500} style={{ whiteSpace: 'nowrap' }}>Max Version</Table.Td>
<Table.Td>{selectedVersionData.max_dispatcharr_version}</Table.Td>
</Table.Tr>
)}
{selectedVersionData.commit_sha_short && (
<Table.Tr>
<Table.Td fw={500} style={{ whiteSpace: 'nowrap' }}>Commit</Table.Td>
<Table.Td>
{manifest.registry_url ? (
<Text
size="xs"
component="a"
href={`${manifest.registry_url}/commit/${selectedVersionData.commit_sha}`}
target="_blank"
rel="noopener noreferrer"
c="blue"
>
{selectedVersionData.commit_sha_short}
</Text>
) : (
selectedVersionData.commit_sha_short
)}
</Table.Td>
</Table.Tr>
)}
{selectedVersionData.url && (
<Table.Tr>
<Table.Td fw={500} style={{ whiteSpace: 'nowrap' }}>Download</Table.Td>
<Table.Td>
<Text
size="xs"
component="a"
href={selectedVersionData.url}
target="_blank"
rel="noopener noreferrer"
c="blue"
>
{selectedVersionData.url.split('/').pop()}
</Text>
</Table.Td>
</Table.Tr>
)}
</Table.Tbody>
</Table>
)}
</>
);
})()}
</Stack>
);
};
export default PluginDetailPanel;

2
frontend/src/components/ProgramDetailModal.jsx
View file

@ -16,7 +16,7 @@ import API from '../api';
import useVideoStore from '../store/useVideoStore';
import useSettingsStore from '../store/settings';
import { getShowVideoUrl } from '../utils/cards/RecordingCardUtils';
import { formatSeasonEpisode } from '../pages/guideUtils';
import { formatSeasonEpisode } from '../utils/guideUtils';
import {
format,
initializeTime,

71
frontend/src/components/Sidebar.jsx
View file

@ -1,12 +1,7 @@
import React, { useRef, useState, useMemo } from 'react';
import { Link, useLocation } from 'react-router-dom';
import { copyToClipboard } from '../utils';
import {
Copy,
LogOut,
ChevronDown,
ChevronRight,
} from 'lucide-react';
import { Copy, LogOut, ChevronDown, ChevronRight, Heart } from 'lucide-react';
import { getOrderedNavItems } from '../config/navigation';
import {
Avatar,
@ -19,6 +14,7 @@ import {
ActionIcon,
AppShellNavbar,
ScrollArea,
Tooltip,
} from '@mantine/core';
import logo from '../images/logo.png';
import useChannelsStore from '../store/channels';
@ -29,6 +25,21 @@ import { USER_LEVELS } from '../constants';
import UserForm from './forms/User';
import NotificationCenter from './NotificationCenter';
const DonateButton = ({ tooltipPosition = 'top' }) => (
<Tooltip label="Support Dispatcharr" position={tooltipPosition}>
<ActionIcon
component="a"
href="https://opencollective.com/dispatcharr/contribute"
target="_blank"
rel="noopener noreferrer"
variant="transparent"
color="pink"
>
<Heart size={20} />
</ActionIcon>
</Tooltip>
);
const NavLink = ({ item, isActive, collapsed }) => {
const IconComponent = item.icon;
return (
@ -328,24 +339,56 @@ const Sidebar = ({ collapsed, toggleDrawer, drawerWidth, miniDrawerWidth }) => {
{!collapsed && (
<Group
gap="xs"
wrap="nowrap"
style={{ padding: '0 16px 16px', justifyContent: 'space-between' }}
>
<Text size="xs" c="dimmed">
v{appVersion?.version || '0.0.0'}
{appVersion?.timestamp ? `-${appVersion.timestamp}` : ''}
</Text>
{isAuthenticated && <NotificationCenter />}
<Tooltip
label={`v${appVersion?.version || '0.0.0'}${appVersion?.timestamp ? `-${appVersion.timestamp}` : ''}`}
position="top"
>
<Text
size="xs"
c="dimmed"
style={{
overflow: 'hidden',
textOverflow: 'ellipsis',
whiteSpace: 'nowrap',
minWidth: 0,
flex: 1,
cursor: 'pointer',
}}
onClick={() =>
copyToClipboard(
`v${appVersion?.version || '0.0.0'}${appVersion?.timestamp ? `-${appVersion.timestamp}` : ''}`,
{
successTitle: 'Copied',
successMessage: 'Version copied to clipboard',
}
)
}
>
v{appVersion?.version || '0.0.0'}
{appVersion?.timestamp ? `-${appVersion.timestamp}` : ''}
</Text>
</Tooltip>
<Group gap="xs" wrap="nowrap">
<DonateButton />
{isAuthenticated && <NotificationCenter />}
</Group>
</Group>
)}
{collapsed && isAuthenticated && (
{collapsed && (
<Box
style={{
padding: '0 16px 16px',
display: 'flex',
justifyContent: 'center',
flexDirection: 'column',
alignItems: 'center',
gap: 8,
}}
>
<NotificationCenter />
{isAuthenticated && <NotificationCenter />}
<DonateButton tooltipPosition="right" />
</Box>
)}

7
frontend/src/components/__tests__/FloatingVideo.test.jsx
View file

@ -150,6 +150,10 @@ describe('FloatingVideo', () => {
type: 'mpegts',
url: 'http://example.com/stream.ts',
isLive: true,
}),
expect.objectContaining({
enableWorker: true,
enableStashBuffer: false,
})
);
});
@ -239,8 +243,9 @@ describe('FloatingVideo', () => {
const { container } = render(<FloatingVideo />);
const video = container.querySelector('video');
// Simulate video loaded event to clear loading state
// Simulate video loaded and canplay events to clear loading state and show overlay
fireEvent.loadedData(video);
fireEvent.canPlay(video);
expect(screen.getAllByText('Test Movie').length).toBeGreaterThanOrEqual(
1

2
frontend/src/components/__tests__/GuideRow.test.jsx
View file

@ -6,7 +6,7 @@ import {
CHANNEL_WIDTH,
HOUR_WIDTH,
PROGRAM_HEIGHT,
} from '../../pages/guideUtils';
} from '../../utils/guideUtils';
// Mock logo import
vi.mock('../../images/logo.png', () => ({

49
frontend/src/components/__tests__/HourTimeline.test.jsx
View file

@ -3,7 +3,7 @@ import { render, screen, fireEvent } from '@testing-library/react';
import '@testing-library/jest-dom';
import HourTimeline from '../HourTimeline';
import { format } from '../../utils/dateTimeUtils';
import { HOUR_WIDTH } from '../../pages/guideUtils';
import { HOUR_WIDTH } from '../../utils/guideUtils';
// Mock date utilities
vi.mock('../../utils/dateTimeUtils', () => ({
@ -108,7 +108,9 @@ describe('HourTimeline', () => {
/>
);
const hourBlocks = container.querySelectorAll('[style*="cursor: pointer"]');
const hourBlocks = container.querySelectorAll(
'[style*="cursor: pointer"]'
);
hourBlocks.forEach((block) => {
expect(block).toHaveAttribute('w', `${HOUR_WIDTH}`);
expect(block).toHaveAttribute('h', '40px');
@ -126,16 +128,16 @@ describe('HourTimeline', () => {
/>
);
const hourBlocks = container.querySelectorAll('[style*="cursor: pointer"]');
const hourBlocks = container.querySelectorAll(
'[style*="cursor: pointer"]'
);
expect(hourBlocks[0]).toHaveStyle({
backgroundColor: '#1B2421',
});
});
it('should apply special styling for new day blocks', () => {
const newDayTimeline = [
{ time: mockTime3, isNewDay: true },
];
const newDayTimeline = [{ time: mockTime3, isNewDay: true }];
const { container } = render(
<HourTimeline
@ -154,9 +156,7 @@ describe('HourTimeline', () => {
});
it('should apply bold font weight to day label on new day', () => {
const newDayTimeline = [
{ time: mockTime3, isNewDay: true },
];
const newDayTimeline = [{ time: mockTime3, isNewDay: true }];
const { container } = render(
<HourTimeline
@ -198,7 +198,9 @@ describe('HourTimeline', () => {
/>
);
const markers = container.querySelectorAll('[style*="background-color: rgb(113, 128, 150);"]');
const markers = container.querySelectorAll(
'[style*="background-color: rgb(113, 128, 150);"]'
);
expect(markers.length).toBe(3); // 15, 30, 45 minute markers
});
@ -212,7 +214,9 @@ describe('HourTimeline', () => {
/>
);
const markers = container.querySelectorAll('[style*="backgroundColor: #718096"]');
const markers = container.querySelectorAll(
'[style*="backgroundColor: #718096"]'
);
const positions = ['25%', '50%', '75%'];
markers.forEach((marker, index) => {
@ -238,10 +242,15 @@ describe('HourTimeline', () => {
/>
);
const hourBlocks = container.querySelectorAll('[style*="cursor: pointer"]');
const hourBlocks = container.querySelectorAll(
'[style*="cursor: pointer"]'
);
fireEvent.click(hourBlocks[0]);
expect(mockHandleTimeClick).toHaveBeenCalledWith(mockTime1, expect.any(Object));
expect(mockHandleTimeClick).toHaveBeenCalledWith(
mockTime1,
expect.any(Object)
);
});
it('should call handleTimeClick with correct time for each block', () => {
@ -254,13 +263,21 @@ describe('HourTimeline', () => {
/>
);
const hourBlocks = container.querySelectorAll('[style*="cursor: pointer"]');
const hourBlocks = container.querySelectorAll(
'[style*="cursor: pointer"]'
);
fireEvent.click(hourBlocks[0]);
expect(mockHandleTimeClick).toHaveBeenCalledWith(mockTime1, expect.any(Object));
expect(mockHandleTimeClick).toHaveBeenCalledWith(
mockTime1,
expect.any(Object)
);
fireEvent.click(hourBlocks[1]);
expect(mockHandleTimeClick).toHaveBeenCalledWith(mockTime2, expect.any(Object));
expect(mockHandleTimeClick).toHaveBeenCalledWith(
mockTime2,
expect.any(Object)
);
});
});

1
frontend/src/components/__tests__/M3URefreshNotification.test.jsx
View file

@ -57,6 +57,7 @@ vi.mock('@mantine/core', async () => {
// Mock lucide-react icons
vi.mock('lucide-react', () => ({
ListOrdered: () => <div data-testid="icon-list-ordered" />,
CircleCheck: () => <div data-testid="circle-check-icon" />,
}));

119
frontend/src/components/__tests__/NotificationCenter.test.jsx
View file

@ -33,8 +33,12 @@ vi.mock('@mantine/core', async () => {
{children}
</div>
),
PopoverTarget: ({ children }) => <div data-testid="popover-target">{children}</div>,
PopoverDropdown: ({ children }) => <div data-testid="popover-dropdown">{children}</div>,
PopoverTarget: ({ children }) => (
<div data-testid="popover-target">{children}</div>
),
PopoverDropdown: ({ children }) => (
<div data-testid="popover-dropdown">{children}</div>
),
Indicator: ({ children, label, disabled, processing }) => (
<div
data-testid="indicator"
@ -46,18 +50,53 @@ vi.mock('@mantine/core', async () => {
</div>
),
ActionIcon: ({ children, onClick, 'aria-label': ariaLabel, ...props }) => (
<button onClick={onClick} aria-label={ariaLabel} data-testid={`action-icon-${ariaLabel}`} {...props}>
<button
onClick={onClick}
aria-label={ariaLabel}
data-testid={`action-icon-${ariaLabel}`}
{...props}
>
{children}
</button>
),
ScrollAreaAutosize: ({ children }) => <div data-testid="scroll-area">{children}</div>,
Badge: ({ children, ...props }) => <span data-testid="badge" {...props}>{children}</span>,
Card: ({ children, ...props }) => <div data-testid="notification-card" {...props}>{children}</div>,
ThemeIcon: ({ children, ...props }) => <div data-testid="theme-icon" {...props}>{children}</div>,
Group: ({ children, ...props }) => <div data-testid="group" {...props}>{children}</div>,
Stack: ({ children, ...props }) => <div data-testid="stack" {...props}>{children}</div>,
Box: ({ children, ...props }) => <div data-testid="box" {...props}>{children}</div>,
Text: ({ children, ...props }) => <span data-testid="text" {...props}>{children}</span>,
ScrollAreaAutosize: ({ children }) => (
<div data-testid="scroll-area">{children}</div>
),
Badge: ({ children, ...props }) => (
<span data-testid="badge" {...props}>
{children}
</span>
),
Card: ({ children, ...props }) => (
<div data-testid="notification-card" {...props}>
{children}
</div>
),
ThemeIcon: ({ children, ...props }) => (
<div data-testid="theme-icon" {...props}>
{children}
</div>
),
Group: ({ children, ...props }) => (
<div data-testid="group" {...props}>
{children}
</div>
),
Stack: ({ children, ...props }) => (
<div data-testid="stack" {...props}>
{children}
</div>
),
Box: ({ children, ...props }) => (
<div data-testid="box" {...props}>
{children}
</div>
),
Text: ({ children, ...props }) => (
<span data-testid="text" {...props}>
{children}
</span>
),
Button: ({ children, onClick, ...props }) => (
<button onClick={onClick} data-testid="button" {...props}>
{children}
@ -84,14 +123,19 @@ vi.mock('@mantine/core', async () => {
// Mock lucide-react icons
vi.mock('lucide-react', () => ({
ListOrdered: () => <span data-testid="icon-list-ordered">ListOrdered</span>,
Bell: () => <span data-testid="bell-icon">Bell</span>,
Check: () => <span data-testid="check-icon">Check</span>,
CheckCheck: () => <span data-testid="checkcheck-icon">CheckCheck</span>,
Download: () => <span data-testid="download-icon">Download</span>,
ExternalLink: () => <span data-testid="external-link-icon">ExternalLink</span>,
ExternalLink: () => (
<span data-testid="external-link-icon">ExternalLink</span>
),
Info: () => <span data-testid="info-icon">Info</span>,
Settings: () => <span data-testid="settings-icon">Settings</span>,
AlertTriangle: () => <span data-testid="alert-triangle-icon">AlertTriangle</span>,
AlertTriangle: () => (
<span data-testid="alert-triangle-icon">AlertTriangle</span>
),
Megaphone: () => <span data-testid="megaphone-icon">Megaphone</span>,
X: () => <span data-testid="x-icon">X</span>,
Eye: () => <span data-testid="eye-icon">Eye</span>,
@ -279,8 +323,12 @@ describe('NotificationCenter', () => {
});
it('should handle API errors gracefully', async () => {
const consoleError = vi.spyOn(console, 'error').mockImplementation(() => {});
NotificationUtils.getNotifications.mockRejectedValue(new Error('Network error'));
const consoleError = vi
.spyOn(console, 'error')
.mockImplementation(() => {});
NotificationUtils.getNotifications.mockRejectedValue(
new Error('Network error')
);
renderComponent();
@ -311,7 +359,7 @@ describe('NotificationCenter', () => {
fireEvent.click(screen.getByLabelText('Notifications'));
const eyeButtons = screen.getAllByTestId(/action-icon-/);
const toggleButton = eyeButtons.find(btn =>
const toggleButton = eyeButtons.find((btn) =>
btn.querySelector('[data-testid="eye-icon"]')
);
fireEvent.click(toggleButton);
@ -376,7 +424,10 @@ describe('NotificationCenter', () => {
fireEvent.click(xIcons[0].closest('button'));
await waitFor(() => {
expect(NotificationUtils.dismissNotification).toHaveBeenCalledWith(1, 'dismissed');
expect(NotificationUtils.dismissNotification).toHaveBeenCalledWith(
1,
'dismissed'
);
});
});
@ -445,7 +496,10 @@ describe('NotificationCenter', () => {
fireEvent.click(applyButton);
await waitFor(() => {
expect(NotificationUtils.dismissNotification).toHaveBeenCalledWith(2, 'applied');
expect(NotificationUtils.dismissNotification).toHaveBeenCalledWith(
2,
'applied'
);
expect(onSettingAction).toHaveBeenCalledWith(mockNotifications[1]);
});
});
@ -458,7 +512,10 @@ describe('NotificationCenter', () => {
fireEvent.click(ignoreButton);
await waitFor(() => {
expect(NotificationUtils.dismissNotification).toHaveBeenCalledWith(2, 'dismissed');
expect(NotificationUtils.dismissNotification).toHaveBeenCalledWith(
2,
'dismissed'
);
});
});
@ -520,7 +577,7 @@ describe('NotificationCenter', () => {
fireEvent.click(screen.getByLabelText('Notifications'));
const eyeButtons = screen.getAllByTestId(/action-icon-/);
const toggleButton = eyeButtons.find(btn =>
const toggleButton = eyeButtons.find((btn) =>
btn.querySelector('[data-testid="eye-icon"]')
);
fireEvent.click(toggleButton);
@ -599,7 +656,7 @@ describe('NotificationCenter', () => {
fireEvent.click(screen.getByLabelText('Notifications'));
const eyeButtons = screen.getAllByTestId(/action-icon-/);
const toggleButton = eyeButtons.find(btn =>
const toggleButton = eyeButtons.find((btn) =>
btn.querySelector('[data-testid="eye-icon"]')
);
fireEvent.click(toggleButton);
@ -610,8 +667,12 @@ describe('NotificationCenter', () => {
describe('Error Handling', () => {
it('should handle dismiss notification errors', async () => {
const consoleError = vi.spyOn(console, 'error').mockImplementation(() => {});
NotificationUtils.dismissNotification.mockRejectedValue(new Error('API error'));
const consoleError = vi
.spyOn(console, 'error')
.mockImplementation(() => {});
NotificationUtils.dismissNotification.mockRejectedValue(
new Error('API error')
);
renderComponent();
fireEvent.click(screen.getByLabelText('Notifications'));
@ -630,8 +691,12 @@ describe('NotificationCenter', () => {
});
it('should handle dismiss all notifications errors', async () => {
const consoleError = vi.spyOn(console, 'error').mockImplementation(() => {});
NotificationUtils.dismissAllNotifications.mockRejectedValue(new Error('API error'));
const consoleError = vi
.spyOn(console, 'error')
.mockImplementation(() => {});
NotificationUtils.dismissAllNotifications.mockRejectedValue(
new Error('API error')
);
renderComponent();
fireEvent.click(screen.getByLabelText('Notifications'));
@ -655,7 +720,9 @@ describe('NotificationCenter', () => {
renderComponent();
fireEvent.click(screen.getByLabelText('Notifications'));
const expectedDate = new Date('2024-01-01T10:00:00Z').toLocaleDateString();
const expectedDate = new Date(
'2024-01-01T10:00:00Z'
).toLocaleDateString();
expect(screen.getByText(expectedDate)).toBeInTheDocument();
});
});

5
frontend/src/components/__tests__/ProgramDetailModal.test.jsx
View file

@ -21,9 +21,10 @@ vi.mock('../../utils/dateTimeUtils.js', () => ({
useDateTimeFormat: vi.fn(() => ({ timeFormat: 'h:mm A' })),
}));
vi.mock('../../pages/guideUtils', () => ({
vi.mock('../../utils/guideUtils', () => ({
formatSeasonEpisode: vi.fn((s, e) => {
if (s != null && e != null) return `S${String(s).padStart(2, '0')}E${String(e).padStart(2, '0')}`;
if (s != null && e != null)
return `S${String(s).padStart(2, '0')}E${String(e).padStart(2, '0')}`;
if (s != null) return `S${String(s).padStart(2, '0')}`;
if (e != null) return `E${String(e).padStart(2, '0')}`;
return null;

172
frontend/src/components/__tests__/SeriesModal.test.jsx
View file

@ -26,6 +26,7 @@ vi.mock('../../utils', () => ({
// Mock lucide-react icons
vi.mock('lucide-react', () => ({
ListOrdered: () => <div data-testid="icon-list-ordered" />,
Play: () => <div data-testid="play-icon" />,
Copy: () => <div data-testid="copy-icon" />,
}));
@ -40,27 +41,60 @@ vi.mock('@mantine/core', async () => {
if (!opened) return null;
return (
<div data-testid="modal" data-title={title} data-size={size}>
<button onClick={onClose} data-testid="modal-close">Close</button>
<button onClick={onClose} data-testid="modal-close">
Close
</button>
<div>{children}</div>
</div>
);
},
Box: ({ children, ...props }) => <div data-testid="box" {...props}>{children}</div>,
Box: ({ children, ...props }) => (
<div data-testid="box" {...props}>
{children}
</div>
),
Button: ({ children, onClick, disabled, ...props }) => (
<button onClick={onClick} disabled={disabled} data-testid="button" {...props}>
<button
onClick={onClick}
disabled={disabled}
data-testid="button"
{...props}
>
{children}
</button>
),
Flex: ({ children, ...props }) => <div data-testid="flex" {...props}>{children}</div>,
Group: ({ children, ...props }) => <div data-testid="group" {...props}>{children}</div>,
Flex: ({ children, ...props }) => (
<div data-testid="flex" {...props}>
{children}
</div>
),
Group: ({ children, ...props }) => (
<div data-testid="group" {...props}>
{children}
</div>
),
Image: ({ src, alt, ...props }) => (
<img src={src} alt={alt} data-testid="image" {...props} />
),
Text: ({ children, ...props }) => <div data-testid="text" {...props}>{children}</div>,
Title: ({ children, order, ...props }) => (
<div data-testid="title" data-order={order} {...props}>{children}</div>
Text: ({ children, ...props }) => (
<div data-testid="text" {...props}>
{children}
</div>
),
Select: ({ value, onChange, data, label, placeholder, disabled, ...props }) => (
Title: ({ children, order, ...props }) => (
<div data-testid="title" data-order={order} {...props}>
{children}
</div>
),
Select: ({
value,
onChange,
data,
label,
placeholder,
disabled,
...props
}) => (
<div data-testid="select" data-label={label}>
<select
value={value || ''}
@ -77,39 +111,76 @@ vi.mock('@mantine/core', async () => {
</select>
</div>
),
Badge: ({ children, ...props }) => <a data-testid="badge" {...props}>{children}</a>,
Badge: ({ children, ...props }) => (
<a data-testid="badge" {...props}>
{children}
</a>
),
Loader: (props) => <div data-testid="loader" {...props} />,
Stack: ({ children, ...props }) => <div data-testid="stack" {...props}>{children}</div>,
Stack: ({ children, ...props }) => (
<div data-testid="stack" {...props}>
{children}
</div>
),
ActionIcon: ({ children, onClick, disabled, ...props }) => (
<button onClick={onClick} disabled={disabled} data-testid="action-icon" {...props}>
<button
onClick={onClick}
disabled={disabled}
data-testid="action-icon"
{...props}
>
{children}
</button>
),
Tabs: ({ children, value, onChange, ...props }) => (
<div data-testid="tabs" data-value={value} {...props}>
<div onClick={(e) => {
const tab = e.target.closest('[data-tab-value]');
if (tab) onChange?.(tab.dataset.tabValue);
}}>
<div
onClick={(e) => {
const tab = e.target.closest('[data-tab-value]');
if (tab) onChange?.(tab.dataset.tabValue);
}}
>
{children}
</div>
</div>
),
TabsList: ({ children }) => <div data-testid="tabs-list">{children}</div>,
TabsTab: ({ children, value }) => (
<button data-testid="tabs-tab" data-tab-value={value}>{children}</button>
<button data-testid="tabs-tab" data-tab-value={value}>
{children}
</button>
),
TabsPanel: ({ children, value }) => (
<div data-testid="tabs-panel" data-value={value}>{children}</div>
<div data-testid="tabs-panel" data-value={value}>
{children}
</div>
),
Table: ({ children, ...props }) => (
<table data-testid="table" {...props}>
{children}
</table>
),
TableThead: ({ children }) => (
<thead data-testid="table-thead">{children}</thead>
),
TableTbody: ({ children }) => (
<tbody data-testid="table-tbody">{children}</tbody>
),
Table: ({ children, ...props }) => <table data-testid="table" {...props}>{children}</table>,
TableThead: ({ children }) => <thead data-testid="table-thead">{children}</thead>,
TableTbody: ({ children }) => <tbody data-testid="table-tbody">{children}</tbody>,
TableTr: ({ children, onClick, ...props }) => (
<tr onClick={onClick} data-testid="table-tr" {...props}>{children}</tr>
<tr onClick={onClick} data-testid="table-tr" {...props}>
{children}
</tr>
),
TableTh: ({ children, ...props }) => (
<th data-testid="table-th" {...props}>
{children}
</th>
),
TableTd: ({ children, ...props }) => (
<td data-testid="table-td" {...props}>
{children}
</td>
),
TableTh: ({ children, ...props }) => <th data-testid="table-th" {...props}>{children}</th>,
TableTd: ({ children, ...props }) => <td data-testid="table-td" {...props}>{children}</td>,
Divider: (props) => <hr data-testid="divider" {...props} />,
};
});
@ -168,7 +239,7 @@ describe('SeriesModal', () => {
m3u_account: { name: 'Provider 2' },
stream_name: 'Test Series 720p',
quality_info: null,
}
},
];
beforeEach(() => {
@ -187,9 +258,15 @@ describe('SeriesModal', () => {
environment: { env_mode: 'prod' },
};
useVODStore.mockImplementation((selector) => selector ? selector(mockVODStore) : mockVODStore);
useVideoStore.mockImplementation((selector) => selector ? selector(mockVideoStore) : mockVideoStore);
useSettingsStore.mockImplementation((selector) => selector ? selector(mockSettingsStore) : mockSettingsStore);
useVODStore.mockImplementation((selector) =>
selector ? selector(mockVODStore) : mockVODStore
);
useVideoStore.mockImplementation((selector) =>
selector ? selector(mockVideoStore) : mockVideoStore
);
useSettingsStore.mockImplementation((selector) =>
selector ? selector(mockSettingsStore) : mockSettingsStore
);
copyToClipboard.mockResolvedValue(undefined);
});
@ -325,23 +402,37 @@ describe('SeriesModal', () => {
it('should display IMDB link when imdb_id exists', async () => {
render(
<SeriesModal series={mockDetailedSeries} opened={true} onClose={vi.fn()} />
<SeriesModal
series={mockDetailedSeries}
opened={true}
onClose={vi.fn()}
/>
);
await waitFor(() => {
const link = screen.getByText(/IMDB/i).closest('a');
expect(link).toHaveAttribute('href', 'https://www.imdb.com/title/tt1234567');
expect(link).toHaveAttribute(
'href',
'https://www.imdb.com/title/tt1234567'
);
});
});
it('should display TMDB link when tmdb_id exists', async () => {
render(
<SeriesModal series={mockDetailedSeries} opened={true} onClose={vi.fn()} />
<SeriesModal
series={mockDetailedSeries}
opened={true}
onClose={vi.fn()}
/>
);
await waitFor(() => {
const link = screen.getByText(/TMDB/i).closest('a');
expect(link).toHaveAttribute('href', 'https://www.themoviedb.org/tv/12345');
expect(link).toHaveAttribute(
'href',
'https://www.themoviedb.org/tv/12345'
);
});
});
});
@ -446,7 +537,12 @@ describe('SeriesModal', () => {
});
it('should sort episodes by episode number', async () => {
const episode2 = { ...mockEpisode, id: 2, episode_number: 2, name: 'Second Episode' };
const episode2 = {
...mockEpisode,
id: 2,
episode_number: 2,
name: 'Second Episode',
};
mockVODStore.fetchSeriesInfo.mockResolvedValue({
...mockDetailedSeries,
episodesList: [episode2, mockEpisode],
@ -592,7 +688,12 @@ describe('SeriesModal', () => {
describe('Season Tabs', () => {
it('should create tabs for each season', async () => {
const season2Episode = { ...mockEpisode, id: 2, season_number: 2, episode_num: 1 };
const season2Episode = {
...mockEpisode,
id: 2,
season_number: 2,
episode_num: 1,
};
mockVODStore.fetchSeriesInfo.mockResolvedValue({
...mockDetailedSeries,
episodesList: [mockEpisode, season2Episode],
@ -670,7 +771,6 @@ describe('SeriesModal', () => {
<SeriesModal series={mockSeries} opened={true} onClose={vi.fn()} />
);
await waitFor(() => {
expect(screen.getByText(/Provider 1 - 1080p/)).toBeInTheDocument();
});
@ -797,7 +897,7 @@ describe('SeriesModal', () => {
);
await waitFor(() => {
expect(screen.getByText('Test Account')).toBeInTheDocument()
expect(screen.getByText('Test Account')).toBeInTheDocument();
});
});
});

4
frontend/src/components/__tests__/Sidebar.test.jsx
View file

@ -56,6 +56,9 @@ vi.mock('lucide-react', () => ({
ChevronRight: () => <div data-testid="chevron-right-icon" />,
MonitorCog: () => <div data-testid="monitor-cog-icon" />,
Blocks: () => <div data-testid="blocks-icon" />,
Heart: () => <div data-testid="heart-icon" />,
Package: () => <div data-testid="package-icon" />,
Download: () => <div data-testid="download-icon" />,
}));
// Mock UserForm component
@ -114,6 +117,7 @@ vi.mock('@mantine/core', async () => {
</nav>
),
ScrollArea: ({ children }) => <div>{children}</div>,
Tooltip: ({ children }) => <>{children}</>,
};
});

769
frontend/src/components/cards/AvailablePluginCard.jsx Normal file
View file

@ -0,0 +1,769 @@
import React, { useState } from 'react';
import {
ActionIcon,
Avatar,
Badge,
Box,
Button,
Card,
Group,
Loader,
Modal,
Stack,
Switch,
Text,
Tooltip,
} from '@mantine/core';
import { AlertTriangle, Ban, Check, Download, FlaskConical, Info, RefreshCw, RotateCcw, ShieldAlert, ShieldCheck, Trash2 } from 'lucide-react';
import { useNavigate, useLocation } from 'react-router-dom';
import API from '../../api';
import { usePluginStore } from '../../store/plugins';
import PluginDetailPanel from '../PluginDetailPanel.jsx';
import { compareVersions } from '../pluginUtils.js';
const RepoBadge = ({ isOfficial, repoName, signatureVerified }) => {
if (isOfficial) {
const badge = (
<Badge
size="xs"
variant="filled"
style={{ backgroundColor: signatureVerified === false ? 'var(--mantine-color-red-9)' : '#14917E' }}
leftSection={signatureVerified != null ? (signatureVerified ? <ShieldCheck size={10} /> : <ShieldAlert size={10} />) : undefined}
>
Official Repo
</Badge>
);
return signatureVerified != null ? (
<Tooltip label={signatureVerified ? 'Verified Signature' : 'Invalid Signature'}>{badge}</Tooltip>
) : badge;
}
if (!repoName) return null;
const badge = (
<Badge
size="xs"
variant="filled"
color={signatureVerified === false ? 'red.9' : 'gray'}
leftSection={signatureVerified != null ? (signatureVerified ? <ShieldCheck size={10} /> : <ShieldAlert size={10} />) : undefined}
>
{repoName}
</Badge>
);
return signatureVerified != null ? (
<Tooltip label={signatureVerified ? 'Verified Signature' : 'Invalid Signature'}>{badge}</Tooltip>
) : badge;
};
const StatusBadge = ({ status, deprecated, isPrerelease, isLatestDowngrade, installedSourceRepoName }) => {
if (status === 'installed') {
const baseLabel = isPrerelease ? 'Prerelease' : 'Installed';
if (!deprecated) {
return (
<Badge size="xs" variant="light" color={isPrerelease ? 'violet' : 'green'} leftSection={isPrerelease ? <FlaskConical size={8} /> : <Check size={8} />}>
{baseLabel}
</Badge>
);
}
return (
<Tooltip label={`${isPrerelease ? 'Prerelease installed' : 'Installed'}, but this plugin has been deprecated by its maintainer`}>
<Badge size="xs" variant="light" color={isPrerelease ? 'red' : 'orange'} leftSection={<Ban size={8} />}>
{baseLabel} · Deprecated
</Badge>
</Tooltip>
);
}
if (status === 'update_available') {
const baseLabel = isLatestDowngrade ? 'Newer Installed' : 'Update Available';
if (!deprecated) {
return (
<Badge size="xs" variant="light" color={isLatestDowngrade ? 'orange' : 'yellow'} leftSection={isLatestDowngrade ? <AlertTriangle size={8} /> : <RefreshCw size={8} />}>
{baseLabel}
</Badge>
);
}
return (
<Tooltip label="Update available, but this plugin has been deprecated by its maintainer">
<Badge size="xs" variant="light" color="red" leftSection={<Ban size={8} />}>
{baseLabel} · Deprecated
</Badge>
</Tooltip>
);
}
if (status === 'unmanaged' || status === 'different_repo') {
const tooltip = status === 'unmanaged'
? (deprecated ? 'Installed manually (deprecated) - installing from this repo will take over management' : 'Installed manually - installing from this repo will take over management')
: `Managed by ${installedSourceRepoName || 'another repo'}${deprecated ? ' (deprecated)' : ''}`;
return (
<Tooltip label={tooltip}>
<Badge size="xs" variant="light" color={deprecated ? 'red' : 'orange'} leftSection={deprecated ? <Ban size={8} /> : <Check size={8} />}>
{deprecated ? 'Installed · Deprecated' : 'Installed'}
</Badge>
</Tooltip>
);
}
if (deprecated) {
return (
<Tooltip label="This plugin has been marked as deprecated by its maintainer">
<Badge size="xs" variant="light" color="red" leftSection={<Ban size={8} />}>
Deprecated
</Badge>
</Tooltip>
);
}
return null;
};
const AvailablePluginCard = ({ plugin, appVersion, multiRepo = false, autoOpenDetail = false, onDetailClose, onInstalled, onUninstalled, onBeforeInstall }) => {
const meetsMinVersion = !plugin.min_dispatcharr_version || compareVersions(appVersion, plugin.min_dispatcharr_version) >= 0;
const meetsMaxVersion = !plugin.max_dispatcharr_version || compareVersions(appVersion, plugin.max_dispatcharr_version) <= 0;
const meetsVersion = meetsMinVersion && meetsMaxVersion;
const [detailOpen, setDetailOpen] = useState(false);
const [detail, setDetail] = useState(null);
const [detailLoading, setDetailLoading] = useState(false);
const [selectedVersion, setSelectedVersion] = useState(null);
const [installing, setInstalling] = useState(false);
const [confirmOpen, setConfirmOpen] = useState(false);
const [restartPromptOpen, setRestartPromptOpen] = useState(false);
const [installAction, setInstallAction] = useState(null); // 'installed' | 'updated' | 'downgraded'
const [pendingInstall, setPendingInstall] = useState(null);
const [installedKey, setInstalledKey] = useState(null);
const [enableNow, setEnableNow] = useState(false);
const [enabling, setEnabling] = useState(false);
const [pluginIsDisabled, setPluginIsDisabled] = useState(false);
const [uninstallConfirmOpen, setUninstallConfirmOpen] = useState(false);
const [uninstalling, setUninstalling] = useState(false);
const [deprecationWarnOpen, setDeprecationWarnOpen] = useState(false);
const [pendingDeprecatedInstall, setPendingDeprecatedInstall] = useState(null);
const installPlugin = usePluginStore((s) => s.installPlugin);
const navigate = useNavigate();
const { pathname } = useLocation();
const onMyPlugins = pathname === '/plugins';
const isLatestDowngrade = plugin.install_status === 'update_available' &&
plugin.latest_version && plugin.installed_version &&
compareVersions(plugin.latest_version, plugin.installed_version) < 0;
const doInstall = (params) => {
if (plugin.deprecated) {
setPendingDeprecatedInstall(params);
setDeprecationWarnOpen(true);
return;
}
setPendingInstall(params);
setConfirmOpen(true);
};
const confirmDeprecatedInstall = () => {
setDeprecationWarnOpen(false);
if (pendingDeprecatedInstall) {
setPendingInstall(pendingDeprecatedInstall);
setPendingDeprecatedInstall(null);
setConfirmOpen(true);
}
};
const confirmAndInstall = () => {
setConfirmOpen(false);
if (pendingInstall) executeInstall(pendingInstall);
};
const executeInstall = async (params) => {
const wasInstalled = plugin.installed;
const wasDowngrade = plugin.installed_version && params.version &&
compareVersions(params.version, plugin.installed_version) < 0;
onBeforeInstall?.(plugin.slug);
setInstalling(true);
const result = await installPlugin(params);
setInstalling(false);
setPendingInstall(null);
if (result?.success) {
setInstallAction(wasDowngrade ? 'downgraded' : wasInstalled ? 'updated' : 'installed');
setInstalledKey(result.plugin?.key || params.slug);
setPluginIsDisabled(result.plugin?.enabled === false);
setEnableNow(false);
setRestartPromptOpen(true);
onInstalled?.(plugin.slug);
}
};
const [uninstallDoneOpen, setUninstallDoneOpen] = useState(false);
const handleDismissRestart = async (andNavigate = false) => {
if (enableNow && installedKey) {
setEnabling(true);
try {
await API.setPluginEnabled(installedKey, true);
} finally {
setEnabling(false);
}
}
setRestartPromptOpen(false);
if (andNavigate) navigate('/plugins');
};
const handleUninstall = async () => {
const key = plugin.key || installedKey;
if (!key) return;
setUninstalling(true);
try {
const resp = await API.deletePlugin(key);
if (resp?.success) {
onUninstalled?.(plugin.slug);
usePluginStore.getState().invalidatePlugins();
usePluginStore.getState().fetchAvailablePlugins();
setUninstallConfirmOpen(false);
setUninstallDoneOpen(true);
}
} finally {
setUninstalling(false);
}
};
const handleMoreInfo = async () => {
setDetailOpen(true);
if (detailLoading) return;
if (!plugin.manifest_url) {
// No per-plugin manifest — synthesize from top-level repo entry (latest only)
setDetail({
manifest: {
description: plugin.description,
author: plugin.author,
license: plugin.license,
versions: plugin.latest_version ? [{
version: plugin.latest_version,
url: plugin.latest_url,
checksum_sha256: plugin.latest_sha256,
min_dispatcharr_version: plugin.min_dispatcharr_version,
max_dispatcharr_version: plugin.max_dispatcharr_version,
build_timestamp: plugin.last_updated,
}] : [],
latest: plugin.latest_version ? { version: plugin.latest_version } : null,
},
signature_verified: plugin.signature_verified ?? null,
});
if (plugin.latest_version) setSelectedVersion(plugin.latest_version);
return;
}
setDetailLoading(true);
const result = await API.getPluginDetailManifest(plugin.repo_id, plugin.manifest_url);
if (result) {
setDetail(result);
if (result.manifest?.versions?.length) {
setSelectedVersion(result.manifest.versions[0].version);
}
}
setDetailLoading(false);
};
React.useEffect(() => {
if (autoOpenDetail) handleMoreInfo();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
const latestInstallParams = {
repo_id: plugin.repo_id,
slug: plugin.slug,
version: plugin.latest_version,
download_url: plugin.latest_url,
sha256: plugin.latest_sha256,
min_dispatcharr_version: plugin.min_dispatcharr_version,
max_dispatcharr_version: plugin.max_dispatcharr_version,
};
return (
<Card
shadow="sm"
radius="md"
withBorder
style={{
display: 'flex',
flexDirection: 'column',
minHeight: 220,
backgroundColor: '#27272A',
...(multiRepo && plugin.is_official_repo ? { borderColor: '#0e6459' } : {}),
}}
>
<Group justify="space-between" mb="xs" align="flex-start" wrap="nowrap">
<Group gap="sm" align="flex-start" wrap="nowrap" style={{ minWidth: 0, flex: 1 }}>
<Avatar
src={plugin.icon_url}
radius="sm"
size={48}
alt={`${plugin.name} logo`}
>
{plugin.name?.[0]?.toUpperCase()}
</Avatar>
<Box style={{ minWidth: 0, flex: 1 }}>
<Text fw={600} lineClamp={1}>
{plugin.name}
</Text>
<Group gap={6} align="center" wrap="nowrap">
{plugin.author && (
<Text size="xs" c="dimmed" truncate style={{ minWidth: 0, maxWidth: '100%' }}>
{plugin.author}
</Text>
)}
<StatusBadge
status={plugin.install_status}
deprecated={plugin.deprecated}
isPrerelease={plugin.installed_version_is_prerelease}
isLatestDowngrade={isLatestDowngrade}
installedSourceRepoName={plugin.installed_source_repo_name}
/>
</Group>
</Box>
</Group>
<Group gap={4} wrap="nowrap" style={{ flexShrink: 0 }}>
<RepoBadge
isOfficial={plugin.is_official_repo}
repoName={plugin.repo_name}
signatureVerified={plugin.signature_verified}
/>
</Group>
</Group>
<Box style={{ flex: 1, minHeight: 0, overflow: 'hidden', display: 'flex', flexDirection: 'column' }}>
<div style={{ overflow: 'hidden' }}>
<Text size="sm" c="dimmed" lineClamp={3} mb={0}>
{plugin.description}
</Text>
</div>
<Stack gap={2} mt="auto" pt={4} style={{ flexShrink: 0 }}>
<Group gap="xs" wrap="wrap">
{plugin.latest_version && (
<Badge size="xs" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>LATEST</span>
v{plugin.latest_version}
</Badge>
)}
{plugin.license && (
<Badge
size="xs"
variant="default"
component="a"
href={`https://spdx.org/licenses/${encodeURIComponent(plugin.license)}.html`}
target="_blank"
rel="noopener noreferrer"
style={{ cursor: 'pointer' }}
>
<span style={{ opacity: 0.5, marginRight: 4 }}>LICENSE</span>
{plugin.license}
</Badge>
)}
{plugin.min_dispatcharr_version && (
<Badge size="xs" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>MIN</span>
{plugin.min_dispatcharr_version}
</Badge>
)}
{plugin.max_dispatcharr_version && (
<Badge size="xs" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>MAX</span>
{plugin.max_dispatcharr_version}
</Badge>
)}
{plugin.last_updated && (
<Badge size="xs" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>UPDATED</span>
{new Date(plugin.last_updated).toLocaleDateString()}
</Badge>
)}
</Group>
</Stack>
</Box>
<Group justify="space-between" mt="sm" align="center" wrap="nowrap">
{!meetsVersion && (() => {
const parts = [];
if (!meetsMinVersion) parts.push(`${plugin.min_dispatcharr_version} or newer`);
if (!meetsMaxVersion) parts.push(`${plugin.max_dispatcharr_version} or older`);
const label = !meetsMinVersion
? `Min ${plugin.min_dispatcharr_version}`
: `Max ${plugin.max_dispatcharr_version}`;
return (
<Tooltip label={`Incompatible: requires Dispatcharr ${parts.join(' and ')} (you have v${appVersion})`}>
<Group gap={4} align="center" wrap="nowrap">
<AlertTriangle size={14} color="var(--mantine-color-yellow-6)" />
<Text size="xs" c="yellow">{label}</Text>
</Group>
</Tooltip>
);
})()}
{meetsVersion && <span />}
<Group gap="xs" wrap="nowrap">
<Button
size="xs"
variant="default"
leftSection={<Info size={14} />}
onClick={handleMoreInfo}
>
More Info
</Button>
{(plugin.install_status === 'unmanaged') && plugin.latest_version && plugin.latest_url && (
<Tooltip label="Installed manually - installing from this repo will take over management">
<Button
size="xs"
variant="filled"
color="orange"
leftSection={installing ? <Loader size={14} /> : <Download size={14} />}
disabled={!meetsVersion || installing}
onClick={() => doInstall(latestInstallParams)}
>
{installing ? 'Installing...' : 'Overwrite'}
</Button>
</Tooltip>
)}
{(plugin.install_status === 'different_repo') && plugin.latest_url && (
<Tooltip label={`Managed by ${plugin.installed_source_repo_name || 'another repo'} - installing will transfer management to this repo`}>
<Button
size="xs"
variant="filled"
color="orange"
leftSection={installing ? <Loader size={14} /> : <Download size={14} />}
disabled={!meetsVersion || installing}
onClick={() => doInstall(latestInstallParams)}
>
{installing ? 'Installing...' : 'Overwrite'}
</Button>
</Tooltip>
)}
{(plugin.install_status === 'installed') && (
<Button
size="xs"
variant="light"
color="red"
leftSection={<Trash2 size={14} />}
onClick={() => setUninstallConfirmOpen(true)}
>
Uninstall
</Button>
)}
{(plugin.install_status === 'update_available') && (
<Button
size="xs"
variant="filled"
color={isLatestDowngrade ? 'orange' : 'yellow'}
leftSection={installing ? <Loader size={14} /> : isLatestDowngrade ? <AlertTriangle size={14} /> : <RefreshCw size={14} />}
disabled={!meetsVersion || installing}
onClick={() => doInstall(latestInstallParams)}
>
{installing
? (isLatestDowngrade ? 'Downgrading...' : 'Updating...')
: (isLatestDowngrade ? 'Downgrade' : 'Update')}
</Button>
)}
{(!plugin.install_status || plugin.install_status === 'not_installed') && plugin.latest_url && (
<Button
size="xs"
variant="filled"
leftSection={installing ? <Loader size={14} /> : <Download size={14} />}
disabled={!meetsVersion || installing}
onClick={() => doInstall(latestInstallParams)}
>
{installing ? 'Installing...' : 'Install'}
</Button>
)}
</Group>
</Group>
{/* Detail Modal */}
<Modal
opened={detailOpen}
onClose={() => { setDetailOpen(false); onDetailClose?.(); }}
title={
<Group gap="xs" align="center">
<Avatar
src={plugin.icon_url}
radius="sm"
size={28}
alt={`${plugin.name} logo`}
>
{plugin.name?.[0]?.toUpperCase()}
</Avatar>
<Text fw={600}>{plugin.name}</Text>
<RepoBadge
isOfficial={plugin.is_official_repo}
repoName={plugin.repo_name}
signatureVerified={detail?.signature_verified ?? plugin.signature_verified}
/>
</Group>
}
size="lg"
>
<PluginDetailPanel
detail={detail}
detailLoading={detailLoading}
selectedVersion={selectedVersion}
onVersionChange={setSelectedVersion}
installedVersion={plugin.installed_version}
installedVersionIsPrerelease={!!plugin.installed_version_is_prerelease}
appVersion={appVersion}
installing={installing}
uninstalling={uninstalling}
onInstall={doInstall}
onUninstall={() => setUninstallConfirmOpen(true)}
installStatus={plugin.install_status}
installedSourceRepoName={plugin.installed_source_repo_name}
repoId={plugin.repo_id}
slug={plugin.slug}
/>
</Modal>
{/* Deprecation warning modal */}
<Modal
opened={deprecationWarnOpen}
onClose={() => { setDeprecationWarnOpen(false); setPendingDeprecatedInstall(null); }}
zIndex={300}
title={
<Group gap="xs" align="center">
<Ban size={18} color="var(--mantine-color-red-6)" />
<Text fw={600}>Deprecated Plugin</Text>
</Group>
}
size="sm"
>
<Stack gap="md">
<Text size="sm">
<b>{plugin.name}</b> has been marked as <b>deprecated</b> by its maintainer.
</Text>
<Text size="sm" c="dimmed">
Deprecated plugins may no longer receive updates or fixes, and could stop working with future
versions of Dispatcharr. It is recommended to look for an alternative.
</Text>
<Text size="sm" fw={500}>Do you still want to proceed?</Text>
<Group justify="flex-end" gap="xs">
<Button
size="xs"
variant="default"
onClick={() => { setDeprecationWarnOpen(false); setPendingDeprecatedInstall(null); }}
>
Cancel
</Button>
<Button
size="xs"
color="red"
leftSection={<Ban size={14} />}
onClick={confirmDeprecatedInstall}
>
Install Anyway
</Button>
</Group>
</Stack>
</Modal>
{/* Unified install confirmation modal */}
{(() => {
const isDowngrade = pendingInstall && plugin.installed_version &&
compareVersions(pendingInstall.version, plugin.installed_version) < 0;
const isUpdate = pendingInstall && plugin.installed_version &&
!isDowngrade &&
compareVersions(pendingInstall.version, plugin.installed_version) > 0;
const isBadSig = plugin.signature_verified === false;
const actionLabel = isDowngrade ? 'Downgrade' : isUpdate ? 'Update' : 'Install';
const btnColor = (isDowngrade && isBadSig) ? 'red' : isDowngrade ? 'orange' : isBadSig ? 'red' : undefined;
return (
<Modal
opened={confirmOpen}
onClose={() => { setConfirmOpen(false); setPendingInstall(null); }}
zIndex={300}
title={
<Group gap="xs" align="center">
{isBadSig
? <ShieldAlert size={18} color="var(--mantine-color-red-6)" />
: isDowngrade
? <AlertTriangle size={18} color="var(--mantine-color-orange-6)" />
: <Download size={18} />}
<Text fw={600}>Confirm {actionLabel}</Text>
</Group>
}
size="sm"
>
<Stack gap="md">
<Text size="sm">
You are about to {actionLabel.toLowerCase()} <b>{plugin.name}</b>{' '}
{isUpdate || isDowngrade
? <>from <b>v{plugin.installed_version}</b> to <b>v{pendingInstall?.version}</b></>
: <><b>v{pendingInstall?.version}</b></>}
{plugin.repo_name ? <> from <b>{plugin.repo_name}</b></> : ''}.
</Text>
<Text size="sm" c="dimmed">
Plugins run server-side code with full access to your Dispatcharr instance and its
data. Only install plugins from developers you trust. Malicious plugins could read
or modify data, call internal APIs, or perform unwanted actions.
</Text>
{isDowngrade && (
<Text size="sm" c="orange">
<b>Warning:</b> Downgrading may cause issues with saved settings or data.
</Text>
)}
{isBadSig && (
<Text size="sm" c="red">
<b>Warning:</b> This repository has an invalid or unverified signature.
Installing plugins from unverified sources may be risky.
</Text>
)}
{plugin.install_status === 'unmanaged' && (
<Text size="sm" c="orange">
<b>Note:</b> This plugin was installed manually. Installing from this repo
will bring it under repo management and enable future update checks.
</Text>
)}
{plugin.install_status === 'different_repo' && (
<Text size="sm" c="orange">
<b>Note:</b> This plugin is currently managed
by <b>{plugin.installed_source_repo_name || 'another repo'}</b>.
Installing will transfer management to this repo.
</Text>
)}
<Text size="sm" fw={500}>Are you sure you want to proceed?</Text>
<Group justify="flex-end" gap="xs">
<Button
size="xs"
variant="default"
onClick={() => { setConfirmOpen(false); setPendingInstall(null); }}
>
Cancel
</Button>
<Button
size="xs"
color={btnColor}
onClick={confirmAndInstall}
>
{actionLabel}
</Button>
</Group>
</Stack>
</Modal>
);
})()}
{/* Uninstall confirmation modal */}
<Modal
opened={uninstallConfirmOpen}
onClose={() => setUninstallConfirmOpen(false)}
zIndex={300}
title={
<Group gap="xs" align="center">
<Trash2 size={18} color="var(--mantine-color-red-6)" />
<Text fw={600}>Uninstall Plugin</Text>
</Group>
}
size="sm"
>
<Stack gap="md">
<Text size="sm">
Are you sure you want to uninstall <b>{plugin.name}</b>? This will
remove the plugin files and all associated settings.
</Text>
<Group justify="flex-end" gap="xs">
<Button
size="xs"
variant="default"
onClick={() => setUninstallConfirmOpen(false)}
>
Cancel
</Button>
<Button
size="xs"
color="red"
loading={uninstalling}
onClick={handleUninstall}
>
Uninstall
</Button>
</Group>
</Stack>
</Modal>
{/* Post-uninstall notice */}
<Modal
opened={uninstallDoneOpen}
onClose={() => setUninstallDoneOpen(false)}
zIndex={300}
title={
<Group gap="xs" align="center">
<Trash2 size={18} color="var(--mantine-color-green-6)" />
<Text fw={600}>Plugin Uninstalled</Text>
</Group>
}
size="sm"
>
<Stack gap="md">
<Text size="sm">
<b>{plugin.name}</b> has been uninstalled successfully.
</Text>
<Text size="sm">
A restart of Dispatcharr may be required to fully unload the plugin.
</Text>
<Group justify="flex-end">
<Button size="xs" variant="default" onClick={() => setUninstallDoneOpen(false)}>
Done
</Button>
</Group>
</Stack>
</Modal>
{/* Post-install restart prompt */}
<Modal
opened={restartPromptOpen}
onClose={() => setRestartPromptOpen(false)}
zIndex={300}
title={
<Group gap="xs" align="center">
<RotateCcw size={18} color="var(--mantine-color-blue-6)" />
<Text fw={600}>
Plugin {installAction === 'installed' ? 'Installed' : installAction === 'downgraded' ? 'Downgraded' : 'Updated'}
</Text>
</Group>
}
size="sm"
>
<Stack gap="md">
<Text size="sm">
<b>{plugin.name}</b> has been {installAction || 'installed'} successfully.
</Text>
<Text size="sm">
A restart of Dispatcharr may be required for the plugin to be fully loaded.
</Text>
{pluginIsDisabled && (
<>
<Text size="xs" c="dimmed">
This plugin is currently disabled. You can enable it now or at any time from My Plugins.
</Text>
<Group justify="space-between" align="center">
<Text size="sm">Enable plugin</Text>
<Switch
size="sm"
checked={enableNow}
onChange={(e) => setEnableNow(e.currentTarget.checked)}
/>
</Group>
</>
)}
<Group justify="flex-end" gap="xs">
<Button
size="xs"
variant="default"
loading={enabling}
onClick={() => handleDismissRestart(false)}
>
Done
</Button>
{!onMyPlugins && (
<Button
size="xs"
loading={enabling}
onClick={() => handleDismissRestart(true)}
>
Go to My Plugins
</Button>
)}
</Group>
</Stack>
</Modal>
</Card>
);
};
export default AvailablePluginCard;

544
frontend/src/components/cards/PluginCard.jsx
View file

@ -2,23 +2,29 @@ import React, { useState } from 'react';
import { showNotification } from '../../utils/notificationUtils.js';
import { Field } from '../Field.jsx';
import {
ActionIcon,
Anchor,
Box,
Avatar,
Badge,
Box,
Button,
Card,
Divider,
Group,
Loader,
Modal,
Stack,
Switch,
Tabs,
Text,
UnstyledButton,
Badge,
Tooltip,
} from '@mantine/core';
import { ChevronDown, ChevronRight, Trash2 } from 'lucide-react';
import { Ban, Check, FlaskConical, Info, RefreshCw, Settings, Trash2, Zap } from 'lucide-react';
import { getConfirmationDetails } from '../../utils/cards/PluginCardUtils.js';
import { SUBSCRIPTION_EVENTS } from '../../constants.js';
import useSettingsStore from '../../store/settings.jsx';
import { usePluginStore } from '../../store/plugins.jsx';
import API from '../../api';
import PluginDetailPanel from '../PluginDetailPanel.jsx';
import { compareVersions } from '../pluginUtils.js';
const PluginFieldList = ({ plugin, settings, updateField }) => {
return plugin.fields.map((f) => (
@ -42,19 +48,19 @@ const PluginActionList = ({
return (
<Group key={action.id} justify="space-between">
<div>
<Text>{action.label}</Text>
<Text size="sm">{action.label}</Text>
{action.description && (
<Text size="sm" c="dimmed">
<Text size="xs" c="dimmed">
{action.description}
</Text>
)}
{events.length > 0 && (
<>
<Text size="xs" style={{ paddingTop: 10 }}>
<Text size="xs" style={{ paddingTop: 6 }}>
Event Triggers
</Text>
{events.map((event) => (
<Badge key={`${action.id}:${event}`} size="sm" variant="light" color="green">
<Badge key={`${action.id}:${event}`} size="xs" variant="light" color="green">
{SUBSCRIPTION_EVENTS[event] || event}
</Badge>
))}
@ -82,17 +88,17 @@ const PluginActionStatus = ({ running, lastResult }) => {
return (
<>
{running && (
<Text size="sm" c="dimmed">
<Text size="xs" c="dimmed">
Running action… please wait
</Text>
)}
{!running && lastResult?.file && (
<Text size="sm" c="dimmed">
<Text size="xs" c="dimmed">
Output: {lastResult.file}
</Text>
)}
{!running && lastResult?.error && (
<Text size="sm" c="red">
<Text size="xs" c="red">
Error: {String(lastResult.error)}
</Text>
)}
@ -109,27 +115,92 @@ const PluginCard = ({
onRequestDelete,
onRequestConfirm,
}) => {
const appVersion = useSettingsStore((s) => s.version?.version || '');
const [settings, setSettings] = useState(plugin.settings || {});
const [saving, setSaving] = useState(false);
const [runningActionId, setRunningActionId] = useState(null);
const [enabled, setEnabled] = useState(!!plugin.enabled);
const [lastResult, setLastResult] = useState(null);
const [expanded, setExpanded] = useState(!!plugin.enabled);
const [modalOpen, setModalOpen] = useState(false);
const [modalTab, setModalTab] = useState('settings');
const [detail, setDetail] = useState(null);
const [detailLoading, setDetailLoading] = useState(false);
const [selectedVersion, setSelectedVersion] = useState(null);
const [installing, setInstalling] = useState(false);
const [uninstalling] = useState(false);
// Keep local enabled state in sync with props (e.g., after import + enable)
const installPlugin = usePluginStore((s) => s.installPlugin);
// Keep local enabled state in sync with props
React.useEffect(() => {
setEnabled(!!plugin.enabled);
}, [plugin.enabled]);
React.useEffect(() => {
if (!plugin.enabled) {
setExpanded(false);
}
}, [plugin.enabled]);
// Sync settings if plugin changes identity
React.useEffect(() => {
setSettings(plugin.settings || {});
}, [plugin.key, plugin.settings]);
const hasActions = !plugin.missing && enabled && plugin.actions?.length > 0;
const isManaged = !!(plugin.slug && plugin.source_repo);
const fetchDetail = async () => {
if (detailLoading || !isManaged) return;
// Find the available plugin entry for manifest_url
let avail = usePluginStore.getState().availablePlugins.find(
(ap) => ap.slug === plugin.slug && ap.repo_id === plugin.source_repo
);
if (!avail) {
setDetailLoading(true);
try {
await usePluginStore.getState().fetchAvailablePlugins();
avail = usePluginStore.getState().availablePlugins.find(
(ap) => ap.slug === plugin.slug && ap.repo_id === plugin.source_repo
);
} catch { /* ignore */ }
}
if (!avail) { setDetailLoading(false); return; }
if (!avail.manifest_url) {
// Synthesize from top-level entry
setDetail({
manifest: {
description: avail.description,
author: avail.author,
license: avail.license,
repo_url: avail.repo_url,
discord_thread: avail.discord_thread,
registry_url: avail.registry_url,
versions: avail.latest_version ? [{
version: avail.latest_version,
url: avail.latest_url,
checksum_sha256: avail.latest_sha256,
min_dispatcharr_version: avail.min_dispatcharr_version,
max_dispatcharr_version: avail.max_dispatcharr_version,
build_timestamp: avail.last_updated,
}] : [],
latest: avail.latest_version ? { version: avail.latest_version } : null,
},
signature_verified: avail.signature_verified ?? null,
_avail: avail,
});
if (avail.latest_version) setSelectedVersion(avail.latest_version);
setDetailLoading(false);
return;
}
setDetailLoading(true);
try {
const result = await API.getPluginDetailManifest(avail.repo_id, avail.manifest_url);
if (result) {
setDetail({ ...result, _avail: avail });
if (result.manifest?.versions?.length) {
setSelectedVersion(result.manifest.versions[0].version);
}
}
} finally {
setDetailLoading(false);
}
};
const updateField = (id, val) => {
setSettings((prev) => ({ ...prev, [id]: val }));
};
@ -170,7 +241,6 @@ const PluginCard = ({
if (next && !plugin.ever_enabled && onRequireTrust) {
const ok = await onRequireTrust(plugin);
if (!ok) {
// Revert
setEnabled(false);
return;
}
@ -183,7 +253,7 @@ const PluginCard = ({
setEnabled(previous);
return;
}
} catch (e) {
} catch {
setEnabled(previous);
}
};
@ -191,17 +261,12 @@ const PluginCard = ({
const handlePluginRun = async (a) => {
try {
// Determine if confirmation is required from action metadata or fallback field
const { requireConfirm, confirmTitle, confirmMessage } =
getConfirmationDetails(a, plugin, settings);
if (requireConfirm) {
const confirmed = await onRequestConfirm(confirmTitle, confirmMessage);
if (!confirmed) {
// User canceled, abort the action
return;
}
if (!confirmed) return;
}
setRunningActionId(a.id);
@ -210,7 +275,7 @@ const PluginCard = ({
// Save settings before running to ensure backend uses latest values
try {
await onSaveSettings(plugin.key, settings);
} catch (e) {
} catch {
/* ignore, run anyway */
}
const resp = await onRunAction(plugin.key, a.id);
@ -236,149 +301,320 @@ const PluginCard = ({
}
};
const toggleExpanded = () => {
setExpanded((prev) => !prev);
const hasFields = !missing && enabled && plugin.fields?.length > 0;
const openModal = (tab) => {
setModalTab(tab);
setModalOpen(true);
if (tab === 'details') fetchDetail();
};
const handleDetailInstall = async (params) => {
const selVer = params.version;
const isDown = plugin.version && compareVersions(selVer, plugin.version) < 0;
const action = isDown ? 'downgrade' : 'update';
const confirmed = await onRequestConfirm(
`${isDown ? 'Downgrade' : 'Update'} ${plugin.name}?`,
`${isDown ? 'Downgrade' : 'Update'} from v${plugin.version} to v${selVer}?`
);
if (!confirmed) return;
setInstalling(true);
try {
const result = await installPlugin(params);
if (result?.success) {
showNotification({
title: plugin.name,
message: `Successfully ${action === 'downgrade' ? 'downgraded' : 'updated'} to v${selVer}`,
color: 'green',
});
usePluginStore.getState().invalidatePlugins();
}
} finally {
setInstalling(false);
}
};
const handleDetailUninstall = () => {
onRequestDelete && onRequestDelete(plugin);
};
return (
<Card
shadow="sm"
radius="md"
withBorder
style={{ opacity: !missing && enabled ? 1 : 0.6 }}
>
<Group justify="space-between" mb="xs" align="flex-start" wrap="nowrap">
<Group
gap="sm"
align="flex-start"
wrap="nowrap"
style={{ minWidth: 0, flex: 1 }}
>
<ActionIcon
variant="subtle"
size="sm"
onClick={toggleExpanded}
title={expanded ? 'Collapse settings' : 'Expand settings'}
>
{expanded ? <ChevronDown size={16} /> : <ChevronRight size={16} />}
</ActionIcon>
{plugin.logo_url && (
<div style={{ position: 'relative' }}>
<Card
shadow="sm"
radius="md"
withBorder
style={{
display: 'flex',
flexDirection: 'column',
height: '100%',
minHeight: 220,
backgroundColor: '#27272A',
opacity: !missing && enabled ? 1 : 0.6,
}}
>
{/* Header: avatar, name/author, badges, toggle */}
<Group justify="space-between" mb="xs" align="flex-start" wrap="nowrap">
<Group gap="sm" align="flex-start" wrap="nowrap" style={{ minWidth: 0, flex: 1 }}>
<Avatar
src={plugin.logo_url}
radius="sm"
size={44}
size={48}
alt={`${plugin.name} logo`}
/>
)}
<UnstyledButton
onClick={toggleExpanded}
style={{ minWidth: 0, flex: 1, textAlign: 'left' }}
>
onClick={isManaged ? () => openModal('details') : undefined}
style={isManaged ? { cursor: 'pointer' } : undefined}
>
{plugin.name?.[0]?.toUpperCase()}
</Avatar>
<Box style={{ minWidth: 0, flex: 1 }}>
<Text fw={600}>{plugin.name}</Text>
<Text size="sm" c="dimmed">
{plugin.description}
<Text
fw={600}
lineClamp={1}
onClick={isManaged ? () => openModal('details') : undefined}
style={isManaged ? { cursor: 'pointer' } : undefined}
>
{plugin.name}
</Text>
{(plugin.author || plugin.help_url) && (
<Group gap="xs" mt={2}>
{plugin.author && (
<Text size="xs" c="dimmed">
By {plugin.author}
</Text>
)}
{plugin.help_url && (
<Anchor
href={plugin.help_url}
target="_blank"
rel="noreferrer"
size="xs"
onClick={(e) => e.stopPropagation()}
>
Docs
</Anchor>
)}
</Group>
)}
<Group gap={6} align="center" wrap="nowrap">
{plugin.author && (
<Text
size="xs"
c="dimmed"
truncate
onClick={isManaged ? () => openModal('details') : undefined}
style={{ minWidth: 0, maxWidth: '100%', ...(isManaged ? { cursor: 'pointer' } : {}) }}
>
{plugin.author}
</Text>
)}
{plugin.help_url && (
<Anchor
href={plugin.help_url}
target="_blank"
rel="noreferrer"
size="xs"
>
Docs
</Anchor>
)}
</Group>
</Box>
</UnstyledButton>
</Group>
<Group gap={6} wrap="nowrap" align="center" style={{ flexShrink: 0 }}>
{plugin.is_managed && plugin.installed_version_is_prerelease ? (
<Tooltip label={plugin.deprecated ? 'Prerelease installed (deprecated), click for details' : 'Prerelease installed, click for details'}>
<Badge
size="xs"
variant="light"
color={plugin.deprecated ? 'red' : 'violet'}
leftSection={detailLoading ? <Loader size={8} /> : plugin.deprecated ? <Ban size={8} /> : <FlaskConical size={8} />}
style={{ cursor: 'pointer' }}
onClick={() => openModal('details')}
>
{plugin.deprecated ? 'Prerelease · Deprecated' : 'Prerelease'}
</Badge>
</Tooltip>
) : plugin.update_available ? (
<Tooltip label={plugin.deprecated ? `Update available: v${plugin.latest_version} (deprecated)` : `Update available: v${plugin.latest_version}`}>
<Badge
size="xs"
variant="light"
color={plugin.deprecated ? 'red' : 'yellow'}
leftSection={detailLoading ? <Loader size={8} /> : plugin.deprecated ? <Ban size={8} /> : <RefreshCw size={8} />}
style={{ cursor: 'pointer' }}
onClick={() => openModal('details')}
>
{plugin.deprecated ? 'Update · Deprecated' : 'Update'}
</Badge>
</Tooltip>
) : plugin.is_managed ? (
<Tooltip label={plugin.deprecated ? 'Installed (deprecated), click for details' : 'View plugin details'}>
<Badge
size="xs"
variant="light"
color={plugin.deprecated ? 'orange' : 'green'}
leftSection={detailLoading ? <Loader size={8} /> : plugin.deprecated ? <Ban size={8} /> : <Check size={8} />}
style={{ cursor: 'pointer' }}
onClick={() => openModal('details')}
>
{plugin.deprecated ? 'Deprecated' : 'Up to Date'}
</Badge>
</Tooltip>
) : (
<Badge size="xs" variant="light" color="gray">
Unmanaged
</Badge>
)}
<Switch
checked={!missing && enabled}
onChange={handleEnableChange()}
size="xs"
onLabel="On"
offLabel="Off"
disabled={missing}
/>
</Group>
</Group>
<Group gap="xs" align="center" wrap="nowrap" style={{ flexShrink: 0 }}>
<ActionIcon
variant="subtle"
{/* Description */}
<div style={{ overflow: 'hidden' }}>
<Text size="sm" c="dimmed" lineClamp={3} mb={0}>
{plugin.description}
</Text>
</div>
{/* Status warnings */}
{(missing || plugin.legacy) && (
<Text size="xs" c={missing ? 'red' : 'yellow'} mt="xs">
{missing
? 'Missing plugin files. Re-import or delete this entry.'
: 'Please update or ask the developer to add plugin.json.'}
</Text>
)}
{/* Bottom metadata pills */}
<Stack gap={2} mt="auto" pt={4} style={{ flexShrink: 0 }}>
<Group gap="xs" wrap="wrap">
<Badge size="xs" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>VERSION</span>
v{plugin.version || '1.0.0'}
</Badge>
{plugin.is_managed && plugin.source_repo_name && (
<Badge size="xs" variant="default">
<span style={{ opacity: 0.5, marginRight: 4 }}>REPO</span>
{plugin.source_repo_name}
</Badge>
)}
</Group>
</Stack>
{/* Bottom button row */}
<Group justify="flex-end" mt="sm" gap="xs">
{hasFields && (
<Button
size="xs"
variant="default"
leftSection={<Settings size={14} />}
onClick={() => openModal('settings')}
>
Settings
</Button>
)}
{hasActions && (
<Button
size="xs"
variant="light"
color="blue"
leftSection={<Zap size={14} />}
onClick={() => openModal('actions')}
>
Actions
</Button>
)}
<Button
size="xs"
variant="light"
color="red"
title="Delete plugin"
leftSection={<Trash2 size={14} />}
onClick={() => onRequestDelete && onRequestDelete(plugin)}
>
<Trash2 size={16} />
</ActionIcon>
<Text size="xs" c="dimmed">
v{plugin.version || '1.0.0'}
</Text>
<Switch
checked={!missing && enabled}
onChange={handleEnableChange()}
size="xs"
onLabel="On"
offLabel="Off"
disabled={missing}
/>
Uninstall
</Button>
</Group>
</Group>
</Card>
{(missing || plugin.legacy) && (
<Text size="sm" c={missing ? 'red' : 'yellow'}>
{missing
? 'Missing plugin files. Re-import or delete this entry.'
: 'Please update or ask the developer to add plugin.json.'}
</Text>
)}
{/* Settings & Actions Modal */}
<Modal
opened={modalOpen}
onClose={() => setModalOpen(false)}
title={
<Group gap="xs" align="center">
<Avatar src={plugin.logo_url} radius="sm" size={28} alt={`${plugin.name} logo`}>
{plugin.name?.[0]?.toUpperCase()}
</Avatar>
<Text fw={600}>{plugin.name}</Text>
</Group>
}
size="lg"
>
<Tabs value={modalTab} onChange={(tab) => { setModalTab(tab); if (tab === 'details') fetchDetail(); }}>
<Tabs.List>
{isManaged && <Tabs.Tab value="details" leftSection={<Info size={14} />}>Details</Tabs.Tab>}
{hasFields && <Tabs.Tab value="settings" leftSection={<Settings size={14} />}>Settings</Tabs.Tab>}
{hasActions && <Tabs.Tab value="actions" leftSection={<Zap size={14} />}>Actions</Tabs.Tab>}
</Tabs.List>
{expanded &&
!missing &&
enabled &&
plugin.fields &&
plugin.fields.length > 0 && (
<Stack gap="xs" mt="sm">
<PluginFieldList
plugin={plugin}
settings={settings}
updateField={updateField}
/>
<Group>
<Button
loading={saving}
onClick={save}
variant="default"
size="xs"
>
Save Settings
</Button>
</Group>
</Stack>
)}
{expanded &&
!missing &&
enabled &&
plugin.actions &&
plugin.actions.length > 0 && (
<>
<Divider my="sm" />
<Stack gap="xs">
<PluginActionList
plugin={plugin}
enabled={enabled}
runningActionId={runningActionId}
handlePluginRun={handlePluginRun}
{isManaged && (
<Tabs.Panel value="details" pt="md">
<PluginDetailPanel
detail={detail}
detailLoading={detailLoading}
selectedVersion={selectedVersion}
onVersionChange={setSelectedVersion}
installedVersion={plugin.version}
installedVersionIsPrerelease={!!plugin.installed_version_is_prerelease}
appVersion={appVersion}
installing={installing}
uninstalling={uninstalling}
onInstall={handleDetailInstall}
onUninstall={handleDetailUninstall}
installStatus="installed"
repoId={plugin.source_repo}
slug={plugin.slug}
/>
<PluginActionStatus
running={!!runningActionId}
lastResult={lastResult}
/>
</Stack>
</>
)}
</Card>
</Tabs.Panel>
)}
{hasFields && (
<Tabs.Panel value="settings" pt="md">
<Stack gap="md">
<PluginFieldList
plugin={plugin}
settings={settings}
updateField={updateField}
/>
<Group justify="flex-end">
<Button
variant="default"
size="xs"
onClick={() => setModalOpen(false)}
>
Cancel
</Button>
<Button
loading={saving}
onClick={async () => {
await save();
setModalOpen(false);
}}
size="xs"
>
Save
</Button>
</Group>
</Stack>
</Tabs.Panel>
)}
{hasActions && (
<Tabs.Panel value="actions" pt="md">
<Stack gap="sm">
<PluginActionList
plugin={plugin}
enabled={enabled}
runningActionId={runningActionId}
handlePluginRun={handlePluginRun}
/>
<PluginActionStatus
running={!!runningActionId}
lastResult={lastResult}
/>
</Stack>
</Tabs.Panel>
)}
</Tabs>
</Modal>
</div>
);
};

71
frontend/src/components/cards/StreamConnectionCard.jsx
View file

@ -2,6 +2,7 @@ import { useLocation } from 'react-router-dom';
import React, { useEffect, useMemo, useState } from 'react';
import usePlaylistsStore from '../../store/playlists.jsx';
import useSettingsStore from '../../store/settings.jsx';
import useUsersStore from '../../store/users.jsx';
import {
ActionIcon,
Badge,
@ -45,7 +46,6 @@ import {
getChannelStreams,
getLogoUrl,
getM3uAccountsMap,
getMatchingStreamByUrl,
getSelectedStream,
getStartDate,
getStreamOptions,
@ -123,6 +123,8 @@ const StreamConnectionCard = ({
// Get M3U account data from the playlists store
const m3uAccounts = usePlaylistsStore((s) => s.playlists);
// Get users for resolving user_id → username on client rows
const users = useUsersStore((s) => s.users);
// Get settings for speed threshold and environment mode
const settings = useSettingsStore((s) => s.settings);
const env_mode =
@ -138,6 +140,15 @@ const StreamConnectionCard = ({
return getM3uAccountsMap(m3uAccounts);
}, [m3uAccounts]);
// Create a map of user IDs to usernames for quick lookup
const usersMap = useMemo(() => {
const map = {};
users.forEach((u) => {
map[String(u.id)] = u.username;
});
return map;
}, [users]);
// Update M3U profile information when channel data changes
useEffect(() => {
// If the channel data includes M3U profile information, update our state
@ -164,18 +175,13 @@ const StreamConnectionCard = ({
// Use streams in the order returned by the API without sorting
setAvailableStreams(streamData);
// If we have a channel URL, try to find the matching stream
if (channel.url && streamData.length > 0) {
// Try to find matching stream based on URL
const matchingStream = getMatchingStreamByUrl(
streamData,
channel.url
// Match by server-reported stream_id.
if (channel.stream_id && streamData.length > 0) {
const matchingStream = streamData.find(
(s) => s.id.toString() === channel.stream_id.toString()
);
if (matchingStream) {
setActiveStreamId(matchingStream.id.toString());
// If the stream has M3U profile info, save it
if (matchingStream.m3u_profile) {
setCurrentM3UProfile(matchingStream.m3u_profile);
}
@ -190,7 +196,7 @@ const StreamConnectionCard = ({
};
fetchStreams();
}, [channel.channel_id, channel.url, channelsByUUID]);
}, [channel.channel_id, channel.stream_id, channelsByUUID]);
useEffect(() => {
setData(
@ -314,7 +320,8 @@ const StreamConnectionCard = ({
{
header: 'IP Address',
accessorKey: 'ip_address',
size: 150,
grow: true,
minSize: 85,
cell: ({ cell }) => (
<Tooltip label={cell.getValue()}>
<Text size="xs" truncate style={{ maxWidth: '100%' }}>
@ -323,10 +330,29 @@ const StreamConnectionCard = ({
</Tooltip>
),
},
{
id: 'user',
header: 'User',
grow: true,
minSize: 60,
accessorFn: (row) => {
const uid = row.user_id ? String(row.user_id) : null;
if (!uid || uid === '0') return 'Anonymous';
return usersMap[uid] || `User ${uid}`;
},
cell: ({ cell }) => (
<Text size="xs" truncate style={{ maxWidth: '100%' }}>
{cell.getValue()}
</Text>
),
},
// Updated Connected column with tooltip
{
id: 'connected',
header: 'Connected',
grow: 1.5,
minSize: 70,
maxSize: 150,
accessorFn: connectedAccessor(fullDateTimeFormat),
cell: ({ cell }) => (
<Tooltip
@ -336,7 +362,9 @@ const StreamConnectionCard = ({
: 'Unknown connection time'
}
>
<Text size="xs">{cell.getValue()}</Text>
<Text size="xs" truncate style={{ maxWidth: '100%' }}>
{cell.getValue()}
</Text>
</Tooltip>
),
},
@ -344,6 +372,8 @@ const StreamConnectionCard = ({
{
id: 'duration',
header: 'Duration',
size: 82,
minSize: 60,
accessorFn: durationAccessor(),
cell: ({ cell, row }) => {
const exactDuration =
@ -356,7 +386,9 @@ const StreamConnectionCard = ({
: 'Unknown duration'
}
>
<Text size="xs">{cell.getValue()}</Text>
<Text size="xs" style={{ whiteSpace: 'nowrap' }}>
{cell.getValue()}
</Text>
</Tooltip>
);
},
@ -364,10 +396,11 @@ const StreamConnectionCard = ({
{
id: 'actions',
header: 'Actions',
size: 100,
size: 60,
minSize: 40,
},
],
[fullDateTimeFormat]
[fullDateTimeFormat, usersMap]
);
const channelClientsTable = useTable({
@ -383,6 +416,7 @@ const StreamConnectionCard = ({
}),
headerCellRenderFns: {
ip_address: renderHeaderCell,
user: renderHeaderCell,
connected: renderHeaderCell,
duration: renderHeaderCell,
actions: renderHeaderCell,
@ -610,8 +644,9 @@ const StreamConnectionCard = ({
{currentProgram &&
isProgramDescExpanded &&
currentProgram.start_time &&
currentProgram.end_time &&
<ProgramProgress currentProgram={currentProgram} />}
currentProgram.end_time && (
<ProgramProgress currentProgram={currentProgram} />
)}
{/* Add stream selection dropdown and preview button */}
{availableStreams.length > 0 && (

33
frontend/src/components/cards/VodConnectionCard.jsx
View file

@ -1,5 +1,5 @@
// Format duration for content length
import React, { useCallback, useEffect, useState } from 'react';
import React, { useCallback, useEffect, useMemo, useState } from 'react';
import logo from '../../images/logo.png';
import {
ActionIcon,
@ -38,6 +38,7 @@ import {
getMovieDisplayTitle,
getMovieSubtitle,
} from '../../utils/cards/VodConnectionCardUtils.js';
import useUsersStore from '../../store/users.jsx';
const ClientDetails = ({ connection, connectionStartTime }) => {
return (
@ -142,7 +143,10 @@ const ClientDetails = ({ connection, connectionStartTime }) => {
};
const ConnectionProgress = ({ connection, durationSecs }) => {
const { totalTime, currentTime, percentage } = calculateProgress(connection, durationSecs);
const { totalTime, currentTime, percentage } = calculateProgress(
connection,
durationSecs
);
return totalTime > 0 ? (
<Stack gap="xs" mt="sm">
<Group justify="space-between" align="center">
@ -172,6 +176,14 @@ const ConnectionProgress = ({ connection, durationSecs }) => {
const VodConnectionCard = ({ vodContent, stopVODClient }) => {
const { fullDateTimeFormat } = useDateTimeFormat();
const [isClientExpanded, setIsClientExpanded] = useState(false);
const users = useUsersStore((s) => s.users);
const usersMap = useMemo(() => {
const map = {};
users.forEach((u) => {
map[String(u.id)] = u.username;
});
return map;
}, [users]);
const [, setUpdateTrigger] = useState(0); // Force re-renders for progress updates
// Get metadata from the VOD content
@ -377,13 +389,12 @@ const VodConnectionCard = ({ vodContent, stopVODClient }) => {
</Group>
{/* Progress bar - show current position in content */}
{connection &&
metadata.duration_secs &&
{connection && metadata.duration_secs && (
<ConnectionProgress
connection={connection}
durationSecs={metadata.duration_secs}
/>
}
)}
{/* Client information section - collapsible like channel cards */}
{connection && (
@ -403,11 +414,21 @@ const VodConnectionCard = ({ vodContent, stopVODClient }) => {
>
<Group gap={8}>
<Text size="sm" fw={500} color="dimmed">
Client:
Client IP:
</Text>
<Text size="sm" ff={'monospace'}>
{connection.client_ip || 'Unknown IP'}
</Text>
{usersMap[String(connection.user_id)] && (
<>
<Text size="sm" c="dimmed">
User:
</Text>
<Text size="sm">
{usersMap[String(connection.user_id)]}
</Text>
</>
)}
</Group>
<Group gap={8}>

92
frontend/src/components/cards/__tests__/PluginCard.test.jsx
View file

@ -54,6 +54,32 @@ vi.mock('@mantine/core', async () => {
Text: ({ children, ...props }) => <span {...props}>{children}</span>,
UnstyledButton: ({ children, ...props }) => <button {...props}>{children}</button>,
Badge: ({ children, ...props }) => <span {...props}>{children}</span>,
Loader: ({ size }) => <span data-testid="loader" data-size={size} />,
Modal: ({ opened, onClose, title, children }) =>
opened ? (
<div data-testid="modal">
<div data-testid="modal-title">{title}</div>
<button onClick={onClose}>Close Modal</button>
{children}
</div>
) : null,
Tabs: Object.assign(
({ children, value, onChange }) => (
<div data-testid="tabs" data-value={value}>{children}</div>
),
{
List: ({ children }) => <div>{children}</div>,
Tab: ({ children, value, leftSection }) => (
<button data-value={value}>{leftSection}{children}</button>
),
Panel: ({ children, value }) => (
<div data-testid={`tab-panel-${value}`}>{children}</div>
),
}
),
Tooltip: ({ children, label }) => (
<div title={label}>{children}</div>
),
};
});
@ -112,7 +138,7 @@ describe('PluginCard', () => {
expect(screen.getByText('Test Plugin')).toBeInTheDocument();
expect(screen.getByText('A test plugin')).toBeInTheDocument();
expect(screen.getByText('v1.0.0')).toBeInTheDocument();
expect(screen.getByText('By Test Author')).toBeInTheDocument();
expect(screen.getByText('Test Author')).toBeInTheDocument();
});
it('should render plugin logo when logo_url is provided', () => {
@ -166,43 +192,28 @@ describe('PluginCard', () => {
});
});
describe('Expansion/Collapse', () => {
it('should toggle expanded state when clicking chevron button', async () => {
render(<PluginCard plugin={mockPlugin} />);
await waitFor(() => {
fireEvent.click(screen.getByTitle('Collapse settings'));
expect(screen.queryByText('Test Action')).not.toBeInTheDocument();
});
await waitFor(() => {
fireEvent.click(screen.getByTitle('Expand settings'));
});
expect(await screen.findByText('Test Action')).toBeInTheDocument();
});
it('should toggle expanded state when clicking plugin name', () => {
describe('Modal Behavior', () => {
it('should open settings modal when Settings button is clicked', () => {
render(<PluginCard {...defaultProps} />);
expect(screen.getByText('Test Action')).toBeInTheDocument();
const nameButton = screen.getByText('Test Plugin');
fireEvent.click(nameButton);
expect(screen.queryByText('Test Action')).not.toBeInTheDocument();
expect(screen.queryByTestId('modal')).not.toBeInTheDocument();
fireEvent.click(screen.getByText('Settings'));
expect(screen.getByTestId('modal')).toBeInTheDocument();
});
it('should collapse when plugin is disabled', () => {
const { rerender } = render(<PluginCard {...defaultProps} />);
it('should open actions modal when Actions button is clicked', () => {
render(<PluginCard {...defaultProps} />);
const expandButton = screen.getAllByRole('button')[0];
fireEvent.click(expandButton);
expect(screen.queryByTestId('modal')).not.toBeInTheDocument();
fireEvent.click(screen.getByText('Actions'));
expect(screen.getByTestId('modal')).toBeInTheDocument();
});
it('should not show Actions button when plugin is disabled', () => {
const disabledPlugin = { ...mockPlugin, enabled: false };
rerender(<PluginCard {...defaultProps} plugin={disabledPlugin} />);
render(<PluginCard {...defaultProps} plugin={disabledPlugin} />);
expect(screen.queryByText('Test Action')).not.toBeInTheDocument();
expect(screen.queryByText('Actions')).not.toBeInTheDocument();
});
});
@ -278,7 +289,8 @@ describe('PluginCard', () => {
defaultProps.onSaveSettings.mockResolvedValue(true);
render(<PluginCard {...defaultProps} />);
const saveButton = screen.getByText('Save Settings');
fireEvent.click(screen.getByText('Settings'));
const saveButton = screen.getByText('Save');
fireEvent.click(saveButton);
await waitFor(() => {
@ -298,7 +310,8 @@ describe('PluginCard', () => {
defaultProps.onSaveSettings.mockResolvedValue(false);
render(<PluginCard {...defaultProps} />);
const saveButton = screen.getByText('Save Settings');
fireEvent.click(screen.getByText('Settings'));
const saveButton = screen.getByText('Save');
fireEvent.click(saveButton);
await waitFor(() => {
@ -315,7 +328,8 @@ describe('PluginCard', () => {
defaultProps.onSaveSettings.mockRejectedValue(error);
render(<PluginCard {...defaultProps} />);
const saveButton = screen.getByText('Save Settings');
fireEvent.click(screen.getByText('Settings'));
const saveButton = screen.getByText('Save');
fireEvent.click(saveButton);
await waitFor(() => {
@ -332,6 +346,7 @@ describe('PluginCard', () => {
it('should render action buttons', () => {
render(<PluginCard {...defaultProps} />);
fireEvent.click(screen.getByText('Actions'));
expect(screen.getByText('Run Action')).toBeInTheDocument();
});
@ -344,6 +359,7 @@ describe('PluginCard', () => {
render(<PluginCard {...defaultProps} />);
fireEvent.click(screen.getByText('Actions'));
const actionButton = screen.getByText('Run Action');
fireEvent.click(actionButton);
@ -369,6 +385,7 @@ describe('PluginCard', () => {
render(<PluginCard {...defaultProps} />);
fireEvent.click(screen.getByText('Actions'));
const actionButton = screen.getByText('Run Action');
fireEvent.click(actionButton);
@ -391,6 +408,7 @@ describe('PluginCard', () => {
render(<PluginCard {...defaultProps} />);
fireEvent.click(screen.getByText('Actions'));
const actionButton = screen.getByText('Run Action');
fireEvent.click(actionButton);
@ -412,6 +430,7 @@ describe('PluginCard', () => {
render(<PluginCard {...errorProps} />);
fireEvent.click(screen.getByText('Actions'));
const actionButton = screen.getByText('Run Action');
fireEvent.click(actionButton);
@ -439,6 +458,7 @@ describe('PluginCard', () => {
render(<PluginCard {...defaultProps} plugin={pluginWithEvents} />);
fireEvent.click(screen.getByText('Actions'));
expect(screen.getByText('Event Triggers')).toBeInTheDocument();
});
});
@ -447,7 +467,7 @@ describe('PluginCard', () => {
it('should call onRequestDelete when delete button is clicked', () => {
render(<PluginCard {...defaultProps} />);
const deleteButton = screen.getByTitle('Delete plugin');
const deleteButton = screen.getByText('Uninstall');
fireEvent.click(deleteButton);
expect(defaultProps.onRequestDelete).toHaveBeenCalledWith(mockPlugin);
@ -476,8 +496,8 @@ describe('PluginCard', () => {
};
rerender(<PluginCard {...defaultProps} plugin={newPlugin} />);
// Settings should be updated internally
expect(screen.getByText('Save Settings')).toBeInTheDocument();
// Settings button should still be present after key change
expect(screen.getByText('Settings')).toBeInTheDocument();
});
});
});

280
frontend/src/components/cards/__tests__/RecordingCard.test.jsx
View file

@ -109,13 +109,12 @@ vi.mock('@mantine/core', async () => ({
{children}
</span>
),
Tooltip: ({ children, label }) => (
<div data-tooltip={label}>{children}</div>
),
Tooltip: ({ children, label }) => <div data-tooltip={label}>{children}</div>,
}));
// ── lucide-react ───────────────────────────────────────────────────────────────
vi.mock('lucide-react', () => ({
ListOrdered: () => <svg data-testid="icon-list-ordered" />,
AlertTriangle: () => <svg data-testid="icon-alert-triangle" />,
Plus: () => <svg data-testid="icon-plus" />,
Square: () => <svg data-testid="icon-square" />,
@ -140,8 +139,13 @@ vi.mock('../../../images/logo.png', () => ({ default: 'default-logo.png' }));
import useChannelsStore from '../../../store/channels.jsx';
import useSettingsStore from '../../../store/settings.jsx';
import useVideoStore from '../../../store/useVideoStore.jsx';
import { useDateTimeFormat, useTimeHelpers, format, isAfter, isBefore }
from '../../../utils/dateTimeUtils.js';
import {
useDateTimeFormat,
useTimeHelpers,
format,
isAfter,
isBefore,
} from '../../../utils/dateTimeUtils.js';
import { notifications } from '@mantine/notifications';
import * as RecordingCardUtils from '../../../utils/cards/RecordingCardUtils.js';
import dayjs from 'dayjs';
@ -187,7 +191,11 @@ const makeChannel = () => ({
});
/** Wire up all store/utility mocks with sensible defaults */
const setupMocks = ({ now = NOW, recording = makeRecording(), channel = makeChannel() } = {}) => {
const setupMocks = ({
now = NOW,
recording = makeRecording(),
channel = makeChannel(),
} = {}) => {
const nowMoment = makeMoment(now);
const startMoment = makeMoment(recording.start_time);
const endMoment = makeMoment(recording.end_time);
@ -226,9 +234,13 @@ const setupMocks = ({ now = NOW, recording = makeRecording(), channel = makeChan
vi.mocked(RecordingCardUtils.getPosterUrl).mockReturnValue('/poster.jpg');
vi.mocked(RecordingCardUtils.getChannelLogoUrl).mockReturnValue('/logo.png');
vi.mocked(RecordingCardUtils.getRecordingUrl).mockReturnValue('/recordings/test.ts');
vi.mocked(RecordingCardUtils.getRecordingUrl).mockReturnValue(
'/recordings/test.ts'
);
vi.mocked(RecordingCardUtils.getSeasonLabel).mockReturnValue('');
vi.mocked(RecordingCardUtils.getSeriesInfo).mockReturnValue({ seriesId: 's1' });
vi.mocked(RecordingCardUtils.getSeriesInfo).mockReturnValue({
seriesId: 's1',
});
vi.mocked(RecordingCardUtils.getShowVideoUrl).mockReturnValue('/live/ch-1');
return { mockShowVideo, mockFetchRecordings };
@ -237,10 +249,18 @@ const setupMocks = ({ now = NOW, recording = makeRecording(), channel = makeChan
describe('RecordingCard', () => {
beforeEach(() => {
vi.clearAllMocks();
vi.mocked(RecordingCardUtils.stopRecordingById).mockResolvedValue(undefined);
vi.mocked(RecordingCardUtils.deleteRecordingById).mockResolvedValue(undefined);
vi.mocked(RecordingCardUtils.deleteSeriesAndRule).mockResolvedValue(undefined);
vi.mocked(RecordingCardUtils.extendRecordingById).mockResolvedValue(undefined);
vi.mocked(RecordingCardUtils.stopRecordingById).mockResolvedValue(
undefined
);
vi.mocked(RecordingCardUtils.deleteRecordingById).mockResolvedValue(
undefined
);
vi.mocked(RecordingCardUtils.deleteSeriesAndRule).mockResolvedValue(
undefined
);
vi.mocked(RecordingCardUtils.extendRecordingById).mockResolvedValue(
undefined
);
vi.mocked(RecordingCardUtils.runComSkip).mockResolvedValue(undefined);
vi.mocked(RecordingCardUtils.removeRecording).mockReturnValue(undefined);
});
@ -250,17 +270,23 @@ describe('RecordingCard', () => {
describe('rendering', () => {
it('renders the recording title', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('Test Show')).toBeInTheDocument();
});
it('renders "Custom Recording" when no program title', () => {
setupMocks({
recording: makeRecording({ custom_properties: { status: 'completed', program: {} } }),
recording: makeRecording({
custom_properties: { status: 'completed', program: {} },
}),
});
render(
<RecordingCard
recording={makeRecording({ custom_properties: { status: 'completed', program: {} } })}
recording={makeRecording({
custom_properties: { status: 'completed', program: {} },
})}
channel={makeChannel()}
/>
);
@ -269,7 +295,9 @@ describe('RecordingCard', () => {
it('renders channel info', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('501 • HBO')).toBeInTheDocument();
});
@ -281,27 +309,35 @@ describe('RecordingCard', () => {
it('shows description via RecordingSynopsis for non-series completed recording', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByTestId('recording-synopsis')).toBeInTheDocument();
expect(screen.getByText('A test description')).toBeInTheDocument();
});
it('shows sub_title when present', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('Pilot')).toBeInTheDocument();
});
it('shows season/episode label when getSeasonLabel returns a value', () => {
setupMocks();
vi.mocked(RecordingCardUtils.getSeasonLabel).mockReturnValue('S01E02');
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('S01E02')).toBeInTheDocument();
});
it('renders the poster image', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
const img = screen.getByAltText('Test Show');
expect(img).toHaveAttribute('src', '/poster.jpg');
});
@ -312,7 +348,9 @@ describe('RecordingCard', () => {
describe('status badge', () => {
it('shows "Completed" badge for a completed recording', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('Completed')).toBeInTheDocument();
});
@ -320,7 +358,10 @@ describe('RecordingCard', () => {
const recording = makeRecording({
start_time: PAST,
end_time: FUTURE,
custom_properties: { status: 'recording', program: { title: 'Live Show' } },
custom_properties: {
status: 'recording',
program: { title: 'Live Show' },
},
});
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
@ -331,7 +372,10 @@ describe('RecordingCard', () => {
const recording = makeRecording({
start_time: FUTURE,
end_time: FUTURE,
custom_properties: { status: 'scheduled', program: { title: 'Future Show' } },
custom_properties: {
status: 'scheduled',
program: { title: 'Future Show' },
},
});
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
@ -373,8 +417,7 @@ describe('RecordingCard', () => {
// ── Series group ───────────────────────────────────────────────────────────
describe('series group', () => {
const makeSeriesRecording = () =>
makeRecording({ _group_count: 3 });
const makeSeriesRecording = () => makeRecording({ _group_count: 3 });
it('shows "Series" badge when _group_count > 1', () => {
const recording = makeSeriesRecording();
@ -394,7 +437,9 @@ describe('RecordingCard', () => {
const recording = makeSeriesRecording();
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
expect(screen.queryByTestId('recording-synopsis')).not.toBeInTheDocument();
expect(
screen.queryByTestId('recording-synopsis')
).not.toBeInTheDocument();
});
});
@ -500,7 +545,9 @@ describe('RecordingCard', () => {
it('does not show "Watch Live" for a completed recording', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.queryByText('Watch Live')).not.toBeInTheDocument();
});
@ -523,7 +570,9 @@ describe('RecordingCard', () => {
it('shows "Watch" button for a completed recording', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('Watch')).toBeInTheDocument();
});
@ -531,7 +580,10 @@ describe('RecordingCard', () => {
const recording = makeRecording({
start_time: FUTURE,
end_time: FUTURE,
custom_properties: { status: 'scheduled', program: { title: 'Future' } },
custom_properties: {
status: 'scheduled',
program: { title: 'Future' },
},
});
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
@ -540,7 +592,9 @@ describe('RecordingCard', () => {
it('calls showVideo with vod params when Watch is clicked', () => {
const { mockShowVideo } = setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
fireEvent.click(screen.getByText('Watch'));
expect(mockShowVideo).toHaveBeenCalledWith(
'/recordings/test.ts',
@ -552,7 +606,9 @@ describe('RecordingCard', () => {
it('does not call showVideo when Watch is clicked but no file url', () => {
const { mockShowVideo } = setupMocks();
vi.mocked(RecordingCardUtils.getRecordingUrl).mockReturnValue(null);
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
fireEvent.click(screen.getByText('Watch'));
expect(mockShowVideo).not.toHaveBeenCalled();
});
@ -575,7 +631,9 @@ describe('RecordingCard', () => {
describe('"Remove commercials" button', () => {
it('shows "Remove commercials" for a completed recording without comskip', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
expect(screen.getByText('Remove commercials')).toBeInTheDocument();
});
@ -597,7 +655,10 @@ describe('RecordingCard', () => {
const recording = makeRecording({
start_time: FUTURE,
end_time: FUTURE,
custom_properties: { status: 'scheduled', program: { title: 'Future' } },
custom_properties: {
status: 'scheduled',
program: { title: 'Future' },
},
});
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
@ -606,20 +667,31 @@ describe('RecordingCard', () => {
it('calls runComSkip and shows notification on success', async () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
fireEvent.click(screen.getByText('Remove commercials'));
await waitFor(() => {
expect(RecordingCardUtils.runComSkip).toHaveBeenCalledWith(makeRecording());
expect(RecordingCardUtils.runComSkip).toHaveBeenCalledWith(
makeRecording()
);
expect(notifications.show).toHaveBeenCalledWith(
expect.objectContaining({ title: 'Removing commercials', color: 'blue.5' })
expect.objectContaining({
title: 'Removing commercials',
color: 'blue.5',
})
);
});
});
it('does not show notification when runComSkip throws', async () => {
vi.mocked(RecordingCardUtils.runComSkip).mockRejectedValue(new Error('fail'));
vi.mocked(RecordingCardUtils.runComSkip).mockRejectedValue(
new Error('fail')
);
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
fireEvent.click(screen.getByText('Remove commercials'));
await waitFor(() => {
expect(notifications.show).not.toHaveBeenCalled();
@ -634,7 +706,11 @@ describe('RecordingCard', () => {
makeRecording({
start_time: PAST,
end_time: FUTURE,
custom_properties: { status: 'recording', program: { title: 'Live Show' }, file_url: '/f.ts' },
custom_properties: {
status: 'recording',
program: { title: 'Live Show' },
file_url: '/f.ts',
},
});
it('shows extend menu for in-progress recording', () => {
@ -650,9 +726,15 @@ describe('RecordingCard', () => {
render(<RecordingCard recording={recording} channel={makeChannel()} />);
fireEvent.click(screen.getByText('+15 minutes'));
await waitFor(() => {
expect(RecordingCardUtils.extendRecordingById).toHaveBeenCalledWith('rec-1', 15);
expect(RecordingCardUtils.extendRecordingById).toHaveBeenCalledWith(
'rec-1',
15
);
expect(notifications.show).toHaveBeenCalledWith(
expect.objectContaining({ title: 'Recording extended', color: 'teal' })
expect.objectContaining({
title: 'Recording extended',
color: 'teal',
})
);
});
});
@ -663,7 +745,10 @@ describe('RecordingCard', () => {
render(<RecordingCard recording={recording} channel={makeChannel()} />);
fireEvent.click(screen.getByText('+30 minutes'));
await waitFor(() => {
expect(RecordingCardUtils.extendRecordingById).toHaveBeenCalledWith('rec-1', 30);
expect(RecordingCardUtils.extendRecordingById).toHaveBeenCalledWith(
'rec-1',
30
);
});
});
@ -673,12 +758,17 @@ describe('RecordingCard', () => {
render(<RecordingCard recording={recording} channel={makeChannel()} />);
fireEvent.click(screen.getByText('+1 hour'));
await waitFor(() => {
expect(RecordingCardUtils.extendRecordingById).toHaveBeenCalledWith('rec-1', 60);
expect(RecordingCardUtils.extendRecordingById).toHaveBeenCalledWith(
'rec-1',
60
);
});
});
it('shows error notification when extendRecordingById throws', async () => {
vi.mocked(RecordingCardUtils.extendRecordingById).mockRejectedValue(new Error('Network error'));
vi.mocked(RecordingCardUtils.extendRecordingById).mockRejectedValue(
new Error('Network error')
);
const recording = makeInProgress();
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
@ -698,7 +788,11 @@ describe('RecordingCard', () => {
makeRecording({
start_time: PAST,
end_time: FUTURE,
custom_properties: { status: 'recording', program: { title: 'Live Show' }, file_url: '/f.ts' },
custom_properties: {
status: 'recording',
program: { title: 'Live Show' },
file_url: '/f.ts',
},
});
it('shows stop modal when stop button is clicked', () => {
@ -711,7 +805,9 @@ describe('RecordingCard', () => {
fireEvent.click(stopButton);
expect(screen.getByTestId('modal')).toBeInTheDocument();
expect(screen.getByTestId('modal-title')).toHaveTextContent('Stop Recording');
expect(screen.getByTestId('modal-title')).toHaveTextContent(
'Stop Recording'
);
});
it('closes stop modal when Go Back is clicked', () => {
@ -736,7 +832,9 @@ describe('RecordingCard', () => {
fireEvent.click(screen.getAllByText('Stop Recording')[1]);
await waitFor(() => {
expect(RecordingCardUtils.stopRecordingById).toHaveBeenCalledWith('rec-1');
expect(RecordingCardUtils.stopRecordingById).toHaveBeenCalledWith(
'rec-1'
);
expect(mockFetchRecordings).toHaveBeenCalled();
});
});
@ -761,48 +859,71 @@ describe('RecordingCard', () => {
describe('delete recording', () => {
it('shows delete modal for a completed non-series recording', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
expect(screen.getByTestId('modal')).toBeInTheDocument();
expect(screen.getByTestId('modal-title')).toHaveTextContent('Delete Recording');
expect(screen.getByTestId('modal-title')).toHaveTextContent(
'Delete Recording'
);
});
it('shows "Cancel Recording" title for upcoming recording delete', () => {
const recording = makeRecording({
start_time: FUTURE,
end_time: FUTURE,
custom_properties: { status: 'scheduled', program: { title: 'Future Show' } },
custom_properties: {
status: 'scheduled',
program: { title: 'Future Show' },
},
});
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
expect(screen.getByTestId('modal-title')).toHaveTextContent('Cancel Recording');
expect(screen.getByTestId('modal-title')).toHaveTextContent(
'Cancel Recording'
);
});
it('calls removeRecording when delete is confirmed', async () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
fireEvent.click(screen.getByText('Delete'));
await waitFor(() => {
expect(RecordingCardUtils.removeRecording).toHaveBeenCalledWith('rec-1');
expect(RecordingCardUtils.removeRecording).toHaveBeenCalledWith(
'rec-1'
);
});
});
it('closes delete modal after confirming', async () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
fireEvent.click(screen.getByText('Delete'));
@ -813,9 +934,13 @@ describe('RecordingCard', () => {
it('closes delete modal on Go Back click', () => {
setupMocks();
render(<RecordingCard recording={makeRecording()} channel={makeChannel()} />);
render(
<RecordingCard recording={makeRecording()} channel={makeChannel()} />
);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
fireEvent.click(screen.getByText('Go Back'));
@ -842,10 +967,14 @@ describe('RecordingCard', () => {
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
expect(screen.getByTestId('modal-title')).toHaveTextContent('Cancel Series');
expect(screen.getByTestId('modal-title')).toHaveTextContent(
'Cancel Series'
);
});
it('calls deleteRecordingById when "Only this upcoming" is clicked', async () => {
@ -853,12 +982,16 @@ describe('RecordingCard', () => {
const { mockFetchRecordings } = setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
fireEvent.click(screen.getByText('Only this upcoming'));
await waitFor(() => {
expect(RecordingCardUtils.deleteRecordingById).toHaveBeenCalledWith('rec-1');
expect(RecordingCardUtils.deleteRecordingById).toHaveBeenCalledWith(
'rec-1'
);
expect(mockFetchRecordings).toHaveBeenCalled();
});
});
@ -868,7 +1001,9 @@ describe('RecordingCard', () => {
const { mockFetchRecordings } = setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
fireEvent.click(screen.getByText('Entire series + rule'));
@ -883,7 +1018,9 @@ describe('RecordingCard', () => {
setupMocks({ recording });
render(<RecordingCard recording={recording} channel={makeChannel()} />);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
fireEvent.click(screen.getByText('Only this upcoming'));
@ -918,13 +1055,18 @@ describe('RecordingCard', () => {
_group_count: 3,
start_time: FUTURE,
end_time: FUTURE,
custom_properties: { status: 'scheduled', program: { title: 'Series' } },
custom_properties: {
status: 'scheduled',
program: { title: 'Series' },
},
});
const { mockFetchRecordings } = setupMocks({ recording });
mockFetchRecordings.mockRejectedValue(new Error('network'));
render(<RecordingCard recording={recording} channel={makeChannel()} />);
const deleteButton = screen.getByTestId('icon-square-x').closest('button');
const deleteButton = screen
.getByTestId('icon-square-x')
.closest('button');
fireEvent.click(deleteButton);
await expect(
@ -932,4 +1074,4 @@ describe('RecordingCard', () => {
).resolves.not.toThrow();
});
});
});
});

60
frontend/src/components/cards/__tests__/SeriesCard.test.jsx
View file

@ -30,7 +30,13 @@ vi.mock('@mantine/core', async () => ({
),
Stack: ({ children, gap }) => <div data-testid="stack">{children}</div>,
Text: ({ children, size, fw, c, lineClamp, style }) => (
<span data-testid="text" data-size={size} data-fw={fw} data-color={c} style={style}>
<span
data-testid="text"
data-size={size}
data-fw={fw}
data-color={c}
style={style}
>
{children}
</span>
),
@ -38,6 +44,7 @@ vi.mock('@mantine/core', async () => ({
// ── lucide-react ───────────────────────────────────────────────────────────────
vi.mock('lucide-react', () => ({
ListOrdered: () => <svg data-testid="icon-list-ordered" />,
Calendar: () => <svg data-testid="icon-calendar" />,
Play: () => <svg data-testid="icon-play" />,
Star: () => <svg data-testid="icon-star" />,
@ -78,7 +85,12 @@ describe('SeriesCard', () => {
});
it('renders a fallback image when poster_url is missing', () => {
render(<SeriesCard series={makeSeries({ poster_url: null })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ poster_url: null })}
onClick={vi.fn()}
/>
);
const img = screen.getByRole('img');
expect(img).toBeInTheDocument();
});
@ -105,7 +117,12 @@ describe('SeriesCard', () => {
it('renders play icon', () => {
//this only renders when logo.url is missing, but we want to test that the icon itself renders correctly
render(<SeriesCard series={makeSeries({ logo: { url: null } })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ logo: { url: null } })}
onClick={vi.fn()}
/>
);
expect(screen.getByTestId('icon-play')).toBeInTheDocument();
});
@ -119,27 +136,52 @@ describe('SeriesCard', () => {
describe('optional fields', () => {
it('does not crash when year is missing', () => {
render(<SeriesCard series={makeSeries({ year: undefined })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ year: undefined })}
onClick={vi.fn()}
/>
);
expect(screen.getByTestId('series-card')).toBeInTheDocument();
});
it('does not crash when rating is missing', () => {
render(<SeriesCard series={makeSeries({ rating: undefined })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ rating: undefined })}
onClick={vi.fn()}
/>
);
expect(screen.getByTestId('series-card')).toBeInTheDocument();
});
it('does not crash when genre is missing', () => {
render(<SeriesCard series={makeSeries({ genre: undefined })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ genre: undefined })}
onClick={vi.fn()}
/>
);
expect(screen.getByTestId('series-card')).toBeInTheDocument();
});
it('does not crash when description is missing', () => {
render(<SeriesCard series={makeSeries({ description: undefined })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ description: undefined })}
onClick={vi.fn()}
/>
);
expect(screen.getByTestId('series-card')).toBeInTheDocument();
});
it('does not crash when seasons is missing', () => {
render(<SeriesCard series={makeSeries({ seasons: undefined })} onClick={vi.fn()} />);
render(
<SeriesCard
series={makeSeries({ seasons: undefined })}
onClick={vi.fn()}
/>
);
expect(screen.getByTestId('series-card')).toBeInTheDocument();
});
@ -161,4 +203,4 @@ describe('SeriesCard', () => {
expect(onClick).toHaveBeenCalledWith(series);
});
});
});
});

164
frontend/src/components/cards/__tests__/StreamConnectionCard.test.jsx
View file

@ -42,7 +42,6 @@ vi.mock('../../../utils/cards/StreamConnectionCardUtils.js', () => ({
getChannelStreams: vi.fn(() => Promise.resolve([])),
getLogoUrl: vi.fn(() => null),
getM3uAccountsMap: vi.fn(() => ({})),
getMatchingStreamByUrl: vi.fn(() => null),
getSelectedStream: vi.fn(() => null),
getStartDate: vi.fn(() => 'Jan 1 2024 10:00 AM'),
getStreamOptions: vi.fn(() => []),
@ -95,7 +94,12 @@ vi.mock('@mantine/core', () => ({
Center: ({ children }) => <div data-testid="center">{children}</div>,
Group: ({ children }) => <div data-testid="group">{children}</div>,
Progress: ({ value, size, color }) => (
<div data-testid="progress" data-value={value} data-size={size} data-color={color} />
<div
data-testid="progress"
data-value={value}
data-size={size}
data-color={color}
/>
),
Select: ({ value, onChange, label, data, disabled, placeholder }) => (
<select
@ -129,9 +133,7 @@ vi.mock('@mantine/core', () => ({
{children}
</span>
),
Tooltip: ({ children, label }) => (
<div data-tooltip={label}>{children}</div>
),
Tooltip: ({ children, label }) => <div data-tooltip={label}>{children}</div>,
useMantineTheme: vi.fn(() => ({
tailwind: { green: { 5: '#22c55e' } },
})),
@ -139,6 +141,22 @@ vi.mock('@mantine/core', () => ({
// ── lucide-react ──────────────────────────────────────────────────────────────
vi.mock('lucide-react', () => ({
// navigation.js icons (all must be present for the auth→navigation import chain)
ListOrdered: () => <svg data-testid="icon-list-ordered" />,
Play: () => <svg data-testid="icon-play" />,
Database: () => <svg data-testid="icon-database" />,
LayoutGrid: () => <svg data-testid="icon-layout-grid" />,
Settings: () => <svg data-testid="icon-settings" />,
ChartLine: () => <svg data-testid="icon-chart-line" />,
Video: () => <svg data-testid="icon-video" />,
PlugZap: () => <svg data-testid="icon-plug-zap" />,
User: () => <svg data-testid="icon-user" />,
FileImage: () => <svg data-testid="icon-file-image" />,
Webhook: () => <svg data-testid="icon-webhook" />,
Logs: () => <svg data-testid="icon-logs" />,
Blocks: () => <svg data-testid="icon-blocks" />,
MonitorCog: () => <svg data-testid="icon-monitor-cog" />,
// StreamConnectionCard-specific icons
ChevronDown: () => <svg data-testid="icon-chevron-down" />,
ChevronRight: () => <svg data-testid="icon-chevron-right" />,
CirclePlay: () => <svg data-testid="icon-circle-play" />,
@ -150,6 +168,8 @@ vi.mock('lucide-react', () => ({
Timer: () => <svg data-testid="icon-timer" />,
Users: () => <svg data-testid="icon-users" />,
Video: () => <svg data-testid="icon-video" />,
Package: () => <svg data-testid="icon-package" />,
Download: () => <svg data-testid="icon-download" />,
}));
// ── Imports after mocks ───────────────────────────────────────────────────────
@ -160,7 +180,6 @@ import useVideoStore from '../../../store/useVideoStore';
import { showNotification } from '../../../utils/notificationUtils.js';
import {
getChannelStreams,
getMatchingStreamByUrl,
getSelectedStream,
getStreamsByIds,
switchStream,
@ -264,13 +283,17 @@ describe('StreamConnectionCard', () => {
describe('route guard', () => {
it('renders nothing when pathname is not /stats', () => {
setupLocation('/dashboard');
const { container } = render(<StreamConnectionCard {...defaultProps()} />);
const { container } = render(
<StreamConnectionCard {...defaultProps()} />
);
expect(container.firstChild).toBeNull();
});
it('renders nothing when pathname is /channels', () => {
setupLocation('/channels');
const { container } = render(<StreamConnectionCard {...defaultProps()} />);
const { container } = render(
<StreamConnectionCard {...defaultProps()} />
);
expect(container.firstChild).toBeNull();
});
@ -337,7 +360,9 @@ describe('StreamConnectionCard', () => {
vi.mocked(getStreamsByIds).mockResolvedValue([]);
render(
<StreamConnectionCard
{...defaultProps({ channel: makeChannel({ name: '', stream_id: null }) })}
{...defaultProps({
channel: makeChannel({ name: '', stream_id: null }),
})}
/>
);
expect(screen.getByText('Unnamed Channel')).toBeInTheDocument();
@ -465,7 +490,9 @@ describe('StreamConnectionCard', () => {
describe('current program', () => {
it('does not render program section when currentProgram is null', () => {
render(<StreamConnectionCard {...defaultProps({ currentProgram: null })} />);
render(
<StreamConnectionCard {...defaultProps({ currentProgram: null })} />
);
expect(screen.queryByText('Now Playing:')).not.toBeInTheDocument();
});
@ -493,7 +520,9 @@ describe('StreamConnectionCard', () => {
{...defaultProps({ currentProgram: makeCurrentProgram() })}
/>
);
expect(screen.queryByText('Daily news broadcast.')).not.toBeInTheDocument();
expect(
screen.queryByText('Daily news broadcast.')
).not.toBeInTheDocument();
});
it('expands program description when chevron button is clicked', () => {
@ -502,7 +531,9 @@ describe('StreamConnectionCard', () => {
{...defaultProps({ currentProgram: makeCurrentProgram() })}
/>
);
const chevronBtn = screen.getByTestId('icon-chevron-right').closest('button');
const chevronBtn = screen
.getByTestId('icon-chevron-right')
.closest('button');
fireEvent.click(chevronBtn);
expect(screen.getByText('Daily news broadcast.')).toBeInTheDocument();
});
@ -513,12 +544,18 @@ describe('StreamConnectionCard', () => {
{...defaultProps({ currentProgram: makeCurrentProgram() })}
/>
);
const chevronBtn = screen.getByTestId('icon-chevron-right').closest('button');
const chevronBtn = screen
.getByTestId('icon-chevron-right')
.closest('button');
fireEvent.click(chevronBtn);
expect(screen.getByText('Daily news broadcast.')).toBeInTheDocument();
const chevronDownBtn = screen.getByTestId('icon-chevron-down').closest('button');
const chevronDownBtn = screen
.getByTestId('icon-chevron-down')
.closest('button');
fireEvent.click(chevronDownBtn);
expect(screen.queryByText('Daily news broadcast.')).not.toBeInTheDocument();
expect(
screen.queryByText('Daily news broadcast.')
).not.toBeInTheDocument();
});
it('renders program progress when expanded and times are present', () => {
@ -527,7 +564,9 @@ describe('StreamConnectionCard', () => {
{...defaultProps({ currentProgram: makeCurrentProgram() })}
/>
);
const chevronBtn = screen.getByTestId('icon-chevron-right').closest('button');
const chevronBtn = screen
.getByTestId('icon-chevron-right')
.closest('button');
fireEvent.click(chevronBtn);
expect(screen.getByTestId('progress')).toBeInTheDocument();
});
@ -536,11 +575,16 @@ describe('StreamConnectionCard', () => {
render(
<StreamConnectionCard
{...defaultProps({
currentProgram: makeCurrentProgram({ start_time: null, end_time: null }),
currentProgram: makeCurrentProgram({
start_time: null,
end_time: null,
}),
})}
/>
);
const chevronBtn = screen.getByTestId('icon-chevron-right').closest('button');
const chevronBtn = screen
.getByTestId('icon-chevron-right')
.closest('button');
fireEvent.click(chevronBtn);
expect(screen.queryByTestId('progress')).not.toBeInTheDocument();
});
@ -570,9 +614,8 @@ describe('StreamConnectionCard', () => {
{ id: 11, name: 'Stream B', url: 'http://b.com', m3u_profile: null },
]);
// Provide options via getStreamOptions mock
const { getStreamOptions } = await import(
'../../../utils/cards/StreamConnectionCardUtils.js'
);
const { getStreamOptions } =
await import('../../../utils/cards/StreamConnectionCardUtils.js');
vi.mocked(getStreamOptions).mockReturnValue([
{ value: '10', label: 'Stream A' },
{ value: '11', label: 'Stream B' },
@ -583,19 +626,19 @@ describe('StreamConnectionCard', () => {
});
});
it('sets activeStreamId when a matching stream is found by URL', async () => {
it('sets activeStreamId when a matching stream is found by stream_id', async () => {
vi.mocked(getChannelStreams).mockResolvedValue([
{ id: 42, name: 'Stream A', url: 'http://stream.example.com/ch1', m3u_profile: null },
{
id: 42,
name: 'Stream A',
url: 'http://stream.example.com/ch1',
m3u_profile: null,
},
]);
vi.mocked(getMatchingStreamByUrl).mockReturnValue({
id: 42,
name: 'Stream A',
url: 'http://stream.example.com/ch1',
m3u_profile: null,
});
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => {
expect(getMatchingStreamByUrl).toHaveBeenCalled();
// defaultProps channel has stream_id: 42, which matches the stream id above
expect(getChannelStreams).toHaveBeenCalled();
});
});
@ -614,11 +657,15 @@ describe('StreamConnectionCard', () => {
describe('stream switching', () => {
beforeEach(async () => {
vi.mocked(getChannelStreams).mockResolvedValue([
{ id: 10, name: 'Stream A', url: 'http://a.com', m3u_profile: { name: 'M3U A' } },
{
id: 10,
name: 'Stream A',
url: 'http://a.com',
m3u_profile: { name: 'M3U A' },
},
]);
const { getStreamOptions } = await import(
'../../../utils/cards/StreamConnectionCardUtils.js'
);
const { getStreamOptions } =
await import('../../../utils/cards/StreamConnectionCardUtils.js');
vi.mocked(getStreamOptions).mockReturnValue([
{ value: '10', label: 'Stream A' },
]);
@ -633,7 +680,9 @@ describe('StreamConnectionCard', () => {
vi.mocked(switchStream).mockResolvedValue({});
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => screen.getByTestId('select'));
fireEvent.change(screen.getByTestId('select'), { target: { value: '10' } });
fireEvent.change(screen.getByTestId('select'), {
target: { value: '10' },
});
await waitFor(() => {
expect(switchStream).toHaveBeenCalledWith(
expect.objectContaining({ channel_id: 'ch-uuid-1' }),
@ -646,7 +695,9 @@ describe('StreamConnectionCard', () => {
vi.mocked(switchStream).mockResolvedValue({});
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => screen.getByTestId('select'));
fireEvent.change(screen.getByTestId('select'), { target: { value: '10' } });
fireEvent.change(screen.getByTestId('select'), {
target: { value: '10' },
});
await waitFor(() => {
expect(showNotification).toHaveBeenCalledWith(
expect.objectContaining({ color: 'blue.5' })
@ -658,7 +709,9 @@ describe('StreamConnectionCard', () => {
vi.mocked(switchStream).mockRejectedValue(new Error('Switch failed'));
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => screen.getByTestId('select'));
fireEvent.change(screen.getByTestId('select'), { target: { value: '10' } });
fireEvent.change(screen.getByTestId('select'), {
target: { value: '10' },
});
await waitFor(() => {
expect(showNotification).toHaveBeenCalledWith(
expect.objectContaining({ color: 'red.5' })
@ -672,7 +725,9 @@ describe('StreamConnectionCard', () => {
});
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => screen.getByTestId('select'));
fireEvent.change(screen.getByTestId('select'), { target: { value: '10' } });
fireEvent.change(screen.getByTestId('select'), {
target: { value: '10' },
});
await waitFor(() => {
expect(screen.getByText('Updated M3U')).toBeInTheDocument();
});
@ -686,7 +741,9 @@ describe('StreamConnectionCard', () => {
vi.mocked(getChannelStreams).mockResolvedValue([]);
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => {
expect(screen.queryByTestId('icon-circle-play')).not.toBeInTheDocument();
expect(
screen.queryByTestId('icon-circle-play')
).not.toBeInTheDocument();
});
});
@ -694,10 +751,11 @@ describe('StreamConnectionCard', () => {
vi.mocked(getChannelStreams).mockResolvedValue([
{ id: 10, name: 'Stream A', url: 'http://a.com', m3u_profile: null },
]);
const { getStreamOptions } = await import(
'../../../utils/cards/StreamConnectionCardUtils.js'
);
vi.mocked(getStreamOptions).mockReturnValue([{ value: '10', label: 'Stream A' }]);
const { getStreamOptions } =
await import('../../../utils/cards/StreamConnectionCardUtils.js');
vi.mocked(getStreamOptions).mockReturnValue([
{ value: '10', label: 'Stream A' },
]);
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => {
expect(screen.getByTestId('icon-circle-play')).toBeInTheDocument();
@ -712,10 +770,11 @@ describe('StreamConnectionCard', () => {
vi.mocked(getChannelStreams).mockResolvedValue([
{ id: 10, name: 'Stream A', url: 'http://a.com', m3u_profile: null },
]);
const { getStreamOptions } = await import(
'../../../utils/cards/StreamConnectionCardUtils.js'
);
vi.mocked(getStreamOptions).mockReturnValue([{ value: '10', label: 'Stream A' }]);
const { getStreamOptions } =
await import('../../../utils/cards/StreamConnectionCardUtils.js');
vi.mocked(getStreamOptions).mockReturnValue([
{ value: '10', label: 'Stream A' },
]);
render(<StreamConnectionCard {...defaultProps()} />);
await waitFor(() => screen.getByTestId('icon-circle-play'));
@ -738,10 +797,11 @@ describe('StreamConnectionCard', () => {
vi.mocked(getChannelStreams).mockResolvedValue([
{ id: 10, name: 'Stream A', url: 'http://a.com', m3u_profile: null },
]);
const { getStreamOptions } = await import(
'../../../utils/cards/StreamConnectionCardUtils.js'
);
vi.mocked(getStreamOptions).mockReturnValue([{ value: '10', label: 'Stream A' }]);
const { getStreamOptions } =
await import('../../../utils/cards/StreamConnectionCardUtils.js');
vi.mocked(getStreamOptions).mockReturnValue([
{ value: '10', label: 'Stream A' },
]);
render(
<StreamConnectionCard {...defaultProps({ channelsByUUID: {} })} />
@ -837,4 +897,4 @@ describe('StreamConnectionCard', () => {
expect(screen.getAllByText('0 Kbps').length).toBeGreaterThan(0);
});
});
});
});

58
frontend/src/components/cards/__tests__/VODCard.test.jsx
View file

@ -10,12 +10,22 @@ vi.mock('../../../utils/cards/VODCardUtils.js', () => ({
// ── Mantine core ──────────────────────────────────────────────────────────────
vi.mock('@mantine/core', () => ({
ActionIcon: ({ children, onClick, variant, size }) => (
<button data-testid="action-icon" data-variant={variant} data-size={size} onClick={onClick}>
<button
data-testid="action-icon"
data-variant={variant}
data-size={size}
onClick={onClick}
>
{children}
</button>
),
Badge: ({ children, color, variant, size }) => (
<span data-testid="badge" data-color={color} data-variant={variant} data-size={size}>
<span
data-testid="badge"
data-color={color}
data-variant={variant}
data-size={size}
>
{children}
</span>
),
@ -37,14 +47,27 @@ vi.mock('@mantine/core', () => ({
{children}
</div>
),
CardSection: ({ children }) => <div data-testid="card-section">{children}</div>,
CardSection: ({ children }) => (
<div data-testid="card-section">{children}</div>
),
Group: ({ children, justify, gap, wrap }) => (
<div data-testid="group" data-justify={justify} data-gap={gap} data-wrap={wrap}>
<div
data-testid="group"
data-justify={justify}
data-gap={gap}
data-wrap={wrap}
>
{children}
</div>
),
Image: ({ src, alt, height, fallbackSrc, fit }) => (
<img src={src} alt={alt} data-height={height} data-fallback={fallbackSrc} data-fit={fit} />
<img
src={src}
alt={alt}
data-height={height}
data-fallback={fallbackSrc}
data-fit={fit}
/>
),
Stack: ({ children, spacing, gap, p }) => (
<div data-testid="stack" data-spacing={spacing} data-gap={gap} data-p={p}>
@ -68,6 +91,7 @@ vi.mock('@mantine/core', () => ({
// ── lucide-react ──────────────────────────────────────────────────────────────
vi.mock('lucide-react', () => ({
ListOrdered: () => <svg data-testid="icon-list-ordered" />,
Calendar: () => <svg data-testid="icon-calendar" />,
Clock: () => <svg data-testid="icon-clock" />,
Play: () => <svg data-testid="icon-play" />,
@ -75,7 +99,10 @@ vi.mock('lucide-react', () => ({
}));
// ── Imports after mocks ───────────────────────────────────────────────────────
import { formatDuration, getSeasonLabel } from '../../../utils/cards/VODCardUtils.js';
import {
formatDuration,
getSeasonLabel,
} from '../../../utils/cards/VODCardUtils.js';
import VODCard from '../VODCard';
// ── Helpers ───────────────────────────────────────────────────────────────────
@ -113,7 +140,9 @@ const makeEpisode = (overrides = {}) => ({
describe('VODCard', () => {
beforeEach(() => {
vi.clearAllMocks();
vi.mocked(formatDuration).mockImplementation((mins) => (mins ? `${mins}m` : null));
vi.mocked(formatDuration).mockImplementation((mins) =>
mins ? `${mins}m` : null
);
vi.mocked(getSeasonLabel).mockReturnValue('S01E02');
});
@ -193,7 +222,9 @@ describe('VODCard', () => {
it('renders the season label via getSeasonLabel', () => {
render(<VODCard vod={makeEpisode()} onClick={vi.fn()} />);
expect(getSeasonLabel).toHaveBeenCalledWith(expect.objectContaining({ type: 'episode' }));
expect(getSeasonLabel).toHaveBeenCalledWith(
expect.objectContaining({ type: 'episode' })
);
expect(screen.getByText(/S01E02/)).toBeInTheDocument();
});
@ -230,7 +261,9 @@ describe('VODCard', () => {
});
it('does not render an img tag when logo.url is empty string', () => {
render(<VODCard vod={makeMovie({ logo: { url: '' } })} onClick={vi.fn()} />);
render(
<VODCard vod={makeMovie({ logo: { url: '' } })} onClick={vi.fn()} />
);
expect(screen.queryByRole('img')).not.toBeInTheDocument();
});
});
@ -257,8 +290,9 @@ describe('VODCard', () => {
it('does not render genre when absent', () => {
render(<VODCard vod={makeMovie({ genre: null })} onClick={vi.fn()} />);
const badges = screen.queryAllByTestId('badge');
const dimmedBadges = badges.filter((badge) =>
badge.getAttribute('data-color') === 'dimmed');
const dimmedBadges = badges.filter(
(badge) => badge.getAttribute('data-color') === 'dimmed'
);
expect(dimmedBadges.length).toBe(0);
});
});
@ -291,4 +325,4 @@ describe('VODCard', () => {
expect(onClick).toHaveBeenCalledWith(vod);
});
});
});
});

Some files were not shown because too many files have changed in this diff Show more