Mirrors/Dispatcharr
1
0
Fork 1
mirror of https://github.com/Dispatcharr/Dispatcharr.git synced 2026-07-18 00:55:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/Jacob-Lasky/1378

Browse source
This commit is contained in:
SergeantPanda 2026-06-30 12:08:17 -05:00
commit 49a40ba7ed
87 changed files with 10589 additions and 2418 deletions
Download patch file Download diff file Expand all files Collapse all files

95
CHANGELOG.md
View file

@ -7,22 +7,109 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
### Added
- **Channel list with nested streams loads faster.** `GET /api/channels/channels/?include_streams=true` (Channels UI and single-channel fetch) now builds nested stream payloads from the prefetched `channelstream_set` instead of issuing one extra streams M2M query per channel.
- **XC catch-up (timeshift) support**. Adds a native `/timeshift/{user}/{pass}/{stream_id}/{timestamp}/{duration}.ts` endpoint that proxies replay sessions from an Xtream Codes provider to clients such as iPlayTV (Apple TV) and TiviMate. Auth reuses the same `xc_password` custom-property the live `/live/.../{id}.ts` endpoint already uses. Catch-up sessions appear on `/stats` with a violet `TIMESHIFT` badge alongside live sessions and respect per-channel access rules. — Thanks [@cedric-marcoux](https://github.com/cedric-marcoux) (#1242)
- **Catch-up flags** (`tv_archive`, `tv_archive_duration`) denormalized at import time for zero-cost output queries; end-of-refresh SQL rollup updates channels linked to the refreshed account and self-heals stale flags on those channels only (e.g. after catch-up streams are removed or an account is deactivated).
- **Strictly-UTC API surface, automatic per-provider timezone.** `server_info.timezone` is always `UTC` and the XC EPG `start`/`end` strings are emitted in UTC; the proxy converts the requested instant to the serving provider's own reported timezone (`server_info.timezone` captured on account refresh) at request time. No timezone to configure.
- **Multi-provider failover.** Catch-up walks the channel's catch-up streams in order (like live playback): if one provider cannot serve the archive the next is tried, each attempt with its own account context (credentials, reported timezone, user-agent). Accounts that fail with an auth/ban-class status are not retried via their other streams.
- **Provider pool accounting.** Catch-up reserves a provider profile slot (`connection_pool`) before connecting upstream — same contract as live and VOD — and releases it when the session ends. When the default profile is at capacity it walks the account's alternate profiles with their own resolved credentials, and answers `503` when every profile is full.
- **Per-client session pool.** The first request without `session_id` receives a `301` redirect to the same URL with a stable `?session_id=` query parameter; that ID becomes the client identity for stats, stop keys, and pool coordination. Each viewer gets their own Redis-backed pool entry even when watching the same programme; idle sessions can be reclaimed via fingerprint matching (same user and programme, IP + user-agent score). Real scrubs within a session stop the in-flight stream through the standard stop-key mechanism; parallel startup probes (full-file, open-ended, and EOF tail `Range` requests) are deferred with `503` instead of opening extra upstream connections. Stream-limit exemptions for `ignore_same_channel_connections` apply only to sibling requests from the same `session_id`.
- **`xmltv_prev_days_override` under `Settings → EPG`** (default `0` = auto-detect). Verbose timeshift logging follows the standard logger DEBUG level.
- **EPG XMLTV `prev_days` auto-detection** from the provider's largest `tv_archive_duration` (capped at 30 days), overridable via setting or `?prev_days=` URL parameter.
- **Byte path streams directly via `iter_content` + generator yield** (same pattern as the VOD proxy), with an upfront MPEG-TS sync peek that strips any PHP-warning preamble before bytes reach strict demuxers. Throughput stays comfortably above the typical 5 Mbps FHD bitrate so clients don't buffer.
### Performance
- **XMLTV EPG output no longer N+1 queries streams or dummy-program checks.** `generate_epg()` prefetches ordered channel streams once (for custom dummy EPG logo/program parsing when `name_source` is `stream`) and bulk-checks which dummy `EPGData` rows have stored programmes in a single query instead of one `.exists()` per row. Large guides with hundreds of custom-dummy channels issue far fewer SQL round-trips per client refresh.
- **M3U/XC stream refresh is faster on large accounts.** Steady-state refreshes split `bulk_update` into a lightweight touch pass (`last_seen` / `is_stale` only) for unchanged streams and a full column update only when provider metadata or catch-up fields actually change.
- **M3U stream filters compile once per refresh.** Account filters are regex-compiled before batch workers start; accounts with no filters skip per-stream filter checks entirely.
- **M3U refresh releases parse catalogs sooner.** Standard accounts stream-parse the on-disk M3U instead of loading the full file into RAM; `extinf_data` is dropped immediately after batch DB work, before stale cleanup and auto-sync.
- **XC live refresh avoids redundant work during catalog filtering.** `collect_xc_streams` skips disabled categories before building entries and uses a shared URL prefix instead of formatting each stream URL separately. Auto-sync releases per-group logo/EPG caches after each group iteration.
- **Celery workers return RSS after memory-intensive tasks.** `cleanup_memory()` accepts an optional `trim_heap` flag (glibc `malloc_trim`); Celery `task_postrun` enables it after `close_old_connections()` for M3U account/group refresh, EPG, VOD, and channel-matching tasks so worker memory drops back toward baseline instead of ratcheting across successive large jobs.
### Fixed
- **M3U refresh completion counts now reflect actual stream changes.** The "updated" count previously included every existing stream because the summary treated `last_seen` touch-only rows as updates; it now counts only streams whose provider metadata changed. "Total processed" includes unchanged streams separately. The completion message, WebSocket payload, and parsing notification now also report how many streams were **marked stale** (missing from this refresh, pending retention-gated deletion) versus **removed** (deleted this run).
- **M3U group processing retries on poisoned DB connections.** `_db_query_with_retry` now treats psycopg desync errors (`DatabaseError`, e.g. `lost synchronization with server`) as transient; `process_groups` relationship loading uses it so a stale Celery worker connection resets once instead of failing the whole refresh.
- **Auto Channel Sync's Find and Replace preview now matches the rename it performs.** The preview rendered the literal `$1` instead of substituting numbered capture groups, and compiled patterns with a stricter engine than the rename, so patterns like `^*` previewed a change the sync silently skipped. The live rename now uses the same `regex` engine as the preview (with a substitution timeout to bound catastrophic backtracking), both apply the `$1`→`\1` conversion through one shared helper, and a rename whose result exceeds the channel-name column length is truncated instead of aborting the whole sync. (Fixes #1332) — Thanks [@CodeBormen](https://github.com/CodeBormen)
## [0.27.2] - 2026-06-30
### Added
- **New proxy setting: Client Connect Grace Period (`channel_client_wait_period`, default 5s).** Adds a dedicated timeout for channels that have filled their buffer but still have no viewers (`waiting_for_clients`). Previously that window reused `channel_shutdown_delay` (default 0s), so those channels were torn down almost immediately.
### Changed
- **Proxy grace-period settings are now split into three distinct timeouts.** The cleanup watchdog already applied `channel_init_grace_period` while a channel was still connecting (buffer not ready) and reused `channel_shutdown_delay` once `connection_ready_time` was set, including for `waiting_for_clients` with zero viewers. With the default `channel_shutdown_delay` of 0s, a buffered channel waiting for its first viewer was stopped almost immediately; raising shutdown delay was the only workaround, but that also delayed teardown after real disconnects. Behaviour is now:
- **`channel_init_grace_period` (default 60s, max 300s):** how long the proxy may spend connecting and cycling failover streams before giving up on startup.
- **`channel_client_wait_period` (default 5s):** how long a ready channel with no viewers stays up waiting for the first client (the original grace-period use case).
- **`channel_shutdown_delay` (default 0s):** delay after the last client disconnects only; no longer applies when the buffer is ready but no viewer has connected yet.
- **Migration 0026 bumps `channel_init_grace_period` to 60s when the stored value is below 60.** Existing installs on the old 5s default (or any custom value under 60) are raised automatically. Values already at 60s or higher are unchanged. If you previously raised `channel_shutdown_delay` to keep buffered channels alive with no viewers, set `channel_client_wait_period` instead (Settings → Proxy → Advanced).
- **Proxy settings UI: less-used options moved under Advanced.** Settings → Proxy now shows the day-to-day tuning fields by default (`buffering_timeout`, `buffering_speed`, `channel_shutdown_delay`, `new_client_behind_seconds`). **Buffer Chunk TTL**, **Channel Initialization Timeout**, and **Client Connect Grace Period** are tucked under **Show Advanced Settings**.
## [0.27.1] - 2026-06-25
### Security
- Updated `Django` 6.0.5 → 6.0.6, resolving the following CVEs:
- **CVE-2026-6873**: Signed cookie salt namespace collision in `HttpRequest.get_signed_cookie()`.
- **CVE-2026-7666**: Potential unencrypted email transmission via STARTTLS in the SMTP backend.
- **CVE-2026-8404**: Potential private data exposure via case-sensitive `Cache-Control` directives in `UpdateCacheMiddleware`.
- **CVE-2026-35193**: Potential private data exposure via missing `Vary: Authorization` in `UpdateCacheMiddleware`.
- **CVE-2026-48587**: Potential private data exposure via whitespace padding in the `Vary` header.
- Updated frontend npm dependencies to resolve 4 audit vulnerabilities (1 low, 2 moderate, 1 high):
- Updated `vite` 7.3.2 → 7.3.5, resolving **moderate** NTLMv2 hash disclosure via UNC path handling on Windows ([GHSA-v6wh-96g9-6wx3](https://github.com/advisories/GHSA-v6wh-96g9-6wx3)) and **high** `server.fs.deny` bypass on Windows alternate paths ([GHSA-fx2h-pf6j-xcff](https://github.com/advisories/GHSA-fx2h-pf6j-xcff))
- Updated `js-yaml` 4.1.1 → 5.1.0, resolving **moderate** quadratic-complexity DoS in merge key handling via repeated aliases ([GHSA-h67p-54hq-rp68](https://github.com/advisories/GHSA-h67p-54hq-rp68))
- Updated `esbuild` 0.27.3 → 0.28.1, resolving **low** arbitrary file read when running the development server on Windows ([GHSA-g7r4-m6w7-qqqr](https://github.com/advisories/GHSA-g7r4-m6w7-qqqr))
### Added
- **Isolated backend test settings (`dispatcharr.settings_test`).** `python manage.py test` now switches to this module automatically (via `manage.py`). It creates an empty PostgreSQL `test_<dbname>` database (same engine as production), uses the standard Postgres backend instead of geventpool so `TestCase` transactions isolate correctly, and leaves Celery tasks queued (no eager `post_save` signal runs during tests). Set `TEST_USE_SQLITE=1` for an in-memory SQLite fallback when Postgres is unavailable.
### Performance
- **`get_vod_streams` and `get_series` XC API endpoints are faster and no longer exhaust Docker `/dev/shm`.** Large libraries (e.g. 125k movies) previously ran one wide `DISTINCT ON` query with parallel workers, which could fail with `could not resize shared memory segment … No space left on device` on the default 64MB container shm. Both endpoints now fetch display columns via `.values()` (no ORM model instantiation per row). Redundant `category` and `logo` joins were dropped in favor of FK ids; alphabetical sort runs in SQL. Typical full-library response time drops from ~23–28s to ~8–10s with stable shm usage.
- **XMLTV EPG export is faster and no longer balloons worker memory.** `generate_epg()` was reworked end-to-end for large guides. (Fixes #1366)
- Streams incrementally: on a cache miss each chunk is pushed to a Redis list as it is yielded (no `''.join()` in the worker); repeat requests within 300s stream chunks back from Redis. `malloc_trim` runs after cold builds.
- Channel streams are prefetched once (only `id`/`name`) instead of one query per custom-dummy channel; dummy `EPGData` programme existence is bulk-checked in a single query.
- The primary channel id is escaped once per `epg_id` group instead of once per programme (~750k fewer `html.escape` calls on a large guide).
- The channel query no longer JOINs multi-MB `programme_index` blobs per channel (~13s saved on a ~2000-channel guide; indices live in `EPGSourceIndex`).
- Programme export uses `(epg_id, id)` keyset pagination with a per-source `start_time` sort; a matching composite index on `ProgramData` (created `CONCURRENTLY` on PostgreSQL) lets each chunk use an ordered index range scan instead of re-sorting every chunk.
- **EPG grid endpoint releases its payload memory back to the OS.** `/api/epg/grid/` drops the redundant full-list copy when appending dummy programmes and runs `malloc_trim` once the response is sent, so worker RSS no longer ratchets up ~20MB per request.
### Changed
- **`programme_index` moved off `EPGSource` into a dedicated `EPGSourceIndex` table.** The multi-MB byte-offset index was repeatedly pulled into web and Celery workers by ordinary `EPGSource` queries and `select_related` JOINs (which ignore manager-level `defer()`). It now lives in a one-to-one `EPGSourceIndex` row, read only when explicitly accessed through the `EPGSource.programme_index` property, so no list, detail, or JOIN query can load it by accident. Migration `0026` copies existing indices across. No API or index-build behavior change.
- **EPG generation extracted into `apps/output/epg.py`.** All XMLTV output logic (`generate_epg`, `generate_dummy_programs`, `generate_custom_dummy_programs`, `generate_dummy_epg`, and supporting helpers) moved from `apps/output/views.py` into a dedicated module. `views.py` retains the thin HTTP endpoint wrappers and auth checks; `epg.py` handles all content generation. No behavior change.
- **Channel list with nested streams loads faster.** `GET /api/channels/channels/?include_streams=true` (Channels UI and single-channel fetch) now builds nested stream payloads from the prefetched `channelstream_set` instead of issuing one extra streams M2M query per channel.
- Dependency updates:
- `Django` 6.0.5 → 6.0.6 (security patch; see Security section)
- `requests` 2.33.1 → 2.34.2
- `gevent` 26.4.0 → 26.5.0
- `torch` 2.11.0+cpu → 2.12.1+cpu
- `sentence-transformers` 5.4.1 → 5.6.0
- `lxml` 6.1.0 → 6.1.1
### Fixed
- **Channel Initialization Grace Period is honoured during live stream startup.** Preview and playback no longer abort after a hardcoded 10s while the channel is still connecting with an empty buffer; the TS generator init-wait stall check and upstream health monitor now use the configured `channel_init_grace_period` (same as the server cleanup watchdog) instead of `CONNECTION_TIMEOUT`. (Fixes #1380)
- **Channels are marked `active` as soon as the buffer threshold is met and a client is streaming.** Once the initial buffer fills, state is set to `active` immediately when viewers are attached, or `waiting_for_clients` when the buffer is ready but no client is connected yet (e.g. proxy API warmup). The cleanup watchdog no longer waits an extra grace period before promoting to `active`. Proxy settings copy now describes `channel_init_grace_period` as an initialization buffer timeout.
- **DVR recording playback auth is complete for native video, HLS segments, and redirects.** Completed recordings use `/file/` with native `<video src>`, which cannot send `Authorization` headers. Playback accepts `?token=` query-param JWT (matching VOD streams), and the floating video player appends the token when assigning native URLs. The explicit HLS URL route (`/api/channels/recordings/.../hls/...`) now registers the same authenticators as the ViewSet `@action` handlers—previously only header JWT applied on that path, so `?token=` failed for native HLS clients. When the request used `?token=`, rewritten playlist segment URLs and `/file/`↔`/hls/` redirects preserve the token; hls.js clients that authenticate via `Authorization` are unchanged. `QueryParamJWTAuthentication` reads `request.query_params` on DRF requests.
- **In-progress DVR playback no longer jumps to the live edge.** The floating video player still opens in-progress recordings at the start of the seekable range, but hls.js was configured with `liveMaxLatencyDurationCount: 10`, which forced the playhead forward to "now" shortly after playback began. Live-edge sync is now disabled for recording HLS so users can watch, pause, and scrub from the beginning while the recording continues. (Fixes #1329)
- **`refresh_single_m3u_account` no longer re-raises after setting account ERROR.** Matches `refresh_epg_data`: the account status and UI already reflect the failure; Celery no longer marks the task FAILED after the terminal error state is persisted.
- **M3U refresh recovers DB connections and account status after failures.** Celery workers now call `close_old_connections()` before and after every task; `refresh_single_m3u_account` also resets its DB connection at task start and retries account loads once after a connection reset (avoids opaque `list index out of range` errors from poisoned worker connections). Accounts leave `fetching`/`parsing` for a terminal `error` or `success` state when refresh aborts. Error-path status updates use `QuerySet.update()` on a clean connection instead of `save()` on a connection that may already be INTRANS after `refresh_m3u_groups` or batch ORM failures. Failed group downloads now set `error` instead of returning silently. Refresh start replaces stale `last_message` text. `refresh_account_profiles` releases DB connections after each profile failure. (Fixes #1338)
- **EPG refresh recovers DB connections and source status after failures.** `refresh_epg_data` now mirrors the M3U hardening: connection reset at task start/end, retried source load after a poisoned-worker connection reset, `QuerySet.update()` for error status on a clean connection, and a `finally` guard that moves sources stuck in `fetching`/`parsing` to `error`. Programme index builds are skipped when programme parsing fails. `parse_channels_only` now persists the error status when a missing file has no URL to refetch.
- **M3U `custom_properties` JSON normalization.** Legacy rows and some API writes could store a JSON-encoded string instead of an object in `custom_properties`, causing refresh, profile sync, and auto-sync to fail with `'str' object has no attribute 'get'`. M3U account, profile, and group-relation models normalize on `save()`; group settings bulk writes and read/merge paths use shared helpers in `core.utils` without re-parsing dict values.
- **`POST /api/epg/import/` no longer loads `programme_index`.** The dummy-source guard uses a narrow existence query instead of `objects.get()`, so import triggers avoid pulling the multi-MB byte-offset index into the uWSGI worker. Request/response shape is unchanged for the UI, plugins, and external importers.
- **`POST /api/epg/import/` no longer loads the full EPG source row.** The dummy-source guard uses a narrow existence query instead of `objects.get()`, keeping the import trigger lightweight. Request/response shape is unchanged for the UI, plugins, and external importers.
- **XC live playback URL building now normalizes account server URLs.** On-demand live URLs (introduced for Server Group profile rotation in 0.27.0) rebuild `/live/{user}/{pass}/{stream_id}.ts` from current credentials instead of reusing the synced `stream.url`. That path now runs `normalize_server_url()` so pasted API URLs (e.g. `/player_api.php` with query params) are stripped while sub-paths like `/server1` are preserved. `get_transformed_credentials()` normalizes the base URL at the source; VOD movie and episode relations use the same helper instead of constructing a throwaway `XCClient` (which also avoided per-request HTTP session setup). (Fixes #1363)
- **Live proxy now releases geventpool DB connections on more paths.** `stream_ts()` calls `close_old_connections()` before `StreamingHttpResponse` so clients waiting on channel init do not keep a pool slot while the generator polls Redis. `generate_stream_url()`, `get_stream_info_for_switch()`, `get_alternate_streams()`, and `get_connections_left()` release in `finally` blocks after ORM work on stream-manager failover paths that run outside Django's request cycle. TS client disconnect cleanup matches fMP4, and `ChannelService` metadata helpers release after their ORM lookups.
- **OpenAPI schema paths for nested M3U profiles and filters no longer contain escaped slashes.** Router regex prefixes used unnecessary `\/`, which drf-spectacular serialized literally into the schema and broke client generators (e.g. oapi-codegen). Runtime URL matching is unchanged. (Fixes #1384)
- **Auto-sync range conflict warning no longer flags a group's own channels when using a channel-group override.** The M3U group settings "Range conflict" check compared occupant channel groups against the source group being configured. Auto-sync with `group_override` creates channels in the override target group, so those channels were misclassified as conflicts. The frontend now resolves the effective target group (override target when set, otherwise the source group) for classification. Genuine conflicts—manual channels, other accounts, different groups, or user-pinned numbers—still surface. (Fixes #1331) — Thanks [@CodeBormen](https://github.com/CodeBormen)
## [0.27.0] - 2026-06-16

14
CONTRIBUTING.md
View file

@ -144,7 +144,19 @@ Untested code is significantly less likely to be merged.
- Use Django's `TestCase` for unit/integration tests.
- Test files live at `apps/<app>/tests/`.
- Run the test suite with: `uv run python manage.py test`
- Run the backend test suite with:
```bash
python manage.py test
```
`manage.py` automatically uses `dispatcharr.settings_test`, which creates an empty PostgreSQL database `test_<dbname>` (same engine as production), runs migrations, and rolls back each test in a transaction. Your live VOD/channels data is not used.
Optional: `TEST_USE_SQLITE=1` for machines without Postgres (some PostgreSQL-only tests skip automatically).
Tests that exercise Celery task bodies should use `@override_settings(CELERY_TASK_ALWAYS_EAGER=True)` locally. Global eager mode is off because `post_save` signals on M3U/EPG models call `.delay()` and would break `TestCase` transaction isolation.
- Do **not** override with `--settings=dispatcharr.settings` on a live instance.
### Frontend

3
apps/accounts/authentication.py
View file

@ -93,7 +93,8 @@ class QueryParamJWTAuthentication(JWTAuthentication):
where the browser cannot set Authorization headers (e.g. <video src>)."""
def authenticate(self, request):
raw_token = request.GET.get('token')
params = getattr(request, "query_params", request.GET)
raw_token = params.get("token")
if not raw_token:
return None
try:

6
apps/channels/api_urls.py
View file

@ -13,6 +13,7 @@ from .api_views import (
UpdateChannelMembershipAPIView,
BulkUpdateChannelMembershipAPIView,
RecordingViewSet,
RECORDING_PLAYBACK_AUTHENTICATORS,
RecurringRecordingRuleViewSet,
GetChannelStreamsAPIView,
GetChannelStreamStatsAPIView,
@ -53,7 +54,10 @@ urlpatterns = [
path('recordings/bulk-delete-upcoming/', BulkDeleteUpcomingRecordingsAPIView.as_view(), name='bulk_delete_upcoming_recordings'),
path(
'recordings/<int:pk>/hls/<path:seg_path>',
RecordingViewSet.as_view({'get': 'hls'}),
RecordingViewSet.as_view(
{'get': 'hls'},
authentication_classes=RECORDING_PLAYBACK_AUTHENTICATORS,
),
name='recording-hls',
),
path('dvr/comskip-config/', ComskipConfigAPIView.as_view(), name='comskip_config'),

78
apps/channels/api_views.py
View file

@ -3,6 +3,8 @@ from rest_framework.response import Response
from rest_framework.views import APIView
from rest_framework.permissions import AllowAny
from rest_framework.decorators import action
from rest_framework_simplejwt.authentication import JWTAuthentication
from apps.accounts.authentication import ApiKeyAuthentication, QueryParamJWTAuthentication
from rest_framework.parsers import MultiPartParser, FormParser, JSONParser
from drf_spectacular.utils import extend_schema, OpenApiParameter, inline_serializer
from drf_spectacular.types import OpenApiTypes
@ -12,6 +14,7 @@ from django.db import connection, transaction
from django.db.models import Count, F, Prefetch
from django.db.models import Q
import os, json, requests, logging, mimetypes, threading, time
from urllib.parse import urlencode
from datetime import timedelta
from django.utils.http import http_date
from apps.accounts.permissions import (
@ -24,6 +27,7 @@ from apps.accounts.permissions import (
from core.models import UserAgent, CoreSettings
from core.utils import RedisClient, safe_upload_path
from apps.m3u.utils import convert_js_numbered_backreferences
from .models import (
Stream,
@ -365,6 +369,17 @@ class StreamViewSet(viewsets.ModelViewSet):
except re.error as e:
exclude_error = str(e)
# The replace field accepts JS-style $1 backreferences, but the regex
# engine honors \1. Convert once so the preview's "after" matches the
# name the live rename produces (apps/m3u/tasks.py sync_auto_channels
# applies the same conversion on the same engine).
replace_repl = convert_js_numbered_backreferences(replace_pat)
# The live rename caps the result at Channel.name's column length
# before bulk_create; mirror that cap so the preview never shows a
# name the sync would truncate.
name_max_len = Channel._meta.get_field("name").max_length
# Capped at SCAN_CAP to bound memory on huge groups; the
# separate COUNT lets the client surface scan_limit_hit when
# the preview covers only a sample.
@ -387,11 +402,12 @@ class StreamViewSet(viewsets.ModelViewSet):
total_scanned += 1
if find_re is not None:
try:
new_name = find_re.sub(replace_pat, name, timeout=REGEX_TIMEOUT)
new_name = find_re.sub(replace_repl, name, timeout=REGEX_TIMEOUT)
except (TimeoutError, re.error) as e:
find_error = find_error or f"Pattern timed out: {e}"
find_re = None
continue
new_name = new_name[:name_max_len]
if new_name != name:
find_match_count += 1
if len(find_matches) < limit:
@ -3236,12 +3252,41 @@ def _stop_dvr_clients(channel_uuid, recording_id=None):
return stopped
# QueryParamJWTAuthentication supports native <video src> clients that cannot
# send Authorization headers. Authorization still requires an authenticated
# user via _user_can_play_recording; these classes only populate request.user.
RECORDING_PLAYBACK_AUTHENTICATORS = [
JWTAuthentication,
ApiKeyAuthentication,
QueryParamJWTAuthentication,
]
def _recording_auth_query_suffix(request):
"""Suffix for rewritten recording URLs when auth used ?token= (native <video>).
hls.js clients authenticate via Authorization on each XHR and do not need
tokens embedded in playlist segment lines.
"""
from rest_framework.request import Request as DRFRequest
if isinstance(request, DRFRequest):
params = request.query_params
else:
params = request.GET
token = params.get("token")
if not token:
return ""
return "?" + urlencode({"token": token})
class RecordingViewSet(viewsets.ModelViewSet):
queryset = Recording.objects.all()
serializer_class = RecordingSerializer
def get_permissions(self):
# Allow unauthenticated playback of recording files (like other streaming endpoints)
# file/hls use AllowAny so DRF does not reject requests before auth
# classes run; _user_can_play_recording enforces authenticated access.
if self.action in ('file', 'hls'):
return [AllowAny()]
try:
@ -3305,7 +3350,12 @@ class RecordingViewSet(viewsets.ModelViewSet):
except Exception as e:
return Response({"success": False, "error": str(e)}, status=400)
@action(detail=True, methods=["get"], url_path="file")
@action(
detail=True,
methods=["get"],
url_path="file",
authentication_classes=RECORDING_PLAYBACK_AUTHENTICATORS,
)
def file(self, request, pk=None):
"""Stream a completed recording file with HTTP Range support for seeking.
@ -3329,7 +3379,7 @@ class RecordingViewSet(viewsets.ModelViewSet):
if hls_dir and os.path.isdir(hls_dir):
hls_url = request.build_absolute_uri(
f"/api/channels/recordings/{pk}/hls/index.m3u8"
)
) + _recording_auth_query_suffix(request)
return HttpResponseRedirect(hls_url)
if not file_path or not os.path.exists(file_path):
raise Http404("Recording file not found")
@ -3393,7 +3443,12 @@ class RecordingViewSet(viewsets.ModelViewSet):
response["Content-Disposition"] = f"inline; filename=\"{file_name}\""
return response
@action(detail=True, methods=["get"], url_path="hls/(?P<seg_path>.+)")
@action(
detail=True,
methods=["get"],
url_path="hls/(?P<seg_path>.+)",
authentication_classes=RECORDING_PLAYBACK_AUTHENTICATORS,
)
def hls(self, request, pk=None, seg_path=None):
"""Serve HLS playlist and segment files for an in-progress (or completed) recording.
@ -3417,9 +3472,10 @@ class RecordingViewSet(viewsets.ModelViewSet):
cp = recording.custom_properties or {}
file_path = cp.get("file_path")
if seg_path.endswith(".m3u8") and file_path and os.path.exists(file_path) and os.path.getsize(file_path) > 0:
return HttpResponseRedirect(
request.build_absolute_uri(f"/api/channels/recordings/{pk}/file/")
)
file_url = request.build_absolute_uri(
f"/api/channels/recordings/{pk}/file/"
) + _recording_auth_query_suffix(request)
return HttpResponseRedirect(file_url)
raise Http404("HLS content not available for this recording")
# Security: prevent path traversal outside the HLS directory
@ -3432,16 +3488,18 @@ class RecordingViewSet(viewsets.ModelViewSet):
raise Http404(f"HLS file not found: {seg_path}")
if seg_path.endswith(".m3u8"):
# Rewrite relative segment lines to absolute URLs through this API
# Rewrite relative segment lines to absolute URLs through this API.
# Propagate ?token= only for native <video> clients (see helper).
base_url = request.build_absolute_uri(
f"/api/channels/recordings/{pk}/hls/"
)
auth_suffix = _recording_auth_query_suffix(request)
lines = []
with open(requested) as _f:
for line in _f:
stripped = line.strip()
if stripped and not stripped.startswith("#"):
lines.append(f"{base_url}{stripped}\n")
lines.append(f"{base_url}{stripped}{auth_suffix}\n")
else:
lines.append(line)
return HttpResponse("".join(lines), content_type="application/x-mpegURL")

101
apps/channels/migrations/0038_add_catchup_fields.py Normal file
View file

@ -0,0 +1,101 @@
"""Add denormalized catch-up fields to Stream and Channel."""
from django.db import migrations, models
def backfill_stream_catchup(apps, schema_editor):
"""Derive is_catchup/catchup_days from Stream.custom_properties JSON."""
with schema_editor.connection.cursor() as cursor:
cursor.execute("""
UPDATE dispatcharr_channels_stream
SET is_catchup = TRUE,
catchup_days = COALESCE(
CASE WHEN (custom_properties->>'tv_archive_duration') ~ '^\\d+$'
THEN (custom_properties->>'tv_archive_duration')::int
ELSE NULL
END, 7
)
WHERE custom_properties IS NOT NULL
AND custom_properties != 'null'::jsonb
AND (
custom_properties->>'tv_archive' = '1'
-- JSON booleans extract as lowercase 'true' via ->>; the
-- 'True' spelling covers Python-str values stored by
-- older import code.
OR custom_properties->>'tv_archive' = 'true'
OR custom_properties->>'tv_archive' = 'True'
)
""")
def backfill_channel_catchup(apps, schema_editor):
"""Roll up catch-up fields from streams to channels."""
with schema_editor.connection.cursor() as cursor:
cursor.execute("""
UPDATE dispatcharr_channels_channel c SET
is_catchup = EXISTS (
SELECT 1 FROM dispatcharr_channels_channelstream cs
JOIN dispatcharr_channels_stream s ON s.id = cs.stream_id
WHERE cs.channel_id = c.id AND s.is_catchup = TRUE
),
catchup_days = COALESCE((
SELECT MAX(s.catchup_days) FROM dispatcharr_channels_channelstream cs
JOIN dispatcharr_channels_stream s ON s.id = cs.stream_id
WHERE cs.channel_id = c.id AND s.is_catchup = TRUE
), 0)
""")
class Migration(migrations.Migration):
dependencies = [
("dispatcharr_channels", "0037_auto_sync_overhaul"),
]
operations = [
# Stream fields
migrations.AddField(
model_name="stream",
name="is_catchup",
field=models.BooleanField(
default=False,
db_index=True,
help_text="Whether this stream supports catch-up/timeshift (tv_archive=1)",
),
),
migrations.AddField(
model_name="stream",
name="catchup_days",
field=models.PositiveIntegerField(
default=0,
help_text="Number of days of catch-up archive available (tv_archive_duration)",
),
),
# Channel fields
migrations.AddField(
model_name="channel",
name="is_catchup",
field=models.BooleanField(
default=False,
db_index=True,
help_text="Whether any stream on this channel supports catch-up (tv_archive=1)",
),
),
migrations.AddField(
model_name="channel",
name="catchup_days",
field=models.PositiveIntegerField(
default=0,
help_text="Max catch-up archive days across all streams on this channel",
),
),
# Backfill existing data
migrations.RunPython(
backfill_stream_catchup,
reverse_code=migrations.RunPython.noop,
),
migrations.RunPython(
backfill_channel_catchup,
reverse_code=migrations.RunPython.noop,
),
]

22
apps/channels/models.py
View file

@ -135,6 +135,17 @@ class Stream(models.Model):
db_index=True
)
# Populated at import from tv_archive / tv_archive_duration.
is_catchup = models.BooleanField(
default=False,
db_index=True,
help_text="Whether this stream supports catch-up/timeshift (tv_archive=1)",
)
catchup_days = models.PositiveIntegerField(
default=0,
help_text="Number of days of catch-up archive available (tv_archive_duration)",
)
class Meta:
# If you use m3u_account, you might do unique_together = ('name','url','m3u_account')
verbose_name = "Stream"
@ -364,6 +375,17 @@ class Channel(models.Model):
help_text="The M3U account that auto-created this channel"
)
# Populated at import; rolled up via ChannelStream signal / m3u refresh.
is_catchup = models.BooleanField(
default=False,
db_index=True,
help_text="Whether any stream on this channel supports catch-up (tv_archive=1)",
)
catchup_days = models.PositiveIntegerField(
default=0,
help_text="Max catch-up archive days across all streams on this channel",
)
# Hidden channels are excluded from HDHR, M3U, EPG, and XC output queries.
# Auto-sync still recognizes them so they are not recreated when their
# underlying provider stream persists; this is an output-layer concern, not

1
apps/channels/serializers.py
View file

@ -448,6 +448,7 @@ class ChannelSerializer(serializers.ModelSerializer):
"logo_id",
"user_level",
"is_adult",
"is_catchup",
"hidden_from_output",
"auto_created",
"auto_created_by",

19
apps/channels/signals.py
View file

@ -5,7 +5,7 @@ from django.dispatch import receiver
from django.utils.timezone import now, is_aware, make_aware
from celery.result import AsyncResult
from django_celery_beat.models import ClockedSchedule, PeriodicTask
from .models import Channel, Stream, ChannelProfile, ChannelProfileMembership, Recording
from .models import Channel, Stream, ChannelStream, ChannelProfile, ChannelProfileMembership, Recording
from apps.m3u.models import M3UAccount
from apps.epg.tasks import parse_programs_for_tvg_id
import json
@ -369,3 +369,20 @@ def schedule_task_on_save(sender, instance, created, **kwargs):
@receiver(post_delete, sender=Recording)
def revoke_task_on_delete(sender, instance, **kwargs):
revoke_task(instance.task_id)
@receiver([post_save, post_delete], sender=ChannelStream)
def update_channel_catchup_fields(sender, instance, **kwargs):
"""Roll up catch-up flags from active streams (UI path; import uses SQL rollup)."""
from django.db.models import Max
channel = instance.channel
catchup_qs = channel.streams.filter(
is_catchup=True,
m3u_account__is_active=True,
)
max_days = catchup_qs.aggregate(max_days=Max("catchup_days"))["max_days"]
Channel.objects.filter(pk=channel.pk).update(
is_catchup=catchup_qs.exists(),
catchup_days=max_days or 0,
)

100
apps/channels/tests/test_catchup_utils.py Normal file
View file

@ -0,0 +1,100 @@
from unittest.mock import patch
from django.test import RequestFactory, SimpleTestCase, TestCase
from apps.accounts.models import User
from apps.channels.models import Channel, ChannelStream, Stream
from apps.channels.utils import resolve_xc_epg_prev_days
from apps.m3u.models import M3UAccount
class ResolveXcEpgPrevDaysTests(SimpleTestCase):
def setUp(self):
self.factory = RequestFactory()
self.user = User(username="xc-prev", custom_properties={})
def test_url_prev_days_zero_is_explicit(self):
request = self.factory.get("/xmltv.php?prev_days=0")
self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 0)
def test_user_epg_prev_days_used_when_url_omitted(self):
self.user.custom_properties = {"epg_prev_days": 5}
request = self.factory.get("/xmltv.php")
self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 5)
@patch("apps.channels.utils.compute_provider_archive_days_capped", return_value=14)
def test_auto_detect_only_when_no_url_or_user_default(self, mock_compute):
request = self.factory.get("/xmltv.php")
self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 14)
mock_compute.assert_called_once()
@patch("apps.channels.utils.compute_provider_archive_days_capped", return_value=14)
def test_per_channel_epg_skips_global_auto_detect(self, mock_compute):
request = self.factory.get("/player_api.php")
self.assertEqual(
resolve_xc_epg_prev_days(request, self.user, auto_detect_fallback=False),
0,
)
mock_compute.assert_not_called()
@patch("apps.channels.utils.compute_provider_archive_days_capped", return_value=14)
def test_user_default_prevents_auto_detect(self, mock_compute):
self.user.custom_properties = {"epg_prev_days": 3}
request = self.factory.get("/xmltv.php")
self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 3)
mock_compute.assert_not_called()
@patch("apps.channels.utils.compute_provider_archive_days_capped", return_value=14)
@patch("core.models.CoreSettings.get_xmltv_prev_days_override", return_value=7)
def test_epg_settings_override_prevents_auto_detect(self, _override, mock_compute):
request = self.factory.get("/xmltv.php")
self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 7)
mock_compute.assert_not_called()
class CatchupRollupActiveAccountTests(TestCase):
"""Denormalized catch-up flags ignore disabled M3U accounts."""
@classmethod
def setUpTestData(cls):
cls.inactive = M3UAccount.objects.create(
name="catchup-inactive",
server_url="http://example.test",
account_type="XC",
is_active=False,
)
def test_channelstream_signal_ignores_inactive_catchup_stream(self):
channel = Channel.objects.create(name="inactive-only")
stream = Stream.objects.create(
name="inactive-catchup",
url="http://example.test/inactive",
m3u_account=self.inactive,
is_catchup=True,
catchup_days=9,
)
ChannelStream.objects.create(channel=channel, stream=stream, order=0)
channel.refresh_from_db()
self.assertFalse(channel.is_catchup)
self.assertEqual(channel.catchup_days, 0)
def test_rollup_ignores_inactive_catchup_stream(self):
from apps.m3u.tasks import rollup_channel_catchup_fields
channel = Channel.objects.create(name="rollup-inactive-only")
stream = Stream.objects.create(
name="rollup-inactive-catchup",
url="http://example.test/rollup-inactive",
m3u_account=self.inactive,
is_catchup=True,
catchup_days=9,
)
ChannelStream.objects.create(channel=channel, stream=stream, order=0)
Channel.objects.filter(pk=channel.pk).update(is_catchup=True, catchup_days=9)
rollup_channel_catchup_fields(self.inactive.id)
channel.refresh_from_db()
self.assertFalse(channel.is_catchup)
self.assertEqual(channel.catchup_days, 0)

193
apps/channels/tests/test_recording_playback_auth.py Normal file
View file

@ -0,0 +1,193 @@
"""Tests for DVR recording playback authentication (file/hls endpoints)."""
import os
import shutil
import tempfile
from types import SimpleNamespace
from unittest.mock import patch
from django.test import TestCase, override_settings
from django.utils import timezone
from rest_framework.test import APIClient
from rest_framework_simplejwt.tokens import RefreshToken
from apps.channels.api_views import _recording_auth_query_suffix
from apps.channels.models import Channel, Recording
def _make_admin():
from django.contrib.auth import get_user_model
User = get_user_model()
user, _ = User.objects.get_or_create(
username="recording_playback_admin",
defaults={"user_level": User.UserLevel.ADMIN},
)
user.user_level = User.UserLevel.ADMIN
user.set_password("pass")
user.save()
return user
@override_settings(ALLOWED_HOSTS=["testserver"])
@patch("apps.channels.api_views.network_access_allowed", return_value=True)
class RecordingPlaybackAuthTests(TestCase):
def setUp(self):
self.channel = Channel.objects.create(channel_number=42, name="Playback Auth Channel")
self.user = _make_admin()
self.client = APIClient()
self.tmp = tempfile.NamedTemporaryFile(suffix=".mkv", delete=False)
self.tmp.write(b"\x00" * 1024)
self.tmp.close()
self.hls_dir = tempfile.mkdtemp(prefix="dvr_playback_auth_hls_")
with open(os.path.join(self.hls_dir, "index.m3u8"), "w", encoding="utf-8") as playlist:
playlist.write("#EXTM3U\n#EXTINF:4.0,\nseg_00001.ts\n")
with open(os.path.join(self.hls_dir, "seg_00001.ts"), "wb") as segment:
segment.write(b"\x00" * 188)
now = timezone.now()
self.recording = Recording.objects.create(
channel=self.channel,
start_time=now,
end_time=now,
custom_properties={
"status": "completed",
"file_path": self.tmp.name,
"file_name": "test.mkv",
},
)
def tearDown(self):
if os.path.exists(self.tmp.name):
os.unlink(self.tmp.name)
if os.path.isdir(self.hls_dir):
shutil.rmtree(self.hls_dir, ignore_errors=True)
@staticmethod
def _jwt_for(user):
return str(RefreshToken.for_user(user).access_token)
def test_file_requires_authentication(self, _mock_network):
response = self.client.get(
f"/api/channels/recordings/{self.recording.id}/file/"
)
self.assertEqual(response.status_code, 403)
def test_file_accepts_jwt_query_param(self, _mock_network):
token = self._jwt_for(self.user)
response = self.client.get(
f"/api/channels/recordings/{self.recording.id}/file/",
{"token": token},
)
self.assertEqual(response.status_code, 200)
def test_file_redirect_to_hls_preserves_token(self, _mock_network):
pending = os.path.join(self.hls_dir, "pending.mkv")
now = timezone.now()
in_progress = Recording.objects.create(
channel=self.channel,
start_time=now,
end_time=now,
custom_properties={
"status": "recording",
"_hls_dir": self.hls_dir,
"file_path": pending,
},
)
token = self._jwt_for(self.user)
response = self.client.get(
f"/api/channels/recordings/{in_progress.id}/file/",
{"token": token},
)
self.assertEqual(response.status_code, 302)
self.assertIn("token=", response["Location"])
self.assertIn("/hls/index.m3u8", response["Location"])
def test_hls_playlist_rewrites_segments_with_token_when_present(self, _mock_network):
now = timezone.now()
hls_rec = Recording.objects.create(
channel=self.channel,
start_time=now,
end_time=now,
custom_properties={
"status": "recording",
"_hls_dir": self.hls_dir,
},
)
token = self._jwt_for(self.user)
response = self.client.get(
f"/api/channels/recordings/{hls_rec.id}/hls/index.m3u8",
{"token": token},
)
self.assertEqual(response.status_code, 200)
body = response.content.decode("utf-8")
self.assertIn("token=", body)
self.assertIn("seg_00001.ts", body)
def test_hls_playlist_omits_token_when_not_in_request(self, _mock_network):
now = timezone.now()
hls_rec = Recording.objects.create(
channel=self.channel,
start_time=now,
end_time=now,
custom_properties={
"status": "recording",
"_hls_dir": self.hls_dir,
},
)
token = self._jwt_for(self.user)
response = self.client.get(
f"/api/channels/recordings/{hls_rec.id}/hls/index.m3u8",
HTTP_AUTHORIZATION=f"Bearer {token}",
)
self.assertEqual(response.status_code, 200)
body = response.content.decode("utf-8")
self.assertIn("seg_00001.ts", body)
self.assertNotIn("token=", body)
def test_hls_segment_accepts_jwt_query_param(self, _mock_network):
now = timezone.now()
hls_rec = Recording.objects.create(
channel=self.channel,
start_time=now,
end_time=now,
custom_properties={
"status": "recording",
"_hls_dir": self.hls_dir,
},
)
token = self._jwt_for(self.user)
response = self.client.get(
f"/api/channels/recordings/{hls_rec.id}/hls/seg_00001.ts",
{"token": token},
)
self.assertEqual(response.status_code, 200)
def test_hls_redirect_to_file_preserves_token(self, _mock_network):
now = timezone.now()
hls_rec = Recording.objects.create(
channel=self.channel,
start_time=now,
end_time=now,
custom_properties={
"status": "completed",
"file_path": self.tmp.name,
"file_name": "test.mkv",
},
)
token = self._jwt_for(self.user)
response = self.client.get(
f"/api/channels/recordings/{hls_rec.id}/hls/index.m3u8",
{"token": token},
)
self.assertEqual(response.status_code, 302)
self.assertIn("token=", response["Location"])
self.assertIn("/file/", response["Location"])
class RecordingAuthQuerySuffixTests(TestCase):
def test_empty_when_no_token(self):
request = SimpleNamespace(GET={})
self.assertEqual(_recording_auth_query_suffix(request), "")
def test_includes_token_when_present(self):
request = SimpleNamespace(GET={"token": "abc123"})
self.assertEqual(_recording_auth_query_suffix(request), "?token=abc123")

187
apps/channels/tests/test_ts_proxy_teardown.py
View file

@ -1,6 +1,6 @@
"""Tests for multi-worker channel teardown coordination."""
import time
from unittest.mock import MagicMock, patch
from unittest.mock import ANY, MagicMock, patch
from django.test import TestCase
@ -692,9 +692,8 @@ class InitWaitAbortTests(TestCase):
self.assertEqual(generator._init_wait_abort_reason(server, time.time()), "client_gone")
def test_abort_when_connect_stalled_without_buffer(self):
from apps.proxy.config import TSConfig as Config
@patch("apps.proxy.live_proxy.output.ts.generator.ConfigHelper.channel_init_grace_period", return_value=10)
def test_abort_when_connect_stalled_without_buffer(self, _mock_grace):
generator = self._make_generator()
server = MagicMock()
client_manager = MagicMock()
@ -707,9 +706,128 @@ class InitWaitAbortTests(TestCase):
server.redis_client.hget.return_value = b"connecting"
server.redis_client.get.return_value = None
started = time.time() - (getattr(Config, "CONNECTION_TIMEOUT", 10) + 1)
started = time.time() - 11
self.assertEqual(generator._init_wait_abort_reason(server, started), "stalled")
@patch("apps.proxy.live_proxy.output.ts.generator.ConfigHelper.channel_init_grace_period", return_value=30)
def test_no_stall_abort_within_init_grace_period(self, _mock_grace):
generator = self._make_generator()
server = MagicMock()
client_manager = MagicMock()
client_manager.clients = {"client-1"}
server.client_managers = {CHANNEL_ID: client_manager}
buffer = MagicMock()
buffer.index = 0
server.stream_buffers = {CHANNEL_ID: buffer}
server.redis_client = MagicMock()
server.redis_client.hget.return_value = b"connecting"
server.redis_client.get.return_value = None
started = time.time() - 15
self.assertIsNone(generator._init_wait_abort_reason(server, started))
@patch("apps.proxy.live_proxy.output.ts.generator.ConfigHelper.channel_init_grace_period", return_value=30)
def test_stall_abort_after_init_grace_period(self, _mock_grace):
generator = self._make_generator()
server = MagicMock()
client_manager = MagicMock()
client_manager.clients = {"client-1"}
server.client_managers = {CHANNEL_ID: client_manager}
buffer = MagicMock()
buffer.index = 0
server.stream_buffers = {CHANNEL_ID: buffer}
server.redis_client = MagicMock()
server.redis_client.hget.return_value = b"connecting"
server.redis_client.get.return_value = None
started = time.time() - 31
self.assertEqual(generator._init_wait_abort_reason(server, started), "stalled")
class PromoteChannelWhenBufferReadyTests(TestCase):
def _mock_proxy(self, redis_client):
proxy_server = MagicMock()
proxy_server.redis_client = redis_client
return patch(
"apps.proxy.live_proxy.services.channel_service.ProxyServer.get_instance",
return_value=proxy_server,
), proxy_server
@patch("apps.proxy.live_proxy.services.channel_service.ConfigHelper.initial_behind_chunks", return_value=4)
def test_buffer_ready_with_clients_becomes_active(self, _mock_chunks):
redis = MagicMock()
redis.hget.return_value = ChannelState.CONNECTING.encode()
redis.get.return_value = b"4"
redis.scard.return_value = 2
ctx, proxy_server = self._mock_proxy(redis)
with ctx:
result = ChannelService.promote_channel_when_buffer_ready(CHANNEL_ID)
self.assertEqual(result, ChannelState.ACTIVE)
proxy_server.update_channel_state.assert_called_once()
args = proxy_server.update_channel_state.call_args[0]
self.assertEqual(args[1], ChannelState.ACTIVE)
self.assertEqual(args[2]["clients_at_activation"], "2")
@patch("apps.proxy.live_proxy.services.channel_service.ConfigHelper.initial_behind_chunks", return_value=4)
def test_buffer_ready_without_clients_becomes_waiting(self, _mock_chunks):
redis = MagicMock()
redis.hget.return_value = ChannelState.CONNECTING.encode()
redis.get.return_value = b"5"
redis.scard.return_value = 0
ctx, proxy_server = self._mock_proxy(redis)
with ctx:
result = ChannelService.promote_channel_when_buffer_ready(CHANNEL_ID)
self.assertEqual(result, ChannelState.WAITING_FOR_CLIENTS)
proxy_server.update_channel_state.assert_called_once_with(
CHANNEL_ID,
ChannelState.WAITING_FOR_CLIENTS,
{
ChannelMetadataField.CONNECTION_READY_TIME: ANY,
ChannelMetadataField.BUFFER_CHUNKS: "5",
},
)
@patch("apps.proxy.live_proxy.services.channel_service.ConfigHelper.initial_behind_chunks", return_value=4)
def test_buffer_not_ready_does_not_promote(self, _mock_chunks):
redis = MagicMock()
redis.hget.return_value = ChannelState.CONNECTING.encode()
redis.get.return_value = b"2"
ctx, proxy_server = self._mock_proxy(redis)
with ctx:
result = ChannelService.promote_channel_when_buffer_ready(CHANNEL_ID)
self.assertIsNone(result)
proxy_server.update_channel_state.assert_not_called()
def test_waiting_for_clients_with_clients_becomes_active(self):
redis = MagicMock()
def hget_side_effect(key, field):
if field == ChannelMetadataField.STATE:
return ChannelState.WAITING_FOR_CLIENTS.encode()
if field == ChannelMetadataField.CONNECTION_READY_TIME:
return b"1700000000.0"
return None
redis.hget.side_effect = hget_side_effect
redis.scard.return_value = 1
ctx, proxy_server = self._mock_proxy(redis)
with ctx:
result = ChannelService.promote_channel_when_buffer_ready(CHANNEL_ID)
self.assertEqual(result, ChannelState.ACTIVE)
proxy_server.update_channel_state.assert_called_once_with(
CHANNEL_ID,
ChannelState.ACTIVE,
{"clients_at_activation": "1"},
)
class UpstreamStopBroadcastTests(TestCase):
def _make_server(self):
@ -805,3 +923,62 @@ class StreamManagerStillOwnerTests(TestCase):
return_value=5,
):
self.assertTrue(manager._still_owner())
class PreActiveNoClientsTimeoutTests(TestCase):
@patch("apps.proxy.live_proxy.server.ConfigHelper.channel_client_wait_period", return_value=5)
def test_buffer_ready_uses_client_wait_period(self, _mock_client_wait):
should_stop, timeout, reason = ProxyServer._pre_active_no_clients_should_stop(
connection_ready_time=1000.0,
start_time=900.0,
now=1006.0,
)
self.assertTrue(should_stop)
self.assertEqual(timeout, 5)
self.assertEqual(reason, "client_wait")
@patch("apps.proxy.live_proxy.server.ConfigHelper.channel_client_wait_period", return_value=5)
def test_buffer_ready_within_client_wait_period(self, _mock_client_wait):
should_stop, timeout, reason = ProxyServer._pre_active_no_clients_should_stop(
connection_ready_time=1000.0,
start_time=900.0,
now=1003.0,
)
self.assertFalse(should_stop)
self.assertEqual(timeout, 5)
self.assertEqual(reason, "client_wait")
@patch("apps.proxy.live_proxy.server.ConfigHelper.channel_init_grace_period", return_value=60)
def test_startup_uses_init_grace_period(self, _mock_init_grace):
should_stop, timeout, reason = ProxyServer._pre_active_no_clients_should_stop(
connection_ready_time=None,
start_time=1000.0,
now=1070.0,
)
self.assertTrue(should_stop)
self.assertEqual(timeout, 60)
self.assertEqual(reason, "startup")
@patch("apps.proxy.live_proxy.server.ConfigHelper.channel_init_grace_period", return_value=60)
def test_startup_within_init_grace_period(self, _mock_init_grace):
should_stop, timeout, reason = ProxyServer._pre_active_no_clients_should_stop(
connection_ready_time=None,
start_time=1000.0,
now=1030.0,
)
self.assertFalse(should_stop)
self.assertEqual(timeout, 60)
self.assertEqual(reason, "startup")
@patch("apps.proxy.live_proxy.server.ConfigHelper.channel_shutdown_delay", return_value=30)
@patch("apps.proxy.live_proxy.server.ConfigHelper.channel_client_wait_period", return_value=5)
def test_buffer_ready_does_not_use_shutdown_delay(self, mock_client_wait, mock_shutdown_delay):
should_stop, _, reason = ProxyServer._pre_active_no_clients_should_stop(
connection_ready_time=1000.0,
start_time=900.0,
now=1006.0,
)
self.assertTrue(should_stop)
self.assertEqual(reason, "client_wait")
mock_client_wait.assert_called_once()
mock_shutdown_delay.assert_not_called()

90
apps/channels/utils.py
View file

@ -1,8 +1,13 @@
import logging
import threading
from django.core.cache import cache
logger = logging.getLogger(__name__)
PROVIDER_ARCHIVE_CACHE_TTL_SECONDS = 300
MAX_AUTO_PREV_DAYS = 30
# Bound memory/DB work per chunk for large libraries (20k+ channels).
EPG_LOGO_APPLY_BATCH_SIZE = 500
EPG_LOGO_APPLY_MAX_ERRORS = 100
@ -23,6 +28,91 @@ def format_channel_number(value, empty=""):
return int(value)
return value
def compute_provider_archive_days_capped():
"""Max ``catchup_days`` across active XC catch-up streams (capped, cached).
Cached briefly so XC XMLTV exports without an explicit ``prev_days`` do not
repeat the aggregate query on every request.
"""
def _scan():
from django.db.models import Max
from apps.channels.models import Stream
result = Stream.objects.filter(
m3u_account__account_type="XC",
m3u_account__is_active=True,
is_catchup=True,
).aggregate(max_days=Max("catchup_days"))
return min(result["max_days"] or 0, MAX_AUTO_PREV_DAYS)
return cache.get_or_set(
"channels:provider_archive_days_capped",
_scan,
PROVIDER_ARCHIVE_CACHE_TTL_SECONDS,
)
def resolve_xc_epg_prev_days(request, user, *, auto_detect_fallback=True):
"""Resolve ``prev_days`` for XC XMLTV and player_api EPG.
Args:
request: HTTP request (reads ``?prev_days=``).
user: Authenticated user (reads ``custom_properties.epg_prev_days``).
auto_detect_fallback: When True (XC XMLTV), fall back to the largest
provider archive depth. When False (per-channel EPG), return 0 so
``xc_get_epg`` can expand to each channel's ``catchup_days``.
Resolution order:
1. URL ``?prev_days=`` (explicit; 0 means no past programmes)
2. ``user.custom_properties.epg_prev_days``
3. ``CoreSettings.epg_settings.xmltv_prev_days_override`` when > 0
4. Auto-detect (only when *auto_detect_fallback* is True)
"""
user_custom = (user.custom_properties or {}) if user else {}
url_prev = request.GET.get("prev_days")
user_prev = user_custom.get("epg_prev_days") if user_custom else None
if url_prev is not None:
try:
return max(0, min(int(url_prev), MAX_AUTO_PREV_DAYS))
except (ValueError, TypeError):
return 0
if user_prev not in (None, ""):
try:
return max(0, min(int(user_prev), MAX_AUTO_PREV_DAYS))
except (ValueError, TypeError):
return 0
from core.models import CoreSettings
try:
override = int(CoreSettings.get_xmltv_prev_days_override() or 0)
except (TypeError, ValueError):
override = 0
if override > 0:
return max(0, min(override, MAX_AUTO_PREV_DAYS))
if auto_detect_fallback:
return compute_provider_archive_days_capped()
return 0
def get_channel_catchup_streams(channel):
"""Active catch-up streams for a channel, in ``channelstream`` order.
Inactive M3U accounts are excluded, matching live dispatch.
"""
if not getattr(channel, "is_catchup", False):
return []
return list(
channel.streams.filter(is_catchup=True, m3u_account__is_active=True)
.order_by("channelstream__order")
.select_related("m3u_account")
)
def increment_stream_count(account):
with lock:
current_usage = active_streams_map.get(account.id, 0)

29
apps/epg/api_views.py
View file

@ -64,8 +64,6 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
from apps.channels.models import Channel
return EPGSource.objects.select_related(
"refresh_task__crontab", "refresh_task__interval"
).defer(
'programme_index'
).annotate(
has_channels=Exists(
Channel.objects.filter(epg_data__epg_source_id=OuterRef('pk'))
@ -986,7 +984,9 @@ class EPGGridAPIView(APIView):
channel_name=name_to_parse,
num_days=1,
program_length_hours=4,
epg_source=epg_source
epg_source=epg_source,
export_lookback=one_hour_ago,
export_cutoff=twenty_four_hours_later,
)
# Custom dummy should always return data (either from patterns or fallback)
@ -1082,13 +1082,19 @@ class EPGGridAPIView(APIView):
f"Error creating standard dummy programs for channel {channel.name} (ID: {channel.id}): {str(e)}"
)
# Combine regular and dummy programs
all_programs = list(serialized_programs) + dummy_programs
# Combine regular and dummy programs in place to avoid copying the large list
serialized_programs.extend(dummy_programs)
logger.debug(
f"EPGGridAPIView: Returning {len(all_programs)} total programs (including {len(dummy_programs)} dummy programs)."
f"EPGGridAPIView: Returning {len(serialized_programs)} total programs (including {len(dummy_programs)} dummy programs)."
)
return Response({"data": all_programs}, status=status.HTTP_200_OK)
# The grid materializes tens of thousands of program dicts plus the
# rendered JSON; trim once the response is sent so worker RSS does not
# ratchet up per request.
from core.utils import spawn_memory_trim
response = Response({"data": serialized_programs}, status=status.HTTP_200_OK)
response._resource_closers.append(spawn_memory_trim)
return response
# ─────────────────────────────
@ -1119,7 +1125,7 @@ class EPGImportAPIView(APIView):
epg_id = request.data.get("id", None)
force = bool(request.data.get("force", False))
# Reject dummy sources without loading programme_index (multi-MB JSON).
# Reject dummy sources with a narrow existence query, no full row load.
if epg_id is not None:
from .models import EPGSource
@ -1236,10 +1242,9 @@ class CurrentProgramsAPIView(APIView):
# Limit to 50 IDs per request
epg_data_ids = epg_data_ids[:50]
# Defer the multi-MB programme_index the JOIN would pull once per row. The lookup reads it via a targeted refresh_from_db
epg_data_entries = EPGData.objects.select_related('epg_source').defer(
'epg_source__programme_index'
).filter(id__in=epg_data_ids)
epg_data_entries = EPGData.objects.select_related('epg_source').filter(
id__in=epg_data_ids
)
# Batch-fetch current programs for all requested EPG entries in one query
db_programs = ProgramData.objects.filter(

40
apps/epg/migrations/0025_programdata_epg_id_index.py Normal file
View file

@ -0,0 +1,40 @@
from django.contrib.postgres.operations import AddIndexConcurrently
from django.db import migrations, models
class AddIndexConcurrentlyIfPostgres(AddIndexConcurrently):
"""Create the index CONCURRENTLY on PostgreSQL (no table lock on large
tables), falling back to a normal blocking AddIndex on other backends
such as the sqlite dev/test fallback."""
def database_forwards(self, app_label, schema_editor, from_state, to_state):
if schema_editor.connection.vendor == 'postgresql':
super().database_forwards(app_label, schema_editor, from_state, to_state)
else:
migrations.AddIndex.database_forwards(
self, app_label, schema_editor, from_state, to_state
)
def database_backwards(self, app_label, schema_editor, from_state, to_state):
if schema_editor.connection.vendor == 'postgresql':
super().database_backwards(app_label, schema_editor, from_state, to_state)
else:
migrations.AddIndex.database_backwards(
self, app_label, schema_editor, from_state, to_state
)
class Migration(migrations.Migration):
# CREATE INDEX CONCURRENTLY cannot run inside a transaction.
atomic = False
dependencies = [
('epg', '0024_remove_epgsource_api_key_epgsource_password_and_more'),
]
operations = [
AddIndexConcurrentlyIfPostgres(
model_name='programdata',
index=models.Index(fields=['epg', 'id'], name='epg_prog_epg_id_idx'),
),
]

52
apps/epg/migrations/0026_epgsourceindex.py Normal file
View file

@ -0,0 +1,52 @@
import django.db.models.deletion
from django.db import migrations, models
def copy_index_forward(apps, schema_editor):
EPGSource = apps.get_model('epg', 'EPGSource')
EPGSourceIndex = apps.get_model('epg', 'EPGSourceIndex')
rows = list(
EPGSource.objects.exclude(programme_index__isnull=True).values_list(
'id', 'programme_index'
)
)
for source_id, data in rows:
EPGSourceIndex.objects.update_or_create(
source_id=source_id, defaults={'data': data}
)
def copy_index_backward(apps, schema_editor):
EPGSource = apps.get_model('epg', 'EPGSource')
EPGSourceIndex = apps.get_model('epg', 'EPGSourceIndex')
for source_id, data in EPGSourceIndex.objects.values_list('source_id', 'data'):
EPGSource.objects.filter(id=source_id).update(programme_index=data)
class Migration(migrations.Migration):
dependencies = [
('epg', '0025_programdata_epg_id_index'),
]
operations = [
migrations.CreateModel(
name='EPGSourceIndex',
fields=[
('source', models.OneToOneField(
on_delete=django.db.models.deletion.CASCADE,
primary_key=True,
related_name='index_record',
serialize=False,
to='epg.epgsource',
)),
('data', models.JSONField(blank=True, default=None, null=True)),
('updated_at', models.DateTimeField(auto_now=True)),
],
),
migrations.RunPython(copy_index_forward, copy_index_backward),
migrations.RemoveField(
model_name='epgsource',
name='programme_index',
),
]

42
apps/epg/models.py
View file

@ -62,12 +62,6 @@ class EPGSource(models.Model):
blank=True,
help_text="Last status message, including success results or error information"
)
programme_index = models.JSONField(
null=True,
blank=True,
default=None,
help_text="Byte-offset index mapping tvg_id to file positions, built after each EPG refresh"
)
created_at = models.DateTimeField(
auto_now_add=True,
help_text="Time when this source was created"
@ -124,6 +118,37 @@ class EPGSource(models.Model):
kwargs['update_fields'].remove('updated_at')
super().save(*args, **kwargs)
@property
def programme_index(self):
"""Byte-offset index for this source, read on demand from the separate
EPGSourceIndex table so the multi-MB blob is never pulled into EPGSource
queries or select_related JOINs. Returns the stored dict or None."""
return (
EPGSourceIndex.objects.filter(source_id=self.pk)
.values_list('data', flat=True)
.first()
)
class EPGSourceIndex(models.Model):
"""Byte-offset programme index for an EPGSource, stored in its own table.
Kept out of EPGSource so the multi-MB JSON blob is only loaded when read
explicitly, never when querying or joining EPGSource rows.
"""
source = models.OneToOneField(
EPGSource,
on_delete=models.CASCADE,
related_name='index_record',
primary_key=True,
)
data = models.JSONField(null=True, blank=True, default=None)
updated_at = models.DateTimeField(auto_now=True)
def __str__(self):
return f"Programme index for source {self.source_id}"
class EPGData(models.Model):
tvg_id = models.CharField(max_length=255, null=True, blank=True, db_index=True)
name = models.CharField(max_length=512)
@ -153,6 +178,11 @@ class ProgramData(models.Model):
program_id = models.CharField(max_length=64, null=True, blank=True, help_text='Schedules Direct programID (e.g. EP123456789). Null for XMLTV sources.')
custom_properties = models.JSONField(default=dict, blank=True, null=True)
class Meta:
indexes = [
models.Index(fields=['epg', 'id'], name='epg_prog_epg_id_idx'),
]
def __str__(self):
return f"{self.title} ({self.start_time} - {self.end_time})"

17
apps/epg/tasks.py
View file

@ -26,7 +26,7 @@ from core.models import UserAgent, CoreSettings
from asgiref.sync import async_to_sync
from channels.layers import get_channel_layer
from .models import EPGSource, EPGData, ProgramData, SDScheduleMD5, SDProgramMD5
from .models import EPGSource, EPGSourceIndex, EPGData, ProgramData, SDScheduleMD5, SDProgramMD5
from core.utils import (
acquire_task_lock,
is_task_lock_held,
@ -653,7 +653,9 @@ def _refresh_epg_data_impl(source_id, force=False):
if source.source_type == 'xmltv':
# Invalidate the byte-offset index before downloading the new file
# so stale offsets are never used during the refresh window.
EPGSource.objects.filter(id=source.id).update(programme_index=None)
EPGSourceIndex.objects.update_or_create(
source_id=source.id, defaults={'data': None}
)
if not fetch_xmltv(source):
logger.error(f"Failed to fetch XMLTV for source {source.name}")
return
@ -4484,7 +4486,7 @@ def _programme_to_dict(elem, start_time, end_time):
def build_programme_index(source_id):
"""
Scan the XML file with raw binary I/O to build a {tvg_id: [byte_offset, ...]} map.
Persists the result to EPGSource.programme_index. Most XMLTV files group programmes
Persists the result to the EPGSourceIndex table. Most XMLTV files group programmes
by channel, but some split a channel across multiple non-contiguous blocks, so we
record block starts up to _OFFSET_CAP and mark only channels that exceed the cap
as interleaved.
@ -4573,7 +4575,9 @@ def build_programme_index(source_id):
'channels': index,
'interleaved_channels': sorted(interleaved_channels),
}
EPGSource.objects.filter(id=source_id).update(programme_index=result)
EPGSourceIndex.objects.update_or_create(
source_id=source_id, defaults={'data': result}
)
@shared_task
@ -4620,9 +4624,8 @@ def find_current_program_for_tvg_id(epg_or_id):
return None
now = timezone.now()
# Force a fresh read of the DB-backed index to avoid using stale related-object
# state when an EPG refresh invalidates/rebuilds the index concurrently.
source.refresh_from_db(fields=['programme_index'])
# The property reads the EPGSourceIndex table fresh on each access, so a
# concurrent refresh invalidating/rebuilding the index can't serve stale state.
index = source.programme_index
if index is not None:

7
apps/epg/tests/test_epg_import_api.py
View file

@ -7,7 +7,7 @@ from django.test import TestCase
from rest_framework import status
from rest_framework.test import APIClient
from apps.epg.models import EPGSource
from apps.epg.models import EPGSource, EPGSourceIndex
User = get_user_model()
@ -47,7 +47,10 @@ class EPGImportAPITests(TestCase):
name="Large Index XMLTV",
source_type="xmltv",
url="http://example.com/epg.xml",
programme_index={
)
EPGSourceIndex.objects.create(
source=source,
data={
"channels": {f"ch.{i}": {"offsets": [0, 100]} for i in range(200)},
"interleaved_channels": [],
},

1
apps/epg/tests/test_programme_index.py
View file

@ -519,7 +519,6 @@ class FindCurrentProgramTests(TestCase):
name="No Index",
source_type="xmltv",
file_path=FIXTURE_XML,
programme_index=None,
)
epg = EPGData.objects.create(
tvg_id="channel.current",

4
apps/m3u/api_urls.py
View file

@ -16,12 +16,12 @@ app_name = "m3u"
router = DefaultRouter()
router.register(r"accounts", M3UAccountViewSet, basename="m3u-account")
router.register(
r"accounts\/(?P<account_id>\d+)\/profiles",
r"accounts/(?P<account_id>\d+)/profiles",
M3UAccountProfileViewSet,
basename="m3u-account-profiles",
)
router.register(
r"accounts\/(?P<account_id>\d+)\/filters",
r"accounts/(?P<account_id>\d+)/filters",
M3UFilterViewSet,
basename="m3u-filters",
)

636
apps/m3u/tasks.py
View file

File diff suppressed because it is too large Load diff

24
apps/m3u/tests/test_memory_cleanup.py
View file

@ -30,7 +30,7 @@ class ProcessM3UBatchCleanupTests(SimpleTestCase):
mock_stream_cls.generate_hash_key = MagicMock(return_value="hash123")
with patch("django.db.connections") as mock_connections:
process_m3u_batch_direct(1, [], {}, ["name", "url"])
process_m3u_batch_direct(1, [], {}, ["name", "url"], compiled_filters=[])
mock_connections.close_all.assert_called()
@patch("apps.m3u.tasks.Stream")
@ -48,9 +48,29 @@ class ProcessM3UBatchCleanupTests(SimpleTestCase):
mock_stream_cls.generate_hash_key = MagicMock(return_value="hash123")
with patch("gc.collect") as mock_gc, patch("django.db.connections"):
process_m3u_batch_direct(1, [], {}, ["name", "url"])
process_m3u_batch_direct(1, [], {}, ["name", "url"], compiled_filters=[])
mock_gc.assert_called()
@patch("apps.m3u.tasks.Stream")
@patch("apps.m3u.tasks.M3UAccount")
def test_precompiled_empty_filters_skip_db_lookup(
self, mock_account_cls, mock_stream_cls,
):
"""When filters are precompiled as empty, batch workers must not query filters."""
from apps.m3u.tasks import process_m3u_batch_direct
mock_account = MagicMock()
mock_account_cls.objects.get.return_value = mock_account
mock_stream_cls.objects.filter.return_value.select_related.return_value.only.return_value = (
[]
)
mock_stream_cls.generate_hash_key = MagicMock(return_value="hash123")
with patch("django.db.connections"):
process_m3u_batch_direct(1, [], {}, ["name", "url"], compiled_filters=[])
mock_account.filters.order_by.assert_not_called()
class LockReleaseTests(SimpleTestCase):
"""Verify task lock is released on all exit paths."""

212
apps/m3u/tests/test_rename_preview_parity.py Normal file
View file

@ -0,0 +1,212 @@
"""
Differential parity tests: the auto-sync rename preview must predict the exact
name the live rename produces, across a broad matrix of regex strategies.
Both the preview and the live rename compile with the third-party `regex`
module (for its JS-aligned syntax and per-call timeout). They can only be
trusted together if, for every find/replace a user might author, the preview's
predicted `after` equals the channel name the sync actually writes.
Each case is run end-to-end: real streams, the real sync_auto_channels rename,
and the real regex-preview endpoint, compared per original stream name.
"""
from django.test import TestCase
from django.utils import timezone
from apps.channels.models import Channel, ChannelGroup, ChannelStream, Stream
from apps.m3u.models import M3UAccount
from apps.m3u.tasks import sync_auto_channels
# Diverse names: distinct word-prefixes (collision-resistant), with quality
# tags, brackets, pipes, ampersands, extra whitespace, underscores, dots, CJK,
# and emoji, to exercise anchors, classes, boundaries, and Unicode handling.
NAMES = [
"Alpha Channel 11",
"Bravo Sports HD",
"Charlie News FHD",
"Delta Movie (2024)",
"Echo [UK] 4K",
"Foxtrot spaced",
"Golf|Pipe|Name",
"Hotel & Inn <x>",
"India 日本語 77",
"Juliet 📺 88",
"Kilo_under_9",
"Lima.dot.name",
]
# (find, replace) pairs spanning common user strategies and edge cases.
STRATEGIES = [
# --- capture groups ---
(r"(.+) Channel (\d+)", r"$1 #$2"),
(r"(\w+) (\w+)", r"$2 $1"),
(r"(.+)", r"$1 - $1"),
(r"(.+)", r"[$1]"),
(r"(.+) (\d+)$", r"$2 $1"),
# --- strip / delete ---
(r" (HD|FHD|4K|SD)\b", r""),
(r"\s+", r" "),
(r"[\[\(].*?[\]\)]", r""),
(r"\d+", r""),
(r"[_.]", r" "),
# --- anchors / inserts ---
(r"^", r"NEW "),
(r"$", r" LIVE"),
(r"^(\w+)", r"<$1>"),
# --- char classes / Unicode (divergence hunters) ---
(r"\w+", r"W"),
(r"\b\w", r"_"),
(r"[A-Z]", r"*"),
(r"\s", r"_"),
(r"[^\x00-\x7F]+", r"?"),
# --- non-capturing / lookaround / in-pattern backref ---
(r"(?:Channel|Movie|News) ", r""),
(r"(\w)\1", r"$1"),
(r"\w+(?= )", r"X"),
(r"(?<=\d)\d", r"#"),
# --- literal $ and odd replacements ---
(r" ", r" $ "),
(r"o", r"0"),
# --- invalid group references (rejected by both engines) ---
(r"(.+)", r"$2"),
(r"(.+)", r"$10"),
# --- rename that expands past Channel.name's column length ---
(r"(.+)", r"$1" * 40),
# --- regex-module syntax: quantified anchor, JS-style and duplicate
# named groups (these transform; both paths use the regex module) ---
(r"^*", r"$"),
(r"(?<season>\d+)", r"S$1"),
(r"(?P<n>x)(?P<n>y)", r"z"),
]
class RenamePreviewParityTests(TestCase):
@classmethod
def setUpClass(cls):
super().setUpClass()
from rest_framework.test import APIClient
from apps.accounts.models import User
admin = User.objects.create_superuser(
username="admin_rename_parity", password="pw", user_level=10
)
cls.client_api = APIClient()
cls.client_api.force_authenticate(user=admin)
cls.account = M3UAccount.objects.create(
name="Rename Parity Provider",
server_url="http://example.com/test.m3u",
)
def _sync(self):
return sync_auto_channels(
self.account.id,
scan_start_time=(
timezone.now() - timezone.timedelta(minutes=1)
).isoformat(),
)
def _run_case(self, group_name, find, replace):
"""Returns a list of human-readable mismatch strings (empty == parity)."""
group = ChannelGroup.objects.create(name=group_name)
from apps.channels.models import ChannelGroupM3UAccount
ChannelGroupM3UAccount.objects.create(
m3u_account=self.account,
channel_group=group,
enabled=True,
auto_channel_sync=True,
auto_sync_channel_start=1000,
custom_properties={
"name_regex_pattern": find,
"name_replace_pattern": replace,
},
)
for i, name in enumerate(NAMES):
Stream.objects.create(
name=name,
url=f"http://example.com/{group_name}_{i}.m3u8",
m3u_account=self.account,
channel_group=group,
tvg_id=f"{group_name}-{i}",
last_seen=timezone.now(),
)
# --- live rename via sync ---
result = self._sync()
if result.get("status") != "ok":
return [f"[{find!r} -> {replace!r}] sync status={result.get('status')}"]
channels = Channel.objects.filter(
auto_created_by=self.account, channel_group=group
)
cs_rows = ChannelStream.objects.filter(
channel__in=channels
).select_related("channel", "stream")
sync_map = {row.stream.name: row.channel.name for row in cs_rows}
# --- preview endpoint ---
response = self.client_api.get(
"/api/channels/streams/regex-preview/",
{
"channel_group": group_name,
"find": find,
"replace": replace,
"limit": 50,
},
)
if response.status_code != 200:
return [f"[{find!r} -> {replace!r}] preview HTTP {response.status_code}"]
data = response.data
# When the preview reports find_error it predicts no rename at all.
preview_map = {name: name for name in NAMES}
if "find_error" not in data:
for m in data.get("find_matches", []):
preview_map[m["before"]] = m["after"]
mismatches = []
for name in NAMES:
sync_after = sync_map.get(name)
if sync_after is None:
mismatches.append(
f"[{find!r} -> {replace!r}] {name!r}: no channel created"
)
continue
preview_after = preview_map[name]
if preview_after != sync_after:
mismatches.append(
f"[{find!r} -> {replace!r}] {name!r}: "
f"preview={preview_after!r} sync={sync_after!r} "
f"(find_error={data.get('find_error')!r})"
)
return mismatches
def test_preview_predicts_rename_across_strategies(self):
all_mismatches = []
for idx, (find, replace) in enumerate(STRATEGIES):
all_mismatches.extend(
self._run_case(f"ParityG{idx}", find, replace)
)
self.assertEqual(
all_mismatches,
[],
"Preview diverged from the live rename:\n"
+ "\n".join(all_mismatches),
)
def test_overlong_rename_is_bounded_not_aborting_sync(self):
# A rename that expands past Channel.name's column length must not
# abort the bulk_create sync. Both sync and preview cap at the column
# length so the channel is created (truncated) and the preview shows
# the same bounded name.
max_len = Channel._meta.get_field("name").max_length
mismatches = self._run_case("OverlongG", r"(.+)", "$1" * 40)
self.assertEqual(mismatches, [], "\n".join(mismatches))
group = ChannelGroup.objects.get(name="OverlongG")
channels = Channel.objects.filter(
auto_created_by=self.account, channel_group=group
)
self.assertEqual(channels.count(), len(NAMES))
self.assertTrue(all(len(c.name) <= max_len for c in channels))
self.assertTrue(any(len(c.name) == max_len for c in channels))

156
apps/m3u/tests/test_stream_filters.py Normal file
View file

@ -0,0 +1,156 @@
"""Tests for M3U stream filter compilation and batch application."""
from unittest.mock import MagicMock, patch
from django.test import SimpleTestCase
from apps.m3u.tasks import (
_compile_m3u_stream_filters,
_stream_passes_m3u_filters,
process_m3u_batch_direct,
)
class CompileM3UStreamFiltersTests(SimpleTestCase):
def test_compiles_case_insensitive_when_configured(self):
filter_obj = MagicMock()
filter_obj.regex_pattern = "news"
filter_obj.custom_properties = {"case_sensitive": False}
compiled = _compile_m3u_stream_filters([filter_obj])
self.assertEqual(len(compiled), 1)
pattern, _ = compiled[0]
self.assertTrue(pattern.search("NEWS"))
def test_compiles_case_sensitive_by_default(self):
filter_obj = MagicMock()
filter_obj.regex_pattern = "news"
filter_obj.custom_properties = {}
compiled = _compile_m3u_stream_filters([filter_obj])
pattern, _ = compiled[0]
self.assertIsNone(pattern.search("NEWS"))
self.assertTrue(pattern.search("news"))
class StreamPassesM3UFiltersTests(SimpleTestCase):
def _compiled(self, *, filter_type="name", exclude=False, pattern="Adult"):
filter_obj = MagicMock()
filter_obj.filter_type = filter_type
filter_obj.exclude = exclude
filter_obj.regex_pattern = pattern
filter_obj.custom_properties = {}
return _compile_m3u_stream_filters([filter_obj])
def test_include_filter_passes_matching_stream(self):
compiled = self._compiled(exclude=False)
self.assertTrue(
_stream_passes_m3u_filters("Adult Channel", "http://x", "News", compiled)
)
def test_include_filter_passes_non_matching_stream(self):
"""Non-matching streams still pass unless a matching exclude filter hits."""
compiled = self._compiled(exclude=False, pattern="news")
self.assertTrue(
_stream_passes_m3u_filters("Sports", "http://x", "Sports", compiled)
)
def test_exclude_filter_rejects_matching_stream(self):
compiled = self._compiled(exclude=True, pattern="Adult")
self.assertFalse(
_stream_passes_m3u_filters("Adult Channel", "http://x", "News", compiled)
)
def test_url_filter_type_targets_url(self):
compiled = self._compiled(filter_type="url", exclude=True, pattern="blocked")
self.assertFalse(
_stream_passes_m3u_filters("OK", "http://blocked.example/live", "News", compiled)
)
self.assertTrue(
_stream_passes_m3u_filters("blocked name", "http://ok.example/live", "News", compiled)
)
def test_group_filter_type_targets_group(self):
compiled = self._compiled(filter_type="group", exclude=True, pattern="Hidden")
self.assertFalse(
_stream_passes_m3u_filters("Channel", "http://x", "Hidden Group", compiled)
)
class ProcessM3UBatchFilterTests(SimpleTestCase):
def _mock_stream_meta(self, mock_stream_cls, max_length=255):
mock_field = MagicMock()
mock_field.max_length = max_length
mock_stream_cls._meta.get_field.return_value = mock_field
@patch("apps.m3u.tasks._bulk_update_stream_refresh_batches")
@patch("apps.m3u.tasks.Stream")
@patch("apps.m3u.tasks.M3UAccount")
def test_exclude_filter_skips_stream_import(
self, mock_account_cls, mock_stream_cls, mock_bulk_update,
):
self._mock_stream_meta(mock_stream_cls)
mock_account = MagicMock()
mock_account.account_type = "STD"
mock_account_cls.objects.get.return_value = mock_account
mock_stream_cls.objects.filter.return_value.select_related.return_value.only.return_value = (
[]
)
mock_stream_cls.generate_hash_key = MagicMock(return_value="hash123")
filter_obj = MagicMock()
filter_obj.regex_pattern = "skip-me"
filter_obj.filter_type = "name"
filter_obj.exclude = True
filter_obj.custom_properties = {}
compiled = _compile_m3u_stream_filters([filter_obj])
batch = [{
"name": "skip-me channel",
"url": "http://example/live",
"attributes": {"group-title": "News"},
"vlc_opts": {},
}]
with patch("django.db.connections"):
result = process_m3u_batch_direct(
1, batch, {"News": 1}, ["name", "url"], compiled_filters=compiled,
)
self.assertIn("0 created", result)
mock_stream_cls.objects.bulk_create.assert_not_called()
mock_bulk_update.assert_called_once_with([], [], batch_size=200)
@patch("apps.m3u.tasks._bulk_update_stream_refresh_batches")
@patch("apps.m3u.tasks.Stream")
@patch("apps.m3u.tasks.M3UAccount")
def test_no_filters_imports_matching_stream(
self, mock_account_cls, mock_stream_cls, mock_bulk_update,
):
self._mock_stream_meta(mock_stream_cls)
mock_account = MagicMock()
mock_account.account_type = "STD"
mock_account_cls.objects.get.return_value = mock_account
mock_stream_cls.objects.filter.return_value.select_related.return_value.only.return_value = (
[]
)
mock_stream_cls.generate_hash_key = MagicMock(return_value="hash123")
mock_stream_cls.objects.bulk_create.return_value = []
batch = [{
"name": "News One",
"url": "http://example/live",
"attributes": {"group-title": "News"},
"vlc_opts": {},
}]
with patch("django.db.connections"), patch(
"apps.m3u.tasks.transaction.atomic",
):
result = process_m3u_batch_direct(
1, batch, {"News": 1}, ["name", "url"], compiled_filters=[],
)
self.assertIn("1 created", result)
mock_stream_cls.objects.bulk_create.assert_called_once()

150
apps/m3u/tests/test_sync_correctness.py
View file

@ -644,6 +644,34 @@ class NumbersInRangeLookupTests(TestCase):
)
self.assertFalse(occupant["has_channel_number_override"])
def test_group_override_channel_reports_target_group(self):
# When auto-sync routes channels into a different group via
# group_override, the occupant's channel_group_id is the override
# target, not the source group being configured. The frontend relies
# on this to recognize override-routed channels as the config's own
# output (effectiveSyncGroupId), so the warning does not flag them.
account = _make_account()
source = _make_group(name="SourceGrp")
target = _make_group(name="TargetGrp")
Channel.objects.create(
name="Routed",
channel_number=3210,
channel_group=target,
auto_created=True,
auto_created_by=account,
)
client = self._client()
response = client.get(
"/api/channels/channels/numbers-in-range/?start=3210&end=3210"
)
occupant = response.data["occupants"][0]
self.assertEqual(occupant["channel_group_id"], target.id)
self.assertNotEqual(occupant["channel_group_id"], source.id)
self.assertTrue(occupant["auto_created"])
self.assertEqual(occupant["auto_created_by_account_id"], account.id)
def test_manual_channel_exposed_with_auto_created_false(self):
# Manual channels are always a real collision worth surfacing.
# The response must flag them with auto_created=False and a null
@ -735,6 +763,128 @@ class RegexPreviewTests(TestCase):
self.assertEqual(response.data["total_scanned"], 3)
self.assertFalse(response.data["scan_limit_hit"])
def test_find_replace_applies_numbered_capture_group(self):
# The replace field accepts JS-style $1 backreferences, but the regex
# engine expects \1. Without the conversion the preview echoes the
# literal "$1", so the previewed "after" disagrees with the name the
# live rename produces.
account = self._make_account()
group = _make_group(name="Sports")
Stream.objects.create(
name="High Limit Racing at Eagle @ Jun 9 7:00 PM",
url="http://example.com/hlr.m3u8",
m3u_account=account,
channel_group=group,
last_seen=timezone.now(),
)
client = self._client()
response = client.get(
"/api/channels/streams/regex-preview/",
{"channel_group": "Sports", "find": r"(.+) @.*", "replace": "$1"},
)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.data["find_match_count"], 1)
after = response.data["find_matches"][0]["after"]
self.assertEqual(after, "High Limit Racing at Eagle")
self.assertNotIn("$1", after)
def test_preview_after_matches_live_sync_rename(self):
# Guards the defect class: the preview and the live rename are
# separate code paths that must convert the replacement identically,
# so the preview can never promise an output the sync would not yield.
name = "High Limit Racing at Eagle @ Jun 9 7:00 PM"
account = self._make_account()
group = _make_group(name="Racing")
_attach_group_to_account(
account,
group,
custom_properties={
"name_regex_pattern": r"(.+) @.*",
"name_replace_pattern": "$1",
},
)
_make_stream(account, group, name=name, tvg_id="hlr")
result = _sync(account)
self.assertEqual(result.get("status"), "ok")
channel = Channel.objects.get(auto_created=True, auto_created_by=account)
live_name = channel.name
client = self._client()
response = client.get(
"/api/channels/streams/regex-preview/",
{"channel_group": "Racing", "find": r"(.+) @.*", "replace": "$1"},
)
self.assertEqual(response.status_code, 200)
preview_after = response.data["find_matches"][0]["after"]
self.assertEqual(preview_after, live_name)
self.assertEqual(preview_after, "High Limit Racing at Eagle")
def test_regex_engine_pattern_transforms_in_preview(self):
# Both the preview and the live rename use the regex module, which is
# more permissive than stdlib re and matches the JS-style syntax the UI
# authors. A quantified anchor like "^*" (which stdlib re rejects)
# compiles and transforms rather than reporting an error.
account = self._make_account()
group = _make_group(name="Sports")
Stream.objects.create(
name="Doc95",
url="http://example.com/doc95.m3u8",
m3u_account=account,
channel_group=group,
last_seen=timezone.now(),
)
client = self._client()
response = client.get(
"/api/channels/streams/regex-preview/",
{"channel_group": "Sports", "find": "^*", "replace": "$"},
)
self.assertEqual(response.status_code, 200)
self.assertNotIn("find_error", response.data)
self.assertEqual(response.data["find_match_count"], 1)
# ^* matches the empty string at every position, so the literal $
# replacement is inserted between characters.
self.assertEqual(
response.data["find_matches"][0]["after"], "$D$o$c$9$5$"
)
def test_preview_and_sync_agree_on_regex_only_pattern(self):
# Parity guard for the engine alignment: a pattern valid in regex but
# not stdlib re must transform identically in the sync and the preview,
# rather than diverging (the sync no longer silently keeps the
# original name for these patterns).
name = "Doc95"
account = self._make_account()
group = _make_group(name="Docs")
_attach_group_to_account(
account,
group,
custom_properties={
"name_regex_pattern": "^*",
"name_replace_pattern": "$",
},
)
_make_stream(account, group, name=name, tvg_id="doc95")
result = _sync(account)
self.assertEqual(result.get("status"), "ok")
channel = Channel.objects.get(auto_created=True, auto_created_by=account)
live_name = channel.name
self.assertNotEqual(live_name, name)
client = self._client()
response = client.get(
"/api/channels/streams/regex-preview/",
{"channel_group": "Docs", "find": "^*", "replace": "$"},
)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.data["find_matches"][0]["after"], live_name)
def test_filter_returns_matched_names_with_count(self):
account = self._make_account()
group = _make_group(name="Sports")

13
apps/m3u/utils.py
View file

@ -1,4 +1,5 @@
# apps/m3u/utils.py
import regex
import threading
import logging
from django.db import models
@ -9,6 +10,18 @@ active_streams_map = {}
logger = logging.getLogger(__name__)
def convert_js_numbered_backreferences(replacement):
"""Translate JS-style ``$1``/``$2`` backreferences to Python ``\\1``/``\\2``.
Auto-sync replace patterns are authored in JS regex syntax, but Python's
regex engines honor backslash backreferences, not ``$1``. The live rename
and the UI preview must convert identically, so both call this single
helper and cannot drift apart (otherwise the preview promises an output
the sync would never produce).
"""
return regex.sub(r"\$(\d+)", r"\\\1", replacement)
def normalize_stream_url(url):
"""
Normalize stream URLs for compatibility with FFmpeg.

1732
apps/output/epg.py Normal file
View file

File diff suppressed because it is too large Load diff

239
apps/output/streaming_chunk_cache.py Normal file
View file

@ -0,0 +1,239 @@
"""Single-flight Redis chunk cache for large streaming HTTP responses."""
import logging
import time
from django.http import StreamingHttpResponse
logger = logging.getLogger(__name__)
STATUS_BUILDING = "building"
STATUS_READY = "ready"
STATUS_ERROR = "error"
DEFAULT_CACHE_TTL = 300
DEFAULT_LOCK_TTL = 120
DEFAULT_POLL_INTERVAL = 0.05
DEFAULT_MAX_FOLLOWER_WAIT = 600
def _chunks_key(base_key):
return f"{base_key}:chunks"
def _ready_key(base_key):
return f"{base_key}:ready"
def _status_key(base_key):
return f"{base_key}:status"
def _lock_key(base_key):
return f"{base_key}:lock"
def _decode_chunk(chunk):
if chunk is None:
return None
if isinstance(chunk, bytes):
return chunk.decode("utf-8")
return chunk
def _encode_chunk(chunk):
if isinstance(chunk, bytes):
return chunk
return chunk.encode("utf-8")
def _poll_wait(interval):
try:
from core.utils import _is_gevent_monkey_patched
if _is_gevent_monkey_patched():
import gevent
gevent.sleep(interval)
return
except ImportError:
pass
time.sleep(interval)
def _get_redis():
from django_redis import get_redis_connection
return get_redis_connection("default")
def _get_status(redis, base_key):
raw = redis.get(_status_key(base_key))
if raw is None:
return None
return _decode_chunk(raw)
def _clear_build_keys(redis, base_key):
redis.delete(
_chunks_key(base_key),
_status_key(base_key),
_ready_key(base_key),
_lock_key(base_key),
)
def _try_acquire_lock(redis, base_key, lock_ttl):
return bool(redis.set(_lock_key(base_key), "1", nx=True, ex=lock_ttl))
def _refresh_build_ttl(redis, base_key, lock_ttl):
redis.expire(_lock_key(base_key), lock_ttl)
redis.expire(_status_key(base_key), lock_ttl)
redis.expire(_chunks_key(base_key), lock_ttl)
def _stream_ready(redis, base_key):
offset = 0
chunks_key = _chunks_key(base_key)
while True:
chunk = redis.lindex(chunks_key, offset)
if chunk is None:
break
yield _decode_chunk(chunk)
offset += 1
def _stream_build(redis, base_key, source, cache_ttl, lock_ttl):
"""Leader: stream to client and append each chunk to Redis."""
chunks_key = _chunks_key(base_key)
status_key = _status_key(base_key)
try:
from django.core.cache import cache as django_cache
django_cache.delete(base_key) # clear any non-chunked entry under this key
redis.delete(chunks_key, _ready_key(base_key))
redis.set(status_key, STATUS_BUILDING, ex=lock_ttl)
refresh_interval = max(1, lock_ttl // 4)
last_refresh = 0.0
for chunk in source():
redis.rpush(chunks_key, _encode_chunk(chunk))
now = time.monotonic()
if now - last_refresh >= refresh_interval:
_refresh_build_ttl(redis, base_key, lock_ttl)
last_refresh = now
yield chunk
redis.set(status_key, STATUS_READY)
redis.set(_ready_key(base_key), "1")
redis.expire(chunks_key, cache_ttl)
redis.expire(status_key, cache_ttl)
redis.expire(_ready_key(base_key), cache_ttl)
logger.debug("Cached response in %s chunks", redis.llen(chunks_key))
except Exception:
logger.exception("Chunk cache build failed for %s", base_key)
redis.delete(chunks_key)
redis.set(status_key, STATUS_ERROR, ex=60)
raise
finally:
redis.delete(_lock_key(base_key))
def _stream_follow(redis, base_key, source, cache_ttl, lock_ttl, poll_interval, max_follower_wait):
"""Follower: read chunks as the leader writes them."""
offset = 0
deadline = time.monotonic() + max_follower_wait
idle_polls = 0
chunks_key = _chunks_key(base_key)
lock_key = _lock_key(base_key)
while True:
chunk = redis.lindex(chunks_key, offset)
if chunk is not None:
idle_polls = 0
yield _decode_chunk(chunk)
offset += 1
continue
status = _get_status(redis, base_key)
if status == STATUS_READY:
break
if status == STATUS_ERROR:
_clear_build_keys(redis, base_key)
if offset == 0 and _try_acquire_lock(redis, base_key, lock_ttl):
yield from _stream_build(redis, base_key, source, cache_ttl, lock_ttl)
return
raise RuntimeError("Chunk cache build failed")
if time.monotonic() >= deadline:
if offset == 0 and _try_acquire_lock(redis, base_key, lock_ttl):
logger.warning("Chunk cache follower timed out; rebuilding %s", base_key)
yield from _stream_build(redis, base_key, source, cache_ttl, lock_ttl)
return
logger.warning("Chunk cache follower timed out after partial read for %s", base_key)
break
lock_active = bool(redis.exists(lock_key))
if status != STATUS_BUILDING and not lock_active:
idle_polls += 1
if offset == 0 and idle_polls >= max(1, int(1.0 / poll_interval)):
if _try_acquire_lock(redis, base_key, lock_ttl):
logger.warning("Chunk cache leader lost; rebuilding %s", base_key)
yield from _stream_build(redis, base_key, source, cache_ttl, lock_ttl)
return
else:
idle_polls = 0
_poll_wait(poll_interval)
def stream_cached_response(
cache_key,
source,
*,
content_type="application/xml",
filename=None,
cache_ttl=DEFAULT_CACHE_TTL,
lock_ttl=DEFAULT_LOCK_TTL,
poll_interval=DEFAULT_POLL_INTERVAL,
max_follower_wait=DEFAULT_MAX_FOLLOWER_WAIT,
redis=None,
):
"""
Stream a large response with single-flight Redis chunk caching.
``source`` must be a callable returning a chunk iterator. Only the leader
invokes it; concurrent followers replay chunks already written to Redis, so
the expensive ``source`` runs at most once per ``cache_key``.
"""
if redis is None:
redis = _get_redis()
if redis.get(_ready_key(cache_key)):
logger.debug("Serving response from chunk cache")
stream = _stream_ready(redis, cache_key)
else:
status = _get_status(redis, cache_key)
if status == STATUS_ERROR:
_clear_build_keys(redis, cache_key)
if _try_acquire_lock(redis, cache_key, lock_ttl):
logger.debug("Building response (cache leader)")
stream = _stream_build(redis, cache_key, source, cache_ttl, lock_ttl)
else:
logger.debug("Following in-flight cache build")
stream = _stream_follow(
redis,
cache_key,
source,
cache_ttl,
lock_ttl,
poll_interval,
max_follower_wait,
)
response = StreamingHttpResponse(stream, content_type=content_type)
if filename:
response["Content-Disposition"] = f'attachment; filename="{filename}"'
response["Cache-Control"] = "no-cache"
return response

187
apps/output/test_streaming_chunk_cache.py Normal file
View file

@ -0,0 +1,187 @@
import threading
import time
from unittest import TestCase
from apps.output.streaming_chunk_cache import (
STATUS_BUILDING,
STATUS_READY,
_chunks_key,
_lock_key,
_ready_key,
_status_key,
stream_cached_response,
)
class FakeRedis:
"""Minimal Redis stand-in for chunk-cache unit tests."""
def __init__(self):
self._strings = {}
self._lists = {}
self._expires_at = {}
def _purge_expired(self):
now = time.monotonic()
expired = [key for key, deadline in self._expires_at.items() if deadline <= now]
for key in expired:
self._strings.pop(key, None)
self._lists.pop(key, None)
self._expires_at.pop(key, None)
def get(self, key):
self._purge_expired()
return self._strings.get(key)
def set(self, key, value, nx=False, ex=None):
self._purge_expired()
if nx and key in self._strings:
return None
self._strings[key] = value
if ex is not None:
self._expires_at[key] = time.monotonic() + ex
return True
def delete(self, *keys):
for key in keys:
self._strings.pop(key, None)
self._lists.pop(key, None)
self._expires_at.pop(key, None)
def exists(self, key):
self._purge_expired()
return key in self._strings or key in self._lists
def expire(self, key, ttl):
if key in self._strings or key in self._lists:
self._expires_at[key] = time.monotonic() + ttl
return True
def rpush(self, key, value):
self._lists.setdefault(key, []).append(value)
def lindex(self, key, offset):
items = self._lists.get(key, [])
if offset < len(items):
return items[offset]
return None
def llen(self, key):
return len(self._lists.get(key, []))
def _consume(response):
return b"".join(response.streaming_content).decode("utf-8")
class StreamingChunkCacheTests(TestCase):
def test_leader_caches_chunks_and_sets_ready(self):
redis = FakeRedis()
calls = []
def source():
calls.append(1)
yield "<tv>"
yield "</tv>"
body = _consume(stream_cached_response("cache:test", source, redis=redis))
self.assertEqual(body, "<tv></tv>")
self.assertEqual(calls, [1])
self.assertEqual(redis.get(_ready_key("cache:test")), "1")
self.assertEqual(redis.get(_status_key("cache:test")), STATUS_READY)
self.assertEqual(redis.llen(_chunks_key("cache:test")), 2)
self.assertFalse(redis.exists(_lock_key("cache:test")))
def test_cache_hit_skips_source(self):
redis = FakeRedis()
calls = []
def source():
calls.append(1)
yield "<tv>"
yield "</tv>"
_consume(stream_cached_response("cache:test", source, redis=redis))
calls.clear()
body = _consume(stream_cached_response("cache:test", source, redis=redis))
self.assertEqual(body, "<tv></tv>")
self.assertEqual(calls, [])
def test_follower_reads_leader_chunks_without_rebuilding(self):
redis = FakeRedis()
base = "cache:follow"
leader_started = threading.Event()
rebuild_calls = []
def slow_source():
rebuild_calls.append(1)
leader_started.set()
yield "a"
time.sleep(0.05)
yield "b"
def forbidden_source():
rebuild_calls.append(2)
yield "SHOULD_NOT_RUN"
def leader():
_consume(
stream_cached_response(
base,
slow_source,
redis=redis,
poll_interval=0.01,
)
)
leader_thread = threading.Thread(target=leader)
leader_thread.start()
leader_started.wait(timeout=5)
follower_body = _consume(
stream_cached_response(
base,
forbidden_source,
redis=redis,
poll_interval=0.01,
)
)
leader_thread.join(timeout=5)
self.assertEqual(follower_body, "ab")
self.assertEqual(rebuild_calls, [1])
def test_only_one_leader_when_two_clients_start_together(self):
redis = FakeRedis()
build_calls = []
barrier = threading.Barrier(2)
results = {}
def source():
build_calls.append(threading.current_thread().name)
yield "x"
def worker():
barrier.wait()
results[threading.current_thread().name] = _consume(
stream_cached_response(
"cache:race",
source,
redis=redis,
poll_interval=0.01,
)
)
threads = [
threading.Thread(target=worker, name="t1"),
threading.Thread(target=worker, name="t2"),
]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=10)
self.assertEqual(results["t1"], "x")
self.assertEqual(results["t2"], "x")
self.assertEqual(len(build_calls), 1)

854
apps/output/tests.py
View file

@ -1,31 +1,131 @@
from django.test import TestCase, Client
from django.test import TestCase, Client, SimpleTestCase, RequestFactory
from django.urls import reverse
from apps.channels.models import Channel, ChannelGroup
from unittest import skipUnless
from unittest.mock import patch
from uuid import uuid4
from django.db import connection
from django.test.utils import CaptureQueriesContext
from apps.channels.models import Channel, ChannelGroup, ChannelProfile, ChannelProfileMembership
from apps.epg.models import EPGData, EPGSource
from apps.accounts.models import User
from apps.m3u.models import M3UAccount
from apps.output.views import xc_get_series, xc_get_vod_streams
from apps.vod.models import (
M3UMovieRelation,
M3USeriesRelation,
Movie,
Series,
VODCategory,
VODLogo,
)
import xml.etree.ElementTree as ET
from datetime import timedelta
def _response_text(response):
"""Read body from HttpResponse or StreamingHttpResponse."""
if getattr(response, "streaming", False):
return b"".join(response.streaming_content).decode()
return response.content.decode()
def _epg_response_without_redis(cache_key, source, **kwargs):
"""Test helper: stream EPG directly without Redis chunk caching."""
from django.http import StreamingHttpResponse
response = StreamingHttpResponse(source(), content_type="application/xml")
response["Content-Disposition"] = 'attachment; filename="Dispatcharr.xml"'
response["Cache-Control"] = "no-cache"
return response
class OutputEndpointTestMixin:
"""Isolate HTTP endpoint tests from network ACL, logging, DB teardown, and Redis."""
class OutputM3UTest(TestCase):
def setUp(self):
super().setUp()
self._network_patch = patch(
"apps.output.views.network_access_allowed",
return_value=True,
)
self._epg_teardown_patch = patch("apps.output.epg._epg_export_teardown")
self._log_event_patch = patch("apps.output.views.log_system_event")
self._epg_log_event_patch = patch("apps.output.epg.log_system_event")
self._close_db_patch = patch("django.db.close_old_connections")
self._epg_cache_patch = patch(
"apps.output.epg.stream_cached_response",
side_effect=_epg_response_without_redis,
)
self._network_patch.start()
self._epg_teardown_patch.start()
self._log_event_patch.start()
self._epg_log_event_patch.start()
self._close_db_patch.start()
self._epg_cache_patch.start()
def tearDown(self):
from django.core.cache import cache
cache.clear()
self._epg_cache_patch.stop()
self._close_db_patch.stop()
self._epg_log_event_patch.stop()
self._log_event_patch.stop()
self._epg_teardown_patch.stop()
self._network_patch.stop()
super().tearDown()
def _create_isolated_profile(self, prefix):
"""New profiles auto-include every channel via signal; clear that for tests."""
profile = ChannelProfile.objects.create(name=f"{prefix}-{uuid4().hex[:8]}")
ChannelProfileMembership.objects.filter(channel_profile=profile).delete()
return profile
def _add_channel_to_profile(self, profile, group, **kwargs):
channel = Channel.objects.create(channel_group=group, **kwargs)
ChannelProfileMembership.objects.create(
channel_profile=profile,
channel=channel,
enabled=True,
)
return channel
class OutputM3UTest(OutputEndpointTestMixin, TestCase):
def setUp(self):
super().setUp()
self.client = Client()
self.group = ChannelGroup.objects.create(name=f"M3U Group {uuid4().hex[:8]}")
self.profile = self._create_isolated_profile("m3u")
self._add_channel_to_profile(
self.profile,
self.group,
channel_number=1.0,
name="Test M3U Channel",
)
def _m3u_url(self):
return reverse("output:m3u_endpoint", kwargs={"profile_name": self.profile.name})
def test_generate_m3u_response(self):
"""
Test that the M3U endpoint returns a valid M3U file.
"""
url = reverse('output:generate_m3u')
response = self.client.get(url)
response = self.client.get(self._m3u_url())
self.assertEqual(response.status_code, 200)
content = response.content.decode()
content = _response_text(response)
self.assertIn("#EXTM3U", content)
def test_generate_m3u_response_post_empty_body(self):
"""
Test that a POST request with an empty body returns 200 OK.
"""
url = reverse('output:generate_m3u')
response = self.client.post(url, data=None, content_type='application/x-www-form-urlencoded')
content = response.content.decode()
response = self.client.post(
self._m3u_url(),
data=None,
content_type="application/x-www-form-urlencoded",
)
content = _response_text(response)
self.assertEqual(response.status_code, 200, "POST with empty body should return 200 OK")
self.assertIn("#EXTM3U", content)
@ -34,35 +134,40 @@ class OutputM3UTest(TestCase):
"""
Test that a POST request with a non-empty body returns 403 Forbidden.
"""
url = reverse('output:generate_m3u')
response = self.client.post(url, data={'evilstring': 'muhahaha'})
response = self.client.post(self._m3u_url(), data={"evilstring": "muhahaha"})
self.assertEqual(response.status_code, 403, "POST with body should return 403 Forbidden")
self.assertIn("POST requests with body are not allowed, body is:", response.content.decode())
self.assertIn("POST requests with body are not allowed", _response_text(response))
class OutputEPGXMLEscapingTest(TestCase):
class OutputEPGXMLEscapingTest(OutputEndpointTestMixin, TestCase):
"""Test XML escaping of channel_id attributes in EPG generation"""
def setUp(self):
super().setUp()
self.client = Client()
self.group = ChannelGroup.objects.create(name="Test Group")
self.group = ChannelGroup.objects.create(name=f"Test Group {uuid4().hex[:8]}")
self.profile = self._create_isolated_profile("epg-xml")
def _add_channel(self, **kwargs):
return self._add_channel_to_profile(self.profile, self.group, **kwargs)
def _epg_url(self, query="tvg_id_source=tvg_id&days=0&prev_days=0"):
base = reverse("output:epg_endpoint", kwargs={"profile_name": self.profile.name})
return f"{base}?{query}"
def test_channel_id_with_ampersand(self):
"""Test channel ID with ampersand is properly escaped"""
channel = Channel.objects.create(
self._add_channel(
channel_number=1.0,
name="Test Channel",
tvg_id="News & Sports",
channel_group=self.group
)
url = reverse('output:generate_epg') + '?tvg_id_source=tvg_id'
response = self.client.get(url)
response = self.client.get(self._epg_url())
self.assertEqual(response.status_code, 200)
content = response.content.decode()
content = _response_text(response)
# Should contain escaped ampersand
self.assertIn('id="News &amp; Sports"', content)
@ -76,17 +181,15 @@ class OutputEPGXMLEscapingTest(TestCase):
def test_channel_id_with_angle_brackets(self):
"""Test channel ID with < and > characters"""
channel = Channel.objects.create(
self._add_channel(
channel_number=2.0,
name="HD Channel",
tvg_id="Channel <HD>",
channel_group=self.group
)
url = reverse('output:generate_epg') + '?tvg_id_source=tvg_id'
response = self.client.get(url)
response = self.client.get(self._epg_url())
content = response.content.decode()
content = _response_text(response)
self.assertIn('id="Channel &lt;HD&gt;"', content)
try:
@ -96,23 +199,28 @@ class OutputEPGXMLEscapingTest(TestCase):
def test_channel_id_with_all_special_chars(self):
"""Test channel ID with all XML special characters"""
channel = Channel.objects.create(
expected_id = 'Test & "Special" <Chars>'
self._add_channel(
channel_number=3.0,
name="Complex Channel",
tvg_id='Test & "Special" <Chars>',
channel_group=self.group
tvg_id=expected_id,
)
url = reverse('output:generate_epg') + '?tvg_id_source=tvg_id'
response = self.client.get(url)
response = self.client.get(self._epg_url())
content = response.content.decode()
content = _response_text(response)
self.assertIn('id="Test &amp; &quot;Special&quot; &lt;Chars&gt;"', content)
try:
tree = ET.fromstring(content)
# Verify we can find the channel with correct ID in parsed tree
channel_elem = tree.find('.//channel[@id="Test & \\"Special\\" <Chars>"]')
channel_elem = next(
(
elem
for elem in tree.findall(".//channel")
if elem.get("id") == expected_id
),
None,
)
self.assertIsNotNone(channel_elem)
except ET.ParseError as e:
self.fail(f"Generated EPG with all special chars is not valid XML: {e}")
@ -121,25 +229,689 @@ class OutputEPGXMLEscapingTest(TestCase):
"""Test that programme elements also have escaped channel attributes"""
epg_source = EPGSource.objects.create(name="Test EPG", source_type="dummy")
epg_data = EPGData.objects.create(name="Test EPG Data", epg_source=epg_source)
channel = Channel.objects.create(
self._add_channel(
channel_number=4.0,
name="Program Test",
tvg_id="News & Sports",
epg_data=epg_data,
channel_group=self.group
)
url = reverse('output:generate_epg') + '?tvg_id_source=tvg_id'
response = self.client.get(url)
response = self.client.get(self._epg_url())
content = response.content.decode()
content = _response_text(response)
# Check programme elements have escaped channel attributes
self.assertIn('channel="News &amp; Sports"', content)
try:
tree = ET.fromstring(content)
programmes = tree.findall('.//programme[@channel="News & Sports"]')
programmes = [
programme
for programme in tree.findall(".//programme")
if programme.get("channel") == "News & Sports"
]
self.assertGreater(len(programmes), 0)
except ET.ParseError as e:
self.fail(f"Generated EPG with programme elements is not valid XML: {e}")
def test_programmes_emitted_in_start_time_order(self):
"""Programmes for a channel are emitted in start_time order, not insert order."""
from django.utils import timezone
from apps.epg.models import ProgramData
epg_source = EPGSource.objects.create(name="Real EPG", source_type="xmltv")
epg_data = EPGData.objects.create(name="Station", epg_source=epg_source, tvg_id="station1")
self._add_channel(
channel_number=149.0,
name="Food Network",
tvg_id="station1",
epg_data=epg_data,
)
now = timezone.now()
# Insert out of chronological order so id order != start_time order.
ProgramData.objects.create(
epg=epg_data,
start_time=now + timedelta(days=3),
end_time=now + timedelta(days=3, hours=1),
title="Third",
tvg_id="station1",
)
ProgramData.objects.create(
epg=epg_data,
start_time=now + timedelta(days=1),
end_time=now + timedelta(days=1, hours=1),
title="First",
tvg_id="station1",
)
ProgramData.objects.create(
epg=epg_data,
start_time=now + timedelta(days=2),
end_time=now + timedelta(days=2, hours=1),
title="Second",
tvg_id="station1",
)
content = _response_text(self.client.get(self._epg_url("tvg_id_source=tvg_id&days=7")))
self.assertLess(content.find('<title>First</title>'), content.find('<title>Second</title>'))
self.assertLess(content.find('<title>Second</title>'), content.find('<title>Third</title>'))
class OutputEPGCustomDummyTest(TestCase):
"""Custom dummy EPG must not fall back to default when pattern matched but event is outside window."""
def setUp(self):
self.group = ChannelGroup.objects.create(name="Sports Group")
def test_custom_dummy_outside_window_fills_with_ended_programmes(self):
from django.utils import timezone
from apps.output.views import generate_dummy_programs
epg_source = EPGSource.objects.create(
name="NHL Dummy",
source_type="dummy",
custom_properties={
"title_pattern": r"(?<league>.*)\s\d+:\s(?<team1>.*?)(?:\s+vs\s+)(?<team2>.*?)\s*@.*",
"time_pattern": r"(?<hour>\d{1,2}):(?<minute>\d{2})\s*(?<ampm>AM|PM)",
"date_pattern": r"@ (?<month>[A-Za-z]+)\s+(?<day>\d{1,2})",
"timezone": "US/Eastern",
"program_duration": 180,
},
)
channel_name = (
"NHL 01: Washington Capitals vs Philadelphia Flyers @ April 16 07:30 PM ET"
)
now = timezone.now()
lookback = now - timedelta(days=7)
programs = generate_dummy_programs(
channel_id="nhl01",
channel_name=channel_name,
num_days=7,
epg_source=epg_source,
export_lookback=lookback,
export_cutoff=now + timedelta(days=7),
)
self.assertGreater(len(programs), 0)
self.assertTrue(
all(p['end_time'] >= lookback for p in programs),
"All programmes should fall inside the export window",
)
self.assertTrue(
any('Ended' in p['description'] for p in programs),
"Past events outside the window should still show ended filler",
)
for program in programs:
start = program['start_time']
self.assertEqual(start.second, 0)
self.assertEqual(start.microsecond, 0)
self.assertIn(
start.minute, (0, 30),
"Filler programmes should start on half-hour boundaries",
)
self.assertGreaterEqual(programs[0]['start_time'], lookback)
def test_custom_dummy_future_event_fills_grid_window_with_upcoming(self):
"""Grid-style window: future event should show upcoming filler, not empty."""
from django.utils import timezone
from apps.output.epg import _programme_overlaps_export_window, generate_dummy_programs
epg_source = EPGSource.objects.create(
name="NHL Dummy Future",
source_type="dummy",
custom_properties={
"title_pattern": r"(?<league>.*)\s\d+:\s(?<team1>.*?)(?:\s+vs\s+)(?<team2>.*?)\s*@.*",
"time_pattern": r"(?<hour>\d{1,2}):(?<minute>\d{2})\s*(?<ampm>AM|PM)",
"date_pattern": r"@ (?<month>[A-Za-z]+)\s+(?<day>\d{1,2})",
"timezone": "US/Eastern",
"program_duration": 180,
},
)
now = timezone.now()
grid_start = now - timedelta(hours=1)
grid_end = now + timedelta(hours=24)
future = now + timedelta(days=3)
channel_name = (
f"NHL 01: Washington Capitals vs Philadelphia Flyers @ "
f"{future.strftime('%B')} {future.day} 07:30 PM ET"
)
programs = generate_dummy_programs(
channel_id="nhl01",
channel_name=channel_name,
num_days=1,
epg_source=epg_source,
export_lookback=grid_start,
export_cutoff=grid_end,
)
self.assertGreater(len(programs), 0)
self.assertTrue(
all(
_programme_overlaps_export_window(
p["start_time"], p["end_time"], grid_start, grid_end
)
for p in programs
),
"All programmes should overlap the grid query window",
)
self.assertTrue(
any("Upcoming" in p.get("description", "") for p in programs),
"Future events outside the window should show upcoming filler",
)
class OutputEPGHelperTest(SimpleTestCase):
def test_ceil_to_half_hour_on_boundary(self):
from django.utils import timezone
from apps.output.epg import _ceil_to_half_hour
dt = timezone.now().replace(minute=30, second=0, microsecond=0)
self.assertEqual(_ceil_to_half_hour(dt), dt)
def test_ceil_to_half_hour_rounds_up(self):
from django.utils import timezone
from apps.output.epg import _ceil_to_half_hour
dt = timezone.now().replace(minute=17, second=42, microsecond=123456)
aligned = _ceil_to_half_hour(dt)
self.assertEqual(aligned.minute, 30)
self.assertEqual(aligned.second, 0)
self.assertGreaterEqual(aligned, dt.replace(microsecond=0))
def test_ceil_to_half_hour_past_boundary_second(self):
from django.utils import timezone
from apps.output.epg import _ceil_to_half_hour
dt = timezone.now().replace(minute=0, second=52, microsecond=123456)
aligned = _ceil_to_half_hour(dt)
self.assertEqual(aligned.minute, 30)
self.assertEqual(aligned.second, 0)
self.assertGreaterEqual(aligned, dt.replace(microsecond=0))
class XcVodSeriesDistinctTests(TestCase):
def setUp(self):
self.factory = RequestFactory()
self.user = User.objects.create_user(
username=f"xc-{uuid4().hex[:8]}",
password="pass",
custom_properties={"xc_password": "xcpass"},
)
self.request = self.factory.get("/player_api.php")
def _account(self, name, *, priority=0, is_active=True):
return M3UAccount.objects.create(
name=name,
server_url="http://example.com",
priority=priority,
is_active=is_active,
)
def test_vod_streams_picks_highest_priority_relation(self):
low = self._account(f"low-{uuid4().hex[:6]}", priority=1)
high = self._account(f"high-{uuid4().hex[:6]}", priority=10)
movie = Movie.objects.create(name="Shared Movie", year=2020)
M3UMovieRelation.objects.create(
m3u_account=low,
movie=movie,
stream_id="low-stream",
container_extension="mkv",
)
M3UMovieRelation.objects.create(
m3u_account=high,
movie=movie,
stream_id="high-stream",
container_extension="mp4",
)
streams = xc_get_vod_streams(self.request, self.user)
self.assertEqual(len(streams), 1)
self.assertEqual(streams[0]["name"], "Shared Movie")
self.assertEqual(streams[0]["container_extension"], "mp4")
def test_vod_streams_excludes_inactive_accounts(self):
active = self._account(f"active-{uuid4().hex[:6]}", priority=1)
inactive = self._account(
f"inactive-{uuid4().hex[:6]}", priority=99, is_active=False
)
active_movie = Movie.objects.create(name="Active Movie")
inactive_movie = Movie.objects.create(name="Inactive Only Movie")
M3UMovieRelation.objects.create(
m3u_account=active,
movie=active_movie,
stream_id="active-1",
)
M3UMovieRelation.objects.create(
m3u_account=inactive,
movie=inactive_movie,
stream_id="inactive-1",
)
streams = xc_get_vod_streams(self.request, self.user)
names = {s["name"] for s in streams}
self.assertEqual(names, {"Active Movie"})
def test_vod_streams_category_filter(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
action = VODCategory.objects.create(name="Action", category_type="movie")
comedy = VODCategory.objects.create(name="Comedy", category_type="movie")
action_movie = Movie.objects.create(name="Action Movie")
comedy_movie = Movie.objects.create(name="Comedy Movie")
M3UMovieRelation.objects.create(
m3u_account=account,
movie=action_movie,
category=action,
stream_id="action-1",
)
M3UMovieRelation.objects.create(
m3u_account=account,
movie=comedy_movie,
category=comedy,
stream_id="comedy-1",
)
streams = xc_get_vod_streams(self.request, self.user, category_id=action.id)
self.assertEqual(len(streams), 1)
self.assertEqual(streams[0]["name"], "Action Movie")
self.assertEqual(streams[0]["category_id"], str(action.id))
def test_vod_streams_sorted_alphabetically_by_name(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
zebra = Movie.objects.create(name="Zebra Film")
apple = Movie.objects.create(name="Apple Film")
M3UMovieRelation.objects.create(
m3u_account=account, movie=zebra, stream_id="z-1"
)
M3UMovieRelation.objects.create(
m3u_account=account, movie=apple, stream_id="a-1"
)
streams = xc_get_vod_streams(self.request, self.user)
self.assertEqual([s["name"] for s in streams], ["Apple Film", "Zebra Film"])
def test_vod_streams_includes_metadata_fields(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
movie = Movie.objects.create(
name="Rich Movie",
description="A plot",
genre="Drama",
year=2021,
rating="8",
custom_properties={
"director": "Dir",
"actors": "Cast",
"release_date": "2021-01-01",
"youtube_trailer": "yt123",
},
)
M3UMovieRelation.objects.create(
m3u_account=account,
movie=movie,
stream_id="rich-1",
container_extension="avi",
)
stream = xc_get_vod_streams(self.request, self.user)[0]
self.assertEqual(stream["plot"], "A plot")
self.assertEqual(stream["genre"], "Drama")
self.assertEqual(stream["year"], 2021)
self.assertEqual(stream["director"], "Dir")
self.assertEqual(stream["cast"], "Cast")
self.assertEqual(stream["release_date"], "2021-01-01")
self.assertEqual(stream["trailer"], "yt123")
self.assertEqual(stream["container_extension"], "avi")
def test_vod_streams_stream_icon_uses_logo_id_without_logo_join(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
logo = VODLogo.objects.create(name="Poster", url="http://example.com/poster.png")
movie = Movie.objects.create(name="Logo Movie", logo=logo)
M3UMovieRelation.objects.create(
m3u_account=account,
movie=movie,
stream_id="logo-1",
)
stream = xc_get_vod_streams(self.request, self.user)[0]
self.assertIn(f"/{logo.id}/", stream["stream_icon"])
def test_series_picks_highest_priority_relation(self):
low = self._account(f"low-{uuid4().hex[:6]}", priority=1)
high = self._account(f"high-{uuid4().hex[:6]}", priority=10)
series = Series.objects.create(name="Shared Series", year=2019)
M3USeriesRelation.objects.create(
m3u_account=low,
series=series,
external_series_id="low-series",
)
high_rel = M3USeriesRelation.objects.create(
m3u_account=high,
series=series,
external_series_id="high-series",
)
results = xc_get_series(self.request, self.user)
self.assertEqual(len(results), 1)
self.assertEqual(results[0]["name"], "Shared Series")
self.assertEqual(results[0]["series_id"], high_rel.id)
def test_series_excludes_inactive_accounts(self):
active = self._account(f"active-{uuid4().hex[:6]}")
inactive = self._account(f"inactive-{uuid4().hex[:6]}", is_active=False)
active_series = Series.objects.create(name="Active Series")
inactive_series = Series.objects.create(name="Inactive Only Series")
M3USeriesRelation.objects.create(
m3u_account=active,
series=active_series,
external_series_id="active-s",
)
M3USeriesRelation.objects.create(
m3u_account=inactive,
series=inactive_series,
external_series_id="inactive-s",
)
results = xc_get_series(self.request, self.user)
self.assertEqual({r["name"] for r in results}, {"Active Series"})
def test_series_sorted_alphabetically_by_name(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
z = Series.objects.create(name="Zulu Show")
a = Series.objects.create(name="Alpha Show")
M3USeriesRelation.objects.create(
m3u_account=account, series=z, external_series_id="z"
)
M3USeriesRelation.objects.create(
m3u_account=account, series=a, external_series_id="a"
)
results = xc_get_series(self.request, self.user)
self.assertEqual([r["name"] for r in results], ["Alpha Show", "Zulu Show"])
@skipUnless(connection.vendor == "postgresql", "PostgreSQL-specific query shape")
def test_vod_streams_dedupe_query_avoids_movie_join(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
movie = Movie.objects.create(name="Query Shape Movie")
M3UMovieRelation.objects.create(
m3u_account=account, movie=movie, stream_id="qs-1"
)
with CaptureQueriesContext(connection) as ctx:
xc_get_vod_streams(self.request, self.user)
distinct_queries = [q for q in ctx.captured_queries if "DISTINCT" in q["sql"]]
self.assertEqual(len(distinct_queries), 1)
self.assertNotIn('"vod_movie"', distinct_queries[0]["sql"])
self.assertNotIn('"vod_vodlogo"', distinct_queries[0]["sql"])
fetch_queries = [
q
for q in ctx.captured_queries
if '"vod_movie"' in q["sql"] and "DISTINCT" not in q["sql"]
]
self.assertGreaterEqual(len(fetch_queries), 1)
fetch_sql = fetch_queries[0]["sql"]
self.assertNotIn('"vod_vodlogo"', fetch_sql)
self.assertNotIn('"vod_vodcategory"', fetch_sql)
@skipUnless(connection.vendor == "postgresql", "PostgreSQL-specific query shape")
def test_series_dedupe_query_avoids_series_join(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
series = Series.objects.create(name="Query Shape Series")
M3USeriesRelation.objects.create(
m3u_account=account, series=series, external_series_id="qs-s"
)
with CaptureQueriesContext(connection) as ctx:
xc_get_series(self.request, self.user)
distinct_queries = [q for q in ctx.captured_queries if "DISTINCT" in q["sql"]]
self.assertEqual(len(distinct_queries), 1)
self.assertNotIn('"vod_series"', distinct_queries[0]["sql"])
fetch_queries = [
q
for q in ctx.captured_queries
if '"vod_series"' in q["sql"] and "DISTINCT" not in q["sql"]
]
self.assertGreaterEqual(len(fetch_queries), 1)
fetch_sql = fetch_queries[0]["sql"]
self.assertNotIn('"vod_vodlogo"', fetch_sql)
self.assertNotIn('"vod_vodcategory"', fetch_sql)
XC_VOD_STREAM_KEYS = frozenset({
"num", "name", "stream_type", "stream_id", "stream_icon", "rating",
"rating_5based", "added", "is_adult", "tmdb_id", "imdb_id", "trailer",
"plot", "genre", "year", "director", "cast", "release_date", "category_id",
"category_ids", "container_extension", "custom_sid", "direct_source",
})
XC_SERIES_KEYS = frozenset({
"num", "name", "series_id", "cover", "plot", "cast", "director", "genre",
"release_date", "releaseDate", "last_modified", "rating", "rating_5based",
"backdrop_path", "youtube_trailer", "episode_run_time", "category_id",
"category_ids", "tmdb_id", "imdb_id",
})
class XcVodSeriesRegressionTests(TestCase):
"""Full output-shape and edge-case regressions for XC list endpoints."""
def setUp(self):
self.factory = RequestFactory()
self.user = User.objects.create_user(
username=f"xc-reg-{uuid4().hex[:8]}",
password="pass",
custom_properties={"xc_password": "xcpass"},
)
self.request = self.factory.get("/player_api.php")
def _account(self, name, *, priority=0):
return M3UAccount.objects.create(
name=name,
server_url="http://example.com",
priority=priority,
)
def test_vod_streams_empty_library(self):
self.assertEqual(xc_get_vod_streams(self.request, self.user), [])
def test_series_empty_library(self):
self.assertEqual(xc_get_series(self.request, self.user), [])
def test_vod_streams_response_keys(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
movie = Movie.objects.create(name="Schema Movie", rating="10")
M3UMovieRelation.objects.create(
m3u_account=account, movie=movie, stream_id="schema-1"
)
stream = xc_get_vod_streams(self.request, self.user)[0]
self.assertEqual(set(stream.keys()), XC_VOD_STREAM_KEYS)
self.assertEqual(stream["stream_type"], "movie")
self.assertEqual(stream["stream_id"], movie.id)
self.assertEqual(stream["rating_5based"], 5.0)
self.assertEqual(stream["custom_sid"], None)
self.assertEqual(stream["direct_source"], "")
def test_vod_streams_null_optional_fields(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
movie = Movie.objects.create(name="Sparse Movie")
M3UMovieRelation.objects.create(
m3u_account=account,
movie=movie,
stream_id="sparse-1",
container_extension=None,
)
stream = xc_get_vod_streams(self.request, self.user)[0]
self.assertIsNone(stream["stream_icon"])
self.assertEqual(stream["category_id"], "0")
self.assertEqual(stream["category_ids"], [])
self.assertEqual(stream["container_extension"], "mp4")
self.assertEqual(stream["plot"], "")
self.assertEqual(stream["trailer"], "")
self.assertEqual(stream["tmdb_id"], "")
self.assertEqual(stream["imdb_id"], "")
def test_vod_streams_category_from_winning_relation(self):
"""Category must come from the highest-priority relation, not any relation."""
low = self._account(f"low-{uuid4().hex[:6]}", priority=1)
high = self._account(f"high-{uuid4().hex[:6]}", priority=10)
action = VODCategory.objects.create(name="Action", category_type="movie")
comedy = VODCategory.objects.create(name="Comedy", category_type="movie")
movie = Movie.objects.create(name="Dual Category Movie")
M3UMovieRelation.objects.create(
m3u_account=low,
movie=movie,
category=action,
stream_id="low-cat",
)
M3UMovieRelation.objects.create(
m3u_account=high,
movie=movie,
category=comedy,
stream_id="high-cat",
)
stream = xc_get_vod_streams(self.request, self.user)[0]
self.assertEqual(stream["category_id"], str(comedy.id))
self.assertEqual(stream["category_ids"], [comedy.id])
def test_series_response_keys_and_metadata(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
logo = VODLogo.objects.create(name="Cover", url="http://example.com/cover.png")
category = VODCategory.objects.create(name="Drama", category_type="series")
series = Series.objects.create(
name="Schema Series",
description="Series plot",
genre="Sci-Fi",
year=2022,
rating="8",
tmdb_id="tm123",
imdb_id="tt123",
logo=logo,
custom_properties={
"cast": "Actor A",
"director": "Director B",
"release_date": "2022-06-01",
"backdrop_path": ["/img1.jpg"],
"youtube_trailer": "yt-series",
"episode_run_time": "45",
},
)
relation = M3USeriesRelation.objects.create(
m3u_account=account,
series=series,
category=category,
external_series_id="schema-s",
)
row = xc_get_series(self.request, self.user)[0]
self.assertEqual(set(row.keys()), XC_SERIES_KEYS)
self.assertEqual(row["series_id"], relation.id)
self.assertIn(f"/{logo.id}/", row["cover"])
self.assertEqual(row["plot"], "Series plot")
self.assertEqual(row["cast"], "Actor A")
self.assertEqual(row["director"], "Director B")
self.assertEqual(row["genre"], "Sci-Fi")
self.assertEqual(row["release_date"], "2022-06-01")
self.assertEqual(row["releaseDate"], "2022-06-01")
self.assertEqual(row["backdrop_path"], ["/img1.jpg"])
self.assertEqual(row["youtube_trailer"], "yt-series")
self.assertEqual(row["episode_run_time"], "45")
self.assertEqual(row["tmdb_id"], "tm123")
self.assertEqual(row["imdb_id"], "tt123")
self.assertEqual(row["category_id"], str(category.id))
self.assertEqual(row["category_ids"], [category.id])
self.assertEqual(row["last_modified"], str(int(relation.updated_at.timestamp())))
def test_series_null_optional_fields(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
series = Series.objects.create(name="Sparse Series")
M3USeriesRelation.objects.create(
m3u_account=account,
series=series,
external_series_id="sparse-s",
)
row = xc_get_series(self.request, self.user)[0]
self.assertIsNone(row["cover"])
self.assertEqual(row["category_id"], "0")
self.assertEqual(row["category_ids"], [])
self.assertEqual(row["release_date"], "")
self.assertEqual(row["releaseDate"], "")
self.assertEqual(row["backdrop_path"], [])
self.assertEqual(row["youtube_trailer"], "")
self.assertEqual(row["episode_run_time"], "")
def test_series_release_date_falls_back_to_year(self):
account = self._account(f"acct-{uuid4().hex[:6]}")
series = Series.objects.create(name="Year Only", year=2018)
M3USeriesRelation.objects.create(
m3u_account=account,
series=series,
external_series_id="year-s",
)
row = xc_get_series(self.request, self.user)[0]
self.assertEqual(row["release_date"], "2018")
self.assertEqual(row["releaseDate"], "2018")
def test_priority_tiebreaker_uses_lower_relation_id(self):
"""Same priority: DISTINCT ON tie-breaks on relation id ascending."""
a1 = self._account(f"a1-{uuid4().hex[:6]}", priority=5)
a2 = self._account(f"a2-{uuid4().hex[:6]}", priority=5)
movie = Movie.objects.create(name="Tie Movie")
first = M3UMovieRelation.objects.create(
m3u_account=a1,
movie=movie,
stream_id="first",
container_extension="mkv",
)
M3UMovieRelation.objects.create(
m3u_account=a2,
movie=movie,
stream_id="second",
container_extension="mp4",
)
stream = xc_get_vod_streams(self.request, self.user)[0]
self.assertEqual(stream["container_extension"], first.container_extension)
class GenerateEpgPrevDaysTests(SimpleTestCase):
"""Profile EPG keeps legacy prev_days=0 unless URL or user setting says otherwise."""
def setUp(self):
self.factory = RequestFactory()
@patch("apps.output.epg.stream_cached_response")
@patch("apps.output.epg.Channel.objects")
def test_non_xc_epg_defaults_prev_days_to_zero(self, _channels, mock_cache):
from apps.output.epg import generate_epg
mock_cache.side_effect = lambda cache_key, _source, **_kwargs: cache_key
request = self.factory.get("/epg/")
cache_key = generate_epg(request, profile_name="test", user=None)
self.assertIn(":p=0:", cache_key)

1954
apps/output/views.py
View file

File diff suppressed because it is too large Load diff

18
apps/proxy/config.py
View file

@ -42,7 +42,8 @@ class BaseConfig:
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 5,
"channel_init_grace_period": 60,
"channel_client_wait_period": 5,
"new_client_behind_seconds": 5,
}
@ -133,9 +134,15 @@ class TSConfig(BaseConfig):
@classmethod
def get_channel_init_grace_period(cls):
"""Get channel init grace period from database or default"""
"""Max seconds to wait for initial buffer fill during channel startup."""
settings = cls.get_proxy_settings()
return settings.get("channel_init_grace_period", 5)
return settings.get("channel_init_grace_period", 60)
@classmethod
def get_channel_client_wait_period(cls):
"""Seconds to keep a ready channel alive waiting for the first client to connect."""
settings = cls.get_proxy_settings()
return settings.get("channel_client_wait_period", 5)
# Dynamic property access for these settings
@property
@ -154,5 +161,6 @@ class TSConfig(BaseConfig):
def CHANNEL_INIT_GRACE_PERIOD(self):
return self.get_channel_init_grace_period()
@property
def CHANNEL_CLIENT_WAIT_PERIOD(self):
return self.get_channel_client_wait_period()

14
apps/proxy/live_proxy/channel_status.py
View file

@ -407,6 +407,20 @@ class ChannelStatus:
if channel_name:
info['channel_name'] = channel_name
info['is_timeshift'] = bool(metadata.get(ChannelMetadataField.IS_TIMESHIFT))
for key, field in (
('logo_id', ChannelMetadataField.LOGO_ID),
('m3u_profile_id', ChannelMetadataField.M3U_PROFILE),
):
raw = metadata.get(field)
if not raw:
continue
try:
info[key] = int(raw)
except (TypeError, ValueError):
pass
stream_id_bytes = metadata.get(ChannelMetadataField.STREAM_ID)
if stream_id_bytes:
try:

2
apps/proxy/live_proxy/client_manager.py
View file

@ -301,6 +301,8 @@ class ClientManager:
# Trigger channel stats update via WebSocket
self._trigger_stats_update()
ChannelService.promote_channel_when_buffer_ready(self.channel_id)
# Get total clients across all workers
total_clients = self.get_total_client_count()
logger.info(f"New client connected: {client_id} (local: {len(self.clients)}, total: {total_clients})")

7
apps/proxy/live_proxy/config_helper.py
View file

@ -107,9 +107,14 @@ class ConfigHelper:
@staticmethod
def channel_init_grace_period():
"""Get channel initialization grace period in seconds"""
"""Max seconds to wait for initial buffer fill during channel startup."""
return Config.get_channel_init_grace_period()
@staticmethod
def channel_client_wait_period():
"""Seconds to keep a ready channel alive waiting for the first client to connect."""
return Config.get_channel_client_wait_period()
@staticmethod
def chunk_timeout():
"""

2
apps/proxy/live_proxy/constants.py
View file

@ -54,6 +54,8 @@ class ChannelMetadataField:
STREAM_ID = "stream_id"
CHANNEL_NAME = "channel_name"
STREAM_NAME = "stream_name"
IS_TIMESHIFT = "is_timeshift"
LOGO_ID = "logo_id"
# Profile fields
STREAM_PROFILE = "stream_profile"

22
apps/proxy/live_proxy/input/manager.py
View file

@ -1420,6 +1420,12 @@ class StreamManager:
"""Check if connection retry is allowed"""
return self.retry_count < self.max_retries
def _health_inactivity_threshold(self):
"""How long without data before marking the stream unhealthy."""
if self.connected and getattr(self.buffer, 'index', 0) == 0:
return ConfigHelper.channel_init_grace_period()
return getattr(Config, 'CONNECTION_TIMEOUT', 10)
def _monitor_health(self):
"""Monitor stream health and set flags for the main loop to handle recovery"""
consecutive_unhealthy_checks = 0
@ -1435,7 +1441,7 @@ class StreamManager:
try:
now = time.time()
inactivity_duration = now - self.last_data_time
timeout_threshold = getattr(Config, 'CONNECTION_TIMEOUT', 10)
timeout_threshold = self._health_inactivity_threshold()
if inactivity_duration > timeout_threshold and self.connected:
if self.healthy:
@ -1830,19 +1836,9 @@ class StreamManager:
timer.start()
return False
# We have enough buffer, proceed with state change
update_data = {
ChannelMetadataField.STATE: ChannelState.WAITING_FOR_CLIENTS,
ChannelMetadataField.CONNECTION_READY_TIME: current_time,
ChannelMetadataField.STATE_CHANGED_AT: current_time,
ChannelMetadataField.BUFFER_CHUNKS: str(current_buffer_index)
}
redis_client.hset(metadata_key, mapping=update_data)
from ..services.channel_service import ChannelService
# Get configured grace period or default
grace_period = ConfigHelper.channel_init_grace_period()
logger.info(f"STREAM MANAGER: Updated channel {channel_id} state: {current_state or 'None'} -> {ChannelState.WAITING_FOR_CLIENTS} with {current_buffer_index} buffer chunks")
logger.info(f"Started initial connection grace period ({grace_period}s) for channel {channel_id}")
ChannelService.promote_channel_when_buffer_ready(channel_id)
else:
logger.debug(f"Not changing state: channel {channel_id} already in {current_state} state")
except Exception as e:

6
apps/proxy/live_proxy/output/ts/generator.py
View file

@ -159,8 +159,8 @@ class StreamGenerator:
return "client_gone"
elapsed = time.time() - initialization_start
connection_timeout = getattr(Config, 'CONNECTION_TIMEOUT', 10)
if elapsed < connection_timeout or not proxy_server.redis_client:
init_grace_period = ConfigHelper.channel_init_grace_period()
if elapsed < init_grace_period or not proxy_server.redis_client:
return None
metadata_key = RedisKeys.channel_metadata(self.channel_id)
@ -204,7 +204,7 @@ class StreamGenerator:
logger.warning(
f"[{self.client_id}] Channel {self.channel_id} stalled in connecting state "
f"with no buffer data after "
f"{getattr(Config, 'CONNECTION_TIMEOUT', 10)}s, aborting init wait"
f"{ConfigHelper.channel_init_grace_period()}s, aborting init wait"
)
yield create_ts_packet('error', "Error: Connection stalled")
return False

87
apps/proxy/live_proxy/server.py
View file

@ -789,7 +789,7 @@ class ProxyServer:
attempt_key = RedisKeys.connection_attempt(channel_id)
self.redis_client.setex(attempt_key, 60, str(time.time()))
logger.info(f"Channel {channel_id} in {ChannelState.CONNECTING} state - will start grace period after connection")
logger.info(f"Channel {channel_id} in {ChannelState.CONNECTING} state - waiting for buffer to fill")
return True
except Exception as e:
@ -910,6 +910,26 @@ class ProxyServer:
delay = ConfigHelper.channel_shutdown_delay()
return max(int(delay * 2), 60)
@staticmethod
def _pre_active_no_clients_should_stop(connection_ready_time, start_time, now=None):
"""
Decide whether a pre-active channel with zero clients should be stopped.
Returns (should_stop, timeout_seconds, reason) where reason is
'client_wait' (buffer ready, waiting for first viewer) or 'startup'
(still connecting / filling buffer).
"""
now = now if now is not None else time.time()
if connection_ready_time:
elapsed = now - connection_ready_time
timeout = ConfigHelper.channel_client_wait_period()
return elapsed > timeout, timeout, "client_wait"
if start_time:
elapsed = now - start_time
timeout = ConfigHelper.channel_init_grace_period()
return elapsed > timeout, timeout, "startup"
return False, None, None
def _wait_for_shutdown_delay(self, channel_id):
"""
Wait until shutdown_delay has elapsed since the Redis disconnect
@ -1758,7 +1778,7 @@ class ProxyServer:
if time.time() % 30 < 1: # Every ~30 seconds
logger.info(f"Channel {channel_id} has {total_clients} clients, state: {channel_state}")
# If in connecting or waiting_for_clients state, check grace period
# Pre-active channels: init timeouts and buffer-ready promotion
if channel_state in [ChannelState.INITIALIZING, ChannelState.CONNECTING, ChannelState.WAITING_FOR_CLIENTS]:
# Check if channel is already stopping
if self.redis_client:
@ -1799,52 +1819,34 @@ class ProxyServer:
start_time = connection_attempt_time or init_time
if start_time:
# Check which timeout to apply based on channel lifecycle
if connection_ready_time:
# Already reached ready - use shutdown_delay
time_since_ready = time.time() - connection_ready_time
shutdown_delay = ConfigHelper.channel_shutdown_delay()
if time_since_ready > shutdown_delay:
should_stop, timeout, reason = (
self._pre_active_no_clients_should_stop(
connection_ready_time,
start_time,
)
)
if should_stop:
if reason == "client_wait":
time_since_ready = time.time() - connection_ready_time
logger.warning(
f"Channel {channel_id} in {channel_state} state with 0 clients for {time_since_ready:.1f}s "
f"(after reaching ready, shutdown_delay: {shutdown_delay}s) - stopping channel"
f"(buffer ready, no client connected, client_wait_period: {timeout}s) - stopping channel"
)
self._coordinated_stop_channel(channel_id)
continue
else:
# Never reached ready - use grace_period timeout
time_since_start = time.time() - start_time
connecting_timeout = ConfigHelper.channel_init_grace_period()
if time_since_start > connecting_timeout:
else:
time_since_start = time.time() - start_time
logger.warning(
f"Channel {channel_id} stuck in {channel_state} state for {time_since_start:.1f}s "
f"with no clients (timeout: {connecting_timeout}s) - stopping channel due to upstream issues"
f"with no clients (timeout: {timeout}s) - stopping channel due to upstream issues"
)
self._coordinated_stop_channel(channel_id)
continue
elif connection_ready_time:
# We have clients now, but check grace period for state transition
grace_period = ConfigHelper.channel_init_grace_period()
time_since_ready = time.time() - connection_ready_time
self._coordinated_stop_channel(channel_id)
continue
elif (
channel_state == ChannelState.WAITING_FOR_CLIENTS
and total_clients > 0
):
from .services.channel_service import ChannelService
logger.debug(f"GRACE PERIOD CHECK: Channel {channel_id} in {channel_state} state, "
f"time_since_ready={time_since_ready:.1f}s, grace_period={grace_period}s, "
f"total_clients={total_clients}")
if time_since_ready <= grace_period:
# Still within grace period
logger.debug(f"Channel {channel_id} in grace period - {time_since_ready:.1f}s of {grace_period}s elapsed")
continue
else:
# Grace period expired with clients - mark channel as active
logger.info(f"Grace period expired with {total_clients} clients - marking channel {channel_id} as active")
if self.update_channel_state(channel_id, ChannelState.ACTIVE, {
"grace_period_ended_at": str(time.time()),
"clients_at_activation": str(total_clients)
}):
logger.info(f"Channel {channel_id} activated with {total_clients} clients after grace period")
ChannelService.promote_channel_when_buffer_ready(channel_id)
# If active and no clients, start normal shutdown procedure
elif channel_state not in [ChannelState.CONNECTING, ChannelState.WAITING_FOR_CLIENTS] and total_clients == 0:
# Check if channel is already stopping
@ -2093,6 +2095,9 @@ class ProxyServer:
if not channel_id:
continue
if channel_id.startswith("timeshift_"):
continue
# Get metadata first
metadata = self.redis_client.hgetall(key)
if not metadata:

72
apps/proxy/live_proxy/services/channel_service.py
View file

@ -192,6 +192,78 @@ class ChannelService:
ChannelState.CONNECTING,
)
@staticmethod
def promote_channel_when_buffer_ready(channel_id):
"""
Promote channel state once the initial buffer threshold is met.
- connecting/initializing + buffer ready + clients -> active
- connecting/initializing + buffer ready + no clients -> waiting_for_clients
- waiting_for_clients + clients -> active
Returns the resulting state, or None when no promotion applies.
"""
proxy_server = ProxyServer.get_instance()
redis_client = proxy_server.redis_client
if not redis_client:
return None
metadata_key = RedisKeys.channel_metadata(channel_id)
state_raw = redis_client.hget(metadata_key, ChannelMetadataField.STATE)
if not state_raw:
return None
state = state_raw.decode() if isinstance(state_raw, bytes) else state_raw
if state == ChannelState.ACTIVE:
return ChannelState.ACTIVE
if state == ChannelState.WAITING_FOR_CLIENTS:
ready_raw = redis_client.hget(
metadata_key, ChannelMetadataField.CONNECTION_READY_TIME
)
if not ready_raw:
return None
client_count = redis_client.scard(RedisKeys.clients(channel_id)) or 0
if client_count <= 0:
return ChannelState.WAITING_FOR_CLIENTS
proxy_server.update_channel_state(
channel_id,
ChannelState.ACTIVE,
{"clients_at_activation": str(client_count)},
)
return ChannelState.ACTIVE
if state not in (ChannelState.INITIALIZING, ChannelState.CONNECTING):
return None
try:
buffer_index = int(redis_client.get(RedisKeys.buffer_index(channel_id)) or 0)
except (TypeError, ValueError):
buffer_index = 0
chunks_needed = ConfigHelper.initial_behind_chunks()
if buffer_index < chunks_needed:
return None
client_count = redis_client.scard(RedisKeys.clients(channel_id)) or 0
new_state = (
ChannelState.ACTIVE if client_count > 0 else ChannelState.WAITING_FOR_CLIENTS
)
current_time = str(time.time())
extra = {
ChannelMetadataField.CONNECTION_READY_TIME: current_time,
ChannelMetadataField.BUFFER_CHUNKS: str(buffer_index),
}
if new_state == ChannelState.ACTIVE:
extra["clients_at_activation"] = str(client_count)
proxy_server.update_channel_state(channel_id, new_state, extra)
logger.info(
f"Channel {channel_id} buffer ready ({buffer_index}/{chunks_needed} chunks) "
f"-> {new_state} (clients={client_count})"
)
return new_state
@staticmethod
def initialize_channel(channel_id, stream_url, user_agent, transcode=False, stream_profile_value=None, stream_id=None, m3u_profile_id=None, channel_name=None, stream_name=None):
"""

143
apps/proxy/live_proxy/tests/test_proxy_settings.py Normal file
View file

@ -0,0 +1,143 @@
"""Tests for proxy settings defaults, serializer validation, and migration 0026."""
from importlib import import_module
from unittest.mock import patch
from django.apps import apps
from django.test import SimpleTestCase, TestCase
from apps.proxy.config import TSConfig
from core.models import CoreSettings
from core.serializers import ProxySettingsSerializer
MIGRATION_0026 = import_module("core.migrations.0026_add_channel_client_wait_period")
class TSConfigProxySettingsDefaultsTests(SimpleTestCase):
@patch.object(TSConfig, "get_proxy_settings", return_value={})
def test_channel_init_grace_period_default(self, _mock_settings):
self.assertEqual(TSConfig.get_channel_init_grace_period(), 60)
@patch.object(TSConfig, "get_proxy_settings", return_value={})
def test_channel_client_wait_period_default(self, _mock_settings):
self.assertEqual(TSConfig.get_channel_client_wait_period(), 5)
@patch.object(
TSConfig,
"get_proxy_settings",
return_value={
"channel_init_grace_period": 120,
"channel_client_wait_period": 15,
},
)
def test_settings_override_db_values(self, _mock_settings):
self.assertEqual(TSConfig.get_channel_init_grace_period(), 120)
self.assertEqual(TSConfig.get_channel_client_wait_period(), 15)
class ProxySettingsSerializerTests(SimpleTestCase):
def _valid_payload(self, **overrides):
payload = {
"buffering_timeout": 15,
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 60,
"channel_client_wait_period": 5,
"new_client_behind_seconds": 5,
}
payload.update(overrides)
return payload
def test_accepts_new_client_wait_period(self):
serializer = ProxySettingsSerializer(data=self._valid_payload())
self.assertTrue(serializer.is_valid(), serializer.errors)
self.assertEqual(serializer.validated_data["channel_client_wait_period"], 5)
def test_init_grace_period_allows_up_to_300(self):
serializer = ProxySettingsSerializer(
data=self._valid_payload(channel_init_grace_period=300)
)
self.assertTrue(serializer.is_valid(), serializer.errors)
def test_init_grace_period_rejects_above_300(self):
serializer = ProxySettingsSerializer(
data=self._valid_payload(channel_init_grace_period=301)
)
self.assertFalse(serializer.is_valid())
self.assertIn("channel_init_grace_period", serializer.errors)
class CoreSettingsProxyDefaultsTests(TestCase):
def test_get_proxy_settings_defaults_when_missing(self):
CoreSettings.objects.filter(key="proxy_settings").delete()
defaults = CoreSettings.get_proxy_settings()
self.assertEqual(defaults["channel_init_grace_period"], 60)
self.assertEqual(defaults["channel_client_wait_period"], 5)
class Migration0026ProxySettingsTests(TestCase):
def _run_migration_forward(self):
MIGRATION_0026.add_channel_client_wait_period(apps, None)
def _set_proxy_settings(self, value):
settings_obj, _ = CoreSettings.objects.get_or_create(
key="proxy_settings",
defaults={"name": "Proxy Settings", "value": value},
)
settings_obj.value = value
settings_obj.save(update_fields=["value"])
return settings_obj
def test_bumps_legacy_init_grace_and_adds_client_wait(self):
settings_obj = self._set_proxy_settings(
{
"buffering_timeout": 15,
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 5,
"new_client_behind_seconds": 5,
}
)
self._run_migration_forward()
settings_obj.refresh_from_db()
self.assertEqual(settings_obj.value["channel_init_grace_period"], 60)
self.assertEqual(settings_obj.value["channel_client_wait_period"], 5)
def test_bumps_init_grace_below_new_default(self):
settings_obj = self._set_proxy_settings(
{
"buffering_timeout": 15,
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 45,
"new_client_behind_seconds": 5,
}
)
self._run_migration_forward()
settings_obj.refresh_from_db()
self.assertEqual(settings_obj.value["channel_init_grace_period"], 60)
def test_preserves_init_grace_at_or_above_new_default(self):
settings_obj = self._set_proxy_settings(
{
"buffering_timeout": 15,
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 90,
"new_client_behind_seconds": 5,
}
)
self._run_migration_forward()
settings_obj.refresh_from_db()
self.assertEqual(settings_obj.value["channel_init_grace_period"], 90)
self.assertEqual(settings_obj.value["channel_client_wait_period"], 5)

56
apps/proxy/utils.py
View file

@ -1,6 +1,7 @@
import logging
from core.utils import RedisClient
from apps.proxy.vod_proxy.multi_worker_connection_manager import MultiWorkerVODConnectionManager, get_vod_client_stop_key
from apps.proxy.live_proxy.redis_keys import RedisKeys
from core.models import CoreSettings
from apps.proxy.live_proxy.services.channel_service import ChannelService
@ -61,6 +62,15 @@ def attempt_stream_termination(user_id, requesting_client_id, active_connections
result = ChannelService.stop_client(t['media_id'], t['client_id'])
if result.get("status") == "error":
logger.warning(f"[stream limits][{requesting_client_id}] Failed to stop client {t['client_id']} on channel {t['media_id']}")
elif t['type'] == 'timeshift':
# Same Redis stop key as live; timeshift generator polls it every 5s.
redis_client = RedisClient.get_client()
if not redis_client:
# Deny the new stream if we cannot stop the old one.
return False
stop_key = RedisKeys.client_stop(t['media_id'], t['client_id'])
redis_client.setex(stop_key, 60, "true")
logger.info(f"[stream limits][{requesting_client_id}] Set stop key for timeshift client {t['client_id']}")
else:
connection_manager = MultiWorkerVODConnectionManager.get_instance()
redis_client = connection_manager.redis_client
@ -106,13 +116,14 @@ def get_user_active_connections(user_id):
if user_id is None or (client_user_id and int(client_user_id) == user_id):
try:
logger.debug(f"[stream limits] Found LIVE connection for user {user_id} on channel {channel_id} with client ID {client_id}")
conn_type = 'timeshift' if channel_id.startswith('timeshift_') else 'live'
logger.debug(f"[stream limits] Found {conn_type.upper()} connection for user {user_id} on channel {channel_id} with client ID {client_id}")
connected_at = float(connected_at) if connected_at else 0
connections.append({
'media_id': channel_id,
'client_id': client_id,
'connected_at': connected_at,
'type': 'live',
'type': conn_type,
})
except (ValueError, TypeError):
pass
@ -172,6 +183,22 @@ def check_user_stream_limits(user, client_id, media_id=None):
logger.debug(f"[stream limits][{client_id}] Same-channel reconnect for {media_id} allowed (ignore_same_channel=True)")
return True
# Timeshift sibling range/probe requests share one provider slot per
# session_id. Each distinct client/session still consumes its own slot.
if ignore_same_channel and media_id:
media_id_str = str(media_id)
for conn in active_connections:
if conn.get('type') != 'timeshift':
continue
if conn.get('client_id') != client_id:
continue
conn_media_id = str(conn.get('media_id') or '')
if conn_media_id == media_id_str or conn_media_id.startswith(f"{media_id_str}_"):
logger.debug(
f"[stream limits][{client_id}] Same timeshift session probe for {media_id} allowed (ignore_same_channel=True)"
)
return True
if user_stream_count >= user.stream_limit:
if user_limit_settings.get("terminate_on_limit_exceeded", True) == False:
return False
@ -186,3 +213,28 @@ def check_user_stream_limits(user, client_id, media_id=None):
return False
return True
_TS_PACKET_SIZE = 188
_TS_SYNC_BYTE = 0x47
def find_ts_sync(buf):
"""Return byte offset of the first valid MPEG-TS sync chain in *buf*, or -1.
Args:
buf: Raw bytes from an upstream HTTP response (typically the first 1 KB).
Returns:
Offset of the first 0x47 byte that starts three consecutive 188-byte
packets, or -1. Used to strip PHP/HTML preamble before streaming.
"""
end = len(buf) - 2 * _TS_PACKET_SIZE
for i in range(0, end):
if (
buf[i] == _TS_SYNC_BYTE
and buf[i + _TS_PACKET_SIZE] == _TS_SYNC_BYTE
and buf[i + 2 * _TS_PACKET_SIZE] == _TS_SYNC_BYTE
):
return i
return -1

0
apps/timeshift/__init__.py Normal file
View file

7
apps/timeshift/apps.py Normal file
View file

@ -0,0 +1,7 @@
from django.apps import AppConfig
class TimeshiftConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "apps.timeshift"
verbose_name = "Timeshift"

252
apps/timeshift/helpers.py Normal file
View file

@ -0,0 +1,252 @@
"""URL builders and timestamp helpers for XC catch-up."""
import logging
import re
from collections import namedtuple
from datetime import datetime, timezone
from urllib.parse import quote
from zoneinfo import ZoneInfo
logger = logging.getLogger(__name__)
# Credentials for the profile whose pool slot was reserved (not raw account fields).
TimeshiftCredentials = namedtuple(
"TimeshiftCredentials", ("server_url", "username", "password")
)
DEFAULT_DURATION_MINUTES = 120
DURATION_BUFFER_MINUTES = 5
MAX_DURATION_MINUTES = 480
# Wall-clock shapes seen from XC / iPlayTV / TiviMate clients. Compiled once.
_CATCHUP_WALL_CLOCK_RE = re.compile(
r"^"
r"(?P<date>\d{4}-\d{2}-\d{2})"
r"(?P<dtsep>[:_]| )"
r"(?P<hour>\d{2})"
r"(?P<hmsep>[-:])"
r"(?P<minute>\d{2})"
r"(?:"
r":"
r"(?P<second>\d{2})"
r")?"
r"$"
)
def normalize_catchup_timestamp_input(timestamp_str):
"""Map a client catch-up timestamp to an ISO-8601 string for ``fromisoformat``.
Supported inputs:
- ``YYYY-MM-DD:HH-MM`` (iPlayTV/TiviMate colon-dash)
- ``YYYY-MM-DD_HH-MM`` (XC underscore)
- ``YYYY-MM-DD:HH:MM[:SS]`` (XC colon time in catch-up URLs)
- ``YYYY-MM-DD HH:MM[:SS]`` (EPG / SQL datetime)
- Unix epoch seconds (10 digits) or milliseconds (13 digits)
Returns:
An ISO-8601 date-time string (``YYYY-MM-DDTHH:MM:SS``), or None if
the value does not match a known catch-up shape.
"""
if timestamp_str is None:
return None
if not isinstance(timestamp_str, str):
timestamp_str = str(timestamp_str)
value = timestamp_str.strip()
if not value:
return None
if value.isdigit():
length = len(value)
if length == 10:
dt = datetime.fromtimestamp(int(value), tz=timezone.utc)
return dt.replace(tzinfo=None).isoformat(timespec="seconds")
if length == 13:
dt = datetime.fromtimestamp(int(value) / 1000, tz=timezone.utc)
return dt.replace(tzinfo=None).isoformat(timespec="seconds")
return None
match = _CATCHUP_WALL_CLOCK_RE.match(value)
if not match:
return None
parts = match.groupdict()
second = parts["second"] or "00"
return f"{parts['date']}T{parts['hour']}:{parts['minute']}:{second}"
def parse_catchup_timestamp(timestamp_str):
"""Parse a catch-up timestamp string into a naive UTC wall-clock datetime.
See ``normalize_catchup_timestamp_input`` for supported input shapes.
Returns:
A naive datetime on success, or None.
"""
iso_value = normalize_catchup_timestamp_input(timestamp_str)
if iso_value is None:
if timestamp_str is not None and str(timestamp_str).strip():
logger.debug(
"Timeshift: unrecognised catch-up timestamp: %r", timestamp_str
)
return None
try:
return datetime.fromisoformat(iso_value)
except ValueError:
logger.debug(
"Timeshift: invalid catch-up timestamp after normalize: %r -> %r",
timestamp_str,
iso_value,
)
return None
def _reshape_timestamp(timestamp, strftime_fmt, label):
dt = parse_catchup_timestamp(timestamp)
if dt is None:
logger.error(
"Timeshift %s reshape failed for %r: unrecognised format", label, timestamp
)
return timestamp
return dt.strftime(strftime_fmt)
def convert_timestamp_to_provider_tz(timestamp_str, provider_tz_name):
"""Convert a UTC catch-up timestamp to the provider's local zone.
Args:
timestamp_str: UTC wall-clock in ``YYYY-MM-DD:HH-MM`` or underscore form.
provider_tz_name: IANA zone from the provider's ``server_info.timezone``
(e.g. ``Europe/Brussels``). Falsy, ``UTC``, or unknown: no conversion.
Returns:
``YYYY-MM-DD:HH-MM`` in the provider zone, or the input unchanged on skip/failure.
"""
if not provider_tz_name or provider_tz_name == "UTC":
return timestamp_str
dt = parse_catchup_timestamp(timestamp_str)
if dt is None:
return timestamp_str
try:
target = ZoneInfo(provider_tz_name)
except Exception:
logger.warning(
"Timeshift: unknown provider timezone %r, no conversion applied",
provider_tz_name,
)
return timestamp_str
# timezone.utc, not ZoneInfo("UTC"): avoids mis-set Docker /etc/timezone.
local_dt = dt.replace(tzinfo=timezone.utc).astimezone(target)
return local_dt.strftime("%Y-%m-%d:%H-%M")
def get_programme_duration(channel, timestamp_str):
"""Look up catch-up duration in minutes from EPG.
Args:
channel: Channel with optional ``epg_data`` relation loaded.
timestamp_str: Programme start in UTC (same shape as the client URL).
Returns:
Programme length plus a small buffer, capped at ``MAX_DURATION_MINUTES``,
or ``DEFAULT_DURATION_MINUTES`` when EPG lookup fails.
"""
try:
dt = parse_catchup_timestamp(timestamp_str)
if dt is None:
return DEFAULT_DURATION_MINUTES
# EPG times are timezone-aware; parsed value must be too.
dt = dt.replace(tzinfo=timezone.utc)
if not channel.epg_data:
return DEFAULT_DURATION_MINUTES
programme = channel.epg_data.programs.filter(
start_time__lte=dt, end_time__gt=dt
).first()
if not programme:
return DEFAULT_DURATION_MINUTES
duration_seconds = (programme.end_time - programme.start_time).total_seconds()
duration_minutes = int(duration_seconds / 60) + DURATION_BUFFER_MINUTES
return min(duration_minutes, MAX_DURATION_MINUTES)
except Exception:
return DEFAULT_DURATION_MINUTES
def build_timeshift_url_format_a(creds, stream_id, timestamp, duration_minutes):
"""QUERY layout: ``/streaming/timeshift.php?username=...&start=...``"""
return (
f"{creds.server_url.rstrip('/')}/streaming/timeshift.php"
f"?username={quote(str(creds.username), safe='')}"
f"&password={quote(str(creds.password), safe='')}"
f"&stream={stream_id}"
f"&start={timestamp}"
f"&duration={duration_minutes}"
)
def build_timeshift_url_format_b(creds, stream_id, timestamp, duration_minutes):
"""PATH layout: ``/timeshift/{user}/{pass}/{dur}/{start}/{id}.ts``"""
return (
f"{creds.server_url.rstrip('/')}/timeshift"
f"/{quote(str(creds.username), safe='')}"
f"/{quote(str(creds.password), safe='')}"
f"/{duration_minutes}"
f"/{timestamp}"
f"/{stream_id}.ts"
)
def build_timeshift_candidate_urls(creds, stream_id, timestamp, duration_minutes):
"""Build ordered upstream URL candidates (PATH forms first, QUERY last).
Args:
creds: ``TimeshiftCredentials`` for the reserved profile.
stream_id: Provider stream id from the catch-up stream's custom properties.
timestamp: Already converted to the serving provider's local zone.
duration_minutes: Archive window length passed to the provider.
Returns:
List of URL strings to try in order. QUERY forms are last because some
providers return live TV even when ``start`` is set.
"""
dt = parse_catchup_timestamp(timestamp)
if dt is None:
colon_dash_ts = timestamp
underscore_ts = timestamp
colon_seconds_ts = timestamp
sql_ts = timestamp
else:
colon_dash_ts = dt.strftime("%Y-%m-%d:%H-%M")
underscore_ts = dt.strftime("%Y-%m-%d_%H-%M")
colon_seconds_ts = dt.strftime("%Y-%m-%d:%H:%M:%S")
sql_ts = dt.strftime("%Y-%m-%d %H:%M:%S")
return [
build_timeshift_url_format_b(creds, stream_id, colon_dash_ts, duration_minutes),
build_timeshift_url_format_b(creds, stream_id, underscore_ts, duration_minutes),
build_timeshift_url_format_b(creds, stream_id, colon_seconds_ts, duration_minutes),
build_timeshift_url_format_a(creds, stream_id, underscore_ts, duration_minutes),
build_timeshift_url_format_a(creds, stream_id, sql_ts, duration_minutes),
build_timeshift_url_format_a(creds, stream_id, colon_dash_ts, duration_minutes),
build_timeshift_url_format_a(creds, stream_id, colon_seconds_ts, duration_minutes),
]
def format_timestamp_as_colon_dash(timestamp):
"""Reshape to ``YYYY-MM-DD:HH-MM`` without timezone conversion."""
return _reshape_timestamp(timestamp, "%Y-%m-%d:%H-%M", "colon-dash")
def format_timestamp_as_colon_seconds(timestamp):
"""Reshape to ``YYYY-MM-DD:HH:MM:SS`` without timezone conversion."""
return _reshape_timestamp(timestamp, "%Y-%m-%d:%H:%M:%S", "colon-seconds")
def format_timestamp_as_underscore(timestamp):
"""Reshape to ``YYYY-MM-DD_HH-MM`` without timezone conversion."""
return _reshape_timestamp(timestamp, "%Y-%m-%d_%H-%M", "underscore")
def format_timestamp_as_sql_datetime(timestamp):
"""Reshape to ``YYYY-MM-DD HH:MM:SS`` without timezone conversion."""
return _reshape_timestamp(timestamp, "%Y-%m-%d %H:%M:%S", "SQL")

0
apps/timeshift/tests/__init__.py Normal file
View file

325
apps/timeshift/tests/test_helpers.py Normal file
View file

@ -0,0 +1,325 @@
"""Tests for `apps.timeshift.helpers`: timestamp shape conversion and URL build."""
from datetime import datetime, timezone
from django.test import TestCase
from apps.timeshift.helpers import (
TimeshiftCredentials,
build_timeshift_candidate_urls,
build_timeshift_url_format_a,
build_timeshift_url_format_b,
convert_timestamp_to_provider_tz,
format_timestamp_as_colon_dash,
format_timestamp_as_colon_seconds,
format_timestamp_as_sql_datetime,
format_timestamp_as_underscore,
normalize_catchup_timestamp_input,
parse_catchup_timestamp,
)
def _make_creds():
# The builders consume resolved per-profile credentials, never an account
# object — get_transformed_credentials() produces these in the view.
return TimeshiftCredentials("http://example.test", "user", "pass")
class TimestampFormatTests(TestCase):
"""Timestamp reshape functions change format only; no timezone conversion."""
def test_normalize_colon_dash_shape(self):
self.assertEqual(
normalize_catchup_timestamp_input("2026-05-21:12-55"),
"2026-05-21T12:55:00",
)
def test_normalize_colon_seconds_xc_format(self):
self.assertEqual(
normalize_catchup_timestamp_input("2026-06-23:04:00:00"),
"2026-06-23T04:00:00",
)
def test_normalize_epg_sql_format(self):
self.assertEqual(
normalize_catchup_timestamp_input("2026-06-23 04:00:00"),
"2026-06-23T04:00:00",
)
def test_normalize_unix_epoch_seconds(self):
epoch = str(int(datetime(2026, 6, 23, 4, 0, 0, tzinfo=timezone.utc).timestamp()))
self.assertEqual(
normalize_catchup_timestamp_input(epoch),
"2026-06-23T04:00:00",
)
def test_normalize_unix_epoch_milliseconds(self):
epoch_ms = str(
int(datetime(2026, 6, 23, 4, 0, 0, tzinfo=timezone.utc).timestamp() * 1000)
)
self.assertEqual(
normalize_catchup_timestamp_input(epoch_ms),
"2026-06-23T04:00:00",
)
def test_normalize_rejects_garbage(self):
self.assertIsNone(normalize_catchup_timestamp_input("garbage"))
self.assertIsNone(normalize_catchup_timestamp_input(""))
self.assertIsNone(normalize_catchup_timestamp_input("12345"))
def test_parse_rejects_invalid_calendar_date(self):
self.assertIsNone(parse_catchup_timestamp("2026-13-45:04-00"))
def test_parse_colon_dash_format(self):
dt = parse_catchup_timestamp("2026-05-21:12-55")
self.assertEqual(dt, datetime(2026, 5, 21, 12, 55, 0))
def test_parse_underscore_format(self):
dt = parse_catchup_timestamp("2026-05-21_12-55")
self.assertEqual(dt, datetime(2026, 5, 21, 12, 55, 0))
def test_parse_colon_minutes_without_seconds(self):
dt = parse_catchup_timestamp("2026-06-23:04:00")
self.assertEqual(dt, datetime(2026, 6, 23, 4, 0, 0))
def test_parse_colon_seconds_xc_format(self):
dt = parse_catchup_timestamp("2026-06-23:04:00:00")
self.assertEqual(dt, datetime(2026, 6, 23, 4, 0, 0))
def test_parse_epg_sql_format(self):
dt = parse_catchup_timestamp("2026-06-23 04:00:00")
self.assertEqual(dt, datetime(2026, 6, 23, 4, 0, 0))
def test_format_colon_dash_from_colon_seconds(self):
self.assertEqual(
format_timestamp_as_colon_dash("2026-06-23:04:00:00"),
"2026-06-23:04-00",
)
def test_format_colon_seconds_from_colon_dash(self):
self.assertEqual(
format_timestamp_as_colon_seconds("2026-06-23:04-00"),
"2026-06-23:04:00:00",
)
def test_format_colon_seconds_from_unix_epoch(self):
epoch = str(int(datetime(2026, 6, 23, 4, 0, 0, tzinfo=timezone.utc).timestamp()))
self.assertEqual(
format_timestamp_as_colon_dash(epoch),
"2026-06-23:04-00",
)
def test_format_sql_reshapes_without_tz_conversion(self):
self.assertEqual(
format_timestamp_as_sql_datetime("2026-05-12:17-00"),
"2026-05-12 17:00:00",
)
def test_format_sql_accepts_underscore_input(self):
self.assertEqual(
format_timestamp_as_sql_datetime("2026-05-12_17-00"),
"2026-05-12 17:00:00",
)
def test_format_sql_invalid_falls_back(self):
self.assertEqual(format_timestamp_as_sql_datetime("garbage"), "garbage")
def test_format_underscore_from_colon_dash(self):
self.assertEqual(
format_timestamp_as_underscore("2026-05-21:12-55"),
"2026-05-21_12-55",
)
def test_format_underscore_idempotent(self):
# Underscore input → underscore output (no change)
self.assertEqual(
format_timestamp_as_underscore("2026-05-21_12-55"),
"2026-05-21_12-55",
)
def test_format_underscore_invalid_falls_back(self):
self.assertEqual(format_timestamp_as_underscore("garbage"), "garbage")
class BuildTimeshiftUrlTests(TestCase):
def setUp(self):
self.creds = _make_creds()
def test_format_a_passes_dash_shape_unchanged(self):
url = build_timeshift_url_format_a(
self.creds, "22372", "2026-05-12:19-00", 40
)
self.assertIn("start=2026-05-12:19-00", url)
self.assertIn("stream=22372", url)
self.assertIn("duration=40", url)
def test_format_a_passes_sql_shape_unchanged(self):
url = build_timeshift_url_format_a(
self.creds, "22372", "2026-05-12 19:00:00", 40
)
self.assertIn("start=2026-05-12 19:00:00", url)
def test_format_b_path_with_dash_shape(self):
url = build_timeshift_url_format_b(
self.creds, "22372", "2026-05-12:19-00", 40
)
self.assertIn("/40/2026-05-12:19-00/22372.ts", url)
class CandidateOrderingTests(TestCase):
"""`build_timeshift_candidate_urls` must try the PATH form (which seeks the
archive) before the QUERY form (which returns LIVE on some providers,
silently ignoring the requested timestamp). Regression guard for the
"catch-up plays the live stream instead of the requested programme" bug."""
def setUp(self):
self.creds = _make_creds()
def _is_path_form(self, url):
return "/timeshift/" in url and url.endswith(".ts") and "timeshift.php" not in url
def _is_query_form(self, url):
return "timeshift.php?" in url
def test_every_path_candidate_precedes_every_query_candidate(self):
urls = build_timeshift_candidate_urls(self.creds, "22372", "2026-05-12:19-00", 40)
path_indices = [i for i, u in enumerate(urls) if self._is_path_form(u)]
query_indices = [i for i, u in enumerate(urls) if self._is_query_form(u)]
# Each URL is classified as exactly one form.
self.assertEqual(len(path_indices) + len(query_indices), len(urls))
self.assertTrue(path_indices and query_indices)
# The last PATH candidate still comes before the first QUERY candidate.
self.assertLess(max(path_indices), min(query_indices))
def test_first_candidate_is_path_form_with_canonical_dash_timestamp(self):
urls = build_timeshift_candidate_urls(self.creds, "22372", "2026-05-12:19-00", 40)
self.assertTrue(self._is_path_form(urls[0]))
# Canonical colon-dash timestamp, passed through unchanged.
self.assertIn("/40/2026-05-12:19-00/22372.ts", urls[0])
def test_accepts_colon_seconds_input_timestamp(self):
urls = build_timeshift_candidate_urls(
self.creds, "22372", "2026-06-23:04:00:00", 40
)
self.assertTrue(self._is_path_form(urls[0]))
self.assertIn("/40/2026-06-23:04-00/22372.ts", urls[0])
self.assertIn("/40/2026-06-23:04:00:00/22372.ts", urls[2])
def test_accepts_underscore_input_timestamp(self):
# Client may send the underscore shape; PATH form still leads.
urls = build_timeshift_candidate_urls(self.creds, "22372", "2026-05-12_19-00", 40)
self.assertTrue(self._is_path_form(urls[0]))
class ConvertTimestampToProviderTzTests(TestCase):
"""`convert_timestamp_to_provider_tz` shifts a UTC catch-up timestamp into the
serving provider's local zone (XC providers index archives in their own zone),
DST-correct, and is a no-op when the zone is UTC/unknown/missing."""
def test_utc_to_brussels_summer_is_plus_two(self):
# June → CEST (+02:00): 17:00 UTC == 19:00 Brussels (the 19h JT case).
self.assertEqual(
convert_timestamp_to_provider_tz("2026-06-08:17-00", "Europe/Brussels"),
"2026-06-08:19-00",
)
def test_utc_to_brussels_winter_is_plus_one(self):
# January → CET (+01:00): 17:00 UTC == 18:00 Brussels (DST handled).
self.assertEqual(
convert_timestamp_to_provider_tz("2026-01-08:17-00", "Europe/Brussels"),
"2026-01-08:18-00",
)
def test_day_rollover(self):
# 23:30 UTC + 2h (CEST) crosses midnight into the next day.
self.assertEqual(
convert_timestamp_to_provider_tz("2026-06-08:23-30", "Europe/Brussels"),
"2026-06-09:01-30",
)
def test_underscore_input_returns_colon_dash(self):
self.assertEqual(
convert_timestamp_to_provider_tz("2026-06-08_17-00", "Europe/Brussels"),
"2026-06-08:19-00",
)
def test_utc_zone_is_noop(self):
self.assertEqual(
convert_timestamp_to_provider_tz("2026-06-08:17-00", "UTC"),
"2026-06-08:17-00",
)
def test_none_zone_is_noop(self):
self.assertEqual(
convert_timestamp_to_provider_tz("2026-06-08:17-00", None),
"2026-06-08:17-00",
)
def test_unknown_zone_is_noop(self):
self.assertEqual(
convert_timestamp_to_provider_tz("2026-06-08:17-00", "Mars/Phobos"),
"2026-06-08:17-00",
)
def test_utc_to_brussels_from_unix_epoch(self):
epoch = str(int(datetime(2026, 6, 8, 17, 0, 0, tzinfo=timezone.utc).timestamp()))
self.assertEqual(
convert_timestamp_to_provider_tz(epoch, "Europe/Brussels"),
"2026-06-08:19-00",
)
def test_garbage_timestamp_passthrough(self):
self.assertEqual(
convert_timestamp_to_provider_tz("garbage", "Europe/Brussels"),
"garbage",
)
class GetProgrammeDurationTests(TestCase):
"""Duration window resolution: programme length + buffer, capped, with a
safe default whenever the EPG lookup cannot resolve."""
def _channel_with_programme(self, minutes):
from datetime import datetime, timedelta, timezone as dt_timezone
from unittest.mock import MagicMock
start = datetime(2026, 6, 8, 17, 0, tzinfo=dt_timezone.utc)
programme = MagicMock(
start_time=start, end_time=start + timedelta(minutes=minutes)
)
channel = MagicMock()
channel.epg_data.programs.filter.return_value.first.return_value = programme
return channel
def test_duration_is_programme_length_plus_buffer(self):
from apps.timeshift.helpers import get_programme_duration
# 40-minute programme + 5-minute buffer.
self.assertEqual(
get_programme_duration(self._channel_with_programme(40), "2026-06-08:17-00"),
45,
)
def test_duration_capped_at_max(self):
from apps.timeshift.helpers import get_programme_duration
self.assertEqual(
get_programme_duration(self._channel_with_programme(1000), "2026-06-08:17-00"),
480,
)
def test_no_epg_data_falls_back_to_default(self):
from unittest.mock import MagicMock
from apps.timeshift.helpers import get_programme_duration
channel = MagicMock(epg_data=None)
self.assertEqual(get_programme_duration(channel, "2026-06-08:17-00"), 120)
def test_no_matching_programme_falls_back_to_default(self):
from unittest.mock import MagicMock
from apps.timeshift.helpers import get_programme_duration
channel = MagicMock()
channel.epg_data.programs.filter.return_value.first.return_value = None
self.assertEqual(get_programme_duration(channel, "2026-06-08:17-00"), 120)
def test_garbage_timestamp_falls_back_to_default(self):
from unittest.mock import MagicMock
from apps.timeshift.helpers import get_programme_duration
self.assertEqual(get_programme_duration(MagicMock(), "garbage"), 120)

2050
apps/timeshift/tests/test_views.py Normal file
View file

File diff suppressed because it is too large Load diff

1355
apps/timeshift/views.py Normal file
View file

File diff suppressed because it is too large Load diff

3
core/api_views.py
View file

@ -225,7 +225,8 @@ class ProxySettingsViewSet(viewsets.ViewSet):
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 5,
"channel_init_grace_period": 60,
"channel_client_wait_period": 5,
"new_client_behind_seconds": 5,
}
settings_obj, created = CoreSettings.objects.get_or_create(

51
core/migrations/0026_add_channel_client_wait_period.py Normal file
View file

@ -0,0 +1,51 @@
from django.db import migrations
PROXY_SETTINGS_KEY = "proxy_settings"
def add_channel_client_wait_period(apps, schema_editor):
CoreSettings = apps.get_model("core", "CoreSettings")
try:
obj = CoreSettings.objects.get(key=PROXY_SETTINGS_KEY)
except CoreSettings.DoesNotExist:
return
value = obj.value if isinstance(obj.value, dict) else {}
# Add the new client-connect grace period default.
value.setdefault("channel_client_wait_period", 5)
# channel_init_grace_period was repurposed in 0.27.1 as the channel startup
# timeout (replacing a hardcoded 10s). Values below the new 60s default are
# too short when a channel has many failover streams to cycle through.
current_init = value.get("channel_init_grace_period", 5)
if current_init < 60:
value["channel_init_grace_period"] = 60
obj.value = value
obj.save()
def remove_channel_client_wait_period(apps, schema_editor):
CoreSettings = apps.get_model("core", "CoreSettings")
try:
obj = CoreSettings.objects.get(key=PROXY_SETTINGS_KEY)
except CoreSettings.DoesNotExist:
return
value = obj.value if isinstance(obj.value, dict) else {}
value.pop("channel_client_wait_period", None)
obj.value = value
obj.save()
class Migration(migrations.Migration):
dependencies = [
("core", "0025_move_preferred_region_and_auto_import_to_system_settings"),
]
operations = [
migrations.RunPython(add_channel_client_wait_period, remove_channel_client_wait_period),
]

13
core/models.py
View file

@ -280,8 +280,18 @@ class CoreSettings(models.Model):
"epg_match_ignore_prefixes": [],
"epg_match_ignore_suffixes": [],
"epg_match_ignore_custom": [],
# XC catch-up: forced XMLTV lookback (0 = auto-detect, capped at 30).
"xmltv_prev_days_override": 0,
})
@classmethod
def get_xmltv_prev_days_override(cls):
"""Global XC XMLTV prev_days default (0 = auto-detect from provider archives)."""
try:
return int(cls.get_epg_settings().get("xmltv_prev_days_override", 0) or 0)
except (TypeError, ValueError):
return 0
@classmethod
def _safe_string_list(cls, value):
"""Return a list of strings, filtering out non-list or non-string values."""
@ -394,7 +404,8 @@ class CoreSettings(models.Model):
"buffering_speed": 1.0,
"redis_chunk_ttl": 60,
"channel_shutdown_delay": 0,
"channel_init_grace_period": 5,
"channel_init_grace_period": 60,
"channel_client_wait_period": 5,
"new_client_behind_seconds": 5,
})

16
core/serializers.py
View file

@ -96,7 +96,8 @@ class ProxySettingsSerializer(serializers.Serializer):
buffering_speed = serializers.FloatField(min_value=0.1, max_value=10.0)
redis_chunk_ttl = serializers.IntegerField(min_value=10, max_value=3600)
channel_shutdown_delay = serializers.IntegerField(min_value=0, max_value=300)
channel_init_grace_period = serializers.IntegerField(min_value=0, max_value=60)
channel_init_grace_period = serializers.IntegerField(min_value=0, max_value=300)
channel_client_wait_period = serializers.IntegerField(min_value=0, max_value=300, required=False, default=5)
new_client_behind_seconds = serializers.IntegerField(min_value=0, max_value=120, required=False, default=5)
def validate_buffering_timeout(self, value):
@ -120,8 +121,17 @@ class ProxySettingsSerializer(serializers.Serializer):
return value
def validate_channel_init_grace_period(self, value):
if value < 0 or value > 60:
raise serializers.ValidationError("Channel init grace period must be between 0 and 60 seconds")
if value < 0 or value > 300:
raise serializers.ValidationError(
"Channel initialization timeout must be between 0 and 300 seconds"
)
return value
def validate_channel_client_wait_period(self, value):
if value < 0 or value > 300:
raise serializers.ValidationError(
"Client connect grace period must be between 0 and 300 seconds"
)
return value
def validate_new_client_behind_seconds(self, value):

8
core/tasks.py
View file

@ -398,12 +398,16 @@ def scan_and_process_files():
def _rebuild_programme_indices():
"""Queue index builds for active EPG sources that are missing their DB index."""
try:
from django.db.models import Q
from apps.epg.tasks import build_programme_index_task
sources = EPGSource.objects.filter(
is_active=True,
programme_index__isnull=True,
).exclude(source_type__in=('dummy', 'schedules_direct'))
).exclude(
source_type__in=('dummy', 'schedules_direct')
).filter(
Q(index_record__isnull=True) | Q(index_record__data__isnull=True)
)
count = 0
for source in sources:

19
core/tests.py
View file

@ -1,8 +1,8 @@
from unittest.mock import patch, MagicMock
from django.test import TestCase
from django.test import TestCase, SimpleTestCase
from apps.epg.models import EPGSource
from apps.epg.models import EPGSource, EPGSourceIndex
from core.models import CoreSettings, DVR_SETTINGS_KEY, EPG_SETTINGS_KEY
@ -37,14 +37,10 @@ class DispatcharrUserAgentTests(TestCase):
class ProgrammeIndexRebuildTests(TestCase):
def test_startup_rebuild_does_not_lock_out_queued_build_task(self):
EPGSource.objects.update(
programme_index={"channels": {}, "interleaved_channels": []}
)
source = EPGSource.objects.create(
name="Missing Index",
source_type="xmltv",
is_active=True,
programme_index=None,
)
class FakeRedis:
@ -301,3 +297,14 @@ class DropDBCommandTlsTest(TestCase):
host='localhost', port=5432,
autocommit=True,
)
class MallocTrimTests(SimpleTestCase):
def test_trim_is_noop_when_libc_has_no_malloc_trim(self):
from core.utils import trim_c_allocator_heap
fake_libc = MagicMock(spec=[])
with patch('ctypes.util.find_library', return_value='libc.so.6'), patch(
'ctypes.CDLL', return_value=fake_libc
):
self.assertFalse(trim_c_allocator_heap())

48
core/utils.py
View file

@ -546,13 +546,34 @@ def monitor_memory_usage(func):
return result
return wrapper
def cleanup_memory(log_usage=False, force_collection=True):
def trim_c_allocator_heap():
"""Return unused C heap pages to the OS where supported (glibc malloc_trim)."""
try:
import ctypes
import ctypes.util
libc_name = ctypes.util.find_library("c")
if not libc_name:
return False
libc = ctypes.CDLL(libc_name)
if not hasattr(libc, "malloc_trim"):
return False
libc.malloc_trim(0)
return True
except Exception:
logger.debug("malloc_trim unavailable or failed", exc_info=True)
return False
def cleanup_memory(log_usage=False, force_collection=True, trim_heap=False):
"""
Comprehensive memory cleanup function to reduce memory footprint
Args:
log_usage: Whether to log memory usage before and after cleanup
force_collection: Whether to force garbage collection
trim_heap: Return freed C heap pages to the OS. Only use after DB
connections are closed (e.g. Celery task_postrun).
"""
logger.trace("Starting memory cleanup django memory cleanup")
# Skip logging if log level is not set to debug or more verbose (like trace)
@ -587,8 +608,33 @@ def cleanup_memory(log_usage=False, force_collection=True):
logger.debug(f"Memory after cleanup: {after_mem:.2f} MB (change: {after_mem-before_mem:.2f} MB)")
except (ImportError, Exception):
pass
if trim_heap:
trim_c_allocator_heap()
logger.trace("Memory cleanup complete for django")
def spawn_memory_trim(close_connections=False):
"""Reclaim a request's heap pages: GC, then return freed C pages to the OS.
On gevent uWSGI workers the trim runs in a spawned greenlet so it never
blocks the caller; Celery prefork workers (no gevent hub) run it inline.
Set close_connections=True when called from a streaming generator's teardown
so the pooled DB connection is released first.
"""
def _run():
cleanup_memory(force_collection=True, trim_heap=True)
if close_connections:
from django.db import close_old_connections
close_old_connections()
if _is_gevent_monkey_patched():
import gevent
gevent.spawn(_run)
else:
_run()
def safe_upload_path(filename: str, base_dir) -> str:
"""Return a safe absolute path for an uploaded file within base_dir.

3
dispatcharr/celery.py
View file

@ -99,6 +99,7 @@ def cleanup_task_memory(**kwargs):
memory_intensive_tasks = [
'apps.m3u.tasks.refresh_single_m3u_account',
'apps.m3u.tasks.refresh_m3u_accounts',
'apps.m3u.tasks.refresh_m3u_groups',
'apps.m3u.tasks.process_m3u_batch',
'apps.m3u.tasks.process_xc_category',
'apps.m3u.tasks.sync_auto_channels',
@ -121,7 +122,7 @@ def cleanup_task_memory(**kwargs):
from core.utils import cleanup_memory
# Use the comprehensive cleanup function
cleanup_memory(log_usage=True, force_collection=True)
cleanup_memory(log_usage=True, force_collection=True, trim_heap=True)
# Log memory usage if psutil is installed
try:

1
dispatcharr/settings.py
View file

@ -99,6 +99,7 @@ INSTALLED_APPS = [
"django_filters",
"django_celery_beat",
"apps.plugins",
"apps.timeshift.apps.TimeshiftConfig",
]
# EPG Processing optimization settings

58
dispatcharr/settings_test.py Normal file
View file

@ -0,0 +1,58 @@
"""
Django settings for running the backend test suite in isolation.
Always use this module instead of dispatcharr.settings when running tests:
python manage.py test
`manage.py` selects this module automatically for the ``test`` command.
Django creates a separate empty database (``test_<POSTGRES_DB>``) and runs
migrations — your live data under /data/db is not used.
Why not dispatcharr.settings?
- Production/AIO points at the live ``dispatcharr`` database.
- django-db-geventpool breaks TestCase transaction isolation on pooled connections.
SQLite (``TEST_USE_SQLITE=1``) is an optional fallback for machines without
Postgres; production and CI should use the default Postgres test database.
"""
import os
from dispatcharr.settings import * # noqa: F401,F403
# Fast password hashing for tests.
PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
# Do NOT run Celery tasks inline during tests. post_save signals on M3UAccount and
# EPGSource call .delay(); eager mode runs them inside TestCase transactions and
# closes/poisons the DB connection for subsequent queries in the same test.
CELERY_TASK_ALWAYS_EAGER = False
CELERY_TASK_EAGER_PROPAGATES = False
_use_sqlite = os.environ.get("TEST_USE_SQLITE", "").lower() in ("1", "true", "yes")
if _use_sqlite:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": ":memory:",
}
}
else:
# Default: PostgreSQL with Django-managed test_dispatcharr (matches production).
# Uses the standard backend (not geventpool) so TestCase transactions isolate.
_pg_name = os.environ.get("POSTGRES_DB", "dispatcharr")
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": _pg_name,
"USER": os.environ.get("POSTGRES_USER", "dispatch"),
"PASSWORD": os.environ.get("POSTGRES_PASSWORD", "secret"),
"HOST": os.environ.get("POSTGRES_HOST", "localhost"),
"PORT": int(os.environ.get("POSTGRES_PORT", 5432)),
"TEST": {
"NAME": "test_" + _pg_name,
},
}
}

6
dispatcharr/urls.py
View file

@ -7,6 +7,7 @@ from .routing import websocket_urlpatterns
from apps.output.views import xc_player_api, xc_panel_api, xc_get, xc_xmltv
from apps.proxy.live_proxy.views import stream_xc
from apps.proxy.vod_proxy.views import stream_xc_movie, stream_xc_episode
from apps.timeshift.views import timeshift_proxy
urlpatterns = [
# API Routes
@ -41,6 +42,11 @@ urlpatterns = [
stream_xc,
name="xc_stream_endpoint",
),
path(
"timeshift/<str:username>/<str:password>/<str:stream_id>/<str:timestamp>/<str:duration>",
timeshift_proxy,
name="timeshift_proxy",
),
# XC VOD endpoints
path(
"movie/<str:username>/<str:password>/<str:stream_id>.<str:extension>",

240
frontend/package-lock.json generated
View file

@ -57,7 +57,7 @@
"globals": "^15.15.0",
"jsdom": "^27.0.0",
"prettier": "^3.5.3",
"vite": "^7.1.7",
"vite": "^7.3.5",
"vitest": "^4.1.8"
}
},
@ -595,9 +595,9 @@
"license": "MIT"
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.3.tgz",
"integrity": "sha512-9fJMTNFTWZMh5qwrBItuziu834eOCUcEqymSH7pY+zoMVEZg3gcPuBNxH1EvfVYe9h0x/Ptw8KBzv7qxb7l8dg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz",
"integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==",
"cpu": [
"ppc64"
],
@ -612,9 +612,9 @@
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.3.tgz",
"integrity": "sha512-i5D1hPY7GIQmXlXhs2w8AWHhenb00+GxjxRncS2ZM7YNVGNfaMxgzSGuO8o8SJzRc/oZwU2bcScvVERk03QhzA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz",
"integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==",
"cpu": [
"arm"
],
@ -629,9 +629,9 @@
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.3.tgz",
"integrity": "sha512-YdghPYUmj/FX2SYKJ0OZxf+iaKgMsKHVPF1MAq/P8WirnSpCStzKJFjOjzsW0QQ7oIAiccHdcqjbHmJxRb/dmg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz",
"integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==",
"cpu": [
"arm64"
],
@ -646,9 +646,9 @@
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.3.tgz",
"integrity": "sha512-IN/0BNTkHtk8lkOM8JWAYFg4ORxBkZQf9zXiEOfERX/CzxW3Vg1ewAhU7QSWQpVIzTW+b8Xy+lGzdYXV6UZObQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz",
"integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==",
"cpu": [
"x64"
],
@ -663,9 +663,9 @@
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.3.tgz",
"integrity": "sha512-Re491k7ByTVRy0t3EKWajdLIr0gz2kKKfzafkth4Q8A5n1xTHrkqZgLLjFEHVD+AXdUGgQMq+Godfq45mGpCKg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz",
"integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==",
"cpu": [
"arm64"
],
@ -680,9 +680,9 @@
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.3.tgz",
"integrity": "sha512-vHk/hA7/1AckjGzRqi6wbo+jaShzRowYip6rt6q7VYEDX4LEy1pZfDpdxCBnGtl+A5zq8iXDcyuxwtv3hNtHFg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz",
"integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==",
"cpu": [
"x64"
],
@ -697,9 +697,9 @@
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.3.tgz",
"integrity": "sha512-ipTYM2fjt3kQAYOvo6vcxJx3nBYAzPjgTCk7QEgZG8AUO3ydUhvelmhrbOheMnGOlaSFUoHXB6un+A7q4ygY9w==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz",
"integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==",
"cpu": [
"arm64"
],
@ -714,9 +714,9 @@
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.3.tgz",
"integrity": "sha512-dDk0X87T7mI6U3K9VjWtHOXqwAMJBNN2r7bejDsc+j03SEjtD9HrOl8gVFByeM0aJksoUuUVU9TBaZa2rgj0oA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz",
"integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==",
"cpu": [
"x64"
],
@ -731,9 +731,9 @@
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.3.tgz",
"integrity": "sha512-s6nPv2QkSupJwLYyfS+gwdirm0ukyTFNl3KTgZEAiJDd+iHZcbTPPcWCcRYH+WlNbwChgH2QkE9NSlNrMT8Gfw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz",
"integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==",
"cpu": [
"arm"
],
@ -748,9 +748,9 @@
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.3.tgz",
"integrity": "sha512-sZOuFz/xWnZ4KH3YfFrKCf1WyPZHakVzTiqji3WDc0BCl2kBwiJLCXpzLzUBLgmp4veFZdvN5ChW4Eq/8Fc2Fg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz",
"integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==",
"cpu": [
"arm64"
],
@ -765,9 +765,9 @@
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.3.tgz",
"integrity": "sha512-yGlQYjdxtLdh0a3jHjuwOrxQjOZYD/C9PfdbgJJF3TIZWnm/tMd/RcNiLngiu4iwcBAOezdnSLAwQDPqTmtTYg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz",
"integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==",
"cpu": [
"ia32"
],
@ -782,9 +782,9 @@
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.3.tgz",
"integrity": "sha512-WO60Sn8ly3gtzhyjATDgieJNet/KqsDlX5nRC5Y3oTFcS1l0KWba+SEa9Ja1GfDqSF1z6hif/SkpQJbL63cgOA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz",
"integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==",
"cpu": [
"loong64"
],
@ -799,9 +799,9 @@
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.3.tgz",
"integrity": "sha512-APsymYA6sGcZ4pD6k+UxbDjOFSvPWyZhjaiPyl/f79xKxwTnrn5QUnXR5prvetuaSMsb4jgeHewIDCIWljrSxw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz",
"integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==",
"cpu": [
"mips64el"
],
@ -816,9 +816,9 @@
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.3.tgz",
"integrity": "sha512-eizBnTeBefojtDb9nSh4vvVQ3V9Qf9Df01PfawPcRzJH4gFSgrObw+LveUyDoKU3kxi5+9RJTCWlj4FjYXVPEA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz",
"integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==",
"cpu": [
"ppc64"
],
@ -833,9 +833,9 @@
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.3.tgz",
"integrity": "sha512-3Emwh0r5wmfm3ssTWRQSyVhbOHvqegUDRd0WhmXKX2mkHJe1SFCMJhagUleMq+Uci34wLSipf8Lagt4LlpRFWQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz",
"integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==",
"cpu": [
"riscv64"
],
@ -850,9 +850,9 @@
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.3.tgz",
"integrity": "sha512-pBHUx9LzXWBc7MFIEEL0yD/ZVtNgLytvx60gES28GcWMqil8ElCYR4kvbV2BDqsHOvVDRrOxGySBM9Fcv744hw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz",
"integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==",
"cpu": [
"s390x"
],
@ -867,9 +867,9 @@
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.3.tgz",
"integrity": "sha512-Czi8yzXUWIQYAtL/2y6vogER8pvcsOsk5cpwL4Gk5nJqH5UZiVByIY8Eorm5R13gq+DQKYg0+JyQoytLQas4dA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz",
"integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==",
"cpu": [
"x64"
],
@ -884,9 +884,9 @@
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.3.tgz",
"integrity": "sha512-sDpk0RgmTCR/5HguIZa9n9u+HVKf40fbEUt+iTzSnCaGvY9kFP0YKBWZtJaraonFnqef5SlJ8/TiPAxzyS+UoA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz",
"integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==",
"cpu": [
"arm64"
],
@ -901,9 +901,9 @@
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.3.tgz",
"integrity": "sha512-P14lFKJl/DdaE00LItAukUdZO5iqNH7+PjoBm+fLQjtxfcfFE20Xf5CrLsmZdq5LFFZzb5JMZ9grUwvtVYzjiA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz",
"integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==",
"cpu": [
"x64"
],
@ -918,9 +918,9 @@
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.3.tgz",
"integrity": "sha512-AIcMP77AvirGbRl/UZFTq5hjXK+2wC7qFRGoHSDrZ5v5b8DK/GYpXW3CPRL53NkvDqb9D+alBiC/dV0Fb7eJcw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz",
"integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==",
"cpu": [
"arm64"
],
@ -935,9 +935,9 @@
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.3.tgz",
"integrity": "sha512-DnW2sRrBzA+YnE70LKqnM3P+z8vehfJWHXECbwBmH/CU51z6FiqTQTHFenPlHmo3a8UgpLyH3PT+87OViOh1AQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz",
"integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==",
"cpu": [
"x64"
],
@ -952,9 +952,9 @@
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.3.tgz",
"integrity": "sha512-NinAEgr/etERPTsZJ7aEZQvvg/A6IsZG/LgZy+81wON2huV7SrK3e63dU0XhyZP4RKGyTm7aOgmQk0bGp0fy2g==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz",
"integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==",
"cpu": [
"arm64"
],
@ -969,9 +969,9 @@
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.3.tgz",
"integrity": "sha512-PanZ+nEz+eWoBJ8/f8HKxTTD172SKwdXebZ0ndd953gt1HRBbhMsaNqjTyYLGLPdoWHy4zLU7bDVJztF5f3BHA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz",
"integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==",
"cpu": [
"x64"
],
@ -986,9 +986,9 @@
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.3.tgz",
"integrity": "sha512-B2t59lWWYrbRDw/tjiWOuzSsFh1Y/E95ofKz7rIVYSQkUYBjfSgf6oeYPNWHToFRr2zx52JKApIcAS/D5TUBnA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz",
"integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==",
"cpu": [
"arm64"
],
@ -1003,9 +1003,9 @@
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.3.tgz",
"integrity": "sha512-QLKSFeXNS8+tHW7tZpMtjlNb7HKau0QDpwm49u0vUp9y1WOF+PEzkU84y9GqYaAVW8aH8f3GcBck26jh54cX4Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz",
"integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==",
"cpu": [
"ia32"
],
@ -1020,9 +1020,9 @@
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.3.tgz",
"integrity": "sha512-4uJGhsxuptu3OcpVAzli+/gWusVGwZZHTlS63hh++ehExkVT8SgiEf7/uC/PclrPPkLhZqGgCTjd0VWLo6xMqA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz",
"integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==",
"cpu": [
"x64"
],
@ -3163,9 +3163,9 @@
"license": "MIT"
},
"node_modules/esbuild": {
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.3.tgz",
"integrity": "sha512-8VwMnyGCONIs6cWue2IdpHxHnAjzxnw2Zr7MkVxB2vjmQ2ivqGFb4LEG3SMnv0Gb2F/G/2yA8zUaiL1gywDCCg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz",
"integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
@ -3176,32 +3176,32 @@
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.27.3",
"@esbuild/android-arm": "0.27.3",
"@esbuild/android-arm64": "0.27.3",
"@esbuild/android-x64": "0.27.3",
"@esbuild/darwin-arm64": "0.27.3",
"@esbuild/darwin-x64": "0.27.3",
"@esbuild/freebsd-arm64": "0.27.3",
"@esbuild/freebsd-x64": "0.27.3",
"@esbuild/linux-arm": "0.27.3",
"@esbuild/linux-arm64": "0.27.3",
"@esbuild/linux-ia32": "0.27.3",
"@esbuild/linux-loong64": "0.27.3",
"@esbuild/linux-mips64el": "0.27.3",
"@esbuild/linux-ppc64": "0.27.3",
"@esbuild/linux-riscv64": "0.27.3",
"@esbuild/linux-s390x": "0.27.3",
"@esbuild/linux-x64": "0.27.3",
"@esbuild/netbsd-arm64": "0.27.3",
"@esbuild/netbsd-x64": "0.27.3",
"@esbuild/openbsd-arm64": "0.27.3",
"@esbuild/openbsd-x64": "0.27.3",
"@esbuild/openharmony-arm64": "0.27.3",
"@esbuild/sunos-x64": "0.27.3",
"@esbuild/win32-arm64": "0.27.3",
"@esbuild/win32-ia32": "0.27.3",
"@esbuild/win32-x64": "0.27.3"
"@esbuild/aix-ppc64": "0.28.1",
"@esbuild/android-arm": "0.28.1",
"@esbuild/android-arm64": "0.28.1",
"@esbuild/android-x64": "0.28.1",
"@esbuild/darwin-arm64": "0.28.1",
"@esbuild/darwin-x64": "0.28.1",
"@esbuild/freebsd-arm64": "0.28.1",
"@esbuild/freebsd-x64": "0.28.1",
"@esbuild/linux-arm": "0.28.1",
"@esbuild/linux-arm64": "0.28.1",
"@esbuild/linux-ia32": "0.28.1",
"@esbuild/linux-loong64": "0.28.1",
"@esbuild/linux-mips64el": "0.28.1",
"@esbuild/linux-ppc64": "0.28.1",
"@esbuild/linux-riscv64": "0.28.1",
"@esbuild/linux-s390x": "0.28.1",
"@esbuild/linux-x64": "0.28.1",
"@esbuild/netbsd-arm64": "0.28.1",
"@esbuild/netbsd-x64": "0.28.1",
"@esbuild/openbsd-arm64": "0.28.1",
"@esbuild/openbsd-x64": "0.28.1",
"@esbuild/openharmony-arm64": "0.28.1",
"@esbuild/sunos-x64": "0.28.1",
"@esbuild/win32-arm64": "0.28.1",
"@esbuild/win32-ia32": "0.28.1",
"@esbuild/win32-x64": "0.28.1"
}
},
"node_modules/escape-string-regexp": {
@ -3814,16 +3814,26 @@
"license": "MIT"
},
"node_modules/js-yaml": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"version": "5.1.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-5.1.0.tgz",
"integrity": "sha512-s8VA5jkR8f22S3NAXmhKPFqGUduqZGlsufabVOgN14iTdw/RXcym7bKkbwjxLK9Yw2lEvvmJjFp119+KPeo8Kg==",
"dev": true,
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/puzrin"
},
{
"type": "github",
"url": "https://github.com/sponsors/nodeca"
}
],
"license": "MIT",
"dependencies": {
"argparse": "^2.0.1"
},
"bin": {
"js-yaml": "bin/js-yaml.js"
"js-yaml": "bin/js-yaml.mjs"
}
},
"node_modules/jsdom": {
@ -5425,9 +5435,9 @@
}
},
"node_modules/vite": {
"version": "7.3.2",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.2.tgz",
"integrity": "sha512-Bby3NOsna2jsjfLVOHKes8sGwgl4TT0E6vvpYgnAYDIF/tie7MRaFthmKuHx1NSXjiTueXH3do80FMQgvEktRg==",
"version": "7.3.5",
"resolved": "https://registry.npmjs.org/vite/-/vite-7.3.5.tgz",
"integrity": "sha512-KuOaNhcnGFN2zIPGA7wRmzF+lJA1sea7rHq17aiJ++9lzY1WWG6Jpwqwe1KNbRVPIqHmr8GLYx7jbrQcN/7/ww==",
"dev": true,
"license": "MIT",
"dependencies": {

7
frontend/package.json
View file

@ -61,16 +61,17 @@
"globals": "^15.15.0",
"jsdom": "^27.0.0",
"prettier": "^3.5.3",
"vite": "^7.1.7",
"vite": "^7.3.5",
"vitest": "^4.1.8"
},
"resolutions": {
"vite": "7.1.7",
"vite": "7.3.5",
"react": "19.1.0",
"react-dom": "19.1.0"
},
"overrides": {
"js-yaml": "^4.1.1",
"esbuild": "^0.28.1",
"js-yaml": "^5.1.0",
"minimatch": "^10.2.1"
}
}

2
frontend/src/WebSocket.jsx
View file

@ -203,7 +203,7 @@ export const WebsocketProvider = ({ children }) => {
notifications.update({
id,
title: 'Commercials removed',
message: `${title} — kept ${parsedEvent.data.segments_kept} segments`,
message: `${title}: kept ${parsedEvent.data.segments_kept} segments`,
color: 'green.5',
loading: false,
autoClose: 4000,

133
frontend/src/components/FloatingVideo.jsx
View file

@ -16,6 +16,20 @@ import {
savePlayerPrefs,
} from '../utils/components/FloatingVideoUtils.js';
// Native <video src> cannot send Authorization headers. Append ?token= at playback
// time (not when building the URL in cards/modals) so the JWT is fresh. hls.js
// paths authenticate via xhrSetup instead and do not need this helper.
const withRecordingAuthToken = (url) => {
if (!url || typeof url !== 'string') return url;
if (!url.includes('/api/channels/recordings/')) return url;
const token = useAuthStore.getState().accessToken;
if (!token) return url;
const [base, query = ''] = url.split('?');
const params = new URLSearchParams(query);
params.set('token', token);
return `${base}?${params.toString()}`;
};
const ResizeHandles = ({ startResize }) => {
const HANDLE_SIZE = 18;
const HANDLE_OFFSET = 0;
@ -333,72 +347,75 @@ export default function FloatingVideo() {
let hls = null;
if (isHls && Hls.isSupported()) {
hls = new Hls({
// Open at the very beginning of the recording rather than the live
// edge. Without this, an in-progress recording would start at "now"
// and hide everything already recorded. hls.js applies this AFTER
// MEDIA_ATTACHED, so the listener-driven `seekToStart()` above is
// also kept as a safety net for the Safari native-HLS path and for
// edge cases where this initial-position logic loses to the user's
// first interaction.
startPosition: 0,
// Allow seeking back to the start of the recording, regardless of
// current playhead position. Recordings can be hours long and the
// user may want to scrub anywhere; we explicitly disable buffer
// eviction by setting a very large back-buffer length.
backBufferLength: 90 * 60, // 90 minutes
maxBufferLength: 60,
maxMaxBufferLength: 600,
// For an in-progress recording, hls.js refreshes the playlist on
// its target-duration cadence; let it follow the live edge but keep
// the full DVR window seekable.
liveSyncDurationCount: 3,
liveMaxLatencyDurationCount: 10,
enableWorker: true,
lowLatencyMode: false,
// Inject the JWT into every playlist + segment XHR. Read the token
// from the auth store at request time rather than capturing the
// closure value at hls.js init, so a refreshed access token mid-
// playback is picked up on the next segment fetch.
xhrSetup: (xhr) => {
const token = useAuthStore.getState().accessToken;
if (token) {
xhr.setRequestHeader('Authorization', `Bearer ${token}`);
}
},
});
hls.on(Hls.Events.ERROR, (_evt, data) => {
if (data.fatal) {
// eslint-disable-next-line no-console
console.error('HLS fatal error:', data.type, data.details);
if (data.type === Hls.ErrorTypes.NETWORK_ERROR) {
try {
hls.startLoad();
} catch {
// ignore
try {
hls = new Hls({
// Open at the very beginning of the recording rather than the live
// edge. Without this, an in-progress recording would start at "now"
// and hide everything already recorded. hls.js applies this AFTER
// MEDIA_ATTACHED, so the listener-driven `seekToStart()` above is
// also kept as a safety net for the Safari native-HLS path and for
// edge cases where this initial-position logic loses to the user's
// first interaction.
startPosition: 0,
// Allow seeking back to the start of the recording, regardless of
// current playhead position. Recordings can be hours long and the
// user may want to scrub anywhere; we explicitly disable buffer
// eviction by setting a very large back-buffer length.
backBufferLength: 90 * 60, // 90 minutes
maxBufferLength: 60,
maxMaxBufferLength: 600,
// Leave liveMaxLatencyDurationCount at the hls.js default (Infinity).
// A finite value forces the playhead to the live edge during playback.
enableWorker: true,
lowLatencyMode: false,
// Inject the JWT into every playlist + segment XHR. Read the token
// from the auth store at request time rather than capturing the
// closure value at hls.js init, so a refreshed access token mid-
// playback is picked up on the next segment fetch.
xhrSetup: (xhr) => {
const token = useAuthStore.getState().accessToken;
if (token) {
xhr.setRequestHeader('Authorization', `Bearer ${token}`);
}
} else if (data.type === Hls.ErrorTypes.MEDIA_ERROR) {
try {
hls.recoverMediaError();
} catch {
// ignore
},
});
hls.on(Hls.Events.ERROR, (_evt, data) => {
if (data.fatal) {
// eslint-disable-next-line no-console
console.error('HLS fatal error:', data.type, data.details);
if (data.type === Hls.ErrorTypes.NETWORK_ERROR) {
try {
hls.startLoad();
} catch {
// ignore
}
} else if (data.type === Hls.ErrorTypes.MEDIA_ERROR) {
try {
hls.recoverMediaError();
} catch {
// ignore
}
} else {
setLoadError(`HLS playback error: ${data.details || data.type}`);
}
} else {
setLoadError(`HLS playback error: ${data.details || data.type}`);
}
}
});
hls.attachMedia(video);
hls.on(Hls.Events.MEDIA_ATTACHED, () => {
hls.loadSource(streamUrl);
});
});
hls.attachMedia(video);
hls.on(Hls.Events.MEDIA_ATTACHED, () => {
hls.loadSource(streamUrl);
});
} catch (error) {
setIsLoading(false);
setLoadError(`HLS initialization error: ${error.message}`);
return;
}
} else if (isHls && video.canPlayType('application/vnd.apple.mpegurl')) {
// Safari path: native HLS support, including seekable DVR windows.
video.src = streamUrl;
video.src = withRecordingAuthToken(streamUrl);
video.load();
} else {
// Plain progressive file (MKV/MP4): native HTML5.
video.src = streamUrl;
video.src = withRecordingAuthToken(streamUrl);
video.load();
}

36
frontend/src/components/M3URefreshNotification.jsx
View file

@ -52,6 +52,22 @@ const M3uSetupSuccess = ({ data }) => {
};
// One-line outcome summary for the notification body.
const buildStreamSummary = (data) => {
if (data.streams_processed == null && data.streams_created == null) {
return null;
}
const created = data.streams_created || 0;
const updated = data.streams_updated || 0;
const stale = data.streams_stale || 0;
const removed = data.streams_deleted || 0;
const processed = data.streams_processed || 0;
return (
`Streams: ${created} created, ${updated} updated, ` +
`${stale} marked stale, ${removed} removed. ` +
`Total processed: ${processed}.`
);
};
const buildAutoSyncSummary = (data) => {
const created = data.channels_created || 0;
const updated = data.channels_updated || 0;
@ -211,21 +227,29 @@ export default function M3URefreshNotification() {
let body = message;
let autoClose = 2000;
// Surface auto-sync counts attached to the parsing-complete event
// so the channel-side outcome appears in the notification body.
// Surface stream and auto-sync counts attached to the parsing-complete
// event so the outcome appears in the notification body.
if (data.progress == 100 && data.action === 'parsing') {
const streamSummary = buildStreamSummary(data);
const autoSyncSummary = buildAutoSyncSummary(data);
const failed = data.channels_failed || 0;
const failedDetails = Array.isArray(data.failed_stream_details)
? data.failed_stream_details
: [];
if (autoSyncSummary) {
if (streamSummary || autoSyncSummary) {
body = (
<Stack gap={4}>
<Text size="sm">{message}</Text>
<Text size="xs" c="dimmed">
{autoSyncSummary}
</Text>
{streamSummary && (
<Text size="xs" c="dimmed">
{streamSummary}
</Text>
)}
{autoSyncSummary && (
<Text size="xs" c="dimmed">
{autoSyncSummary}
</Text>
)}
{failed > 0 && failedDetails.length > 0 && (
<Button
size="xs"

94
frontend/src/components/__tests__/FloatingVideo.test.jsx
View file

@ -20,8 +20,49 @@ vi.mock('mpegts.js', () => ({
},
}));
const mockHlsInstance = {
attachMedia: vi.fn(),
loadSource: vi.fn(),
destroy: vi.fn(),
on: vi.fn(),
};
let capturedHlsConfig = null;
let forceHlsInitError = false;
vi.mock('hls.js', () => ({
default: class MockHls {
static isSupported = vi.fn(() => true);
static Events = {
ERROR: 'error',
MEDIA_ATTACHED: 'media_attached',
};
static ErrorTypes = {
NETWORK_ERROR: 'networkError',
MEDIA_ERROR: 'mediaError',
};
constructor(config) {
if (forceHlsInitError) {
throw new Error('Illegal hls.js config');
}
capturedHlsConfig = config;
Object.assign(this, mockHlsInstance);
}
},
}));
vi.mock('../../store/auth', () => ({
default: {
getState: vi.fn(() => ({ accessToken: 'test-token' })),
},
}));
// Import the mocked module after mocking
const mpegts = (await import('mpegts.js')).default;
const Hls = (await import('hls.js')).default;
// Mock react-draggable
vi.mock('react-draggable', () => ({
@ -53,6 +94,8 @@ describe('FloatingVideo', () => {
beforeEach(async () => {
vi.clearAllMocks();
capturedHlsConfig = null;
forceHlsInitError = false;
// Mock HTMLVideoElement methods
HTMLVideoElement.prototype.load = vi.fn();
@ -239,6 +282,57 @@ describe('FloatingVideo', () => {
expect(video.poster).toBe('http://example.com/poster.jpg');
});
it('should disable live-edge sync for in-progress recording HLS', () => {
useVideoStore.mockImplementation((selector) => {
const state = {
isVisible: true,
streamUrl:
'http://example.com/api/channels/recordings/1/hls/index.m3u8',
contentType: 'vod',
metadata: { name: 'News Recording' },
hideVideo: mockHideVideo,
};
return selector ? selector(state) : state;
});
Hls.isSupported.mockReturnValue(true);
render(<FloatingVideo />);
expect(capturedHlsConfig).toEqual(
expect.objectContaining({
startPosition: 0,
})
);
expect(capturedHlsConfig).not.toHaveProperty(
'liveMaxLatencyDurationCount'
);
expect(capturedHlsConfig).not.toHaveProperty('liveSyncDurationCount');
});
it('shows an in-player error when hls.js config is invalid', () => {
useVideoStore.mockImplementation((selector) => {
const state = {
isVisible: true,
streamUrl:
'http://example.com/api/channels/recordings/1/hls/index.m3u8',
contentType: 'vod',
metadata: { name: 'News Recording' },
hideVideo: mockHideVideo,
};
return selector ? selector(state) : state;
});
Hls.isSupported.mockReturnValue(true);
forceHlsInitError = true;
render(<FloatingVideo />);
expect(
screen.getByText(/HLS initialization error: Illegal hls.js config/i)
).toBeInTheDocument();
});
it('should show metadata overlay', () => {
const { container } = render(<FloatingVideo />);
const video = container.querySelector('video');

33
frontend/src/components/__tests__/M3URefreshNotification.test.jsx
View file

@ -851,4 +851,37 @@ describe('M3URefreshNotification', () => {
expect(call[0].autoClose).toBe(12000);
});
});
describe('Stream count rendering on parsing complete', () => {
it('inlines stream summary including marked stale count', async () => {
mockPlaylistsStore.refreshProgress = {
1: {
account: 1,
action: 'parsing',
progress: 100,
status: 'success',
streams_created: 2,
streams_updated: 5,
streams_stale: 18,
streams_deleted: 3,
streams_processed: 1200,
},
};
renderWithProviders(<M3URefreshNotification />);
await waitFor(() => {
expect(showNotification).toHaveBeenCalled();
});
const call = showNotification.mock.calls.find(
(c) => typeof c[0]?.message === 'object'
);
expect(call).toBeDefined();
const { container } = render(<>{call[0].message}</>);
expect(container.textContent).toContain('Stream parsing complete!');
expect(container.textContent).toContain('18 marked stale');
expect(container.textContent).toContain('3 removed');
expect(container.textContent).toContain('Total processed: 1200');
});
});
});

18
frontend/src/components/cards/StreamConnectionCard.jsx
View file

@ -22,6 +22,7 @@ import {
Gauge,
HardDriveDownload,
HardDriveUpload,
Rewind,
SquareX,
Timer,
Users,
@ -486,7 +487,10 @@ const StreamConnectionCard = ({
}, [channel.name, channel.stream_id]);
const channelName =
channel.name || previewedStream?.name || 'Unnamed Channel';
channel.name ||
channel.channel_name ||
previewedStream?.name ||
'Unnamed Channel';
const uptime = channel.uptime || 0;
const bitrates = channel.bitrates || [];
const totalBytes = channel.total_bytes || 0;
@ -661,6 +665,18 @@ const StreamConnectionCard = ({
{/* Add stream information badges */}
<Group gap="xs" mt="5">
{channel.is_timeshift && (
<Tooltip label="Catch-up (timeshift)">
<Badge
size="sm"
variant="light"
color="violet"
leftSection={<Rewind size={12} />}
>
TIMESHIFT
</Badge>
</Tooltip>
)}
{channel.resolution && (
<Tooltip label="Video resolution">
<Badge size="sm" variant="light" color="red">

13
frontend/src/components/forms/LiveGroupFilter.jsx
View file

@ -34,6 +34,7 @@ import {
getRegexOptions,
getStreamsRegexPreview,
isExpectedOccupantForGroup,
effectiveSyncGroupId,
isGroupVisible,
rangeFor,
} from '../../utils/forms/LiveGroupFilterUtils.js';
@ -131,7 +132,12 @@ const LiveGroupFilter = ({
// (in-memory range overlap with sibling groups) sources so the sweep
// can refresh form-overlap synchronously without firing HTTP for
// groups that did not change.
const scheduleConflictScan = (groupId, rawStart, rawEnd) => {
const scheduleConflictScan = (
groupId,
rawStart,
rawEnd,
expectedGroupId = groupId
) => {
if (conflictTimersRef.current[groupId]) {
clearTimeout(conflictTimersRef.current[groupId]);
}
@ -156,7 +162,7 @@ const LiveGroupFilter = ({
? result.occupants
: [];
const unexpected = occupants.filter(
(o) => !isExpectedOccupantForGroup(o, groupId, playlist)
(o) => !isExpectedOccupantForGroup(o, expectedGroupId, playlist)
);
setConflictSource(groupId, 'occupant', unexpected.length > 0);
} catch (e) {
@ -221,7 +227,8 @@ const LiveGroupFilter = ({
scheduleConflictScan(
g.channel_group,
range.startRaw,
g.auto_sync_channel_end
g.auto_sync_channel_end,
effectiveSyncGroupId(g)
);
}
}

81
frontend/src/components/forms/settings/EpgSettingsForm.jsx Normal file
View file

@ -0,0 +1,81 @@
import useSettingsStore from '../../../store/settings.jsx';
import React, { useEffect, useState } from 'react';
import { useForm } from '@mantine/form';
import {
Alert,
Button,
Flex,
NumberInput,
Stack,
Text,
} from '@mantine/core';
import { EPG_SETTINGS_OPTIONS } from '../../../constants.js';
import {
getChangedSettings,
parseSettings,
saveChangedSettings,
} from '../../../utils/pages/SettingsUtils.js';
import { getEpgSettingsFormInitialValues } from '../../../utils/forms/settings/EpgSettingsFormUtils.js';
const EpgSettingsForm = React.memo(({ active }) => {
const settings = useSettingsStore((s) => s.settings);
const [saved, setSaved] = useState(false);
const form = useForm({
mode: 'controlled',
initialValues: getEpgSettingsFormInitialValues(),
});
useEffect(() => {
if (!active) setSaved(false);
}, [active]);
useEffect(() => {
if (settings) {
const parsed = parseSettings(settings);
form.setFieldValue(
'xmltv_prev_days_override',
parsed.xmltv_prev_days_override ?? 0,
);
}
}, [settings]);
const onSubmit = async () => {
setSaved(false);
const changedSettings = getChangedSettings(form.getValues(), settings);
try {
await saveChangedSettings(settings, changedSettings);
setSaved(true);
} catch (error) {
console.error('Error saving EPG settings:', error);
}
};
const prevDaysConfig = EPG_SETTINGS_OPTIONS.xmltv_prev_days_override;
return (
<form onSubmit={form.onSubmit(onSubmit)}>
<Stack gap="md">
{saved && (
<Alert variant="light" color="green" title="Saved Successfully" />
)}
<NumberInput
label={prevDaysConfig.label}
description={prevDaysConfig.description}
min={0}
max={30}
{...form.getInputProps('xmltv_prev_days_override')}
/>
<Text size="xs" c="dimmed">
Per-user defaults and URL parameters still override this global value.
EPG channel matching options are configured from the Channels page.
</Text>
<Flex justify="flex-end">
<Button type="submit">Save</Button>
</Flex>
</Stack>
</form>
);
});
export default EpgSettingsForm;

159
frontend/src/components/forms/settings/ProxySettingsForm.jsx
View file

@ -5,80 +5,119 @@ import { updateSetting } from '../../../utils/pages/SettingsUtils.js';
import {
Alert,
Button,
Collapse,
Flex,
NumberInput,
Stack,
TextInput,
} from '@mantine/core';
import { ChevronDown, ChevronRight } from 'lucide-react';
import { PROXY_SETTINGS_OPTIONS } from '../../../constants.js';
import {
getProxySettingDefaults,
getProxySettingsFormInitialValues,
} from '../../../utils/forms/settings/ProxySettingsFormUtils.js';
const isNumericField = (key) => {
return [
'buffering_timeout',
'redis_chunk_ttl',
'channel_shutdown_delay',
'channel_init_grace_period',
'channel_client_wait_period',
'new_client_behind_seconds',
].includes(key);
};
const isFloatField = (key) => key === 'buffering_speed';
const getNumericFieldMax = (key) => {
if (key === 'buffering_timeout') return 300;
if (key === 'redis_chunk_ttl') return 3600;
if (key === 'channel_shutdown_delay') return 300;
if (key === 'channel_client_wait_period') return 300;
if (key === 'new_client_behind_seconds') return 120;
return 300;
};
const renderProxySettingField = (key, config, proxySettingsForm) => {
if (isNumericField(key)) {
return (
<NumberInput
key={key}
label={config.label}
{...proxySettingsForm.getInputProps(key)}
description={config.description || null}
min={0}
max={getNumericFieldMax(key)}
/>
);
}
if (isFloatField(key)) {
return (
<NumberInput
key={key}
label={config.label}
{...proxySettingsForm.getInputProps(key)}
description={config.description || null}
min={0.0}
max={10.0}
step={0.01}
precision={1}
/>
);
}
return (
<TextInput
key={key}
label={config.label}
{...proxySettingsForm.getInputProps(key)}
description={config.description || null}
/>
);
};
const ProxySettingsOptions = React.memo(({ proxySettingsForm }) => {
const isNumericField = (key) => {
// Determine if this field should be a NumberInput
return [
'buffering_timeout',
'redis_chunk_ttl',
'channel_shutdown_delay',
'channel_init_grace_period',
'new_client_behind_seconds',
].includes(key);
};
const isFloatField = (key) => {
return key === 'buffering_speed';
};
const getNumericFieldMax = (key) => {
return key === 'buffering_timeout'
? 300
: key === 'redis_chunk_ttl'
? 3600
: key === 'channel_shutdown_delay'
? 300
: key === 'new_client_behind_seconds'
? 120
: 60;
};
const [advancedOpen, setAdvancedOpen] = useState(false);
const entries = Object.entries(PROXY_SETTINGS_OPTIONS);
const mainEntries = entries.filter(([, config]) => !config.advanced);
const advancedEntries = entries.filter(([, config]) => config.advanced);
return (
<>
{Object.entries(PROXY_SETTINGS_OPTIONS).map(([key, config]) => {
if (isNumericField(key)) {
return (
<NumberInput
key={key}
label={config.label}
{...proxySettingsForm.getInputProps(key)}
description={config.description || null}
min={0}
max={getNumericFieldMax(key)}
/>
);
} else if (isFloatField(key)) {
return (
<NumberInput
key={key}
label={config.label}
{...proxySettingsForm.getInputProps(key)}
description={config.description || null}
min={0.0}
max={10.0}
step={0.01}
precision={1}
/>
);
} else {
return (
<TextInput
key={key}
label={config.label}
{...proxySettingsForm.getInputProps(key)}
description={config.description || null}
/>
);
}
})}
{mainEntries.map(([key, config]) =>
renderProxySettingField(key, config, proxySettingsForm)
)}
{advancedEntries.length > 0 && (
<>
<Button
variant="subtle"
size="xs"
leftSection={
advancedOpen ? (
<ChevronDown size={12} />
) : (
<ChevronRight size={12} />
)
}
onClick={() => setAdvancedOpen((open) => !open)}
c="dimmed"
styles={{ root: { alignSelf: 'flex-start' } }}
>
{advancedOpen ? 'Hide' : 'Show'} Advanced Settings
</Button>
<Collapse in={advancedOpen}>
<Stack gap="sm">
{advancedEntries.map(([key, config]) =>
renderProxySettingField(key, config, proxySettingsForm)
)}
</Stack>
</Collapse>
</>
)}
</>
);
});

46
frontend/src/components/forms/settings/__tests__/ProxySettingsForm.test.jsx
View file

@ -14,6 +14,21 @@ vi.mock('../../../../constants.js', () => ({
description: 'Speed multiplier',
},
redis_url: { label: 'Redis URL', description: 'Redis connection URL' },
redis_chunk_ttl: {
label: 'Buffer Chunk TTL',
advanced: true,
description: 'Chunk TTL',
},
channel_init_grace_period: {
label: 'Channel Initialization Timeout',
advanced: true,
description: 'Init timeout',
},
channel_client_wait_period: {
label: 'Client Connect Grace Period',
advanced: true,
description: 'Advanced grace period',
},
},
}));
@ -71,6 +86,8 @@ vi.mock('@mantine/core', () => ({
</div>
),
Stack: ({ children }) => <div>{children}</div>,
Collapse: ({ in: isOpen, children }) =>
isOpen ? <div data-testid="collapse-open">{children}</div> : null,
TextInput: ({ label, description, ...rest }) => (
<div>
<label>{label}</label>
@ -353,11 +370,38 @@ describe('ProxySettingsForm', () => {
// ── ProxySettingsOptions field routing ─────────────────────────────────────
describe('ProxySettingsOptions field routing', () => {
it('calls getInputProps for each PROXY_SETTINGS_OPTIONS key', () => {
it('binds main settings on initial render', () => {
render(<ProxySettingsForm active={true} />);
expect(formMock.getInputProps).toHaveBeenCalledWith('buffering_timeout');
expect(formMock.getInputProps).toHaveBeenCalledWith('buffering_speed');
expect(formMock.getInputProps).toHaveBeenCalledWith('redis_url');
});
it('hides advanced settings until expanded', () => {
render(<ProxySettingsForm active={true} />);
expect(
screen.queryByTestId('number-input-Client Connect Grace Period')
).not.toBeInTheDocument();
expect(
screen.queryByTestId('number-input-Channel Initialization Timeout')
).not.toBeInTheDocument();
expect(
screen.queryByTestId('number-input-Buffer Chunk TTL')
).not.toBeInTheDocument();
expect(screen.getByText('Show Advanced Settings')).toBeInTheDocument();
});
it('shows advanced settings when expanded', () => {
render(<ProxySettingsForm active={true} />);
fireEvent.click(screen.getByText('Show Advanced Settings'));
expect(screen.getByTestId('collapse-open')).toBeInTheDocument();
expect(
screen.getByTestId('number-input-Client Connect Grace Period')
).toBeInTheDocument();
expect(
screen.getByTestId('number-input-Channel Initialization Timeout')
).toBeInTheDocument();
expect(screen.getByTestId('number-input-Buffer Chunk TTL')).toBeInTheDocument();
});
});
});

23
frontend/src/constants.js
View file

@ -43,17 +43,26 @@ export const PROXY_SETTINGS_OPTIONS = {
},
redis_chunk_ttl: {
label: 'Buffer Chunk TTL',
advanced: true,
description:
'Time-to-live for buffer chunks in seconds (how long stream data is cached)',
},
channel_shutdown_delay: {
label: 'Channel Shutdown Delay',
description:
'Delay in seconds before shutting down a channel after last client disconnects',
'Delay in seconds before shutting down a channel after the last client disconnects',
},
channel_init_grace_period: {
label: 'Channel Initialization Grace Period',
description: 'Grace period in seconds during channel initialization',
label: 'Channel Initialization Timeout',
advanced: true,
description:
'Maximum seconds to wait for the initial buffer to fill while a channel is connecting. Channels that never receive enough buffered data are stopped after this limit.',
},
channel_client_wait_period: {
label: 'Client Connect Grace Period',
advanced: true,
description:
'Seconds to keep a buffered channel alive when no viewer is connected yet. Rarely needed unless you start channels programmatically.',
},
new_client_behind_seconds: {
label: 'New Client Buffer (seconds)',
@ -62,6 +71,14 @@ export const PROXY_SETTINGS_OPTIONS = {
},
};
export const EPG_SETTINGS_OPTIONS = {
xmltv_prev_days_override: {
label: 'XMLTV prev_days Override (catch-up)',
description:
'Days of past programmes in the XC EPG output. 0 = auto-detect from the providers’ tv_archive_duration (capped at 30).',
},
};
export const USER_LIMITS_OPTIONS = {
terminate_on_limit_exceeded: {
label: 'Terminate on Limit Exceeded',

16
frontend/src/pages/Settings.jsx
View file

@ -45,6 +45,9 @@ const DvrSettingsForm = React.lazy(
const SystemSettingsForm = React.lazy(
() => import('../components/forms/settings/SystemSettingsForm.jsx')
);
const EpgSettingsForm = React.lazy(
() => import('../components/forms/settings/EpgSettingsForm.jsx')
);
const NavOrderForm = React.lazy(
() => import('../components/forms/settings/NavOrderForm.jsx')
);
@ -122,6 +125,19 @@ const SettingsPage = () => {
</AccordionPanel>
</AccordionItem>
<AccordionItem value="epg-settings">
<AccordionControl>EPG</AccordionControl>
<AccordionPanel>
<ErrorBoundary>
<Suspense fallback={<Loader />}>
<EpgSettingsForm
active={accordianValue === 'epg-settings'}
/>
</Suspense>
</ErrorBoundary>
</AccordionPanel>
</AccordionItem>
<AccordionItem value="system-settings">
<AccordionControl>System Settings</AccordionControl>
<AccordionPanel>

20
frontend/src/utils/cards/__tests__/RecordingCardUtils.test.js
View file

@ -272,34 +272,36 @@ describe('RecordingCardUtils', () => {
describe('getRecordingUrl', () => {
it('returns file_url when available', () => {
const customProps = { file_url: '/recordings/file.mp4' };
const customProps = { file_url: '/api/channels/recordings/67/file/' };
const result = getRecordingUrl(customProps, 'production');
expect(result).toBe('/recordings/file.mp4');
expect(result).toBe('/api/channels/recordings/67/file/');
});
it('returns output_file_url when file_url is not available', () => {
const customProps = { output_file_url: '/output/file.mp4' };
const customProps = { output_file_url: '/api/channels/recordings/1/file/' };
const result = getRecordingUrl(customProps, 'production');
expect(result).toBe('/output/file.mp4');
expect(result).toBe('/api/channels/recordings/1/file/');
});
it('prefers file_url over output_file_url', () => {
const customProps = {
file_url: '/recordings/file.mp4',
output_file_url: '/output/file.mp4',
file_url: '/api/channels/recordings/2/file/',
output_file_url: '/api/channels/recordings/3/file/',
};
const result = getRecordingUrl(customProps, 'production');
expect(result).toBe('/recordings/file.mp4');
expect(result).toBe('/api/channels/recordings/2/file/');
});
it('prepends dev server URL in dev mode for relative paths', () => {
const customProps = { file_url: '/recordings/file.mp4' };
const customProps = { file_url: '/api/channels/recordings/4/hls/index.m3u8' };
const result = getRecordingUrl(customProps, 'dev');
expect(result).toMatch(/^https?:\/\/.*:5656\/recordings\/file\.mp4$/);
expect(result).toMatch(
/^https?:\/\/.*:5656\/api\/channels\/recordings\/4\/hls\/index\.m3u8$/
);
});
it('does not prepend dev URL for absolute URLs', () => {

12
frontend/src/utils/forms/LiveGroupFilterUtils.js
View file

@ -57,6 +57,18 @@ export const isExpectedOccupantForGroup = (
);
};
// The group the sync's own channels actually land in. A group_override
// routes auto-created channels into a different ChannelGroup, so the
// conflict check must recognize occupants of that target group as this
// config's own output rather than flagging them against the source group.
export const effectiveSyncGroupId = (group) => {
const override = group?.custom_properties?.group_override;
if (override !== undefined && override !== null && override !== '') {
return Number(override);
}
return group?.channel_group;
};
export const rangeFor = (g) => {
if (!g.enabled || !g.auto_channel_sync) return null;
const mode = g.custom_properties?.channel_numbering_mode || 'fixed';

111
frontend/src/utils/forms/__tests__/LiveGroupFilterUtils.test.js
View file

@ -4,6 +4,7 @@ import {
getChannelsInRange,
getStreamsRegexPreview,
isExpectedOccupantForGroup,
effectiveSyncGroupId,
rangeFor,
abortTimers,
getRegexOptions,
@ -227,6 +228,116 @@ describe('LiveGroupFilterUtils', () => {
});
});
// ── effectiveSyncGroupId ───────────────────────────────────────────────────
describe('effectiveSyncGroupId', () => {
it('returns the source channel_group when there is no override', () => {
expect(effectiveSyncGroupId(makeGroup({ channel_group: 7 }))).toBe(7);
});
it('returns the group_override target when set', () => {
const group = makeGroup({
channel_group: 7,
custom_properties: { group_override: 9 },
});
expect(effectiveSyncGroupId(group)).toBe(9);
});
it('coerces a string-stored group_override to a number', () => {
const group = makeGroup({
channel_group: 7,
custom_properties: { group_override: '9' },
});
expect(effectiveSyncGroupId(group)).toBe(9);
});
it('falls back to the source group when group_override is blank', () => {
const group = makeGroup({
channel_group: 7,
custom_properties: { group_override: '' },
});
expect(effectiveSyncGroupId(group)).toBe(7);
});
// Regression guard for the group-override range-conflict false positive:
// the auto-sync's own channels land in the override target group, so
// comparing against the source group (pre-fix) flags them as a conflict,
// while comparing against the effective target recognizes them as this
// config's own output.
it("makes group-override occupants count as this group's own", () => {
const group = makeGroup({
channel_group: 7,
custom_properties: { group_override: 9 },
});
const occupant = makeOccupant({ channel_group_id: 9 });
// Pre-fix comparison (source group) treats own channels as a conflict.
expect(
isExpectedOccupantForGroup(
occupant,
group.channel_group,
makePlaylist()
)
).toBe(false);
// Comparing against the effective target recognizes them as expected.
expect(
isExpectedOccupantForGroup(
occupant,
effectiveSyncGroupId(group),
makePlaylist()
)
).toBe(true);
});
// Guards against over-suppression: resolving the effective target group
// must still surface genuine collisions in an override config's range.
// Only the config's own output (auto-created, this account, in the
// target group, unpinned) is excluded.
it('still flags genuine collisions in a group-override config', () => {
const group = makeGroup({
channel_group: 7,
custom_properties: { group_override: 9 },
});
const target = effectiveSyncGroupId(group);
// Manual channel sitting in the range.
expect(
isExpectedOccupantForGroup(
makeOccupant({ channel_group_id: 9, auto_created: false }),
target,
makePlaylist()
)
).toBe(false);
// Auto-created by a different account.
expect(
isExpectedOccupantForGroup(
makeOccupant({
channel_group_id: 9,
auto_created_by_account_id: 999,
}),
target,
makePlaylist()
)
).toBe(false);
// A channel in a different group than the override target.
expect(
isExpectedOccupantForGroup(
makeOccupant({ channel_group_id: 123 }),
target,
makePlaylist()
)
).toBe(false);
// A user-pinned channel number.
expect(
isExpectedOccupantForGroup(
makeOccupant({
channel_group_id: 9,
has_channel_number_override: true,
}),
target,
makePlaylist()
)
).toBe(false);
});
});
// ── rangeFor ──────────────────────────────────────────────────────────────
describe('rangeFor', () => {
it('returns null when group is disabled', () => {

3
frontend/src/utils/forms/settings/EpgSettingsFormUtils.js Normal file
View file

@ -0,0 +1,3 @@
export const getEpgSettingsFormInitialValues = () => ({
xmltv_prev_days_override: 0,
});

3
frontend/src/utils/forms/settings/ProxySettingsFormUtils.js
View file

@ -13,7 +13,8 @@ export const getProxySettingDefaults = () => {
buffering_speed: 1.0,
redis_chunk_ttl: 60,
channel_shutdown_delay: 0,
channel_init_grace_period: 5,
channel_init_grace_period: 60,
channel_client_wait_period: 5,
new_client_behind_seconds: 5,
};
};

4
frontend/src/utils/forms/settings/__tests__/ProxySettingsFormUtils.test.js
View file

@ -60,7 +60,8 @@ describe('ProxySettingsFormUtils', () => {
buffering_speed: 1.0,
redis_chunk_ttl: 60,
channel_shutdown_delay: 0,
channel_init_grace_period: 5,
channel_init_grace_period: 60,
channel_client_wait_period: 5,
new_client_behind_seconds: 5,
});
});
@ -81,6 +82,7 @@ describe('ProxySettingsFormUtils', () => {
expect(typeof result.redis_chunk_ttl).toBe('number');
expect(typeof result.channel_shutdown_delay).toBe('number');
expect(typeof result.channel_init_grace_period).toBe('number');
expect(typeof result.channel_client_wait_period).toBe('number');
expect(typeof result.new_client_behind_seconds).toBe('number');
});
});

22
frontend/src/utils/pages/SettingsUtils.js
View file

@ -39,6 +39,7 @@ export const saveChangedSettings = async (settings, changedSettings) => {
'epg_match_ignore_prefixes',
'epg_match_ignore_suffixes',
'epg_match_ignore_custom',
'xmltv_prev_days_override',
];
const dvrFields = [
'tv_template',
@ -125,6 +126,7 @@ export const saveChangedSettings = async (settings, changedSettings) => {
'retention_count',
'schedule_day_of_week',
'max_system_events',
'xmltv_prev_days_override',
];
if (numericFields.includes(formKey) && value != null) {
value = typeof value === 'number' ? value : parseInt(value, 10);
@ -222,6 +224,20 @@ export const getChangedSettings = (values, settings) => {
continue;
}
if (settingKey === 'xmltv_prev_days_override') {
const baseline = Number(
settings['epg_settings']?.value?.xmltv_prev_days_override ?? 0,
);
const nextVal =
typeof actualValue === 'number'
? actualValue
: parseInt(actualValue, 10) || 0;
if (nextVal !== baseline) {
changedSettings[settingKey] = nextVal;
}
continue;
}
// Convert array values (like m3u_hash_key) to comma-separated strings for comparison
if (Array.isArray(actualValue)) {
actualValue = actualValue.join(',');
@ -296,6 +312,12 @@ export const parseSettings = (settings) => {
epgSettings && Array.isArray(epgSettings.epg_match_ignore_custom)
? epgSettings.epg_match_ignore_custom
: [];
parsed.xmltv_prev_days_override =
epgSettings && epgSettings.xmltv_prev_days_override != null
? typeof epgSettings.xmltv_prev_days_override === 'number'
? epgSettings.xmltv_prev_days_override
: parseInt(epgSettings.xmltv_prev_days_override, 10) || 0
: 0;
// DVR settings - direct mapping with underscore keys
const dvrSettings = settings['dvr_settings']?.value;

7
manage.py
View file

@ -5,7 +5,12 @@ import sys
def main():
"""Run administrative tasks."""
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'dispatcharr.settings')
# Use isolated test DB settings for `manage.py test` (empty test_<dbname>).
# Override with --settings=... on the command line if needed.
if len(sys.argv) > 1 and sys.argv[1] == "test":
os.environ["DJANGO_SETTINGS_MODULE"] = "dispatcharr.settings_test"
else:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "dispatcharr.settings")
try:
from django.core.management import execute_from_command_line
except ImportError as exc:

12
pyproject.toml
View file

@ -6,18 +6,18 @@ license = "AGPL-3.0-only"
requires-python = ">=3.13"
dynamic = ["version"]
dependencies = [
"Django==6.0.5",
"Django==6.0.6",
"psycopg[binary]",
"celery[redis]==5.6.3",
"djangorestframework==3.17.1",
"requests==2.33.1",
"requests==2.34.2",
"psutil==7.2.2",
"pillow",
"drf-spectacular>=0.29.0",
"streamlink",
"python-vlc",
"yt-dlp",
"gevent==26.4.0",
"gevent==26.5.0",
"django-db-geventpool",
"daphne",
"uwsgi",
@ -28,14 +28,14 @@ dependencies = [
"regex",
"tzlocal",
"pytz",
"torch==2.11.0+cpu",
"sentence-transformers==5.4.1",
"torch==2.12.1+cpu",
"sentence-transformers==5.6.0",
"channels",
"channels-redis==4.3.0",
"django-filter",
"django-redis",
"django-celery-beat>=2.9.0",
"lxml==6.1.0",
"lxml==6.1.1",
"packaging",
]

4
tests/test_process_label.py
View file

@ -17,13 +17,13 @@ class ProcessLabelTests(SimpleTestCase):
self.assertEqual(role, "uwsgi")
def test_uwsgi_labeled_when_worker_module_present(self):
fake_uwsgi = type("uwsgi", (), {"worker_id": lambda: 2})()
fake_uwsgi = type("uwsgi", (), {"worker_id": staticmethod(lambda: 2)})()
with patch.dict("sys.modules", {"uwsgi": fake_uwsgi}):
role = get_process_role(["/dispatcharrpy/bin/python", "-c", "pass"])
self.assertEqual(role, "uwsgi")
def test_uwsgi_master_not_labeled_as_uwsgi(self):
fake_uwsgi = type("uwsgi", (), {"worker_id": lambda: 0})()
fake_uwsgi = type("uwsgi", (), {"worker_id": staticmethod(lambda: 0)})()
with patch.dict("sys.modules", {"uwsgi": fake_uwsgi}):
role = get_process_role(["/dispatcharrpy/bin/python", "-c", "pass"])
self.assertEqual(role, "django")

4
tests/test_user_preferences.py
View file

@ -128,13 +128,13 @@ class UserPreferencesAPITests(TestCase):
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
def test_patch_me_cannot_escalate_privileges(self):
"""Test PATCH /me/ rejects attempts to change user_level or is_staff"""
"""PATCH /me/ ignores privilege fields; they are stripped before save."""
original_level = self.user.user_level
data = {"user_level": 99, "is_staff": True, "is_superuser": True}
response = self.client.patch(self.me_url, data, format="json")
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.user.refresh_from_db()
self.assertEqual(self.user.user_level, original_level)

2
version.py
View file

@ -1,5 +1,5 @@
"""
Dispatcharr version information.
"""
__version__ = '0.27.0' # Follow semantic versioning (MAJOR.MINOR.PATCH)
__version__ = '0.27.2' # Follow semantic versioning (MAJOR.MINOR.PATCH)
__timestamp__ = None # Set during CI/CD build process