diff --git a/CHANGELOG.md b/CHANGELOG.md index 38c590bd..83cf0f3e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Watch DVR recordings live while they are still recording**: `run_recording` has been refactored from a single TS file capture to an FFmpeg HLS segmentation pipeline. While recording is in progress, the `file_url` exposed on the recording points to a new `/api/channels/recordings/{id}/hls/index.m3u8` endpoint instead of the eventual MKV path, so any HLS-capable client (the built-in web player, VLC, infuse, channels DVR clients, etc.) can join the stream at any time and watch from the live edge. When the recording ends, the segments are concatenated into the final MKV, `file_url` is updated to point to the existing `/file/` endpoint, and the HLS working directory is cleaned up. Hitting `/file/` while a recording is still in progress now redirects to the HLS playlist rather than 404, and hitting `/hls/index.m3u8` after the recording has finalized redirects to `/file/`, so URLs cached by clients in either form continue to work across the transition. +- New HLS playback endpoint (`RecordingViewSet.hls`) serves `.m3u8` and `.ts` files out of the recording's working directory with path traversal protection (`os.path.realpath` containment check) and `AllowAny` permissions consistent with the existing `/file/` endpoint. Playlist files are rewritten on the fly so each segment line becomes an absolute URL routed back through this endpoint, preserving authentication and path isolation. +- Explicit `path('recordings//hls/', ...)` route registered in `apps/channels/api_urls.py` _before_ `router.urls`. DRF's `DefaultRouter` appends a mandatory trailing slash to every URL it generates, but HLS players request segments by their natural filenames (`seg_00001.ts`) without a trailing slash, so the explicit route is required for the player to ever reach the view. +- `DISPATCHARR_WEB_HOST` environment variable for modular deployments. The Celery container needs to reach the uWSGI / web container to fetch HLS segments while recording (FFmpeg pulls from the public stream URL, the same way any other client does). `get_dvr_stream_base_url()` resolves the base URL deterministically: AIO / dev / debug containers reach uWSGI on `127.0.0.1:$DISPATCHARR_PORT` since Celery and uWSGI share the container, while modular deployments use `http://$DISPATCHARR_WEB_HOST:$DISPATCHARR_PORT` and default `DISPATCHARR_WEB_HOST` to `web` (the compose service name). Documented as a commented-out override in `docker/docker-compose.yml`. +- HLS viewer heartbeat. Every `.ts` request refreshes a Redis key `dvr:hls_viewer:{id}` with a 20 second TTL. After concat succeeds, the recording task waits for that key to expire naturally before deleting the HLS working directory, so an actively-watching client is never cut off mid-segment when the recording ends. Cleanup happens within 20 seconds of the last client stopping, with a 4 hour safety cap as a guard against a stuck Redis state. - **VOD start/stop notifications**: the frontend now shows a toast notification when a VOD stream starts or stops. `vod_started` and `vod_stopped` WebSocket events are fired from the backend when a new provider connection is opened or the last active stream on a session ends. The 1-second delayed-cleanup window is used as a settle period before firing `vod_stopped`, so seek reconnects that re-establish `active_streams` within that window suppress the notification. A `vod_stats` WebSocket push is sent alongside every event to keep the Stats page connection table in sync in real time. `vod_start` and `vod_stop` system events are also written to the system event log for each transition, and are now selectable as integration trigger events (webhook/script) in the Connect page. - **Channel Profiles column in Users table**: users can now see all channel profiles assigned to each user directly in the Users table. Added tooltips for profile names to handle overflow, adjusted column sizing to prevent overflow between columns, and improved the table layout for better readability. (Closes #819) — Thanks [@damien-alt-sudo](https://github.com/damien-alt-sudo) - **Plugin warning & disclaimer components**: extracted shared plugin warning UI into a new `PluginWarnings.jsx` component and normalized warning/disclaimer usage across all plugin action modals. New reusable components: `PluginSecurityWarning` (untrusted code), `PluginSupportDisclaimer` (community support scope), `PluginDowngradeWarning` (version downgrade), `PluginInfoNote` (informational), and `PluginRestartWarning` (backend restart on import). Updated `Plugins.jsx`, `AvailablePluginCard.jsx`, and `PluginCard.jsx` to use the shared components. — Thanks [@sethwv](https://github.com/sethwv) @@ -42,9 +47,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Eliminated per-tick DB queries from the channel stats system. Previously `get_basic_channel_info()` in `channel_status.py` issued a `Stream.objects.filter()` and an `M3UAccountProfile.objects.filter()` on every stats tick for each active channel to resolve display names. Channel name and stream name are now written into the Redis metadata hash at channel-init time (via `initialize_channel`) and read back directly during stats collection, zero DB queries per tick. `m3u_profile_name` is now resolved on the frontend from the already-loaded playlists store rather than being pushed from the backend. - Eliminated a redundant `Channel` DB lookup inside `ChannelService.initialize_channel()`. `views.py` already fetches the `Channel` object via `get_stream_object()` before calling `initialize_channel()`; `channel.name` is now passed as an optional `channel_name` parameter, so the service uses the caller-supplied value and only falls back to a DB query when the name is not provided (e.g. stream-preview paths). A matching `stream_name` parameter was added for the same reason; the `stream_name` DB query is skipped entirely on normal channel start since a channel name is always available. - Eliminated a redundant `Stream` DB lookup on stream switches. `get_stream_info_for_switch()` already fetches the `Stream` object to build the URL; it now includes `stream_name` in its return dict. `change_stream_url()` captures that value and passes it through to `_update_channel_metadata()`, which skips its own `Stream.objects.filter()` when the name is already known. +- **Dedicated thread-pool Celery worker for DVR recordings**. `run_recording` is long-running and almost entirely I/O-bound: it loops in short ticks polling FFmpeg, the DB, and Redis for the full duration of the recording. Running it on the default prefork worker pool (`--autoscale=6,1`) meant at most 6 concurrent recordings, and only if no other background work was running. M3U refreshes, EPG parsing, channel matching, comskip, etc. all competed for the same 6 slots, so if every worker was busy when a recording's start time arrived, the task would queue in Redis and FFmpeg would not start until a worker freed up, causing the user to miss the beginning of the show. + - A second Celery worker is now started alongside the default one, configured with `--pool=threads --concurrency=20` and bound to a dedicated `dvr` queue. Threads fit this workload because every blocking call in `run_recording` releases the GIL (sleep, subprocess wait, file / DB / Redis I/O) and FFmpeg itself runs in a separate OS process. A `task_routes` entry in `dispatcharr/celery.py` routes `apps.channels.tasks.run_recording` to the `dvr` queue regardless of how it is dispatched (Celery Beat `PeriodicTask`, `.delay()`, etc.). The default prefork worker now subscribes only to the `celery` queue (`-Q celery -n default@%h --autoscale=6,1`), so recordings can no longer starve background tasks and background tasks can no longer delay recordings. + + - Net result: up to 20 concurrent recordings, zero startup delay regardless of EPG / M3U refresh activity, and a memory cost of roughly 80 to 120 MB for the second always-on worker process. Each additional concurrent recording costs nearly nothing on top because all 20 threads share that single process. Applied to AIO (`docker/uwsgi.ini`), dev (`docker/uwsgi.dev.ini`), debug (`docker/uwsgi.debug.ini`), and the modular celery container (`docker/entrypoint.celery.sh`). ### Changed +- DVR recording pipeline rewritten around FFmpeg + HLS. The previous implementation wrote a single growing `.ts` file via the proxy and remuxed it to `.mkv` at the end; the new implementation runs FFmpeg with `-f hls`, regenerates monotonic PTS to handle erratic IPTV source timestamps, shifts output timestamps so they start at 0 (fixing negative-PTS streams that prevented HLS segment boundary detection), and concatenates the resulting segments into the final MKV at the end of the recording. Server-restart resume now continues segment numbering from the previous session rather than overwriting earlier segments. +- Stall detection added to the recording loop. Once the first segment confirms the stream is flowing, the loop watches both segment count and segment file mtime. If neither advances for the configured stall window (e.g. when an upstream proxy ghost-kills the client), FFmpeg is signaled and the recording ends cleanly rather than hanging until `end_time`. Recording duration is honored via SIGINT so FFmpeg writes `#EXT-X-ENDLIST` cleanly into the final playlist. +- `end_time` extension is now picked up by the running recording without restarting FFmpeg. The DB poll loop simply updates the in-memory deadline. +- Recording cancellation cleanup updated for the new layout. `RecordingViewSet.destroy()` now removes the HLS working directory via a `_safe_rmtree` helper (with the same `allowed_roots` containment check used for the existing `_safe_remove`) instead of trying to delete the obsolete `_temp_file_path`. +- HLS working directory lifecycle in `custom_properties` is now two-phase. After concat succeeds, only `file_url` and `output_file_url` are updated so new client requests are redirected to the final MKV via `/file/`; `_hls_dir` is intentionally preserved through the viewer-wait grace period so in-flight `.ts` requests keep resolving. `_hls_dir` is cleared from `custom_properties` only after `shutil.rmtree` actually removes the directory. - **Column resizing in `CustomTable`**: column widths are now propagated to body cells via CSS custom properties (`--header-{id}-size`) injected on the table wrapper, rather than reading `column.getSize()` directly in each cell's React style. This decouples body-cell widths from React renders so that memoized rows (which skip re-renders for performance) still reflect resize changes instantly via CSS cascade. - Decoupled row expansion from row selection in `CustomTable`, expanding a channel row no longer also selects it. Added an `onRowExpansionChange` callback to `useTable` so callers can react to expansion changes independently of selection state. - Fixed a stale-closure bug in memoized channel row checkboxes, `selectedTableIdsRef` now ensures the checkbox `onChange` handler and `handleShiftSelect` always read the current selection set rather than the stale set captured at render time. Without this, clicking any checkbox after the first would silently deselect previously checked rows.