diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c49c40f..ee3d0df5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,13 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -### Fixed - -- **Channel start/stop notifications missing name**: the "channel started" toast never showed up and the "channel stopped" toast showed the raw UUID instead of the channel name. `channel_name` is now written into the Redis metadata hash at init time and included in every stats payload pushed over WebSocket; the frontend notification functions read `ch.channel_name` from the stats payload directly, falling back to the store lookup and then a formatted placeholder. Both notifications now display the correct channel name. -- **Channel form reset on group creation**: creating a new channel group from within the channel create/edit form no longer wipes all filled-in form fields. The newly created group is also automatically selected in the channel group field after it is saved. (Fixes #545) -- **EPG channel name truncation**: EPG sources that include long `` values (e.g. event-based channels with descriptions appended to the name) would crash the channel-parse task with a `value too long for type character varying(255)` PostgreSQL error and silently discard the entire batch. The `EPGData.name`, `Stream.name`, and `Channel.name` fields have been widened to 512 characters, and names exceeding this limit are now truncated with a warning log rather than aborting the import. (Fixes #1134) -- **Channel start/client connect notifications suppressed on first stats poll after page load**: after logging in or navigating to the Stats page, the notification logic was not firing for any connections present in the first stats response, even for streams that had started after the page loaded. Replaced the flag-based approach with a `pageLoadTime` module-level constant compared against each client's `connected_at` Unix timestamp from Redis; connections that pre-date the page load are filtered out, while genuinely new ones fire immediately. `get_basic_channel_info` now also includes `connected_at` in each client entry so this check works for the Stats page's API poll path as well as the WebSocket path. - ### Added - **EPG Program Search API** (`GET /api/epg/programs/search/`): a new endpoint for querying EPG program data with rich filtering and query support. - Thanks [@northernpowerhouse](https://github.com/northernpowerhouse) @@ -25,14 +18,101 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Access control**: results are scoped to channels the requesting user can access. `user_level` and the per-user adult-content filter are both enforced, matching the access model used by the M3U playlist, XC API, and stream proxy. Admin users receive unfiltered results. - Full OpenAPI/Swagger documentation available. - Requires `IsStandardUser` permission (user level ≥ 1). +- **Per-user IP/CIDR network allowlists**. Admins can now assign IP address and CIDR range restrictions to individual user accounts via the API & XC tab on the user edit form. When a user has one or more allowed ranges configured, requests from IPs outside that list are rejected with `403 Forbidden` regardless of the global network access policy; if no ranges are configured, the user inherits global settings unchanged. The existing `network_access_allowed()` utility is extended with an optional `user` argument so the per-user check is enforced at all access-controlled entry points (M3U/EPG, Streams, XC API, UI) without duplicating IP-matching logic. Per-user restrictions are stored in `custom_properties['allowed_networks']`; no model changes or migrations are required. — Thanks [@sethwv](https://github.com/sethwv) +- **`reset_user_network` management command**: `manage.py reset_user_network ` clears the per-user `allowed_networks` restriction for the specified account, restoring it to global-policy inheritance. Useful for recovering a user locked out by a misconfigured allowlist. — Thanks [@sethwv](https://github.com/sethwv) +- **Auto-sync overhaul**: comprehensive rebuild of the M3U auto-channel-sync flow. Introduces a per-field override system, hide-from-output flag, range-bounded auto-numbering with a re-pack helper, multi-stream channel safety, multi-provider shared-range merging, and an across-the-board move from per-row writes to bulk operations. (Closes #1196) — Thanks [@CodeBormen](https://github.com/CodeBormen) + - **Per-field channel overrides for auto-synced channels.** A new `ChannelOverride` table (one-to-one with `Channel`) holds user-specified values for `name`, `channel_number`, `channel_group`, `logo`, `tvg_id`, `tvc_guide_stationid`, `epg_data`, and `stream_profile`. Sync writes only to `Channel.*` columns; the override row takes precedence at read time via a new `with_effective_values()` queryset helper that coalesces both sources at the SQL layer. Every output surface that consumes channel data reads through this helper so overrides surface consistently: HDHR (`/hdhr/lineup.json` and per-profile variants, `/hdhr/discover.json`), M3U (`/output/m3u`, per-profile variants), EPG (`/output/epg`, per-profile variants), XC API (`get_live_streams`, `xmltv.php`), the channels list/detail endpoints, and the TV Guide summary endpoint (`GET /api/channels/channels/summary/`). Channel API responses now include both the raw `name`/`channel_number`/etc. fields AND new `effective_*` annotations plus the `override` object, so frontend consumers can show both "what the user picked" and "what the provider sent" simultaneously. + - **Per-field reset-to-provider icon.** FK pickers (channel group, logo, EPG, stream profile) and scalar inputs (name, channel_number, tvg_id, tvc_guide_stationid) display a small reset icon ("Provider: " subtext + Undo2 button) when the field has an active override on an auto-synced channel. Clicking it sets the form value back to the provider value; the override for that field is cleared on save while other overrides remain intact. + - **Auto-created channel source attribution.** The channel edit form shows an "Auto-created from: / " label at the top of auto-synced channels so the user can see which M3U account and which stream produced the row. + - **Hide-from-output flag (`hidden_from_output`).** A user-toggleable boolean that excludes a channel from HDHR, M3U, EPG, and XC output without deleting it. Hidden channels are preserved across auto-sync refreshes and excluded from auto-cleanup. The channels table shows an EyeOff icon on hidden rows. + - **Channels table visibility filter and inline indicators.** The table header has a new visibility selector ("Active Only" / "Hidden Only" / "Show All") plus a "Has Overrides" filter that narrows to channels with an active override row. Each row's name cell renders a yellow Pencil icon when overrides are active (tooltip lists the overridden field names) and an EyeOff icon when the channel is hidden. + - **Bulk hide / unhide in the bulk edit modal.** Selecting any number of channels and toggling `hidden_from_output` in bulk edit applies the change in a single PATCH; auto-routes through the override-aware bulk endpoint so it works equivalently for auto-synced and manual channels. + - **Auto-sync channel ranges per group.** `ChannelGroupM3UAccount` accepts `auto_sync_channel_start` (lower bound, inclusive) and `auto_sync_channel_end` (upper bound, inclusive; nullable for unbounded fill). Channels created by auto-sync are assigned numbers within the configured range; streams that do not fit are surfaced in the failure detail modal with a typed "RANGE_EXHAUSTED" reason. + - **Per-group inline configuration in the M3U Live group filter.** Each group row now exposes a Sync toggle, a Numbering Mode selector (Provider / Next Available / Fixed), and Start/End channel-number inputs alongside the existing fields. The numbering modes control how each stream's channel number is chosen during sync: Provider tries the M3U-supplied number first then falls back to next-available; Next Available always picks the lowest free number from 1 upward; Fixed picks sequentially from the configured Start. + - **Per-group "Configure" modal.** A gear-icon button on each group row opens a `GroupConfigureModal` containing the full set of advanced options. Fields surfaced in the modal: Channel Sort Order (provider order / name / tvg_id / updated_at, with reverse toggle), Compact Numbering toggle and "Re-pack now" button, Group Override (re-route this group's auto-created channels into a different `ChannelGroup` so multiple provider groups merge into one logical group), Name Regex Find/Replace pattern, Exclude Regex pattern, Force Dummy EPG, and a live regex preview pane. + - **Live regex preview pane.** A new `GET /api/channels/streams/regex-preview/` endpoint scans the group's streams (capped at 5000) for the find / match / exclude patterns the user is editing and returns up to 10 sample matches per pattern plus accurate total counts. The frontend debounces the call (500ms) and runs it inside the configure modal so the user sees live evidence of what their patterns will match before saving. + - **Live overlap warning across rows.** As the user edits Start/End values across multiple groups in the same provider, an `AbortController`-debounced scan calls a new `GET /api/channels/channels/numbers-in-range/` endpoint to detect (a) cross-row range overlaps (in-memory, all group pairs) and (b) channels already occupying the configured range that are not the expected occupants for that group. An informational warning surfaces inline; configurations are not blocked. + - **Compact numbering with re-pack helper.** When `compact_numbering=True` on a group's account relation, sync packs visible auto-sync channels sequentially into the configured range. A "Re-pack now" button in the group config modal runs the pack manually via a new `POST /api/m3u/accounts/{id}/repack-group/` endpoint. Hidden channels do not occupy slots; un-hiding shifts visible numbers down to fill the gap. Override-pinned channel numbers are respected as fixed anchors. Single-channel hide / unhide via the post-save signal triggers an incremental `assign_compact_numbers_for_channels` pass without requiring a full sync. + - **Multi-stream channel safety in sync.** A user-attached second stream on an auto-created channel no longer causes the channel to be deleted when one of those streams disappears from the provider. The deletion path filters out channels where at least one current stream remains alive, and per-stream iteration mutates the same `Channel` instance so `bulk_update` writes the merged final state. + - **Orphan channel cleanup mode (3-state, account-scoped).** Stored under `M3UAccount.custom_properties.orphan_channel_cleanup` and surfaced as a SegmentedControl at the top of the M3U account's group-settings page. Three positions: `always` (default; removes every auto-channel whose source stream has disappeared), `preserve_customized` (removes orphans without a `ChannelOverride` row, preserves those with one), and `never` (preserves all orphans, intended for users with manual redundancy/failover stream setups). Hidden channels are universally preserved regardless of mode. + - **Failure modal grouped by reason.** The auto-sync completion notification's failure detail modal groups entries by typed reason (`RANGE_EXHAUSTED`, `INTEGRITY_ERROR`, `OTHER`) with collapsible sections and a per-group cap. The in-memory cap was raised from 50 to 1000 so realistic multi-provider failure sets are not truncated. The notification's auto-close timeout extends from 4s to 12s when failures are present so users have time to see and click into the modal. + - **Multi-provider shared-range merging.** Two M3U accounts can target the same group with overlapping channel-number ranges. Sync seeds `used_numbers` globally so the second provider's channels pick up where the first left off without colliding. The overlap warning in the group-settings UI surfaces this configuration as informational; the merge is intentional, not a hard error. + - **Selection summary in the bulk edit modal.** The bulk edit form shows `Selection: X auto-synced, Y manual` at the top so users can see how a single set of changes will route between override rows and direct writes. + - **Delete-Playlist preview.** A new `GET /api/m3u/accounts/{id}/auto-created-channels-count/` endpoint returns the count and up to 5 sample names of auto-created channels owned by the account. The Delete Playlist confirmation dialog uses this to show the user exactly how many channels will cascade away before they confirm. + - **Custom `Channel.objects` manager.** A thin manager wraps the existing `with_effective_values()` helper so new code can call `Channel.objects.with_effective_values(...)` directly. The module-level helper remains the canonical implementation; existing call sites are unchanged. + +### Changed + +- **Global Network Access settings use tag-style inputs**: the IP/CIDR range fields in the Network Access settings panel now use tag-style chip inputs instead of a plain text field, making it easier to add, review, and remove individual addresses or ranges. — Thanks [@sethwv](https://github.com/sethwv) +- **Auto-sync overhaul** sync and channel behavior changes for the new override system. — Thanks [@CodeBormen](https://github.com/CodeBormen) + - **`Channel.channel_number` is now nullable.** Auto-sync may produce channels without an assigned number (for example, a sync run where the configured range was exhausted before this stream could be slotted, or a hidden channel in compact mode whose slot was released for visible channels). Existing channels are unchanged. HDHR / M3U / EPG / XC output endpoints filter out channels with `null` channel_number, so a number-less row is invisible to clients but visible in the channels page where the user can assign one. The 0037 auto-sync overhaul migration includes a reverse-direction backfill on the `channel_number` AlterField step so a rollback that re-imposes NOT NULL still succeeds even if NULLs are present. + - **M3U account delete always cascades auto-created channels.** The delete dialog is a single confirmation showing the count of affected channels (fetched from the new auto-created-channels-count endpoint). Auto-created channels are removed regardless of `hidden_from_output` or override state: override rows cascade via the one-to-one FK; hidden status does not preserve the channel because there is no provider left to populate it on the next sync. Manual channels are unaffected; their non-provider streams remain intact, and only streams owned by the deleted account are removed. The destroy response now returns `{"deleted_channels": N}` (HTTP 200) instead of an empty 204 so the confirmation toast can show the actual count. + - **Bulk channel edit auto-routes auto-created channels through the override path.** When a bulk edit includes auto-synced channels, the changed fields are written to each channel's `ChannelOverride` row instead of the raw `Channel.*` columns, so the edit survives the next sync. Manual channels in the same selection write directly to `Channel.*`. A "Clear all overrides" affordance pre-clears overrides on the selection before applying new edits in the same submit (clear runs before the routing PATCH so a same-submit clear cannot wipe just-written overrides). + - **Inline edits on the channels table route through the override row for auto-synced channels.** Editing a cell directly in the table (number, name, group, EPG, logo) now writes to `ChannelOverride.` for auto-synced rows so the edit survives the next refresh; manual rows continue to write directly. The save mechanic and validation are unchanged from the user's perspective. + - **Channel-number duplicates are explicitly allowed.** Sync's `used_numbers` seed still avoids assigning the same number to two newly-created auto channels in the same run, so accidental duplication during sync remains impossible. Manual or override-driven duplication is permitted; downstream client behavior on duplicates varies by client. + +### Fixed + +- **Bulk channel edit silently failed on API rejection.** The bulk-edit submit handler in `ChannelBatch.jsx` only wrote `console.error` when the PATCH was rejected; the user saw the spinner stop but received no notification, leading them to assume the save succeeded. The catch block now surfaces a red toast with the server-provided detail (or a generic fallback), and the form stays open with the in-progress selection intact so the user can correct and retry. — Thanks [@CodeBormen](https://github.com/CodeBormen) + +### Performance + +- **Auto-sync at scale**: the new override-aware sync flow is more capable than the prior path but the implementation choices below keep it viable on libraries with thousands of channels. — Thanks [@CodeBormen](https://github.com/CodeBormen) + - **Bulk writes throughout `sync_auto_channels`.** Per-row `Channel.objects.create()` and `.save()` calls were replaced with `bulk_create()` and `bulk_update()` paths that batch the entire group's create + update sets into single round-trips. The renumber pass collects all dirty channels into one list and flushes with a single `bulk_update` at the end of the loop. `ChannelStream.order` writes were similarly consolidated into a single `bulk_update`. + - **Single-pass collision detection via a global `used_numbers` set.** Choosing a free channel number was previously an `O(N)` DB query per stream. The new path seeds a `set()` of all reserved numbers (existing channels + override pins) once per run; `_next_available_number()` is `O(cluster size)` against that set. The seed deliberately excludes only this account's visible auto-created channels (the rows about to be reassigned), so cross-account and hidden-channel reservations are honored without extra queries. + - **SQL-level effective-value coalescing via `with_effective_values()`.** All channel-data consumers (HDHR, M3U, EPG, XC, channel list / detail, TV Guide summary) read effective values directly from a single annotated queryset that left-joins the override row at the database layer. Prevents N+1 `ChannelOverride` lookups across the channel scan and lets the same query serve the join. + - **EPG dispatch deduplication.** `bulk_create` and `bulk_update` bypass `post_save`, so the post-loop step explicitly dispatches `parse_programs_for_tvg_id.delay` once per unique `epg_data_id` actually changed in the run, not per channel. Avoids fan-out where one EPG source change otherwise queued thousands of redundant parse tasks. + - **Logo and EPGData lookup caching during sync.** `sync_auto_channels` now caches Logo and EPGData lookups in a per-run dict so repeated provider URLs / IDs across many streams do not produce duplicate `Logo.objects.get_or_create` or `EPGData.objects.filter` calls. + - **Override-aware bulk PATCH endpoint.** `update_channels_with_override_routing` partitions the bulk-edit selection into auto-created (override write) vs manual (direct write) once, then issues two bulk PATCHes instead of N single-row updates. + +## [0.24.0] - 2026-05-03 + +### Security + +- **HDHomeRun discovery endpoints now respect the `M3U_EPG` network access policy**. `DiscoverAPIView`, `LineupAPIView`, `LineupStatusAPIView`, and `HDHRDeviceXMLAPIView` were marked `AllowAny` so HDHR clients (Plex, Emby, Jellyfin, Channels DVR, etc.) can discover the tuner without authenticating, but they were not gated by any network allowlist. The lineup enumerates every channel name and per-channel UUID stream URL, so any client that could reach the server could full-enumerate the lineup. All four views now call `network_access_allowed(request, "M3U_EPG")` and return `403 Forbidden` for clients outside the allowlist, matching the gating already applied to the M3U and EPG endpoints (and matching what the Network Access settings UI already advertised: "Limit access to M3U, EPG, and HDHR URLs"). Operators with a restrictive `M3U_EPG` policy will see HDHR discovery start being blocked for off-LAN clients on upgrade; loosen the policy if remote HDHR access is required. +- **Removed `dangerouslySetInnerHTML` from the M3U Profile regex preview**. The "Matched Text" preview in the M3U Profile editor built an HTML string by interpolating the user's sample input into a `` wrapper and rendering it via `dangerouslySetInnerHTML`. A crafted sample input (admin-only, so self-XSS only) could inject arbitrary HTML into the preview pane. The preview now returns an array of plain strings and `` React elements, so user input is always treated as text by React. +- **Authorization on DVR recording playback endpoints**. `RecordingViewSet.file` and `RecordingViewSet.hls` now require an authenticated session and enforce a per-user channel-access check before serving any bytes. Admins (`user_level >= 10`) are always allowed; standard users are allowed only when the recording's source channel is visible under their channel-profile assignments and within their `user_level`, mirroring the same logic used by `stream_xc` for live channels. Unauthenticated requests now receive `403 Forbidden` instead of being served. The pre-existing `network_access_allowed(request, "STREAMS")` perimeter check is retained as a separate, prior gate so external IPs can be blocked from streaming entirely even with a valid token. +- Updated `lxml` 6.0.3 → 6.1.0, resolving the following CVE: + - **CVE-2026-41066**: External entity injection (XXE) in `iterparse()` and `ETCompatXMLParser`. +- Updated frontend npm dependencies to resolve 5 audit vulnerabilities (1 moderate, 4 high): + - Updated `@xmldom/xmldom` 0.8.12 → 0.8.13, resolving **high** uncontrolled recursion in XML serialization causing DoS ([GHSA-2v35-w6hq-6mfw](https://github.com/advisories/GHSA-2v35-w6hq-6mfw)), **high** XML injection via unvalidated `DocumentType` serialization ([GHSA-f6ww-3ggp-fr8h](https://github.com/advisories/GHSA-f6ww-3ggp-fr8h)), **high** XML node injection via unvalidated processing instruction serialization ([GHSA-x6wf-f3px-wcqx](https://github.com/advisories/GHSA-x6wf-f3px-wcqx)), and **high** XML node injection via unvalidated comment serialization ([GHSA-j759-j44w-7fr8](https://github.com/advisories/GHSA-j759-j44w-7fr8)) + - Updated `postcss` 8.5.6 → 8.5.13, resolving **moderate** XSS via unescaped `` in CSS stringify output ([GHSA-qx2v-qp2m-jg93](https://github.com/advisories/GHSA-qx2v-qp2m-jg93)) + +### Fixed + +- **EPG program times shifted by the host system's UTC offset when `/etc/localtime` is bind-mounted into the container**. Mounting `/etc/localtime` from a non-UTC host causes PostgreSQL to silently resolve the `'UTC'` timezone name to the host's local timezone (e.g. CDT, CET) rather than actual UTC - even though `SHOW timezone` returns `UTC` and the zoneinfo file exists. This made PostgreSQL format all stored `timestamptz` values with the host's UTC offset, and psycopg2 returned datetimes shifted by that offset, causing every EPG program time to be read back N hours wrong and written to the XML output incorrectly. The fix registers a Django `connection_created` signal that issues `SET TIME ZONE 'UTC0'` on every new database connection. `UTC0` is a POSIX timezone string that bypasses the broken zoneinfo name-lookup path entirely; it resolves unconditionally to UTC+00 regardless of the host timezone or what files are mounted. (Fixes #651) +- **Xtream Codes `player_api.php` missing `active_cons` and reporting wrong `max_connections`**. The `user_info` block returned by the XC API did not include the `active_cons` field, which Enigma2 clients (XStreamity, XKlass) read unconditionally and crash with `KeyError: 'active_cons'` when it is absent. `max_connections` was also hardcoded to the system-wide tuner count for every user, ignoring per-user `stream_limit` configuration. `xc_get_info` now reports `max_connections` as the user's `stream_limit` when set, falling back to the system tuner count for unlimited users; `active_cons` is the user's own active connection count when they have a per-user limit, or the system-wide active connection count when they do not (so unlimited clients can still see how much of the global tuner pool is in use). The existing `get_user_active_connections` helper was generalized to accept `user_id=None` for the system-wide query rather than duplicating its Redis scan logic. (Fixes #990) +- **Plugin discovery re-running on every connect event**. The `PluginManager` cache hit check used `if self._registry and ...`, which always evaluated false when zero plugins were installed because an empty dict is falsy. Every Connect event (`recording_start`, `client_connect`, `channel_start`, etc.) was therefore triggering a full filesystem walk of `/data/plugins` and emitting `Discovering plugins (no DB sync) in /data/plugins` / `Discovered 0 plugin(s)` log lines. Tracked separately via a new `_discovery_completed` flag so the cache short-circuits subsequent calls regardless of registry contents, dropping discovery to once per worker process lifetime. +- **Empty show/season folders left behind after deleting a recording**. Deleting a recording removes the MKV (or HLS working directory if still in progress) but previously left the parent show / season directories on disk even when they no longer contained any other recordings. After file cleanup, `RecordingViewSet.destroy` now walks up from the deleted file's parent directory and removes any now-empty directories, stopping at the `/data/recordings` library root so the root itself is never touched. +- **Premature DVR concat on graceful Celery worker shutdown**. A `worker_shutting_down` signal handler in `apps/channels/tasks.py` now sets a module-level `_DVR_SHUTTING_DOWN` flag. The `run_recording` loop checks this flag after FFmpeg exits and, if the recording's `end_time` has not yet passed, skips concatenation and persists `status="interrupted"` with `interrupted_reason="server_shutdown"`. The HLS working directory is preserved across the restart so the recovery path on the next worker startup can resume segment numbering from where it left off rather than truncating the show. +- **Channel start/stop notifications missing name**: the "channel started" toast never showed up and the "channel stopped" toast showed the raw UUID instead of the channel name. `channel_name` is now written into the Redis metadata hash at init time and included in every stats payload pushed over WebSocket; the frontend notification functions read `ch.channel_name` from the stats payload directly, falling back to the store lookup and then a formatted placeholder. Both notifications now display the correct channel name. +- **Channel form reset on group creation**: creating a new channel group from within the channel create/edit form no longer wipes all filled-in form fields. The newly created group is also automatically selected in the channel group field after it is saved. (Fixes #545) +- **EPG channel name truncation**: EPG sources that include long `` values (e.g. event-based channels with descriptions appended to the name) would crash the channel-parse task with a `value too long for type character varying(255)` PostgreSQL error and silently discard the entire batch. The `EPGData.name`, `Stream.name`, and `Channel.name` fields have been widened to 512 characters, and names exceeding this limit are now truncated with a warning log rather than aborting the import. (Fixes #1134) +- **Channel start/client connect notifications suppressed on first stats poll after page load**: after logging in or navigating to the Stats page, the notification logic was not firing for any connections present in the first stats response, even for streams that had started after the page loaded. Replaced the flag-based approach with a `pageLoadTime` module-level constant compared against each client's `connected_at` Unix timestamp from Redis; connections that pre-date the page load are filtered out, while genuinely new ones fire immediately. `get_basic_channel_info` now also includes `connected_at` in each client entry so this check works for the Stats page's API poll path as well as the WebSocket path. + +### Added + +- **In-progress recording playback from the DVR page**. The Watch button on a recording card is now enabled while the recording is still in progress, and the in-app floating player can play the live HLS playlist with full timeshift / scrub-back to the start of the recording. + - The frontend now bundles `hls.js` and routes any `.m3u8` URL through it (Chrome, Edge, Firefox, and other Chromium-based browsers). Native HLS via `