Mirrors/Dispatcharr
1
0
Fork 1
mirror of https://github.com/Dispatcharr/Dispatcharr.git synced 2026-07-17 16:50:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Merge pull request #1215 from CodeBormen/feat/1196-auto-sync-overhaul

Browse source 
Feat/1196 auto sync overhaul
This commit is contained in:
SergeantPanda 2026-05-04 18:51:18 -05:00 • committed by GitHub
commit 438c90e96d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
45 changed files with 10169 additions and 1802 deletions
Download patch file Download diff file Expand all files Collapse all files

40
CHANGELOG.md
View file

@ -11,10 +11,50 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Per-user IP/CIDR network allowlists**. Admins can now assign IP address and CIDR range restrictions to individual user accounts via the API & XC tab on the user edit form. When a user has one or more allowed ranges configured, requests from IPs outside that list are rejected with `403 Forbidden` regardless of the global network access policy; if no ranges are configured, the user inherits global settings unchanged. The existing `network_access_allowed()` utility is extended with an optional `user` argument so the per-user check is enforced at all access-controlled entry points (M3U/EPG, Streams, XC API, UI) without duplicating IP-matching logic. Per-user restrictions are stored in `custom_properties['allowed_networks']`; no model changes or migrations are required. — Thanks [@sethwv](https://github.com/sethwv)
- **`reset_user_network` management command**: `manage.py reset_user_network <username>` clears the per-user `allowed_networks` restriction for the specified account, restoring it to global-policy inheritance. Useful for recovering a user locked out by a misconfigured allowlist. — Thanks [@sethwv](https://github.com/sethwv)
- **Auto-sync overhaul**: comprehensive rebuild of the M3U auto-channel-sync flow. Introduces a per-field override system, hide-from-output flag, range-bounded auto-numbering with a re-pack helper, multi-stream channel safety, multi-provider shared-range merging, and an across-the-board move from per-row writes to bulk operations. (Closes #1196) — Thanks [@CodeBormen](https://github.com/CodeBormen)
- **Per-field channel overrides for auto-synced channels.** A new `ChannelOverride` table (one-to-one with `Channel`) holds user-specified values for `name`, `channel_number`, `channel_group`, `logo`, `tvg_id`, `tvc_guide_stationid`, `epg_data`, and `stream_profile`. Sync writes only to `Channel.*` columns; the override row takes precedence at read time via a new `with_effective_values()` queryset helper that coalesces both sources at the SQL layer. Every output surface that consumes channel data reads through this helper so overrides surface consistently: HDHR (`/hdhr/lineup.json` and per-profile variants, `/hdhr/discover.json`), M3U (`/output/m3u`, per-profile variants), EPG (`/output/epg`, per-profile variants), XC API (`get_live_streams`, `xmltv.php`), the channels list/detail endpoints, and the TV Guide summary endpoint (`GET /api/channels/channels/summary/`). Channel API responses now include both the raw `name`/`channel_number`/etc. fields AND new `effective_*` annotations plus the `override` object, so frontend consumers can show both "what the user picked" and "what the provider sent" simultaneously.
- **Per-field reset-to-provider icon.** FK pickers (channel group, logo, EPG, stream profile) and scalar inputs (name, channel_number, tvg_id, tvc_guide_stationid) display a small reset icon ("Provider: <value>" subtext + Undo2 button) when the field has an active override on an auto-synced channel. Clicking it sets the form value back to the provider value; the override for that field is cleared on save while other overrides remain intact.
- **Auto-created channel source attribution.** The channel edit form shows an "Auto-created from: <provider name> / <stream name>" label at the top of auto-synced channels so the user can see which M3U account and which stream produced the row.
- **Hide-from-output flag (`hidden_from_output`).** A user-toggleable boolean that excludes a channel from HDHR, M3U, EPG, and XC output without deleting it. Hidden channels are preserved across auto-sync refreshes and excluded from auto-cleanup. The channels table shows an EyeOff icon on hidden rows.
- **Channels table visibility filter and inline indicators.** The table header has a new visibility selector ("Active Only" / "Hidden Only" / "Show All") plus a "Has Overrides" filter that narrows to channels with an active override row. Each row's name cell renders a yellow Pencil icon when overrides are active (tooltip lists the overridden field names) and an EyeOff icon when the channel is hidden.
- **Bulk hide / unhide in the bulk edit modal.** Selecting any number of channels and toggling `hidden_from_output` in bulk edit applies the change in a single PATCH; auto-routes through the override-aware bulk endpoint so it works equivalently for auto-synced and manual channels.
- **Auto-sync channel ranges per group.** `ChannelGroupM3UAccount` accepts `auto_sync_channel_start` (lower bound, inclusive) and `auto_sync_channel_end` (upper bound, inclusive; nullable for unbounded fill). Channels created by auto-sync are assigned numbers within the configured range; streams that do not fit are surfaced in the failure detail modal with a typed "RANGE_EXHAUSTED" reason.
- **Per-group inline configuration in the M3U Live group filter.** Each group row now exposes a Sync toggle, a Numbering Mode selector (Provider / Next Available / Fixed), and Start/End channel-number inputs alongside the existing fields. The numbering modes control how each stream's channel number is chosen during sync: Provider tries the M3U-supplied number first then falls back to next-available; Next Available always picks the lowest free number from 1 upward; Fixed picks sequentially from the configured Start.
- **Per-group "Configure" modal.** A gear-icon button on each group row opens a `GroupConfigureModal` containing the full set of advanced options. Fields surfaced in the modal: Channel Sort Order (provider order / name / tvg_id / updated_at, with reverse toggle), Compact Numbering toggle and "Re-pack now" button, Group Override (re-route this group's auto-created channels into a different `ChannelGroup` so multiple provider groups merge into one logical group), Name Regex Find/Replace pattern, Exclude Regex pattern, Force Dummy EPG, and a live regex preview pane.
- **Live regex preview pane.** A new `GET /api/channels/streams/regex-preview/` endpoint scans the group's streams (capped at 5000) for the find / match / exclude patterns the user is editing and returns up to 10 sample matches per pattern plus accurate total counts. The frontend debounces the call (500ms) and runs it inside the configure modal so the user sees live evidence of what their patterns will match before saving.
- **Live overlap warning across rows.** As the user edits Start/End values across multiple groups in the same provider, an `AbortController`-debounced scan calls a new `GET /api/channels/channels/numbers-in-range/` endpoint to detect (a) cross-row range overlaps (in-memory, all group pairs) and (b) channels already occupying the configured range that are not the expected occupants for that group. An informational warning surfaces inline; configurations are not blocked.
- **Compact numbering with re-pack helper.** When `compact_numbering=True` on a group's account relation, sync packs visible auto-sync channels sequentially into the configured range. A "Re-pack now" button in the group config modal runs the pack manually via a new `POST /api/m3u/accounts/{id}/repack-group/` endpoint. Hidden channels do not occupy slots; un-hiding shifts visible numbers down to fill the gap. Override-pinned channel numbers are respected as fixed anchors. Single-channel hide / unhide via the post-save signal triggers an incremental `assign_compact_numbers_for_channels` pass without requiring a full sync.
- **Multi-stream channel safety in sync.** A user-attached second stream on an auto-created channel no longer causes the channel to be deleted when one of those streams disappears from the provider. The deletion path filters out channels where at least one current stream remains alive, and per-stream iteration mutates the same `Channel` instance so `bulk_update` writes the merged final state.
- **Orphan channel cleanup mode (3-state, account-scoped).** Stored under `M3UAccount.custom_properties.orphan_channel_cleanup` and surfaced as a SegmentedControl at the top of the M3U account's group-settings page. Three positions: `always` (default; removes every auto-channel whose source stream has disappeared), `preserve_customized` (removes orphans without a `ChannelOverride` row, preserves those with one), and `never` (preserves all orphans, intended for users with manual redundancy/failover stream setups). Hidden channels are universally preserved regardless of mode.
- **Failure modal grouped by reason.** The auto-sync completion notification's failure detail modal groups entries by typed reason (`RANGE_EXHAUSTED`, `INTEGRITY_ERROR`, `OTHER`) with collapsible sections and a per-group cap. The in-memory cap was raised from 50 to 1000 so realistic multi-provider failure sets are not truncated. The notification's auto-close timeout extends from 4s to 12s when failures are present so users have time to see and click into the modal.
- **Multi-provider shared-range merging.** Two M3U accounts can target the same group with overlapping channel-number ranges. Sync seeds `used_numbers` globally so the second provider's channels pick up where the first left off without colliding. The overlap warning in the group-settings UI surfaces this configuration as informational; the merge is intentional, not a hard error.
- **Selection summary in the bulk edit modal.** The bulk edit form shows `Selection: X auto-synced, Y manual` at the top so users can see how a single set of changes will route between override rows and direct writes.
- **Delete-Playlist preview.** A new `GET /api/m3u/accounts/{id}/auto-created-channels-count/` endpoint returns the count and up to 5 sample names of auto-created channels owned by the account. The Delete Playlist confirmation dialog uses this to show the user exactly how many channels will cascade away before they confirm.
- **Custom `Channel.objects` manager.** A thin manager wraps the existing `with_effective_values()` helper so new code can call `Channel.objects.with_effective_values(...)` directly. The module-level helper remains the canonical implementation; existing call sites are unchanged.
### Changed
- **Global Network Access settings use tag-style inputs**: the IP/CIDR range fields in the Network Access settings panel now use tag-style chip inputs instead of a plain text field, making it easier to add, review, and remove individual addresses or ranges. — Thanks [@sethwv](https://github.com/sethwv)
- **Auto-sync overhaul** sync and channel behavior changes for the new override system. — Thanks [@CodeBormen](https://github.com/CodeBormen)
- **`Channel.channel_number` is now nullable.** Auto-sync may produce channels without an assigned number (for example, a sync run where the configured range was exhausted before this stream could be slotted, or a hidden channel in compact mode whose slot was released for visible channels). Existing channels are unchanged. HDHR / M3U / EPG / XC output endpoints filter out channels with `null` channel_number, so a number-less row is invisible to clients but visible in the channels page where the user can assign one. The 0037 auto-sync overhaul migration includes a reverse-direction backfill on the `channel_number` AlterField step so a rollback that re-imposes NOT NULL still succeeds even if NULLs are present.
- **M3U account delete always cascades auto-created channels.** The delete dialog is a single confirmation showing the count of affected channels (fetched from the new auto-created-channels-count endpoint). Auto-created channels are removed regardless of `hidden_from_output` or override state: override rows cascade via the one-to-one FK; hidden status does not preserve the channel because there is no provider left to populate it on the next sync. Manual channels are unaffected; their non-provider streams remain intact, and only streams owned by the deleted account are removed. The destroy response now returns `{"deleted_channels": N}` (HTTP 200) instead of an empty 204 so the confirmation toast can show the actual count.
- **Bulk channel edit auto-routes auto-created channels through the override path.** When a bulk edit includes auto-synced channels, the changed fields are written to each channel's `ChannelOverride` row instead of the raw `Channel.*` columns, so the edit survives the next sync. Manual channels in the same selection write directly to `Channel.*`. A "Clear all overrides" affordance pre-clears overrides on the selection before applying new edits in the same submit (clear runs before the routing PATCH so a same-submit clear cannot wipe just-written overrides).
- **Inline edits on the channels table route through the override row for auto-synced channels.** Editing a cell directly in the table (number, name, group, EPG, logo) now writes to `ChannelOverride.<field>` for auto-synced rows so the edit survives the next refresh; manual rows continue to write directly. The save mechanic and validation are unchanged from the user's perspective.
- **Channel-number duplicates are explicitly allowed.** Sync's `used_numbers` seed still avoids assigning the same number to two newly-created auto channels in the same run, so accidental duplication during sync remains impossible. Manual or override-driven duplication is permitted; downstream client behavior on duplicates varies by client.
### Fixed
- **Bulk channel edit silently failed on API rejection.** The bulk-edit submit handler in `ChannelBatch.jsx` only wrote `console.error` when the PATCH was rejected; the user saw the spinner stop but received no notification, leading them to assume the save succeeded. The catch block now surfaces a red toast with the server-provided detail (or a generic fallback), and the form stays open with the in-progress selection intact so the user can correct and retry. — Thanks [@CodeBormen](https://github.com/CodeBormen)
### Performance
- **Auto-sync at scale**: the new override-aware sync flow is more capable than the prior path but the implementation choices below keep it viable on libraries with thousands of channels. — Thanks [@CodeBormen](https://github.com/CodeBormen)
- **Bulk writes throughout `sync_auto_channels`.** Per-row `Channel.objects.create()` and `.save()` calls were replaced with `bulk_create()` and `bulk_update()` paths that batch the entire group's create + update sets into single round-trips. The renumber pass collects all dirty channels into one list and flushes with a single `bulk_update` at the end of the loop. `ChannelStream.order` writes were similarly consolidated into a single `bulk_update`.
- **Single-pass collision detection via a global `used_numbers` set.** Choosing a free channel number was previously an `O(N)` DB query per stream. The new path seeds a `set()` of all reserved numbers (existing channels + override pins) once per run; `_next_available_number()` is `O(cluster size)` against that set. The seed deliberately excludes only this account's visible auto-created channels (the rows about to be reassigned), so cross-account and hidden-channel reservations are honored without extra queries.
- **SQL-level effective-value coalescing via `with_effective_values()`.** All channel-data consumers (HDHR, M3U, EPG, XC, channel list / detail, TV Guide summary) read effective values directly from a single annotated queryset that left-joins the override row at the database layer. Prevents N+1 `ChannelOverride` lookups across the channel scan and lets the same query serve the join.
- **EPG dispatch deduplication.** `bulk_create` and `bulk_update` bypass `post_save`, so the post-loop step explicitly dispatches `parse_programs_for_tvg_id.delay` once per unique `epg_data_id` actually changed in the run, not per channel. Avoids fan-out where one EPG source change otherwise queued thousands of redundant parse tasks.
- **Logo and EPGData lookup caching during sync.** `sync_auto_channels` now caches Logo and EPGData lookups in a per-run dict so repeated provider URLs / IDs across many streams do not produce duplicate `Logo.objects.get_or_create` or `EPGData.objects.filter` calls.
- **Override-aware bulk PATCH endpoint.** `update_channels_with_override_routing` partitions the bulk-edit selection into auto-created (override write) vs manual (direct write) once, then issues two bulk PATCHes instead of N single-row updates.
## [0.24.0] - 2026-05-03

713
apps/channels/api_views.py
View file

@ -8,8 +8,8 @@ from drf_spectacular.utils import extend_schema, OpenApiParameter, inline_serial
from drf_spectacular.types import OpenApiTypes
from rest_framework import serializers
from django.shortcuts import get_object_or_404, get_list_or_404
from django.db import transaction
from django.db.models import Count, F
from django.db import connection, transaction
from django.db.models import Count, F, Prefetch
from django.db.models import Q
import os, json, requests, logging, mimetypes, threading, time
from datetime import timedelta
@ -200,6 +200,249 @@ class StreamViewSet(viewsets.ModelViewSet):
# Return the response with the list of IDs
return Response(list(stream_ids))
@extend_schema(
parameters=[
OpenApiParameter(
name="channel_group",
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
required=True,
description="Channel group name to scope the preview to.",
),
OpenApiParameter(
name="find",
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
required=False,
description=(
"Find regex for the rename preview. When supplied, "
"the response includes find_matches and find_match_count."
),
),
OpenApiParameter(
name="replace",
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
required=False,
description=(
"Replacement string used with the find pattern. "
"Defaults to empty string when omitted."
),
),
OpenApiParameter(
name="match",
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
required=False,
description=(
"Filter regex for the include preview. When supplied, "
"the response includes filter_matches and filter_match_count."
),
),
OpenApiParameter(
name="exclude",
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
required=False,
description=(
"Filter regex for the exclude preview. When supplied, "
"the response includes exclude_matches and "
"exclude_match_count."
),
),
OpenApiParameter(
name="limit",
type=OpenApiTypes.INT,
location=OpenApiParameter.QUERY,
required=False,
description="Max preview entries per match list (default 10, capped at 50).",
),
],
responses={
200: inline_serializer(
name="StreamRegexPreviewResponse",
fields={
"total_in_group": serializers.IntegerField(),
"total_scanned": serializers.IntegerField(),
"scan_limit_hit": serializers.BooleanField(),
"find_matches": serializers.ListField(
child=serializers.DictField(), required=False
),
"find_match_count": serializers.IntegerField(required=False),
"filter_matches": serializers.ListField(
child=serializers.DictField(), required=False
),
"filter_match_count": serializers.IntegerField(required=False),
"exclude_matches": serializers.ListField(
child=serializers.DictField(), required=False
),
"exclude_match_count": serializers.IntegerField(required=False),
"find_error": serializers.CharField(required=False),
"match_error": serializers.CharField(required=False),
"exclude_error": serializers.CharField(required=False),
},
)
},
description=(
"Returns regex preview info for a group's streams. Used by the "
"auto-sync gear modal so users can see how their find/replace "
"or filter pattern affects real stream names before saving. "
"Caps in-memory iteration at SCAN_CAP streams per call so the "
"endpoint stays bounded even on groups with tens of thousands "
"of streams; the caller surfaces total_in_group and "
"scan_limit_hit so users know whether the preview is complete."
),
)
@action(detail=False, methods=["get"], url_path="regex-preview")
def regex_preview(self, request, *args, **kwargs):
# `regex` (third-party) supports a per-call timeout that bounds
# catastrophic backtracking; paired with PATTERN_MAX_LEN to keep
# the endpoint safe under adversarial input.
import regex as re
SCAN_CAP = 5000
PATTERN_MAX_LEN = 512
REGEX_TIMEOUT = 0.1
group_name = request.query_params.get("channel_group")
if not group_name:
return Response(
{"detail": "channel_group is required"},
status=status.HTTP_400_BAD_REQUEST,
)
# Group names are not unique across M3U accounts (two providers
# can both publish a "Sports" group). Scope to the calling
# account so the sample reflects only the user's edits.
m3u_account_id = request.query_params.get("m3u_account_id")
if m3u_account_id is not None:
try:
m3u_account_id = int(m3u_account_id)
except (TypeError, ValueError):
return Response(
{"detail": "m3u_account_id must be an integer"},
status=status.HTTP_400_BAD_REQUEST,
)
find_pat = request.query_params.get("find") or None
replace_pat = request.query_params.get("replace") or ""
match_pat = request.query_params.get("match") or None
exclude_pat = request.query_params.get("exclude") or None
for label, value in (
("find", find_pat),
("replace", replace_pat),
("match", match_pat),
("exclude", exclude_pat),
):
if value is not None and len(value) > PATTERN_MAX_LEN:
return Response(
{"detail": f"{label} exceeds {PATTERN_MAX_LEN} characters"},
status=status.HTTP_400_BAD_REQUEST,
)
try:
limit = int(request.query_params.get("limit", 10))
except (TypeError, ValueError):
limit = 10
limit = max(1, min(limit, 50))
find_re = None
match_re = None
exclude_re = None
find_error = None
match_error = None
exclude_error = None
if find_pat:
try:
find_re = re.compile(find_pat)
except re.error as e:
find_error = str(e)
if match_pat:
try:
match_re = re.compile(match_pat)
except re.error as e:
match_error = str(e)
if exclude_pat:
try:
exclude_re = re.compile(exclude_pat)
except re.error as e:
exclude_error = str(e)
# Capped at SCAN_CAP to bound memory on huge groups; the
# separate COUNT lets the client surface scan_limit_hit when
# the preview covers only a sample.
base_qs = Stream.objects.filter(channel_group__name=group_name)
if m3u_account_id is not None:
base_qs = base_qs.filter(m3u_account_id=m3u_account_id)
names_iter = base_qs.values_list("name", flat=True)[:SCAN_CAP]
total_in_group = base_qs.count()
find_matches = []
filter_matches = []
exclude_matches = []
find_match_count = 0
filter_match_count = 0
exclude_match_count = 0
total_scanned = 0
# Abort a pattern on timeout to bound CPU; partial counts and
# an `*_error` field still flow back to the client.
for name in names_iter:
total_scanned += 1
if find_re is not None:
try:
new_name = find_re.sub(replace_pat, name, timeout=REGEX_TIMEOUT)
except (TimeoutError, re.error) as e:
find_error = find_error or f"Pattern timed out: {e}"
find_re = None
continue
if new_name != name:
find_match_count += 1
if len(find_matches) < limit:
find_matches.append({"before": name, "after": new_name})
if match_re is not None:
try:
matched = match_re.search(name, timeout=REGEX_TIMEOUT)
except (TimeoutError, re.error) as e:
match_error = match_error or f"Pattern timed out: {e}"
match_re = None
continue
if matched:
filter_match_count += 1
if len(filter_matches) < limit:
filter_matches.append({"name": name, "matches": True})
if exclude_re is not None:
try:
matched = exclude_re.search(name, timeout=REGEX_TIMEOUT)
except (TimeoutError, re.error) as e:
exclude_error = exclude_error or f"Pattern timed out: {e}"
exclude_re = None
continue
if matched:
exclude_match_count += 1
if len(exclude_matches) < limit:
exclude_matches.append({"name": name, "matches": True})
response_payload = {
"total_in_group": total_in_group,
"total_scanned": total_scanned,
"scan_limit_hit": total_in_group > SCAN_CAP,
}
if find_pat:
response_payload["find_matches"] = find_matches
response_payload["find_match_count"] = find_match_count
if find_error:
response_payload["find_error"] = find_error
if match_pat:
response_payload["filter_matches"] = filter_matches
response_payload["filter_match_count"] = filter_match_count
if match_error:
response_payload["match_error"] = match_error
if exclude_pat:
response_payload["exclude_matches"] = exclude_matches
response_payload["exclude_match_count"] = exclude_match_count
if exclude_error:
response_payload["exclude_error"] = exclude_error
return Response(response_payload)
@action(detail=False, methods=["get"], url_path="groups")
def get_groups(self, request, *args, **kwargs):
# Get unique ChannelGroup names that are linked to streams
@ -322,8 +565,56 @@ class ChannelGroupViewSet(viewsets.ModelViewSet):
return [Authenticated()]
def get_queryset(self):
"""Return channel groups with prefetched relations for efficient counting"""
return ChannelGroup.objects.prefetch_related('channels', 'm3u_accounts').all()
# Annotate both counts at the SQL level so the serializer methods
# can read them from the object rather than issuing a COUNT per row.
# `distinct=True` is required when multiple reverse-FK annotations
# share the same queryset to avoid row-multiplication artifacts.
# m3u_accounts is still prefetched for the nested serializer data.
return (
ChannelGroup.objects
.annotate(
channel_count=Count('channels', distinct=True),
m3u_account_count=Count('m3u_accounts', distinct=True),
)
.prefetch_related('m3u_accounts')
.all()
)
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
# Evaluate the queryset once so the annotations and prefetch cache are
# populated together, then extract IDs from the in-memory objects.
# A second .values_list() call would fire a separate SQL query.
groups = list(queryset)
group_ids = [g.id for g in groups]
# Pre-aggregate stream counts for all (account, group) pairs in a
# single query so the nested ChannelGroupM3UAccountSerializer never
# fires a COUNT per row.
counts_qs = (
Stream.objects.filter(channel_group_id__in=group_ids)
.values('m3u_account_id', 'channel_group_id')
.annotate(c=Count('id'))
)
stream_counts = {
(row['m3u_account_id'], row['channel_group_id']): row['c']
for row in counts_qs
}
page = self.paginate_queryset(groups)
if page is not None:
serializer = self.get_serializer(
page, many=True,
context={**self.get_serializer_context(), 'stream_counts': stream_counts},
)
return self.get_paginated_response(serializer.data)
serializer = self.get_serializer(
groups, many=True,
context={**self.get_serializer_context(), 'stream_counts': stream_counts},
)
return Response(serializer.data)
def update(self, request, *args, **kwargs):
"""Override update to check M3U associations"""
@ -563,17 +854,31 @@ class ChannelViewSet(viewsets.ModelViewSet):
return [Authenticated()]
def get_queryset(self):
qs = (
super()
.get_queryset()
.select_related(
# get_ids and summary only need the filter conditions, not the full
# object graph. Skipping the 5 select_related joins and 2 prefetch
# queries for those actions cuts their DB cost significantly.
action = getattr(self, "action", None)
qs = super().get_queryset()
if action not in ("get_ids", "summary"):
qs = qs.select_related(
"channel_group",
"logo",
"epg_data",
"stream_profile",
"override",
"auto_created_by",
).prefetch_related(
"streams",
# Default-attr prefetch shares the cache with M2M writes;
# a named `to_attr` would isolate it and trigger N+1.
Prefetch(
"channelstream_set",
queryset=ChannelStream.objects.select_related(
"stream__m3u_account"
).order_by("order"),
),
)
.prefetch_related("streams")
)
channel_group = self.request.query_params.get("channel_group")
if channel_group:
@ -587,6 +892,8 @@ class ChannelViewSet(viewsets.ModelViewSet):
show_disabled_param = self.request.query_params.get("show_disabled", None)
only_streamless = self.request.query_params.get("only_streamless", None)
only_stale = self.request.query_params.get("only_stale", None)
only_has_overrides = self.request.query_params.get("only_has_overrides", None)
visibility_filter = self.request.query_params.get("visibility_filter", "active")
if channel_profile_id:
try:
@ -609,6 +916,18 @@ class ChannelViewSet(viewsets.ModelViewSet):
if only_stale:
# Filter channels that have at least one related stream marked as stale
q_filters &= Q(streams__is_stale=True)
if only_has_overrides:
q_filters &= Q(override__isnull=False)
# Visibility filter applies to list-style reads only; retrieve /
# update / delete must still reach a hidden channel by id so the
# frontend can unhide. Summary powers the TV Guide and follows
# the same hidden semantic as downstream clients.
if self.action in ("list", "get_ids", "summary"):
if visibility_filter == "hidden":
q_filters &= Q(hidden_from_output=True)
elif visibility_filter != "all":
q_filters &= Q(hidden_from_output=False)
if self.request.user.user_level < 10:
filters["user_level__lte"] = self.request.user.user_level
@ -630,6 +949,14 @@ class ChannelViewSet(viewsets.ModelViewSet):
self.request.query_params.get("include_streams", "false") == "true"
)
context["include_streams"] = include_streams
# source_stream is only needed by the channel edit form. For get_ids
# and summary the channelstream_set prefetch is skipped entirely, so
# source_stream cannot be computed without hitting the DB per channel.
# For list and write/retrieve paths the prefetch is present, so we
# can populate it from memory without extra queries.
context["include_source_stream"] = action not in (
"get_ids", "summary"
) if (action := getattr(self, "action", None)) else False
return context
@extend_schema(
@ -756,14 +1083,44 @@ class ChannelViewSet(viewsets.ModelViewSet):
status=status.HTTP_400_BAD_REQUEST,
)
# Capture override intent from the raw payload: presence of the
# key distinguishes "no change" from explicit null ("clear"),
# which `validated_data` would collapse.
override_intents = {}
for channel_data in data:
if "override" in channel_data:
cid = channel_data.get("id")
override_intents[cid] = channel_data["override"]
# Capture hide / unhide transitions before setattr overwrites
# the in-memory channels. bulk_update skips post_save, so the
# compact-mode assign / release runs explicitly below.
unhide_transition_ids = [
channel.id
for channel, validated_data in validated_updates
if validated_data.get("hidden_from_output") is False
and channel.hidden_from_output is True
]
hide_transition_candidates = [
channel
for channel, validated_data in validated_updates
if validated_data.get("hidden_from_output") is True
and channel.hidden_from_output is False
and channel.channel_number is not None
and channel.auto_created
and channel.auto_created_by_id
]
# Apply all updates in a transaction
with transaction.atomic():
streams_updates = []
for channel, validated_data in validated_updates:
# Pop streams before setattr loop — M2M fields can't be set via setattr
# Streams (M2M) and override (reverse OneToOne) cannot
# ride the setattr loop; handle each separately below.
streams = validated_data.pop("streams", None)
if streams is not None:
streams_updates.append((channel, streams))
validated_data.pop("override", None)
for key, value in validated_data.items():
setattr(channel, key, value)
@ -783,6 +1140,179 @@ class ChannelViewSet(viewsets.ModelViewSet):
batch_size=100
)
# On unhide under compact mode, assign a number so the
# channel is immediately addressable by clients.
if unhide_transition_ids:
from .compact_numbering import (
assign_compact_numbers_for_channels,
)
assign_compact_numbers_for_channels(unhide_transition_ids)
# On hide under compact mode, release the channel_number
# so the slot is reused. The bulk path bypasses the
# post_save signal that handles single-row hides.
if hide_transition_candidates:
from .compact_numbering import (
get_group_relation_for_channel,
is_compact_group,
)
ids_to_release = []
for ch in hide_transition_candidates:
relation = get_group_relation_for_channel(ch)
if relation and is_compact_group(relation):
ids_to_release.append(ch.id)
if ids_to_release:
Channel.objects.filter(id__in=ids_to_release).update(
channel_number=None
)
# Refresh in-memory copies so the response shape
# reflects the cleared numbers.
for ch in channels_to_update:
if ch.id in ids_to_release:
ch.channel_number = None
# Override (reverse OneToOne) needs a separate write path.
if override_intents:
from apps.channels.models import ChannelOverride
from apps.channels.managers import OVERRIDABLE_FIELDS
override_fields = OVERRIDABLE_FIELDS
# Block override mutations on manual channels. Mixed
# selections with override:null are tolerated because
# clearing a non-existent row is a no-op.
manual_with_override = []
for channel, _ in validated_updates:
if channel.id not in override_intents:
continue
intent = override_intents[channel.id]
if (
intent is not None
and intent != {}
and not channel.auto_created
):
manual_with_override.append(channel.id)
if manual_with_override:
return Response(
{
"errors": [
{
"channel_id": cid,
"error": (
"Cannot set override on a manual channel; "
"overrides only apply to auto-created channels."
),
}
for cid in manual_with_override
]
},
status=status.HTTP_400_BAD_REQUEST,
)
channels_to_clear = []
overrides_to_upsert = []
for channel, _ in validated_updates:
if channel.id not in override_intents:
continue
intent = override_intents[channel.id]
if intent is None:
channels_to_clear.append(channel.id)
elif intent == {}:
# Empty dict means no override intent; treat as no-op.
continue
else:
defaults = {
f: intent.get(f)
for f in override_fields
if f in intent
}
# Coerce FK aliases (logo, channel_group, ...) to
# the *_id columns ChannelOverride actually stores.
for raw, mapped in (
("logo", "logo_id"),
("channel_group", "channel_group_id"),
("epg_data", "epg_data_id"),
("stream_profile", "stream_profile_id"),
):
if raw in intent and mapped not in defaults:
val = intent[raw]
defaults[mapped] = (
val.id if hasattr(val, "id") else val
)
overrides_to_upsert.append((channel.id, defaults))
if channels_to_clear:
ChannelOverride.objects.filter(
channel_id__in=channels_to_clear
).delete()
# Bulk upsert keeps a 1000-channel batch to two
# statements (one INSERT, one UPDATE) instead of the
# per-row SELECT + INSERT-or-UPDATE that update_or_create
# would generate.
if overrides_to_upsert:
existing_overrides = {
o.channel_id: o
for o in ChannelOverride.objects.filter(
channel_id__in=[
cid for cid, _ in overrides_to_upsert
]
)
}
to_create = []
to_update = []
update_field_set = set()
for channel_id, defaults in overrides_to_upsert:
existing = existing_overrides.get(channel_id)
if existing:
for f, v in defaults.items():
setattr(existing, f, v)
update_field_set.add(f)
to_update.append(existing)
else:
to_create.append(
ChannelOverride(
channel_id=channel_id, **defaults
)
)
if to_update:
ChannelOverride.objects.bulk_update(
to_update,
fields=list(update_field_set),
batch_size=200,
)
if to_create:
ChannelOverride.objects.bulk_create(
to_create, batch_size=200
)
# Drop override rows that ended up all-null; an empty
# override would falsely surface as active in the UI.
touched_ids = [cid for cid, _ in overrides_to_upsert]
empty_overrides = [
o for o in ChannelOverride.objects.filter(
channel_id__in=touched_ids
)
if not o.has_any_override()
]
if empty_overrides:
ChannelOverride.objects.filter(
id__in=[o.id for o in empty_overrides]
).delete()
# Queryset writes leave the reverse-OneToOne cache stale;
# clear it so the serializer reads the new override state.
touched_channel_ids = {cid for cid, _ in overrides_to_upsert}
touched_channel_ids.update(channels_to_clear)
if touched_channel_ids:
for channel, _ in validated_updates:
if channel.id not in touched_channel_ids:
continue
try:
channel._state.fields_cache.pop("override", None)
except AttributeError:
pass
if hasattr(channel, "_channel_override_cache"):
delattr(channel, "_channel_override_cache")
# Handle streams M2M updates separately
for channel, streams in streams_updates:
normalized_ids = [
@ -1032,21 +1562,158 @@ class ChannelViewSet(viewsets.ModelViewSet):
@action(detail=False, methods=["get"], url_path="summary")
def summary(self, request, *args, **kwargs):
"""Return a lightweight list of channels with only the fields needed by the TV Guide."""
queryset = self.filter_queryset(self.get_queryset())
data = list(
queryset.values(
"id",
"name",
"logo_id",
"channel_number",
"uuid",
"epg_data_id",
"channel_group_id",
)
"""Return a lightweight list of channels with only the fields needed by the TV Guide.
The TV Guide is a downstream output surface like HDHR / M3U / EPG /
XC and must reflect the user's overrides. Effective values are
coalesced at the SQL layer; the annotated columns are renamed
back to the raw field names on the way out so the response
shape stays unchanged for the frontend.
"""
from .managers import with_effective_values
queryset = with_effective_values(
self.filter_queryset(self.get_queryset())
)
data = [
{
"id": row["id"],
"uuid": row["uuid"],
"name": row["effective_name"],
"logo_id": row["effective_logo_id"],
"channel_number": row["effective_channel_number"],
"epg_data_id": row["effective_epg_data_id"],
"channel_group_id": row["effective_channel_group_id"],
}
for row in queryset.values(
"id",
"uuid",
"effective_name",
"effective_logo_id",
"effective_channel_number",
"effective_epg_data_id",
"effective_channel_group_id",
)
]
return Response(data)
@extend_schema(
parameters=[
OpenApiParameter(
name="start",
type=OpenApiTypes.NUMBER,
location=OpenApiParameter.QUERY,
required=True,
description="Inclusive lower bound of the range to scan.",
),
OpenApiParameter(
name="end",
type=OpenApiTypes.NUMBER,
location=OpenApiParameter.QUERY,
required=False,
description=(
"Inclusive upper bound. If omitted or equal to start, "
"behaves as a single-number lookup."
),
),
],
responses={
200: inline_serializer(
name="ChannelsInRangeResponse",
fields={
"occupants": serializers.ListField(
child=serializers.DictField()
)
},
)
},
description=(
"Returns the channels (including those whose effective number is "
"set via override) currently occupying numbers within the given "
"range. Used by the group settings form to surface inline range "
"conflict warnings. Capped at 50 entries to bound the response "
"payload; the frontend only needs to know whether any conflicts "
"exist after filtering, not the entire list."
),
)
@action(detail=False, methods=["get"], url_path="numbers-in-range")
def numbers_in_range(self, request, *args, **kwargs):
from .managers import with_effective_values
raw_start = request.query_params.get("start")
raw_end = request.query_params.get("end")
if raw_start is None or raw_start == "":
return Response({"occupants": []})
try:
start = float(raw_start)
except (TypeError, ValueError):
return Response(
{"detail": "Invalid start value"},
status=status.HTTP_400_BAD_REQUEST,
)
try:
end = float(raw_end) if raw_end not in (None, "") else start
except (TypeError, ValueError):
return Response(
{"detail": "Invalid end value"},
status=status.HTTP_400_BAD_REQUEST,
)
if end < start:
start, end = end, start
queryset = (
with_effective_values(
Channel.objects.all(), select_related_fks=True
)
.filter(
effective_channel_number__gte=start,
effective_channel_number__lte=end,
)
.order_by("effective_channel_number")[:50]
)
occupants = []
for occupant in queryset:
effective_group = getattr(
occupant, "effective_channel_group_obj", None
)
group_name = (
getattr(effective_group, "name", None)
if effective_group is not None
else None
)
effective_group_id = getattr(
occupant, "effective_channel_group_id", None
)
override = getattr(occupant, "override", None)
override_sets_number = bool(
override is not None and override.channel_number is not None
)
occupants.append(
{
"id": occupant.id,
"name": getattr(
occupant, "effective_name", occupant.name
),
"channel_number": getattr(
occupant,
"effective_channel_number",
occupant.channel_number,
),
"channel_group": group_name,
"channel_group_id": effective_group_id,
"auto_created": bool(occupant.auto_created),
"auto_created_by_account_id": (
occupant.auto_created_by_id
if occupant.auto_created_by_id
else None
),
"has_channel_number_override": override_sets_number,
}
)
return Response({"occupants": occupants})
@extend_schema(
methods=["POST"],
description="Retrieve channels by a list of UUIDs using POST to avoid URL length limitations",

363
apps/channels/compact_numbering.py Normal file
View file

@ -0,0 +1,363 @@
"""Compact channel numbering helpers.
The per-group `compact_numbering` custom_property is opt-in. When enabled
on a ChannelGroupM3UAccount, the group's auto-created channels get packed
contiguously into the group's [start, end] range:
* Visible without channel_number override: assigned sequentially
* Hidden without channel_number override: channel_number set to NULL
(released; the slot becomes available for visible channels)
* Override-pinned (any visibility): untouched; the override's
channel_number is treated as a global reservation that other channels
skip when packing
Used by sync_auto_channels (full-pack pass), the post_save Channel signal
(single-channel unhide), the bulk-edit endpoint (bulk unhide), and a
manual per-group re-pack endpoint.
Trade-off the user opts into: channel numbers may shift when hide / unhide
state changes. To pin a number through hide/unhide cycles, set a
channel_number override - the override is honored as a reservation.
"""
import logging
from django.db import transaction
from .models import Channel, ChannelOverride, ChannelGroupM3UAccount
from apps.m3u.tasks import _custom_properties_as_dict, _next_available_number
from core.utils import (
acquire_task_lock,
natural_sort_key,
release_task_lock,
)
logger = logging.getLogger(__name__)
def is_compact_group(group_relation):
"""Return True if the given ChannelGroupM3UAccount is in compact mode."""
cp = _custom_properties_as_dict(group_relation.custom_properties)
return bool(cp.get("compact_numbering"))
def get_group_relation_for_channel(channel):
"""Resolve the ChannelGroupM3UAccount that owns this auto-created
channel. Returns None for manual channels, or when the relation has
been deleted."""
if (
not channel.auto_created
or not channel.auto_created_by_id
or not channel.channel_group_id
):
return None
try:
return ChannelGroupM3UAccount.objects.get(
m3u_account_id=channel.auto_created_by_id,
channel_group_id=channel.channel_group_id,
)
except ChannelGroupM3UAccount.DoesNotExist:
return None
def build_reserved_set(exclude_channel_ids=None, range_start=None, range_end=None):
"""Return the set of channel numbers currently 'claimed' system-wide
for the purposes of a compact pack:
* Every ChannelOverride.channel_number value (overrides reserve
their effective number; duplicates across overrides are allowed
and collapse to a single reservation via set semantics)
* Every Channel.channel_number value EXCEPT for channels in the
passed-in exclude set (those are about to be reassigned)
When the caller knows the target group has a bounded range, pass
`range_start` and `range_end` to scope the scan to numbers that
could possibly collide. Without scoping, a single signal-driven
unhide reads every channel_number in the database; with scoping it
reads at most the values within [start, end] which for typical
cable-style ranges is hundreds rather than tens of thousands.
Float vs int normalization is unnecessary because Python treats
50 and 50.0 as equal and produces the same hash, so set membership
works directly across both types.
"""
exclude_channel_ids = set(exclude_channel_ids or [])
override_qs = ChannelOverride.objects.filter(channel_number__isnull=False)
other_qs = Channel.objects.exclude(channel_number__isnull=True).exclude(
id__in=exclude_channel_ids
)
if range_start is not None:
override_qs = override_qs.filter(channel_number__gte=range_start)
other_qs = other_qs.filter(channel_number__gte=range_start)
if range_end is not None:
override_qs = override_qs.filter(channel_number__lte=range_end)
other_qs = other_qs.filter(channel_number__lte=range_end)
reserved = set(override_qs.values_list("channel_number", flat=True))
reserved.update(other_qs.values_list("channel_number", flat=True))
reserved.discard(None)
return reserved
def _channel_has_number_override(channel):
try:
ov = channel.override
except ChannelOverride.DoesNotExist:
return False
return ov is not None and ov.channel_number is not None
def assign_compact_numbers_for_channels(channel_ids):
"""For each channel ID in the input that became eligible for a number
(visible, auto-created, no number override, in a compact-mode group),
assign the next available channel number in the group's [start, end]
range. Channels whose group is not in compact mode are skipped silently
so callers can pass mixed batches without filtering up front.
Used by both the post_save signal (single-channel unhide) and the bulk
edit endpoint (bulk unhide). Returns dict {channel_id: number_or_None}.
"""
if not channel_ids:
return {}
channels = list(
Channel.objects.filter(
id__in=channel_ids,
auto_created=True,
auto_created_by__isnull=False,
hidden_from_output=False,
channel_number__isnull=True,
).select_related("override", "channel_group", "auto_created_by")
)
if not channels:
return {}
# Group channels by (account_id, group_id) so each unique pair's
# ChannelGroupM3UAccount is resolved with a single SELECT rather than
# one per channel.
by_pair = {}
for ch in channels:
if _channel_has_number_override(ch):
continue
key = (ch.auto_created_by_id, ch.channel_group_id)
by_pair.setdefault(key, []).append(ch)
if not by_pair:
return {}
pair_keys = list(by_pair.keys())
relations_by_pair = {}
relations_qs = ChannelGroupM3UAccount.objects.filter(
m3u_account_id__in={k[0] for k in pair_keys},
channel_group_id__in={k[1] for k in pair_keys},
)
for rel in relations_qs:
relations_by_pair[(rel.m3u_account_id, rel.channel_group_id)] = rel
by_relation = {}
for key, group_channels in by_pair.items():
rel = relations_by_pair.get(key)
if rel is None or not is_compact_group(rel):
continue
by_relation[rel.id] = (rel, group_channels)
# Group by account so writes share the `refresh_single_m3u_account`
# lock used by sync_auto_channels and the manual repack endpoint.
# If sync is in flight for an account, defer to it.
by_account = {}
for rel, group_channels in by_relation.values():
by_account.setdefault(rel.m3u_account_id, []).append(
(rel, group_channels)
)
results = {}
for account_id, account_pairs in by_account.items():
if not acquire_task_lock(
"refresh_single_m3u_account", account_id
):
logger.info(
"Compact unhide deferred for account %s: refresh in progress; "
"next sync will assign numbers.",
account_id,
)
for _, group_channels in account_pairs:
for ch in group_channels:
results[ch.id] = None
continue
# try/finally release: the Redis lock is not transactional, so a
# transaction.on_commit release would leak the lock when an
# outer atomic rolls back, blocking subsequent syncs until TTL.
try:
with transaction.atomic():
for rel, group_channels in account_pairs:
start = int(rel.auto_sync_channel_start or 1)
end = (
int(rel.auto_sync_channel_end)
if rel.auto_sync_channel_end
else None
)
reserved = build_reserved_set(
exclude_channel_ids=[c.id for c in group_channels],
range_start=start,
range_end=end,
)
to_update = []
for ch in group_channels:
next_num = _next_available_number(
reserved, start, end=end
)
if next_num is None:
results[ch.id] = None
continue
ch.channel_number = next_num
reserved.add(next_num)
results[ch.id] = next_num
to_update.append(ch)
if to_update:
Channel.objects.bulk_update(
to_update, ["channel_number"], batch_size=100
)
finally:
try:
release_task_lock(
"refresh_single_m3u_account", account_id
)
except Exception as e:
logger.warning(
"Failed to release compact-unhide lock for account "
"%s: %s",
account_id,
e,
)
return results
def repack_group(group_relation):
"""Renumber every auto-created channel in the given group+account.
Visible non-override channels are assigned sequentially in
[start, end] using the group's configured channel_sort_order.
Hidden non-override channels have their channel_number set to
None (slot released). Override-pinned channels are untouched.
Returns a dict with assigned/released/failed counts. ``failed``
counts visible channels that could not fit because the range was
exhausted; their channel_number is set to None so the state is
unambiguous instead of stuck at a stale number.
All writes run inside a single transaction so concurrent readers
(HDHR/M3U/EPG output paths) never observe a half-packed state.
"""
with transaction.atomic():
return _repack_inner(group_relation)
def _repack_inner(group_relation):
account_id = group_relation.m3u_account_id
group_id = group_relation.channel_group_id
cp = _custom_properties_as_dict(group_relation.custom_properties)
sort_order = cp.get("channel_sort_order") or ""
sort_reverse = bool(cp.get("channel_sort_reverse"))
start = int(group_relation.auto_sync_channel_start or 1)
end = (
int(group_relation.auto_sync_channel_end)
if group_relation.auto_sync_channel_end
else None
)
channels = list(
Channel.objects.filter(
auto_created=True,
auto_created_by_id=account_id,
channel_group_id=group_id,
).select_related("override")
)
visible = []
hidden = []
pinned = []
for ch in channels:
if _channel_has_number_override(ch):
pinned.append(ch)
elif ch.hidden_from_output:
hidden.append(ch)
else:
visible.append(ch)
# Sort the visible set by the group's configured channel_sort_order.
# Provider order (the default) preserves DB-iteration order which is
# roughly creation order; treat unrecognized values the same way.
if sort_order == "name":
visible.sort(
key=lambda c: natural_sort_key(c.name or ""),
reverse=sort_reverse,
)
elif sort_order == "tvg_id":
visible.sort(
key=lambda c: c.tvg_id or "",
reverse=sort_reverse,
)
elif sort_order == "updated_at":
visible.sort(
key=lambda c: c.updated_at,
reverse=sort_reverse,
)
# Exclude every channel in this group: the pinned channel's raw value
# is irrelevant (the override is reserved globally and cleared below).
# Scope to the group's range when bounded to keep the set small.
affected_ids = [c.id for c in (visible + hidden + pinned)]
reserved = build_reserved_set(
exclude_channel_ids=affected_ids,
range_start=start,
range_end=end,
)
assigned_count = 0
failed_count = 0
visible_to_update = []
for ch in visible:
next_num = _next_available_number(reserved, start, end=end)
if next_num is None:
failed_count += 1
if ch.channel_number is not None:
ch.channel_number = None
visible_to_update.append(ch)
continue
if ch.channel_number != next_num:
ch.channel_number = next_num
visible_to_update.append(ch)
reserved.add(next_num)
assigned_count += 1
if visible_to_update:
Channel.objects.bulk_update(
visible_to_update, ["channel_number"], batch_size=100
)
released_count = 0
hidden_with_num = [c for c in hidden if c.channel_number is not None]
if hidden_with_num:
for c in hidden_with_num:
c.channel_number = None
Channel.objects.bulk_update(
hidden_with_num, ["channel_number"], batch_size=100
)
released_count = len(hidden_with_num)
# Pinned channels: clear raw channel_number. The override controls
# their effective number; leaving a stale raw value would pollute
# uniqueness checks and could resurrect on override clear.
pinned_with_num = [c for c in pinned if c.channel_number is not None]
if pinned_with_num:
for c in pinned_with_num:
c.channel_number = None
Channel.objects.bulk_update(
pinned_with_num, ["channel_number"], batch_size=100
)
return {
"assigned": assigned_count,
"released": released_count,
"failed": failed_count,
}

55
apps/channels/managers.py Normal file
View file

@ -0,0 +1,55 @@
"""
Queryset helpers that resolve effective Channel field values.
Each Channel can optionally have a related ChannelOverride row carrying user
edits to any subset of its user-facing fields. Sync never touches the override
row; provider metadata flows directly into Channel.* and the override table
sits alongside with a nullable value per field. The helpers here coalesce the
two sources into `effective_*` annotations so output querysets can sort,
filter, and emit values correctly at SQL level (avoiding 23+ Python-side
resolutions across the codebase).
"""
from django.db.models.functions import Coalesce
OVERRIDABLE_FIELDS = (
"name",
"channel_number",
"channel_group_id",
"logo_id",
"tvg_id",
"tvc_guide_stationid",
"epg_data_id",
"stream_profile_id",
)
def with_effective_values(queryset, select_related_fks=False):
"""
Annotate the channels queryset with `effective_*` columns that resolve to
the override value when set, otherwise fall back to the channel's own
value. Always eagerly loads the override one-to-one to avoid N+1 when the
caller reads annotated attributes and then the related override.
Pass `select_related_fks=True` when the output path will access FK objects
through the `effective_*_obj` Channel properties; this pulls the override's
logo, channel_group, epg_data, and stream_profile in the same query so
those accessors do not trigger per-row lookups.
"""
annotations = {
f"effective_{field}": Coalesce(
f"override__{field}",
field,
)
for field in OVERRIDABLE_FIELDS
}
qs = queryset.select_related("override").annotate(**annotations)
if select_related_fks:
qs = qs.select_related(
"override__logo",
"override__channel_group",
"override__epg_data",
"override__stream_profile",
)
return qs

156
apps/channels/migrations/0037_auto_sync_overhaul.py Normal file
View file

@ -0,0 +1,156 @@
"""
Auto-sync overhaul (FR #1196): per-field channel overrides, hide-from-output
flag, configurable auto-sync number range, and nullable channel_number for
compact-numbering slot release.
Bundled operations (in forward order; reversed on rollback):
1. AddField Channel.hidden_from_output
2. CreateModel ChannelOverride (one-to-one Channel)
3. AddField ChannelGroupM3UAccount.auto_sync_channel_end
4. RunPython backfill_auto_created_by_null (orphan re-attribution / demotion)
5. AlterField Channel.channel_number (nullable)
6. RunPython noop / reverse_backfill_channel_number_nulls
(rollback-safety hook; runs FIRST on un-apply, fills NULLs
so step 5 reverse can re-impose NOT NULL)
"""
import django.db.models.deletion
from django.db import migrations, models
from django.db.models import Max
def backfill_auto_created_by_null(apps, schema_editor):
"""
Re-attribute or demote `auto_created=True, auto_created_by=NULL` rows.
Sync only touches rows where `auto_created_by=account`, so orphans
accumulate indefinitely. Best-effort re-attribute via the channel's
streams' single owning account; otherwise demote to manual by clearing
`auto_created`. The channel and any user customization survive; sync
will not touch a demoted row again.
"""
Channel = apps.get_model("dispatcharr_channels", "Channel")
ChannelStream = apps.get_model("dispatcharr_channels", "ChannelStream")
orphans = Channel.objects.filter(auto_created=True, auto_created_by__isnull=True)
total = orphans.count()
if total == 0:
return
print(f"\n Found {total} auto_created channels with NULL auto_created_by")
reattributed = 0
demoted = 0
for channel in orphans.iterator(chunk_size=200):
account_ids = set(
ChannelStream.objects.filter(channel=channel)
.values_list("stream__m3u_account_id", flat=True)
)
account_ids.discard(None)
if len(account_ids) == 1:
channel.auto_created_by_id = next(iter(account_ids))
channel.save(update_fields=["auto_created_by"])
reattributed += 1
else:
channel.auto_created = False
channel.save(update_fields=["auto_created"])
demoted += 1
print(
f" Re-attributed: {reattributed}, demoted to manual "
f"(ambiguous/no streams): {demoted}"
)
def reverse_auto_created_by_null(apps, schema_editor):
# Forward decisions cannot be cleanly reverted (no record of the
# original NULL state). Leaving the re-attributions and demotions in
# place is safer than restoring NULLs the schema may not accept.
pass
def noop(apps, schema_editor):
pass
def reverse_backfill_channel_number_nulls(apps, schema_editor):
"""
Rollback-safety hook for the channel_number nullable AlterField.
Runs FIRST on un-apply (operations reverse in list order) and assigns
sequential channel numbers above the current max to any NULL rows so
the AlterField reverse (nullable to NOT NULL) succeeds without a
constraint violation. The user can re-hide or re-number these
channels after they have rolled back.
"""
Channel = apps.get_model("dispatcharr_channels", "Channel")
null_qs = Channel.objects.filter(channel_number__isnull=True)
null_count = null_qs.count()
if null_count == 0:
return
max_num = Channel.objects.aggregate(m=Max("channel_number"))["m"] or 0.0
next_num = float(max_num) + 1.0
print(
f"\n Backfilling channel_number on {null_count} NULL row(s) "
f"starting at {int(next_num)} so rollback can re-impose NOT NULL"
)
for ch in null_qs.order_by("id"):
ch.channel_number = next_num
ch.save(update_fields=["channel_number"])
next_num += 1.0
class Migration(migrations.Migration):
dependencies = [
('core', '022_default_user_limit_settings'),
('dispatcharr_channels', '0036_alter_stream_name'),
('epg', '0022_alter_epgdata_name'),
]
operations = [
migrations.AddField(
model_name='channel',
name='hidden_from_output',
field=models.BooleanField(
db_index=True,
default=False,
help_text='Exclude this channel from downstream client output (HDHR, M3U, EPG, XC). Auto-sync still updates provider metadata.',
),
),
migrations.CreateModel(
name='ChannelOverride',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(blank=True, max_length=512, null=True)),
('channel_number', models.FloatField(blank=True, null=True)),
('tvg_id', models.CharField(blank=True, max_length=255, null=True)),
('tvc_guide_stationid', models.CharField(blank=True, max_length=255, null=True)),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('channel', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='override', to='dispatcharr_channels.channel')),
('channel_group', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='+', to='dispatcharr_channels.channelgroup')),
('epg_data', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='+', to='epg.epgdata')),
('logo', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='+', to='dispatcharr_channels.logo')),
('stream_profile', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='+', to='core.streamprofile')),
],
),
migrations.AddField(
model_name='channelgroupm3uaccount',
name='auto_sync_channel_end',
field=models.FloatField(
blank=True,
help_text='Optional upper bound for auto-created channel numbers in this group. Leave blank for unlimited fill. Overflow streams are skipped and reported in the completion notification.',
null=True,
),
),
migrations.RunPython(backfill_auto_created_by_null, reverse_auto_created_by_null),
migrations.AlterField(
model_name='channel',
name='channel_number',
field=models.FloatField(blank=True, db_index=True, null=True),
),
migrations.RunPython(noop, reverse_backfill_channel_number_nulls),
]

149
apps/channels/models.py
View file

@ -291,9 +291,23 @@ class ChannelManager(models.Manager):
def active(self):
return self.all()
def with_effective_values(self, select_related_fks=False):
"""
Chainable shortcut for the override-aware queryset annotations,
delegating to the canonical helper in ``apps.channels.managers``
so the function form (``with_effective_values(qs)``) and the
manager form (``Channel.objects.with_effective_values()``) are
both valid entry points and stay in sync.
"""
from apps.channels.managers import with_effective_values
return with_effective_values(
self.get_queryset(), select_related_fks=select_related_fks
)
class Channel(models.Model):
channel_number = models.FloatField(db_index=True)
channel_number = models.FloatField(db_index=True, null=True, blank=True)
name = models.CharField(max_length=512)
logo = models.ForeignKey(
"Logo",
@ -360,6 +374,16 @@ class Channel(models.Model):
help_text="The M3U account that auto-created this channel"
)
# Hidden channels are excluded from HDHR, M3U, EPG, and XC output queries.
# Auto-sync still recognizes them so they are not recreated when their
# underlying provider stream persists; this is an output-layer concern, not
# a sync-time flag.
hidden_from_output = models.BooleanField(
default=False,
db_index=True,
help_text="Exclude this channel from downstream client output (HDHR, M3U, EPG, XC). Auto-sync still updates provider metadata."
)
created_at = models.DateTimeField(
auto_now_add=True,
help_text="Timestamp when this channel was created"
@ -369,6 +393,8 @@ class Channel(models.Model):
help_text="Timestamp when this channel was last updated"
)
objects = ChannelManager()
def clean(self):
# Enforce unique channel_number within a given group
existing = Channel.objects.filter(
@ -384,11 +410,46 @@ class Channel(models.Model):
@classmethod
def get_next_available_channel_number(cls, starting_from=1):
used_numbers = set(cls.objects.all().values_list("channel_number", flat=True))
n = starting_from
while n in used_numbers:
n += 1
return n
# Both raw and override channel numbers are reserved. Handing out a
# raw number currently masked by an override would create a deferred
# collision once the override is cleared.
from apps.channels.compact_numbering import build_reserved_set
from apps.m3u.tasks import _next_available_number
reserved = build_reserved_set()
return _next_available_number(reserved, starting_from)
def _resolved_override(self):
"""Return the related ChannelOverride or None, tolerant of no row."""
try:
return self.override
except ChannelOverride.DoesNotExist:
return None
def _resolve_effective_fk(self, field_name):
"""Pick the override's FK object if set, otherwise the channel's own."""
override = self._resolved_override()
if override is not None:
override_val = getattr(override, field_name, None)
if override_val is not None:
return override_val
return getattr(self, field_name)
@property
def effective_logo_obj(self):
return self._resolve_effective_fk("logo")
@property
def effective_channel_group_obj(self):
return self._resolve_effective_fk("channel_group")
@property
def effective_epg_data_obj(self):
return self._resolve_effective_fk("epg_data")
@property
def effective_stream_profile_obj(self):
return self._resolve_effective_fk("stream_profile")
# @TODO: honor stream's stream profile
def get_stream_profile(self):
@ -840,6 +901,76 @@ class Channel(models.Model):
return True
class ChannelOverride(models.Model):
"""
Per-field user overrides for auto-synced channels.
Each nullable column represents a user-provided value that takes precedence
over the matching field on the related Channel. Sync writes only to
Channel.* fields and never to this table, so provider metadata keeps
flowing while user customizations persist across refreshes. Output
querysets resolve the effective value via
`apps.channels.managers.with_effective_values`.
"""
channel = models.OneToOneField(
Channel,
on_delete=models.CASCADE,
related_name="override",
)
name = models.CharField(max_length=512, null=True, blank=True)
channel_number = models.FloatField(null=True, blank=True)
channel_group = models.ForeignKey(
ChannelGroup,
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="+",
)
logo = models.ForeignKey(
"Logo",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="+",
)
tvg_id = models.CharField(max_length=255, null=True, blank=True)
tvc_guide_stationid = models.CharField(max_length=255, null=True, blank=True)
epg_data = models.ForeignKey(
EPGData,
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="+",
)
stream_profile = models.ForeignKey(
StreamProfile,
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="+",
)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
def __str__(self):
return f"Override for channel {self.channel_id}"
def has_any_override(self) -> bool:
return any(
getattr(self, field) is not None
for field in (
"name",
"channel_number",
"channel_group_id",
"logo_id",
"tvg_id",
"tvc_guide_stationid",
"epg_data_id",
"stream_profile_id",
)
)
class ChannelProfile(models.Model):
name = models.CharField(max_length=100, unique=True)
@ -887,6 +1018,12 @@ class ChannelGroupM3UAccount(models.Model):
blank=True,
help_text='Starting channel number for auto-created channels in this group'
)
# Optional upper bound; out-of-range streams fail. NULL = unlimited.
auto_sync_channel_end = models.FloatField(
null=True,
blank=True,
help_text='Optional upper bound for auto-created channel numbers in this group. Leave blank for unlimited fill. Overflow streams are skipped and reported in the completion notification.'
)
last_seen = models.DateTimeField(
default=timezone.now,
db_index=True,

424
apps/channels/serializers.py
View file

@ -6,6 +6,7 @@ from .models import (
Stream,
Channel,
ChannelGroup,
ChannelOverride,
ChannelStream,
ChannelGroupM3UAccount,
Logo,
@ -17,6 +18,7 @@ from .models import (
from apps.epg.serializers import EPGDataSerializer
from core.models import StreamProfile
from apps.epg.models import EPGData
from django.db import connection, transaction
from django.urls import reverse
from rest_framework import serializers
from django.utils import timezone
@ -154,12 +156,52 @@ class ChannelGroupM3UAccountSerializer(serializers.ModelSerializer):
m3u_accounts = serializers.IntegerField(source="m3u_accounts.id", read_only=True)
enabled = serializers.BooleanField()
auto_channel_sync = serializers.BooleanField(default=False)
auto_sync_channel_start = serializers.FloatField(allow_null=True, required=False)
auto_sync_channel_start = serializers.FloatField(
allow_null=True, required=False, min_value=1
)
auto_sync_channel_end = serializers.FloatField(
allow_null=True, required=False, min_value=1
)
custom_properties = serializers.JSONField(required=False)
# Provider stream count for this group+account. Lets users size an
# optional end-range without first running a blind sync.
stream_count = serializers.SerializerMethodField()
class Meta:
model = ChannelGroupM3UAccount
fields = ["m3u_accounts", "channel_group", "enabled", "auto_channel_sync", "auto_sync_channel_start", "custom_properties", "is_stale", "last_seen"]
fields = [
"m3u_accounts",
"channel_group",
"enabled",
"auto_channel_sync",
"auto_sync_channel_start",
"auto_sync_channel_end",
"custom_properties",
"is_stale",
"last_seen",
"stream_count",
]
def get_stream_count(self, obj):
"""
Return the number of streams for this (m3u_account, channel_group)
pair. A parent serializer (e.g. M3UAccountSerializer) may seed
``context["stream_counts"]`` with a pre-aggregated dict keyed by
``(m3u_account_id, channel_group_id)`` to avoid one COUNT per row;
when present, it is used as the source of truth. The per-row
COUNT fallback is correct for stand-alone serialization (rare,
low-volume) and exists so direct ChannelGroupM3UAccount queries
do not require callers to know the seeding pattern.
"""
counts = self.context.get("stream_counts")
if counts is not None:
return counts.get((obj.m3u_account_id, obj.channel_group_id), 0)
from apps.channels.models import Stream
return Stream.objects.filter(
m3u_account_id=obj.m3u_account_id,
channel_group_id=obj.channel_group_id,
).count()
def to_representation(self, instance):
data = super().to_representation(instance)
@ -179,6 +221,26 @@ class ChannelGroupM3UAccountSerializer(serializers.ModelSerializer):
return super().to_internal_value(data)
def validate(self, attrs):
# Partial PATCHes only carry submitted fields; fill missing
# start/end from the instance so the validator catches a PATCH
# that lowers end past the existing start.
start = attrs.get("auto_sync_channel_start")
end = attrs.get("auto_sync_channel_end")
if start is None and self.instance is not None:
start = self.instance.auto_sync_channel_start
if end is None and self.instance is not None:
end = self.instance.auto_sync_channel_end
if start is not None and end is not None and end < start:
raise serializers.ValidationError(
{
"auto_sync_channel_end": (
"End must be greater than or equal to start."
)
}
)
return super().validate(attrs)
#
# Channel Group
#
@ -195,12 +257,14 @@ class ChannelGroupSerializer(serializers.ModelSerializer):
fields = ["id", "name", "channel_count", "m3u_account_count", "m3u_accounts"]
def get_channel_count(self, obj):
"""Get count of channels in this group"""
return obj.channels.count()
# Use the queryset annotation when available (list path); fall back
# to a live query for retrieve/create/update where it isn't set.
v = getattr(obj, 'channel_count', None)
return v if v is not None else obj.channels.count()
def get_m3u_account_count(self, obj):
"""Get count of M3U accounts associated with this group"""
return obj.m3u_accounts.count()
v = getattr(obj, 'm3u_account_count', None)
return v if v is not None else obj.m3u_accounts.count()
class ChannelProfileSerializer(serializers.ModelSerializer):
@ -240,6 +304,62 @@ class BulkChannelProfileMembershipSerializer(serializers.Serializer):
return value
#
# Channel override
#
# Nullable per-field overrides resolved over the parent Channel in read
# paths. Embedded in ChannelSerializer so clients can upsert/clear in the
# same PATCH that targets direct channel fields.
class ChannelOverrideSerializer(serializers.ModelSerializer):
# HDHR clients reject negative GuideNumber and zero is not a real
# provider value, so reject both at the API boundary.
channel_number = serializers.FloatField(
allow_null=True, required=False, min_value=0.0001
)
channel_group_id = serializers.PrimaryKeyRelatedField(
queryset=ChannelGroup.objects.all(),
source="channel_group",
allow_null=True,
required=False,
)
logo_id = serializers.PrimaryKeyRelatedField(
queryset=Logo.objects.all(),
source="logo",
allow_null=True,
required=False,
)
epg_data_id = serializers.PrimaryKeyRelatedField(
queryset=EPGData.objects.all(),
source="epg_data",
allow_null=True,
required=False,
)
stream_profile_id = serializers.PrimaryKeyRelatedField(
queryset=StreamProfile.objects.all(),
source="stream_profile",
allow_null=True,
required=False,
)
class Meta:
model = ChannelOverride
fields = [
"name",
"channel_number",
"channel_group_id",
"logo_id",
"tvg_id",
"tvc_guide_stationid",
"epg_data_id",
"stream_profile_id",
]
extra_kwargs = {
"name": {"allow_null": True, "required": False},
"tvg_id": {"allow_null": True, "required": False},
"tvc_guide_stationid": {"allow_null": True, "required": False},
}
#
# Channel
#
@ -280,6 +400,32 @@ class ChannelSerializer(serializers.ModelSerializer):
)
auto_created_by_name = serializers.SerializerMethodField()
override = ChannelOverrideSerializer(
required=False,
allow_null=True,
help_text=(
"Per-field overrides for an auto-created channel. "
'Send {"override": {"name": "ESPN"}} to upsert the listed '
'fields, {"override": {"name": null}} to clear specific fields '
'while leaving others, or {"override": null} to delete the '
"override row entirely. Omitting the key leaves any existing "
"override unchanged. Only valid for auto_created=True channels. "
"Duplicate channel_number values across channels are permitted; "
"downstream client behavior on duplicates varies by client."
),
)
source_stream = serializers.SerializerMethodField()
# Effective fields coalesce override over channel column. Consumers
# display these; raw fields remain in the response so the edit form
# can show them as "Provider: X" subtext.
effective_name = serializers.SerializerMethodField()
effective_channel_number = serializers.SerializerMethodField()
effective_channel_group_id = serializers.SerializerMethodField()
effective_logo_id = serializers.SerializerMethodField()
effective_tvg_id = serializers.SerializerMethodField()
effective_tvc_guide_stationid = serializers.SerializerMethodField()
effective_epg_data_id = serializers.SerializerMethodField()
effective_stream_profile_id = serializers.SerializerMethodField()
class Meta:
model = Channel
@ -297,11 +443,88 @@ class ChannelSerializer(serializers.ModelSerializer):
"logo_id",
"user_level",
"is_adult",
"hidden_from_output",
"auto_created",
"auto_created_by",
"auto_created_by_name",
"override",
"source_stream",
"effective_name",
"effective_channel_number",
"effective_channel_group_id",
"effective_logo_id",
"effective_tvg_id",
"effective_tvc_guide_stationid",
"effective_epg_data_id",
"effective_stream_profile_id",
]
def _effective_value(self, obj, field_name):
override = getattr(obj, "_channel_override_cache", None)
if override is None:
try:
override = obj.override
except ChannelOverride.DoesNotExist:
override = None
obj._channel_override_cache = override
if override is not None:
value = getattr(override, field_name, None)
if value is not None:
return value
return getattr(obj, field_name, None)
def get_effective_name(self, obj):
return self._effective_value(obj, "name")
def get_effective_channel_number(self, obj):
return self._effective_value(obj, "channel_number")
def get_effective_channel_group_id(self, obj):
return self._effective_value(obj, "channel_group_id")
def get_effective_logo_id(self, obj):
return self._effective_value(obj, "logo_id")
def get_effective_tvg_id(self, obj):
return self._effective_value(obj, "tvg_id")
def get_effective_tvc_guide_stationid(self, obj):
return self._effective_value(obj, "tvc_guide_stationid")
def get_effective_epg_data_id(self, obj):
return self._effective_value(obj, "epg_data_id")
def get_effective_stream_profile_id(self, obj):
return self._effective_value(obj, "stream_profile_id")
def get_source_stream(self, obj):
"""
Return the originating provider stream for an auto-created channel.
Surfaces the provider stream's name and owning M3U account so the
frontend can render "Auto-created from: <provider> / <stream name>"
in the channel edit form. Returns None for manual channels.
"""
if not self.context.get("include_source_stream", False):
return None
if not obj.auto_created:
return None
# Viewset prefetches `channelstream_set` ordered by `order`, so
# `.all()[0]` reuses the cache and returns the lowest-order entry.
prefetched_list = list(obj.channelstream_set.all())
if not prefetched_list:
return None
cs = prefetched_list[0]
if not cs.stream:
return None
stream = cs.stream
return {
"id": stream.id,
"name": stream.name,
"account_id": stream.m3u_account_id,
"account_name": getattr(stream.m3u_account, "name", None),
}
def to_representation(self, instance):
include_streams = self.context.get("include_streams", False)
@ -309,14 +532,14 @@ class ChannelSerializer(serializers.ModelSerializer):
self.fields["streams"] = serializers.SerializerMethodField()
return super().to_representation(instance)
else:
# Fix: For PATCH/PUT responses, ensure streams are ordered
# Read from the prefetched channelstream_set (ordered by the
# viewset's Prefetch); chaining .order_by() rebuilds the
# queryset and fires one SELECT per row in list responses.
representation = super().to_representation(instance)
if "streams" in representation:
representation["streams"] = list(
instance.streams.all()
.order_by("channelstream__order")
.values_list("id", flat=True)
)
representation["streams"] = [
cs.stream_id for cs in instance.channelstream_set.all()
]
return representation
def get_logo(self, obj):
@ -330,6 +553,7 @@ class ChannelSerializer(serializers.ModelSerializer):
def create(self, validated_data):
streams = validated_data.pop("streams", [])
override_data = validated_data.pop("override", None)
channel_number = validated_data.pop(
"channel_number", Channel.get_next_available_channel_number()
)
@ -341,60 +565,146 @@ class ChannelSerializer(serializers.ModelSerializer):
default_group, _ = ChannelGroup.objects.get_or_create(name="Default Group")
validated_data["channel_group"] = default_group
channel = Channel.objects.create(**validated_data)
# Atomic wrapper keeps the channel insert and its override row
# in the same transaction so a failure on either rolls both back.
with transaction.atomic():
channel = Channel.objects.create(**validated_data)
# Add streams in the specified order
for index, stream in enumerate(streams):
ChannelStream.objects.create(
channel=channel, stream_id=stream.id, order=index
)
# Add streams in the specified order
for index, stream in enumerate(streams):
ChannelStream.objects.create(
channel=channel, stream_id=stream.id, order=index
)
if override_data:
# Manual channels (auto_created=False) have no provider
# value to override; reject the override payload here so a
# programmatic client can't write a semantically meaningless
# row that the frontend would then surface as "Overrides
# active".
if not channel.auto_created:
raise serializers.ValidationError(
{
"override": (
"Cannot set override on a manual channel; "
"overrides only apply to auto-created channels."
)
}
)
obj = ChannelOverride.objects.create(channel=channel, **override_data)
# Drop an all-null override row; an empty override would
# falsely surface as active in the UI.
if not obj.has_any_override():
obj.delete()
return channel
def update(self, instance, validated_data):
"""
PATCH handler for Channel rows. The ``override`` key carries
per-field user overrides for auto-created channels and follows
these rules:
* key absent from payload: no change to existing overrides
* ``{"override": {"field": value}}``: upsert those fields
* ``{"override": {"field": null}}``: clear those specific fields
* ``{"override": null}``: delete the override row entirely
Key presence is what distinguishes "no change" from "delete";
an explicit null means delete. Override mutations are rejected
on manual channels (auto_created=False) since there is no
provider value to override.
"""
streams = validated_data.pop("streams", None)
has_override_key = "override" in self.initial_data
override_data = validated_data.pop("override", None)
# Update standard fields
for attr, value in validated_data.items():
setattr(instance, attr, value)
instance.save()
if streams is not None:
# Normalize stream IDs
normalized_ids = [
stream.id if hasattr(stream, "id") else stream for stream in streams
]
# Get current mapping of stream_id -> ChannelStream
current_links = {
cs.stream_id: cs for cs in instance.channelstream_set.all()
}
# Track existing stream IDs
existing_ids = set(current_links.keys())
new_ids = set(normalized_ids)
# Delete any links not in the new list
to_remove = existing_ids - new_ids
if to_remove:
instance.channelstream_set.filter(stream_id__in=to_remove).delete()
# Update or create with new order
to_update = []
for order, stream_id in enumerate(normalized_ids):
if stream_id in current_links:
cs = current_links[stream_id]
if cs.order != order:
cs.order = order
to_update.append(cs)
else:
ChannelStream.objects.create(
channel=instance, stream_id=stream_id, order=order
# Block override mutations on manual channels (no provider
# value to override). Clearing is a tolerated no-op.
if (
has_override_key
and override_data is not None
and override_data != {}
and not instance.auto_created
):
raise serializers.ValidationError(
{
"override": (
"Cannot set override on a manual channel; "
"overrides only apply to auto-created channels."
)
}
)
if to_update:
ChannelStream.objects.bulk_update(to_update, ["order"])
# Atomic so a failure on the override row rolls back the
# channel update too.
with transaction.atomic():
# Skip save() when only override keys were submitted; a
# no-op UPDATE would bump updated_at and bust caches.
if validated_data:
for attr, value in validated_data.items():
setattr(instance, attr, value)
instance.save()
if has_override_key:
if override_data is None:
# Explicit null: remove the override row.
ChannelOverride.objects.filter(channel=instance).delete()
elif override_data == {}:
# Empty dict has no field intent; no-op.
pass
else:
obj, _ = ChannelOverride.objects.update_or_create(
channel=instance, defaults=override_data
)
# Drop an all-null override; would falsely surface
# as active in the UI.
if not obj.has_any_override():
obj.delete()
# Queryset writes leave the reverse-OneToOne cache stale;
# clear it so to_representation reads the new state.
try:
instance._state.fields_cache.pop("override", None)
except AttributeError:
pass
if hasattr(instance, "_channel_override_cache"):
delattr(instance, "_channel_override_cache")
if streams is not None:
# Normalize stream IDs
normalized_ids = [
stream.id if hasattr(stream, "id") else stream for stream in streams
]
# Get current mapping of stream_id -> ChannelStream
current_links = {
cs.stream_id: cs for cs in instance.channelstream_set.all()
}
# Track existing stream IDs
existing_ids = set(current_links.keys())
new_ids = set(normalized_ids)
# Delete any links not in the new list
to_remove = existing_ids - new_ids
if to_remove:
instance.channelstream_set.filter(stream_id__in=to_remove).delete()
# Update or create with new order
to_update = []
for order, stream_id in enumerate(normalized_ids):
if stream_id in current_links:
cs = current_links[stream_id]
if cs.order != order:
cs.order = order
to_update.append(cs)
else:
ChannelStream.objects.create(
channel=instance, stream_id=stream_id, order=order
)
if to_update:
ChannelStream.objects.bulk_update(to_update, ["order"])
return instance

133
apps/channels/signals.py
View file

@ -1,6 +1,6 @@
# apps/channels/signals.py
from django.db.models.signals import m2m_changed, pre_save, post_save, post_delete
from django.db.models.signals import m2m_changed, pre_save, post_save, post_delete, pre_delete
from django.dispatch import receiver
from django.utils.timezone import now, is_aware, make_aware
from celery.result import AsyncResult
@ -60,6 +60,137 @@ def generate_custom_stream_hash(sender, instance, created, **kwargs):
# Use update to avoid triggering signals again
Stream.objects.filter(id=instance.id).update(stream_hash=instance.stream_hash)
@receiver(pre_delete, sender=Channel)
def stop_proxy_session_before_channel_delete(sender, instance, **kwargs):
"""
When a Channel is deleted, stop any active TS proxy session for it first.
Without this, the proxy's Redis state (ts_proxy:channel:{uuid}:*) survives
the Channel row and the connected clients' "Stop" button hits
`ChannelService.stop_channel(uuid)`, which calls `Channel.objects.get(uuid=...)`
and crashes with DoesNotExist (reported as 'Channel not found' in the UI).
Users then cannot close the stream; source-side connection limits stay
consumed. Covers manual deletes, bulk deletes, and sync-driven deletes
via the same signal path.
"""
try:
from apps.proxy.ts_proxy.services.channel_service import ChannelService
channel_uuid = str(instance.uuid) if instance.uuid else None
if not channel_uuid:
return
# Best-effort: if the channel has no active session the service
# returns a benign 'Channel not found' result, which is ignored.
ChannelService.stop_channel(channel_uuid)
except Exception as e:
# Never block a channel delete on proxy cleanup failure. Log and
# continue so at least the DB row is removed.
logger.warning(
"Failed to stop proxy session before deleting channel %s: %s",
getattr(instance, "id", "<unknown>"),
e,
)
@receiver(post_save, sender=Channel)
def assign_compact_number_on_unhide(sender, instance, created, **kwargs):
"""When a channel transitions from hidden to visible under compact
numbering, immediately assign the next available number from its
group's range. Without this, an unhide would leave the channel at
NULL until the next M3U refresh, which is too long a delay for what
is meant to feel like an instant action.
Bails out for the common cases where assignment is not appropriate:
a fresh channel (newly created), a channel that already has a
number, a hidden channel, a manual channel, or a channel whose group
is not in compact mode. The assignment helper does the same checks
defensively, but bailing here keeps the signal out of the helper
code path on every channel save.
"""
if created:
return
# Skip the signal when update_fields proves hidden_from_output was not
# touched. Sync sets update_fields on every save, so this keeps the
# signal off the sync hot path.
update_fields = kwargs.get("update_fields")
if update_fields is not None and "hidden_from_output" not in update_fields:
return
if instance.hidden_from_output:
return
if instance.channel_number is not None:
return
if not instance.auto_created or not instance.auto_created_by_id:
return
try:
from .compact_numbering import assign_compact_numbers_for_channels
assign_compact_numbers_for_channels([instance.id])
# The helper writes via queryset .update() which skips
# in-memory state. Reload so a same-request serializer
# response carries the assigned number, not stale None.
try:
instance.refresh_from_db(fields=["channel_number"])
except Exception:
# Refresh failure (race with delete) is non-fatal.
pass
except Exception as e:
# Do not propagate. The save succeeded; the assignment is
# recoverable on next sync or manual repack.
logger.warning(
"Compact unhide assignment failed for channel %s: %s",
instance.id,
e,
)
@receiver(post_save, sender=Channel)
def release_compact_number_on_hide(sender, instance, created, **kwargs):
"""When a channel transitions from visible to hidden under compact
numbering, immediately release its channel_number slot. Without
this, the slot stays occupied until the next sync's repack pass,
so the user sees their hidden channels still consuming numbers in
the table for an indeterminate window. Mirror image of
`assign_compact_number_on_unhide` above.
Bails out for the common cases where release is not appropriate:
a fresh channel (newly created), a non-hidden channel, a channel
that has no number to release, a manual channel, a channel without
a known auto_created_by, or a channel whose group is not in
compact mode.
"""
if created:
return
update_fields = kwargs.get("update_fields")
if update_fields is not None and "hidden_from_output" not in update_fields:
return
if not instance.hidden_from_output:
return
if instance.channel_number is None:
return
if not instance.auto_created or not instance.auto_created_by_id:
return
try:
from .compact_numbering import (
get_group_relation_for_channel,
is_compact_group,
)
relation = get_group_relation_for_channel(instance)
if not relation or not is_compact_group(relation):
return
except Exception:
return
# Release the slot. Queryset .update() bypasses the post_save signal
# chain that this handler is itself running inside.
Channel.objects.filter(id=instance.id).update(channel_number=None)
# Refresh in-memory instance so the same-request serializer
# response surfaces channel_number=None instead of stale value.
try:
instance.refresh_from_db(fields=["channel_number"])
except Exception:
pass
@receiver(post_save, sender=Channel)
def refresh_epg_programs(sender, instance, created, **kwargs):
"""

132
apps/channels/tests/test_channel_api.py
View file

@ -3,7 +3,7 @@ from django.contrib.auth import get_user_model
from rest_framework.test import APIClient
from rest_framework import status
from apps.channels.models import Channel, ChannelGroup
from apps.channels.models import Channel, ChannelGroup, ChannelOverride
User = get_user_model()
@ -209,3 +209,133 @@ class ChannelBulkEditAPITests(TestCase):
self.assertEqual(self.channel1.name, "Only Name Changed")
self.assertEqual(self.channel1.channel_number, original_channel_number)
self.assertEqual(self.channel1.tvg_id, original_tvg_id)
def test_bulk_swap_clear_and_assign_same_number(self):
# User clears channel A's override (which currently pins #10) and
# in the same bulk request sets channel B's override.channel_number
# to #10. Both halves of the swap must succeed; the resulting
# state has A unpinned and B pinned at #10.
auto_a = Channel.objects.create(
channel_number=1.0,
name="Auto A",
tvg_id="auto_a",
channel_group=self.group1,
auto_created=True,
)
ChannelOverride.objects.create(channel=auto_a, channel_number=10.0)
auto_b = Channel.objects.create(
channel_number=2.0,
name="Auto B",
tvg_id="auto_b",
channel_group=self.group1,
auto_created=True,
)
data = [
{"id": auto_a.id, "override": None},
{"id": auto_b.id, "override": {"channel_number": 10.0}},
]
response = self.client.patch(self.bulk_edit_url, data, format="json")
self.assertEqual(
response.status_code,
status.HTTP_200_OK,
f"Expected 200; got {response.status_code} body={response.data}",
)
self.assertFalse(
ChannelOverride.objects.filter(channel=auto_a).exists()
)
b_override = ChannelOverride.objects.get(channel=auto_b)
self.assertEqual(b_override.channel_number, 10.0)
class ChannelSummaryEffectiveValuesTests(TestCase):
"""
The /api/channels/channels/summary/ endpoint feeds the TV Guide.
Like every downstream output surface, it must reflect the user's
overrides (name, channel_number, logo_id, epg_data_id,
channel_group_id) instead of the raw provider values, otherwise
the in-app guide would silently disagree with HDHR / M3U / EPG /
XC clients on the same channel set.
"""
def setUp(self):
from django.contrib.auth import get_user_model
from rest_framework.test import APIClient
from apps.channels.models import ChannelOverride
User = get_user_model()
self.user = User.objects.create_user(
username="summary_admin", password="x"
)
self.user.user_level = 10
self.user.save()
self.client = APIClient()
self.client.force_authenticate(user=self.user)
self.group = ChannelGroup.objects.create(name="Summary Group")
self.other_group = ChannelGroup.objects.create(name="Other")
self.channel = Channel.objects.create(
channel_number=10.0,
name="Provider Name",
channel_group=self.group,
auto_created=True,
)
ChannelOverride.objects.create(
channel=self.channel,
name="Override Name",
channel_number=99.0,
channel_group=self.other_group,
)
def test_summary_returns_effective_values(self):
response = self.client.get("/api/channels/channels/summary/")
self.assertEqual(response.status_code, 200)
row = next(r for r in response.data if r["id"] == self.channel.id)
self.assertEqual(row["name"], "Override Name")
self.assertEqual(row["channel_number"], 99.0)
self.assertEqual(row["channel_group_id"], self.other_group.id)
class ChannelManagerEffectiveValuesTests(TestCase):
"""
The chainable ``Channel.objects.with_effective_values()`` shortcut
must return rows with the same ``effective_*`` annotations the
module-level helper produces, since both forms are documented
entry points and a divergence would silently change output for
one set of callers.
"""
def test_manager_shortcut_matches_module_helper(self):
from apps.channels.managers import with_effective_values
group = ChannelGroup.objects.create(name="Manager Test")
channel = Channel.objects.create(
channel_number=42.0,
name="Original Name",
channel_group=group,
auto_created=True,
)
ChannelOverride.objects.create(
channel=channel,
name="Renamed",
channel_number=99.0,
)
helper_row = with_effective_values(
Channel.objects.filter(id=channel.id)
).get()
shortcut_row = (
Channel.objects.with_effective_values()
.filter(id=channel.id)
.get()
)
self.assertEqual(helper_row.effective_name, "Renamed")
self.assertEqual(shortcut_row.effective_name, "Renamed")
self.assertEqual(helper_row.effective_channel_number, 99.0)
self.assertEqual(shortcut_row.effective_channel_number, 99.0)
self.assertEqual(
helper_row.effective_channel_group_id,
shortcut_row.effective_channel_group_id,
)

12
apps/channels/utils.py
View file

@ -4,6 +4,18 @@ lock = threading.Lock()
# Dictionary to track usage: {account_id: current_usage}
active_streams_map = {}
def format_channel_number(value, empty=""):
"""Display formatting for an effective channel_number. Returns int for
whole-valued floats (so ``123.0`` renders as ``123``), the float as-is
for fractional values, or ``empty`` when the value is ``None``.
"""
if value is None:
return empty
if value == int(value):
return int(value)
return value
def increment_stream_count(account):
with lock:
current_usage = active_streams_map.get(account.id, 0)

32
apps/hdhr/api_views.py
View file

@ -125,30 +125,38 @@ class LineupAPIView(APIView):
if blocked is not None:
return blocked
from apps.channels.managers import with_effective_values
from apps.channels.utils import format_channel_number
if profile is not None:
channel_profile = ChannelProfile.objects.get(name=profile)
channels = Channel.objects.filter(
base_qs = Channel.objects.filter(
channelprofilemembership__channel_profile=channel_profile,
channelprofilemembership__enabled=True,
).order_by("channel_number")
)
else:
channels = Channel.objects.all().order_by("channel_number")
base_qs = Channel.objects.all()
channels = (
with_effective_values(base_qs)
.exclude(hidden_from_output=True)
.order_by("effective_channel_number")
)
lineup = []
for ch in channels:
# Format channel number as integer if it has no decimal component
if ch.channel_number is not None:
if ch.channel_number == int(ch.channel_number):
formatted_channel_number = str(int(ch.channel_number))
else:
formatted_channel_number = str(ch.channel_number)
else:
formatted_channel_number = ""
# HDHR clients reject lineup entries with empty/non-numeric
# GuideNumber and may drop the whole lineup. With nullable
# channel_number, skip rows that have no usable number.
formatted = format_channel_number(ch.effective_channel_number, empty=None)
if formatted is None:
continue
formatted_channel_number = str(formatted)
lineup.append(
{
"GuideNumber": formatted_channel_number,
"GuideName": ch.name,
"GuideName": ch.effective_name,
"URL": request.build_absolute_uri(f"/proxy/ts/stream/{ch.uuid}"),
"Guide_ID": formatted_channel_number,
"Station": formatted_channel_number,

30
apps/hdhr/views.py
View file

@ -84,15 +84,27 @@ class LineupAPIView(APIView):
description="Retrieve the available channel lineup",
)
def get(self, request):
channels = Channel.objects.all().order_by("channel_number")
lineup = [
{
"GuideNumber": str(ch.channel_number),
"GuideName": ch.name,
"URL": request.build_absolute_uri(f"/proxy/ts/stream/{ch.uuid}"),
}
for ch in channels
]
from apps.channels.managers import with_effective_values
from apps.channels.utils import format_channel_number
channels = (
with_effective_values(Channel.objects.all())
.exclude(hidden_from_output=True)
.order_by("effective_channel_number")
)
lineup = []
for ch in channels:
formatted = format_channel_number(ch.effective_channel_number, empty=None)
if formatted is None:
continue
formatted_channel_number = str(formatted)
lineup.append(
{
"GuideNumber": formatted_channel_number,
"GuideName": ch.effective_name,
"URL": request.build_absolute_uri(f"/proxy/ts/stream/{ch.uuid}"),
}
)
return JsonResponse(lineup, safe=False)

261
apps/m3u/api_views.py
View file

@ -17,6 +17,9 @@ from rest_framework.decorators import action
from django.conf import settings
from .tasks import refresh_m3u_groups
import json
import logging
logger = logging.getLogger(__name__)
from .models import M3UAccount, M3UFilter, ServerGroup, M3UAccountProfile
from core.models import UserAgent
@ -41,7 +44,7 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
queryset = M3UAccount.objects.select_related(
"refresh_task__crontab", "refresh_task__interval"
).prefetch_related("channel_group", "profiles")
).prefetch_related("channel_group", "profiles", "filters")
serializer_class = M3UAccountSerializer
def get_permissions(self):
@ -50,6 +53,41 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
except KeyError:
return [Authenticated()]
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
# Pre-aggregate stream counts for all accounts in one query so the
# nested ChannelGroupM3UAccountSerializer never issues a COUNT per
# group row. The serializer checks for this key and skips its own
# per-instance query when it is present.
from apps.channels.models import Stream
from django.db.models import Count
account_ids = list(queryset.values_list("id", flat=True))
counts_qs = (
Stream.objects.filter(m3u_account_id__in=account_ids)
.values("m3u_account_id", "channel_group_id")
.annotate(c=Count("id"))
)
stream_counts = {
(row["m3u_account_id"], row["channel_group_id"]): row["c"]
for row in counts_qs
}
page = self.paginate_queryset(queryset)
if page is not None:
serializer = self.get_serializer(
page, many=True,
context={**self.get_serializer_context(), "stream_counts": stream_counts},
)
return self.get_paginated_response(serializer.data)
serializer = self.get_serializer(
queryset, many=True,
context={**self.get_serializer_context(), "stream_counts": stream_counts},
)
return Response(serializer.data)
def create(self, request, *args, **kwargs):
# Handle file upload first, if any
file_path = None
@ -225,6 +263,198 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
# Continue with regular partial update
return super().partial_update(request, *args, **kwargs)
def destroy(self, request, *args, **kwargs):
"""
Delete an M3U account and all auto-created channels attributed
to it. Auto-created channels with no surviving provider have no
useful state (they cannot sync, their streams are about to
cascade away), so the delete is unconditional: the only
question for the user is whether to confirm. Manual channels
are untouched, even if they include streams from this account;
those streams cascade away independently and the channels
survive with their other streams. The legacy
``?cleanup_channels`` query parameter is accepted for backward
compatibility but ignored.
"""
instance = self.get_object()
from apps.channels.models import Channel
from apps.proxy.ts_proxy.services.channel_service import (
ChannelService,
)
# Snapshot channels so proxy sessions can be stopped outside
# the DB transaction. The pre_delete signal would otherwise
# fire ChannelService.stop_channel (Redis pub / hgetall /
# setex) per channel inside the atomic, holding the DB
# connection across thousands of blocking RPCs and gumming up
# the connection pool.
channels_to_delete = list(
Channel.objects.filter(
auto_created=True,
auto_created_by=instance,
).values_list("id", "uuid")
)
for _, channel_uuid in channels_to_delete:
if not channel_uuid:
continue
try:
ChannelService.stop_channel(str(channel_uuid))
except Exception as e:
logger.warning(
"Failed to stop proxy session for channel %s "
"during account cleanup: %s",
channel_uuid,
e,
)
channel_ids = [cid for cid, _ in channels_to_delete]
# Channel + account writes share an atomic so an account
# delete failure rolls back the channel deletes too. The
# pre_delete signal will fire again here but its proxy stop
# is fast on already-stopped channels (a single Redis check
# returns "not found" immediately).
with transaction.atomic():
if channel_ids:
_, per_model = Channel.objects.filter(
id__in=channel_ids
).delete()
deleted_channels = per_model.get(
"dispatcharr_channels.Channel", 0
)
else:
deleted_channels = 0
response = super().destroy(request, *args, **kwargs)
# Surface the channel count alongside the standard 204; the
# confirmation toast renders the number to acknowledge what
# the cascade actually removed.
if response.status_code == status.HTTP_204_NO_CONTENT:
return Response(
{"deleted_channels": deleted_channels},
status=status.HTTP_200_OK,
)
return response
@extend_schema(
responses={
200: {
"type": "object",
"properties": {
"count": {"type": "integer"},
"sample_names": {"type": "array", "items": {"type": "string"}},
},
},
},
)
@action(detail=True, methods=["get"], url_path="auto-created-channels-count")
def auto_created_channels_count(self, request, pk=None):
"""
Preview how many auto-created channels would be removed if the account
were deleted with cleanup_channels=true. The frontend calls this when
the user clicks Delete, to render a truthful confirmation dialog
("Also delete N channels auto-created by this provider?").
"""
account = self.get_object()
from apps.channels.models import Channel
qs = Channel.objects.filter(
auto_created=True, auto_created_by=account
)
count = qs.count()
sample_names = list(qs.values_list("name", flat=True)[:5])
return Response({"count": count, "sample_names": sample_names})
@extend_schema(
parameters=[
OpenApiParameter(
name="channel_group_id",
type=OpenApiTypes.INT,
location=OpenApiParameter.QUERY,
required=True,
description=(
"ID of the ChannelGroup whose auto-created channels "
"should be repacked."
),
),
],
responses={
200: {
"type": "object",
"properties": {
"assigned": {"type": "integer"},
"released": {"type": "integer"},
"failed": {"type": "integer"},
},
},
},
)
@action(detail=True, methods=["post"], url_path="repack-group")
def repack_group(self, request, pk=None):
"""
Manually re-pack visible channels in one of this account's
groups into the group's [start, end] range. Override-pinned
numbers are treated as reservations and skipped. Hidden channels
without overrides have their channel_number set to NULL.
Useful when the user has just finished customizing channels
(setting overrides as pins, hiding unwanted streams) and wants
the result reflected immediately rather than on the next M3U
refresh. Also acts as a one-shot cleanup for groups that aren't
running in compact mode but have accumulated gaps.
"""
account = self.get_object()
group_id_raw = request.query_params.get("channel_group_id")
if not group_id_raw:
return Response(
{"detail": "channel_group_id is required"},
status=status.HTTP_400_BAD_REQUEST,
)
try:
group_id = int(group_id_raw)
except (TypeError, ValueError):
return Response(
{"detail": "channel_group_id must be an integer"},
status=status.HTTP_400_BAD_REQUEST,
)
from apps.channels.models import ChannelGroupM3UAccount
from apps.channels.compact_numbering import repack_group as _repack
from core.utils import acquire_task_lock, release_task_lock
# Share the lock that wraps the entire refresh-plus-sync pipeline
# (`refresh_single_m3u_account`). The narrower
# `refresh_m3u_account_groups` lock is released before
# `sync_auto_channels` runs, so it would not protect this writer
# from racing against the channel_number writes inside sync.
if not acquire_task_lock("refresh_single_m3u_account", account.id):
return Response(
{"detail": "An M3U refresh is in progress for this account."},
status=status.HTTP_409_CONFLICT,
)
try:
# Re-fetch under the lock so a sync that just released its lock
# cannot leave the cached group_relation reflecting pre-sync
# custom_properties (auto_sync_channel_start/end, etc.).
try:
group_relation = ChannelGroupM3UAccount.objects.get(
m3u_account=account, channel_group_id=group_id
)
except ChannelGroupM3UAccount.DoesNotExist:
return Response(
{"detail": "Group is not associated with this account"},
status=status.HTTP_404_NOT_FOUND,
)
result = _repack(group_relation)
finally:
try:
release_task_lock("refresh_single_m3u_account", account.id)
except Exception as e:
logger.warning(
f"Failed to release repack lock for account "
f"{account.id}: {e}"
)
return Response(result)
@action(detail=True, methods=["post"], url_path="refresh-vod")
def refresh_vod(self, request, pk=None):
"""Trigger VOD content refresh for XtreamCodes accounts"""
@ -270,6 +500,33 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
category_settings = request.data.get("category_settings", [])
try:
for setting in group_settings:
start = setting.get("auto_sync_channel_start")
end = setting.get("auto_sync_channel_end")
if (start is not None and start < 1) or (
end is not None and end < 1
):
return Response(
{
"error": (
f"Channel group {setting.get('channel_group')}: "
f"channel range must be >= 1."
)
},
status=status.HTTP_400_BAD_REQUEST,
)
if start is not None and end is not None and end < start:
return Response(
{
"error": (
f"Channel group {setting.get('channel_group')}: "
f"auto_sync_channel_end must be >= "
f"auto_sync_channel_start."
)
},
status=status.HTTP_400_BAD_REQUEST,
)
with transaction.atomic():
group_objects = [
ChannelGroupM3UAccount(
@ -278,6 +535,7 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
enabled=setting.get("enabled", True),
auto_channel_sync=setting.get("auto_channel_sync", False),
auto_sync_channel_start=setting.get("auto_sync_channel_start"),
auto_sync_channel_end=setting.get("auto_sync_channel_end"),
custom_properties=setting.get("custom_properties", {}),
)
for setting in group_settings
@ -293,6 +551,7 @@ class M3UAccountViewSet(viewsets.ModelViewSet):
"enabled",
"auto_channel_sync",
"auto_sync_channel_start",
"auto_sync_channel_end",
"custom_properties",
],
)

1
apps/m3u/models.py
View file

@ -100,7 +100,6 @@ class M3UAccount(models.Model):
default=0,
help_text="Priority for VOD provider selection (higher numbers = higher priority). Used when multiple providers offer the same content.",
)
def __str__(self):
return self.name

35
apps/m3u/serializers.py
View file

@ -193,6 +193,24 @@ class M3UAccountSerializer(serializers.ModelSerializer):
}
def to_representation(self, instance):
# When the list() view pre-aggregates stream counts for all accounts
# in a single query, it seeds "stream_counts" into the context before
# serialization. Avoid issuing a redundant per-instance COUNT in that
# case. The per-instance fallback handles direct serialization (e.g.
# retrieve, create) where only one account is in scope.
if "stream_counts" not in self.context:
from django.db.models import Count
from apps.channels.models import Stream
counts_qs = (
Stream.objects.filter(m3u_account_id=instance.id)
.values("channel_group_id")
.annotate(c=Count("id"))
)
self.context["stream_counts"] = {
(instance.id, row["channel_group_id"]): row["c"] for row in counts_qs
}
data = super().to_representation(instance)
# Parse custom_properties to get VOD preference and auto_enable_new_groups settings
@ -247,10 +265,17 @@ class M3UAccountSerializer(serializers.ModelSerializer):
auto_enable_new_groups_vod = validated_data.pop("auto_enable_new_groups_vod", None)
auto_enable_new_groups_series = validated_data.pop("auto_enable_new_groups_series", None)
# Get existing custom_properties
custom_props = instance.custom_properties or {}
# Merge client-supplied custom_properties over the existing blob
# so unrelated keys persist. The dedicated preference fields below
# overwrite their corresponding keys; clients should set those via
# the typed top-level fields rather than the custom_properties
# payload.
incoming_custom = validated_data.get("custom_properties") or {}
custom_props = {
**(instance.custom_properties or {}),
**incoming_custom,
}
# Update preferences
if enable_vod is not None:
custom_props["enable_vod"] = enable_vod
if auto_enable_new_groups_live is not None:
@ -345,7 +370,9 @@ class M3UAccountSerializer(serializers.ModelSerializer):
return instance
def get_filters(self, obj):
filters = obj.filters.order_by("order")
# Sort over the prefetch cache; .order_by() would fire one SELECT
# per account (viewset prefetches "filters").
filters = sorted(obj.filters.all(), key=lambda f: f.order)
return M3UFilterSerializer(filters, many=True).data
def get_earliest_expiration(self, obj):

1132
apps/m3u/tasks.py
View file

File diff suppressed because it is too large Load diff

143
apps/m3u/tests/test_account_destroy.py Normal file
View file

@ -0,0 +1,143 @@
"""
DELETE /api/m3u/accounts/{id}/ behavior.
The endpoint always cascade-deletes auto-created channels owned by the
account. Manual channels survive even if some of their streams were
owned by the deleted account; only the streams from that account go
away with the account, and the channel keeps any streams sourced from
other accounts. The legacy ``cleanup_channels`` query parameter is
accepted but ignored.
"""
from unittest.mock import patch
from django.contrib.auth import get_user_model
from django.test import TestCase
from django.utils import timezone
from rest_framework import status
from rest_framework.test import APIClient
from apps.channels.models import (
Channel,
ChannelGroup,
ChannelStream,
Stream,
)
from apps.m3u.models import M3UAccount
User = get_user_model()
class M3UAccountDestroyTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(
username="destroyer", password="testpass123"
)
self.user.user_level = 10
self.user.save()
self.client = APIClient()
self.client.force_authenticate(user=self.user)
# Patching the proxy stop here keeps the Redis-backed call out
# of the test path. The endpoint loops over every auto-created
# channel before the DB transaction; the real implementation is
# exercised by integration tests, not unit tests.
self._stop_patch = patch(
"apps.proxy.ts_proxy.services.channel_service."
"ChannelService.stop_channel"
)
self._stop_patch.start()
self.addCleanup(self._stop_patch.stop)
self.account = M3UAccount.objects.create(
name="ProviderA",
server_url="http://example.com/a.m3u",
)
self.other_account = M3UAccount.objects.create(
name="ProviderB",
server_url="http://example.com/b.m3u",
)
self.group = ChannelGroup.objects.create(name="News")
def _make_stream(self, account, name="ESPN"):
return Stream.objects.create(
name=name,
url=f"http://example.com/{name.lower()}.m3u8",
m3u_account=account,
channel_group=self.group,
tvg_id=name.lower(),
last_seen=timezone.now(),
)
def test_cascade_deletes_auto_created_channels(self):
# Two auto-created channels under the account being deleted.
for n in (101.0, 102.0):
ch = Channel.objects.create(
channel_number=n,
name=f"Auto {n}",
channel_group=self.group,
auto_created=True,
auto_created_by=self.account,
)
ChannelStream.objects.create(
channel=ch,
stream=self._make_stream(self.account, name=f"Auto{int(n)}"),
order=0,
)
response = self.client.delete(
f"/api/m3u/accounts/{self.account.id}/"
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["deleted_channels"], 2)
self.assertFalse(M3UAccount.objects.filter(id=self.account.id).exists())
self.assertFalse(
Channel.objects.filter(auto_created_by_id=self.account.id).exists()
)
def test_manual_channel_survives_with_other_provider_streams(self):
# Manual channel with one stream from each account. The provider
# account's stream goes away with the account; the other
# account's stream stays, and the channel survives.
manual = Channel.objects.create(
channel_number=200.0,
name="Manual",
channel_group=self.group,
auto_created=False,
)
provider_stream = self._make_stream(self.account, name="ProviderA")
other_stream = self._make_stream(self.other_account, name="ProviderB")
ChannelStream.objects.create(channel=manual, stream=provider_stream, order=0)
ChannelStream.objects.create(channel=manual, stream=other_stream, order=1)
response = self.client.delete(
f"/api/m3u/accounts/{self.account.id}/"
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["deleted_channels"], 0)
manual.refresh_from_db()
remaining = list(manual.channelstream_set.values_list("stream__id", flat=True))
self.assertEqual(remaining, [other_stream.id])
def test_legacy_cleanup_channels_param_is_ignored(self):
# Behavior must be identical with or without the deprecated
# ``cleanup_channels`` query parameter.
ch = Channel.objects.create(
channel_number=300.0,
name="Auto",
channel_group=self.group,
auto_created=True,
auto_created_by=self.account,
)
response = self.client.delete(
f"/api/m3u/accounts/{self.account.id}/?cleanup_channels=false"
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["deleted_channels"], 1)
self.assertFalse(Channel.objects.filter(id=ch.id).exists())
def test_no_op_when_account_has_no_auto_created_channels(self):
response = self.client.delete(
f"/api/m3u/accounts/{self.account.id}/"
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["deleted_channels"], 0)

240
apps/m3u/tests/test_sync_compound.py Normal file
View file

@ -0,0 +1,240 @@
"""
Compound-fixture sync tests.
Where individual sync tests cover one variation at a time (multi-stream,
hidden, override, manual), this module seeds all of them in the same
fixture and asserts the constraints still hold when sync sees the full
mix on a single account. The point is to catch interactions that pass
each isolated test but break when the conditions overlap.
"""
from datetime import timedelta
from django.test import TestCase
from django.utils import timezone
from apps.channels.models import (
Channel,
ChannelGroup,
ChannelGroupM3UAccount,
ChannelOverride,
ChannelStream,
Stream,
)
from apps.m3u.models import M3UAccount
from apps.m3u.tasks import sync_auto_channels
def _scan_start_time():
return (timezone.now() - timedelta(minutes=1)).isoformat()
class CompoundFixtureSyncTests(TestCase):
"""
Single fixture covers: multi-stream auto channel, hidden auto channel,
auto channel with channel_number override, manual channel that shares
the group with auto-created rows. After sync runs, every channel must
end up in the state its individual test would have asserted.
"""
def setUp(self):
self.account = M3UAccount.objects.create(
name="Compound Provider",
server_url="http://example.com/compound.m3u",
)
self.group = ChannelGroup.objects.create(name="Compound Group")
self.relation = ChannelGroupM3UAccount.objects.create(
m3u_account=self.account,
channel_group=self.group,
enabled=True,
auto_channel_sync=True,
auto_sync_channel_start=100,
auto_sync_channel_end=199,
)
now = timezone.now()
# ── Multi-stream auto channel: two streams, one fresh, one stale.
# The channel must survive the stale stream's disappearance because
# the fresh one keeps the channel alive.
self.multi_stream_a = Stream.objects.create(
name="MultiCh HD",
url="http://example.com/multi-a.m3u8",
m3u_account=self.account,
channel_group=self.group,
tvg_id="multi",
last_seen=now,
)
self.multi_stream_b = Stream.objects.create(
name="MultiCh HD",
url="http://example.com/multi-b.m3u8",
m3u_account=self.account,
channel_group=self.group,
tvg_id="multi",
last_seen=now - timedelta(days=2),
)
self.multi_channel = Channel.objects.create(
name="MultiCh HD",
channel_number=100,
channel_group=self.group,
auto_created=True,
auto_created_by=self.account,
)
ChannelStream.objects.create(
channel=self.multi_channel, stream=self.multi_stream_a, order=0
)
ChannelStream.objects.create(
channel=self.multi_channel, stream=self.multi_stream_b, order=1
)
# ── Hidden auto channel: visible from the table's perspective only
# to admins with the Hidden filter on. Sync must not reuse its
# channel_number for a different channel just because it's hidden.
self.hidden_stream = Stream.objects.create(
name="HiddenCh",
url="http://example.com/hidden.m3u8",
m3u_account=self.account,
channel_group=self.group,
tvg_id="hidden",
last_seen=now,
)
self.hidden_channel = Channel.objects.create(
name="HiddenCh",
channel_number=101,
channel_group=self.group,
auto_created=True,
auto_created_by=self.account,
hidden_from_output=True,
)
ChannelStream.objects.create(
channel=self.hidden_channel, stream=self.hidden_stream, order=0
)
# ── Overridden auto channel: user pinned channel_number to 150 via
# an override row. Sync must not clobber the override; the
# effective channel_number stays 150 even though the raw column
# may evolve.
self.overridden_stream = Stream.objects.create(
name="OverriddenCh",
url="http://example.com/overridden.m3u8",
m3u_account=self.account,
channel_group=self.group,
tvg_id="overridden",
last_seen=now,
)
self.overridden_channel = Channel.objects.create(
name="OverriddenCh",
channel_number=102,
channel_group=self.group,
auto_created=True,
auto_created_by=self.account,
)
ChannelStream.objects.create(
channel=self.overridden_channel,
stream=self.overridden_stream,
order=0,
)
ChannelOverride.objects.create(
channel=self.overridden_channel,
channel_number=150,
name="My Pinned Name",
)
# ── Manual channel sharing the group: auto_created=False, user
# picked channel_number 175 themselves. Sync must not touch this
# row at all.
self.manual_channel = Channel.objects.create(
name="ManualCh",
channel_number=175,
channel_group=self.group,
auto_created=False,
)
# New stream that has no existing channel; sync should create a
# fresh channel for it within the configured range and skip
# 100-102 (in use), 150 (overridden), 175 (manual).
self.new_stream = Stream.objects.create(
name="FreshCh",
url="http://example.com/fresh.m3u8",
m3u_account=self.account,
channel_group=self.group,
tvg_id="fresh",
last_seen=now,
)
def test_compound_fixture_each_invariant_holds_after_sync(self):
sync_auto_channels(self.account.id, scan_start_time=_scan_start_time())
# Multi-stream channel survives.
self.assertTrue(
Channel.objects.filter(id=self.multi_channel.id).exists(),
"Multi-stream channel was deleted even though one stream is alive",
)
# Hidden channel survives and remains hidden.
self.hidden_channel.refresh_from_db()
self.assertTrue(self.hidden_channel.hidden_from_output)
# Overridden channel: the override row is intact, not cleared by
# sync. The pinned channel_number persists.
override = ChannelOverride.objects.get(channel=self.overridden_channel)
self.assertEqual(override.channel_number, 150)
self.assertEqual(override.name, "My Pinned Name")
# Manual channel is untouched.
self.manual_channel.refresh_from_db()
self.assertFalse(self.manual_channel.auto_created)
self.assertEqual(self.manual_channel.channel_number, 175)
# The fresh stream becomes a new auto channel; its channel_number
# falls inside the configured range and must not collide with any
# existing number (100-102 used by the auto channels, 150 pinned
# by the override, 175 the manual channel).
fresh_stream_id = self.new_stream.id
new_channel_qs = Channel.objects.filter(
channelstream__stream_id=fresh_stream_id,
auto_created=True,
auto_created_by=self.account,
).distinct()
self.assertEqual(
new_channel_qs.count(),
1,
"Sync did not create exactly one channel for the new stream",
)
new_number = new_channel_qs.first().channel_number
self.assertIsNotNone(new_number)
self.assertGreaterEqual(new_number, 100)
self.assertLessEqual(new_number, 199)
self.assertNotIn(new_number, {100, 101, 102, 150, 175})
def test_hidden_channel_number_not_reassigned_to_new_stream(self):
# Targeted assertion isolated from the broader fixture invariants
# so the failure mode is unambiguous when this single property
# regresses.
sync_auto_channels(self.account.id, scan_start_time=_scan_start_time())
new_channel = (
Channel.objects.filter(
channelstream__stream_id=self.new_stream.id,
auto_created=True,
auto_created_by=self.account,
)
.distinct()
.first()
)
self.assertIsNotNone(new_channel)
self.assertNotEqual(
new_channel.channel_number,
self.hidden_channel.channel_number,
"Hidden channel's number was reassigned; hidden channels must "
"still occupy their slot in the used_numbers set",
)
def test_override_channel_number_preserved_through_sync(self):
# If a future regression caused sync to write to ChannelOverride
# (it must not), this test catches it because the override would
# change after refresh.
original_pin = 150
sync_auto_channels(self.account.id, scan_start_time=_scan_start_time())
override = ChannelOverride.objects.get(channel=self.overridden_channel)
self.assertEqual(override.channel_number, original_pin)

2051
apps/m3u/tests/test_sync_correctness.py Normal file
View file

File diff suppressed because it is too large Load diff

343
apps/output/views.py
View file

@ -2,6 +2,7 @@ from django.http import HttpResponse, JsonResponse, Http404, HttpResponseForbidd
from rest_framework.response import Response
from django.urls import reverse
from apps.channels.models import Channel, ChannelProfile, ChannelGroup, Stream
from apps.channels.utils import format_channel_number
from django.db.models import Prefetch
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_http_methods
@ -138,7 +139,7 @@ def generate_m3u(request, profile_name=None, user=None):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channels = Channel.objects.filter(**filters).select_related('channel_group', 'logo').order_by("channel_number")
base_qs = Channel.objects.filter(**filters).select_related('channel_group', 'logo')
else:
# User has specific limited profiles assigned
filters = {
@ -149,11 +150,9 @@ def generate_m3u(request, profile_name=None, user=None):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channels = Channel.objects.filter(**filters).select_related('channel_group', 'logo').distinct().order_by("channel_number")
base_qs = Channel.objects.filter(**filters).select_related('channel_group', 'logo').distinct()
else:
channels = Channel.objects.filter(user_level__lte=user.user_level).select_related('channel_group', 'logo').order_by(
"channel_number"
)
base_qs = Channel.objects.filter(user_level__lte=user.user_level).select_related('channel_group', 'logo')
else:
if profile_name is not None:
@ -162,12 +161,22 @@ def generate_m3u(request, profile_name=None, user=None):
except ChannelProfile.DoesNotExist:
logger.warning("Requested channel profile (%s) during m3u generation does not exist", profile_name)
raise Http404(f"Channel profile '{profile_name}' not found")
channels = Channel.objects.filter(
base_qs = Channel.objects.filter(
channelprofilemembership__channel_profile=channel_profile,
channelprofilemembership__enabled=True
).select_related('channel_group', 'logo').order_by('channel_number')
).select_related('channel_group', 'logo')
else:
channels = Channel.objects.select_related('channel_group', 'logo').order_by("channel_number")
base_qs = Channel.objects.select_related('channel_group', 'logo')
# Resolve effective (override | provider) values at SQL level so ordering,
# naming, and logo resolution honor user overrides. `exclude(hidden_from_output=True)`
# is the consumer-facing hide guarantee.
from apps.channels.managers import with_effective_values
channels = (
with_effective_values(base_qs, select_related_fks=True)
.exclude(hidden_from_output=True)
.order_by("effective_channel_number")
)
# Check if the request wants to use direct logo URLs instead of cache
use_cached_logos = request.GET.get('cachedlogos', 'true').lower() != 'false'
@ -212,53 +221,55 @@ def generate_m3u(request, profile_name=None, user=None):
m3u_content = f'#EXTM3U x-tvg-url="{epg_url}" url-tvg="{epg_url}"\n'
# Start building M3U content
channel_count = 0
for channel in channels:
group_title = channel.channel_group.name if channel.channel_group else "Default"
channel_count += 1
effective_group = channel.effective_channel_group_obj
effective_logo = channel.effective_logo_obj
effective_name = channel.effective_name
effective_tvg_id_val = channel.effective_tvg_id
effective_tvc_guide = channel.effective_tvc_guide_stationid
effective_number = channel.effective_channel_number
# Format channel number as integer if it has no decimal component
if channel.channel_number is not None:
if channel.channel_number == int(channel.channel_number):
formatted_channel_number = int(channel.channel_number)
else:
formatted_channel_number = channel.channel_number
else:
formatted_channel_number = ""
group_title = effective_group.name if effective_group else "Default"
formatted_channel_number = format_channel_number(effective_number)
# Determine the tvg-id based on the selected source
if tvg_id_source == 'tvg_id' and channel.tvg_id:
tvg_id = channel.tvg_id
elif tvg_id_source == 'gracenote' and channel.tvc_guide_stationid:
tvg_id = channel.tvc_guide_stationid
if tvg_id_source == 'tvg_id' and effective_tvg_id_val:
tvg_id = effective_tvg_id_val
elif tvg_id_source == 'gracenote' and effective_tvc_guide:
tvg_id = effective_tvc_guide
else:
# Default to channel number (original behavior)
tvg_id = str(formatted_channel_number) if formatted_channel_number != "" else str(channel.id)
tvg_name = channel.name
tvg_name = effective_name
tvg_logo = ""
if channel.logo:
if effective_logo:
if use_cached_logos:
# Use cached logo as before
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[channel.logo.id]))
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[effective_logo.id]))
else:
# Try to find direct logo URL from channel's streams
direct_logo = channel.logo.url if channel.logo.url.startswith(('http://', 'https://')) else None
direct_logo = effective_logo.url if effective_logo.url.startswith(('http://', 'https://')) else None
# If direct logo found, use it; otherwise fall back to cached version
if direct_logo:
tvg_logo = direct_logo
else:
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[channel.logo.id]))
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[effective_logo.id]))
# create possible gracenote id insertion
tvc_guide_stationid = ""
if channel.tvc_guide_stationid:
if effective_tvc_guide:
tvc_guide_stationid = (
f'tvc-guide-stationid="{channel.tvc_guide_stationid}" '
f'tvc-guide-stationid="{effective_tvc_guide}" '
)
extinf_line = (
f'#EXTINF:-1 tvg-id="{tvg_id}" tvg-name="{tvg_name}" tvg-logo="{tvg_logo}" '
f'tvg-chno="{formatted_channel_number}" {tvc_guide_stationid}group-title="{group_title}",{channel.name}\n'
f'tvg-chno="{formatted_channel_number}" {tvc_guide_stationid}group-title="{group_title}",{effective_name}\n'
)
# Determine the stream URL based on request type
@ -293,7 +304,7 @@ def generate_m3u(request, profile_name=None, user=None):
event_type='m3u_download',
profile=profile_name or 'all',
user=user.username if user else 'anonymous',
channels=channels.count(),
channels=channel_count,
client_ip=client_ip,
user_agent=user_agent,
)
@ -1328,7 +1339,7 @@ def generate_epg(request, profile_name=None, user=None):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channels = Channel.objects.filter(**filters).select_related('logo', 'epg_data__epg_source').order_by("channel_number")
base_qs = Channel.objects.filter(**filters).select_related('logo', 'epg_data__epg_source')
else:
# User has specific limited profiles assigned
filters = {
@ -1339,11 +1350,9 @@ def generate_epg(request, profile_name=None, user=None):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channels = Channel.objects.filter(**filters).select_related('logo', 'epg_data__epg_source').distinct().order_by("channel_number")
base_qs = Channel.objects.filter(**filters).select_related('logo', 'epg_data__epg_source').distinct()
else:
channels = Channel.objects.filter(user_level__lte=user.user_level).select_related('logo', 'epg_data__epg_source').order_by(
"channel_number"
)
base_qs = Channel.objects.filter(user_level__lte=user.user_level).select_related('logo', 'epg_data__epg_source')
else:
if profile_name is not None:
try:
@ -1351,12 +1360,21 @@ def generate_epg(request, profile_name=None, user=None):
except ChannelProfile.DoesNotExist:
logger.warning("Requested channel profile (%s) during epg generation does not exist", profile_name)
raise Http404(f"Channel profile '{profile_name}' not found")
channels = Channel.objects.filter(
base_qs = Channel.objects.filter(
channelprofilemembership__channel_profile=channel_profile,
channelprofilemembership__enabled=True,
).select_related('logo', 'epg_data__epg_source').order_by("channel_number")
).select_related('logo', 'epg_data__epg_source')
else:
channels = Channel.objects.all().select_related('logo', 'epg_data__epg_source').order_by("channel_number")
base_qs = Channel.objects.all().select_related('logo', 'epg_data__epg_source')
# Resolve effective values at SQL level and exclude hidden channels
# so output ordering/display honors user overrides.
from apps.channels.managers import with_effective_values
channels = (
with_effective_values(base_qs, select_related_fks=True)
.exclude(hidden_from_output=True)
.order_by("effective_channel_number")
)
# For dummy EPG, use either the specified value or default to 3 days
@ -1376,15 +1394,17 @@ def generate_epg(request, profile_name=None, user=None):
# First pass: assign integers for channels that already have integer numbers
for channel in channels:
if channel.channel_number == int(channel.channel_number):
num = int(channel.channel_number)
effective_num = channel.effective_channel_number
if effective_num is not None and effective_num == int(effective_num):
num = int(effective_num)
channel_num_map[channel.id] = num
used_numbers.add(num)
# Second pass: assign integers for channels with float numbers
for channel in channels:
if channel.channel_number != int(channel.channel_number):
candidate = int(channel.channel_number)
effective_num = channel.effective_channel_number
if effective_num is not None and effective_num != int(effective_num):
candidate = int(effective_num)
while candidate in used_numbers:
candidate += 1
channel_num_map[channel.id] = candidate
@ -1392,38 +1412,37 @@ def generate_epg(request, profile_name=None, user=None):
# Process channels for the <channel> section
for channel in channels:
effective_name = channel.effective_name
effective_epg_data = channel.effective_epg_data_obj
effective_logo = channel.effective_logo_obj
effective_number = channel.effective_channel_number
# user is set only for XC clients, which require integer channel numbers
if user is not None:
formatted_channel_number = channel_num_map[channel.id]
else:
if channel.channel_number is not None:
if channel.channel_number == int(channel.channel_number):
formatted_channel_number = int(channel.channel_number)
else:
formatted_channel_number = channel.channel_number
else:
formatted_channel_number = ""
formatted_channel_number = format_channel_number(effective_number)
# Determine the channel ID based on the selected source
if tvg_id_source == 'tvg_id' and channel.tvg_id:
channel_id = channel.tvg_id
elif tvg_id_source == 'gracenote' and channel.tvc_guide_stationid:
channel_id = channel.tvc_guide_stationid
if tvg_id_source == 'tvg_id' and channel.effective_tvg_id:
channel_id = channel.effective_tvg_id
elif tvg_id_source == 'gracenote' and channel.effective_tvc_guide_stationid:
channel_id = channel.effective_tvc_guide_stationid
else:
channel_id = str(formatted_channel_number) if formatted_channel_number != "" else str(channel.id)
tvg_logo = ""
# Check if this is a custom dummy EPG with channel logo URL template
if channel.epg_data and channel.epg_data.epg_source and channel.epg_data.epg_source.source_type == 'dummy':
epg_source = channel.epg_data.epg_source
if effective_epg_data and effective_epg_data.epg_source and effective_epg_data.epg_source.source_type == 'dummy':
epg_source = effective_epg_data.epg_source
if epg_source.custom_properties:
custom_props = epg_source.custom_properties
channel_logo_url_template = custom_props.get('channel_logo_url', '')
if channel_logo_url_template:
# Determine which name to use for pattern matching (same logic as program generation)
pattern_match_name = channel.name
pattern_match_name = effective_name
name_source = custom_props.get('name_source')
if name_source == 'stream':
@ -1465,20 +1484,20 @@ def generate_epg(request, profile_name=None, user=None):
tvg_logo = channel_logo_url_template
logger.debug(f"Built channel logo URL from template: {tvg_logo}")
except Exception as e:
logger.warning(f"Failed to build channel logo URL for {channel.name}: {e}")
logger.warning(f"Failed to build channel logo URL for {effective_name}: {e}")
# If no custom dummy logo, use regular logo logic
if not tvg_logo and channel.logo:
if not tvg_logo and effective_logo:
if use_cached_logos:
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[channel.logo.id]))
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[effective_logo.id]))
else:
# Use direct URL if available, otherwise fall back to cached version
direct_logo = channel.logo.url if channel.logo.url.startswith(('http://', 'https://')) else None
direct_logo = effective_logo.url if effective_logo.url.startswith(('http://', 'https://')) else None
if direct_logo:
tvg_logo = direct_logo
else:
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[channel.logo.id]))
display_name = channel.name
tvg_logo = build_absolute_uri_with_port(request, reverse('api:channels:logo-cache', args=[effective_logo.id]))
display_name = effective_name
xml_lines.append(f' <channel id="{html.escape(channel_id)}">')
xml_lines.append(f' <display-name>{html.escape(display_name)}</display-name>')
xml_lines.append(f' <icon src="{html.escape(tvg_logo)}" />')
@ -1495,30 +1514,29 @@ def generate_epg(request, profile_name=None, user=None):
dummy_epg_checked = {} # epg_data_id -> bool (has stored programs)
for channel in channels:
effective_name = channel.effective_name
effective_epg_data = channel.effective_epg_data_obj
effective_epg_data_id = channel.effective_epg_data_id
effective_number = channel.effective_channel_number
# Determine channel_id (same logic as channel section)
if tvg_id_source == 'tvg_id' and channel.tvg_id:
channel_id = channel.tvg_id
elif tvg_id_source == 'gracenote' and channel.tvc_guide_stationid:
channel_id = channel.tvc_guide_stationid
if tvg_id_source == 'tvg_id' and channel.effective_tvg_id:
channel_id = channel.effective_tvg_id
elif tvg_id_source == 'gracenote' and channel.effective_tvc_guide_stationid:
channel_id = channel.effective_tvc_guide_stationid
else:
if user is not None:
formatted_channel_number = channel_num_map[channel.id]
else:
if channel.channel_number is not None:
if channel.channel_number == int(channel.channel_number):
formatted_channel_number = int(channel.channel_number)
else:
formatted_channel_number = channel.channel_number
else:
formatted_channel_number = ""
formatted_channel_number = format_channel_number(effective_number)
channel_id = str(formatted_channel_number) if formatted_channel_number != "" else str(channel.id)
display_name = channel.epg_data.name if channel.epg_data else channel.name
pattern_match_name = channel.name
display_name = effective_epg_data.name if effective_epg_data else effective_name
pattern_match_name = effective_name
# Check if we should use stream name instead of channel name
if channel.epg_data and channel.epg_data.epg_source:
epg_source = channel.epg_data.epg_source
if effective_epg_data and effective_epg_data.epg_source:
epg_source = effective_epg_data.epg_source
if epg_source.custom_properties:
custom_props = epg_source.custom_properties
name_source = custom_props.get('name_source')
@ -1532,22 +1550,21 @@ def generate_epg(request, profile_name=None, user=None):
pattern_match_name = stream.name
logger.debug(f"Using stream name for parsing: {pattern_match_name} (stream index: {stream_index})")
else:
logger.warning(f"Stream index {stream_index} not found for channel {channel.name}, falling back to channel name")
logger.warning(f"Stream index {stream_index} not found for channel {effective_name}, falling back to channel name")
if not channel.epg_data:
if not effective_epg_data:
dummy_program_list.append((channel_id, pattern_match_name, None))
else:
if channel.epg_data.epg_source and channel.epg_data.epg_source.source_type == 'dummy':
epg_data_id = channel.epg_data_id
if epg_data_id not in dummy_epg_checked:
dummy_epg_checked[epg_data_id] = channel.epg_data.programs.exists()
if dummy_epg_checked[epg_data_id]:
real_epg_map.setdefault(epg_data_id, []).append(channel_id)
if effective_epg_data.epg_source and effective_epg_data.epg_source.source_type == 'dummy':
if effective_epg_data_id not in dummy_epg_checked:
dummy_epg_checked[effective_epg_data_id] = effective_epg_data.programs.exists()
if dummy_epg_checked[effective_epg_data_id]:
real_epg_map.setdefault(effective_epg_data_id, []).append(channel_id)
else:
dummy_program_list.append((channel_id, pattern_match_name, channel.epg_data.epg_source))
dummy_program_list.append((channel_id, pattern_match_name, effective_epg_data.epg_source))
continue
real_epg_map.setdefault(channel.epg_data_id, []).append(channel_id)
real_epg_map.setdefault(effective_epg_data_id, []).append(channel_id)
# Emit dummy programmes
for channel_id, pattern_match_name, epg_source in dummy_program_list:
@ -1880,11 +1897,13 @@ def generate_epg(request, profile_name=None, user=None):
client_id, client_ip, user_agent = get_client_identifier(request)
event_cache_key = f"epg_download:{user.username if user else 'anonymous'}:{profile_name or 'all'}:{client_id}"
if not cache.get(event_cache_key):
# `len()` reuses the queryset's iteration cache populated above;
# `count()` would issue a separate SELECT COUNT(*).
log_system_event(
event_type='epg_download',
profile=profile_name or 'all',
user=user.username if user else 'anonymous',
channels=channels.count(),
channels=len(channels),
client_ip=client_ip,
user_agent=user_agent,
)
@ -2103,8 +2122,18 @@ def xc_xmltv(request):
def xc_get_live_categories(user):
from django.db.models import Min
from django.db.models.functions import Coalesce
response = []
# Rank categories by the minimum EFFECTIVE channel number across their
# visible (not hidden_from_output) channels so overridden numbers drive the
# ordering, not the underlying provider values.
effective_min = Min(
Coalesce("channels__override__channel_number", "channels__channel_number")
)
hidden_exclusion = {"channels__hidden_from_output": False}
if user.user_level < 10:
user_profile_count = user.channel_profiles.count()
@ -2112,20 +2141,25 @@ def xc_get_live_categories(user):
if user_profile_count == 0:
# No profile filtering - user sees all channel groups
channel_groups = ChannelGroup.objects.filter(
channels__isnull=False, channels__user_level__lte=user.user_level
).distinct().annotate(min_channel_number=Min('channels__channel_number')).order_by('min_channel_number')
channels__isnull=False,
channels__user_level__lte=user.user_level,
**hidden_exclusion,
).distinct().annotate(min_channel_number=effective_min).order_by('min_channel_number')
else:
# User has specific limited profiles assigned
filters = {
"channels__channelprofilemembership__enabled": True,
"channels__user_level": 0,
"channels__channelprofilemembership__channel_profile__in": user.channel_profiles.all()
"channels__channelprofilemembership__channel_profile__in": user.channel_profiles.all(),
**hidden_exclusion,
}
channel_groups = ChannelGroup.objects.filter(**filters).distinct().annotate(min_channel_number=Min('channels__channel_number')).order_by('min_channel_number')
channel_groups = ChannelGroup.objects.filter(**filters).distinct().annotate(min_channel_number=effective_min).order_by('min_channel_number')
else:
channel_groups = ChannelGroup.objects.filter(
channels__isnull=False, channels__user_level__lte=user.user_level
).distinct().annotate(min_channel_number=Min('channels__channel_number')).order_by('min_channel_number')
channels__isnull=False,
channels__user_level__lte=user.user_level,
**hidden_exclusion,
).distinct().annotate(min_channel_number=effective_min).order_by('min_channel_number')
for group in channel_groups:
response.append(
@ -2140,6 +2174,8 @@ def xc_get_live_categories(user):
def xc_get_live_streams(request, user, category_id=None):
from apps.channels.managers import with_effective_values
streams = []
if user.user_level < 10:
@ -2154,7 +2190,7 @@ def xc_get_live_streams(request, user, category_id=None):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channels = Channel.objects.filter(**filters).select_related('channel_group', 'logo').order_by("channel_number")
base_qs = Channel.objects.filter(**filters).select_related('channel_group', 'logo')
else:
# User has specific limited profiles assigned
filters = {
@ -2167,14 +2203,20 @@ def xc_get_live_streams(request, user, category_id=None):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channels = Channel.objects.filter(**filters).select_related('channel_group', 'logo').distinct().order_by("channel_number")
base_qs = Channel.objects.filter(**filters).select_related('channel_group', 'logo').distinct()
else:
if not category_id:
channels = Channel.objects.filter(user_level__lte=user.user_level).select_related('channel_group', 'logo').order_by("channel_number")
base_qs = Channel.objects.filter(user_level__lte=user.user_level).select_related('channel_group', 'logo')
else:
channels = Channel.objects.filter(
base_qs = Channel.objects.filter(
channel_group__id=category_id, user_level__lte=user.user_level
).select_related('channel_group', 'logo').order_by("channel_number")
).select_related('channel_group', 'logo')
channels = (
with_effective_values(base_qs, select_related_fks=True)
.exclude(hidden_from_output=True)
.order_by("effective_channel_number")
)
# Resolve the fallback group ID once to avoid a get_or_create query per null-group channel
_default_group_id = None
@ -2190,21 +2232,21 @@ def xc_get_live_streams(request, user, category_id=None):
channel_num_map = {} # Maps channel.id -> integer channel number for XC
used_numbers = set() # Track all assigned integer channel numbers
# First pass: assign integers for channels that already have integer numbers
# First pass: assign integers for channels that already have integer effective numbers
for channel in channels:
if channel.channel_number == int(channel.channel_number):
# Already an integer, use it directly
num = int(channel.channel_number)
effective_num = channel.effective_channel_number
if effective_num is not None and effective_num == int(effective_num):
num = int(effective_num)
channel_num_map[channel.id] = num
used_numbers.add(num)
# Second pass: assign integers for channels with float numbers
# Find next available number to avoid collisions
for channel in channels:
if channel.channel_number != int(channel.channel_number):
effective_num = channel.effective_channel_number
if effective_num is not None and effective_num != int(effective_num):
# Has decimal component, need to find available integer
# Start from truncated value and increment until we find an unused number
candidate = int(channel.channel_number)
candidate = int(effective_num)
while candidate in used_numbers:
candidate += 1
channel_num_map[channel.id] = candidate
@ -2213,26 +2255,28 @@ def xc_get_live_streams(request, user, category_id=None):
# Build the streams list with the collision-free channel numbers
for channel in channels:
channel_num_int = channel_num_map[channel.id]
effective_logo = channel.effective_logo_obj
effective_group = channel.effective_channel_group_obj
streams.append(
{
"num": channel_num_int,
"name": channel.name,
"name": channel.effective_name,
"stream_type": "live",
"stream_id": channel.id,
"stream_icon": (
None
if not channel.logo
if not effective_logo
else build_absolute_uri_with_port(
request,
reverse("api:channels:logo-cache", args=[channel.logo.id])
reverse("api:channels:logo-cache", args=[effective_logo.id])
)
),
"epg_channel_id": str(channel_num_int),
"added": str(int(channel.created_at.timestamp())),
"is_adult": int(channel.is_adult),
"category_id": str(channel.channel_group.id if channel.channel_group else _get_default_group_id()),
"category_ids": [channel.channel_group.id if channel.channel_group else _get_default_group_id()],
"category_id": str(effective_group.id if effective_group else _get_default_group_id()),
"category_ids": [effective_group.id if effective_group else _get_default_group_id()],
"custom_sid": None,
"tv_archive": 0,
"direct_source": "",
@ -2244,11 +2288,19 @@ def xc_get_live_streams(request, user, category_id=None):
def xc_get_epg(request, user, short=False):
from apps.channels.managers import with_effective_values
channel_id = request.GET.get('stream_id')
if not channel_id:
raise Http404()
channel = None
# Apply effective-value annotation + hidden-exclusion at every channel
# resolution path so a single channel lookup honors the same visibility
# rules as xc_get_live_streams.
def _annotate(qs):
return with_effective_values(qs, select_related_fks=True).exclude(hidden_from_output=True)
if user.user_level < 10:
user_profile_count = user.channel_profiles.count()
@ -2262,7 +2314,7 @@ def xc_get_epg(request, user, short=False):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channel = Channel.objects.filter(**filters).select_related('epg_data__epg_source').first()
channel = _annotate(Channel.objects.filter(**filters).select_related('epg_data__epg_source')).first()
else:
# User has specific limited profiles assigned
filters = {
@ -2274,44 +2326,58 @@ def xc_get_epg(request, user, short=False):
# Hide adult content if user preference is set
if (user.custom_properties or {}).get('hide_adult_content', False):
filters["is_adult"] = False
channel = Channel.objects.filter(**filters).select_related('epg_data__epg_source').distinct().first()
channel = _annotate(Channel.objects.filter(**filters).select_related('epg_data__epg_source').distinct()).first()
if not channel:
raise Http404()
else:
channel = get_object_or_404(Channel.objects.select_related('epg_data__epg_source'), id=channel_id)
channel = _annotate(Channel.objects.filter(id=channel_id).select_related('epg_data__epg_source')).first()
if not channel:
raise Http404()
if not channel:
raise Http404()
# Calculate the collision-free integer channel number for this channel
# This must match the logic in xc_get_live_streams to ensure consistency
# Get all channels in the same category for collision detection
category_channels = Channel.objects.filter(
channel_group=channel.channel_group
).order_by("channel_number")
# This must match the logic in xc_get_live_streams to ensure consistency.
# The category channels must be filtered by the channel's EFFECTIVE group
# (an override can move a channel into a different group), then annotated
# so the comparison runs on effective numbers.
effective_group = channel.effective_channel_group_obj
category_channels = (
with_effective_values(
Channel.objects.filter(channel_group=effective_group) if effective_group else Channel.objects.none()
)
.exclude(hidden_from_output=True)
.order_by("effective_channel_number")
)
channel_num_map = {}
used_numbers = set()
# First pass: assign integers for channels that already have integer numbers
# First pass: assign integers for channels that already have integer effective numbers
for ch in category_channels:
if ch.channel_number == int(ch.channel_number):
num = int(ch.channel_number)
effective_num = ch.effective_channel_number
if effective_num is not None and effective_num == int(effective_num):
num = int(effective_num)
channel_num_map[ch.id] = num
used_numbers.add(num)
# Second pass: assign integers for channels with float numbers
# Second pass: assign integers for channels with float effective numbers
for ch in category_channels:
if ch.channel_number != int(ch.channel_number):
candidate = int(ch.channel_number)
effective_num = ch.effective_channel_number
if effective_num is not None and effective_num != int(effective_num):
candidate = int(effective_num)
while candidate in used_numbers:
candidate += 1
channel_num_map[ch.id] = candidate
used_numbers.add(candidate)
# Get the mapped integer for this specific channel
channel_num_int = channel_num_map.get(channel.id, int(channel.channel_number))
channel_num_int = channel_num_map.get(
channel.id,
int(channel.effective_channel_number) if channel.effective_channel_number is not None else 0,
)
limit = int(request.GET.get('limit', 4))
user_custom = user.custom_properties or {}
@ -2328,25 +2394,28 @@ def xc_get_epg(request, user, short=False):
now = django_timezone.now()
lookback_cutoff = now - timedelta(days=prev_days)
forward_cutoff = now + timedelta(days=num_days) if num_days > 0 else None
if channel.epg_data:
effective_epg_data = channel.effective_epg_data_obj
effective_name = channel.effective_name
if effective_epg_data:
# Check if this is a dummy EPG that generates on-demand
if channel.epg_data.epg_source and channel.epg_data.epg_source.source_type == 'dummy':
if not channel.epg_data.programs.exists():
if effective_epg_data.epg_source and effective_epg_data.epg_source.source_type == 'dummy':
if not effective_epg_data.programs.exists():
# Generate on-demand using custom patterns
programs = generate_dummy_programs(
channel_id=channel_id,
channel_name=channel.name,
epg_source=channel.epg_data.epg_source
channel_name=effective_name,
epg_source=effective_epg_data.epg_source
)
else:
# Has stored programs, use them
if short:
# Short EPG: current and upcoming only (never historical), limited count
programs = channel.epg_data.programs.filter(
programs = effective_epg_data.programs.filter(
end_time__gt=now
).order_by('start_time')[:limit]
else:
qs = channel.epg_data.programs.filter(end_time__gt=lookback_cutoff)
qs = effective_epg_data.programs.filter(end_time__gt=lookback_cutoff)
if forward_cutoff:
qs = qs.filter(start_time__lt=forward_cutoff)
programs = qs.order_by('start_time')
@ -2354,17 +2423,17 @@ def xc_get_epg(request, user, short=False):
# Regular EPG with stored programs
if short:
# Short EPG: current and upcoming only (never historical), limited count
programs = channel.epg_data.programs.filter(
programs = effective_epg_data.programs.filter(
end_time__gt=now
).order_by('start_time')[:limit]
else:
qs = channel.epg_data.programs.filter(end_time__gt=lookback_cutoff)
qs = effective_epg_data.programs.filter(end_time__gt=lookback_cutoff)
if forward_cutoff:
qs = qs.filter(start_time__lt=forward_cutoff)
programs = qs.order_by('start_time')
else:
# No EPG data assigned, generate default dummy
programs = generate_dummy_programs(channel_id=channel_id, channel_name=channel.name, epg_source=None)
programs = generate_dummy_programs(channel_id=channel_id, channel_name=effective_name, epg_source=None)
output = {"epg_listings": []}
@ -2385,7 +2454,7 @@ def xc_get_epg(request, user, short=False):
# epg_id refers to the EPG source/channel mapping in XC panels
# Use the actual EPGData ID when available, otherwise fall back to 0
epg_id = str(channel.epg_data.id) if channel.epg_data else "0"
epg_id = str(effective_epg_data.id) if effective_epg_data else "0"
program_output = {
"id": program_id,

95
frontend/src/api.js
View file

@ -271,6 +271,77 @@ export default class API {
}
}
// Repack visible auto-created channels into [start, end]; override
// pins are reservations and hidden non-pinned channels release their
// number.
static async repackGroupChannels(accountId, channelGroupId) {
try {
const params = new URLSearchParams({
channel_group_id: String(channelGroupId),
});
const url = `${host}/api/m3u/accounts/${accountId}/repack-group/?${params.toString()}`;
return await request(url, { method: 'POST' });
} catch (e) {
errorNotification('Failed to re-pack group channels', e);
return null;
}
}
// Returns occupants whose effective channel_number falls in [start, end].
// Pass `signal` from an AbortController on per-keystroke calls so an
// out-of-order response cannot overwrite newer state.
static async getChannelsInRange(start, end, { signal } = {}) {
try {
const params = new URLSearchParams({ start: String(start) });
if (end !== undefined && end !== null && end !== '') {
params.set('end', String(end));
}
const url = `${host}/api/channels/channels/numbers-in-range/?${params.toString()}`;
return await request(url, { signal });
} catch (e) {
if (e?.name === 'AbortError') {
throw e;
}
// Silent failure is correct here: the warning is purely advisory and
// should not block the user from saving when the server is flaky.
return { occupants: [] };
}
}
// Server-side regex preview for a group's streams. Returns find_matches,
// filter_matches, and exclude_matches plus accurate counts across the
// whole group (capped at 5000 scanned streams server-side). Used by the
// auto-sync gear modal so the user sees real matches and totals rather
// than a small client-side sample. The three patterns are independent;
// any combination can be supplied per call.
static async getStreamsRegexPreview(
channelGroupName,
{ find, replace, match, exclude, limit = 10, signal, m3uAccountId } = {}
) {
try {
const params = new URLSearchParams({
channel_group: channelGroupName,
});
if (m3uAccountId !== undefined && m3uAccountId !== null) {
params.set('m3u_account_id', String(m3uAccountId));
}
if (find) params.set('find', find);
if (replace !== undefined && replace !== null) {
params.set('replace', replace);
}
if (match) params.set('match', match);
if (exclude) params.set('exclude', exclude);
params.set('limit', String(limit));
const url = `${host}/api/channels/streams/regex-preview/?${params.toString()}`;
return await request(url, { signal });
} catch (e) {
if (e?.name === 'AbortError') {
throw e;
}
return null;
}
}
static async queryChannels(params) {
try {
API.lastQueryParams = params;
@ -1341,14 +1412,34 @@ export default class API {
}
static async deletePlaylist(id) {
// Cascade-deletes auto-created channels owned by the account; the
// response includes the deleted count for the confirmation toast.
try {
await request(`${host}/api/m3u/accounts/${id}/`, {
const response = await request(`${host}/api/m3u/accounts/${id}/`, {
method: 'DELETE',
});
usePlaylistsStore.getState().removePlaylists([id]);
return response || {};
} catch (e) {
errorNotification(`Failed to delete playlist ${id}`, e);
throw e;
}
}
// Used by the Delete Playlist confirmation dialog to render an accurate
// "Also delete N auto-created channels?" option before the user commits.
static async getPlaylistAutoCreatedChannelsCount(id) {
try {
const response = await request(
`${host}/api/m3u/accounts/${id}/auto-created-channels-count/`
);
return response;
} catch (e) {
console.error(
`Failed to fetch auto-created channel count for playlist ${id}`,
e
);
return { count: 0, sample_names: [] };
}
}

156
frontend/src/components/M3URefreshNotification.jsx
View file

@ -5,7 +5,15 @@ import useStreamsStore from '../store/streams';
import useChannelsStore from '../store/channels';
import useEPGsStore from '../store/epgs';
import useVODStore from '../store/useVODStore';
import { Stack, Button, Group } from '@mantine/core';
import {
Stack,
Button,
Group,
Modal,
ScrollArea,
Text,
Code,
} from '@mantine/core';
import API from '../api';
import { useNavigate } from 'react-router-dom';
import { CircleCheck } from 'lucide-react';
@ -43,6 +51,47 @@ const M3uSetupSuccess = ({ data }) => {
);
};
// One-line outcome summary for the notification body.
const buildAutoSyncSummary = (data) => {
const created = data.channels_created || 0;
const updated = data.channels_updated || 0;
const deleted = data.channels_deleted || 0;
const failed = data.channels_failed || 0;
if (!created && !updated && !deleted && !failed) return null;
const parts = [];
if (created) parts.push(`${created} created`);
if (updated) parts.push(`${updated} updated`);
if (deleted) parts.push(`${deleted} deleted`);
if (failed) parts.push(`${failed} failed`);
return `Auto-sync: ${parts.join(', ')}.`;
};
// Human labels for the typed failure reasons attached to each
// failed_stream_details entry. Unknown reasons fall back to the raw
// key so a future reason added on the backend still surfaces in the
// modal without a code change here.
const FAILURE_REASON_LABELS = {
RANGE_EXHAUSTED: 'Channel number range exhausted',
INTEGRITY_ERROR: 'Database integrity error',
OTHER: 'Other errors',
};
const FAILURES_PER_GROUP_LIMIT = 50;
// Group entries by their reason key, preserving insertion order so
// section ordering reflects the sync's encounter order. Entries
// without a reason field bucket into OTHER so a backend that does
// not classify failures still renders correctly.
const groupFailuresByReason = (entries) => {
const buckets = new Map();
for (const entry of entries) {
const key = entry?.reason || 'OTHER';
if (!buckets.has(key)) buckets.set(key, []);
buckets.get(key).push(entry);
}
return buckets;
};
export default function M3URefreshNotification() {
const playlists = usePlaylistsStore((s) => s.playlists);
const refreshProgress = usePlaylistsStore((s) => s.refreshProgress);
@ -54,6 +103,9 @@ export default function M3URefreshNotification() {
const fetchCategories = useVODStore((s) => s.fetchCategories);
const [notificationStatus, setNotificationStatus] = useState({});
// Modal payload for the "Click for details" affordance on syncs that
// produced failed_stream_details. Null when the modal is closed.
const [failureModal, setFailureModal] = useState(null);
const handleM3UUpdate = (data) => {
// Skip if status hasn't changed
@ -148,7 +200,7 @@ export default function M3URefreshNotification() {
const handleProgressNotification = (playlist, data) => {
const baseMessage = getActionMessage(data.action);
const message =
let message =
data.progress == 0
? `${baseMessage} starting...`
: `${baseMessage} complete!`;
@ -157,11 +209,49 @@ export default function M3URefreshNotification() {
triggerPostCompletionFetches(data.action);
}
let body = message;
let autoClose = 2000;
// Surface auto-sync counts attached to the parsing-complete event
// so the channel-side outcome appears in the notification body.
if (data.progress == 100 && data.action === 'parsing') {
const autoSyncSummary = buildAutoSyncSummary(data);
const failed = data.channels_failed || 0;
const failedDetails = Array.isArray(data.failed_stream_details)
? data.failed_stream_details
: [];
if (autoSyncSummary) {
body = (
<Stack gap={4}>
<Text size="sm">{message}</Text>
<Text size="xs" c="dimmed">
{autoSyncSummary}
</Text>
{failed > 0 && failedDetails.length > 0 && (
<Button
size="xs"
variant="subtle"
color="yellow"
onClick={() =>
setFailureModal({
playlistName: playlist.name,
failedDetails,
})
}
>
Click for details
</Button>
)}
</Stack>
);
autoClose = failed > 0 ? 12000 : 4000;
}
}
showNotification({
title: `M3U Processing: ${playlist.name}`,
message,
message: body,
loading: data.progress == 0,
autoClose: 2000,
autoClose,
icon: data.progress == 100 ? <CircleCheck /> : null,
});
};
@ -182,5 +272,61 @@ export default function M3URefreshNotification() {
Object.values(refreshProgress).map((data) => handleM3UUpdate(data));
}, [playlists, refreshProgress]);
return <></>;
return (
<Modal
opened={!!failureModal}
onClose={() => setFailureModal(null)}
title={
failureModal
? `Auto-sync failures: ${failureModal.playlistName}`
: 'Auto-sync failures'
}
size="lg"
>
<Stack>
<Text size="sm" c="dimmed">
The following streams could not be synced. Failures are grouped by
cause so the most common issues surface first.
</Text>
<ScrollArea h={360}>
<Stack gap="md">
{Array.from(
groupFailuresByReason(failureModal?.failedDetails || []).entries()
).map(([reasonKey, entries]) => {
const label = FAILURE_REASON_LABELS[reasonKey] || reasonKey;
const visible = entries.slice(0, FAILURES_PER_GROUP_LIMIT);
const hidden = entries.length - visible.length;
return (
<Stack key={reasonKey} gap="xs">
<Text size="sm" fw={600}>
{label} ({entries.length})
</Text>
{visible.map((entry, idx) => (
<Code
block
key={`${reasonKey}-${entry.stream_id ?? 'na'}-${idx}`}
>
{[
entry.stream_name && `Stream: ${entry.stream_name}`,
entry.group && `Group: ${entry.group}`,
entry.error && `Error: ${entry.error}`,
]
.filter(Boolean)
.join('\n')}
</Code>
))}
{hidden > 0 && (
<Text size="xs" c="dimmed">
Showing first {visible.length} of {entries.length}.
Remaining {hidden} entries are recorded in the server log.
</Text>
)}
</Stack>
);
})}
</Stack>
</ScrollArea>
</Stack>
</Modal>
);
}

137
frontend/src/components/__tests__/M3URefreshNotification.test.jsx
View file

@ -52,6 +52,19 @@ vi.mock('@mantine/core', async () => {
Button: ({ children, onClick }) => (
<button onClick={onClick}>{children}</button>
),
// Stub for the auto-sync failure-details modal.
Modal: ({ children, opened, onClose, title }) =>
opened ? (
<div data-testid="modal" role="dialog" aria-label={title}>
<button data-testid="modal-close" onClick={onClose}>
close
</button>
{children}
</div>
) : null,
ScrollArea: ({ children }) => <div>{children}</div>,
Text: ({ children }) => <span>{children}</span>,
Code: ({ children }) => <pre>{children}</pre>,
};
});
@ -714,4 +727,128 @@ describe('M3URefreshNotification', () => {
);
});
});
// The parsing-complete event payload carries auto-sync counts
// (channels_created/updated/deleted/failed) and a failed_stream_details
// array. The component surfaces the counts inline in the notification
// body and exposes a "Click for details" affordance when failures exist.
describe('Auto-sync count rendering on parsing complete', () => {
it('inlines auto-sync summary when counts are present', async () => {
mockPlaylistsStore.refreshProgress = {
1: {
account: 1,
action: 'parsing',
progress: 100,
status: 'success',
channels_created: 12,
channels_updated: 3,
channels_deleted: 1,
channels_failed: 0,
failed_stream_details: [],
},
};
renderWithProviders(<M3URefreshNotification />);
await waitFor(() => {
expect(showNotification).toHaveBeenCalled();
});
// The message arg is JSX; render it and confirm both the
// base message and the auto-sync summary text appear.
const call = showNotification.mock.calls.find(
(c) => typeof c[0]?.message === 'object'
);
expect(call).toBeDefined();
const { container } = render(<>{call[0].message}</>);
expect(container.textContent).toContain('Stream parsing complete!');
expect(container.textContent).toContain('12 created');
expect(container.textContent).toContain('3 updated');
expect(container.textContent).toContain('1 deleted');
});
it('shows "Click for details" button when failures exist', async () => {
mockPlaylistsStore.refreshProgress = {
1: {
account: 1,
action: 'parsing',
progress: 100,
status: 'success',
channels_created: 5,
channels_updated: 0,
channels_deleted: 0,
channels_failed: 2,
failed_stream_details: [
{
stream_name: 'BadStream1',
group: 'Sports',
error: 'Range exhausted',
},
{
stream_name: 'BadStream2',
group: 'News',
error: 'Channel number conflict',
},
],
},
};
renderWithProviders(<M3URefreshNotification />);
await waitFor(() => {
expect(showNotification).toHaveBeenCalled();
});
const call = showNotification.mock.calls.find(
(c) => typeof c[0]?.message === 'object'
);
expect(call).toBeDefined();
const { container } = render(<>{call[0].message}</>);
expect(container.textContent).toContain('2 failed');
expect(container.textContent).toContain('Click for details');
});
it('falls back to plain string body when no auto-sync counts arrive', async () => {
// Older payload shape (or non-parsing actions) get the original
// simple "X complete!" string body, no JSX wrapping.
mockPlaylistsStore.refreshProgress = {
1: {
account: 1,
action: 'downloading',
progress: 100,
status: 'success',
},
};
renderWithProviders(<M3URefreshNotification />);
await waitFor(() => {
expect(showNotification).toHaveBeenCalled();
});
const call = showNotification.mock.calls[0];
expect(typeof call[0].message).toBe('string');
});
it('uses extended autoClose when failures are present', async () => {
mockPlaylistsStore.refreshProgress = {
1: {
account: 1,
action: 'parsing',
progress: 100,
status: 'success',
channels_created: 1,
channels_failed: 1,
failed_stream_details: [{ stream_name: 'X', group: 'Y', error: 'Z' }],
},
};
renderWithProviders(<M3URefreshNotification />);
await waitFor(() => {
expect(showNotification).toHaveBeenCalled();
});
const call = showNotification.mock.calls[0];
// 12000ms when failures > 0; 4000ms when summary present but no
// failures; 2000ms when no auto-sync counts.
expect(call[0].autoClose).toBe(12000);
});
});
});

281
frontend/src/components/forms/Channel.jsx
View file

@ -33,7 +33,7 @@ import {
UnstyledButton,
useMantineTheme,
} from '@mantine/core';
import { ListOrdered, SquarePlus, X, Zap } from 'lucide-react';
import { ListOrdered, SquarePlus, Undo2, X, Zap } from 'lucide-react';
import useEPGsStore from '../../store/epgs';
import { FixedSizeList as List } from 'react-window';
import { USER_LEVEL_LABELS, USER_LEVELS } from '../../constants';
@ -43,11 +43,18 @@ import {
} from '../../utils/notificationUtils.js';
import {
addChannel,
clearChannelOverrides,
createLogo,
getChannelFormDefaultValues,
getFkProviderHint,
getFormattedValues,
getProviderFormValue,
getProviderHint,
handleEpgUpdate,
isFormFieldOverridden,
matchChannelEpg,
OVERRIDABLE_FIELDS,
OVERRIDE_FIELD_LABELS,
requeryChannels,
} from '../../utils/forms/ChannelUtils.js';
@ -56,13 +63,48 @@ const validationSchema = Yup.object({
channel_group_id: Yup.string().required('Channel group is required'),
});
const ChannelForm = ({ channel = null, isOpen, onClose }) => {
// Provider hint plus a reset-to-provider icon for auto-synced
// channels; rendered as the field's `description` prop.
const ProviderHintRow = ({ channel, field, formValue, hintText, onReset }) => {
if (!hintText) return null;
const overridden = isFormFieldOverridden(channel, field, formValue);
return (
<span style={{ display: 'inline-flex', alignItems: 'center', gap: 4 }}>
<Text size="xs" c="dimmed" component="span">
{hintText}
</Text>
{overridden && (
<Tooltip label="Reset to provider value" withArrow>
<ActionIcon
size="xs"
variant="subtle"
color="orange"
onClick={onReset}
aria-label={`Reset ${field} to provider value`}
>
<Undo2 size={11} />
</ActionIcon>
</Tooltip>
)}
</span>
);
};
const ChannelForm = ({ channel: channelProp = null, isOpen, onClose }) => {
const theme = useMantineTheme();
const listRef = useRef(null);
const logoListRef = useRef(null);
const groupListRef = useRef(null);
// Local copy so in-modal mutations (clear overrides, etc.) update the
// form immediately. Reset on each open from `channelProp`; mutated in
// place with API responses.
const [channel, setChannel] = useState(channelProp);
useEffect(() => {
setChannel(channelProp);
}, [channelProp]);
const channelGroups = useChannelsStore((s) => s.channelGroups);
const {
@ -309,7 +351,45 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
resolver: yupResolver(validationSchema),
});
const clearOverrides = async () => {
if (!channel) return;
try {
const updated = await clearChannelOverrides(channel.id);
// Update local state first so the form reflects the cleared
// overrides immediately; the table-store refresh is best-effort.
if (updated && typeof updated === 'object') {
setChannel(updated);
}
requeryChannels();
showNotification({
title: 'Overrides Cleared',
message: 'Channel values now follow the provider.',
color: 'green',
});
} catch (error) {
showNotification({
title: 'Clear Failed',
message:
error?.body?.detail || error?.message || 'Could not clear overrides.',
color: 'red',
});
}
};
// Computed from live form values so per-field resets update the
// Clear-all button immediately, before submit.
const watchedFormValues = watch();
const overriddenFieldLabels = useMemo(() => {
if (!channel) return [];
return OVERRIDABLE_FIELDS.filter((field) =>
isFormFieldOverridden(channel, field, watchedFormValues[field])
).map((field) => OVERRIDE_FIELD_LABELS[field]);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [channel, JSON.stringify(watchedFormValues)]);
const hasAnyOverride = overriddenFieldLabels.length > 0;
const onSubmit = async (values) => {
let saveFailed = false;
try {
const formattedValues = getFormattedValues(values);
@ -324,8 +404,29 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
}
} catch (error) {
console.error('Error saving channel:', error);
saveFailed = true;
showNotification({
title: 'Save Failed',
message:
error?.body?.detail || error?.message || 'Failed to save channel.',
color: 'red',
});
}
if (saveFailed) {
// Keep the form open with the user's edits intact so they can correct
// a validation error without retyping.
return;
}
showNotification({
title: 'Saved',
message: channel
? `Channel "${values.name}" updated.`
: `Channel "${values.name}" created.`,
color: 'green',
});
reset();
requeryChannels();
@ -408,6 +509,17 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
styles={{ content: { '--mantine-color-body': '#27272A' } }}
>
<form onSubmit={handleSubmit(onSubmit)}>
{channel?.auto_created && channel?.source_stream && (
<Text size="xs" c="dimmed" mb="xs">
Auto-created from:{' '}
<Text component="span" fw={500} c="gray.3">
{channel.source_stream.account_name || 'Unknown provider'}
</Text>
{channel.source_stream.name
? ` / ${channel.source_stream.name}`
: ''}
</Text>
)}
<Group justify="space-between" align="top">
<Stack gap="5" style={{ flex: 1 }}>
<TextInput
@ -430,6 +542,19 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
)}
</Group>
}
description={
<ProviderHintRow
channel={channel}
field="name"
formValue={watch('name')}
hintText={getProviderHint(channel, 'name')}
onReset={() =>
setValue('name', getProviderFormValue(channel, 'name'), {
shouldDirty: true,
})
}
/>
}
{...register('name')}
error={errors.name?.message}
size="xs"
@ -448,6 +573,24 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
id="channel_group_id"
name="channel_group_id"
label="Channel Group"
description={
<ProviderHintRow
channel={channel}
field="channel_group_id"
formValue={watch('channel_group_id')}
hintText={getFkProviderHint(
channel,
'channel_group_id',
channelGroups
)}
onReset={() =>
setValue(
'channel_group_id',
getProviderFormValue(channel, 'channel_group_id')
)
}
/>
}
readOnly
value={
channelGroups[watch('channel_group_id')]
@ -535,6 +678,27 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
id="stream_profile_id"
label="Stream Profile"
name="stream_profile_id"
description={
<ProviderHintRow
channel={channel}
field="stream_profile_id"
formValue={watch('stream_profile_id')}
hintText={getFkProviderHint(
channel,
'stream_profile_id',
streamProfiles.reduce((acc, p) => {
acc[p.id] = p;
return acc;
}, {})
)}
onReset={() =>
setValue(
'stream_profile_id',
getProviderFormValue(channel, 'stream_profile_id')
)
}
/>
}
value={watch('stream_profile_id')}
onChange={(value) => {
setValue('stream_profile_id', value);
@ -605,6 +769,24 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
)}
</Group>
}
description={
<ProviderHintRow
channel={channel}
field="logo_id"
formValue={watch('logo_id')}
hintText={getFkProviderHint(
channel,
'logo_id',
channelLogos
)}
onReset={() =>
setValue(
'logo_id',
getProviderFormValue(channel, 'logo_id')
)
}
/>
}
readOnly
value={channelLogos[watch('logo_id')]?.name || 'Default'}
onClick={() => {
@ -738,6 +920,38 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
/>
</Box>
</Tooltip>
<Tooltip
label="Hides this channel from HDHR, M3U, EPG, and XC client output and preserves it from auto-cleanup. To hide channels per-user, use channel profiles instead."
withArrow
multiline
w={320}
>
<Box>
<Switch
label="Hidden"
checked={watch('hidden_from_output')}
onChange={(event) =>
setValue('hidden_from_output', event.currentTarget.checked)
}
size="md"
/>
</Box>
</Tooltip>
{channel?.auto_created && hasAnyOverride && (
<Tooltip
label={`Currently overriding: ${overriddenFieldLabels.join(', ')}. Clear all overrides to follow the provider values again on the next refresh.`}
withArrow
>
<Button
variant="light"
color="orange"
size="xs"
onClick={clearOverrides}
>
Clear All Overrides ({overriddenFieldLabels.length})
</Button>
</Tooltip>
)}
</Stack>
<Divider size="sm" orientation="vertical" />
@ -747,6 +961,21 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
id="channel_number"
name="channel_number"
label="Channel # (blank to auto-assign)"
description={
<ProviderHintRow
channel={channel}
field="channel_number"
formValue={watch('channel_number')}
hintText={getProviderHint(channel, 'channel_number')}
onReset={() =>
setValue(
'channel_number',
getProviderFormValue(channel, 'channel_number'),
{ shouldDirty: true }
)
}
/>
}
value={watch('channel_number')}
onChange={(value) => setValue('channel_number', value)}
error={errors.channel_number?.message}
@ -775,6 +1004,21 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
)}
</Group>
}
description={
<ProviderHintRow
channel={channel}
field="tvg_id"
formValue={watch('tvg_id')}
hintText={getProviderHint(channel, 'tvg_id')}
onReset={() =>
setValue(
'tvg_id',
getProviderFormValue(channel, 'tvg_id'),
{ shouldDirty: true }
)
}
/>
}
{...register('tvg_id')}
error={errors.tvg_id?.message}
size="xs"
@ -784,6 +1028,21 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
id="tvc_guide_stationid"
name="tvc_guide_stationid"
label="Gracenote StationId"
description={
<ProviderHintRow
channel={channel}
field="tvc_guide_stationid"
formValue={watch('tvc_guide_stationid')}
hintText={getProviderHint(channel, 'tvc_guide_stationid')}
onReset={() =>
setValue(
'tvc_guide_stationid',
getProviderFormValue(channel, 'tvc_guide_stationid'),
{ shouldDirty: true }
)
}
/>
}
{...register('tvc_guide_stationid')}
error={errors.tvc_guide_stationid?.message}
size="xs"
@ -829,6 +1088,24 @@ const ChannelForm = ({ channel = null, isOpen, onClose }) => {
</Button>
</Group>
}
description={
<ProviderHintRow
channel={channel}
field="epg_data_id"
formValue={watch('epg_data_id')}
hintText={getFkProviderHint(
channel,
'epg_data_id',
tvgsById
)}
onReset={() =>
setValue(
'epg_data_id',
getProviderFormValue(channel, 'epg_data_id')
)
}
/>
}
readOnly
value={(() => {
const tvg = tvgsById[watch('epg_data_id')];

113
frontend/src/components/forms/ChannelBatch.jsx
View file

@ -54,6 +54,7 @@ import {
setChannelNamesFromEpg,
setChannelTvgIdsFromEpg,
updateChannels,
updateChannelsWithOverrideRouting,
} from '../../utils/forms/ChannelBatchUtils.js';
const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
@ -123,14 +124,40 @@ const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
stream_profile_id: '-1',
user_level: '-1',
is_adult: '-1',
hidden_from_output: '-1',
clear_overrides: '-1',
},
});
// Surfaces auto-vs-manual routing in the selection. Falls back to a
// single total when the table store only has a partial view (e.g.
// cross-page selects). Kept separate from getConfirmationMessage so
// the line does not count against the no-changes guard.
const getSelectionSummary = () => {
const channelsById = useChannelsTableStore
.getState()
.channels.reduce((acc, c) => {
acc[c.id] = c;
return acc;
}, {});
let autoCount = 0;
let manualCount = 0;
for (const id of channelIds) {
const c = channelsById[id];
if (!c) continue;
if (c.auto_created) autoCount++;
else manualCount++;
}
const resolved = autoCount + manualCount;
return resolved === channelIds.length
? `Selection: ${autoCount} auto-synced, ${manualCount} manual`
: `Selection: ${channelIds.length} channels`;
};
// Build confirmation message based on selected changes
const getConfirmationMessage = () => {
const values = form.getValues();
return [
const lines = [
getRegexNameChange(regexFind, regexReplace),
getChannelGroupChange(selectedChannelGroup, channelGroups),
getLogoChange(selectedLogoId, channelLogos),
@ -138,7 +165,18 @@ const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
getUserLevelChange(values.user_level, USER_LEVEL_LABELS),
getMatureContentChange(values.is_adult),
getEpgChange(selectedDummyEpgId, epgs),
].filter(Boolean);
];
if (values.hidden_from_output && values.hidden_from_output !== '-1') {
lines.push(
`• Hidden: ${values.hidden_from_output === 'true' ? 'Yes' : 'No'}`
);
}
if (values.clear_overrides === 'clear') {
lines.push(
'• Clear all overrides on auto-synced channels in selection, then apply the edits above as new overrides'
);
}
return lines.filter(Boolean);
};
const handleSubmit = () => {
@ -167,14 +205,37 @@ const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
setIsSubmitting(true);
try {
const formValues = form.getValues();
const shouldClearOverrides = formValues.clear_overrides === 'clear';
const values = buildSubmitValues(
form.getValues(),
formValues,
selectedChannelGroup,
selectedLogoId
);
// Clear runs before the routing PATCH (not in parallel) so a
// late-landing clear cannot wipe the freshly-written override
// fields.
if (shouldClearOverrides && channelIds.length > 0) {
await updateChannels(channelIds, { override: null });
}
if (Object.keys(values).length > 0) {
await updateChannels(channelIds, values);
// Route auto-created channels to override.X (survives sync)
// and manual channels to direct Channel.X writes; auto_created
// is read from the table store.
const channelsById = useChannelsTableStore
.getState()
.channels.reduce((acc, c) => {
acc[c.id] = c;
return acc;
}, {});
await updateChannelsWithOverrideRouting(
channelIds,
values,
channelsById
);
}
if (regexFind.trim().length > 0) {
@ -197,7 +258,16 @@ const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
]);
onClose();
} catch (error) {
console.error('Failed to update channels:', error);
// Keep the form open with the user's edits intact so they can correct
// a validation error without retyping the bulk selection.
showNotification({
title: 'Bulk Update Failed',
message:
error?.body?.detail ||
error?.message ||
'Failed to apply changes to the selected channels.',
color: 'red',
});
} finally {
setIsSubmitting(false);
}
@ -457,6 +527,9 @@ const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
styles={{ hannontent: { '--mantine-color-body': '#27272A' } }}
>
<form onSubmit={form.onSubmit(handleSubmit)}>
<Text size="xs" c="dimmed" mb="xs">
{getSelectionSummary()}
</Text>
<Group justify="space-between" align="top">
<Stack gap="5" style={{ flex: 1 }}>
<Paper withBorder p="xs" radius="md">
@ -799,6 +872,31 @@ const ChannelBatchForm = ({ channelIds, isOpen, onClose }) => {
{ value: 'false', label: 'No' },
]}
/>
<Select
size="xs"
label="Hidden"
description="Hidden channels are excluded from HDHR, M3U, EPG, and XC output. Use channel profiles to hide per-user."
{...form.getInputProps('hidden_from_output')}
key={form.key('hidden_from_output')}
data={[
{ value: '-1', label: '(no change)' },
{ value: 'true', label: 'Yes' },
{ value: 'false', label: 'No' },
]}
/>
<Select
size="xs"
label="Overrides (auto-synced channels only)"
description="Clearing removes all user overrides and lets the next sync write provider values again. Applies only to auto-synced channels in the selection."
{...form.getInputProps('clear_overrides')}
key={form.key('clear_overrides')}
data={[
{ value: '-1', label: '(no change)' },
{ value: 'clear', label: 'Clear all overrides' },
]}
/>
</Stack>
</Group>
<Flex mih={50} gap="xs" justify="flex-end" align="flex-end">
@ -905,6 +1003,9 @@ This action cannot be undone.`}
style={{ backgroundColor: 'rgba(0, 0, 0, 0.2)' }}
>
<Stack gap="xs">
<Text size="sm" c="dimmed" style={{ fontFamily: 'monospace' }}>
{getSelectionSummary()}
</Text>
{getConfirmationMessage().map((change, index) => (
<Text
key={index}

50
frontend/src/components/forms/GroupConfigureModal.jsx Normal file
View file

@ -0,0 +1,50 @@
// Per-group "advanced config" modal opened from the gear icon on each row
// in LiveGroupFilter / M3UGroupFilter. The inline row keeps only the core
// Sync toggle, Numbering Mode, and Start/End inputs. This modal renders
// the full Advanced Options MultiSelect plus its conditional fields,
// passed in as children by the parent so it continues to own the group
// list and field logic.
import React from 'react';
import { Button, Modal, Stack, Group, Text } from '@mantine/core';
const GroupConfigureModal = ({ opened, onDone, onCancel, group, children }) => {
if (!group) return null;
const streamCount =
typeof group.stream_count === 'number' ? group.stream_count : null;
return (
<Modal
opened={opened}
onClose={onCancel}
withCloseButton={false}
size="lg"
title={
<Group gap="xs">
<Text fw={600}>Configure: {group.name}</Text>
{streamCount !== null && (
<Text size="xs" c="dimmed">
({streamCount} stream{streamCount === 1 ? '' : 's'} available)
</Text>
)}
</Group>
}
styles={{ content: { '--mantine-color-body': '#27272A' } }}
>
<Stack gap="md">{children}</Stack>
{/* Done keeps in-memory edits routed into the parent's groupStates
(parent's Save and Refresh persists them). Cancel reverts to
the open-time snapshot. */}
<Group justify="flex-end" gap="xs" mt="md">
<Button variant="default" onClick={onCancel} size="xs">
Cancel
</Button>
<Button variant="filled" color="blue" onClick={onDone} size="xs">
Done
</Button>
</Group>
</Modal>
);
};
export default GroupConfigureModal;

2645
frontend/src/components/forms/LiveGroupFilter.jsx
View file

File diff suppressed because it is too large Load diff

4
frontend/src/components/forms/Logo.jsx
View file

@ -250,6 +250,10 @@ const LogoForm = ({ logo = null, isOpen, onClose, onSuccess }) => {
onClose={onClose}
title={logo ? 'Edit Logo' : 'Add Logo'}
size="md"
// Render above any other open modal (e.g. the per-group gear modal
// in LiveGroupFilter) when this is invoked from one. Default Mantine
// modal zIndex is 200; bumping to 1000 here keeps it on top.
zIndex={1000}
>
<form onSubmit={handleSubmit(onSubmit)}>
<Stack spacing="md">

19
frontend/src/components/forms/M3U.jsx
View file

@ -19,6 +19,7 @@ import {
Switch,
Box,
PasswordInput,
Tooltip,
} from '@mantine/core';
import M3UGroupFilter from './M3UGroupFilter';
import useChannelsStore from '../../store/channels';
@ -459,16 +460,20 @@ const M3U = ({
{...form.getInputProps('priority')}
key={form.key('priority')}
/>
<Checkbox
label="Is Active"
description="Enable or disable this M3U account"
{...form.getInputProps('is_active', { type: 'checkbox' })}
key={form.key('is_active')}
/>
</Stack>
</Group>
<Divider my="md" />
<Flex gap="xl" wrap="wrap">
<Checkbox
label="Is Active"
description="Enable or disable this M3U account"
{...form.getInputProps('is_active', { type: 'checkbox' })}
key={form.key('is_active')}
/>
</Flex>
<Flex mih={50} gap="xs" justify="flex-end" align="flex-end">
{playlist && (
<>

19
frontend/src/components/forms/M3UGroupFilter.jsx
View file

@ -14,6 +14,7 @@ import useVODStore from '../../store/useVODStore';
import { notifications } from '@mantine/notifications';
import LiveGroupFilter from './LiveGroupFilter';
import VODCategoryFilter from './VODCategoryFilter';
import { detectGroupReservationOverlaps } from '../../utils/forms/GroupSyncUtils';
const M3UGroupFilter = ({ playlist = null, isOpen, onClose }) => {
const channelGroups = useChannelsStore((s) => s.channelGroups);
@ -55,7 +56,7 @@ const M3UGroupFilter = ({ playlist = null, isOpen, onClose }) => {
typeof group.custom_properties === 'string'
? JSON.parse(group.custom_properties)
: group.custom_properties;
} catch (e) {
} catch {
customProps = {};
}
}
@ -64,6 +65,7 @@ const M3UGroupFilter = ({ playlist = null, isOpen, onClose }) => {
name: channelGroups[group.channel_group].name,
auto_channel_sync: group.auto_channel_sync || false,
auto_sync_channel_start: group.auto_sync_channel_start || 1.0,
auto_sync_channel_end: group.auto_sync_channel_end ?? null,
custom_properties: customProps,
};
})
@ -83,6 +85,21 @@ const M3UGroupFilter = ({ playlist = null, isOpen, onClose }) => {
}, [isOpen, playlist, fetchCategories]);
const submit = async () => {
// Advisory only: overlapping ranges are sometimes intentional (for
// example, two providers carrying the same category that should
// merge into one shared number range). The form already shows a
// warning triangle on each affected group with the specific overlap
// names on hover, so the toast just confirms the save proceeded.
const overlaps = detectGroupReservationOverlaps(groupStates);
if (overlaps.length > 0) {
notifications.show({
title: 'Overlapping channel number ranges',
message: `Saved with ${overlaps.length} overlapping range pair${overlaps.length === 1 ? '' : 's'}. Hover the warning icon on each group for details. Sync will assign whichever numbers are free at run time.`,
color: 'yellow',
autoClose: 6000,
});
}
setIsLoading(true);
try {
// Prepare groupStates for API

135
frontend/src/components/forms/__tests__/Channel.test.jsx
View file

@ -20,12 +20,25 @@ vi.mock('../../../utils/notificationUtils.js', () => ({
vi.mock('../../../utils/forms/ChannelUtils.js', () => ({
addChannel: vi.fn(),
clearChannelOverrides: vi.fn(),
createLogo: vi.fn(),
getChannelFormDefaultValues: vi.fn(),
// Stubs return null so the form's `description` prop renders cleanly.
getProviderHint: vi.fn(() => null),
getFkProviderHint: vi.fn(() => null),
getProviderFormValue: vi.fn(() => ''),
getFormattedValues: vi.fn(),
handleEpgUpdate: vi.fn(),
isFormFieldOverridden: vi.fn(() => false),
matchChannelEpg: vi.fn(),
normalizeFieldValue: vi.fn((field, value) => value),
OVERRIDABLE_FIELDS: ['name', 'channel_number'],
OVERRIDE_FIELD_LABELS: {
name: 'Name',
channel_number: 'Channel Number',
},
requeryChannels: vi.fn(),
buildOverridePayload: vi.fn(() => null),
}));
// ── Child component mocks ──────────────────────────────────────────────────────
@ -232,17 +245,29 @@ vi.mock('@mantine/core', async () => ({
{children}
</div>
),
Switch: ({ label, checked, onChange }) => (
<label>
<input
data-testid="switch-mature"
type="checkbox"
checked={checked}
onChange={onChange}
/>
{label}
</label>
),
Switch: ({ label, checked, onChange }) => {
// The form renders multiple switches (Mature Content, Hide from
// Clients, and future flags), so derive a unique testid per label.
// The Mature Content switch keeps "switch-mature" so tests that look
// it up by that legacy id continue to work.
const key =
typeof label === 'string' && label.toLowerCase().includes('mature')
? 'switch-mature'
: `switch-${String(label || '')
.toLowerCase()
.replace(/\s+/g, '-')}`;
return (
<label>
<input
data-testid={key}
type="checkbox"
checked={checked}
onChange={onChange}
/>
{label}
</label>
);
},
Text: ({ children, size, c, style }) => (
<span data-size={size} data-color={c} style={style}>
{children}
@ -1014,17 +1039,29 @@ describe('ChannelForm', () => {
});
});
it('still calls onClose when submission throws', async () => {
vi.mocked(ChannelUtils.addChannel).mockRejectedValue(
new Error('Server error')
);
it('does NOT close the form when submission throws (preserves user edits)', async () => {
// Regression guard: a save failure (validation error from the
// backend, network failure, etc.) must not call reset() + onClose()
// unconditionally; doing so drops the user's in-progress edits. The
// form early-returns on saveFailed and surfaces an error notification
// so the user can correct the error and retry without retyping.
const error = new Error('Server error');
error.body = { detail: 'channel_number must be unique' };
vi.mocked(ChannelUtils.addChannel).mockRejectedValue(error);
setupMocks();
const onClose = vi.fn();
render(<ChannelForm {...defaultProps({ onClose })} />);
fireEvent.click(screen.getByText('Submit'));
await waitFor(() => {
expect(onClose).toHaveBeenCalled();
expect(vi.mocked(ChannelUtils.addChannel)).toHaveBeenCalled();
});
expect(onClose).not.toHaveBeenCalled();
expect(showNotification).toHaveBeenCalledWith(
expect.objectContaining({
color: 'red',
message: 'channel_number must be unique',
})
);
});
});
@ -1080,4 +1117,70 @@ describe('ChannelForm', () => {
expect(epgInput).toHaveValue('EPG Source 1 - ESPN');
});
});
// ── Override reactivity ────────────────────────────────────────────────────
//
// The "Currently overriding" tooltip and "Clear All Overrides (N)" button
// are derived state computed from the form's watched values. The derivation
// must recompute when any watched value changes, otherwise the affordance
// becomes stale and the user loses both the override count and the clear
// action between edits.
describe('override reactivity', () => {
const autoCreatedChannel = () =>
makeChannel({ id: 'ch-auto', auto_created: true });
it('updates "Clear All Overrides" affordance when a watched value diverges from provider', () => {
const channel = autoCreatedChannel();
// Initial render: no field is overridden, so the Clear All button
// must NOT be visible.
vi.mocked(ChannelUtils.isFormFieldOverridden).mockReturnValue(false);
setupMocks({ channel });
const { rerender } = render(
<ChannelForm {...defaultProps()} channel={channel} />
);
expect(screen.queryByText(/Clear All Overrides/)).not.toBeInTheDocument();
// Simulate the user typing into the name field. The mocked watch()
// now returns a different name, AND isFormFieldOverridden now
// reports the name field as overridden. The derived state must
// recompute and surface the Clear All affordance.
vi.mocked(ChannelUtils.isFormFieldOverridden).mockImplementation(
(_ch, field) => field === 'name'
);
setupMocks({
channel,
formOverrides: { watchValues: { name: 'My Override' } },
});
rerender(<ChannelForm {...defaultProps()} channel={channel} />);
expect(screen.getByText(/Clear All Overrides \(1\)/)).toBeInTheDocument();
});
it('hides "Clear All Overrides" when the last divergent field reverts to provider', () => {
const channel = autoCreatedChannel();
vi.mocked(ChannelUtils.isFormFieldOverridden).mockImplementation(
(_ch, field) => field === 'name'
);
setupMocks({
channel,
formOverrides: { watchValues: { name: 'My Override' } },
});
const { rerender } = render(
<ChannelForm {...defaultProps()} channel={channel} />
);
expect(screen.getByText(/Clear All Overrides \(1\)/)).toBeInTheDocument();
// User types the provider value back into the field. No fields are
// overridden anymore, so the affordance must disappear.
vi.mocked(ChannelUtils.isFormFieldOverridden).mockReturnValue(false);
setupMocks({
channel,
formOverrides: { watchValues: { name: 'Test Channel' } },
});
rerender(<ChannelForm {...defaultProps()} channel={channel} />);
expect(screen.queryByText(/Clear All Overrides/)).not.toBeInTheDocument();
});
});
});

121
frontend/src/components/forms/__tests__/ChannelBatch.test.jsx
View file

@ -41,6 +41,7 @@ vi.mock('../../../utils/forms/ChannelBatchUtils.js', () => ({
setChannelNamesFromEpg: vi.fn(),
setChannelTvgIdsFromEpg: vi.fn(),
updateChannels: vi.fn(),
updateChannelsWithOverrideRouting: vi.fn(),
}));
// ── Sub-component mocks ────────────────────────────────────────────────────────
@ -260,6 +261,9 @@ const setupMocks = (overrides = {}) => {
vi.mocked(useChannelsTableStore).mockImplementation((sel) =>
sel({ channels: overrides.pageChannels ?? [] })
);
useChannelsTableStore.getState = vi.fn(() => ({
channels: overrides.pageChannels ?? [],
}));
vi.mocked(useStreamProfilesStore).mockImplementation((sel) =>
sel({ profiles: overrides.profiles ?? makeProfiles() })
@ -304,6 +308,9 @@ const setupMocks = (overrides = {}) => {
});
vi.mocked(ChannelBatchUtils.buildEpgAssociations).mockResolvedValue(null);
vi.mocked(ChannelBatchUtils.updateChannels).mockResolvedValue(undefined);
vi.mocked(
ChannelBatchUtils.updateChannelsWithOverrideRouting
).mockResolvedValue(undefined);
vi.mocked(ChannelBatchUtils.bulkRegexRenameChannels).mockResolvedValue(
undefined
);
@ -434,6 +441,53 @@ describe('ChannelBatchForm', () => {
screen.queryByTestId('confirmation-dialog')
).not.toBeInTheDocument();
});
it('runs clear-overrides BEFORE the routing PATCH so a same-submit clear cannot wipe just-written overrides', async () => {
// If clear and routing fired in parallel via Promise.all, the
// server could process them in either order. When clear lands
// last, the routing PATCH's freshly-written override fields are
// wiped silently. Awaiting clear before routing makes the user
// intent deterministic.
const callOrder = [];
const isWarningSuppressed = vi
.fn()
.mockImplementation((key) => key === 'batch-update-channels');
setupMocks({
isWarningSuppressed,
formValues: { clear_overrides: 'clear' },
});
vi.mocked(ChannelBatchUtils.buildSubmitValues).mockReturnValue({
name: 'Renamed',
});
// clear PATCH resolves slowly; routing PATCH resolves fast. If
// they were fired in parallel, routing would `await` first; the
// sequential code awaits clear first regardless of resolution
// speed.
vi.mocked(ChannelBatchUtils.updateChannels).mockImplementation(
() =>
new Promise((resolve) => {
setTimeout(() => {
callOrder.push('clear');
resolve(undefined);
}, 30);
})
);
vi.mocked(
ChannelBatchUtils.updateChannelsWithOverrideRouting
).mockImplementation(async () => {
callOrder.push('routing');
return undefined;
});
const onClose = vi.fn();
renderForm({ onClose });
fireEvent.click(screen.getByText('Submit'));
await waitFor(() => {
expect(onClose).toHaveBeenCalled();
});
expect(callOrder).toEqual(['clear', 'routing']);
});
});
// ── Confirmation dialog ────────────────────────────────────────────────────
@ -455,7 +509,7 @@ describe('ChannelBatchForm', () => {
).not.toBeInTheDocument();
});
it('calls updateChannels on confirm', async () => {
it('routes confirm through updateChannelsWithOverrideRouting so auto-created channels write to override.X', async () => {
setupMocks({
formValues: {
stream_profile_id: '1', // survives: not '-1' or '0', so kept as-is after parseInt...
@ -474,10 +528,9 @@ describe('ChannelBatchForm', () => {
fireEvent.click(screen.getByTestId('dialog-confirm'));
await waitFor(() => {
expect(ChannelBatchUtils.updateChannels).toHaveBeenCalledWith(
CHANNEL_IDS,
{ stream_profile_id: '1' }
);
expect(
ChannelBatchUtils.updateChannelsWithOverrideRouting
).toHaveBeenCalledWith(CHANNEL_IDS, { stream_profile_id: '1' }, {});
});
});
@ -859,6 +912,64 @@ describe('ChannelBatchForm', () => {
waitFor(() => fireEvent.click(screen.getByTestId('dialog-confirm')))
).resolves.not.toThrow();
});
it('selection summary reflects current channelIds when prop changes', () => {
// Reactivity guard: the selection summary at the top of the form
// must update when the parent passes a different channelIds prop.
// A stale summary misleads the user about how many channels (and
// of which kind) are about to be edited.
setupMocks({
pageChannels: [
{ id: 1, auto_created: true },
{ id: 2, auto_created: false },
{ id: 3, auto_created: true },
{ id: 4, auto_created: false },
],
});
const { rerender } = renderForm({ channelIds: [1, 2] });
// 1 auto + 1 manual.
expect(screen.getByText(/1 auto-synced, 1 manual/)).toBeInTheDocument();
rerender(
<ChannelBatchForm
channelIds={[1, 3, 4]}
isOpen={true}
onClose={vi.fn()}
/>
);
// After prop change: 2 auto + 1 manual.
expect(screen.getByText(/2 auto-synced, 1 manual/)).toBeInTheDocument();
});
it('keeps the form open and surfaces an error notification when batch submit rejects', async () => {
// A bulk PATCH rejection (server validation, network) must not close
// the form silently; the user needs the error message to correct the
// issue and retry without losing the in-progress selection.
setupMocks();
vi.mocked(ChannelBatchUtils.getChannelGroupChange).mockReturnValue(
'• Channel Group: Sports'
);
vi.mocked(
ChannelBatchUtils.updateChannelsWithOverrideRouting
).mockRejectedValue(new Error('Server error'));
const onClose = vi.fn();
renderForm({ onClose });
fireEvent.click(screen.getByText('Submit'));
fireEvent.click(screen.getByTestId('dialog-confirm'));
await waitFor(() => {
expect(
ChannelBatchUtils.updateChannelsWithOverrideRouting
).toHaveBeenCalled();
});
expect(onClose).not.toHaveBeenCalled();
expect(screen.getByTestId('modal')).toBeInTheDocument();
expect(showNotification).toHaveBeenCalledWith(
expect.objectContaining({ color: 'red' })
);
});
});
// ── Batch update confirmation message ──────────────────────────────────────

95
frontend/src/components/forms/__tests__/GroupConfigureModal.test.jsx Normal file
View file

@ -0,0 +1,95 @@
import { render, screen, fireEvent } from '@testing-library/react';
import { describe, it, expect, vi } from 'vitest';
// Contract test for the per-group Configure modal. Covers the footer
// affordances added so users have an unambiguous bottom-of-modal action:
// Done keeps in-memory edits, Cancel triggers the parent's revert path.
// The corner X is gone; Esc / click-outside route the modal's onClose
// to onCancel.
vi.mock('@mantine/core', () => ({
Modal: ({ opened, onClose, withCloseButton, children, title }) =>
opened ? (
<div
data-testid="modal"
data-with-close-button={String(withCloseButton ?? true)}
onClick={(e) => {
if (e.target.dataset.testid === 'modal-overlay') onClose?.();
}}
>
<div data-testid="modal-title">{title}</div>
{children}
</div>
) : null,
Stack: ({ children }) => <div>{children}</div>,
Group: ({ children }) => <div>{children}</div>,
Text: ({ children }) => <span>{children}</span>,
Button: ({ children, onClick }) => (
<button onClick={onClick}>{children}</button>
),
}));
import GroupConfigureModal from '../GroupConfigureModal';
const baseGroup = { channel_group: 1, name: 'Sports', stream_count: 42 };
const renderModal = (overrides = {}) => {
const onDone = vi.fn();
const onCancel = vi.fn();
render(
<GroupConfigureModal
opened
onDone={onDone}
onCancel={onCancel}
group={baseGroup}
{...overrides}
>
<div data-testid="advanced-options">advanced options content</div>
</GroupConfigureModal>
);
return { onDone, onCancel };
};
describe('GroupConfigureModal footer affordances', () => {
it('returns null when no group is provided', () => {
const { container } = render(
<GroupConfigureModal opened group={null}>
child
</GroupConfigureModal>
);
expect(container.firstChild).toBeNull();
});
it('renders Done and Cancel buttons in the footer', () => {
renderModal();
expect(screen.getByText('Done')).toBeInTheDocument();
expect(screen.getByText('Cancel')).toBeInTheDocument();
});
it('removes the corner X close button so the footer is the only action', () => {
renderModal();
expect(screen.getByTestId('modal')).toHaveAttribute(
'data-with-close-button',
'false'
);
});
it('clicking Done calls onDone and not onCancel', () => {
const { onDone, onCancel } = renderModal();
fireEvent.click(screen.getByText('Done'));
expect(onDone).toHaveBeenCalledTimes(1);
expect(onCancel).not.toHaveBeenCalled();
});
it('clicking Cancel calls onCancel and not onDone', () => {
const { onDone, onCancel } = renderModal();
fireEvent.click(screen.getByText('Cancel'));
expect(onCancel).toHaveBeenCalledTimes(1);
expect(onDone).not.toHaveBeenCalled();
});
it('renders the children passed in (advanced options content)', () => {
renderModal();
expect(screen.getByTestId('advanced-options')).toBeInTheDocument();
});
});

251
frontend/src/components/forms/__tests__/LiveGroupFilter.test.jsx Normal file
View file

@ -0,0 +1,251 @@
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
// Targeted Vitest for the orphan-cleanup SegmentedControl that lives at
// the top of the M3U account's group-settings page. Covers default-mode
// resolution, click-to-PATCH, and optimistic-update / error-revert
// behavior. Other parts of LiveGroupFilter (per-group inline config,
// gear modal, regex preview, overlap warning) are not unit-tested here
// because they depend on external APIs and live data; they are exercised
// via the in-repo manual UI walkthrough.
// ── Store mocks ────────────────────────────────────────────────────────────────
vi.mock('../../../store/channels', () => ({ default: vi.fn() }));
vi.mock('../../../store/streamProfiles', () => ({ default: vi.fn() }));
// ── Hook mocks ─────────────────────────────────────────────────────────────────
vi.mock('../../../hooks/useSmartLogos', () => ({
useChannelLogoSelection: vi.fn(() => ({
logos: {},
ensureLogosLoaded: vi.fn(),
isLoading: false,
})),
}));
// ── Module mocks ───────────────────────────────────────────────────────────────
vi.mock('@mantine/notifications', () => ({
notifications: { show: vi.fn() },
}));
vi.mock('../../../api', () => ({
default: {
updatePlaylist: vi.fn(() => Promise.resolve({})),
getEPGs: vi.fn(() => Promise.resolve([])),
getChannelsInRange: vi.fn(() => Promise.resolve({ data: [] })),
getStreamsRegexPreview: vi.fn(() => Promise.resolve({ data: {} })),
repackGroupChannels: vi.fn(() => Promise.resolve({})),
},
}));
vi.mock('../../../utils/forms/GroupSyncUtils', () => ({
getGroupReservation: vi.fn(() => null),
}));
// ── Child component mocks ──────────────────────────────────────────────────────
vi.mock('../GroupConfigureModal', () => ({
default: ({ children }) => <div data-testid="group-config-modal">{children}</div>,
}));
vi.mock('../Logo', () => ({ default: () => null }));
vi.mock('../../LazyLogo', () => ({ default: () => null }));
vi.mock('../../../images/logo.png', () => ({ default: 'default-logo.png' }));
vi.mock('react-window', () => ({
FixedSizeList: ({ children, itemCount }) => (
<div data-testid="fixed-size-list">
{Array.from({ length: itemCount }, (_, index) =>
children({ index, style: {} })
)}
</div>
),
}));
vi.mock('lucide-react', () => ({
Info: () => <svg data-testid="icon-info" />,
CircleCheck: () => <svg data-testid="icon-check" />,
CircleX: () => <svg data-testid="icon-x" />,
Settings: () => <svg data-testid="icon-cog" />,
AlertTriangle: () => <svg data-testid="icon-warn" />,
RefreshCw: () => <svg data-testid="icon-refresh" />,
}));
// ── @mantine/core minimal mocks ────────────────────────────────────────────────
vi.mock('@mantine/core', () => ({
TextInput: ({ value, onChange, placeholder }) => (
<input value={value ?? ''} onChange={onChange} placeholder={placeholder} />
),
Button: ({ children, onClick }) => <button onClick={onClick}>{children}</button>,
Checkbox: ({ label, checked, onChange }) => (
<label>
<input type="checkbox" checked={!!checked} onChange={onChange} />
{label}
</label>
),
Flex: ({ children }) => <div>{children}</div>,
Select: ({ value, onChange, data }) => (
<select value={value ?? ''} onChange={(e) => onChange?.(e.target.value)}>
{(data ?? []).map((opt) => {
const v = typeof opt === 'string' ? opt : opt.value;
const l = typeof opt === 'string' ? opt : opt.label;
return (
<option key={v} value={v}>
{l}
</option>
);
})}
</select>
),
Stack: ({ children }) => <div>{children}</div>,
Group: ({ children }) => <div>{children}</div>,
SimpleGrid: ({ children }) => <div>{children}</div>,
Text: ({ children }) => <span>{children}</span>,
NumberInput: ({ value, onChange }) => (
<input
type="number"
value={value ?? ''}
onChange={(e) => onChange?.(Number(e.target.value))}
/>
),
Divider: ({ label }) => <hr aria-label={label} />,
Alert: ({ children }) => <div role="alert">{children}</div>,
Box: ({ children }) => <div>{children}</div>,
MultiSelect: ({ value, onChange }) => (
<select multiple value={value ?? []} onChange={onChange}>
{(value ?? []).map((v) => (
<option key={v} value={v}>
{v}
</option>
))}
</select>
),
Tooltip: ({ children }) => <>{children}</>,
Popover: ({ children }) => <div>{children}</div>,
ScrollArea: ({ children }) => <div>{children}</div>,
Center: ({ children }) => <div>{children}</div>,
SegmentedControl: ({ value, onChange, data }) => (
<div data-testid="segmented-control" data-value={value}>
{data.map((opt) => (
<button
key={opt.value}
data-testid={`segmented-${opt.value}`}
data-active={value === opt.value}
onClick={() => onChange?.(opt.value)}
>
{opt.label}
</button>
))}
</div>
),
ActionIcon: ({ children, onClick }) => <button onClick={onClick}>{children}</button>,
Switch: ({ label, checked, onChange }) => (
<label>
<input type="checkbox" checked={!!checked} onChange={onChange} />
{label}
</label>
),
}));
// ── Imports after mocks ────────────────────────────────────────────────────────
import LiveGroupFilter from '../LiveGroupFilter';
import API from '../../../api';
import { notifications } from '@mantine/notifications';
import useChannelsStore from '../../../store/channels';
import useStreamProfilesStore from '../../../store/streamProfiles';
// ──────────────────────────────────────────────────────────────────────────────
const makePlaylist = (overrides = {}) => ({
id: 7,
channel_groups: [],
custom_properties: null,
...overrides,
});
const setupStores = () => {
vi.mocked(useChannelsStore).mockImplementation((sel) =>
sel({ channelGroups: {}, profiles: [] })
);
vi.mocked(useStreamProfilesStore).mockImplementation((sel) =>
sel({ profiles: [], fetchProfiles: vi.fn() })
);
};
const renderFilter = (playlistOverrides = {}) =>
render(
<LiveGroupFilter
playlist={makePlaylist(playlistOverrides)}
groupStates={[]}
setGroupStates={vi.fn()}
autoEnableNewGroupsLive={false}
setAutoEnableNewGroupsLive={vi.fn()}
/>
);
// The orphan-cleanup SegmentedControl shares its mocked testid with the
// status-filter SegmentedControl that lives elsewhere in the form, so
// disambiguate by walking from a uniquely-testided child button up to
// its parent SegmentedControl element.
const findCleanupControl = () =>
screen.getByTestId('segmented-always').closest('[data-testid="segmented-control"]');
describe('LiveGroupFilter orphan-cleanup SegmentedControl', () => {
beforeEach(() => {
vi.clearAllMocks();
setupStores();
});
it('defaults to "always" when custom_properties is null', () => {
renderFilter({ custom_properties: null });
expect(findCleanupControl()).toHaveAttribute('data-value', 'always');
});
it('defaults to "always" when the orphan_channel_cleanup key is absent', () => {
renderFilter({ custom_properties: { compact_numbering: true } });
expect(findCleanupControl()).toHaveAttribute('data-value', 'always');
});
it('reads the persisted mode from custom_properties on mount', () => {
renderFilter({
custom_properties: { orphan_channel_cleanup: 'preserve_customized' },
});
expect(findCleanupControl()).toHaveAttribute(
'data-value',
'preserve_customized'
);
});
it('PATCHes the playlist with merged custom_properties on click and updates the displayed value', async () => {
renderFilter({
custom_properties: { compact_numbering: true },
});
fireEvent.click(screen.getByTestId('segmented-never'));
await waitFor(() => {
expect(API.updatePlaylist).toHaveBeenCalledWith({
id: 7,
custom_properties: {
compact_numbering: true,
orphan_channel_cleanup: 'never',
},
});
});
expect(findCleanupControl()).toHaveAttribute('data-value', 'never');
});
it('reverts to the previous mode and surfaces an error toast when the PATCH fails', async () => {
vi.mocked(API.updatePlaylist).mockRejectedValueOnce(
new Error('Server error')
);
renderFilter({
custom_properties: { orphan_channel_cleanup: 'always' },
});
fireEvent.click(screen.getByTestId('segmented-never'));
await waitFor(() => {
expect(notifications.show).toHaveBeenCalledWith(
expect.objectContaining({ color: 'red' })
);
});
expect(findCleanupControl()).toHaveAttribute('data-value', 'always');
});
});

117
frontend/src/components/tables/ChannelsTable.jsx
View file

@ -38,7 +38,10 @@ import {
ArrowUpDown,
ArrowDownWideNarrow,
Search,
EyeOff,
Pencil,
} from 'lucide-react';
import { listOverriddenFields } from '../../utils/forms/ChannelUtils.js';
import {
Box,
TextInput,
@ -241,11 +244,10 @@ const ChannelRowActions = React.memo(
</Box>
);
},
// Custom comparator: only re-render when the actual channel changes.
// The row object is a new TanStack Table reference on each render, but
// row.original.id is stable. Callbacks read fresh data at call time.
(prevProps, nextProps) =>
prevProps.row.original.id === nextProps.row.original.id
// Custom comparator: skip re-render when the channel's data object hasn't
// changed. row.original is stable when the underlying channel hasn't been
// updated; it becomes a new reference when the store replaces that channel.
(prevProps, nextProps) => prevProps.row.original === nextProps.row.original
);
const ChannelsTable = ({ onReady }) => {
@ -326,6 +328,9 @@ const ChannelsTable = ({ onReady }) => {
const [showOnlyStreamlessChannels, setShowOnlyStreamlessChannels] =
useState(false);
const [showOnlyStaleChannels, setShowOnlyStaleChannels] = useState(false);
const [showOnlyOverriddenChannels, setShowOnlyOverriddenChannels] =
useState(false);
const [visibilityFilter, setVisibilityFilter] = useState('active');
const [paginationString, setPaginationString] = useState('');
const [filters, setFilters] = useState({
@ -429,6 +434,14 @@ const ChannelsTable = ({ onReady }) => {
if (showOnlyStaleChannels === true) {
params.append('only_stale', true);
}
if (showOnlyOverriddenChannels === true) {
params.append('only_has_overrides', true);
}
// The backend defaults to "active"; send other choices explicitly so
// hidden rows surface when the user opts into "Hidden Only" or "Show All".
if (visibilityFilter && visibilityFilter !== 'active') {
params.append('visibility_filter', visibilityFilter);
}
// Apply sorting
if (sorting.length > 0) {
@ -525,6 +538,8 @@ const ChannelsTable = ({ onReady }) => {
selectedProfileId,
showOnlyStreamlessChannels,
showOnlyStaleChannels,
showOnlyOverriddenChannels,
visibilityFilter,
]);
const stopPropagation = useCallback((e) => {
@ -559,36 +574,30 @@ const ChannelsTable = ({ onReady }) => {
}));
};
const editChannel = async (ch = null, opts = {}) => {
// If forceAdd is set, always open a blank form
const editChannel = useCallback(async (ch = null, opts = {}) => {
if (opts.forceAdd) {
setChannel(null);
setChannelModalOpen(true);
return;
}
// Use table's selected state instead of store state to avoid stale selections
const currentSelection = table ? table.selectedTableIds : [];
console.log('editChannel called with:', {
ch,
currentSelection,
tableExists: !!table,
});
const currentSelection =
useChannelsTableStore.getState().selectedChannelIds;
console.log('editChannel called with:', { ch, currentSelection });
if (currentSelection.length > 1) {
setChannelBatchModalOpen(true);
} else {
// If no channel object is passed but we have a selection, get the selected channel
let channelToEdit = ch;
if (!channelToEdit && currentSelection.length === 1) {
const selectedId = currentSelection[0];
// Use table data since that's what's currently displayed
channelToEdit = data.find((d) => d.id === selectedId);
channelToEdit = useChannelsTableStore
.getState()
.channels.find((d) => d.id === selectedId);
}
setChannel(channelToEdit);
setChannelModalOpen(true);
}
};
}, []);
const deleteChannel = async (id) => {
console.log(`Deleting channel with ID: ${id}`);
@ -914,7 +923,10 @@ const ChannelsTable = ({ onReady }) => {
},
{
id: 'channel_number',
accessorKey: 'channel_number',
// Prefer the backend-resolved effective_channel_number so overrides
// show through to the table. Inline save still writes to the
// override row via buildInlinePatch in EditableCell.
accessorFn: (row) => row.effective_channel_number ?? row.channel_number,
size: columnSizing.channel_number || 40,
minSize: 30,
maxSize: 100,
@ -922,16 +934,53 @@ const ChannelsTable = ({ onReady }) => {
},
{
id: 'name',
accessorKey: 'name',
accessorFn: (row) => row.effective_name ?? row.name,
size: columnSizing.name || 200,
minSize: 100,
grow: true,
cell: (props) => <EditableTextCell {...props} />,
cell: (props) => {
const row = props.row?.original || {};
const overriddenLabels = listOverriddenFields(row);
return (
<Flex align="center" gap={6} style={{ minWidth: 0 }}>
<Box style={{ minWidth: 0, flex: 1 }}>
<EditableTextCell {...props} />
</Box>
{overriddenLabels.length > 0 && (
<Tooltip
label={`Overrides active: ${overriddenLabels.join(', ')}`}
>
<Box
component="span"
role="img"
aria-label={`Overrides active: ${overriddenLabels.join(', ')}`}
style={{ display: 'inline-flex' }}
>
<Pencil size={14} color="#eab308" aria-hidden="true" />
</Box>
</Tooltip>
)}
{row.hidden_from_output && (
<Tooltip label="Hidden from HDHR, M3U, EPG, and XC output.">
<Box
component="span"
role="img"
aria-label="Hidden from HDHR, M3U, EPG, and XC output"
style={{ display: 'inline-flex' }}
>
<EyeOff size={14} color="#9ca3af" aria-hidden="true" />
</Box>
</Tooltip>
)}
</Flex>
);
},
},
{
id: 'epg',
header: 'EPG',
accessorKey: 'epg_data_id',
// Effective EPG id so overridden EPG assignments show in the table.
accessorFn: (row) => row.effective_epg_data_id ?? row.epg_data_id,
cell: (props) => (
<EditableEPGCell
{...props}
@ -945,10 +994,13 @@ const ChannelsTable = ({ onReady }) => {
},
{
id: 'channel_group',
accessorFn: (row) =>
channelGroups[row.channel_group_id]
? channelGroups[row.channel_group_id].name
: '',
accessorFn: (row) => {
const effectiveGroupId =
row.effective_channel_group_id ?? row.channel_group_id;
return channelGroups[effectiveGroupId]
? channelGroups[effectiveGroupId].name
: '';
},
cell: (props) => (
<EditableGroupCell {...props} channelGroups={channelGroups} />
),
@ -957,10 +1009,7 @@ const ChannelsTable = ({ onReady }) => {
},
{
id: 'logo',
accessorFn: (row) => {
// Just pass the logo_id directly, not the full logo object
return row.logo_id;
},
accessorFn: (row) => row.effective_logo_id ?? row.logo_id,
size: 75,
minSize: 50,
maxSize: 120,
@ -1001,7 +1050,7 @@ const ChannelsTable = ({ onReady }) => {
// from the store, so we don't need to recreate columns when logos load.
// Note: tvgsLoaded is intentionally excluded - EditableEPGCell handles loading state internally
// eslint-disable-next-line react-hooks/exhaustive-deps
[selectedProfileId, channelGroups, theme, tvgsById, epgs]
[selectedProfileId, channelGroups, theme, tvgsById, epgs, editChannel]
);
const renderHeaderCell = (header) => {
@ -1515,6 +1564,10 @@ const ChannelsTable = ({ onReady }) => {
setShowOnlyStreamlessChannels={setShowOnlyStreamlessChannels}
showOnlyStaleChannels={showOnlyStaleChannels}
setShowOnlyStaleChannels={setShowOnlyStaleChannels}
showOnlyOverriddenChannels={showOnlyOverriddenChannels}
setShowOnlyOverriddenChannels={setShowOnlyOverriddenChannels}
visibilityFilter={visibilityFilter}
setVisibilityFilter={setVisibilityFilter}
/>
{/* Table or ghost empty state inside Paper */}

56
frontend/src/components/tables/ChannelsTable/ChannelTableHeader.jsx
View file

@ -118,6 +118,10 @@ const ChannelTableHeader = ({
setShowOnlyStreamlessChannels,
showOnlyStaleChannels,
setShowOnlyStaleChannels,
showOnlyOverriddenChannels,
setShowOnlyOverriddenChannels,
visibilityFilter,
setVisibilityFilter,
}) => {
const theme = useMantineTheme();
@ -237,6 +241,12 @@ const ChannelTableHeader = ({
}
};
const toggleShowOnlyOverriddenChannels = () => {
if (setShowOnlyOverriddenChannels) {
setShowOnlyOverriddenChannels(!showOnlyOverriddenChannels);
}
};
const toggleHeaderPinned = () => {
setHeaderPinned(!headerPinned);
};
@ -327,11 +337,55 @@ const ChannelTableHeader = ({
<Menu.Item
onClick={toggleShowOnlyStaleChannels}
leftSection={
showOnlyStaleChannels ? <SquareCheck size={18} /> : <Square size={18} />
showOnlyStaleChannels ? (
<SquareCheck size={18} />
) : (
<Square size={18} />
)
}
>
<Text size="xs">Has Stale Streams</Text>
</Menu.Item>
<Menu.Item
onClick={toggleShowOnlyOverriddenChannels}
leftSection={
showOnlyOverriddenChannels ? (
<SquareCheck size={18} />
) : (
<Square size={18} />
)
}
>
<Text size="xs">Has Overrides</Text>
</Menu.Item>
<Menu.Divider />
<Menu.Label>
<Text size="xs">Visibility</Text>
</Menu.Label>
{[
{ value: 'active', label: 'Active Only' },
{ value: 'hidden', label: 'Hidden Only' },
{ value: 'all', label: 'Show All' },
].map(({ value, label }) => (
<Menu.Item
key={value}
onClick={() =>
setVisibilityFilter && setVisibilityFilter(value)
}
leftSection={
visibilityFilter === value ? (
<SquareCheck size={18} />
) : (
<Square size={18} />
)
}
>
<Text size="xs">{label}</Text>
</Menu.Item>
))}
</Menu.Dropdown>
</Menu>

117
frontend/src/components/tables/ChannelsTable/EditableCell.jsx
View file

@ -18,6 +18,54 @@ import {
import API from '../../../api';
import useChannelsTableStore from '../../../store/channelsTable';
import useLogosStore from '../../../store/logos';
import {
OVERRIDABLE_FIELDS,
normalizeFieldValue,
} from '../../../utils/forms/ChannelUtils.js';
import { showNotification } from '../../../utils/notificationUtils.js';
// Surfaces server-side validation failures so the user knows the
// inline edit was rejected (otherwise the cell silently reverts).
// Exported so unit tests can verify the message composition without
// mounting the component.
export const notifyInlineSaveError = (columnId, error) => {
const detail =
error?.body?.detail ||
error?.body?.[columnId]?.[0] ||
error?.body?.error ||
error?.message ||
'Server rejected the change';
showNotification({
title: 'Edit not saved',
message: String(detail),
color: 'red',
autoClose: 5000,
});
};
// Inline edits on auto-synced channels route into the override row so
// sync cannot overwrite them. If the new value matches the provider's,
// clear that field's override instead of writing a duplicate. Manual
// channels keep direct Channel.* writes.
const buildInlinePatch = (rowOriginal, fieldId, newValue) => {
if (rowOriginal?.auto_created && OVERRIDABLE_FIELDS.includes(fieldId)) {
// Normalize both sides so a stringified form value compares
// cleanly against the typed provider value.
const formValue = normalizeFieldValue(fieldId, newValue);
const providerValue = normalizeFieldValue(fieldId, rowOriginal[fieldId]);
const overrideFieldValue = formValue === providerValue ? null : formValue;
return {
id: rowOriginal.id,
override: { [fieldId]: overrideFieldValue },
};
}
const normalized =
newValue === undefined || newValue === '' ? null : newValue;
return {
id: rowOriginal.id,
[fieldId]: normalized,
};
};
// Lightweight wrapper that only renders full editable cell when unlocked
// This prevents 250+ heavy component instances when table is locked
@ -103,10 +151,9 @@ const EditableTextCellInner = ({ row, column, getValue, onBlur }) => {
}
try {
const response = await API.updateChannel({
id: row.original.id,
[column.id]: newValue || null,
});
const response = await API.updateChannel(
buildInlinePatch(row.original, column.id, newValue)
);
previousValue.current = newValue;
// Update the table store to reflect the change
@ -114,11 +161,13 @@ const EditableTextCellInner = ({ row, column, getValue, onBlur }) => {
useChannelsTableStore.getState().updateChannel(response);
}
} catch (error) {
// Revert on error
// Surface server-side errors (e.g. max_length=512, validator
// rejection) so the user knows the change was not saved.
notifyInlineSaveError(column.id, error);
setValue(previousValue.current || '');
}
},
[row.original.id, column.id]
[row.original, column.id]
);
useEffect(() => {
@ -249,10 +298,9 @@ const EditableNumberCellInner = ({ row, column, getValue, onBlur }) => {
}
try {
const response = await API.updateChannel({
id: row.original.id,
[column.id]: newValue,
});
const response = await API.updateChannel(
buildInlinePatch(row.original, column.id, newValue)
);
previousValue.current = newValue;
// Update the table store to reflect the change
@ -266,11 +314,13 @@ const EditableNumberCellInner = ({ row, column, getValue, onBlur }) => {
}
}
} catch (error) {
// Revert on error
// Surface server-side errors (channel_number out-of-range,
// collision, etc.) so the user knows the change was not saved.
notifyInlineSaveError(column.id, error);
setValue(previousValue.current);
}
},
[row.original.id, column.id, onBlur]
[row.original, column.id, onBlur]
);
useEffect(() => {
@ -366,10 +416,13 @@ const EditableGroupCellInner = ({
}
try {
const response = await API.updateChannel({
id: row.original.id,
channel_group_id: parseInt(newGroupId, 10),
});
const response = await API.updateChannel(
buildInlinePatch(
row.original,
'channel_group_id',
parseInt(newGroupId, 10)
)
);
previousGroupId.current = newGroupId;
// Update the table store to reflect the change
@ -378,9 +431,10 @@ const EditableGroupCellInner = ({
}
} catch (error) {
console.error('Failed to update channel group:', error);
notifyInlineSaveError(column.id, error);
}
},
[row.original.id]
[row.original]
);
const handleChange = (newGroupId) => {
@ -537,11 +591,13 @@ const EditableEPGCellInner = ({
}
try {
const response = await API.updateChannel({
id: row.original.id,
epg_data_id:
newEpgDataId === 'null' ? null : parseInt(newEpgDataId, 10),
});
const response = await API.updateChannel(
buildInlinePatch(
row.original,
'epg_data_id',
newEpgDataId === 'null' ? null : parseInt(newEpgDataId, 10)
)
);
previousEpgDataId.current = newEpgDataId;
// Update the table store to reflect the change
@ -550,9 +606,10 @@ const EditableEPGCellInner = ({
}
} catch (error) {
console.error('Failed to update EPG:', error);
notifyInlineSaveError(column.id, error);
}
},
[row.original.id]
[row.original]
);
const handleChange = (newEpgDataId) => {
@ -704,10 +761,13 @@ const EditableLogoCellInner = ({ row, logoId, onBlur }) => {
}
try {
const response = await API.updateChannel({
id: row.original.id,
logo_id: newLogoId === 'null' ? null : parseInt(newLogoId, 10),
});
const response = await API.updateChannel(
buildInlinePatch(
row.original,
'logo_id',
newLogoId === 'null' ? null : parseInt(newLogoId, 10)
)
);
previousLogoId.current = newLogoId;
// Update the table store to reflect the change
@ -716,9 +776,10 @@ const EditableLogoCellInner = ({ row, logoId, onBlur }) => {
}
} catch (error) {
console.error('Failed to update logo:', error);
notifyInlineSaveError(column.id, error);
}
},
[row.original.id]
[row.original]
);
const handleChange = (newLogoId) => {

82
frontend/src/components/tables/ChannelsTable/__tests__/EditableCell.test.jsx Normal file
View file

@ -0,0 +1,82 @@
// @vitest-environment jsdom
import { describe, it, expect, beforeEach, vi } from 'vitest';
// Mock the notification utility before importing the module under test
// so the helper picks up the mock.
vi.mock('../../../../utils/notificationUtils.js', () => ({
showNotification: vi.fn(),
}));
// Mock other heavy dependencies so the import doesn't pull in stores.
vi.mock('../../../../api', () => ({ default: {} }));
vi.mock('../../../../store/channelsTable', () => ({ default: { getState: vi.fn() } }));
vi.mock('../../../../store/logos', () => ({ default: vi.fn() }));
vi.mock('../../../../utils/forms/ChannelUtils.js', () => ({
OVERRIDABLE_FIELDS: new Set(['name', 'channel_number', 'tvg_id']),
normalizeFieldValue: (v) => v,
}));
import { notifyInlineSaveError } from '../EditableCell.jsx';
import { showNotification } from '../../../../utils/notificationUtils.js';
describe('EditableCell.notifyInlineSaveError (F-12 regression guard)', () => {
beforeEach(() => {
vi.clearAllMocks();
});
it('surfaces a red notification when an inline save fails (the fix)', () => {
// Regression guard: the previous implementation silently reverted
// the field on save error with no user feedback. The user could
// see their input vanish without knowing why. This helper must
// produce a visible error notification.
const error = new Error('Bad request');
notifyInlineSaveError('name', error);
expect(showNotification).toHaveBeenCalledTimes(1);
const arg = showNotification.mock.calls[0][0];
expect(arg.color).toBe('red');
expect(arg.title).toMatch(/not saved|error/i);
expect(arg.message).toBeTruthy();
});
it('extracts a per-field validator message from a structured DRF response', () => {
// DRF serializer errors come back as { field: ["error string"] }.
// The helper must dig into the field-keyed array and surface the
// first message rather than rendering "[object Object]" or
// generic fallback.
const apiError = {
message: 'Request failed',
body: {
channel_number: ['Channel number 0 is below the allowed minimum.'],
},
};
notifyInlineSaveError('channel_number', apiError);
const arg = showNotification.mock.calls[0][0];
expect(arg.message).toContain('below the allowed minimum');
});
it('extracts a top-level "detail" key when DRF emits a non-field error', () => {
const apiError = {
message: 'Request failed',
body: { detail: 'Channel name exceeds max_length=512' },
};
notifyInlineSaveError('name', apiError);
const arg = showNotification.mock.calls[0][0];
expect(arg.message).toContain('max_length');
});
it('falls back to the error.message when no body shape is available', () => {
notifyInlineSaveError('name', new Error('Network error'));
const arg = showNotification.mock.calls[0][0];
expect(arg.message).toContain('Network error');
});
it('uses a generic fallback when nothing useful is on the error', () => {
notifyInlineSaveError('name', {});
const arg = showNotification.mock.calls[0][0];
expect(arg.message).toBeTruthy();
expect(typeof arg.message).toBe('string');
});
});

75
frontend/src/components/tables/M3UsTable.jsx
View file

@ -135,6 +135,12 @@ const M3UTable = () => {
const [confirmDeleteOpen, setConfirmDeleteOpen] = useState(false);
const [deleteTarget, setDeleteTarget] = useState(null);
const [playlistToDelete, setPlaylistToDelete] = useState(null);
// Auto-created channel preview shown in the delete confirmation so the
// user sees what cascades along with the account.
const [autoChannelsInfo, setAutoChannelsInfo] = useState({
count: 0,
sample_names: [],
});
const [data, setData] = useState([]);
const [sorting, setSorting] = useState([{ id: 'name', desc: '' }]);
const [deleting, setDeleting] = useState(false);
@ -404,8 +410,32 @@ const M3UTable = () => {
setPlaylistToDelete(playlist);
setDeleteTarget(id);
// Skip warning if it's been suppressed
if (isWarningSuppressed('delete-m3u')) {
// Fetch how many auto-created channels this playlist owns. Populates the
// confirmation message so the user can decide whether to also delete
// them. On failure, surface "unknown" so the user is not misled into
// thinking there are zero auto-created channels.
let info;
try {
const result = await API.getPlaylistAutoCreatedChannelsCount(id);
info = result || { count: 0, sample_names: [] };
} catch {
info = {
count: null,
sample_names: [],
countUnavailable: true,
};
}
setAutoChannelsInfo(info);
// Skip the warning when it has been suppressed AND the account has
// no auto-created channels. When the account did create channels (or
// the count could not be resolved), the dialog still opens so the
// user sees and confirms what cascades.
if (
isWarningSuppressed('delete-m3u') &&
info.count === 0 &&
!info.countUnavailable
) {
return executeDeletePlaylist(id);
}
@ -421,6 +451,7 @@ const M3UTable = () => {
setDeleting(false);
setIsLoading(false);
setConfirmDeleteOpen(false);
setAutoChannelsInfo({ count: 0, sample_names: [] });
}
};
@ -1028,15 +1059,49 @@ const M3UTable = () => {
title="Confirm M3U Account Deletion"
message={
playlistToDelete ? (
<div style={{ whiteSpace: 'pre-line' }}>
{`Are you sure you want to delete the following M3U account?
<div>
<div style={{ whiteSpace: 'pre-line', marginBottom: 12 }}>
{`Delete the following M3U account?
Name: ${playlistToDelete.name}
Type: ${playlistToDelete.account_type === 'XC' ? 'Xtream Codes' : 'Standard'}
Server: ${playlistToDelete.server_url || 'Local file'}
This will remove all related streams and may affect channels using these streams.
Streams owned by this provider will be removed. Manual channels that include those streams will lose them, but the channels and any other streams on them survive.
This action cannot be undone.`}
</div>
{autoChannelsInfo.countUnavailable ? (
<div
style={{
background: 'rgba(234,179,8,0.08)',
border: '1px solid rgba(234,179,8,0.3)',
borderRadius: 4,
padding: 10,
marginTop: 6,
}}
>
<Text size="sm" fw={600}>
Auto-synced channel count is unavailable; any channels
auto-created by this provider will be deleted with the
account.
</Text>
</div>
) : autoChannelsInfo.count > 0 ? (
<div
style={{
background: 'rgba(234,179,8,0.08)',
border: '1px solid rgba(234,179,8,0.3)',
borderRadius: 4,
padding: 10,
marginTop: 6,
}}
>
<Text size="sm" fw={600}>
{`${autoChannelsInfo.count} auto-synced channel${autoChannelsInfo.count === 1 ? '' : 's'} created by this provider will also be deleted.`}
</Text>
</div>
) : null}
</div>
) : (
'Are you sure you want to delete this M3U account? This action cannot be undone.'

50
frontend/src/utils/forms/ChannelBatchUtils.js
View file

@ -49,6 +49,45 @@ export const updateChannels = (channelIds, values) => {
return API.updateChannels(channelIds, values);
};
// Auto-created channels route override-able fields to override.* so they
// survive the next sync; manual channels write to the raw Channel columns.
// Non-overridable fields (status/permission flags) always write raw.
export const updateChannelsWithOverrideRouting = async (
channelIds,
values,
channelsById
) => {
const { OVERRIDABLE_FIELDS } = await import('./ChannelUtils.js');
const overrideKeys = new Set(OVERRIDABLE_FIELDS);
const rawValues = {};
const overrideValues = {};
for (const [key, val] of Object.entries(values)) {
if (overrideKeys.has(key)) {
overrideValues[key] = val;
} else {
rawValues[key] = val;
}
}
const body = [];
for (const id of channelIds) {
const channel = channelsById?.[id];
const isAuto = !!channel?.auto_created;
const item = { id, ...rawValues };
if (Object.keys(overrideValues).length > 0) {
if (isAuto) {
item.override = { ...overrideValues };
} else {
Object.assign(item, overrideValues);
}
}
body.push(item);
}
return API.bulkUpdateChannels(body);
};
export const bulkRegexRenameChannels = (
channelIds,
regexFind,
@ -152,6 +191,17 @@ export const buildSubmitValues = (
values.is_adult = values.is_adult === 'true';
}
if (values.hidden_from_output === '-1' || values.hidden_from_output === undefined) {
delete values.hidden_from_output;
} else {
values.hidden_from_output = values.hidden_from_output === 'true';
}
// clear_overrides is a UI-only flag; the caller splits it out and routes a
// follow-up PATCH to just the auto-created subset. Strip it from the main
// PATCH body.
delete values.clear_overrides;
return values;
};

215
frontend/src/utils/forms/ChannelUtils.js
View file

@ -1,5 +1,30 @@
import API from '../../api.js';
// Fields that users can override on an auto-synced channel. Mirrors
// `OVERRIDABLE_FIELDS` in `apps/channels/managers.py`. Keep the two in sync.
export const OVERRIDABLE_FIELDS = [
'name',
'channel_number',
'channel_group_id',
'logo_id',
'tvg_id',
'tvc_guide_stationid',
'epg_data_id',
'stream_profile_id',
];
// Display labels for the override fields above.
export const OVERRIDE_FIELD_LABELS = {
name: 'Name',
channel_number: 'Channel Number',
channel_group_id: 'Channel Group',
logo_id: 'Logo',
tvg_id: 'TVG-ID',
tvc_guide_stationid: 'Gracenote Station ID',
epg_data_id: 'EPG',
stream_profile_id: 'Stream Profile',
};
export const matchChannelEpg = (channel) => {
return API.matchChannelEpg(channel.id);
};
@ -19,27 +44,165 @@ export const requeryChannels = () => {
API.requeryChannels();
};
// PATCH semantic: `override: null` deletes the override row; per-field
// nulls only clear the matching field.
export const clearChannelOverrides = (channelId) => {
return API.updateChannel({ id: channelId, override: null });
};
// Coerce a form value to the backend's storage shape so equality
// comparisons against channel data don't drift on type mismatches.
export const normalizeFieldValue = (field, value) => {
if (value === '' || value === null || value === undefined || value === '-1') {
return null;
}
// The stream_profile and logo pickers encode "(use default)" as '0',
// which is semantically a null FK on the Channel row.
if ((field === 'stream_profile_id' || field === 'logo_id') && value === '0') {
return null;
}
if (field === 'channel_number') {
const n = parseFloat(value);
return Number.isFinite(n) ? n : null;
}
if (
field === 'channel_group_id' ||
field === 'logo_id' ||
field === 'epg_data_id' ||
field === 'stream_profile_id'
) {
const n = parseInt(value, 10);
return Number.isFinite(n) ? n : null;
}
return value;
};
// Per-field form shape for FK pickers (string for popovers, raw for
// the EPG select); used by the reset-to-provider affordance.
const PROVIDER_FORM_VALUE_BUILDERS = {
channel_group_id: (channel) =>
channel?.channel_group_id != null ? `${channel.channel_group_id}` : '',
stream_profile_id: (channel) =>
channel?.stream_profile_id != null ? `${channel.stream_profile_id}` : '0',
logo_id: (channel) => (channel?.logo_id != null ? `${channel.logo_id}` : ''),
epg_data_id: (channel) => channel?.epg_data_id ?? '',
};
export const getProviderFormValue = (channel, field) => {
const builder = PROVIDER_FORM_VALUE_BUILDERS[field];
if (builder) return builder(channel);
return channel?.[field] ?? '';
};
// Form value differs from the channel's provider value. Manual
// channels always return false (no provider value to compare to).
export const isFormFieldOverridden = (channel, field, formValue) => {
if (!channel?.auto_created) return false;
const normalizedForm = normalizeFieldValue(field, formValue);
const normalizedProvider = normalizeFieldValue(field, channel[field]);
return normalizedForm !== normalizedProvider;
};
// Human labels for the table's overrides indicator tooltip.
export const listOverriddenFields = (channel) => {
if (!channel?.override) return [];
return OVERRIDABLE_FIELDS.filter((field) => {
const value = channel.override[field];
return value !== null && value !== undefined;
}).map((field) => OVERRIDE_FIELD_LABELS[field]);
};
// "Provider: <value>" subtext for auto-synced channels (null for manual).
export const getProviderHint = (channel, field) => {
if (!channel?.auto_created) return null;
const providerValue = channel[field];
const display =
providerValue === null ||
providerValue === undefined ||
providerValue === ''
? '(empty)'
: providerValue;
return `Provider: ${display}`;
};
// FK provider hint that resolves the ID to a display name via lookup.
export const getFkProviderHint = (channel, field, lookup) => {
if (!channel?.auto_created) return null;
const providerId = channel[field];
if (providerId === null || providerId === undefined) {
return 'Provider: (none)';
}
const entry = lookup?.[providerId];
const display = entry?.name || entry?.tvg_id || String(providerId);
return `Provider: ${display}`;
};
// Build the override PATCH payload by diffing form values against
// provider values. Matching fields become null (clear); diverging
// fields carry the form value.
export const buildOverridePayload = (channel, formattedValues) => {
if (!channel) return undefined;
const payload = {};
let anyOverride = false;
for (const field of OVERRIDABLE_FIELDS) {
const formValue = normalizeFieldValue(field, formattedValues[field]);
const providerValue = normalizeFieldValue(field, channel[field]);
if (formValue === null && providerValue === null) continue;
if (formValue !== providerValue) {
payload[field] = formValue;
if (formValue !== null) anyOverride = true;
} else {
payload[field] = null;
}
}
if (!anyOverride) {
// Every field matches provider; explicit null deletes the row.
return null;
}
return payload;
};
// Prefer the backend-resolved effective_* so the form loads with the
// overridden value; fall back to the raw field otherwise.
const effective = (channel, field) => {
if (!channel) return undefined;
const effKey = `effective_${field}`;
if (effKey in channel && channel[effKey] !== undefined) {
return channel[effKey];
}
return channel[field];
};
export const getChannelFormDefaultValues = (channel, channelGroups) => {
const name = effective(channel, 'name') ?? '';
const channelNumber = effective(channel, 'channel_number');
const groupId = effective(channel, 'channel_group_id');
const streamProfileId = effective(channel, 'stream_profile_id');
const tvgId = effective(channel, 'tvg_id');
const gracenoteId = effective(channel, 'tvc_guide_stationid');
const epgDataId = effective(channel, 'epg_data_id');
const logoId = effective(channel, 'logo_id');
return {
name: channel?.name || '',
name: name || '',
channel_number:
channel?.channel_number !== null && channel?.channel_number !== undefined
? channel.channel_number
channelNumber !== null && channelNumber !== undefined
? channelNumber
: '',
channel_group_id: channel?.channel_group_id
? `${channel.channel_group_id}`
channel_group_id: groupId
? `${groupId}`
: Object.keys(channelGroups).length > 0
? Object.keys(channelGroups)[0]
: '',
stream_profile_id: channel?.stream_profile_id
? `${channel.stream_profile_id}`
: '0',
tvg_id: channel?.tvg_id || '',
tvc_guide_stationid: channel?.tvc_guide_stationid || '',
epg_data_id: channel?.epg_data_id ?? '',
logo_id: channel?.logo_id ? `${channel.logo_id}` : '',
stream_profile_id: streamProfileId ? `${streamProfileId}` : '0',
tvg_id: tvgId || '',
tvc_guide_stationid: gracenoteId || '',
epg_data_id: epgDataId ?? '',
logo_id: logoId ? `${logoId}` : '',
user_level: `${channel?.user_level ?? '0'}`,
is_adult: channel?.is_adult ?? false,
hidden_from_output: channel?.hidden_from_output ?? false,
};
};
@ -70,15 +233,32 @@ export const handleEpgUpdate = async (
formattedValues,
channelStreams
) => {
// If there's an EPG to set, use our enhanced endpoint
// Auto-synced channels route identity edits into the override row. Sync
// keeps writing provider values to Channel.* unmodified, so the override
// is what actually persists user changes across refreshes. `hidden_from_output`
// stays as a direct Channel field even for auto-created channels because
// it is a status flag, not a value replacement.
if (channel.auto_created) {
const overridePayload = buildOverridePayload(channel, formattedValues);
const payload = {
id: channel.id,
hidden_from_output: formattedValues.hidden_from_output,
};
if (overridePayload !== undefined) {
payload.override = overridePayload;
}
await updateChannel(payload);
return;
}
// Manual channels: existing behavior preserved. When the EPG has changed,
// the dedicated set-EPG endpoint triggers an EPG refresh; other field
// updates go through the regular PATCH and are skipped entirely when
// there is nothing besides epg_data_id to update.
if (values.epg_data_id !== (channel.epg_data_id ?? '')) {
// Use the special endpoint to set EPG and trigger refresh
await setChannelEPG(channel, values);
// Remove epg_data_id from values since we've handled it separately
const { epg_data_id: _epg_data_id, ...otherValues } = formattedValues;
// Update other channel fields if needed
if (Object.keys(otherValues).length > 0) {
await updateChannel({
id: channel.id,
@ -87,7 +267,6 @@ export const handleEpgUpdate = async (
});
}
} else {
// No EPG change, regular update
await updateChannel({
id: channel.id,
...formattedValues,

70
frontend/src/utils/forms/GroupSyncUtils.js Normal file
View file

@ -0,0 +1,70 @@
// Utilities for per-group auto-sync configuration in LiveGroupFilter /
// M3UGroupFilter. Range reservations (groups with both start and end set
// on Fixed or Provider numbering) claim channel numbers exclusively, so
// overlapping reservations across groups would produce conflicts during
// sync. Validation runs client-side at save-time to block obvious
// misconfigurations before the request is sent; the backend enforces
// the same rule across accounts as a safety net.
const MODE_WITHOUT_RANGE = 'next_available';
// Returns null if the group does not participate in reservations (disabled,
// not auto-syncing, unbounded, or in Next Available mode), otherwise a
// [start, end] pair of integers with start <= end.
export const getGroupReservation = (group) => {
if (!group || !group.enabled || !group.auto_channel_sync) return null;
const mode = group.custom_properties?.channel_numbering_mode || 'fixed';
if (mode === MODE_WITHOUT_RANGE) return null;
const endRaw = group.auto_sync_channel_end;
if (endRaw === null || endRaw === undefined || endRaw === '') return null;
const startRaw =
mode === 'provider'
? (group.custom_properties?.channel_numbering_fallback ?? 1)
: (group.auto_sync_channel_start ?? 1);
const start = Math.floor(Number(startRaw));
const end = Math.floor(Number(endRaw));
if (!Number.isFinite(start) || !Number.isFinite(end)) return null;
if (end < start) return null;
return [start, end];
};
// Given an array of groupStates from the M3U Group Filter modal, returns
// overlap records describing pairs of groups whose reservations intersect.
// Shape: [{ a: { channel_group, name, start, end }, b: {...} }, ...]
// Callers surface these as user-facing errors and block submit.
export const detectGroupReservationOverlaps = (groupStates) => {
const reservations = [];
for (const g of groupStates || []) {
const range = getGroupReservation(g);
if (!range) continue;
reservations.push({
channel_group: g.channel_group,
name: g.name,
start: range[0],
end: range[1],
});
}
const conflicts = [];
for (let i = 0; i < reservations.length; i += 1) {
for (let j = i + 1; j < reservations.length; j += 1) {
const a = reservations[i];
const b = reservations[j];
if (a.start <= b.end && b.start <= a.end) {
conflicts.push({ a, b });
}
}
}
return conflicts;
};
// Human-readable summary of an overlap list for a notification body.
export const formatOverlapMessage = (conflicts) => {
if (!conflicts || conflicts.length === 0) return '';
const lines = conflicts.slice(0, 5).map(({ a, b }) => {
return `"${a.name}" [${a.start}-${a.end}] overlaps "${b.name}" [${b.start}-${b.end}]`;
});
if (conflicts.length > 5) {
lines.push(`... and ${conflicts.length - 5} more.`);
}
return lines.join('\n');
};

105
frontend/src/utils/forms/__tests__/ChannelBatchUtils.bulkOverrideRouting.test.js Normal file
View file

@ -0,0 +1,105 @@
// @vitest-environment jsdom
import { describe, it, expect, beforeEach, vi } from 'vitest';
// Mock API.bulkUpdateChannels so we can assert on the body shape the
// helper sends. We're testing routing logic, not network behavior.
vi.mock('../../../api.js', () => ({
default: {
bulkUpdateChannels: vi.fn(async (body) => ({ ok: true, body })),
updateChannels: vi.fn(),
},
}));
import { updateChannelsWithOverrideRouting } from '../ChannelBatchUtils.js';
import API from '../../../api.js';
describe('updateChannelsWithOverrideRouting (manual finding #4)', () => {
beforeEach(() => {
vi.clearAllMocks();
});
it('routes override-able fields to override.X for auto-created channels', async () => {
// User-reported: bulk-edit "name" on auto-channels lost the change
// on next sync because it wrote Channel.name (raw) instead of
// override.name. The Pencil indicator never appeared because no
// override row was created.
const channelsById = {
1: { id: 1, auto_created: true },
2: { id: 2, auto_created: true },
};
await updateChannelsWithOverrideRouting(
[1, 2],
{ name: 'BulkRename' },
channelsById,
);
expect(API.bulkUpdateChannels).toHaveBeenCalledTimes(1);
const body = API.bulkUpdateChannels.mock.calls[0][0];
expect(body).toEqual([
{ id: 1, override: { name: 'BulkRename' } },
{ id: 2, override: { name: 'BulkRename' } },
]);
});
it('keeps raw Channel.X writes for manual channels (auto_created=false)', async () => {
const channelsById = {
1: { id: 1, auto_created: false },
};
await updateChannelsWithOverrideRouting(
[1],
{ name: 'ManualRename' },
channelsById,
);
const body = API.bulkUpdateChannels.mock.calls[0][0];
expect(body).toEqual([{ id: 1, name: 'ManualRename' }]);
});
it('splits a mixed selection: auto-created → override, manual → raw', async () => {
const channelsById = {
1: { id: 1, auto_created: true },
2: { id: 2, auto_created: false },
3: { id: 3, auto_created: true },
};
await updateChannelsWithOverrideRouting(
[1, 2, 3],
{ name: 'Mixed', tvg_id: 'mixed.tvg' },
channelsById,
);
const body = API.bulkUpdateChannels.mock.calls[0][0];
expect(body).toEqual([
{ id: 1, override: { name: 'Mixed', tvg_id: 'mixed.tvg' } },
{ id: 2, name: 'Mixed', tvg_id: 'mixed.tvg' },
{ id: 3, override: { name: 'Mixed', tvg_id: 'mixed.tvg' } },
]);
});
it('always-raw fields (hidden_from_output, user_level, is_adult) bypass override routing', async () => {
// hidden_from_output is a status flag, not a value override - it goes to
// Channel.hidden_from_output directly even on auto-created channels.
const channelsById = {
1: { id: 1, auto_created: true },
};
await updateChannelsWithOverrideRouting(
[1],
{ hidden_from_output: true, name: 'Renamed' },
channelsById,
);
const body = API.bulkUpdateChannels.mock.calls[0][0];
expect(body).toEqual([
{ id: 1, hidden_from_output: true, override: { name: 'Renamed' } },
]);
});
it('falls back to raw when channelsById lookup is missing the row', async () => {
// Defensive: if the channel store doesn't have the row (paginated
// off, recent creation), default behavior should not crash and
// should not silently drop the change.
await updateChannelsWithOverrideRouting(
[99],
{ name: 'Defensive' },
{}, // empty lookup
);
const body = API.bulkUpdateChannels.mock.calls[0][0];
expect(body).toEqual([{ id: 99, name: 'Defensive' }]);
});
});

340
frontend/src/utils/forms/__tests__/ChannelUtils.test.js
View file

@ -8,6 +8,12 @@ import {
getChannelFormDefaultValues,
getFormattedValues,
handleEpgUpdate,
getProviderHint,
getFkProviderHint,
normalizeFieldValue,
buildOverridePayload,
listOverriddenFields,
clearChannelOverrides,
} from '../ChannelUtils.js';
// ── API mock ───────────────────────────────────────────────────────────────────
@ -134,6 +140,7 @@ describe('ChannelUtils', () => {
logo_id: '10',
user_level: '1',
is_adult: false,
hidden_from_output: false,
});
});
@ -230,6 +237,7 @@ describe('ChannelUtils', () => {
logo_id: '',
user_level: '0',
is_adult: false,
hidden_from_output: false,
});
});
});
@ -458,4 +466,336 @@ describe('ChannelUtils', () => {
).rejects.toThrow('Update error');
});
});
// ── normalizeFieldValue ──────────────────────────────────────────────────────
describe('normalizeFieldValue', () => {
it('returns null for empty string', () => {
expect(normalizeFieldValue('name', '')).toBeNull();
});
it('returns null for null', () => {
expect(normalizeFieldValue('name', null)).toBeNull();
});
it('returns null for undefined', () => {
expect(normalizeFieldValue('name', undefined)).toBeNull();
});
it('returns null for the "-1" sentinel', () => {
expect(normalizeFieldValue('name', '-1')).toBeNull();
});
it('coerces channel_number string "10" to numeric 10', () => {
expect(normalizeFieldValue('channel_number', '10')).toBe(10);
});
it('coerces channel_number "5.5" to 5.5 (preserves decimal)', () => {
expect(normalizeFieldValue('channel_number', '5.5')).toBe(5.5);
});
it('coerces logo_id string "10" to integer 10', () => {
expect(normalizeFieldValue('logo_id', '10')).toBe(10);
});
it('coerces channel_group_id string "3" to integer 3', () => {
expect(normalizeFieldValue('channel_group_id', '3')).toBe(3);
});
it('coerces epg_data_id string "7" to integer 7', () => {
expect(normalizeFieldValue('epg_data_id', '7')).toBe(7);
});
it('coerces stream_profile_id string "2" to integer 2', () => {
expect(normalizeFieldValue('stream_profile_id', '2')).toBe(2);
});
it('treats stream_profile_id "0" as null (the "use default" sentinel)', () => {
expect(normalizeFieldValue('stream_profile_id', '0')).toBeNull();
});
it('treats logo_id "0" as null (the "Default" picker option)', () => {
expect(normalizeFieldValue('logo_id', '0')).toBeNull();
});
it('returns string field unchanged', () => {
expect(normalizeFieldValue('name', 'ESPN HD')).toBe('ESPN HD');
});
it('returns null for non-numeric channel_number', () => {
expect(normalizeFieldValue('channel_number', 'abc')).toBeNull();
});
it('returns null for non-integer FK id input', () => {
// parseInt('abc') is NaN, which is_not_finite, returns null.
expect(normalizeFieldValue('logo_id', 'abc')).toBeNull();
});
});
// ── normalizeFieldValue: sentinel battery ────────────────────────────────────
// Every overridable form field gets walked through the sentinel matrix so
// future field additions inherit coverage automatically. Add the field to
// OVERRIDABLE_FIELDS in ChannelUtils.js, then add its row to the matrix
// below.
describe('normalizeFieldValue: sentinel battery', () => {
// Sentinels common to all fields that must always normalize to null.
const universalNullSentinels = ['', null, undefined, '-1'];
// (field, expectations) pairs for every overridable form field.
// Each row documents what the field accepts as input and what
// normalizeFieldValue must return for the canonical inputs.
const matrix = [
{
field: 'name',
kind: 'string',
passthrough: [['ESPN HD', 'ESPN HD']],
zeroSentinelIsNull: false,
},
{
field: 'tvg_id',
kind: 'string',
passthrough: [['hbo.us', 'hbo.us']],
zeroSentinelIsNull: false,
},
{
field: 'tvc_guide_stationid',
kind: 'string',
passthrough: [['hbo-station', 'hbo-station']],
zeroSentinelIsNull: false,
},
{
field: 'channel_number',
kind: 'numeric',
passthrough: [
['10', 10],
['5.5', 5.5],
['0', 0],
],
zeroSentinelIsNull: false,
},
{
field: 'channel_group_id',
kind: 'fk-int',
passthrough: [
['3', 3],
['0', 0],
],
zeroSentinelIsNull: false,
},
{
field: 'epg_data_id',
kind: 'fk-int',
passthrough: [
['7', 7],
['0', 0],
],
zeroSentinelIsNull: false,
},
{
field: 'logo_id',
kind: 'fk-int',
passthrough: [['10', 10]],
// '0' is a domain sentinel: the picker's "Default" option.
zeroSentinelIsNull: true,
},
{
field: 'stream_profile_id',
kind: 'fk-int',
passthrough: [['2', 2]],
// '0' is a domain sentinel: "(use default)" in the form.
zeroSentinelIsNull: true,
},
];
matrix.forEach(({ field, passthrough, zeroSentinelIsNull }) => {
describe(`field: ${field}`, () => {
universalNullSentinels.forEach((sentinel) => {
it(`normalizes ${JSON.stringify(sentinel)} to null`, () => {
expect(normalizeFieldValue(field, sentinel)).toBeNull();
});
});
if (zeroSentinelIsNull) {
it(`treats "0" as the domain sentinel and returns null`, () => {
expect(normalizeFieldValue(field, '0')).toBeNull();
});
}
passthrough.forEach(([input, expected]) => {
it(`coerces ${JSON.stringify(input)} to ${JSON.stringify(expected)}`, () => {
expect(normalizeFieldValue(field, input)).toBe(expected);
});
});
});
});
});
// ── getProviderHint / getFkProviderHint ──────────────────────────────────────
describe('getProviderHint', () => {
it('returns null for null channel', () => {
expect(getProviderHint(null, 'name')).toBeNull();
});
it('returns null for manual (non-auto) channel', () => {
const ch = makeChannel({ auto_created: false });
expect(getProviderHint(ch, 'name')).toBeNull();
});
it('returns "Provider: <value>" for auto-created channel with value', () => {
const ch = makeChannel({ auto_created: true, name: 'ESPN' });
expect(getProviderHint(ch, 'name')).toBe('Provider: ESPN');
});
it('renders "(empty)" placeholder for null/empty provider value', () => {
const ch = makeChannel({ auto_created: true, tvg_id: null });
expect(getProviderHint(ch, 'tvg_id')).toBe('Provider: (empty)');
});
it('renders "(empty)" for empty string provider value', () => {
const ch = makeChannel({ auto_created: true, tvg_id: '' });
expect(getProviderHint(ch, 'tvg_id')).toBe('Provider: (empty)');
});
});
describe('getFkProviderHint', () => {
it('returns null for null channel', () => {
expect(getFkProviderHint(null, 'channel_group_id', {})).toBeNull();
});
it('returns null for manual channel', () => {
const ch = makeChannel({ auto_created: false });
expect(getFkProviderHint(ch, 'channel_group_id', {})).toBeNull();
});
it('returns "(none)" when provider FK id is null', () => {
const ch = makeChannel({ auto_created: true, channel_group_id: null });
expect(getFkProviderHint(ch, 'channel_group_id', {})).toBe(
'Provider: (none)'
);
});
it('returns the lookup name when FK id resolves', () => {
const ch = makeChannel({ auto_created: true, channel_group_id: 5 });
const lookup = { 5: { name: 'Sports' } };
expect(getFkProviderHint(ch, 'channel_group_id', lookup)).toBe(
'Provider: Sports'
);
});
it('falls back to tvg_id when name is missing', () => {
const ch = makeChannel({ auto_created: true, epg_data_id: 9 });
const lookup = { 9: { tvg_id: 'espn.us' } };
expect(getFkProviderHint(ch, 'epg_data_id', lookup)).toBe(
'Provider: espn.us'
);
});
it('falls back to stringified id when lookup misses', () => {
const ch = makeChannel({ auto_created: true, logo_id: 99 });
expect(getFkProviderHint(ch, 'logo_id', {})).toBe('Provider: 99');
});
});
// ── listOverriddenFields ─────────────────────────────────────────────────────
describe('listOverriddenFields', () => {
it('returns [] for channel without override', () => {
const ch = makeChannel({ override: null });
expect(listOverriddenFields(ch)).toEqual([]);
});
it('returns [] for null channel', () => {
expect(listOverriddenFields(null)).toEqual([]);
});
it('returns labels for all fields where override has a value', () => {
const ch = makeChannel({
override: { name: 'ESPN HD', channel_number: 100, logo_id: null },
});
const labels = listOverriddenFields(ch);
expect(labels).toContain('Name');
expect(labels).toContain('Channel Number');
expect(labels).not.toContain('Logo');
});
});
// ── buildOverridePayload ─────────────────────────────────────────────────────
describe('buildOverridePayload', () => {
it('returns undefined for null channel', () => {
expect(buildOverridePayload(null, {})).toBeUndefined();
});
it('returns null when every form value matches provider', () => {
// No override needed => signal "delete the override row" via null.
const ch = makeChannel();
const formattedValues = {
name: ch.name,
channel_number: ch.channel_number,
channel_group_id: ch.channel_group_id,
logo_id: ch.logo_id,
tvg_id: ch.tvg_id,
tvc_guide_stationid: ch.tvc_guide_stationid,
epg_data_id: ch.epg_data_id,
stream_profile_id: ch.stream_profile_id,
};
expect(buildOverridePayload(ch, formattedValues)).toBeNull();
});
it('emits the diverging field with form value, others as null (clear)', () => {
const ch = makeChannel();
const formattedValues = {
name: 'ESPN-NEW',
channel_number: ch.channel_number,
channel_group_id: ch.channel_group_id,
logo_id: ch.logo_id,
tvg_id: ch.tvg_id,
tvc_guide_stationid: ch.tvc_guide_stationid,
epg_data_id: ch.epg_data_id,
stream_profile_id: ch.stream_profile_id,
};
const payload = buildOverridePayload(ch, formattedValues);
expect(payload).not.toBeNull();
expect(payload.name).toBe('ESPN-NEW');
// Fields that match provider are emitted as null (clear that
// override field; ensures returning to provider value is cleanly
// expressed through one PATCH).
expect(payload.channel_number).toBeNull();
expect(payload.tvg_id).toBeNull();
});
it('coerces FK id "10" string to int 10 before comparing against provider 10', () => {
// Without normalization, "10" !== 10 would falsely emit logo_id as
// a divergence; the test pins normalizeFieldValue's role in the
// diff path.
const ch = makeChannel({ logo_id: 10 });
const formattedValues = {
name: ch.name,
channel_number: ch.channel_number,
channel_group_id: ch.channel_group_id,
logo_id: '10',
tvg_id: ch.tvg_id,
tvc_guide_stationid: ch.tvc_guide_stationid,
epg_data_id: ch.epg_data_id,
stream_profile_id: ch.stream_profile_id,
};
// No actual divergence; helper should return null.
expect(buildOverridePayload(ch, formattedValues)).toBeNull();
});
});
// ── clearChannelOverrides ────────────────────────────────────────────────────
describe('clearChannelOverrides', () => {
it('PATCHes the channel with override:null', () => {
API.updateChannel.mockResolvedValue({ id: 1, override: null });
clearChannelOverrides(7);
expect(API.updateChannel).toHaveBeenCalledWith({
id: 7,
override: null,
});
});
});
});

178
frontend/src/utils/forms/__tests__/GroupSyncUtils.test.js Normal file
View file

@ -0,0 +1,178 @@
import { describe, it, expect } from 'vitest';
import {
getGroupReservation,
detectGroupReservationOverlaps,
formatOverlapMessage,
} from '../GroupSyncUtils';
const makeGroup = (overrides = {}) => ({
channel_group: 1,
name: 'Entertainment',
enabled: true,
auto_channel_sync: true,
auto_sync_channel_start: 100,
auto_sync_channel_end: 110,
custom_properties: { channel_numbering_mode: 'fixed' },
...overrides,
});
describe('getGroupReservation', () => {
it('returns [start, end] for a bounded Fixed group', () => {
expect(getGroupReservation(makeGroup())).toEqual([100, 110]);
});
it('returns null when the group is disabled', () => {
expect(getGroupReservation(makeGroup({ enabled: false }))).toBeNull();
});
it('returns null when auto_channel_sync is off', () => {
expect(
getGroupReservation(makeGroup({ auto_channel_sync: false }))
).toBeNull();
});
it('returns null when end is missing (unbounded group does not reserve)', () => {
expect(
getGroupReservation(makeGroup({ auto_sync_channel_end: null }))
).toBeNull();
});
it('returns null for Next Available mode regardless of bounds', () => {
expect(
getGroupReservation(
makeGroup({
custom_properties: { channel_numbering_mode: 'next_available' },
})
)
).toBeNull();
});
it('reads fallback as Start for Provider mode', () => {
expect(
getGroupReservation(
makeGroup({
auto_sync_channel_start: null,
auto_sync_channel_end: 250,
custom_properties: {
channel_numbering_mode: 'provider',
channel_numbering_fallback: 200,
},
})
)
).toEqual([200, 250]);
});
it('returns null when end < start', () => {
expect(
getGroupReservation(
makeGroup({ auto_sync_channel_start: 500, auto_sync_channel_end: 100 })
)
).toBeNull();
});
});
describe('detectGroupReservationOverlaps', () => {
it('returns no conflicts when ranges are disjoint', () => {
const a = makeGroup({
channel_group: 1,
name: 'A',
auto_sync_channel_start: 1,
auto_sync_channel_end: 100,
});
const b = makeGroup({
channel_group: 2,
name: 'B',
auto_sync_channel_start: 101,
auto_sync_channel_end: 200,
});
expect(detectGroupReservationOverlaps([a, b])).toEqual([]);
});
it('flags a straight overlap', () => {
const a = makeGroup({
channel_group: 1,
name: 'A',
auto_sync_channel_start: 1,
auto_sync_channel_end: 100,
});
const b = makeGroup({
channel_group: 2,
name: 'B',
auto_sync_channel_start: 50,
auto_sync_channel_end: 150,
});
const conflicts = detectGroupReservationOverlaps([a, b]);
expect(conflicts).toHaveLength(1);
expect(conflicts[0].a.name).toBe('A');
expect(conflicts[0].b.name).toBe('B');
});
it('flags an enclosed range (one reservation inside another)', () => {
const outer = makeGroup({
channel_group: 1,
name: 'Outer',
auto_sync_channel_start: 1,
auto_sync_channel_end: 1000,
});
const inner = makeGroup({
channel_group: 2,
name: 'Inner',
auto_sync_channel_start: 500,
auto_sync_channel_end: 600,
});
expect(detectGroupReservationOverlaps([outer, inner])).toHaveLength(1);
});
it('ignores groups without reservations', () => {
const unbounded = makeGroup({
channel_group: 1,
name: 'Unbounded',
auto_sync_channel_end: null,
});
const next_avail = makeGroup({
channel_group: 2,
name: 'Next',
custom_properties: { channel_numbering_mode: 'next_available' },
});
expect(detectGroupReservationOverlaps([unbounded, next_avail])).toEqual([]);
});
it('returns all pairwise overlaps when three groups collide', () => {
const groups = [1, 2, 3].map((id) =>
makeGroup({
channel_group: id,
name: `G${id}`,
auto_sync_channel_start: 1,
auto_sync_channel_end: 100,
})
);
expect(detectGroupReservationOverlaps(groups)).toHaveLength(3);
});
});
describe('formatOverlapMessage', () => {
it('produces one line per conflict', () => {
const conflicts = [
{
a: { name: 'A', start: 1, end: 100 },
b: { name: 'B', start: 50, end: 150 },
},
];
const msg = formatOverlapMessage(conflicts);
expect(msg).toContain('"A" [1-100]');
expect(msg).toContain('"B" [50-150]');
});
it('truncates at five entries with a "... and N more" suffix', () => {
const conflicts = Array.from({ length: 8 }, (_, i) => ({
a: { name: `A${i}`, start: 1, end: 10 },
b: { name: `B${i}`, start: 5, end: 15 },
}));
const msg = formatOverlapMessage(conflicts);
expect(msg).toContain('and 3 more');
});
it('returns empty string for no conflicts', () => {
expect(formatOverlapMessage([])).toBe('');
});
});