This update modifies permission classes in multiple viewsets to restrict access based on user roles. The `IsAdmin` permission is now enforced for several actions, including group management and permission listing, ensuring that only administrators can perform sensitive operations. Additionally, a new utility function, `resolve_safe_local_data_path`, is introduced to enhance security when accessing local file paths. The changes improve overall security and maintainability of the codebase.
Extends _open_m3u_text_source() and fetch_m3u_lines() to treat an
uploaded .xz playlist the same way as the existing .m3u.gz path: it is
streamed lazily via lzma.open() rather than loaded fully into memory
(unlike the .zip path, which must read archive members). Uses stdlib
lzma, no new dependency.
Companion to the EPG xz support added for Dispatcharr/Dispatcharr#1414 -
the M3U upload path has the same gzip/zip dispatch structure and would
otherwise hit the same gap for an xz-compressed playlist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Find and Replace preview did not correctly reflect the rename the sync performs, and the rename engine differed from the preview engine.
- The preview rendered the literal $1 instead of the substituted capture group, because the replacement was passed straight into the regex engine, which honors \1, not the JS-style $1 the field accepts.
- The preview compiled patterns with the regex module while the live rename used stdlib re, so patterns valid in regex but not re (for example ^*) previewed a transform the sync silently skipped.
- A rename that expanded a name past the Channel.name column length aborted the whole bulk_create sync, while the preview showed the full name.
- Convert JS-style $1 backreferences to \1 via a shared helper used by both the preview and the live rename.
- Switch the live rename from re.sub to regex.sub, matching the preview engine and the sync's own include/exclude filters, with a timeout to bound catastrophic backtracking on user patterns.
- Cap the rename result at the Channel.name column length in both paths, so an over-length result cannot abort the sync.
- Add unit, integration, and differential parity tests covering the above.
When an Xtream Codes provider returns no live streams on a routine refresh
(a transient upstream failure, a fetch exception, or no enabled category
matching), collect_xc_streams() returns an empty list. The refresh then fell
through to stale-marking and sync_auto_channels, which -- with nothing seen
this refresh -- deletes the account's entire auto-created channel lineup via
its per-group "no streams remaining" branch.
Guard the XC branch of _refresh_single_m3u_account_impl: on an empty result,
set the account to ERROR and return before stale-marking and auto-sync,
mirroring the standard-path empty/failed-download guards already in the
function. A transient empty fetch can no longer destroy channels.
Adds apps/m3u/tests/test_xc_empty_fetch_guard.py covering both the abort
(sync not called, channels preserved, streams not marked stale, status ERROR)
and the healthy path (non-empty fetch still runs sync).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The range-conflict warning classified each channel in the configured range as either this config's own auto-sync output or a real conflict, comparing each occupant against the source group. When a group sets a group_override, auto-sync creates its channels in the override target group, so the config's own channels failed that comparison and were misclassified as a conflict.
- Add effectiveSyncGroupId to resolve the group the sync's channels actually land in (the group_override target when set, otherwise the source group).
- Compare occupants against that effective target, so the config's own output is recognized while genuine conflicts (manual channels, channels from another account, channels in a different group, user-pinned numbers) still warn.
- Add Vitest coverage for the helper, the override case, and an over-suppression guard, plus a backend test asserting the numbers-in-range endpoint reports the override target group.
- Implemented `ensure_custom_properties_dict()` to normalize custom properties across various models and serializers, addressing issues with legacy JSON-encoded strings.
- Updated M3U account and channel group models to ensure custom properties are consistently stored as dictionaries during save operations.
- Enhanced Celery task management by ensuring old DB connections are closed before and after tasks, improving reliability and preventing errors during account refresh operations. (Fixes#1338)
- Implemented `normalize_server_url()` to standardize account server URLs, ensuring that on-demand live URLs are built correctly without including API endpoints or query parameters.
- Updated `get_transformed_credentials()` and stream URL generation in `M3UMovieRelation` and `M3UEpisodeRelation` to utilize the new normalization function, improving URL handling for Xtream Codes accounts.
- Improved memory cleanup by invoking `gc.collect()` after batch processing in `process_m3u_batch_direct` and `collect_xc_streams`, ensuring timely release of resources.
- Added tests to verify that garbage collection is triggered appropriately after batch operations.
Aligns catch-up with how live and VOD manage provider capacity:
- Reserve a provider profile slot (connection_pool) before every upstream
connect, walking the account's active profiles default-first when the
default is at capacity (profile_full/credential_full are transient and
never mark the account decisive). Credentials for the reserved profile
are resolved via get_transformed_credentials — the same credential
extraction live playback uses — so pool accounting and real upstream
usage always agree. All eligible streams blocked on capacity alone
returns 503 (the VOD pool-exhausted precedent).
- Release exactly once via a one-shot Redis ownership token consumed with
a transactional GET+DEL: the generator finally, the response-close
wrapper, failed failover attempts and session takeover all share it, so
no path can double-decrement and a client disconnecting before the
first chunk still releases (Django registers the iterator's close() as
a resource closer). Failures between reservation and the streaming
response owning the slot release before propagating; an ownership-token
write failure releases directly and reports transient unavailability.
- One catch-up session per user and channel: a new request (programme
jump or seek) displaces the user's previous session on that channel —
its slot is released synchronously, its stats are unregistered, and its
generator is stopped through the standard stop-key mechanism — so rapid
seeking cannot stack upstream provider connections.
- rollup_channel_catchup_fields self-heals channels left flagged
is_catchup with no remaining catch-up stream (outside the
account-scoped CTE), covering bulk removals on manual/multi-provider
channels regardless of how the link rows disappeared. A regression test
locks the ChannelStream post_delete signal firing on queryset bulk
deletes.
Backend test suite grows to 94 (slot reservation/release on every
failover outcome, profile walk, mixed capacity-vs-upstream precedence,
exception-path release, token exactly-once semantics, takeover scoping
and ordering, never-started-generator release, rollup self-heal).
- Introduced a new helper function to generate user-facing error messages for RANGE_EXHAUSTED failures, ensuring the fallback range is correctly displayed.
- Updated tests to verify that the error message reflects the fallback range instead of the hidden auto_sync_channel_start, enhancing clarity for users.
- Replaced individual EPG program parse tasks with a centralized dispatch function to streamline guide refresh for newly assigned EPG IDs.
- Implemented batching for guide fetches when multiple EPGs are mapped, reducing redundant API calls and improving efficiency.
- Updated related utility functions to support the new fetching strategy and added tests to ensure correct behavior under various scenarios.
- http_streamer.py: restore HTTPStreamReader to its upstream form and keep
only the find_ts_sync() addition. The response=/extra_headers=/
strip_ts_preamble= extensions had no remaining callers since the timeshift
view moved to direct iter_content streaming (delta shrinks from +235/-83
lines to +28).
- Unify the catchup_days semantics everywhere: a channel's archive depth is
MAX(catchup_days) over its CATCH-UP streams only. The SQL rollup now uses
a FILTER (WHERE s.is_catchup) aggregate and the ChannelStream signal uses
the same MAX aggregation (it previously took the first stream by order,
and the rollup aggregated over all streams including non-catchup ones).
- Migration backfill: also accept the lowercase 'true' that ->> extraction
yields for JSON booleans.
- get_channel_catchup_info(): drop the tv_archive_duration key — its only
caller never used it.
- Document why timeshift termination fails closed when Redis is unavailable
(denying the new stream is what protects the provider connection limit),
and update the stale stop-key comment (5 s cadence, not 100 chunks).
- Removed redundant logic for releasing server group slots in `move_credential_slot_on_profile_switch` and `release_profile_slot`.
- Simplified the `_release_server_group_slot_for_profile` function by eliminating it, as its functionality was integrated into other methods.
- Improved logging in the Stream model for better debugging of profile evaluations.
- Introduced a new method `_stream_assignment_is_reusable` in the Channel model to determine if existing stream assignments can be reused, enhancing efficiency.
- Updated the release logic in `release_profile_slot` to utilize stored credential keys, reducing unnecessary database lookups.
- Simplified error handling in the `get_stream_info_for_switch` function to ensure proper stream release on exceptions.
- Enhanced tests for connection pool management and error handling in the ServerGroupsTable component to improve reliability and user feedback.
- Renamed `_clear_stream_assignment_keys` to `_release_stale_stream_assignment` for clarity and updated its logic to release pool counters.
- Introduced new functions for managing credential slots during profile switches, enhancing the handling of shared connection limits across server groups.
- Removed the `max_streams` field from the `ServerGroup` model and updated related components to reflect this change, simplifying the server group management.
- Updated frontend components to integrate server group management, allowing for dynamic creation and editing of server groups.
- Enhanced error handling in stream URL generation to provide more informative feedback on connection issues.
- Added tests for stale assignment release and credential management during profile switches.
- A duplicate provider number keeps one channel and falls the collider back to a free number.
- A provider number matching an existing channel falls back instead of overwriting it.
The auto-sync overhaul added a [start, end] range and a shared numbering picker, but each mode's UI exposes only a subset of the persisted fields (Provider's "Start #" writes channel_numbering_fallback; Next Available exposes no Start/End) while the backend read auto_sync_channel_start and auto_sync_channel_end in every mode. Switching modes never resets the others, so a stale or auto-computed value silently changed numbering.
- Provider mode honors the provider-supplied number (stream_chno) verbatim; the group's Start (channel_numbering_fallback) and End bound only the fallback for streams the provider did not number. auto_sync_channel_start is no longer applied in provider mode.
- Next Available ignores End, since its UI exposes no range.
- Range enforcement (the overflow-delete) runs in fixed mode only, the one mode with a user-set [start, end] range.
- Provider mode gains the Start>End guard fixed mode already had, so an inverted fallback range cannot fail every numberless stream.
Includes backend and frontend regression tests.
The compact repack read its channels with no ORDER BY, so the pack followed PostgreSQL's physical row order. That order drifts after the
UPDATEs each repack issues, so successive syncs
packed the same channels into different numbers within the configured range. Auto-synced channel numbers reshuffled on every sync even when the provider had not changed.
- Add .order_by("id") to the _repack_inner channel query so the pack is deterministic. id order is creation order, which tracks the provider stream order used by the default "provider" sort.
- Add c.id as a secondary key to the name / tvg_id / updated_at sorts so equal values (e.g. blank tvg_id) break ties deterministically instead of churning.
- Add a deterministic regression test that forces a physical heap reorder (CLUSTER) and asserts two consecutive repacks produce identical channel numbers.
Store credential Redis keys at reserve so release works when the profile row is deleted. Return reserve failure reasons to avoid fingerprint DB queries on logging paths. Document unlimited profile bypass in pool logic and Server Group UI.
Co-authored-by: Cursor <cursoragent@cursor.com>
Streams matched by hash during refresh were not updating channel_group_id,
causing cleanup to delete all streams when group IDs change (e.g., after
migration reset). Adds channel_group_id to comparison, assignment, only()
fetch, and bulk_update in both standard and XC processing paths.
- Compact numbering: resolve the source relation when an override stores channels under the target group, so hiding releases the slot, unhiding assigns one, and repack sees the channels (no more spurious RANGE_EXHAUSTED). Type-safe override match; the bulk path stays single-query on the common path.
- Channel form: keep the clear-override control available when an override's value equals the provider value (isFormFieldOverridden is now existence-aware).
- Tests: compact override hide/unhide/repack, no-override fast-path guard, repack query-scaling guard, and existence-aware override detection.
Cap credential-scoped group counters at profile.max_streams (not group.max_streams),
skip credential counters when fingerprint resolution fails, and always swap profile
counters on update_stream_profile. Restore per-profile-only selection for VOD/live
switches so a second stream can use a different login without invalid HTTP errors.
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds native catch-up/timeshift replay for Xtream Codes providers through
the same HTTPStreamReader transport pipeline as live TV.
Timeshift proxy (apps/timeshift/):
- URL cascade: 3 candidate timestamp formats per provider, per-account
format cache for fast-forward seek performance
- MPEG-TS preamble stripping (shared with HTTPStreamReader)
- Stats integration: timeshift viewers appear on /stats with TIMESHIFT badge
- Auth via hmac.compare_digest on XC password
Catchup detection — denormalized for zero-cost output queries:
- Stream.is_catchup + Stream.catchup_days populated at XC import time
- Channel.has_catchup + Channel.catchup_days + Channel.catchup_provider_stream_id
rolled up via ChannelStream post_save signal (UI path) and explicit SQL
after bulk_create (import path)
- _xc_channel_entry() reads denormalized fields instead of per-channel
custom_properties JSON introspection (eliminates N+1 queries)
- Migration 0038 backfills existing data via raw SQL
XC API enhancements:
- server_info.timezone + start/end + time_now use configured timezone
(triple consistency rule — fixes wrong-programme-plays bug)
- Dynamic has_archive flag + auto prev_days for catch-up channels
- XMLTV timestamps rewritten to local timezone for catch-up clients
HTTPStreamReader extended (apps/proxy/live_proxy/input/http_streamer.py):
- 1 MB pipe buffer via fcntl F_SETPIPE_SZ (eliminates producer/consumer
ping-pong that halved throughput)
- Pre-opened response= for URL cascade workflows
- strip_ts_preamble= for XC servers emitting PHP warnings before TS
- find_ts_sync() as shared utility
- Builds on upstream O_NONBLOCK + select() write loop
Provider stream_id lookup order:
- stream_xc() and xc_get_epg() try internal Channel.id first, fall back
to provider stream_id only when needed (avoids unconditional query on
every request)
Also includes:
- VOD provider cascade in stream_vod() — iterates all M3U relations by
priority when first provider is at capacity
- Defensive null-safety: custom_sid: None → "" in get_live_streams,
get_vod_streams, get_vod_info, get_series_info (fixes iPlayTV crash on
JSON null for string fields)
- Timeshift settings UI (timezone selector, debug toggle)
- StreamConnectionCard violet TIMESHIFT badge
- Orphan cleanup skips timeshift_* virtual channels
Drop auto-fingerprint migration and restore per-profile selection for live/VOD.
Enforce shared limits on reserve using login-scoped group counters, and add
Server Groups UI for manual account assignment per maintainer feedback (#1137).
Co-authored-by: Cursor <cursoragent@cursor.com>
Group M3U/XC accounts and profiles that share the same provider login into auto-assigned ServerGroups keyed by credential fingerprint. Enforce combined Redis limits for live TV and VOD via apps/m3u/connection_pool.py.
- Per-profile fingerprinting (XC transforms and STD stream URLs)
- VOD profile selection tries alternates when default credential pool is full (fixes live then VOD failure)
- Stats UI shows provider login from active stream URL
- Tests: apps/m3u/tests/test_connection_pool.py (11 tests, all passing)
Co-authored-by: Cursor <cursoragent@cursor.com>
