Commit graph

246 commits

Author SHA1 Message Date
SergeantPanda
8f3dc83543 fix(api): Correct OpenAPI schema paths for nested M3U profiles and filters by removing escaped slashes, ensuring proper serialization and compatibility with client generators. Updated related API URL patterns for consistency. (Fixes #1384) 2026-06-25 13:37:29 -05:00
SergeantPanda
b7ee17ffba fix(tasks): Remove unnecessary raise statement in refresh_single_m3u_account function to improve error handling and maintain task flow. 2026-06-25 11:06:07 -05:00
None
a29704af55 fix(channels): account for group_override in the auto-sync range-conflict check
The range-conflict warning classified each channel in the configured range as either this config's own auto-sync output or a real conflict, comparing each occupant against the source group. When a group sets a group_override, auto-sync creates its channels in the override target group, so the config's own channels failed that comparison and were misclassified as a conflict.

- Add effectiveSyncGroupId to resolve the group the sync's channels actually land in (the group_override target when set, otherwise the source group).
- Compare occupants against that effective target, so the config's own output is recognized while genuine conflicts (manual channels, channels from another account, channels in a different group, user-pinned numbers) still warn.
- Add Vitest coverage for the helper, the override case, and an over-suppression guard, plus a backend test asserting the numbers-in-range endpoint reports the override target group.
2026-06-19 23:40:10 -05:00
SergeantPanda
46695588f7 fix(m3u): enhance custom properties handling and DB connection management
- Implemented `ensure_custom_properties_dict()` to normalize custom properties across various models and serializers, addressing issues with legacy JSON-encoded strings.
- Updated M3U account and channel group models to ensure custom properties are consistently stored as dictionaries during save operations.
- Enhanced Celery task management by ensuring old DB connections are closed before and after tasks, improving reliability and preventing errors during account refresh operations. (Fixes #1338)
2026-06-18 14:59:09 -05:00
SergeantPanda
cfe58db222 fix(xc): normalize server URLs for live playback and stream URLs (Fixes #1363)
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
- Implemented `normalize_server_url()` to standardize account server URLs, ensuring that on-demand live URLs are built correctly without including API endpoints or query parameters.
- Updated `get_transformed_credentials()` and stream URL generation in `M3UMovieRelation` and `M3UEpisodeRelation` to utilize the new normalization function, improving URL handling for Xtream Codes accounts.
2026-06-18 10:12:12 -05:00
SergeantPanda
56199aef81 perf(m3u): Enhance memory management and performance during XC stream processing
- Improved memory cleanup by invoking `gc.collect()` after batch processing in `process_m3u_batch_direct` and `collect_xc_streams`, ensuring timely release of resources.
- Added tests to verify that garbage collection is triggered appropriately after batch operations.
2026-06-16 14:24:26 -05:00
SergeantPanda
5744a55b06
Merge pull request #1328 from CodeBormen:fix/1273-provider-numbering
Fix/1273 provider numbering
2026-06-11 17:03:17 -05:00
SergeantPanda
703926bca5 fix(m3u): improve RANGE_EXHAUSTED error reporting for provider mode
- Introduced a new helper function to generate user-facing error messages for RANGE_EXHAUSTED failures, ensuring the fallback range is correctly displayed.
- Updated tests to verify that the error message reflects the fallback range instead of the hidden auto_sync_channel_start, enhancing clarity for users.
2026-06-11 16:58:43 -05:00
SergeantPanda
7ed9b11a89 Refactor Schedules Direct EPG handling and enhance guide fetch logic
- Replaced individual EPG program parse tasks with a centralized dispatch function to streamline guide refresh for newly assigned EPG IDs.
- Implemented batching for guide fetches when multiple EPGs are mapped, reducing redundant API calls and improving efficiency.
- Updated related utility functions to support the new fetching strategy and added tests to ensure correct behavior under various scenarios.
2026-06-11 16:41:30 -05:00
SergeantPanda
95975d5756 refactor(connection_pool): streamline credential slot management
- Removed redundant logic for releasing server group slots in `move_credential_slot_on_profile_switch` and `release_profile_slot`.
- Simplified the `_release_server_group_slot_for_profile` function by eliminating it, as its functionality was integrated into other methods.
2026-06-09 18:38:52 -05:00
SergeantPanda
4fc12bca4a refactor(m3u, channels, proxy): enhance stream assignment logic and error handling
- Improved logging in the Stream model for better debugging of profile evaluations.
- Introduced a new method `_stream_assignment_is_reusable` in the Channel model to determine if existing stream assignments can be reused, enhancing efficiency.
- Updated the release logic in `release_profile_slot` to utilize stored credential keys, reducing unnecessary database lookups.
- Simplified error handling in the `get_stream_info_for_switch` function to ensure proper stream release on exceptions.
- Enhanced tests for connection pool management and error handling in the ServerGroupsTable component to improve reliability and user feedback.
2026-06-09 17:59:12 -05:00
SergeantPanda
bc046b3c92 refactor(m3u, channels): streamline stream assignment and server group management
- Renamed `_clear_stream_assignment_keys` to `_release_stale_stream_assignment` for clarity and updated its logic to release pool counters.
- Introduced new functions for managing credential slots during profile switches, enhancing the handling of shared connection limits across server groups.
- Removed the `max_streams` field from the `ServerGroup` model and updated related components to reflect this change, simplifying the server group management.
- Updated frontend components to integrate server group management, allowing for dynamic creation and editing of server groups.
- Enhanced error handling in stream URL generation to provide more informative feedback on connection issues.
- Added tests for stale assignment release and credential management during profile switches.
2026-06-09 13:40:16 -05:00
SergeantPanda
0f0e855507 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/Goldenfreddy0703/1254 2026-06-09 11:16:39 -05:00
None
abf16fd104 test(m3u): cover provider-number collisions in auto-sync numbering
- A duplicate provider number keeps one channel and falls the collider back to a free number.
- A provider number matching an existing channel falls back instead of overwriting it.
2026-06-08 21:22:16 -05:00
None
57cae0e4be fix(m3u): correct auto-sync numbering field interactions from the auto-sync overhaul
The auto-sync overhaul added a [start, end] range and a shared numbering picker, but each mode's UI exposes only a subset of the persisted fields (Provider's "Start #" writes channel_numbering_fallback; Next Available exposes no Start/End) while the backend read auto_sync_channel_start and auto_sync_channel_end in every mode. Switching modes never resets the others, so a stale or auto-computed value silently changed numbering.

- Provider mode honors the provider-supplied number (stream_chno) verbatim; the group's Start (channel_numbering_fallback) and End bound only the fallback for streams the provider did not number. auto_sync_channel_start is no longer applied in provider mode.
- Next Available ignores End, since its UI exposes no range.
- Range enforcement (the overflow-delete) runs in fixed mode only, the one mode with a user-set [start, end] range.
- Provider mode gains the Start>End guard fixed mode already had, so an inverted fallback range cannot fail every numberless stream.

Includes backend and frontend regression tests.
2026-06-07 22:02:23 -05:00
SergeantPanda
270d0479b4
Merge pull request #1323 from CodeBormen/fix/compact-numbering-idempotency
fix(channels): make compact numbering repack idempotent
2026-06-06 17:49:42 -05:00
None
cae040d45d fix(channels): make compact numbering repack idempotent
The compact repack read its channels with no ORDER BY, so the pack followed PostgreSQL's physical row order. That order drifts after the
UPDATEs each repack issues, so successive syncs
packed the same channels into different numbers within the configured range. Auto-synced channel numbers reshuffled on every sync even when the provider had not changed.

- Add .order_by("id") to the _repack_inner channel query so the pack is deterministic. id order is creation order, which tracks the provider stream order used by the default "provider" sort.
- Add c.id as a secondary key to the name / tvg_id / updated_at sorts so equal values (e.g. blank tvg_id) break ties deterministically instead of churning.
- Add a deterministic regression test that forces a physical heap reorder (CLUSTER) and asserts two consecutive repacks produce identical channel numbers.
2026-06-05 00:01:22 -05:00
Goldenfreddy0703
abdf2d7864 Fix credential release on deleted profile and round 3 review items.
Store credential Redis keys at reserve so release works when the profile row is deleted. Return reserve failure reasons to avoid fingerprint DB queries on logging paths. Document unlimited profile bypass in pool logic and Server Group UI.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-04 19:51:20 -04:00
mwhit
763d1430fd fix: add channel_group_id to M3U stream bulk_update fields
Streams matched by hash during refresh were not updating channel_group_id,
causing cleanup to delete all streams when group IDs change (e.g., after
migration reset). Adds channel_group_id to comparison, assignment, only()
fetch, and bulk_update in both standard and XC processing paths.
2026-06-04 04:14:23 +00:00
mwhit
b6afcafc98 feat: SD data enrichment, artwork improvement, and bug fixes
- Fix channel_group_id not updating on M3U stream refresh
- Fix SD content rating region selection (country-aware filtering)
- Fix SD season/episode metadata iteration
- Add SD data enrichment: entityType, showType, contentAdvisory, originalAirDate, country, runtime, star ratings, event details, character names
- Cap cast to top-billed, remove role noise
- Switch artwork preference to Banner-L1 (branded key art)
- Add content advisory, sports details, runtime to program detail modal
- Logo cache-busting for browser cache invalidation
- SD API compliance: rolling 24h lineup reset, tokenExpires caching, error caching
2026-06-04 03:40:22 +00:00
None
2a5889f5b2 fix(channels): Channel Group Override interactions
- Compact numbering: resolve the source relation when an override stores channels under the target group, so hiding releases the slot, unhiding assigns one, and repack sees the channels (no more spurious RANGE_EXHAUSTED). Type-safe override match; the bulk path stays single-query on the common path.
- Channel form: keep the clear-override control available when an override's value equals the provider value (isFormFieldOverridden is now existence-aware).
- Tests: compact override hide/unhide/repack, no-override fast-path guard, repack query-scaling guard, and existence-aware override detection.
2026-06-02 00:04:44 -05:00
Goldenfreddy0703
6e07dce3f1 Fix maintainer review items without breaking multi-login rotation.
Cap credential-scoped group counters at profile.max_streams (not group.max_streams),
skip credential counters when fingerprint resolution fails, and always swap profile
counters on update_stream_profile. Restore per-profile-only selection for VOD/live
switches so a second stream can use a different login without invalid HTTP errors.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-29 17:07:52 -04:00
Goldenfreddy0703
bb9b95b4a4 Refactor connection pools to manual ServerGroups with profile rotation.
Drop auto-fingerprint migration and restore per-profile selection for live/VOD.
Enforce shared limits on reserve using login-scoped group counters, and add
Server Groups UI for manual account assignment per maintainer feedback (#1137).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-23 20:18:45 -04:00
Goldenfreddy0703
d89f634213 Enforce shared credential connection pools across M3U accounts (#1137)
Group M3U/XC accounts and profiles that share the same provider login into auto-assigned ServerGroups keyed by credential fingerprint. Enforce combined Redis limits for live TV and VOD via apps/m3u/connection_pool.py.

- Per-profile fingerprinting (XC transforms and STD stream URLs)

- VOD profile selection tries alternates when default credential pool is full (fixes live then VOD failure)

- Stats UI shows provider login from active stream URL

- Tests: apps/m3u/tests/test_connection_pool.py (11 tests, all passing)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 03:30:48 -04:00
SergeantPanda
a3511d6363 fix: update profile save method to include expiration date in custom properties 2026-05-17 17:43:15 -05:00
SergeantPanda
e54b70ba08 fix: enhance regex filtering for auto-sync channels to match frontend patterns 2026-05-16 11:41:15 -05:00
SergeantPanda
21913306c4 fix: Preserve base path in URL normalization for XC server and transform credentials. (Fixes #1218)
Some checks failed
CI Pipeline / prepare (push) Has been cancelled
Build and Push Multi-Arch Docker Image / build-and-push (push) Has been cancelled
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Has been cancelled
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
CI Pipeline / create-manifest (push) Has been cancelled
2026-05-14 09:53:10 -05:00
SergeantPanda
dc649ffd88 Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00
None
df7e6422df perf: eliminate three remaining N+1 queries in list endpoints
Layered on top of the prior perf commits, removes additional per-row queries

Channel list (apps/channels): add auto_created_by to select_related so ChannelSerializer.get_auto_created_by_name does not fire one SELECT per row. Replace instance.streams.all().order_by(channelstream__order) in to_representation with a read off the prefetched channelstream_set.

M3U account list (apps/m3u): prefetch_related "filters" on the viewset queryset and sort filters in Python over the prefetch in M3UAccountSerializer.get_filters.

Frontend (EditableCell.jsx): align the saveValue useCallback dependency from [row.original.id] to [row.original] in the Number, Group, EPG, and Logo cells, matching the pattern applied to the Text cell.
2026-05-02 16:48:37 -05:00
SergeantPanda
d0ce2fdd6d perf(M3UAccountSerializer): optimize stream count retrieval to avoid redundant queries 2026-05-02 11:51:13 -05:00
SergeantPanda
d08298b9c3 perf(M3UAccount): optimize list endpoint with pre-aggregated stream counts 2026-05-02 11:50:53 -05:00
None
0c0e802cf2 refactor(orphan-cleanup): store as custom_properties mode
Drop 3UAccount.auto_cleanup_unused_channels field and its migration (apps/m3u/0020). Replace with a 3-state string under M3UAccount.custom_properties.orphan_channel_cleanup: "always" (default), "preserve_customized", "never".

Move the UI from a checkbox on the M3U account form to a SegmentedControl at the top of LiveGroupFilter. Local state mirrors the prop for immediate click feedback; reverts on API error.

M3UAccountSerializer.update() now merges incoming custom_properties before layering the typed preference fields on top.

Add LiveGroupFilter.test.jsx (5 cases). Replace AutoCleanupToggleTests with OrphanCleanupModeTests (5 cases) for the new 3-state contract.
2026-05-01 23:24:40 -05:00
None
b2b108bfe5 rename Channel.user_hidden to Channel.hidden_from_output
The previous name suggested "hidden from a specific user account" rather than "hidden from downstream client output", which was the actual
behavior. The new name is unambiguous and matches the help_text language ("Exclude this channel from downstream client output").

Mechanical rename across the migration, model, serializer, signals, sync logic, output endpoints, frontend forms / tables / utils, tests, and CHANGELOG. 77 occurrences across 20 files.

No behavior change.
2026-05-01 19:44:16 -05:00
None
b9b3ded71c refactor(migrations): merge channels FR #1196 migrations into single 0037
Consolidates 0037_channeloverride_and_user_hidden, 0038_backfill_auto_created_by_null,
0039_channelgroupm3uaccount_auto_sync_channel_end, and 0040_channel_channel_number_nullable
into a single 0037_auto_sync_overhaul migration.

The merged migration runs operations in dependency-respecting order:
  1. AddField    Channel.user_hidden
  2. CreateModel ChannelOverride
  3. AddField    ChannelGroupM3UAccount.auto_sync_channel_end
  4. RunPython   backfill_auto_created_by_null
  5. AlterField  Channel.channel_number (nullable)
  6. RunPython   noop / reverse_backfill_channel_number_nulls

Also, update CHANGELOG.md and update migration references in test
2026-05-01 18:32:23 -05:00
None
6fed444e43 feat: Auto-sync overhaul (FR #1196)
Per-field channel overrides for auto-synced channels, hide-from-output flag, range-bounded auto-numbering with re-pack, multi-stream channel safety, multi-provider shared-range merging, and an across-the-board move from per-row sync writes to bulk operations.

Migrations:

apps/channels:

0036_channeloverride_and_user_hidden,
0037_backfill_auto_created_by_null,
0038_channelgroupm3uaccount_auto_sync_channel_end,
0039_channel_channel_number_nullable

apps/m3u:
0020_m3uaccount_auto_cleanup_unused_channels

See CHANGELOG.md for the full commit log
2026-05-01 14:05:06 -05:00
SergeantPanda
f2069caa62 Refactor: Simplify orphaned channel deletion and optimize M3UAccount fetching in send_m3u_update 2026-04-30 16:11:37 -05:00
SergeantPanda
2ef9ab0dcf Refactor: Remove unused stream deactivation and filtering methods from M3UAccount and M3UFilter classes 2026-04-30 16:01:55 -05:00
SergeantPanda
625dab0357 Enhancement: The default profile in the M3U profiles editor now exposes Search Pattern and Replace Pattern fields
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
2026-04-21 17:20:45 -05:00
SergeantPanda
69e0cba86e Bug Fix: EPG channel name vlaue too long for type character varying(255) (Fixes #1134)
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-04-21 09:23:35 -05:00
SergeantPanda
bfec229ca9 Enhancements:
- Rewrote the M3U line parser as an `iter_m3u_entries` generator that owns the full per-entry state machine. Intermediate directive lines between `#EXTINF` and the stream URL are now handled correctly rather than corrupting the pending entry or being silently misassigned. A `#EXTINF` with no following URL is discarded with a warning instead of carrying over a `url`-less entry into batch processing. Attribute keys are normalised to lowercase during parsing (provider attribute names remain case-insensitive end-to-end). The `#EXTINF` attribute regex is pre-compiled at module load, and attribute lookups use O(1) `dict.get()` instead of linear scans — approximately 10% faster parsing on large M3U files.
- Added support for the `#EXTGRP` directive in M3U files. When a `group-title` attribute is absent from the `#EXTINF` line, the value from a following `#EXTGRP:` line is used as the group. An explicit `group-title` attribute always takes priority. (Closes #1088)
- Added accumulation of `#EXTVLCOPT` directives per entry. Options are stored as a list under `vlc_opts` inside the stream's `custom_properties`, available for downstream use (e.g. passing VLC-specific options to the player). This is for a planned future enhancement and can also be utlized with the API.
2026-04-11 14:57:52 -05:00
SergeantPanda
fd7ac0029d Enhancement: M3U stream name parsing now uses the comma text (the canonical display title per the base #EXTINF spec) as the primary stream name, falling back to tvc-guide-title, then tvg-name, rather than preferring tvg-name first. Providers that use tvg-name as an EPG key and put the human-readable title after the comma will now display the correct name. Providers that duplicate the same value in both fields are unaffected. (Fixes #1081) 2026-04-11 12:46:36 -05:00
SergeantPanda
7083c512be security: Fixed path traversal vulnerability in file uploads. The M3U account upload (apps/m3u/api_views.py), logo upload (apps/channels/api_views.py), and backup upload (apps/backups/api_views.py) all used the uploaded filename directly without sanitization. os.path.join() discards all preceding components when it encounters an absolute path segment, and pathlib's / operator behaves identically; a relative ../ sequence also escapes via OS path resolution at open() time. All three upload paths now strip directory components via Path(name).name and validate the resolved path remains within the intended upload directory. Exploiting any of these required admin credentials. 2026-04-09 21:32:36 -05:00
SergeantPanda
086cc74959 Bug Fix: M3U profile URL rewriting now uses the regex module instead of re across all URL transform code paths (url_utils.transform_url, core/views.py, vod_proxy/_transform_url, tasks.get_transformed_credentials, and the WebSocket live-preview handler in consumers.py). The regex module natively accepts JavaScript/PCRE-style named capture groups ((?<name>...)) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a re.error. As a further improvement, regex also supports variable-length lookbehind assertions (e.g. (?<=a+)), which re rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling regex.sub: $<name>\g<name> and $1/$2/… → \1/\2/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
2026-03-29 17:55:08 -05:00
SergeantPanda
0815b3df14 tests: Add backend test for expiration feature. 2026-03-15 16:42:11 -05:00
SergeantPanda
bd11b0e832 Change notification for account expiry to admin only. 2026-03-15 16:03:41 -05:00
SergeantPanda
09d5f6bb9f Update expiration date handling in M3U components to ensure UTC format 2026-03-15 16:00:23 -05:00
SergeantPanda
1d4603dadd Enhancement: Optimize profile expiration handling and update task naming for clarity 2026-03-15 12:42:34 -05:00
SergeantPanda
b22f9e8e07 Enhancement: Account expiration tracking and notifications for M3U profiles 2026-03-14 18:38:59 -05:00
SergeantPanda
ea9f4f4bbd Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/JCBird1012/1062 2026-03-08 12:00:46 -05:00
SergeantPanda
9876a25e62 Bug Fix: XC stream refresh crashing with a null value in column "name" database error when a provider returns streams with a null or empty name. Affected streams are now assigned a generated fallback name in the format <account name> - <stream_id> so the refresh completes successfully and the stream remains accessible. A warning is logged for each affected stream. 2026-03-07 18:33:17 -06:00