Commit graph

223 commits

Author SHA1 Message Date
None
2a5889f5b2 fix(channels): Channel Group Override interactions
- Compact numbering: resolve the source relation when an override stores channels under the target group, so hiding releases the slot, unhiding assigns one, and repack sees the channels (no more spurious RANGE_EXHAUSTED). Type-safe override match; the bulk path stays single-query on the common path.
- Channel form: keep the clear-override control available when an override's value equals the provider value (isFormFieldOverridden is now existence-aware).
- Tests: compact override hide/unhide/repack, no-override fast-path guard, repack query-scaling guard, and existence-aware override detection.
2026-06-02 00:04:44 -05:00
SergeantPanda
a3511d6363 fix: update profile save method to include expiration date in custom properties 2026-05-17 17:43:15 -05:00
SergeantPanda
e54b70ba08 fix: enhance regex filtering for auto-sync channels to match frontend patterns 2026-05-16 11:41:15 -05:00
SergeantPanda
21913306c4 fix: Preserve base path in URL normalization for XC server and transform credentials. (Fixes #1218)
Some checks failed
CI Pipeline / prepare (push) Has been cancelled
Build and Push Multi-Arch Docker Image / build-and-push (push) Has been cancelled
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Has been cancelled
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
CI Pipeline / create-manifest (push) Has been cancelled
2026-05-14 09:53:10 -05:00
SergeantPanda
dc649ffd88 Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00
None
df7e6422df perf: eliminate three remaining N+1 queries in list endpoints
Layered on top of the prior perf commits, removes additional per-row queries

Channel list (apps/channels): add auto_created_by to select_related so ChannelSerializer.get_auto_created_by_name does not fire one SELECT per row. Replace instance.streams.all().order_by(channelstream__order) in to_representation with a read off the prefetched channelstream_set.

M3U account list (apps/m3u): prefetch_related "filters" on the viewset queryset and sort filters in Python over the prefetch in M3UAccountSerializer.get_filters.

Frontend (EditableCell.jsx): align the saveValue useCallback dependency from [row.original.id] to [row.original] in the Number, Group, EPG, and Logo cells, matching the pattern applied to the Text cell.
2026-05-02 16:48:37 -05:00
SergeantPanda
d0ce2fdd6d perf(M3UAccountSerializer): optimize stream count retrieval to avoid redundant queries 2026-05-02 11:51:13 -05:00
SergeantPanda
d08298b9c3 perf(M3UAccount): optimize list endpoint with pre-aggregated stream counts 2026-05-02 11:50:53 -05:00
None
0c0e802cf2 refactor(orphan-cleanup): store as custom_properties mode
Drop 3UAccount.auto_cleanup_unused_channels field and its migration (apps/m3u/0020). Replace with a 3-state string under M3UAccount.custom_properties.orphan_channel_cleanup: "always" (default), "preserve_customized", "never".

Move the UI from a checkbox on the M3U account form to a SegmentedControl at the top of LiveGroupFilter. Local state mirrors the prop for immediate click feedback; reverts on API error.

M3UAccountSerializer.update() now merges incoming custom_properties before layering the typed preference fields on top.

Add LiveGroupFilter.test.jsx (5 cases). Replace AutoCleanupToggleTests with OrphanCleanupModeTests (5 cases) for the new 3-state contract.
2026-05-01 23:24:40 -05:00
None
b2b108bfe5 rename Channel.user_hidden to Channel.hidden_from_output
The previous name suggested "hidden from a specific user account" rather than "hidden from downstream client output", which was the actual
behavior. The new name is unambiguous and matches the help_text language ("Exclude this channel from downstream client output").

Mechanical rename across the migration, model, serializer, signals, sync logic, output endpoints, frontend forms / tables / utils, tests, and CHANGELOG. 77 occurrences across 20 files.

No behavior change.
2026-05-01 19:44:16 -05:00
None
b9b3ded71c refactor(migrations): merge channels FR #1196 migrations into single 0037
Consolidates 0037_channeloverride_and_user_hidden, 0038_backfill_auto_created_by_null,
0039_channelgroupm3uaccount_auto_sync_channel_end, and 0040_channel_channel_number_nullable
into a single 0037_auto_sync_overhaul migration.

The merged migration runs operations in dependency-respecting order:
  1. AddField    Channel.user_hidden
  2. CreateModel ChannelOverride
  3. AddField    ChannelGroupM3UAccount.auto_sync_channel_end
  4. RunPython   backfill_auto_created_by_null
  5. AlterField  Channel.channel_number (nullable)
  6. RunPython   noop / reverse_backfill_channel_number_nulls

Also, update CHANGELOG.md and update migration references in test
2026-05-01 18:32:23 -05:00
None
6fed444e43 feat: Auto-sync overhaul (FR #1196)
Per-field channel overrides for auto-synced channels, hide-from-output flag, range-bounded auto-numbering with re-pack, multi-stream channel safety, multi-provider shared-range merging, and an across-the-board move from per-row sync writes to bulk operations.

Migrations:

apps/channels:

0036_channeloverride_and_user_hidden,
0037_backfill_auto_created_by_null,
0038_channelgroupm3uaccount_auto_sync_channel_end,
0039_channel_channel_number_nullable

apps/m3u:
0020_m3uaccount_auto_cleanup_unused_channels

See CHANGELOG.md for the full commit log
2026-05-01 14:05:06 -05:00
SergeantPanda
f2069caa62 Refactor: Simplify orphaned channel deletion and optimize M3UAccount fetching in send_m3u_update 2026-04-30 16:11:37 -05:00
SergeantPanda
2ef9ab0dcf Refactor: Remove unused stream deactivation and filtering methods from M3UAccount and M3UFilter classes 2026-04-30 16:01:55 -05:00
SergeantPanda
625dab0357 Enhancement: The default profile in the M3U profiles editor now exposes Search Pattern and Replace Pattern fields
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
2026-04-21 17:20:45 -05:00
SergeantPanda
69e0cba86e Bug Fix: EPG channel name vlaue too long for type character varying(255) (Fixes #1134)
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-04-21 09:23:35 -05:00
SergeantPanda
bfec229ca9 Enhancements:
- Rewrote the M3U line parser as an `iter_m3u_entries` generator that owns the full per-entry state machine. Intermediate directive lines between `#EXTINF` and the stream URL are now handled correctly rather than corrupting the pending entry or being silently misassigned. A `#EXTINF` with no following URL is discarded with a warning instead of carrying over a `url`-less entry into batch processing. Attribute keys are normalised to lowercase during parsing (provider attribute names remain case-insensitive end-to-end). The `#EXTINF` attribute regex is pre-compiled at module load, and attribute lookups use O(1) `dict.get()` instead of linear scans — approximately 10% faster parsing on large M3U files.
- Added support for the `#EXTGRP` directive in M3U files. When a `group-title` attribute is absent from the `#EXTINF` line, the value from a following `#EXTGRP:` line is used as the group. An explicit `group-title` attribute always takes priority. (Closes #1088)
- Added accumulation of `#EXTVLCOPT` directives per entry. Options are stored as a list under `vlc_opts` inside the stream's `custom_properties`, available for downstream use (e.g. passing VLC-specific options to the player). This is for a planned future enhancement and can also be utlized with the API.
2026-04-11 14:57:52 -05:00
SergeantPanda
fd7ac0029d Enhancement: M3U stream name parsing now uses the comma text (the canonical display title per the base #EXTINF spec) as the primary stream name, falling back to tvc-guide-title, then tvg-name, rather than preferring tvg-name first. Providers that use tvg-name as an EPG key and put the human-readable title after the comma will now display the correct name. Providers that duplicate the same value in both fields are unaffected. (Fixes #1081) 2026-04-11 12:46:36 -05:00
SergeantPanda
7083c512be security: Fixed path traversal vulnerability in file uploads. The M3U account upload (apps/m3u/api_views.py), logo upload (apps/channels/api_views.py), and backup upload (apps/backups/api_views.py) all used the uploaded filename directly without sanitization. os.path.join() discards all preceding components when it encounters an absolute path segment, and pathlib's / operator behaves identically; a relative ../ sequence also escapes via OS path resolution at open() time. All three upload paths now strip directory components via Path(name).name and validate the resolved path remains within the intended upload directory. Exploiting any of these required admin credentials. 2026-04-09 21:32:36 -05:00
SergeantPanda
086cc74959 Bug Fix: M3U profile URL rewriting now uses the regex module instead of re across all URL transform code paths (url_utils.transform_url, core/views.py, vod_proxy/_transform_url, tasks.get_transformed_credentials, and the WebSocket live-preview handler in consumers.py). The regex module natively accepts JavaScript/PCRE-style named capture groups ((?<name>...)) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a re.error. As a further improvement, regex also supports variable-length lookbehind assertions (e.g. (?<=a+)), which re rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling regex.sub: $<name>\g<name> and $1/$2/… → \1/\2/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
2026-03-29 17:55:08 -05:00
SergeantPanda
0815b3df14 tests: Add backend test for expiration feature. 2026-03-15 16:42:11 -05:00
SergeantPanda
bd11b0e832 Change notification for account expiry to admin only. 2026-03-15 16:03:41 -05:00
SergeantPanda
09d5f6bb9f Update expiration date handling in M3U components to ensure UTC format 2026-03-15 16:00:23 -05:00
SergeantPanda
1d4603dadd Enhancement: Optimize profile expiration handling and update task naming for clarity 2026-03-15 12:42:34 -05:00
SergeantPanda
b22f9e8e07 Enhancement: Account expiration tracking and notifications for M3U profiles 2026-03-14 18:38:59 -05:00
SergeantPanda
ea9f4f4bbd Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/JCBird1012/1062 2026-03-08 12:00:46 -05:00
SergeantPanda
9876a25e62 Bug Fix: XC stream refresh crashing with a null value in column "name" database error when a provider returns streams with a null or empty name. Affected streams are now assigned a generated fallback name in the format <account name> - <stream_id> so the refresh completes successfully and the stream remains accessible. A warning is logged for each affected stream. 2026-03-07 18:33:17 -06:00
SergeantPanda
3d27161755 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/nickgerrer/1022 2026-03-06 17:22:44 -06:00
Jonathan Caicedo
a51eb37075 fix: M3U EXTINF parsing for attributes ending in double equals
fixes: #1055
2026-03-06 08:10:30 -05:00
None
ce9f7ccbf9 fix(m3u): fix Celery worker memory leak during M3U/XC refresh
Restructure cleanup in refresh_single_m3u_account so del statements
run before gc.collect(), enabling cyclic garbage collection. Add worker_max_memory_per_child as a safety net against memory fragmentation.

Closes #1012, Closes #1053
2026-03-05 13:47:20 -06:00
Jonathan Caicedo
c6cf9443d2 fix: guard is_adult parsing against invalid values
or whatever other fun things providers might shove down the wire...
2026-03-05 14:16:21 -05:00
Nicholas Gerrer
858c81f826 perf: use bulk_create with update_conflicts for group settings save
Replace N+1 update_or_create() loop with bulk_create(update_conflicts=True)
wrapped in transaction.atomic(). On large M3U accounts with many groups,
the previous approach issued individual DB queries per group/category,
which could take minutes on slower hardware like Synology NAS devices.
2026-02-26 01:21:26 -05:00
SergeantPanda
e9a2cb4dbb
Merge pull request #1014 from CodeBormen/fix/861-M3U-downloading-endlessly
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
Fix #861: Prevent M3U/EPG tasks from downloading endlessly
2026-02-24 14:28:36 -06:00
SergeantPanda
74ce971b75 Bug Fix: Cron schedule got overwrote when changing/loading settings (signals issue) 2026-02-24 10:48:17 -06:00
None
e55ef41acb Fix uncovered lock_renewer.stop() paths in refresh_single_m3u_account
Five early-return paths released the Redis lock without stopping the
renewal thread, and the final release_task_lock was outside the finally block making it unreachable on exception. All exit paths now properly
stop the renewer before releasing the lock.
2026-02-23 00:48:07 -06:00
None
c841cfc8fc Fix #861: Prevent M3U endless downloading caused by Redis lock expiry
Large M3U downloads that exceed the 300s Redis lock TTL caused Celery Beat
to re-queue duplicate tasks, creating overlapping downloads that never complete.

Files changed:

- core/utils.py: Add TaskLockRenewer class — a daemon thread that periodically renews Redis lock TTL (every 120s) to prevent expiry during long-running tasks.

- apps/m3u/tasks.py: Apply TaskLockRenewer to refresh_single_m3u_account and refresh_m3u_groups; add HTTP timeout (30s connect, 60s read) to M3U download (the only download path missing one); stream M3U download to disk instead of accumulating in memory; add Celery task time limits (1 hour hard limit).

- apps/epg/tasks.py: Apply TaskLockRenewer to refresh_epg_data and parse_programs_for_tvg_id; add Celery task time limits (30 min / 1 hour).

Verified with a throttled test server: locks renewed at T+120s, T+240s, T+360s; 50,000 streams processed with no duplicate tasks.
2026-02-23 00:34:28 -06:00
SergeantPanda
ac1b43445e
Merge pull request #946 from CodeBormen/fix/945-XC-profile-refresh-extract-wrong-credentials
Fix XC URL sub-path handling in profile refresh and EPG creation
2026-02-16 16:04:53 -06:00
SergeantPanda
717cce92ec
Merge branch 'dev' into cron-scheduling 2026-02-16 10:55:33 -06:00
SergeantPanda
fb2f156fd8 Bug Fix: Fixes a bug with auto channel sync where channel numbers could be duplicated if multiple groups overlapped. 2026-02-13 15:07:24 -06:00
SergeantPanda
2c9f6d7360 Enhancement: For auto channel sync, allow using provider number, fixed start number or next available. (Closes #956, #433) 2026-02-13 14:34:22 -06:00
SergeantPanda
890992a9f9 feat(scheduling): add cron builder and refactor scheduling components (Closes #165)
Add cron expression support for M3U and EPG refreshes with interactive
builder modal and quick reference examples. Refactor backup scheduling
to use shared ScheduleInput component for consistency.

- New CronBuilder modal with preset buttons and custom field editors
- Info popover with common cron expression examples
- Shared ScheduleInput component for interval/cron toggle
2026-02-12 18:08:13 -06:00
None
8de23eec35 Fix XC URL sub-path handling in profile refresh and EPG creation
- Fix credential extraction in get_transformed_credentials() using negative
  indices anchored to the known tail structure instead of hardcoded indices
  that break when server URLs contain sub-paths
- Fix EPG URL construction in M3U form to normalize server URL to origin
  before appending xmltv.php endpoint

XC accounts with sub-paths in their server URL (e.g., http://server.com/portal/a/) caused two failures: profile refresh extracted wrong credentials from the URL path due to hardcoded array indices, and EPG auto-creation appended xmltv.php under the sub-path instead of at the domain root. Both fixes normalize away the sub-path using the same approach the backend's XCClient already uses for API calls.
2026-02-05 20:55:52 -06:00
SergeantPanda
027d114299 Enhancement: XtreamCodes Authentication Optimization: Reduced API calls during XC refresh by 50% by eliminating redundant authentication step. This should help reduce rate-limiting errors.
Some checks failed
Base Image Build / prepare (push) Has been cancelled
CI Pipeline / prepare (push) Has been cancelled
Build and Push Multi-Arch Docker Image / build-and-push (push) Has been cancelled
Frontend Tests / test (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Has been cancelled
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
CI Pipeline / create-manifest (push) Has been cancelled
2026-02-03 12:16:31 -06:00
SergeantPanda
9ffd595c96 Enhancement: Added stream_id (provider stream identifier) and stream_chno (provider channel number) fields to Stream model. For XC accounts, the stream hash now uses the stable stream_id instead of the URL when hashing, ensuring XC streams maintain their identity and channel associations even when account credentials or server URLs change. Supports both XC num and M3U tvg-chno/channel-number attributes. 2026-01-31 17:47:46 -06:00
SergeantPanda
5364123745 - Swagger/OpenAPI Migration: Migrated from drf-yasg (OpenAPI 2.0) to drf-spectacular (OpenAPI 3.0) for API documentation. This provides:
- Native Bearer token authentication support in Swagger UI - users can now enter just the JWT token and the "Bearer " prefix is automatically added
  - Modern OpenAPI 3.0 specification compliance
  - Better auto-generation of request/response schemas
  - Improved documentation accuracy with serializer introspection
2026-01-27 13:33:33 -06:00
SergeantPanda
d33d047a94 Enhancement: - Mature content filtering support:
- Added `is_adult` boolean field to both Stream and Channel models with database indexing for efficient filtering and sorting
  - Automatically populated during M3U/XC refresh operations by extracting `is_adult` value from provider data
  - Type-safe conversion supporting both integer (0/1) and string ("0"/"1") formats from different providers
  - UI controls in channel edit form (Switch with tooltip) and bulk edit form (Select dropdown) for easy management
  - XtreamCodes API support with proper integer formatting (0/1) in live stream responses
  - Automatic propagation from streams to channels during both single and bulk channel creation operations
  - Included in serializers for full API support
  - User-level content filtering: Non-admin users can opt to hide mature content channels across all interfaces (web UI, M3U playlists, EPG data, XtreamCodes API) via "Hide Mature Content" toggle in user settings (stored in custom_properties, admin users always see all content)
2026-01-17 15:00:28 -06:00
SergeantPanda
a84553d15c Enhancement: Stale status indicators for streams and groups: Added is_stale field to both Stream and ChannelGroupM3UAccount models to track items in their grace period (seen in previous refresh but not current). 2026-01-09 13:53:01 -06:00
SergeantPanda
cc9d38212e Enhancement: Groups now follow the same stale retention logic as streams, using the account's stale_stream_days setting. Groups that temporarily disappear from an M3U source are retained for the configured retention period instead of being immediately deleted, preserving user settings and preventing data loss when providers temporarily remove/re-add groups. (Closes #809) 2026-01-09 12:03:55 -06:00
SergeantPanda
73956924f5 Enhancement: Stream group as available hash option: Users can now select 'Group' as a hash key option in Settings → Stream Settings → M3U Hash Key, allowing streams to be differentiated by their group membership in addition to name, URL, TVG-ID, and M3U ID 2025-12-18 15:26:08 -06:00
SergeantPanda
2b58d7d46e Enhancement: Ensure "Uncategorized" categories and relations exist for VOD accounts. This improves content management for movies and series without assigned categories. Closes #627 2025-11-25 17:14:51 -06:00