Commit graph

191 commits

Author SHA1 Message Date
SergeantPanda
7ed9b11a89 Refactor Schedules Direct EPG handling and enhance guide fetch logic
- Replaced individual EPG program parse tasks with a centralized dispatch function to streamline guide refresh for newly assigned EPG IDs.
- Implemented batching for guide fetches when multiple EPGs are mapped, reducing redundant API calls and improving efficiency.
- Updated related utility functions to support the new fetching strategy and added tests to ensure correct behavior under various scenarios.
2026-06-11 16:41:30 -05:00
SergeantPanda
c274bd7fbb Enhance Schedules Direct integration with per-channel guide fetch and improved EPG handling
- Implemented a targeted guide fetch for Schedules Direct when mapping a channel, ensuring immediate data availability without redundant API calls.
- Updated EPG signals to queue guide fetches for newly mapped channels, streamlining the process and improving user experience.
- Refactored existing logic to skip unnecessary program refreshes for dummy sources and ensure proper handling of Schedules Direct data.
- Added unit tests to validate the new fetching behavior and ensure robustness of the integration.
2026-06-11 13:54:19 -05:00
SergeantPanda
843940f552 refactor(http headers): centralize User-Agent construction and streamline API requests
- Introduced `dispatcharr_user_agent`, `dispatcharr_dvr_user_agent`, and `dispatcharr_http_headers` functions in `core.utils` to standardize User-Agent strings and HTTP headers for outbound requests.
- Updated various components, including `LogoViewSet`, `EPGSourceViewSet`, and VOD proxy views, to utilize the new header functions, ensuring consistent User-Agent usage across the application.
- Enhanced the handling of Schedules Direct API requests by including proper User-Agent headers, addressing previous API compliance issues.
2026-06-11 09:45:35 -05:00
SergeantPanda
7086f41d64 feat(epg): overhaul EPG auto-matching logic and improve performance
- Moved matching logic to a dedicated module for better organization and testability.
- Made single-channel auto-matching asynchronous, allowing for larger EPG libraries without hitting HTTP timeouts.
- Enhanced memory management and throughput during EPG matching, including optimizations for fuzzy matching and bulk processing.
- Fixed various reliability issues in the auto-matching process, ensuring accurate channel assignments and improved UI feedback.
- Updated API views and frontend components to reflect changes in the matching process and provide real-time notifications.
- Added tests for EPG matching functionality and name normalization.
- Single-channel and selected-channel auto-match always run, even when the channel already has EPG assigned; match-all (no channel IDs) still only processes channels without EPG.
2026-06-08 18:12:29 -05:00
SergeantPanda
70d900f346 feat(epg): implement auto-apply functionality for EPG logos
Add support for automatically applying channel logos from EPG data during refresh. Introduce a new toggle in the UI for enabling/disabling this feature, and update the backend to handle both channel IDs and EPG source IDs for logo application. Enhance the logo application task to process large libraries efficiently in chunks, improving performance and memory usage. Update changelog to reflect these changes.
2026-06-05 12:49:05 -05:00
SergeantPanda
c33d456743 Enable gevent in uWSGI workers; fix WS delivery and DB exhaustion
- Enable gevent cooperative multitasking in all uWSGI worker configs
  (gevent-early-monkey-patch + import dispatcharr.gevent_patch)
- Rewrite WebSocket group sends to bypass asyncio in gevent workers:
  _gevent_ws_send() replicates the channels_redis 4.x wire format
  directly via synchronous Redis so send_websocket_update() and
  _send_async() no longer fail silently after epoll is patched out
- Fix PostgreSQL connection exhaustion: CONN_MAX_AGE=0 + explicit
  close_old_connections() in stream manager and cleanup watchdog loops
- Fix stream proxy race: register client before the connect-wait loop
  so the cleanup watchdog never sees zero clients on a live channel
- Channel list/logo/profile queryset optimisations: conditional DISTINCT,
  EXISTS semi-joins for filter-options, channel_count annotation to
  eliminate N+1 in LogoSerializer, prefetched memberships in
  ChannelProfileSerializer
- JsonResponse for channel ID list and summary endpoints
2026-05-14 18:55:09 -05:00
SergeantPanda
dc649ffd88 Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00
SergeantPanda
e6c580c9bd feat: Implement delete series rule functionality with title parameter and update related components 2026-05-05 08:40:05 -05:00
SergeantPanda
0822c9aad2 feat: Make EPG channel optional in series rules and enhance search criteria 2026-05-04 21:34:56 -05:00
SergeantPanda
5940efbddf feat: Enhance series rule evaluation and management
- Added support for customizable title and description matching modes in series rules.
- Implemented a preview feature for series rules to show matching upcoming programs.
- Created a new SeriesRuleEditorModal for editing series rules with improved UI.
- Refactored query parsing logic into reusable functions for better maintainability.
- Updated API to handle new parameters for series rule creation and evaluation.
- Enhanced the ProgramRecordingModal and SeriesRecordingModal to integrate the new rule editor.
- Added pagination for program search results in the API.
2026-05-04 20:45:40 -05:00
None
983a6675c5 ui: scope regex preview to the M3U account
- Scope regex previews to the M3U account being edited

- Empty-state message extended from "No streams in this group yet." to "No streams in this group yet. Run an M3U refresh first to populate streams."
2026-05-03 16:53:57 -05:00
None
df7e6422df perf: eliminate three remaining N+1 queries in list endpoints
Layered on top of the prior perf commits, removes additional per-row queries

Channel list (apps/channels): add auto_created_by to select_related so ChannelSerializer.get_auto_created_by_name does not fire one SELECT per row. Replace instance.streams.all().order_by(channelstream__order) in to_representation with a read off the prefetched channelstream_set.

M3U account list (apps/m3u): prefetch_related "filters" on the viewset queryset and sort filters in Python over the prefetch in M3UAccountSerializer.get_filters.

Frontend (EditableCell.jsx): align the saveValue useCallback dependency from [row.original.id] to [row.original] in the Number, Group, EPG, and Logo cells, matching the pattern applied to the Text cell.
2026-05-02 16:48:37 -05:00
SergeantPanda
b470ae68db perf(ChannelGroupViewSet): optimize list method to reduce SQL queries by pre-evaluating queryset 2026-05-02 12:00:06 -05:00
SergeantPanda
133a15496a perf: optimize ChannelGroup and Channel viewsets for efficient querying and serialization 2026-05-02 11:49:49 -05:00
None
b2b108bfe5 rename Channel.user_hidden to Channel.hidden_from_output
The previous name suggested "hidden from a specific user account" rather than "hidden from downstream client output", which was the actual
behavior. The new name is unambiguous and matches the help_text language ("Exclude this channel from downstream client output").

Mechanical rename across the migration, model, serializer, signals, sync logic, output endpoints, frontend forms / tables / utils, tests, and CHANGELOG. 77 occurrences across 20 files.

No behavior change.
2026-05-01 19:44:16 -05:00
None
6fed444e43 feat: Auto-sync overhaul (FR #1196)
Per-field channel overrides for auto-synced channels, hide-from-output flag, range-bounded auto-numbering with re-pack, multi-stream channel safety, multi-provider shared-range merging, and an across-the-board move from per-row sync writes to bulk operations.

Migrations:

apps/channels:

0036_channeloverride_and_user_hidden,
0037_backfill_auto_created_by_null,
0038_channelgroupm3uaccount_auto_sync_channel_end,
0039_channel_channel_number_nullable

apps/m3u:
0020_m3uaccount_auto_cleanup_unused_channels

See CHANGELOG.md for the full commit log
2026-05-01 14:05:06 -05:00
SergeantPanda
a80af16e6a Enhancement: Implement targeted stream-stats refresh for channel table using a new lightweight delta endpoint 2026-04-30 17:46:49 -05:00
SergeantPanda
a7b8e54747 Enhancement: Refactor HLS handling in RecordingViewSet and implement cleanup for empty parent directories 2026-04-26 13:27:32 -05:00
SergeantPanda
0cca34a2dc Enhancement: Implement user authorization for recording playback and inject JWT into HLS requests 2026-04-26 12:25:05 -05:00
SergeantPanda
914c612db9 Enhancement: Implement network access checks for streaming and update recording status handling during worker shutdown 2026-04-26 11:40:46 -05:00
SergeantPanda
14a6bf4473 Initial DVR HLS refactor. 2026-04-26 09:06:48 -05:00
SergeantPanda
452a493df3 Performance: Fixed N+1 UPDATE queries in the stream-order write path 2026-04-19 09:55:56 -05:00
SergeantPanda
c4972f7dc9 fix: clear previous failure entries after successful logo fetch in LogoViewSet and VODLogoViewSet 2026-04-14 13:34:17 -05:00
SergeantPanda
978714a74d fix: improve remote logo fetching with timeout and caching for failures 2026-04-14 13:04:45 -05:00
SergeantPanda
f1a82e8843 Bug Fix: Fixed several incorrect or incomplete OpenAPI (@extend_schema) schemas across the API
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
2026-04-11 17:02:07 -05:00
SergeantPanda
1adbc70f53 Bug Fix: Fix bulk channel edit API when including streams array. (Fixes #883) 2026-04-11 16:11:45 -05:00
SergeantPanda
7083c512be security: Fixed path traversal vulnerability in file uploads. The M3U account upload (apps/m3u/api_views.py), logo upload (apps/channels/api_views.py), and backup upload (apps/backups/api_views.py) all used the uploaded filename directly without sanitization. os.path.join() discards all preceding components when it encounters an absolute path segment, and pathlib's / operator behaves identically; a relative ../ sequence also escapes via OS path resolution at open() time. All three upload paths now strip directory components via Path(name).name and validate the resolved path remains within the intended upload directory. Exploiting any of these required admin credentials. 2026-04-09 21:32:36 -05:00
dekzter
90a600b48f merged in dev 2026-03-26 12:17:43 -04:00
SergeantPanda
b4281c83bb Switch to backend logic instead of frontend. Use '-1' as 'highest' so not break the api. 2026-03-17 09:27:18 -05:00
dekzter
f69a462253 merged in dev 2026-03-13 08:22:44 -04:00
SergeantPanda
39865d3732 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/CodeBormen/943 2026-03-08 18:04:11 -05:00
None
68de1b0723 fix(dvr): add DVR search/filters, fix in-progress scheduling, prevent duplicate recordings
- Add search, channel, and status filter controls to DVR page (matching Guide/VODs pattern)
- Fix post_save signal to schedule recordings for currently-playing programs immediately instead of skipping them
- Remove duplicate evaluate_series_rules.delay() call from series rule creation endpoint to prevent race condition that created two recordings for the same program
2026-03-04 10:21:03 -06:00
None
759688390b fix(dvr): fix series episode list, card layout, rule cleanup, and recording resilience
Backend:
- Add terminal status guard to stop() endpoint -prevents overwriting completed/interrupted/failed with "stopped" (returns 409 Conflict)
- Include currently-airing programs in series rule evaluation by filtering on end_time instead of start_time
- Clean up future recordings when deleting a series rule while preserving previously recorded episodes
- Add logo proxy negative cache to prevent repeated requests to unreachable hosts from blocking Daphne workers
- Add URL validation with HEAD-then-GET fallback and per-worker cache
- Add EPG time-slot matching (80% overlap threshold) to enrich manual recordings with program metadata
- Add multi-source poster resolution pipeline (EPG, VOD, TMDB, OMDb, TVMaze, iTunes) with URL validation at each stage
- Add editable recording metadata endpoint with user_edited flag to prevent EPG auto-overwrite
- Add refresh-artwork endpoint to re-run poster resolution on demand
- Batch-fetch PeriodicTask names in schedule_upcoming to avoid N+1

Frontend:
- Fix series modal showing only one episode — preserve _group_count from categorized prop when merging with store version so the episode list renders correctly
- Fix recording card button positioning — use flex column layout with marginTop auto so buttons align at the bottom regardless of content
- Add delete fallback in RecurringRuleModal for orphaned recordings from deleted rules
- Use end_time filter in upcoming episodes list to include currently-airing episodes
- Add inline metadata editing (title/description) in details modal
- Add refresh artwork button in details modal
- Replace hardcoded /logo.png fallbacks with imported default logo
- Add React.memo with custom comparator to RecordingCard

Tests:
- Add 21 EPG time-slot matching tests (overlap threshold, edge cases)
- Add 16 recording metadata tests (validation, artwork, logo cache)
- Add 17 URL validation tests (HEAD/GET fallback, caching, eviction)
2026-03-04 10:21:03 -06:00
None
f28530a47b fix(dvr): add recording extend, fix cross-recording interference, artwork race, and keepalive gaps
File changes:

Backend

apps/channels/api_views.py — Added extend() action that uses queryset .update() to bypass signals, letting the running task pick up the new end_time via polling. Moved WS recording_stopped notification to synchronous (before HTTP response). Refactored destroy() to delete DB row first, send WS event immediately, defer cleanup to background thread.

apps/channels/signals.py — Added pre_save guard to skip task revocation when status is "recording". Added post_save reentrancy guard to skip artwork prefetch for internal field-only saves (custom_properties, task_id, end_time).

apps/channels/tasks.py — Added 2-second polling loop in streaming that checks for stop, deletion, and extended end_time. Added refresh_from_db() + merge before metadata save to prevent overwriting concurrent artwork prefetch. Replaced inline PeriodicTask cleanup with revoke_task()/_dvr_task_name(). Consolidated redundant DB queries in error path. Fixed TOCTOU os.path.exists + os.remove. Preserved "stopped" status in final-status logic instead of overwriting with "completed".

core/utils.py — Made send_websocket_update gevent-safe by detecting monkey-patching and offloading async_to_sync to gevent's native threadpool, fixing the event loop deadlock that caused cross-recording interference.

apps/proxy/ts_proxy/client_manager.py — Moved _trigger_stats_update to a background thread so remove_client() returns immediately.

apps/proxy/ts_proxy/stream_generator.py — Added Redis-based health check for non-owner workers in _should_send_keepalive(), so keepalives fire correctly even when stream_manager is None.

Frontend

frontend/src/WebSocket.jsx — Added 400ms debounced scheduleRecordingFetch() replacing all inline fetchRecordings() calls. Added recording_extended and recording_updated event handlers. recording_cancelled now does surgical removeRecording() by ID when available.

frontend/src/api.js — Added extendRecording(id, extraMinutes).

frontend/src/components/cards/RecordingCard.jsx — Added Extend menu (+15m/+30m/+1h) for in-progress recordings. Widened Comskip button to stopped/interrupted statuses.

frontend/src/components/forms/ProgramRecordingModal.jsx — Removed manual fetchRecordings() calls, relying on WS-driven debounced refresh.

frontend/src/components/forms/RecordingDetailsModal.jsx — Added stopped to playable statuses. Widened Comskip button. Removed manual fetchRecordings().

frontend/src/components/forms/RecurringRuleModal.jsx — Removed all manual fetchRecordings() calls, relying on WS events.

frontend/src/pages/Guide.jsx — Removed isLoading subscription that caused loading flash on debounced refresh. Removed manual fetchRecordings() after saving a series rule.

frontend/src/store/channels.jsx — Stripped isLoading/error state from fetchRecordings to prevent full-page loading indicators. Added no-op guards in removeRecording.

frontend/src/utils/cards/RecordingCardUtils.js — Added extendRecordingById wrapper.

Tests

apps/channels/tests/test_recording_extend.py (new) — 14 tests: extend endpoint validation, stacked extensions, signal bypass, pre_save guard behavior.

apps/channels/tests/test_recording_stop_cancel.py (new) — 18 tests: stop endpoint, DVR client teardown, cancel was_in_progress flag, run_recording race guards, signal reentrancy guards.

apps/channels/tests/test_ts_proxy_keepalive.py (new) — 21 tests: keepalive owner/non-owner logic, stats update error handling, client removal non-blocking, timing invariants.

apps/channels/tests/test_recording_scheduling.py — Updated mock assertion to match new _stop_dvr_clients call signature.

Summary:

Adds an Extend action for in-progress recordings that bypasses Django signals and lets the running task dynamically pick up the new end_time via its 2-second DB polling loop. Fixes cross-recording interference caused by send_websocket_update deadlocking the gevent event loop. Fixes an artwork race condition where run_recording overwrote concurrent prefetch data. Adds Redis-based keepalive health checks for non-owner workers to prevent DVR read timeouts during source transitions. Replaces manual fetchRecordings() calls throughout the frontend with a debounced WS-driven refresh. 53 new tests.
2026-03-04 10:21:02 -06:00
None
bd1e060bd2 fix(dvr): fix duplicate recording execution and add manual stop
Fixes #940: Duplicate DVR task executions were caused by ClockedSchedule objects being shared across PeriodicTasks, triggering run_recording multiple
times for the same recording. Added an idempotency guard in run_recording that exits early if status is already 'recording', 'completed', or 'stopped'.

revoke_task() now deletes the PeriodicTask and orphaned ClockedSchedule on execution rather than relying on Celery revocation alone.

Fixes #454: Added POST /api/channels/recordings/{id}/stop/ action that marks the recording as stopped and terminates only the specific DVR proxy client (identified via User-Agent: Dispatcharr-DVR/recording-{id}) without disrupting
simultaneous recordings on the same or other channels. The API returns
immediately; DVR client teardown and task revocation happen in a background thread to avoid 504 timeouts on the recordings list endpoint.

Additional changes:
- destroy() now only calls _stop_dvr_clients() for in-progress recordings, preventing accidental stream termination when deleting completed recordings
- WebSocket events differentiated: recording_stopped, recording_cancelled (in-progress cancel), and recording_cancelled (delete) with was_in_progress flag so the frontend shows distinct "Recording deleted" vs "Recording cancelled" notifications

- Frontend: Stop and Cancel split into separate buttons with confirmation
  modals; stopped recordings move immediately to Previously Recorded via
  optimistic status bucketing; removed dead 'Stopped' badge state

- 27 new unit tests covering scheduling, idempotency, revocation, and DVR client isolation
2026-03-04 10:21:02 -06:00
SergeantPanda
c005a48162 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/JCBird1012/1027 2026-03-03 09:15:31 -06:00
SergeantPanda
b098deae76 Got rid of extra api call to query if any channels had no EPG assigned. Moved query to an EXISTS lookup during channel query. 2026-03-02 17:20:55 -06:00
Jonathan Caicedo
a318d919d8 feat: filter channels that contain stale streams
closes: #1025
2026-02-28 12:30:21 -05:00
dekzter
6f4665e6eb merged in dev 2026-02-26 08:20:20 -05:00
SergeantPanda
7db6aded22 add summary endpoint and lightweight channel summary retrieval for TV Guide 2026-02-18 11:51:41 -06:00
dekzter
f1d6b86278 refactored frontend usage of 'all channels' to only pull down IDs, query for individual channels as necessary 2026-02-13 11:46:17 -05:00
None
a170663407 feat: Add sorting by Group and EPG columns to Channels and Streams tables
Changes:
- Backend: Add epg_data__name to ChannelViewSet ordering_fields
- ChannelsTable: Add sort icons to EPG and Group column headers using rightSection prop
- ChannelsTable: Add field mapping for channel_group → channel_group__name and epg → epg_data__name
- StreamsTable: Add sort icon to Group column header for consistency

Adds the ability to sort channels by Group and EPG columns, and streams by Group column.

Closes #854
2026-02-04 21:09:35 -06:00
SergeantPanda
efe915a9b3
Merge pull request #904 from CodeBormen/Feature/866-Search-Filter-Sort-streams-by-TVG-ID
Feature/866 Add TVG-ID column with filter and sort to Streams table
2026-01-31 19:31:48 -06:00
SergeantPanda
9ffd595c96 Enhancement: Added stream_id (provider stream identifier) and stream_chno (provider channel number) fields to Stream model. For XC accounts, the stream hash now uses the stable stream_id instead of the URL when hashing, ensuring XC streams maintain their identity and channel associations even when account credentials or server URLs change. Supports both XC num and M3U tvg-chno/channel-number attributes. 2026-01-31 17:47:46 -06:00
None
f8ec970561 Add TVG-ID column with filter and sort to Streams table
Changes:

- frontend/src/components/tables/StreamsTable.jsx

Added TVG-ID column to streams table
Added search filter input in column header
Added sort button for ascending/descending order
Column follows same pattern as Name, Group, and M3U columns

- apps/channels/api_views.py

Added TVG-ID to filterable fields
Added TVG-ID to sortable fields
Enables backend support for search and sorting

Users can now view, search, and sort streams by their TVG-ID directly in the Streams table. Fulfills #866
2026-01-30 23:37:39 -06:00
SergeantPanda
5364123745 - Swagger/OpenAPI Migration: Migrated from drf-yasg (OpenAPI 2.0) to drf-spectacular (OpenAPI 3.0) for API documentation. This provides:
- Native Bearer token authentication support in Swagger UI - users can now enter just the JWT token and the "Bearer " prefix is automatically added
  - Modern OpenAPI 3.0 specification compliance
  - Better auto-generation of request/response schemas
  - Improved documentation accuracy with serializer introspection
2026-01-27 13:33:33 -06:00
SergeantPanda
2abe07fe1c Update Series Rules API Swagger Documentation to include required items parameter for TYPE_ARRAY schemas 2026-01-27 10:56:04 -06:00
SergeantPanda
497bb819b9 Docs: Added comprehensive Swagger/OpenAPI documentation for all series-rules endpoints 2026-01-27 09:50:27 -06:00
SergeantPanda
655760f652 When moving a channel down (higher number) don't increase channel numbers, decrease.
Some checks failed
CI Pipeline / prepare (push) Has been cancelled
Build and Push Multi-Arch Docker Image / build-and-push (push) Has been cancelled
Frontend Tests / test (push) Has been cancelled
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Has been cancelled
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
CI Pipeline / create-manifest (push) Has been cancelled
2026-01-23 18:10:16 -06:00
SergeantPanda
e3122b1a1b Bug Fix: Fix reordering during drag and drop. 2026-01-23 17:31:46 -06:00