Commit graph

67 commits

Author SHA1 Message Date
None
9d08305dcb fix(dvr): fix recording modal crash, harden recovery pipeline, improve tests
Frontend:
- RecordingDetailsModal: fix ReferenceError crash (editing state used before declaration in useMemo — TDZ in production bundles broke all tile clicks)
- RecordingDetailsModal: fix focus-stealing on description edit by calling Movie()/Series() as plain functions instead of <Movie />/<Series />(inline component refs change every render, causing unmount/remount)
- RecordingDetailsModal: suppress sub_title in modal title after custom save
- ErrorBoundary: log caught errors and show message for easier debugging
- guideUtils: filter terminal recordings from mapRecordingsByProgramId so Guide red dots clear after recording finishes or rule is deleted
- RecordingDetailsModalUtils.test: add missing end_time to test fixtures (filterByUpcoming now uses end_time instead of start_time)

Backend:
- tasks.py: output file collision avoidance (check .mkv/.ts before writing)
- tasks.py: pre-restart TS segment concatenation (direct concat→MKV remux)
- tasks.py: recovery pipeline — recover crashed "recording" status, remux expired recordings, revoke stale tasks, deterministic task IDs, early lock release, 10-min lock TTL for large remux operations
- tasks.py: poster resolver skips external API when title matches channel name
- celery.py: trigger recover_recordings_on_startup via worker_ready signal

Tests:
- Renamed test_dvr_fixes.py → test_recording_pipeline.py (clarity)
- Renamed test_dvr_retry.py → test_db_retry.py (clarity), fixed mock paths for close_old_connections (resolved 3 test failures), replaced synthetic test
- test_recording_stop_cancel: replaced brittle time.sleep(0.5) with deterministic thread-capture pattern
2026-03-04 10:21:03 -06:00
None
bd14e48f8b fix(dvr): harden recording pipeline — idempotency guard, remux sanity checks, timezone fix
- Idempotency guard: fail closed on DB errors instead of proceeding, preventing duplicate tasks from overwriting valid recordings (#641)
- Remux sanity checks: reject MKV output that is <50% of a previous MKV (duplicate-task overwrite) or <10% of the source TS (corrupt first attempt); preserve .ts for manual recovery on all failure paths (#619)
- Recurring rule timezone fix: use user's configured timezone for date window calculation instead of UTC, fixing silent recording loss for users in UTC-negative timezones after 4pm local time (#1042)
- Include .ts path in final remux failure log for easier recovery
2026-03-04 10:21:03 -06:00
None
759688390b fix(dvr): fix series episode list, card layout, rule cleanup, and recording resilience
Backend:
- Add terminal status guard to stop() endpoint -prevents overwriting completed/interrupted/failed with "stopped" (returns 409 Conflict)
- Include currently-airing programs in series rule evaluation by filtering on end_time instead of start_time
- Clean up future recordings when deleting a series rule while preserving previously recorded episodes
- Add logo proxy negative cache to prevent repeated requests to unreachable hosts from blocking Daphne workers
- Add URL validation with HEAD-then-GET fallback and per-worker cache
- Add EPG time-slot matching (80% overlap threshold) to enrich manual recordings with program metadata
- Add multi-source poster resolution pipeline (EPG, VOD, TMDB, OMDb, TVMaze, iTunes) with URL validation at each stage
- Add editable recording metadata endpoint with user_edited flag to prevent EPG auto-overwrite
- Add refresh-artwork endpoint to re-run poster resolution on demand
- Batch-fetch PeriodicTask names in schedule_upcoming to avoid N+1

Frontend:
- Fix series modal showing only one episode — preserve _group_count from categorized prop when merging with store version so the episode list renders correctly
- Fix recording card button positioning — use flex column layout with marginTop auto so buttons align at the bottom regardless of content
- Add delete fallback in RecurringRuleModal for orphaned recordings from deleted rules
- Use end_time filter in upcoming episodes list to include currently-airing episodes
- Add inline metadata editing (title/description) in details modal
- Add refresh artwork button in details modal
- Replace hardcoded /logo.png fallbacks with imported default logo
- Add React.memo with custom comparator to RecordingCard

Tests:
- Add 21 EPG time-slot matching tests (overlap threshold, edge cases)
- Add 16 recording metadata tests (validation, artwork, logo cache)
- Add 17 URL validation tests (HEAD/GET fallback, caching, eviction)
2026-03-04 10:21:03 -06:00
None
cf3165ea88 fix(dvr): add DB retry resilience, fix recording card logos, prevent incorrect artwork
Add exponential backoff retry (_db_retry helper) for transient DB errors during final metadata save, startup recovery, and initial TS proxy
connection. Add FFmpeg remux retry for transient I/O errors. Retry same proxy base before falling back to next candidate on initial connection failure.

Fix DVR card logos not displaying: channel summary API returns logo_id (integer) but frontend expected a nested logo object with cache_url.

Add getChannelLogoUrl() helper that handles both data shapes. Backend artwork prefetch and run_recording now fall back to the channel's own logo when no show-specific artwork is found, ensuring poster_logo_id is always populated.

Prevent incorrect artwork by skipping external API searches (TVMaze, iTunes) when no program title is available - channel names like
"USA A&E SD*" returned unrelated results from fuzzy matching.

Add 13 tests for retry logic (test_dvr_retry.py)
2026-03-04 10:21:03 -06:00
None
e5206545dd fix(dvr): add stream reconnection resilience, fix comskip exit code handling
- Add reconnection loop to DVR streaming: on transient connectivity loss (ReadTimeout, ConnectionError, ChunkedEncodingError), the recording task retries the same TS proxy base up to 5 times with a 2-second delay, appending to the existing file. Counter resets on successful data receipt. TS container format tolerates brief discontinuities; MKV remux normalises timestamps.
- Extract _check_recording_cancelled() helper to consolidate repeated stop/delete DB checks across the streaming and reconnection paths.
- Fix comskip treating exit code 1 (no commercials detected) as a fatal error.
  Use check=False and handle return codes explicitly: 0 = commercials found,
  1 = no commercials (skipped), negative = signal, other = real error.
- Add human-readable reason messages for comskip 'skipped' notifications in
  WebSocket handler.
- Remove redundant conditional Channel import in stream_generator._cleanup() that shadowed the module-level import, causing UnboundLocalError when the conditional branch did not execute.
2026-03-04 10:21:02 -06:00
None
f28530a47b fix(dvr): add recording extend, fix cross-recording interference, artwork race, and keepalive gaps
File changes:

Backend

apps/channels/api_views.py — Added extend() action that uses queryset .update() to bypass signals, letting the running task pick up the new end_time via polling. Moved WS recording_stopped notification to synchronous (before HTTP response). Refactored destroy() to delete DB row first, send WS event immediately, defer cleanup to background thread.

apps/channels/signals.py — Added pre_save guard to skip task revocation when status is "recording". Added post_save reentrancy guard to skip artwork prefetch for internal field-only saves (custom_properties, task_id, end_time).

apps/channels/tasks.py — Added 2-second polling loop in streaming that checks for stop, deletion, and extended end_time. Added refresh_from_db() + merge before metadata save to prevent overwriting concurrent artwork prefetch. Replaced inline PeriodicTask cleanup with revoke_task()/_dvr_task_name(). Consolidated redundant DB queries in error path. Fixed TOCTOU os.path.exists + os.remove. Preserved "stopped" status in final-status logic instead of overwriting with "completed".

core/utils.py — Made send_websocket_update gevent-safe by detecting monkey-patching and offloading async_to_sync to gevent's native threadpool, fixing the event loop deadlock that caused cross-recording interference.

apps/proxy/ts_proxy/client_manager.py — Moved _trigger_stats_update to a background thread so remove_client() returns immediately.

apps/proxy/ts_proxy/stream_generator.py — Added Redis-based health check for non-owner workers in _should_send_keepalive(), so keepalives fire correctly even when stream_manager is None.

Frontend

frontend/src/WebSocket.jsx — Added 400ms debounced scheduleRecordingFetch() replacing all inline fetchRecordings() calls. Added recording_extended and recording_updated event handlers. recording_cancelled now does surgical removeRecording() by ID when available.

frontend/src/api.js — Added extendRecording(id, extraMinutes).

frontend/src/components/cards/RecordingCard.jsx — Added Extend menu (+15m/+30m/+1h) for in-progress recordings. Widened Comskip button to stopped/interrupted statuses.

frontend/src/components/forms/ProgramRecordingModal.jsx — Removed manual fetchRecordings() calls, relying on WS-driven debounced refresh.

frontend/src/components/forms/RecordingDetailsModal.jsx — Added stopped to playable statuses. Widened Comskip button. Removed manual fetchRecordings().

frontend/src/components/forms/RecurringRuleModal.jsx — Removed all manual fetchRecordings() calls, relying on WS events.

frontend/src/pages/Guide.jsx — Removed isLoading subscription that caused loading flash on debounced refresh. Removed manual fetchRecordings() after saving a series rule.

frontend/src/store/channels.jsx — Stripped isLoading/error state from fetchRecordings to prevent full-page loading indicators. Added no-op guards in removeRecording.

frontend/src/utils/cards/RecordingCardUtils.js — Added extendRecordingById wrapper.

Tests

apps/channels/tests/test_recording_extend.py (new) — 14 tests: extend endpoint validation, stacked extensions, signal bypass, pre_save guard behavior.

apps/channels/tests/test_recording_stop_cancel.py (new) — 18 tests: stop endpoint, DVR client teardown, cancel was_in_progress flag, run_recording race guards, signal reentrancy guards.

apps/channels/tests/test_ts_proxy_keepalive.py (new) — 21 tests: keepalive owner/non-owner logic, stats update error handling, client removal non-blocking, timing invariants.

apps/channels/tests/test_recording_scheduling.py — Updated mock assertion to match new _stop_dvr_clients call signature.

Summary:

Adds an Extend action for in-progress recordings that bypasses Django signals and lets the running task dynamically pick up the new end_time via its 2-second DB polling loop. Fixes cross-recording interference caused by send_websocket_update deadlocking the gevent event loop. Fixes an artwork race condition where run_recording overwrote concurrent prefetch data. Adds Redis-based keepalive health checks for non-owner workers to prevent DVR read timeouts during source transitions. Replaces manual fetchRecordings() calls throughout the frontend with a debounced WS-driven refresh. 53 new tests.
2026-03-04 10:21:02 -06:00
None
bd1e060bd2 fix(dvr): fix duplicate recording execution and add manual stop
Fixes #940: Duplicate DVR task executions were caused by ClockedSchedule objects being shared across PeriodicTasks, triggering run_recording multiple
times for the same recording. Added an idempotency guard in run_recording that exits early if status is already 'recording', 'completed', or 'stopped'.

revoke_task() now deletes the PeriodicTask and orphaned ClockedSchedule on execution rather than relying on Celery revocation alone.

Fixes #454: Added POST /api/channels/recordings/{id}/stop/ action that marks the recording as stopped and terminates only the specific DVR proxy client (identified via User-Agent: Dispatcharr-DVR/recording-{id}) without disrupting
simultaneous recordings on the same or other channels. The API returns
immediately; DVR client teardown and task revocation happen in a background thread to avoid 504 timeouts on the recordings list endpoint.

Additional changes:
- destroy() now only calls _stop_dvr_clients() for in-progress recordings, preventing accidental stream termination when deleting completed recordings
- WebSocket events differentiated: recording_stopped, recording_cancelled (in-progress cancel), and recording_cancelled (delete) with was_in_progress flag so the frontend shows distinct "Recording deleted" vs "Recording cancelled" notifications

- Frontend: Stop and Cancel split into separate buttons with confirmation
  modals; stopped recordings move immediately to Previously Recorded via
  optimistic status bucketing; removed dead 'Stopped' badge state

- 27 new unit tests covering scheduling, idempotency, revocation, and DVR client isolation
2026-03-04 10:21:02 -06:00
None
d7b98fef8d Add default/advanced mode toggle and test coverage for EPG matching
Enhanced EPG matching UX by adding radio button toggle to more clearly define optional advanced options. Also added test coverage for stability.

Frontend Changes:
- Added radio button UI to EPGMatchModal for default/advanced mode selection
- Default mode: Uses built-in normalization
- Advanced mode: Shows TagsInput fields for custom prefixes/suffixes/strings
- Mode persists across sessions and settings are preserved when switching modes

Test Coverage:
- Created EPGMatchModal.test.jsx with test cases covering:
  - Component rendering and mode switching
  - Form submission and settings persistence
  - Error handling and UI text variations

- Added test cases to SettingsUtils.test.js for EPG mode handling:
  - epg_match_mode routing to epg_settings group
  - Settings creation and preservation

Since advanced settings are saved persistently but ignored when default settings are used, this adds an easy way in the future to add additional advanced settings to EPG matching like  region influence, match aggression, and match preview, since users can easily switch between modes.
2026-01-31 23:11:20 -06:00
SergeantPanda
0172a7cc00 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/CodeBormen/894 2026-01-31 19:47:57 -06:00
SergeantPanda
9ffd595c96 Enhancement: Added stream_id (provider stream identifier) and stream_chno (provider channel number) fields to Stream model. For XC accounts, the stream hash now uses the stable stream_id instead of the URL when hashing, ensuring XC streams maintain their identity and channel associations even when account credentials or server URLs change. Supports both XC num and M3U tvg-chno/channel-number attributes. 2026-01-31 17:47:46 -06:00
Matt Grutza
a1fa737ccd Update verbiage 2026-01-25 22:08:32 -06:00
Matt Grutza
8d69fd9374 Add configurable EPG matching normalization settings (Feature #771)
Implements user-configurable string removal for EPG matching to improve channel-to-EPG
association accuracy. Settings are non-destructive and only affect EPG matching
normalization, never modifying actual channel display names.
2026-01-25 22:05:28 -06:00
SergeantPanda
0b83137ac6 Enhancement: DVR recording remux fallback strategy: Implemented two-stage TS→MP4→MKV fallback when direct TS→MKV conversion fails due to timestamp issues. On remux failure, system now attempts TS→MP4 conversion (MP4 container handles broken timestamps better) followed by MP4→MKV conversion, automatically recovering from provider timestamp corruption. Failed conversions now properly clean up partial files and preserve source TS for manual recovery.
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-01-19 18:56:23 -06:00
SergeantPanda
d33d047a94 Enhancement: - Mature content filtering support:
- Added `is_adult` boolean field to both Stream and Channel models with database indexing for efficient filtering and sorting
  - Automatically populated during M3U/XC refresh operations by extracting `is_adult` value from provider data
  - Type-safe conversion supporting both integer (0/1) and string ("0"/"1") formats from different providers
  - UI controls in channel edit form (Switch with tooltip) and bulk edit form (Select dropdown) for easy management
  - XtreamCodes API support with proper integer formatting (0/1) in live stream responses
  - Automatic propagation from streams to channels during both single and bulk channel creation operations
  - Included in serializers for full API support
  - User-level content filtering: Non-admin users can opt to hide mature content channels across all interfaces (web UI, M3U playlists, EPG data, XtreamCodes API) via "Hide Mature Content" toggle in user settings (stored in custom_properties, admin users always see all content)
2026-01-17 15:00:28 -06:00
SergeantPanda
edfa497203 Enhancement: Channel Profile membership control for manual channel creation and bulk operations: Extended the existing channel_profile_ids parameter from POST /api/channels/from-stream/ to also support POST /api/channels/ (manual creation) and bulk creation tasks with the same flexible semantics:
- Omitted parameter (default): Channels are added to ALL profiles (preserves backward compatibility)
  - Empty array `[]`: Channels are added to NO profiles
  - Sentinel value `[0]`: Channels are added to ALL profiles (explicit)
  - Specific IDs `[1, 2, ...]`: Channels are added only to the specified profiles
  This allows API consumers to control profile membership across all channel creation methods without requiring all channels to be added to every profile by default.
2026-01-11 17:31:15 -06:00
SergeantPanda
759569b871 Enhancement: Add a priority field to EPGSource and prefer higher-priority sources when matching channels. Also ignore EPG sources where is_active is false during matching, and update serializers/forms/frontend accordingly.(Closes #603, #672) 2025-12-05 09:54:11 -06:00
SergeantPanda
89a23164ff Enhancement: Add system event logging and viewer with M3U/EPG endpoint caching
System Event Logging:
- Add SystemEvent model with 15 event types tracking channel operations, client connections, M3U/EPG activities, and buffering events
- Log detailed metrics for M3U/EPG refresh operations (streams/programs created/updated/deleted)
- Track M3U/EPG downloads with client information (IP address, user agent, profile, channel count)
- Record channel lifecycle events (start, stop, reconnect) with stream and client details
- Monitor client connections/disconnections and buffering events with stream metadata

Event Viewer UI:
- Add SystemEvents component with real-time updates via WebSocket
- Implement pagination, filtering by event type, and configurable auto-refresh
- Display events with color-coded badges and type-specific icons
- Integrate event viewer into Stats page with modal display
- Add event management settings (retention period, refresh rate)

M3U/EPG Endpoint Optimizations:
- Implement content caching with 5-minute TTL to reduce duplicate processing
- Add client-based event deduplication (2-second window) using IP and user agent hashing
- Support HEAD requests for efficient preflight checks
- Cache streamed EPG responses while maintaining streaming behavior for first request
2025-11-20 17:41:06 -06:00
SergeantPanda
3326b9fbdc Bug fix: Add logo URL validation to prevent PostgreSQL btree index errors during bulk channel creation
Fixes #519
2025-10-06 21:11:53 -05:00
OkinawaBoss
8db9689999
Merge branch 'DVR-Update' into dev 2025-10-06 06:25:40 -07:00
Dispatcharr
dea6411e1c Time Zones
- Added time zone settings
2025-10-06 07:46:23 -05:00
SergeantPanda
d1ac5b11e5 Add EPG TVG-ID setting functionality
- Implemented API endpoint to set channel TVG-IDs from EPG data.
- Created Celery task to handle TVG-ID updates for multiple channels.
- Added frontend methods to initiate TVG-ID setting from EPG for both single and batch channel forms.
- Enhanced notifications for task status updates.
2025-10-04 15:20:35 -05:00
SergeantPanda
c73271c617 Bug Fix: Sort order while bulk creating channels will now be in the order they were selected in not reverse order. 2025-10-02 13:23:24 -05:00
SergeantPanda
23f397c805 Enhancement: Add exact Gracenote ID matching to EPG channel mapping 2025-09-25 11:51:10 -05:00
Dispatcharr
6536f35dc0 FIxed bug
Fixed bug that stopped stream from ending
2025-09-19 19:47:59 -05:00
Dispatcharr
eee4ab0725 Update for recurring rules 2025-09-19 16:21:28 -05:00
Dispatcharr
424a450654 DVR Features and bug fixes
Added ability to use custom comskip.ini
Added series recording without reliance on EPG
Fixed comskip bug
Fixed timezone mismatch when scheduling DVR recordings

No migrations completed yet
2025-09-18 10:23:16 -05:00
SergeantPanda
d2d1984797 Switch bulk epg name and logo to backend celery tasks for efficiency and scrape epg channel logo during epg scanning. 2025-09-16 17:17:07 -05:00
SergeantPanda
a846b09ad3 Minor formatting adjustment. 2025-09-16 14:39:04 -05:00
SergeantPanda
60e378b1ce Add support for matching selected channels with EPG data
- Updated API to accept optional channel IDs for EPG matching.
- Enhanced match_epg method to process only specified channels if provided.
- Implemented new task for matching selected channels in the backend.
- Updated frontend to trigger EPG matching for selected channels with notifications.
2025-09-16 14:38:16 -05:00
SergeantPanda
20685b8344 Lower regional bonus. Remove epg_match script. 2025-09-16 14:27:07 -05:00
SergeantPanda
c7235f66ba Use stricter matching for bulk matching. 2025-09-16 14:12:45 -05:00
SergeantPanda
6384f4f56f Add progress notifications for EPG matching process 2025-09-16 13:47:59 -05:00
SergeantPanda
d6bb9e40b2 Implement memory cleanup for ML models after channel matching operations 2025-09-16 13:15:32 -05:00
SergeantPanda
c55dcfd26a Remove unnecessary checking of cache directories. Lets sentence transformers handle it. 2025-09-16 13:01:43 -05:00
SergeantPanda
fedc98f848 Removed unneeded debug logging. 2025-09-16 12:54:19 -05:00
SergeantPanda
d2085d57f8 Add sentence transformers to new matching function. 2025-09-16 12:43:21 -05:00
SergeantPanda
f6be6bc3a9 Don't use matching script 2025-09-16 09:18:41 -05:00
SergeantPanda
f1739f2394 Add EPG auto-match functionality for specific channels and update UI 2025-09-16 08:55:10 -05:00
SergeantPanda
4886426ea0 New feature: Allow specifying channel number during channel creation.
Closes #377
Closes #169 [Feature]: Starting channel number/continue auto numbering from channel number X (user defined)
2025-09-14 12:46:30 -05:00
SergeantPanda
0411fe003a Add ability to specify starting channel numbers to the api. 2025-09-14 12:30:24 -05:00
SergeantPanda
cf7ea09341 Implement asynchronous bulk channel creation from stream IDs with WebSocket progress updates 2025-09-13 19:59:43 -05:00
SergeantPanda
41d7066d6e Fix incorrect paths for DVR and Plugins. 2025-09-13 11:49:04 -05:00
Dispatcharr
c85316b912 Added DVR offset
Added pre/post offset for DVR recordings.
2025-09-11 20:58:42 -05:00
Dispatcharr
f652d2b233 Comskip Update 2025-09-04 13:45:25 -05:00
Dispatcharr
41e32bc08a DVR Updates
Added fallback settings.
Added subtitles to cards.
Add data volume mount to Docker container.
2025-09-04 08:22:13 -05:00
Dispatcharr
00cc83882a DVR update 2025-09-03 21:35:45 -05:00
xham3
5360f38b14 Fix process communicate() deadlock when epg match data overfill subprocess.PIPE buffer. 2025-06-12 15:34:24 -07:00
SergeantPanda
56be7b7194 Added missing dc dependency for auto-match. 2025-05-20 19:17:01 -05:00
SergeantPanda
7c809931d7 Rewored celery memory cleanup logic. 2025-05-18 20:57:37 -05:00
SergeantPanda
1174e2e0c7 EPG processing enhancements. Celery memory management. 2025-05-17 16:42:37 -05:00