Commit graph

334 commits

Author SHA1 Message Date
SergeantPanda
dc649ffd88 Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00
SergeantPanda
fe9f83a6af tests: Add API tests for series rule CRUD operations and enhance delete functionality for title-only rules 2026-05-05 09:58:02 -05:00
SergeantPanda
8f97688bb1 feat: Refactor DVR pre/post offset retrieval to avoid redundancy in series rule evaluation 2026-05-05 08:56:17 -05:00
SergeantPanda
e6c580c9bd feat: Implement delete series rule functionality with title parameter and update related components 2026-05-05 08:40:05 -05:00
SergeantPanda
0822c9aad2 feat: Make EPG channel optional in series rules and enhance search criteria 2026-05-04 21:34:56 -05:00
SergeantPanda
5940efbddf feat: Enhance series rule evaluation and management
- Added support for customizable title and description matching modes in series rules.
- Implemented a preview feature for series rules to show matching upcoming programs.
- Created a new SeriesRuleEditorModal for editing series rules with improved UI.
- Refactored query parsing logic into reusable functions for better maintainability.
- Updated API to handle new parameters for series rule creation and evaluation.
- Enhanced the ProgramRecordingModal and SeriesRecordingModal to integrate the new rule editor.
- Added pagination for program search results in the API.
2026-05-04 20:45:40 -05:00
None
983a6675c5 ui: scope regex preview to the M3U account
- Scope regex previews to the M3U account being edited

- Empty-state message extended from "No streams in this group yet." to "No streams in this group yet. Run an M3U refresh first to populate streams."
2026-05-03 16:53:57 -05:00
None
df7e6422df perf: eliminate three remaining N+1 queries in list endpoints
Layered on top of the prior perf commits, removes additional per-row queries

Channel list (apps/channels): add auto_created_by to select_related so ChannelSerializer.get_auto_created_by_name does not fire one SELECT per row. Replace instance.streams.all().order_by(channelstream__order) in to_representation with a read off the prefetched channelstream_set.

M3U account list (apps/m3u): prefetch_related "filters" on the viewset queryset and sort filters in Python over the prefetch in M3UAccountSerializer.get_filters.

Frontend (EditableCell.jsx): align the saveValue useCallback dependency from [row.original.id] to [row.original] in the Number, Group, EPG, and Logo cells, matching the pattern applied to the Text cell.
2026-05-02 16:48:37 -05:00
SergeantPanda
b470ae68db perf(ChannelGroupViewSet): optimize list method to reduce SQL queries by pre-evaluating queryset 2026-05-02 12:00:06 -05:00
SergeantPanda
f84ef95d3c perf(serializers): optimize channel group counts with queryset annotation fallback 2026-05-02 11:50:13 -05:00
SergeantPanda
133a15496a perf: optimize ChannelGroup and Channel viewsets for efficient querying and serialization 2026-05-02 11:49:49 -05:00
None
b2b108bfe5 rename Channel.user_hidden to Channel.hidden_from_output
The previous name suggested "hidden from a specific user account" rather than "hidden from downstream client output", which was the actual
behavior. The new name is unambiguous and matches the help_text language ("Exclude this channel from downstream client output").

Mechanical rename across the migration, model, serializer, signals, sync logic, output endpoints, frontend forms / tables / utils, tests, and CHANGELOG. 77 occurrences across 20 files.

No behavior change.
2026-05-01 19:44:16 -05:00
None
b9b3ded71c refactor(migrations): merge channels FR #1196 migrations into single 0037
Consolidates 0037_channeloverride_and_user_hidden, 0038_backfill_auto_created_by_null,
0039_channelgroupm3uaccount_auto_sync_channel_end, and 0040_channel_channel_number_nullable
into a single 0037_auto_sync_overhaul migration.

The merged migration runs operations in dependency-respecting order:
  1. AddField    Channel.user_hidden
  2. CreateModel ChannelOverride
  3. AddField    ChannelGroupM3UAccount.auto_sync_channel_end
  4. RunPython   backfill_auto_created_by_null
  5. AlterField  Channel.channel_number (nullable)
  6. RunPython   noop / reverse_backfill_channel_number_nulls

Also, update CHANGELOG.md and update migration references in test
2026-05-01 18:32:23 -05:00
None
6fed444e43 feat: Auto-sync overhaul (FR #1196)
Per-field channel overrides for auto-synced channels, hide-from-output flag, range-bounded auto-numbering with re-pack, multi-stream channel safety, multi-provider shared-range merging, and an across-the-board move from per-row sync writes to bulk operations.

Migrations:

apps/channels:

0036_channeloverride_and_user_hidden,
0037_backfill_auto_created_by_null,
0038_channelgroupm3uaccount_auto_sync_channel_end,
0039_channel_channel_number_nullable

apps/m3u:
0020_m3uaccount_auto_cleanup_unused_channels

See CHANGELOG.md for the full commit log
2026-05-01 14:05:06 -05:00
SergeantPanda
a80af16e6a Enhancement: Implement targeted stream-stats refresh for channel table using a new lightweight delta endpoint 2026-04-30 17:46:49 -05:00
SergeantPanda
db6755a009 Enhancement: Add database index on Stream.name to improve query performance and reduce disk spill during sorting (Fixes #1209) 2026-04-30 14:28:26 -05:00
SergeantPanda
1e5298a8d3 Bug Fix: Implement Redis locks in run_recording and worker_ready to prevent duplicate ffmpeg processes and ensure single execution across workers 2026-04-30 10:15:03 -05:00
SergeantPanda
1eda68db7a Enhancement: Refactor run_recording to improve directory handling and remove unused cancellation check 2026-04-26 13:27:50 -05:00
SergeantPanda
a7b8e54747 Enhancement: Refactor HLS handling in RecordingViewSet and implement cleanup for empty parent directories 2026-04-26 13:27:32 -05:00
SergeantPanda
0cca34a2dc Enhancement: Implement user authorization for recording playback and inject JWT into HLS requests 2026-04-26 12:25:05 -05:00
SergeantPanda
914c612db9 Enhancement: Implement network access checks for streaming and update recording status handling during worker shutdown 2026-04-26 11:40:46 -05:00
SergeantPanda
14a6bf4473 Initial DVR HLS refactor. 2026-04-26 09:06:48 -05:00
SergeantPanda
69e0cba86e Bug Fix: EPG channel name vlaue too long for type character varying(255) (Fixes #1134)
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-04-21 09:23:35 -05:00
SergeantPanda
452a493df3 Performance: Fixed N+1 UPDATE queries in the stream-order write path 2026-04-19 09:55:56 -05:00
SergeantPanda
c4972f7dc9 fix: clear previous failure entries after successful logo fetch in LogoViewSet and VODLogoViewSet 2026-04-14 13:34:17 -05:00
SergeantPanda
978714a74d fix: improve remote logo fetching with timeout and caching for failures 2026-04-14 13:04:45 -05:00
SergeantPanda
f1a82e8843 Bug Fix: Fixed several incorrect or incomplete OpenAPI (@extend_schema) schemas across the API
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
2026-04-11 17:02:07 -05:00
SergeantPanda
1adbc70f53 Bug Fix: Fix bulk channel edit API when including streams array. (Fixes #883) 2026-04-11 16:11:45 -05:00
SergeantPanda
7083c512be security: Fixed path traversal vulnerability in file uploads. The M3U account upload (apps/m3u/api_views.py), logo upload (apps/channels/api_views.py), and backup upload (apps/backups/api_views.py) all used the uploaded filename directly without sanitization. os.path.join() discards all preceding components when it encounters an absolute path segment, and pathlib's / operator behaves identically; a relative ../ sequence also escapes via OS path resolution at open() time. All three upload paths now strip directory components via Path(name).name and validate the resolved path remains within the intended upload directory. Exploiting any of these required admin credentials. 2026-04-09 21:32:36 -05:00
SergeantPanda
b671a72707 fix: resolve decode_responses migration bugs and user-limit regressions
Fix a wave of bugs introduced by the user-priority branch's migration
from manual .decode('utf-8') calls to decode_responses=True on the
metadata Redis client:

core/utils.py
- Rewrite _init_client: fix SyntaxError at line 138, missing
  redis_password/redis_user params, cls._client corruption when
  decode_responses=False was requested, and dead retry backoff logic

apps/proxy/utils.py
- Fix get_user_limit_settings() → get_user_limits_settings() (2 sites)
- Fix VOD scan key vod_persistent_connections:* → vod_persistent_connection:*
- Fix undefined channel_id in VOD loop (use content_uuid from Redis hash)
- Remove leftover print(active_connections) debug statement
- Refactor manual cursor SCAN while-loops to scan_iter()

apps/channels/tasks.py
- Fix closure bug in _d(): md.get(key) → md.get(bkey) (was reading
  the outer loop variable instead of the local bytes key)

apps/proxy/ts_proxy/server.py
- Replace stale b'init_time', b'total_bytes', b'state' byte-string
  key lookups and .decode() calls with plain string equivalents

apps/proxy/ts_proxy/services/channel_service.py
- Fix ChannelMetadataField.STATE.encode() / .OWNER.encode() →
  .STATE / .OWNER (pre-existing, broke under decoded client)

apps/proxy/ts_proxy/views.py
- Fix ChannelMetadataField.OWNER.encode("utf-8") → .OWNER

apps/proxy/ts_proxy/client_manager.py
- Fix cid.decode('utf-8') in remove_ghost_clients(): smembers()
  already returns str with decode_responses=True
2026-03-26 18:41:42 -05:00
dekzter
90a600b48f merged in dev 2026-03-26 12:17:43 -04:00
SergeantPanda
034f0623bf fix(tasks): handle AttributeError in series rule evaluation lock acquisition and release 2026-03-23 17:48:35 -05:00
None
36a8e92d94 fix(dvr): prevent duplicate recordings after EPG refresh
- Replace ProgramData.id-based dedup with stable (tvg_id, start_time, end_time) key from Recording.custom_properties.program. ProgramData IDs change on every EPG refresh; the old dedup always passed after refresh, creating duplicates.

- Fix secondary timeslot guard to compare original program times (stored in custom_properties) instead of offset-adjusted Recording times. With DVR pre/post offsets configured, the old guard never matched.

- Add acquire_task_lock/release_task_lock to serialize concurrent evaluate_series_rules calls. Multiple EPG source refreshes firing evaluate_series_rules.delay() simultaneously raced to create duplicate recordings.
2026-03-20 11:48:06 -05:00
SergeantPanda
406267965a
Merge pull request #1047 from JCBird1012/auto-assign-next-available
fix: single stream channel modal; remove unused ChannelNumberingModal; add "Next Highest Channel" numbering mode
2026-03-17 13:05:14 -05:00
SergeantPanda
b4281c83bb Switch to backend logic instead of frontend. Use '-1' as 'highest' so not break the api. 2026-03-17 09:27:18 -05:00
SergeantPanda
d4e52f70db Merge branch 'dev' into pr/cmcpherson274/1105 2026-03-15 21:02:52 -05:00
SergeantPanda
cd0eba9654
Merge pull request #1098 from CodeBormen/fix/ghost-streams-stuck-initializing
fix: stuck channel states and ghost clients in TS proxy
2026-03-15 20:07:44 -05:00
None
55f0a49d68 fix: improve test coverage and add missing metadata constants
- Rewrite ghost client tests to call real ChannelStatus and ClientManager code paths instead of reimplementing detection logic in test helpers
- Rewrite initializing state tests to invoke StreamManager.run() so the real finally block executes under test
- Replace hardcoded state list tests with assertions against ChannelState.PRE_ACTIVE frozenset
- Add missing SOURCE_BITRATE and FFMPEG_BITRATE constants to ChannelMetadataField (referenced by channel_status.py but never defined, causing AttributeError on detailed stats)
2026-03-15 19:39:15 -05:00
SergeantPanda
c44a3ad5f9 Bug Fix: Stream.last_seen and ChannelGroupM3UAccount.last_seen model defaults now use django.utils.timezone.now instead of datetime.datetime.now, eliminating spurious RuntimeWarning: DateTimeField received a naive datetime warnings emitted during test database creation and on new record creation when USE_TZ=True. 2026-03-15 16:59:51 -05:00
None
40d0bc9567 fix: cap keepalive mode duration to prevent indefinite client hold
- Add MAX_KEEPALIVE_DURATION = 300s to TSConfig
- Track keepalive_start_time in _stream_data_generator; disconnect after cap is exceeded with a warning log
- Reset timer on real data so independent stalls don't accumulate
- Fix pre-existing test helper missing throttle attributes in NonOwnerWorkerKeepaliveTests
2026-03-15 14:22:57 -05:00
None
c65ace1a7c fix: stuck channel states and ghost clients in TS proxy
- Write ERROR via ownership check OR state guard fallback when ownership TTL expired during retries
- Add INITIALIZING to cleanup task grace period monitoring
- Validate client SET entries against metadata hashes in orphaned channel cleanup; remove ghosts and clean up when no real clients remain
- Stats page self-heals by removing ghost SET entries on read
- Extract remove_ghost_clients() helper to ClientManager
- Add ChannelState.PRE_ACTIVE constant
- Fix 6 pre-existing NonOwnerWorkerKeepalive test failures
2026-03-14 19:24:01 -05:00
dekzter
f69a462253 merged in dev 2026-03-13 08:22:44 -04:00
SergeantPanda
39865d3732 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/CodeBormen/943 2026-03-08 18:04:11 -05:00
SergeantPanda
2373044adb Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/CodeBormen/1052 2026-03-06 16:12:52 -06:00
SergeantPanda
9805732582 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/CodeBormen/1023 2026-03-05 11:16:20 -06:00
None
68de1b0723 fix(dvr): add DVR search/filters, fix in-progress scheduling, prevent duplicate recordings
- Add search, channel, and status filter controls to DVR page (matching Guide/VODs pattern)
- Fix post_save signal to schedule recordings for currently-playing programs immediately instead of skipping them
- Remove duplicate evaluate_series_rules.delay() call from series rule creation endpoint to prevent race condition that created two recordings for the same program
2026-03-04 10:21:03 -06:00
None
9d08305dcb fix(dvr): fix recording modal crash, harden recovery pipeline, improve tests
Frontend:
- RecordingDetailsModal: fix ReferenceError crash (editing state used before declaration in useMemo — TDZ in production bundles broke all tile clicks)
- RecordingDetailsModal: fix focus-stealing on description edit by calling Movie()/Series() as plain functions instead of <Movie />/<Series />(inline component refs change every render, causing unmount/remount)
- RecordingDetailsModal: suppress sub_title in modal title after custom save
- ErrorBoundary: log caught errors and show message for easier debugging
- guideUtils: filter terminal recordings from mapRecordingsByProgramId so Guide red dots clear after recording finishes or rule is deleted
- RecordingDetailsModalUtils.test: add missing end_time to test fixtures (filterByUpcoming now uses end_time instead of start_time)

Backend:
- tasks.py: output file collision avoidance (check .mkv/.ts before writing)
- tasks.py: pre-restart TS segment concatenation (direct concat→MKV remux)
- tasks.py: recovery pipeline — recover crashed "recording" status, remux expired recordings, revoke stale tasks, deterministic task IDs, early lock release, 10-min lock TTL for large remux operations
- tasks.py: poster resolver skips external API when title matches channel name
- celery.py: trigger recover_recordings_on_startup via worker_ready signal

Tests:
- Renamed test_dvr_fixes.py → test_recording_pipeline.py (clarity)
- Renamed test_dvr_retry.py → test_db_retry.py (clarity), fixed mock paths for close_old_connections (resolved 3 test failures), replaced synthetic test
- test_recording_stop_cancel: replaced brittle time.sleep(0.5) with deterministic thread-capture pattern
2026-03-04 10:21:03 -06:00
None
bd14e48f8b fix(dvr): harden recording pipeline — idempotency guard, remux sanity checks, timezone fix
- Idempotency guard: fail closed on DB errors instead of proceeding, preventing duplicate tasks from overwriting valid recordings (#641)
- Remux sanity checks: reject MKV output that is <50% of a previous MKV (duplicate-task overwrite) or <10% of the source TS (corrupt first attempt); preserve .ts for manual recovery on all failure paths (#619)
- Recurring rule timezone fix: use user's configured timezone for date window calculation instead of UTC, fixing silent recording loss for users in UTC-negative timezones after 4pm local time (#1042)
- Include .ts path in final remux failure log for easier recovery
2026-03-04 10:21:03 -06:00
None
759688390b fix(dvr): fix series episode list, card layout, rule cleanup, and recording resilience
Backend:
- Add terminal status guard to stop() endpoint -prevents overwriting completed/interrupted/failed with "stopped" (returns 409 Conflict)
- Include currently-airing programs in series rule evaluation by filtering on end_time instead of start_time
- Clean up future recordings when deleting a series rule while preserving previously recorded episodes
- Add logo proxy negative cache to prevent repeated requests to unreachable hosts from blocking Daphne workers
- Add URL validation with HEAD-then-GET fallback and per-worker cache
- Add EPG time-slot matching (80% overlap threshold) to enrich manual recordings with program metadata
- Add multi-source poster resolution pipeline (EPG, VOD, TMDB, OMDb, TVMaze, iTunes) with URL validation at each stage
- Add editable recording metadata endpoint with user_edited flag to prevent EPG auto-overwrite
- Add refresh-artwork endpoint to re-run poster resolution on demand
- Batch-fetch PeriodicTask names in schedule_upcoming to avoid N+1

Frontend:
- Fix series modal showing only one episode — preserve _group_count from categorized prop when merging with store version so the episode list renders correctly
- Fix recording card button positioning — use flex column layout with marginTop auto so buttons align at the bottom regardless of content
- Add delete fallback in RecurringRuleModal for orphaned recordings from deleted rules
- Use end_time filter in upcoming episodes list to include currently-airing episodes
- Add inline metadata editing (title/description) in details modal
- Add refresh artwork button in details modal
- Replace hardcoded /logo.png fallbacks with imported default logo
- Add React.memo with custom comparator to RecordingCard

Tests:
- Add 21 EPG time-slot matching tests (overlap threshold, edge cases)
- Add 16 recording metadata tests (validation, artwork, logo cache)
- Add 17 URL validation tests (HEAD/GET fallback, caching, eviction)
2026-03-04 10:21:03 -06:00
None
cf3165ea88 fix(dvr): add DB retry resilience, fix recording card logos, prevent incorrect artwork
Add exponential backoff retry (_db_retry helper) for transient DB errors during final metadata save, startup recovery, and initial TS proxy
connection. Add FFmpeg remux retry for transient I/O errors. Retry same proxy base before falling back to next candidate on initial connection failure.

Fix DVR card logos not displaying: channel summary API returns logo_id (integer) but frontend expected a nested logo object with cache_url.

Add getChannelLogoUrl() helper that handles both data shapes. Backend artwork prefetch and run_recording now fall back to the channel's own logo when no show-specific artwork is found, ensuring poster_logo_id is always populated.

Prevent incorrect artwork by skipping external API searches (TVMaze, iTunes) when no program title is available - channel names like
"USA A&E SD*" returned unrelated results from fuzzy matching.

Add 13 tests for retry logic (test_dvr_retry.py)
2026-03-04 10:21:03 -06:00