Commit graph

56 commits

Author SHA1 Message Date
francescodg89-crypto
c824f7a17f
Refactor content relation handling for efficiency
Refactor content relation retrieval to use a single DB query for active relations, improving efficiency. Update return values to include candidates for failover handling.
2026-07-02 00:16:25 +02:00
francescodg89-crypto
502ca5bfc8
Add VOD failover logic for M3U relations
Implemented VOD failover logic to iterate over all active M3U relations, selecting the first account with spare capacity instead of relying on a single highest-priority relation. This change enhances reliability by allowing for provider failover in case of account saturation.
2026-06-29 13:15:52 +02:00
SergeantPanda
7989fa3673 fix(proxy): enhance database connection management during VOD playback and stats updates
- Added calls to `close_old_connections()` in `stream_vod()` and `build_vod_stats_data()` to prevent connection leaks during long-lived streaming responses and background stats refreshes.
- Improved connection handling in various components to ensure proper resource cleanup and prevent blocking issues.
2026-06-15 17:32:38 -05:00
SergeantPanda
843940f552 refactor(http headers): centralize User-Agent construction and streamline API requests
- Introduced `dispatcharr_user_agent`, `dispatcharr_dvr_user_agent`, and `dispatcharr_http_headers` functions in `core.utils` to standardize User-Agent strings and HTTP headers for outbound requests.
- Updated various components, including `LogoViewSet`, `EPGSourceViewSet`, and VOD proxy views, to utilize the new header functions, ensuring consistent User-Agent usage across the application.
- Enhanced the handling of Schedules Direct API requests by including proper User-Agent headers, addressing previous API compliance issues.
2026-06-11 09:45:35 -05:00
SergeantPanda
705041ea81 refactor(channel_status, vod_proxy): remove stream URL credential extraction
- Eliminated redundant extraction of provider usernames from stream URLs in both ChannelStatus and VOD stats data functions, simplifying the codebase.
- Updated frontend components to reflect the removal of provider username display, streamlining the UI.
- Enhanced overall readability and maintainability of the code by reducing complexity in handling stream URLs.
2026-06-09 14:46:26 -05:00
SergeantPanda
bc046b3c92 refactor(m3u, channels): streamline stream assignment and server group management
- Renamed `_clear_stream_assignment_keys` to `_release_stale_stream_assignment` for clarity and updated its logic to release pool counters.
- Introduced new functions for managing credential slots during profile switches, enhancing the handling of shared connection limits across server groups.
- Removed the `max_streams` field from the `ServerGroup` model and updated related components to reflect this change, simplifying the server group management.
- Updated frontend components to integrate server group management, allowing for dynamic creation and editing of server groups.
- Enhanced error handling in stream URL generation to provide more informative feedback on connection issues.
- Added tests for stale assignment release and credential management during profile switches.
2026-06-09 13:40:16 -05:00
SergeantPanda
0f0e855507 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/Goldenfreddy0703/1254 2026-06-09 11:16:39 -05:00
R3XCHRIS
ff60bbec1a fix(vod_proxy): fall back to stream_id when content UUID is dead
Issue: When `process_movie_batch` / `process_series_batch`
(apps/vod/tasks.py) creates duplicate vod_movie / vod_episode records
during a refresh, existing M3U*Relation rows get repointed to the new
records. The old UUIDs that external players (Emby / Jellyfin /
ChannelsDVR) cached in `.strm` URLs are left orphaned, and the proxy
404s on every subsequent request — even though the same request
already carries a stable `stream_id` that uniquely identifies a live
relation.

This patch makes `_get_content_and_relation` use a soft UUID lookup
and, if it misses, fall back to resolving content via
`M3UMovieRelation.stream_id` / `M3UEpisodeRelation.stream_id`. The
fallback respects the strictest match first: when
`preferred_m3u_account_id` is also present, that account is queried
first; otherwise the highest-priority active relation matching the
stream_id wins (same ordering the existing relation-selection logic
uses).

This is read-side graceful degradation, complementary to the
write-side root fix proposed in closed #973 (re-use the relation's
existing movie/series during refresh rather than rematching by TMDB
ID). The two PRs together would fully resolve #961: #973 prevents new
orphaning, this PR makes any URL that ships a stream_id survive past
orphaning. URLs without a stream_id (e.g. XC-compat
`/movie/<user>/<pass>/<id>.mkv`) are NOT covered by this patch — they
need the root fix.

Series-UUID lookups (`/proxy/vod/series/<UUID>`) are left UUID-only by
design — players cache episode and movie URLs, not series URLs. Same
pattern can be added as a follow-up if needed.

Tests: 7 new mock-based regression tests covering both branches,
verified against `:dev`. UUID-first happy path is unchanged
(M3U*Relation table never queried when the UUID resolves).

Related: #961 (open), closed #973 (root fix abandoned for inactivity).
2026-06-02 19:29:58 +01:00
SergeantPanda
450b384419 fix(vod_proxy): Fix stream_vod to handle session_id in path for VOD proxy URLs 2026-05-30 20:48:17 -05:00
SergeantPanda
72d1520400 feat(auth): add QueryParamJWTAuthentication for token retrieval via query parameters
feat(vod): include token in stream URL for episode and movie requests (Fixes #1224)
2026-05-30 11:14:59 -05:00
Goldenfreddy0703
bb9b95b4a4 Refactor connection pools to manual ServerGroups with profile rotation.
Drop auto-fingerprint migration and restore per-profile selection for live/VOD.
Enforce shared limits on reserve using login-scoped group counters, and add
Server Groups UI for manual account assignment per maintainer feedback (#1137).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-23 20:18:45 -04:00
Goldenfreddy0703
d89f634213 Enforce shared credential connection pools across M3U accounts (#1137)
Group M3U/XC accounts and profiles that share the same provider login into auto-assigned ServerGroups keyed by credential fingerprint. Enforce combined Redis limits for live TV and VOD via apps/m3u/connection_pool.py.

- Per-profile fingerprinting (XC transforms and STD stream URLs)

- VOD profile selection tries alternates when default credential pool is full (fixes live then VOD failure)

- Stats UI shows provider login from active stream URL

- Tests: apps/m3u/tests/test_connection_pool.py (11 tests, all passing)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 03:30:48 -04:00
Seth Van Niekerk
0d3eed81ba
Code review changes 2026-05-02 16:59:50 -04:00
Seth Van Niekerk
a01814069a
code review & pivot 2026-05-01 09:10:37 -04:00
Seth Van Niekerk
8894d51877
restrict XC access per user by configurable IP allowlist 2026-04-28 16:35:32 -04:00
SergeantPanda
29739ede36 Enhancement: VOD start/stop notifications and system events.
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
2026-04-23 18:03:02 -05:00
SergeantPanda
c3fb9c0073 security: Fixed missing network_access_allowed checks in the VOD proxy. stream_vod, head_vod, stream_xc_movie, and stream_xc_episode were not checking the STREAMS network policy, unlike the equivalent TS proxy endpoints. 2026-04-10 11:09:27 -05:00
SergeantPanda
fa9a7868ff security: proxy streaming endpoints (stream_ts, stream_xc, stream_vod, head_vod, stream_xc_movie, stream_xc_episode) use @permission_classes([AllowAny]) (access is controlled by the per-stream-type network allow-list inside the view body); the UserAgentViewSet, StreamProfileViewSet, CoreSettingsViewSet, and ProxySettingsViewSet gained get_permissions() methods mapping read actions to IsStandardUser and write actions to IsAdmin; and AuthViewSet.logout was updated to return [Authenticated()]. 2026-04-10 10:47:50 -05:00
SergeantPanda
dcefc6541c chore(cleanup): - Removed dead VODConnectionManager class (apps/proxy/vod_proxy/connection_manager.py) and its associated helpers, which had been superseded by MultiWorkerVODConnectionManager. All active code already used the multi-worker implementation. Removed the unused VODConnectionManager import from vod_proxy/views.py, the unscheduled cleanup_vod_connections task from apps/proxy/tasks.py, and the unscheduled cleanup_vod_persistent_connections task from core/tasks.py.
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
- Removed dead VOD URL routes: `VODPlaylistView` (playlist generation), `VODPositionView` (position tracking), and the class-based `VODStatsView` (replaced by the existing function-based `vod_stats` view).
- Removed dead `updateVODPosition()` API method from `frontend/src/api.js`, which called the now-removed position tracking endpoint.
2026-04-09 19:59:50 -05:00
None
763f42cfff fix: pass TLS parameters to all external connection points in modular mode
- Add setup_pg_ssl_env() to export libpq PGSSL* env vars; all child psql/pg_dump/pg_isready commands inherit TLS automatically

- Move TLS client key permission fix before external PG connections (was running after, defeating the fix)

- Enhance key fix to trigger on ownership mismatch (root-owned 0600 key unreadable by app user)

- Extract key fix to shared script (docker/init/00-fix-pg-ssl-key.sh) sourced by both entrypoints

- Default POSTGRES_PASSWORD to empty in modular mode (cert-only auth sends no spurious password)

- Propagate TLS OPTIONS to backup pg_dump/pg_restore subprocess calls via _get_pg_env()

- Pass SSL kwargs to dropdb management command's psycopg2.connect()

- Add REDIS_SSL_PARAMS to VOD proxy Redis connections missed in original TLS PR

- Add 8-scenario TLS integration test suite (PG mTLS, Redis TLS, verify-full, key fix, Celery, regression)

- Add 6 unit tests for _get_pg_env() and dropdb TLS parameter passing
2026-03-30 20:19:27 -05:00
SergeantPanda
086cc74959 Bug Fix: M3U profile URL rewriting now uses the regex module instead of re across all URL transform code paths (url_utils.transform_url, core/views.py, vod_proxy/_transform_url, tasks.get_transformed_credentials, and the WebSocket live-preview handler in consumers.py). The regex module natively accepts JavaScript/PCRE-style named capture groups ((?<name>...)) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a re.error. As a further improvement, regex also supports variable-length lookbehind assertions (e.g. (?<=a+)), which re rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling regex.sub: $<name>\g<name> and $1/$2/… → \1/\2/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
2026-03-29 17:55:08 -05:00
SergeantPanda
38f5156ca6 Enhancement: Show username in VOD cards. 2026-03-28 16:46:06 -05:00
SergeantPanda
aa6fb033d3 fix: update user stream limit checks to include media_id and rename user_limits_settings key 2026-03-26 20:02:54 -05:00
dekzter
a19be96ced fleshed out user limits and termination logic 2026-03-25 17:33:26 -04:00
dekzter
6f4665e6eb merged in dev 2026-02-26 08:20:20 -05:00
None
e217960500 feat: Add Redis authentication support for modular deployment
Add support for authentication when connecting to external Redis instances in modular deployment mode. This enables secure Redis deployments using either password-only authentication (Redis <6) or
username + password authentication (Redis 6+ ACL).

Changes:
- Add REDIS_PASSWORD and REDIS_USER environment variables
- Implement URL encoding for special characters in passwords
- Update all Redis connection points to support auth
- Add comprehensive documentation and examples to docker-compose.yml
- Maintains full backward compatibility (empty defaults = no auth)

All authentication mechanisms have been fully tested including pasword-only authentication, Redis 6+ ACL authentication with username + password, volume-mounted configuration files, and special character handling.

Existing deployments are not effected, authentication support is entirely opt-in using docker-compose.

Also, acknowledging the fact that the docker-compose file for modular deployments has been getting out of hand with auth support, the docker-compose.yml file was re-formatted for better visibility in configuration. This seemed like a better way to go than mandating a .env file.
2026-02-03 21:42:06 -06:00
SergeantPanda
de31826137 refactor: externalize Redis and Celery configuration via environment variables
Replace hardcoded localhost:6379 values throughout codebase with environment-based configuration. Add REDIS_PORT support and allow REDIS_URL override for external Redis services. Configure Celery broker/result backend to use Redis settings with environment variable overrides.

Closes #762
2025-12-18 16:54:59 -06:00
SergeantPanda
2558ea0b0b Enhancement: Add VOD client stop functionality to Stats page 2025-12-17 16:54:10 -06:00
dekzter
50e9075bb5 initial run of a binary and encoded redis client - no more encoding / decoding data into redis, huge PITA (still some outstanding spots I need to patch) 2025-10-25 08:15:39 -04:00
SergeantPanda
7bb4df78c8 Enhancement: Update M3U profile retrieval to include current connection counts and session handling during vod session start. 2025-09-26 09:18:42 -05:00
SergeantPanda
d961d4cad1 Bug fix: VOD will now select the correct M3U profile while starting.Fixes #461 2025-09-26 09:18:22 -05:00
SergeantPanda
b45c6eda38 Better content-type detection 2025-09-07 18:25:39 -05:00
SergeantPanda
adf960753c Changed user-agent for head request to use m3u account 2025-09-07 15:17:19 -05:00
SergeantPanda
c239f0300f Better progress tracking for clients that start a new session on every seek instead of reusing existing session. 2025-09-06 15:36:14 -05:00
SergeantPanda
4ca6bf763e Add position calculation 2025-09-06 10:16:54 -05:00
SergeantPanda
6b9d42fec1 Refactor VODStatsView to use correct field names for content type and M3U profile ID, and update user agent handling 2025-09-05 20:34:43 -05:00
SergeantPanda
18b8462a5f Refactor user-agent handling to use 'client_user_agent' for consistency across VOD connection management 2025-09-05 20:15:11 -05:00
SergeantPanda
1080b1fb94 Refactor stats and vod proxy 2025-09-05 19:30:13 -05:00
SergeantPanda
3590265836 Show all available options for VODs and their corresponding quality. 2025-08-21 15:04:47 -05:00
SergeantPanda
a4df1f1fb8 Allow specifying which account to play from. 2025-08-21 13:14:16 -05:00
SergeantPanda
24f876d09f Add priority for providers so VOD's can be auto selected based on the priority. 2025-08-20 17:38:21 -05:00
SergeantPanda
97b82e5520 Use redis to track provider connections to work with multi-worker uwsgi. 2025-08-19 17:35:51 -05:00
SergeantPanda
fa19525ab9 Remove url from movie table and save custom_properties for series. 2025-08-19 11:25:56 -05:00
SergeantPanda
54404339c2 Fix batch processing using simpler method for duplicate detection. 2025-08-19 10:01:37 -05:00
SergeantPanda
3e16614eab Store full content-length if using head request to initalize. 2025-08-12 17:32:07 -05:00
SergeantPanda
5eeb51585d Support HEAD requests, keep connection alive. 2025-08-12 15:27:39 -05:00
SergeantPanda
be51be7872 Switched to connection tracking by url instead of parameter and 301 redirect instead of 302. 2025-08-12 14:48:35 -05:00
SergeantPanda
6addcebaf5 Fix duplicate connection counting in redis. 2025-08-12 10:59:40 -05:00
SergeantPanda
07966424f8 Fix seeking not working. 2025-08-12 08:29:26 -05:00
SergeantPanda
345247df11 Fix vod streaming. 2025-08-08 08:35:59 -05:00