Commit graph

30 commits

Author SHA1 Message Date
SergeantPanda
391c3412d2 refactor(proxy): Add new client connect grace period setting and update channel initialization grace period defaults. The channel_client_wait_period is introduced to manage channel lifetimes before client connections, while the channel_init_grace_period default is increased to improve failover handling. 2026-06-28 11:31:21 -05:00
SergeantPanda
fd93c0dc3d fix(channel management): Implement channel initialization grace period handling to ensure proper state transitions during live stream startup. Channels now honor the configured grace period for buffer filling, promoting to active state when conditions are met. Updated related tests and documentation for clarity. (Fixes #1380)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
2026-06-25 14:24:30 -05:00
SergeantPanda
f991338376 fix(proxy): update for improved DB connection management
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
- Enhanced the live proxy to release geventpool DB connections more effectively across various paths.
- Implemented `close_old_connections()` in `stream_ts()`, `generate_stream_url()`, `get_stream_info_for_switch()`, `get_alternate_streams()`, `get_connections_left()`, and the cleanup process in `StreamGenerator`, ensuring that clients do not hold onto pool slots unnecessarily during streaming operations.
2026-06-17 17:22:28 -05:00
SergeantPanda
32442e0b64 fix(proxy): enhance channel shutdown management and client reconnection handling
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
- Improved the handling of channel shutdown delays to ensure they reset correctly upon client reconnections, preventing premature shutdowns.
- Implemented logic to cancel pending shutdowns when clients reconnect, ensuring proper resource management and preventing client disconnect issues across uWSGI workers.
- Enhanced logging for shutdown processes and client management to provide clearer insights during channel operations.
2026-06-15 18:53:49 -05:00
SergeantPanda
7989fa3673 fix(proxy): enhance database connection management during VOD playback and stats updates
- Added calls to `close_old_connections()` in `stream_vod()` and `build_vod_stats_data()` to prevent connection leaks during long-lived streaming responses and background stats refreshes.
- Improved connection handling in various components to ensure proper resource cleanup and prevent blocking issues.
2026-06-15 17:32:38 -05:00
SergeantPanda
c389462dde fix(proxy): enhance connection management and event handling during streaming operations
- Improved database connection management by ensuring `close_old_connections()` is called in various methods to prevent connection leaks.
- Updated event dispatching to run asynchronously in gevent, preventing blocking during live-proxy and streaming paths.
2026-06-15 17:14:48 -05:00
SergeantPanda
8f40f6065f fix(proxy): improve resource cleanup and connection management
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
- Added calls to close_old_connections in various components to ensure proper resource cleanup and prevent connection leaks.
2026-06-14 21:55:57 -05:00
SergeantPanda
67239d921d fix(proxy): enhance channel teardown and client management
- Improved channel teardown process to prevent lingering upstream connections and ensure proper resource cleanup.
- Enhanced client management by implementing checks for ghost clients and ensuring accurate client disconnection handling.
- Updated logging to provide clearer insights during channel initialization and teardown, including handling of unavailable channels.
- Refined stream manager behavior to manage ownership transitions more effectively and prevent blocking during shutdown.
2026-06-14 21:41:50 -05:00
SergeantPanda
9393f72cc1 fix(proxy): enhance channel teardown and client management
- Improved handling of channel teardown to prevent client reconnect issues across uWSGI workers, ensuring proper resource cleanup and ownership management.
- Added functionality to clear all client entries from Redis for a channel, enhancing client management during shutdown.
- Updated logging and response mechanisms to inform clients of channel availability during teardown,.
- Enhanced stream manager behavior to ensure upstream connections are properly managed during ownership transitions.
2026-06-14 18:59:59 -05:00
SergeantPanda
99c2b3b4d7 fix(proxy): improve channel stop handling and logging
- Enhanced the `stop_channel` method to prevent blocking during teardown by logging stop events asynchronously.
- Updated stream buffer behavior to ignore late writes during shutdown, improving resource management.
2026-06-14 11:50:50 -05:00
SergeantPanda
4fc12bca4a refactor(m3u, channels, proxy): enhance stream assignment logic and error handling
- Improved logging in the Stream model for better debugging of profile evaluations.
- Introduced a new method `_stream_assignment_is_reusable` in the Channel model to determine if existing stream assignments can be reused, enhancing efficiency.
- Updated the release logic in `release_profile_slot` to utilize stored credential keys, reducing unnecessary database lookups.
- Simplified error handling in the `get_stream_info_for_switch` function to ensure proper stream release on exceptions.
- Enhanced tests for connection pool management and error handling in the ServerGroupsTable component to improve reliability and user feedback.
2026-06-09 17:59:12 -05:00
SergeantPanda
705041ea81 refactor(channel_status, vod_proxy): remove stream URL credential extraction
- Eliminated redundant extraction of provider usernames from stream URLs in both ChannelStatus and VOD stats data functions, simplifying the codebase.
- Updated frontend components to reflect the removal of provider username display, streamlining the UI.
- Enhanced overall readability and maintainability of the code by reducing complexity in handling stream URLs.
2026-06-09 14:46:26 -05:00
SergeantPanda
bc046b3c92 refactor(m3u, channels): streamline stream assignment and server group management
- Renamed `_clear_stream_assignment_keys` to `_release_stale_stream_assignment` for clarity and updated its logic to release pool counters.
- Introduced new functions for managing credential slots during profile switches, enhancing the handling of shared connection limits across server groups.
- Removed the `max_streams` field from the `ServerGroup` model and updated related components to reflect this change, simplifying the server group management.
- Updated frontend components to integrate server group management, allowing for dynamic creation and editing of server groups.
- Enhanced error handling in stream URL generation to provide more informative feedback on connection issues.
- Added tests for stale assignment release and credential management during profile switches.
2026-06-09 13:40:16 -05:00
SergeantPanda
0f0e855507 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/Goldenfreddy0703/1254 2026-06-09 11:16:39 -05:00
SergeantPanda
60a9f6843b refactor: simplify initialization wait logic by removing keepalive packet handling (Fixes #1280) 2026-06-02 20:54:59 -05:00
SergeantPanda
61c187fc72 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/JCBird1012/1255 2026-05-23 21:43:13 -05:00
Goldenfreddy0703
bb9b95b4a4 Refactor connection pools to manual ServerGroups with profile rotation.
Drop auto-fingerprint migration and restore per-profile selection for live/VOD.
Enforce shared limits on reserve using login-scoped group counters, and add
Server Groups UI for manual account assignment per maintainer feedback (#1137).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-23 20:18:45 -04:00
SergeantPanda
59587cbe66 Fix XC stream URL format output handling and improve M3U playlist generation performance. Removed unused imports in apps\output 2026-05-22 11:51:16 -05:00
SergeantPanda
17e6e8c238 chore: Cleanup unused code. 2026-05-21 10:13:28 -05:00
Jonathan Caicedo
6787e4912f refactor: replace multiple hget calls with hmget where fields are known
- channel_status.py: fold output_format and output_profile_id into the
  existing hmget, reducing per-client Redis calls from 3 to 1
- utils.py: collapse 2x and 3x hget per key in scan_iter loops
  (get_user_active_connections) to single hmget calls
- views.py: replace 3x hget on channel metadata in the worker-join path
  of stream_ts with a single hmget
2026-05-20 22:12:12 -04:00
Jonathan Caicedo
e2074c1921 fix: reduce intermittent 503s on stats page and API requests 2026-05-20 21:52:28 -04:00
Goldenfreddy0703
d89f634213 Enforce shared credential connection pools across M3U accounts (#1137)
Group M3U/XC accounts and profiles that share the same provider login into auto-assigned ServerGroups keyed by credential fingerprint. Enforce combined Redis limits for live TV and VOD via apps/m3u/connection_pool.py.

- Per-profile fingerprinting (XC transforms and STD stream URLs)

- VOD profile selection tries alternates when default credential pool is full (fixes live then VOD failure)

- Stats UI shows provider login from active stream URL

- Tests: apps/m3u/tests/test_connection_pool.py (11 tests, all passing)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 03:30:48 -04:00
SergeantPanda
594596d4f5 Enhancement: update channel card tooltip to use stable 'started_at' timestamp and refactor getStartDate utility
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
Base Image Build / prepare (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
2026-05-19 14:37:33 -05:00
SergeantPanda
a71dcb31a0 fix: ensure proper cleanup of client managers on channel disconnect 2026-05-17 19:05:44 -05:00
SergeantPanda
d448561767 Bug Fix: Refactored channel teardown to kill ffmpeg earlier in the process. Removed redundant cleanup task. Added guard to prevent multiple cleanups for the same channel. 2026-05-17 10:03:44 -05:00
SergeantPanda
c68249c314 fix: replace fork()-based subprocess with posix_spawn at all uWSGI spawn sites
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Waiting to run
Base Image Build / prepare (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
gevent registers a pthread_atfork handler that never yields. Any
subprocess.Popen/run call in a uWSGI greenlet hangs indefinitely at
fork() before the child process even starts.

- dispatcharr/gevent_patch.py: new; monkey.patch_all() + psycogreen,
  loaded via uWSGI import= in all four ini files
- live_proxy/input/manager.py: posix_spawn + _SpawnedProcess; removed
  dead forwarder code
- live_proxy/input/http_streamer.py: O_NONBLOCK on relay pipe + EAGAIN
  retry (blocking write to a full pipe stalls the gevent hub)
- live_proxy/utils.py: new posix_spawn_proc() helper with O_NONBLOCK
  stdin, shared by both output managers
- live_proxy/output/fmp4/manager.py, output/profile/manager.py:
  posix_spawn_proc(); _write_all() treats EAGAIN (None) as cooperative
  select.select wait instead of fatal error
- core/views.py (stream_view): posix_spawn; fixed pre-existing bug
  where return StreamingHttpResponse(...) was indented inside
  stream_generator, making the success path always return None
- connect/handlers/script.py: _posix_run() with posix_spawn +
  cooperative select reads + non-blocking waitpid; fixes deadlock when
  a script integration fires on a uWSGI-context event (client_connect
  fires in the live proxy, not Celery)
2026-05-15 16:45:09 -05:00
SergeantPanda
c33d456743 Enable gevent in uWSGI workers; fix WS delivery and DB exhaustion
- Enable gevent cooperative multitasking in all uWSGI worker configs
  (gevent-early-monkey-patch + import dispatcharr.gevent_patch)
- Rewrite WebSocket group sends to bypass asyncio in gevent workers:
  _gevent_ws_send() replicates the channels_redis 4.x wire format
  directly via synchronous Redis so send_websocket_update() and
  _send_async() no longer fail silently after epoll is patched out
- Fix PostgreSQL connection exhaustion: CONN_MAX_AGE=0 + explicit
  close_old_connections() in stream manager and cleanup watchdog loops
- Fix stream proxy race: register client before the connect-wait loop
  so the cleanup watchdog never sees zero clients on a live channel
- Channel list/logo/profile queryset optimisations: conditional DISTINCT,
  EXISTS semi-joins for filter-options, channel_count annotation to
  eliminate N+1 in LogoSerializer, prefetched memberships in
  ChannelProfileSerializer
- JsonResponse for channel ID list and summary endpoints
2026-05-14 18:55:09 -05:00
SergeantPanda
c4de8ce69e feat: Add debug logging for profile management and client disconnect handling 2026-05-12 18:56:08 -05:00
SergeantPanda
1107423893 fix: Prevent redundant shutdown calls in StreamGenerator and clean up profile managers in ProxyServer
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-05-11 17:34:54 -05:00
SergeantPanda
dc649ffd88 Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00