Dispatcharr/dispatcharr
Josh Becker 1fbc23a5ee
fix(db): reap leaked idle connections from the gevent pool (#1418)
The custom psycopg3 gevent pool only evaluates connection age inside
get()/put(). A connection checked out by a greenlet that dies without
returning it — e.g. a live-stream / channel-switch teardown path that
never calls close_old_connections() — is never aged out. It stays counted
against MAX_CONNS but is no longer in the free queue, so once MAX_CONNS
connections leak per worker, get() blocks forever and every DB-backed
request on that worker hangs with a 60s/504 until the container is
restarted. DB-free endpoints (e.g. /api/core/version/) stay fast, which
is the tell.

Add a background reaper greenlet (one per worker, started lazily on first
get() so the gevent hub is up) that periodically closes connections which
are both expired (older than max_lifetime) and IDLE (no in-flight
transaction). The IDLE guard means an active stream or in-flight request
is never disrupted, and get() already refuses to hand out an expired
connection so a reaped connection can't be in active use. This frees the
leaked slot and lets the worker recover on its own — no restart.

Side effect: DATABASE_POOL_CONN_MAX_LIFETIME now actually reclaims leaked
connections (previously it only recycled healthy idle ones). Lower it to
shorten leak-recovery time. Reaper cadence is configurable via
DATABASE_POOL_REAP_INTERVAL (default 30s; 0 disables).

Also covers the M3U-refresh INTRANS leak family (#1338).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 17:05:33 -04:00
..
db fix(db): reap leaked idle connections from the gevent pool (#1418) 2026-07-07 17:05:33 -04:00
__init__.py Initial commit 2025-02-18 11:14:09 -06:00
admin.py Alpha v3 2025-02-21 15:31:59 -06:00
app_initialization.py Enhancement: Refactored app initialization to prevent redundant execution across multiple worker processes. Created dispatcharr.app_initialization utility module with should_skip_initialization() function that prevents custom initialization tasks (backup scheduler sync, developer notifications sync) from running during management commands, in worker processes, or in development servers. This significantly reduces startup overhead in multi-worker deployments (e.g., uWSGI with 10 workers now syncs the scheduler once instead of 10 times). Applied to both CoreConfig and BackupsConfig apps. 2026-02-05 13:48:01 -06:00
asgi.py websockets behind auth, cleaned up errors and bad state handling in websocket.jsx 2025-05-10 08:40:53 -04:00
celery.py fix(m3u): enhance custom properties handling and DB connection management 2026-06-18 14:59:09 -05:00
consumers.py feat(websocket): enhance connection handling with cached IP lookup 2026-05-31 19:28:19 -05:00
gevent_patch.py Enhancement: Update psycopg2 to 3. 2026-05-28 19:27:52 -05:00
jwt_ws_auth.py Enhance JWT authentication error handling and user redirection on token issues. 2025-05-15 14:12:31 -05:00
persistent_lock.py merged in dev 2026-03-26 12:17:43 -04:00
routing.py Websockets, fixed channel name collision, added back in multi-stream per channel support 2025-03-05 17:04:43 -05:00
settings.py feat(settings): add DATABASE_POOL_CONN_MAX_LIFETIME for connection management and update database engine path (Fixes #1343) 2026-06-12 11:31:09 -05:00
settings_test.py enhancement(tests): introduce isolated backend test settings and improve test documentation 2026-06-23 20:54:12 -05:00
urls.py Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00
utils.py refactor: Clean up get_client_ip() function in utils.py 2026-05-09 09:48:53 -05:00
wsgi.py Revert "fix(wsgi): ensure gevent monkey-patching before Django import chain" 2026-03-03 15:33:55 -06:00