- Add reconnection loop to DVR streaming: on transient connectivity loss (ReadTimeout, ConnectionError, ChunkedEncodingError), the recording task retries the same TS proxy base up to 5 times with a 2-second delay, appending to the existing file. Counter resets on successful data receipt. TS container format tolerates brief discontinuities; MKV remux normalises timestamps.
- Extract _check_recording_cancelled() helper to consolidate repeated stop/delete DB checks across the streaming and reconnection paths.
- Fix comskip treating exit code 1 (no commercials detected) as a fatal error.
Use check=False and handle return codes explicitly: 0 = commercials found,
1 = no commercials (skipped), negative = signal, other = real error.
- Add human-readable reason messages for comskip 'skipped' notifications in
WebSocket handler.
- Remove redundant conditional Channel import in stream_generator._cleanup() that shadowed the module-level import, causing UnboundLocalError when the conditional branch did not execute.
File changes:
Backend
apps/channels/api_views.py — Added extend() action that uses queryset .update() to bypass signals, letting the running task pick up the new end_time via polling. Moved WS recording_stopped notification to synchronous (before HTTP response). Refactored destroy() to delete DB row first, send WS event immediately, defer cleanup to background thread.
apps/channels/signals.py — Added pre_save guard to skip task revocation when status is "recording". Added post_save reentrancy guard to skip artwork prefetch for internal field-only saves (custom_properties, task_id, end_time).
apps/channels/tasks.py — Added 2-second polling loop in streaming that checks for stop, deletion, and extended end_time. Added refresh_from_db() + merge before metadata save to prevent overwriting concurrent artwork prefetch. Replaced inline PeriodicTask cleanup with revoke_task()/_dvr_task_name(). Consolidated redundant DB queries in error path. Fixed TOCTOU os.path.exists + os.remove. Preserved "stopped" status in final-status logic instead of overwriting with "completed".
core/utils.py — Made send_websocket_update gevent-safe by detecting monkey-patching and offloading async_to_sync to gevent's native threadpool, fixing the event loop deadlock that caused cross-recording interference.
apps/proxy/ts_proxy/client_manager.py — Moved _trigger_stats_update to a background thread so remove_client() returns immediately.
apps/proxy/ts_proxy/stream_generator.py — Added Redis-based health check for non-owner workers in _should_send_keepalive(), so keepalives fire correctly even when stream_manager is None.
Frontend
frontend/src/WebSocket.jsx — Added 400ms debounced scheduleRecordingFetch() replacing all inline fetchRecordings() calls. Added recording_extended and recording_updated event handlers. recording_cancelled now does surgical removeRecording() by ID when available.
frontend/src/api.js — Added extendRecording(id, extraMinutes).
frontend/src/components/cards/RecordingCard.jsx — Added Extend menu (+15m/+30m/+1h) for in-progress recordings. Widened Comskip button to stopped/interrupted statuses.
frontend/src/components/forms/ProgramRecordingModal.jsx — Removed manual fetchRecordings() calls, relying on WS-driven debounced refresh.
frontend/src/components/forms/RecordingDetailsModal.jsx — Added stopped to playable statuses. Widened Comskip button. Removed manual fetchRecordings().
frontend/src/components/forms/RecurringRuleModal.jsx — Removed all manual fetchRecordings() calls, relying on WS events.
frontend/src/pages/Guide.jsx — Removed isLoading subscription that caused loading flash on debounced refresh. Removed manual fetchRecordings() after saving a series rule.
frontend/src/store/channels.jsx — Stripped isLoading/error state from fetchRecordings to prevent full-page loading indicators. Added no-op guards in removeRecording.
frontend/src/utils/cards/RecordingCardUtils.js — Added extendRecordingById wrapper.
Tests
apps/channels/tests/test_recording_extend.py (new) — 14 tests: extend endpoint validation, stacked extensions, signal bypass, pre_save guard behavior.
apps/channels/tests/test_recording_stop_cancel.py (new) — 18 tests: stop endpoint, DVR client teardown, cancel was_in_progress flag, run_recording race guards, signal reentrancy guards.
apps/channels/tests/test_ts_proxy_keepalive.py (new) — 21 tests: keepalive owner/non-owner logic, stats update error handling, client removal non-blocking, timing invariants.
apps/channels/tests/test_recording_scheduling.py — Updated mock assertion to match new _stop_dvr_clients call signature.
Summary:
Adds an Extend action for in-progress recordings that bypasses Django signals and lets the running task dynamically pick up the new end_time via its 2-second DB polling loop. Fixes cross-recording interference caused by send_websocket_update deadlocking the gevent event loop. Fixes an artwork race condition where run_recording overwrote concurrent prefetch data. Adds Redis-based keepalive health checks for non-owner workers to prevent DVR read timeouts during source transitions. Replaces manual fetchRecordings() calls throughout the frontend with a debounced WS-driven refresh. 53 new tests.
If ProxyServer() raises during singleton construction, _instance was left
as _INITIALIZING permanently, causing all subsequent get_instance() callers
to spin in an infinite gevent.sleep() loop. Wrap construction in try/except
and reset _instance to None on failure so callers can retry.
Also replace the non-atomic setnx() + expire() pair in try_acquire_ownership()
with a single atomic SET NX EX call, consistent with the approach already used
in extend_ownership() and eliminating the race window where a crash between the
two calls could leave a key with no TTL (permanent ownership lock).
- Use atomic SET NX EX instead of separate SETNX + EXPIRE to prevent
zombie locks if the process crashes between the two calls
- Replace time.sleep() with gevent.sleep() in get_instance() spin-wait
to avoid blocking the greenlet hub
- Defer cleanup when ownership is lost but clients are still connected,
giving the new owner time to establish its stream before we tear down
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three interrelated bugs cause TS proxy streams to terminate prematurely
in multi-worker uWSGI deployments:
1. Double ProxyServer instantiation: ProxyConfig.ready() in apps/proxy/apps.py
calls TSProxyServer() directly, bypassing get_instance(). The subsequent
TSProxyConfig.ready() call to get_instance() creates a second instance.
Each instance starts its own cleanup thread, but only one holds channel
data — the orphaned cleanup thread cannot extend ownership.
2. Redis flushdb() on every client init: RedisClient.get_client() in
core/utils.py calls flushdb() whenever a new connection is created.
Celery autoscale workers spawning mid-stream nuke all Redis keys
including ownership, client records, and channel metadata.
3. No recovery from expired ownership: get_channel_owner() has a TOCTOU
bug (two separate GET calls in a lambda). extend_ownership() silently
fails when keys expire. Non-owner cleanup unconditionally kills streams
even when the worker holds the stream_manager.
Fixes:
- Use TSProxyServer.get_instance() in ProxyConfig.ready()
- Remove flushdb() from Redis client initialization
- Use sentinel pattern for gevent-safe singleton (threading.Lock does not
work with gevent greenlets)
- Single GET in get_channel_owner() to avoid TOCTOU race
- Re-acquire expired ownership keys in extend_ownership()
- Attempt re-acquisition before cleanup in non-owner path
Relates to #992, #980
Fix multiple VOD proxy connection counter leak paths that caused
profile_connections to remain permanently elevated after client
disconnects, leaving profiles locked at capacity.
Changes:
- Fix get_stream() race condition: save state under lock with fresh active_streams to prevent overwriting concurrent decrements during scrubbing/seeking (root cause of counter stuck after timeshift)
- Add early active_streams increment in session reuse path to prevent cleanup race between GeneratorExit and new request
- Add rollback on get_stream() returning None (range not satisfiable)
- Add profile_id fallback to closure variable when Redis state is deleted before generator cleanup runs
- Change increment/decrement return types and add warning logs for lock acquisition and state failures
- Clean up debug traceback logging from increment/decrement methods
Three fixes:
- TOCTOU: Replace GET-check-INCR with atomic INCR-first-then-check pattern in both connection managers to prevent concurrent requests exceeding max_streams
- Immediate DECR: Decrement profile counter directly in stream generator exit paths (normal completion, client disconnect, error) instead of deferring to daemon thread cleanup which may never execute
- Rollback: Decrement profile counter on create_connection() failure so the reserved slot is released
Fixes#962
Add support for authentication when connecting to external Redis instances in modular deployment mode. This enables secure Redis deployments using either password-only authentication (Redis <6) or
username + password authentication (Redis 6+ ACL).
Changes:
- Add REDIS_PASSWORD and REDIS_USER environment variables
- Implement URL encoding for special characters in passwords
- Update all Redis connection points to support auth
- Add comprehensive documentation and examples to docker-compose.yml
- Maintains full backward compatibility (empty defaults = no auth)
All authentication mechanisms have been fully tested including pasword-only authentication, Redis 6+ ACL authentication with username + password, volume-mounted configuration files, and special character handling.
Existing deployments are not effected, authentication support is entirely opt-in using docker-compose.
Also, acknowledging the fact that the docker-compose file for modular deployments has been getting out of hand with auth support, the docker-compose.yml file was re-formatted for better visibility in configuration. This seemed like a better way to go than mandating a .env file.
Replace hardcoded localhost:6379 values throughout codebase with environment-based configuration. Add REDIS_PORT support and allow REDIS_URL override for external Redis services. Configure Celery broker/result backend to use Redis settings with environment variable overrides.
Closes#762
System Event Logging:
- Add SystemEvent model with 15 event types tracking channel operations, client connections, M3U/EPG activities, and buffering events
- Log detailed metrics for M3U/EPG refresh operations (streams/programs created/updated/deleted)
- Track M3U/EPG downloads with client information (IP address, user agent, profile, channel count)
- Record channel lifecycle events (start, stop, reconnect) with stream and client details
- Monitor client connections/disconnections and buffering events with stream metadata
Event Viewer UI:
- Add SystemEvents component with real-time updates via WebSocket
- Implement pagination, filtering by event type, and configurable auto-refresh
- Display events with color-coded badges and type-specific icons
- Integrate event viewer into Stats page with modal display
- Add event management settings (retention period, refresh rate)
M3U/EPG Endpoint Optimizations:
- Implement content caching with 5-minute TTL to reduce duplicate processing
- Add client-based event deduplication (2-second window) using IP and user agent hashing
- Support HEAD requests for efficient preflight checks
- Cache streamed EPG responses while maintaining streaming behavior for first request
Improved try_next_stream to not fail if the returned stream is the same. It will now try a different stream.
Force a client to jump ahead in the buffer if they fall to far behind.
