- Register Lua scripts for optimized chunk retrieval in StreamBuffer.
- Implement atomic Lua binary search to find the oldest available chunk in Redis.
- Update StreamGenerator to handle expired chunks more efficiently, reducing TOCTOU race conditions.
The redirect path in stream_ts() released the connection slot via release_stream() but did not reset the connection_allocated flag. Could have could caused a redundant no-op call in the exception handler.
If ProxyServer() raises during singleton construction, _instance was left
as _INITIALIZING permanently, causing all subsequent get_instance() callers
to spin in an infinite gevent.sleep() loop. Wrap construction in try/except
and reset _instance to None on failure so callers can retry.
Also replace the non-atomic setnx() + expire() pair in try_acquire_ownership()
with a single atomic SET NX EX call, consistent with the approach already used
in extend_ownership() and eliminating the race window where a crash between the
two calls could leave a key with no TTL (permanent ownership lock).
- Use atomic SET NX EX instead of separate SETNX + EXPIRE to prevent
zombie locks if the process crashes between the two calls
- Replace time.sleep() with gevent.sleep() in get_instance() spin-wait
to avoid blocking the greenlet hub
- Defer cleanup when ownership is lost but clients are still connected,
giving the new owner time to establish its stream before we tear down
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three interrelated bugs cause TS proxy streams to terminate prematurely
in multi-worker uWSGI deployments:
1. Double ProxyServer instantiation: ProxyConfig.ready() in apps/proxy/apps.py
calls TSProxyServer() directly, bypassing get_instance(). The subsequent
TSProxyConfig.ready() call to get_instance() creates a second instance.
Each instance starts its own cleanup thread, but only one holds channel
data — the orphaned cleanup thread cannot extend ownership.
2. Redis flushdb() on every client init: RedisClient.get_client() in
core/utils.py calls flushdb() whenever a new connection is created.
Celery autoscale workers spawning mid-stream nuke all Redis keys
including ownership, client records, and channel metadata.
3. No recovery from expired ownership: get_channel_owner() has a TOCTOU
bug (two separate GET calls in a lambda). extend_ownership() silently
fails when keys expire. Non-owner cleanup unconditionally kills streams
even when the worker holds the stream_manager.
Fixes:
- Use TSProxyServer.get_instance() in ProxyConfig.ready()
- Remove flushdb() from Redis client initialization
- Use sentinel pattern for gevent-safe singleton (threading.Lock does not
work with gevent greenlets)
- Single GET in get_channel_owner() to avoid TOCTOU race
- Re-acquire expired ownership keys in extend_ownership()
- Attempt re-acquisition before cleanup in non-owner path
Relates to #992, #980
Multiple code paths call release_stream() during channel shutdown (stream generator cleanup, _clean_redis_keys, ChannelService.stop_channel), causing
the profile_connections counter to be decremented multiple times for the same stream. The metadata hash fallback persisted longer than the primary keys, allowing subsequent calls to find stale stream_id/m3u_profile and DECR again.
Changes:
- Clear STREAM_ID and M3U_PROFILE metadata fields after decrementing in both the primary and fallback paths, making release_stream() safe to call multiple times
- Add metadata fallback when stream_profile:{id} key is missing but channel_stream:{id} exists, preventing a counter leak in that edge case
- Replace redundant get_stream() call in views.py with direct Redis reads to avoid side-effect INCR from a method that can allocate connection slots
Fix multiple VOD proxy connection counter leak paths that caused
profile_connections to remain permanently elevated after client
disconnects, leaving profiles locked at capacity.
Changes:
- Fix get_stream() race condition: save state under lock with fresh active_streams to prevent overwriting concurrent decrements during scrubbing/seeking (root cause of counter stuck after timeshift)
- Add early active_streams increment in session reuse path to prevent cleanup race between GeneratorExit and new request
- Add rollback on get_stream() returning None (range not satisfiable)
- Add profile_id fallback to closure variable when Redis state is deleted before generator cleanup runs
- Change increment/decrement return types and add warning logs for lock acquisition and state failures
- Clean up debug traceback logging from increment/decrement methods
Three fixes:
- TOCTOU: Replace GET-check-INCR with atomic INCR-first-then-check pattern in both connection managers to prevent concurrent requests exceeding max_streams
- Immediate DECR: Decrement profile counter directly in stream generator exit paths (normal completion, client disconnect, error) instead of deferring to daemon thread cleanup which may never execute
- Rollback: Decrement profile counter on create_connection() failure so the reserved slot is released
Fixes#962
Fix exception handling in _clean_redis_keys() and stop_channel()
where `except Channel.DoesNotExist` didn't catch ValidationError from invalid UUID format, bypassing the Stream fallback path.
When previewing a stream directly (via stream hash), the cleanup in stream_generator.py tried Channel.objects.get(uuid=stream_hash) which failed with ValidationError since the hash is not a UUID. This
prevented release_stream() from running, permanently leaking the
connection counter. With max_streams=1, a single preview permanently blocked all subsequent streams.
Added Stream.objects.get(stream_hash=...) fallback so the cleanup
path works for both Channel (UUID) and Stream (hash) identifiers.
Also added Stream import to stream_generator.py.
The local `from apps.channels.models import Channel` inside a
conditional block in _handle_client_disconnect() shadowed the top-level import. When the condition was False, Python's function-level scoping caused line 462's Channel.objects.get() to raise UnboundLocalError. Channel is already imported at module level (line 11), making the local import unnecessary.
Discovered when testing proxy changes
Wrap _clean_redis_keys() call in check_if_channel_exists() with
try/except to prevent exception propagation when Channel/Stream
lookup or release_stream() encounters DB or Redis errors.
All 12 call sites for Channel.release_stream() and Stream.release_stream() now check the boolean return value and log warnings when release fails.
Files updated:
- server.py: 4 call sites (zombie cleanup + _clean_redis_keys)
- channel_service.py: 2 call sites (stop_channel)
- stream_generator.py: 1 call site (last client disconnect)
- views.py: 4 call sites (failed init, redirect, init failure, exception)
- url_utils.py: 1 call site (URL generation error)
Also fixed bare except clause in _clean_redis_keys and uncaught
Stream.DoesNotExist in channel_service.stop_channel.
Based on PR #838 by patchy8736
- multi_worker_connection_manager.py: Add missing count > 0 guard to multi-worker remove_connection()
This method doesn't appear to be called anywhere but added the guard in case it is put back into service
- multi_worker_connection_manager.py: Error handler at line 1042: removed connection_manager=self from cleanup() call since the DECR was already done at line 1036. Prevents double-DECR.
- connection_manager.py: Added connection_key_exists check before calling remove_connection(). The fallback DECR from session data now only fires when the connection tracking key has already expired. Prevents double-DECR.
- stream_generator.py: _schedule_channel_shutdown_if_needed() already guards itself, the if logic created conflict between release_stream() and stop_channel(), so if the stream was already released the shutdown would not be scheduled
Fixed four leak paths in the TS proxy streaming initialization:
1. url_utils.py - generate_stream_url(): Wrapped post-get_stream() code in try/except that calls release_stream() on any failure
2. views.py - stream_ts() retry loop: Added release_stream() before returning 503 to clean up the error-checking get_stream() call at line 181 which could INCR without a matching release when the retry loop times out
3. views.py - stream_ts() init failure: Added guarded release_stream() when initialize_channel() returns False, using a connection_allocated flag to prevent incorrect decrements when joining an existing channel
4. views.py - stream_ts() outer except: Added guarded release_stream() as a safety net for unexpected exceptions, only releasing when the flag confirms the slot was allocated and the channel hasn't been initialized yet
The connection_allocated flag prevents double-decrements by tracking whether request flow actually INCR'd the counter (fresh initialization) vs
returned cached results (joining an existing channel on another worker)
Add support for authentication when connecting to external Redis instances in modular deployment mode. This enables secure Redis deployments using either password-only authentication (Redis <6) or
username + password authentication (Redis 6+ ACL).
Changes:
- Add REDIS_PASSWORD and REDIS_USER environment variables
- Implement URL encoding for special characters in passwords
- Update all Redis connection points to support auth
- Add comprehensive documentation and examples to docker-compose.yml
- Maintains full backward compatibility (empty defaults = no auth)
All authentication mechanisms have been fully tested including pasword-only authentication, Redis 6+ ACL authentication with username + password, volume-mounted configuration files, and special character handling.
Existing deployments are not effected, authentication support is entirely opt-in using docker-compose.
Also, acknowledging the fact that the docker-compose file for modular deployments has been getting out of hand with auth support, the docker-compose.yml file was re-formatted for better visibility in configuration. This seemed like a better way to go than mandating a .env file.
Replace hardcoded localhost:6379 values throughout codebase with environment-based configuration. Add REDIS_PORT support and allow REDIS_URL override for external Redis services. Configure Celery broker/result backend to use Redis settings with environment variable overrides.
Closes#762
System Event Logging:
- Add SystemEvent model with 15 event types tracking channel operations, client connections, M3U/EPG activities, and buffering events
- Log detailed metrics for M3U/EPG refresh operations (streams/programs created/updated/deleted)
- Track M3U/EPG downloads with client information (IP address, user agent, profile, channel count)
- Record channel lifecycle events (start, stop, reconnect) with stream and client details
- Monitor client connections/disconnections and buffering events with stream metadata
Event Viewer UI:
- Add SystemEvents component with real-time updates via WebSocket
- Implement pagination, filtering by event type, and configurable auto-refresh
- Display events with color-coded badges and type-specific icons
- Integrate event viewer into Stats page with modal display
- Add event management settings (retention period, refresh rate)
M3U/EPG Endpoint Optimizations:
- Implement content caching with 5-minute TTL to reduce duplicate processing
- Add client-based event deduplication (2-second window) using IP and user agent hashing
- Support HEAD requests for efficient preflight checks
- Cache streamed EPG responses while maintaining streaming behavior for first request
