diff --git a/CHANGELOG.md b/CHANGELOG.md index f555668f..008d1205 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - **Native XZ decompression for EPG sources and uploaded M3U playlists.** `.xz`-compressed XMLTV and M3U files are now decompressed using Python's stdlib `lzma` module. EPG ingestion detects XZ via magic bytes, file extension (including compound names like `.xml.xz`), or MIME type; uploaded M3U `.xz` playlists stream line-by-line like `.gz`. The `/data/epgs` auto-import scanner now includes `.xz` files alongside `.xml`, `.gz`, and `.zip`. (Closes #1414) — Thanks [@MotWakorb](https://github.com/MotWakorb) -- **XC catch-up (timeshift) support**. Adds a native `/timeshift/{user}/{pass}/{stream_id}/{timestamp}/{duration}.ts` endpoint that proxies replay sessions from an Xtream Codes provider to clients such as iPlayTV (Apple TV) and TiviMate. Auth reuses the same `xc_password` custom-property the live `/live/.../{id}.ts` endpoint already uses. Catch-up sessions appear on `/stats` with a violet `TIMESHIFT` badge alongside live sessions and respect per-channel access rules. (Closes #133) — Thanks [@cedric-marcoux](https://github.com/cedric-marcoux) +- **XC catch-up (timeshift) support**. Adds a native `/timeshift/{user}/{pass}/{stream_id}/{timestamp}/{duration}.ts` endpoint that proxies replay sessions from an Xtream Codes provider to clients such as iPlayTV (Apple TV) and TiviMate. Auth reuses the same `xc_password` custom-property the live `/live/.../{id}.ts` endpoint already uses. Catch-up sessions appear on the Stats page in dedicated connection cards (separate from live and VOD) and respect per-channel access rules. (Closes #133) — Thanks [@cedric-marcoux](https://github.com/cedric-marcoux) + - **REST catch-up API for native apps.** `POST /api/catchup/sessions/` (JWT or API key) mints a server-side playback session and returns a tokenless `playback_url` at `/proxy/catchup/?session_id=...` for headerless video players. `DELETE /api/catchup/sessions//` ends the session. OpenAPI docs cover handshake and idle TTL behaviour. + - **Catch-up admin APIs.** `GET /proxy/catchup/stats/` lists active catch-up viewers; `POST /proxy/catchup/programs/` returns batch EPG metadata for those sessions; `POST /proxy/catchup/stop_client/` stops a viewer from the admin UI. + - **Catch-up Stats UI.** Dedicated catch-up connection cards show channel logo, programme preview (watched/remaining timeline), playback position, bitrate, and a stop button. Stats refresh in real time via WebSocket (`timeshift_stats`) with desktop notifications when catch-up sessions start or end. - **Catch-up flags** (`tv_archive`, `tv_archive_duration`) denormalized at import time for zero-cost output queries; end-of-refresh SQL rollup updates channels linked to the refreshed account and self-heals stale flags on those channels only (e.g. after catch-up streams are removed or an account is deactivated). - **Strictly-UTC API surface, automatic per-provider timezone.** `server_info.timezone` is always `UTC` and the XC EPG `start`/`end` strings are emitted in UTC; the proxy converts the requested instant to the serving provider's own reported timezone (`server_info.timezone` captured on account refresh) at request time. No timezone to configure. - **Multi-provider failover.** Catch-up walks the channel's catch-up streams in order (like live playback): if one provider cannot serve the archive the next is tried, each attempt with its own account context (credentials, reported timezone, user-agent). Accounts that fail with an auth/ban-class status are not retried via their other streams. @@ -20,6 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **EPG XMLTV `prev_days` auto-detection** from the provider's largest `tv_archive_duration` (capped at 30 days), overridable via setting or `?prev_days=` URL parameter. - **Byte path streams directly via `iter_content` + generator yield** (same pattern as the VOD proxy), with an upfront MPEG-TS sync peek that strips any PHP-warning preamble before bytes reach strict demuxers. Throughput stays comfortably above the typical 5 Mbps FHD bitrate so clients don't buffer. - **Catch-up indicators in Channels and Streams tables.** Channels and streams with catch-up enabled show a grey history icon beside the name (tooltip includes archive days when known). Expanded channel stream rows show a catch-up badge. Channel and stream API serializers expose `is_catchup` and `catchup_days` for the UI. The Channels and Streams table filter menus include **Only Catch-up** to narrow each list to catch-up entries. +- **Combined connection stats API.** `GET /proxy/stats/` (admin) returns live, VOD, and catch-up connection stats in one JSON response (`live`, `vod`, `catchup`, `timestamp`). The Stats page polls this endpoint instead of separate live and VOD stats requests. - **Frontend unit tests extended to table components.** `ChannelsTable`, `StreamsTable`, `M3UsTable`, `EPGsTable`, `LogosTable`, `CustomTable`, and related header/editable subcomponents now have Vitest + Testing Library suites. Table business logic was extracted into `frontend/src/utils/tables/*` (with shared `M3uTableUtils` for M3U/EPG sort headers) and covered by dedicated util tests. `dateTimeUtils.formatDuration` gained a `human` precision mode for readable content lengths. — Thanks [@nick4810](https://github.com/nick4810) ### Changed diff --git a/apps/api/urls.py b/apps/api/urls.py index b176bc1b..5af829d3 100644 --- a/apps/api/urls.py +++ b/apps/api/urls.py @@ -12,6 +12,7 @@ urlpatterns = [ path('core/', include(('core.api_urls', 'core'), namespace='core')), path('plugins/', include(('apps.plugins.api_urls', 'plugins'), namespace='plugins')), path('vod/', include(('apps.vod.api_urls', 'vod'), namespace='vod')), + path('catchup/', include(('apps.timeshift.api_urls', 'catchup'), namespace='catchup')), path('backups/', include(('apps.backups.api_urls', 'backups'), namespace='backups')), path('connect/', include(('apps.connect.api_urls', 'connect'), namespace='connect')), # path('output/', include(('apps.output.api_urls', 'output'), namespace='output')), diff --git a/apps/channels/tests/test_ts_proxy_keepalive.py b/apps/channels/tests/test_ts_proxy_keepalive.py index 8645a158..84f764ef 100644 --- a/apps/channels/tests/test_ts_proxy_keepalive.py +++ b/apps/channels/tests/test_ts_proxy_keepalive.py @@ -214,11 +214,12 @@ class DoStatsUpdateTests(TestCase): """_do_stats_update must call send_websocket_update with channel_stats.""" cm = self._make_client_manager() - mock_redis = MagicMock() - mock_redis.scan.return_value = (0, []) - with patch("apps.proxy.live_proxy.client_manager.send_websocket_update") as mock_ws, \ - patch("core.utils.RedisClient.get_client", return_value=mock_redis): + patch( + "apps.proxy.live_proxy.channel_status.build_live_channel_stats_data", + return_value={"channels": [], "count": 0}, + ), \ + patch("core.utils.RedisClient.get_client", return_value=MagicMock()): cm._do_stats_update() mock_ws.assert_called_once() @@ -237,19 +238,20 @@ class DoStatsUpdateTests(TestCase): except Exception as e: self.fail(f"_do_stats_update raised an exception: {e}") - def test_do_stats_update_scans_channel_metadata_keys(self): - """Must scan for live:channel:*:metadata pattern.""" + def test_do_stats_update_uses_build_live_channel_stats_data(self): + """Must build live stats via the shared builder.""" cm = self._make_client_manager() - mock_redis = MagicMock() - mock_redis.scan.return_value = (0, []) with patch("apps.proxy.live_proxy.client_manager.send_websocket_update"), \ + patch( + "apps.proxy.live_proxy.channel_status.build_live_channel_stats_data", + return_value={"channels": [{"channel_id": "ch-1"}], "count": 1}, + ) as mock_build, \ patch("core.utils.RedisClient.get_client", return_value=mock_redis): cm._do_stats_update() - scan_call = mock_redis.scan.call_args - self.assertIn("live:channel:*:metadata", str(scan_call)) + mock_build.assert_called_once_with(mock_redis) # --------------------------------------------------------------------------- diff --git a/apps/proxy/live_proxy/channel_status.py b/apps/proxy/live_proxy/channel_status.py index 300f3b0d..58c66fbc 100644 --- a/apps/proxy/live_proxy/channel_status.py +++ b/apps/proxy/live_proxy/channel_status.py @@ -1,3 +1,4 @@ +import re import time from .server import ProxyServer from .redis_keys import RedisKeys @@ -407,8 +408,6 @@ class ChannelStatus: if channel_name: info['channel_name'] = channel_name - info['is_timeshift'] = bool(metadata.get(ChannelMetadataField.IS_TIMESHIFT)) - for key, field in ( ('logo_id', ChannelMetadataField.LOGO_ID), ('m3u_profile_id', ChannelMetadataField.M3U_PROFILE), @@ -433,7 +432,7 @@ class ChannelStatus: info['stream_name'] = stream_name # Add data throughput information to basic info - # TOTAL_BYTES is already present in the hgetall result — avoid a redundant round-trip + # TOTAL_BYTES is already in the hgetall result; skip a redundant round-trip. total_bytes_bytes = metadata.get(ChannelMetadataField.TOTAL_BYTES) if total_bytes_bytes: total_bytes = int(total_bytes_bytes) @@ -552,3 +551,34 @@ class ChannelStatus: except Exception as e: logger.error(f"Error getting channel info: {e}", exc_info=True) return None + + +def build_live_channel_stats_data(redis_client): + """Scan Redis for live channel metadata and build the stats payload.""" + empty = {"channels": [], "count": 0} + if not redis_client: + return empty + + try: + all_channels = [] + channel_pattern = "live:channel:*:metadata" + cursor = 0 + while True: + cursor, keys = redis_client.scan(cursor, match=channel_pattern) + for key in keys: + key_str = key.decode() if isinstance(key, bytes) else key + channel_id_match = re.search(r"live:channel:(.*):metadata", key_str) + if not channel_id_match: + continue + ch_id = channel_id_match.group(1) + channel_info = ChannelStatus.get_basic_channel_info(ch_id) + if channel_info: + all_channels.append(channel_info) + + if cursor == 0: + break + + return {"channels": all_channels, "count": len(all_channels)} + except Exception as e: + logger.error(f"Error building live channel stats: {e}", exc_info=True) + return empty diff --git a/apps/proxy/live_proxy/client_manager.py b/apps/proxy/live_proxy/client_manager.py index 23f8e768..5d7b45da 100644 --- a/apps/proxy/live_proxy/client_manager.py +++ b/apps/proxy/live_proxy/client_manager.py @@ -49,27 +49,14 @@ class ClientManager: def _do_stats_update(self): """Perform the stats update in the background.""" try: - from apps.proxy.live_proxy.channel_status import ChannelStatus + from apps.proxy.live_proxy.channel_status import build_live_channel_stats_data from core.utils import RedisClient redis_client = RedisClient.get_client() if not redis_client: return - all_channels = [] - cursor = 0 - while True: - cursor, keys = redis_client.scan(cursor, match="live:channel:*:metadata", count=100) - for key in keys: - parts = key.split(':') - if len(parts) >= 4: - ch_id = parts[2] - channel_info = ChannelStatus.get_basic_channel_info(ch_id) - if channel_info: - all_channels.append(channel_info) - - if cursor == 0: - break + live_stats = build_live_channel_stats_data(redis_client) send_websocket_update( "updates", @@ -77,7 +64,7 @@ class ClientManager: { "success": True, "type": "channel_stats", - "stats": json.dumps({'channels': all_channels, 'count': len(all_channels)}) + "stats": json.dumps(live_stats), } ) except Exception as e: diff --git a/apps/proxy/live_proxy/constants.py b/apps/proxy/live_proxy/constants.py index 414e1a2f..e0c2d245 100644 --- a/apps/proxy/live_proxy/constants.py +++ b/apps/proxy/live_proxy/constants.py @@ -54,7 +54,8 @@ class ChannelMetadataField: STREAM_ID = "stream_id" CHANNEL_NAME = "channel_name" STREAM_NAME = "stream_name" - IS_TIMESHIFT = "is_timeshift" + CHANNEL_ID = "channel_id" + CHANNEL_UUID = "channel_uuid" LOGO_ID = "logo_id" # Profile fields diff --git a/apps/proxy/live_proxy/server.py b/apps/proxy/live_proxy/server.py index 08a029e3..97c5b68b 100644 --- a/apps/proxy/live_proxy/server.py +++ b/apps/proxy/live_proxy/server.py @@ -2095,9 +2095,6 @@ class ProxyServer: if not channel_id: continue - if channel_id.startswith("timeshift_"): - continue - # Get metadata first metadata = self.redis_client.hgetall(key) if not metadata: diff --git a/apps/proxy/live_proxy/views.py b/apps/proxy/live_proxy/views.py index 7001336f..cf766505 100644 --- a/apps/proxy/live_proxy/views.py +++ b/apps/proxy/live_proxy/views.py @@ -8,7 +8,7 @@ from django.http import StreamingHttpResponse, JsonResponse, HttpResponseRedirec from django.views.decorators.csrf import csrf_exempt from django.shortcuts import get_object_or_404 from .server import ProxyServer -from .channel_status import ChannelStatus +from .channel_status import ChannelStatus, build_live_channel_stats_data from .output.ts.generator import create_stream_generator from .output.fmp4.generator import create_fmp4_stream_generator from .utils import get_client_ip @@ -872,28 +872,7 @@ def channel_status(request, channel_id=None): {"error": f"Channel {channel_id} not found"}, status=404 ) else: - # Basic info for all channels - channel_pattern = "live:channel:*:metadata" - all_channels = [] - - # Extract channel IDs from keys - cursor = 0 - while True: - cursor, keys = proxy_server.redis_client.scan( - cursor, match=channel_pattern - ) - for key in keys: - channel_id_match = re.search( - r"live:channel:(.*):metadata", key - ) - if channel_id_match: - ch_id = channel_id_match.group(1) - channel_info = ChannelStatus.get_basic_channel_info(ch_id) - if channel_info: - all_channels.append(channel_info) - - if cursor == 0: - break + live_stats = build_live_channel_stats_data(proxy_server.redis_client) # Send WebSocket update with the stats # Format it the same way the original Celery task did @@ -903,11 +882,11 @@ def channel_status(request, channel_id=None): { "success": True, "type": "channel_stats", - "stats": json.dumps({'channels': all_channels, 'count': len(all_channels)}) + "stats": json.dumps(live_stats), } ) - return JsonResponse({"channels": all_channels, "count": len(all_channels)}) + return JsonResponse(live_stats) except Exception as e: logger.error(f"Error in channel_status: {e}", exc_info=True) diff --git a/apps/proxy/stats_views.py b/apps/proxy/stats_views.py new file mode 100644 index 00000000..d8ddd32c --- /dev/null +++ b/apps/proxy/stats_views.py @@ -0,0 +1,31 @@ +"""Combined connection stats for live, VOD, and catch-up.""" + +import logging +import time + +from django.http import JsonResponse +from rest_framework.decorators import api_view, permission_classes + +from apps.accounts.permissions import IsAdmin +from apps.proxy.live_proxy.channel_status import build_live_channel_stats_data +from apps.proxy.vod_proxy.views import build_vod_stats_data +from apps.timeshift.stats import build_timeshift_stats_data +from core.utils import RedisClient + +logger = logging.getLogger(__name__) + + +@api_view(["GET"]) +@permission_classes([IsAdmin]) +def combined_stats(request): + """Return live, VOD, and catch-up stats in one response.""" + redis_client = RedisClient.get_client() + if not redis_client: + return JsonResponse({"error": "Redis not available"}, status=500) + + return JsonResponse({ + "live": build_live_channel_stats_data(redis_client), + "vod": build_vod_stats_data(redis_client), + "catchup": build_timeshift_stats_data(redis_client), + "timestamp": time.time(), + }) diff --git a/apps/proxy/tasks.py b/apps/proxy/tasks.py index 6f46eba5..cb30f185 100644 --- a/apps/proxy/tasks.py +++ b/apps/proxy/tasks.py @@ -1,10 +1,9 @@ from celery import shared_task import json import logging -import re import gc from core.utils import RedisClient -from apps.proxy.live_proxy.channel_status import ChannelStatus +from apps.proxy.live_proxy.channel_status import build_live_channel_stats_data from core.utils import send_websocket_update logger = logging.getLogger(__name__) @@ -17,29 +16,10 @@ def fetch_channel_stats(): redis_client = RedisClient.get_client() try: - # Basic info for all channels - channel_pattern = "live:channel:*:metadata" - all_channels = [] - - # Extract channel IDs from keys - cursor = 0 - while True: - cursor, keys = redis_client.scan(cursor, match=channel_pattern) - for key in keys: - channel_id_match = re.search(r"live:channel:(.*):metadata", key) - if channel_id_match: - ch_id = channel_id_match.group(1) - channel_info = ChannelStatus.get_basic_channel_info(ch_id) - if channel_info: - all_channels.append(channel_info) - - if cursor == 0: - break - + live_stats = build_live_channel_stats_data(redis_client) except Exception as e: logger.error(f"Error in channel_status: {e}", exc_info=True) return - # return JsonResponse({'error': str(e)}, status=500) send_websocket_update( "updates", @@ -47,13 +27,11 @@ def fetch_channel_stats(): { "success": True, "type": "channel_stats", - "stats": json.dumps({'channels': all_channels, 'count': len(all_channels)}) + "stats": json.dumps(live_stats), }, collect_garbage=True ) - # Explicitly clean up large data structures - all_channels = None gc.collect() diff --git a/apps/proxy/tests/__init__.py b/apps/proxy/tests/__init__.py new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/apps/proxy/tests/__init__.py @@ -0,0 +1 @@ + diff --git a/apps/proxy/tests/test_combined_stats.py b/apps/proxy/tests/test_combined_stats.py new file mode 100644 index 00000000..a71f7773 --- /dev/null +++ b/apps/proxy/tests/test_combined_stats.py @@ -0,0 +1,110 @@ +"""Tests for combined connection stats API and live stats builder.""" + +import json +from unittest.mock import MagicMock, patch + +from django.test import TestCase +from rest_framework.test import APIRequestFactory, force_authenticate + +from apps.accounts.models import User +from apps.proxy import stats_views +from apps.proxy.live_proxy.channel_status import build_live_channel_stats_data + + +class BuildLiveChannelStatsDataTests(TestCase): + @patch("apps.proxy.live_proxy.channel_status.ChannelStatus.get_basic_channel_info") + def test_builds_channel_list_from_metadata_scan(self, mock_get_info): + mock_get_info.side_effect = lambda ch_id: {"channel_id": ch_id} + + redis = MagicMock() + redis.scan.return_value = ( + 0, + [ + "live:channel:abc-uuid:metadata", + "live:channel:def-uuid:metadata", + ], + ) + + result = build_live_channel_stats_data(redis) + + self.assertEqual(result["count"], 2) + self.assertEqual( + [ch["channel_id"] for ch in result["channels"]], + ["abc-uuid", "def-uuid"], + ) + + def test_returns_empty_when_redis_unavailable(self): + result = build_live_channel_stats_data(None) + self.assertEqual(result, {"channels": [], "count": 0}) + + @patch("apps.proxy.live_proxy.channel_status.ChannelStatus.get_basic_channel_info") + def test_returns_empty_on_error(self, mock_get_info): + mock_get_info.side_effect = RuntimeError("redis blew up") + + redis = MagicMock() + redis.scan.return_value = (0, ["live:channel:abc-uuid:metadata"]) + + result = build_live_channel_stats_data(redis) + + self.assertEqual(result, {"channels": [], "count": 0}) + + +class CombinedStatsApiTests(TestCase): + @classmethod + def setUpTestData(cls): + cls.admin = User.objects.create( + username="combined-stats-admin", + user_level=User.UserLevel.ADMIN, + ) + + def setUp(self): + self.factory = APIRequestFactory() + + @patch("apps.proxy.stats_views.build_timeshift_stats_data") + @patch("apps.proxy.stats_views.build_vod_stats_data") + @patch("apps.proxy.stats_views.build_live_channel_stats_data") + @patch("apps.proxy.stats_views.RedisClient.get_client") + def test_combined_stats_returns_all_sections( + self, + redis_mock, + live_mock, + vod_mock, + catchup_mock, + ): + redis_mock.return_value = MagicMock() + live_mock.return_value = {"channels": [{"channel_id": "ch-1"}], "count": 1} + vod_mock.return_value = { + "vod_connections": [], + "total_connections": 0, + "timestamp": 100.0, + } + catchup_mock.return_value = { + "timeshift_sessions": [], + "total_connections": 0, + "timestamp": 100.0, + } + + request = self.factory.get("/proxy/stats/") + force_authenticate(request, user=self.admin) + response = stats_views.combined_stats(request) + + self.assertEqual(response.status_code, 200) + payload = json.loads(response.content) + self.assertEqual(payload["live"]["count"], 1) + self.assertIn("vod_connections", payload["vod"]) + self.assertIn("timeshift_sessions", payload["catchup"]) + self.assertIn("timestamp", payload) + live_mock.assert_called_once() + vod_mock.assert_called_once() + catchup_mock.assert_called_once() + + @patch("apps.proxy.stats_views.RedisClient.get_client") + def test_combined_stats_redis_unavailable(self, redis_mock): + redis_mock.return_value = None + + request = self.factory.get("/proxy/stats/") + force_authenticate(request, user=self.admin) + response = stats_views.combined_stats(request) + + self.assertEqual(response.status_code, 500) + self.assertIn("error", json.loads(response.content)) diff --git a/apps/proxy/tests/test_stream_limits.py b/apps/proxy/tests/test_stream_limits.py new file mode 100644 index 00000000..4fd3a07d --- /dev/null +++ b/apps/proxy/tests/test_stream_limits.py @@ -0,0 +1,285 @@ +"""Tests for per-user stream limit enforcement in apps.proxy.utils.""" + +from unittest.mock import MagicMock, patch + +from django.test import TestCase + +from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys +from apps.proxy.utils import ( + _STOP_REASON_ADMIN, + _timeshift_stop_channel_id, + attempt_stream_termination, + check_user_stream_limits, + stop_timeshift_client, +) + + +class _FakeRedis: + def __init__(self): + self.store = {} + + def setex(self, key, ttl, value): + self.store[key] = str(value) + + def hget(self, key, field): + hash_value = self.store.get(key) + return hash_value.get(field) if isinstance(hash_value, dict) else None + + def hset(self, key, field=None, value=None, mapping=None, **kwargs): + hash_value = self.store.get(key) + if not isinstance(hash_value, dict): + hash_value = {} + self.store[key] = hash_value + if field is not None and value is not None: + hash_value[str(field)] = str(value) + for f, v in (mapping or {}).items(): + hash_value[str(f)] = str(v) + + def delete(self, *keys): + count = 0 + for key in keys: + if self.store.pop(key, None) is not None: + count += 1 + return count + + def exists(self, key): + return key in self.store + + def sadd(self, key, *members): + existing = self.store.get(key) + if not isinstance(existing, set): + existing = set() + self.store[key] = existing + before = len(existing) + existing.update(str(m) for m in members) + return len(existing) - before + + def smembers(self, key): + value = self.store.get(key) + return set(value) if isinstance(value, set) else set() + + def scan_iter(self, match=None, count=None): # noqa: ARG002 + import fnmatch + + for key in list(self.store): + if match is None or fnmatch.fnmatch(str(key), match): + yield key + + def eval(self, script, numkeys, *keys_and_args): + if numkeys != 1 or len(keys_and_args) != 2: + raise NotImplementedError("FakeRedis eval only supports claim script") + key, token = keys_and_args + current = self.store.get(key) + if current is not None and str(current) == str(token): + self.store.pop(key, None) + return 1 + return 0 + + +class TimeshiftStopChannelIdTests(TestCase): + def setUp(self): + self.redis = _FakeRedis() + self.stats_channel_id = "8_victim" + self.client_id = "victim" + self.programme_vid = "8_2026-06-08-17-00_111" + + def test_resolves_programme_vid_from_stats_metadata(self): + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "programme_vid", self.programme_vid) + + resolved = _timeshift_stop_channel_id( + self.redis, self.stats_channel_id, self.client_id, + ) + self.assertEqual(resolved, self.programme_vid) + + def test_falls_back_to_stats_channel_id_for_legacy_entries(self): + resolved = _timeshift_stop_channel_id( + self.redis, self.stats_channel_id, self.client_id, + ) + self.assertEqual(resolved, self.stats_channel_id) + + +class AttemptStreamTerminationTests(TestCase): + def setUp(self): + self.redis = _FakeRedis() + self.user_id = 5 + self.requesting_client_id = "newsession" + self.stats_channel_id = "8_victim" + self.programme_vid = "8_2026-06-08-17-00_111" + self.victim_client_id = "victim" + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.victim_client_id, + ) + self.redis.hset(client_key, "programme_vid", self.programme_vid) + + def _connections(self): + return [{ + "media_id": self.stats_channel_id, + "client_id": self.victim_client_id, + "connected_at": 1000.0, + "type": "timeshift", + }] + + def _limits_settings(self): + return { + "terminate_oldest": True, + "prioritize_single_client_channels": True, + "ignore_same_channel_connections": False, + } + + def test_timeshift_stop_key_targets_programme_vid_not_stats_channel(self): + with patch("apps.proxy.utils.RedisClient.get_client", + return_value=self.redis), \ + patch("apps.proxy.utils.CoreSettings.get_user_limits_settings", + return_value=self._limits_settings()): + ok = attempt_stream_termination( + self.user_id, self.requesting_client_id, self._connections(), + ) + + self.assertTrue(ok) + programme_stop = RedisKeys.client_stop( + self.programme_vid, self.victim_client_id, + ) + stats_stop = RedisKeys.client_stop( + self.stats_channel_id, self.victim_client_id, + ) + self.assertIn(programme_stop, self.redis.store) + self.assertNotIn(stats_stop, self.redis.store) + + def test_timeshift_limit_stop_uses_generation_key_when_metadata_missing(self): + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.victim_client_id, + ) + self.redis.store.pop(client_key, None) + generation_key = ( + f"timeshift:stream_gen:{self.programme_vid}:{self.victim_client_id}" + ) + self.redis.store[generation_key] = "4" + + with patch("apps.proxy.utils.RedisClient.get_client", + return_value=self.redis), \ + patch("apps.proxy.utils.CoreSettings.get_user_limits_settings", + return_value=self._limits_settings()): + ok = attempt_stream_termination( + self.user_id, self.requesting_client_id, self._connections(), + ) + + self.assertTrue(ok) + programme_stop = RedisKeys.client_stop( + self.programme_vid, self.victim_client_id, + ) + self.assertIn(programme_stop, self.redis.store) + + def test_limit_termination_allows_new_timeshift_when_victim_stopped(self): + user = MagicMock(id=5, username="viewer", stream_limit=1) + new_session = "newsession" + connections = self._connections() + settings = { + **self._limits_settings(), + "terminate_on_limit_exceeded": True, + } + with patch("apps.proxy.utils.get_user_active_connections", + return_value=connections), \ + patch("apps.proxy.utils.RedisClient.get_client", + return_value=self.redis), \ + patch("apps.proxy.utils.CoreSettings.get_user_limits_settings", + return_value=settings): + allowed = check_user_stream_limits( + user, new_session, media_id="8_2026-06-08-17-30", + ) + + self.assertTrue(allowed) + programme_stop = RedisKeys.client_stop( + self.programme_vid, self.victim_client_id, + ) + self.assertIn(programme_stop, self.redis.store) + + def test_live_termination_still_uses_channel_service(self): + connections = [{ + "media_id": "42", + "client_id": "live_client_1", + "connected_at": 1000.0, + "type": "live", + }] + with patch("apps.proxy.utils.CoreSettings.get_user_limits_settings", + return_value=self._limits_settings()), \ + patch("apps.proxy.utils.ChannelService.stop_client", + return_value={"status": "ok"}) as stop_mock: + ok = attempt_stream_termination( + self.user_id, self.requesting_client_id, connections, + ) + + self.assertTrue(ok) + stop_mock.assert_called_once_with("42", "live_client_1") + + +class TimeshiftAdminStopTests(TestCase): + def setUp(self): + self.redis = _FakeRedis() + self.stats_channel_id = "8_victim" + self.programme_vid = "8_2026-06-08-17-00_111" + self.client_id = "victim" + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "programme_vid", self.programme_vid) + self.redis.sadd(RedisKeys.clients(self.stats_channel_id), self.client_id) + self.redis.hset( + RedisKeys.channel_metadata(self.stats_channel_id), + "state", "active", + ) + + def test_admin_stop_client_signals_programme_and_unregisters_stats(self): + session_key = RedisKeys.api_session(self.client_id) + self.redis.hset(session_key, "user_id", "1") + + with patch("apps.timeshift.views._unregister_stats_client") as unregister_mock, \ + patch("apps.timeshift.views._trigger_timeshift_stats_update") as trigger_mock: + result = stop_timeshift_client( + self.redis, self.stats_channel_id, self.client_id, + ) + + self.assertEqual(result["status"], "success") + programme_stop = RedisKeys.client_stop(self.programme_vid, self.client_id) + stats_stop = RedisKeys.client_stop(self.stats_channel_id, self.client_id) + self.assertEqual(self.redis.store.get(programme_stop), _STOP_REASON_ADMIN) + self.assertNotIn(stats_stop, self.redis.store) + self.assertNotIn(session_key, self.redis.store) + unregister_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.client_id, + ) + trigger_mock.assert_called_once_with(self.redis) + + def test_admin_stop_closes_local_upstream_when_registered(self): + from apps.timeshift.views import _close_active_upstream, _register_active_upstream + + upstream = MagicMock() + _register_active_upstream(self.programme_vid, self.client_id, upstream) + + with patch("apps.timeshift.views._unregister_stats_client"): + result = stop_timeshift_client( + self.redis, self.stats_channel_id, self.client_id, + ) + + self.assertEqual(result["status"], "success") + upstream.close.assert_called_once() + _close_active_upstream(self.programme_vid, self.client_id) + upstream.close.assert_called_once() + + def test_admin_stop_uses_generation_key_when_metadata_missing(self): + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.store.pop(client_key, None) + generation_key = f"timeshift:stream_gen:{self.programme_vid}:{self.client_id}" + self.redis.store[generation_key] = "4" + + with patch("apps.timeshift.views._unregister_stats_client"): + result = stop_timeshift_client( + self.redis, self.stats_channel_id, self.client_id, + ) + + self.assertEqual(result["status"], "success") + programme_stop = RedisKeys.client_stop(self.programme_vid, self.client_id) + self.assertEqual(self.redis.store.get(programme_stop), _STOP_REASON_ADMIN) + self.assertIn(self.programme_vid, result["stop_channel_ids"]) diff --git a/apps/proxy/urls.py b/apps/proxy/urls.py index efd93430..470aa588 100644 --- a/apps/proxy/urls.py +++ b/apps/proxy/urls.py @@ -1,9 +1,13 @@ from django.urls import path, include +from apps.proxy import stats_views + app_name = 'proxy' urlpatterns = [ + path('stats/', stats_views.combined_stats, name='combined_stats'), path('ts/', include('apps.proxy.live_proxy.urls')), + path('catchup/', include('apps.timeshift.urls')), path('hls/', include('apps.proxy.hls_proxy.urls')), path('vod/', include('apps.proxy.vod_proxy.urls')), ] diff --git a/apps/proxy/utils.py b/apps/proxy/utils.py index 081658a4..5dcd4d62 100644 --- a/apps/proxy/utils.py +++ b/apps/proxy/utils.py @@ -1,12 +1,145 @@ import logging +import re from core.utils import RedisClient from apps.proxy.vod_proxy.multi_worker_connection_manager import MultiWorkerVODConnectionManager, get_vod_client_stop_key -from apps.proxy.live_proxy.redis_keys import RedisKeys +from apps.timeshift.redis_keys import ( + TimeshiftRedisKeys, + parse_stats_channel_id, +) from core.models import CoreSettings from apps.proxy.live_proxy.services.channel_service import ChannelService logger = logging.getLogger("proxy") +_STOP_REASON_LIMIT = "limit" +_STOP_REASON_ADMIN = "admin" + + +def _timeshift_stop_channel_id(redis_client, stats_channel_id, client_id, fallback=None): + """Return the programme virtual_channel_id used for generator stop keys.""" + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + programme_vid = redis_client.hget(client_key, "programme_vid") + if programme_vid: + if isinstance(programme_vid, bytes): + programme_vid = programme_vid.decode() + return programme_vid + return fallback if fallback is not None else stats_channel_id + + +def _timeshift_generation_channel_ids(redis_client, client_id): + """Return virtual channel ids with active generation counters for a client.""" + if redis_client is None or not client_id: + return [] + prefix = "timeshift:stream_gen:" + suffix = f":{client_id}" + try: + keys = redis_client.scan_iter( + match=f"{prefix}*{suffix}", count=100, + ) + channel_ids = [] + for key in keys: + if isinstance(key, bytes): + key = key.decode() + key = str(key) + if not key.startswith(prefix) or not key.endswith(suffix): + continue + channel_id = key[len(prefix):-len(suffix)] + if channel_id: + channel_ids.append(channel_id) + return channel_ids + except Exception as exc: + logger.debug( + "Timeshift generation stop-key scan failed for %s: %s", + client_id, exc, + ) + return [] + + +def _timeshift_stop_channel_ids(redis_client, stats_channel_id, client_id, fallback=None): + """Return all Redis stop-key channel ids that may be observed by a stream.""" + candidates = [ + _timeshift_stop_channel_id( + redis_client, stats_channel_id, client_id, fallback=fallback, + ) + ] + candidates.extend(_timeshift_generation_channel_ids(redis_client, client_id)) + + seen = set() + ordered = [] + for channel_id in candidates: + if not channel_id or channel_id in seen: + continue + seen.add(channel_id) + ordered.append(channel_id) + return ordered + + +def _set_timeshift_stop_keys(redis_client, stats_channel_id, client_id, reason, fallback=None): + """Signal a timeshift stream to stop across workers.""" + stop_channel_ids = _timeshift_stop_channel_ids( + redis_client, stats_channel_id, client_id, fallback=fallback, + ) + for stop_channel_id in stop_channel_ids: + stop_key = TimeshiftRedisKeys.client_stop(stop_channel_id, client_id) + redis_client.setex(stop_key, 60, reason) + return stop_channel_ids + + +def stop_timeshift_client(redis_client, stats_channel_id, client_id): + """Stop one catch-up viewer (admin Stats UI and ``POST /proxy/catchup/stop_client/``).""" + if redis_client is None: + return {"status": "error", "message": "Redis unavailable"} + if not stats_channel_id or not client_id: + return {"status": "error", "message": "Missing channel or client id"} + + from apps.timeshift.views import ( + _cancel_stats_disconnect_grace, + _cleanup_all_stream_generations, + _close_active_upstream, + _finalize_playback_session_auth, + _trigger_timeshift_stats_update, + _unregister_stats_client, + ) + + try: + stop_channel_ids = _set_timeshift_stop_keys( + redis_client, stats_channel_id, client_id, _STOP_REASON_ADMIN, + ) + for stop_channel_id in stop_channel_ids: + _close_active_upstream(stop_channel_id, client_id) + _cancel_stats_disconnect_grace(redis_client, stats_channel_id, client_id) + _unregister_stats_client(redis_client, stats_channel_id, client_id) + _cleanup_all_stream_generations(redis_client, client_id) + _finalize_playback_session_auth(redis_client, client_id) + try: + from apps.timeshift.views import _superseded_pool_key + redis_client.delete(_superseded_pool_key(client_id)) + except Exception: + pass + except Exception as exc: + logger.error( + "Timeshift admin stop failed for %s on %s: %s", + client_id, stats_channel_id, exc, + ) + return {"status": "error", "message": str(exc)} + + _trigger_timeshift_stats_update(redis_client) + + logger.info( + "Timeshift admin stop: client=%s stats_channel=%s stop_channels=%s", + client_id, stats_channel_id, stop_channel_ids, + ) + return { + "status": "success", + "message": "Timeshift client stop signal sent", + "channel_id": stats_channel_id, + "client_id": client_id, + "stop_channel_id": stop_channel_ids[0] if stop_channel_ids else stats_channel_id, + "stop_channel_ids": stop_channel_ids, + "stop_key_set": True, + "locally_processed": False, + } + def attempt_stream_termination(user_id, requesting_client_id, active_connections): try: @@ -48,9 +181,7 @@ def attempt_stream_termination(user_id, requesting_client_id, active_connections f"on media {target['media_id']} (connected_at={target['connected_at']})" ) - # When counting by unique channel, freeing one connection from a multi-connection - # channel doesn't free a slot — terminate all connections to that channel so the - # unique-channel count actually drops by one. + # When counting by unique channel, stop all connections on that channel. targets = ( [c for c in active_connections if c['media_id'] == target['media_id']] if ignore_same_channel @@ -63,14 +194,20 @@ def attempt_stream_termination(user_id, requesting_client_id, active_connections if result.get("status") == "error": logger.warning(f"[stream limits][{requesting_client_id}] Failed to stop client {t['client_id']} on channel {t['media_id']}") elif t['type'] == 'timeshift': - # Same Redis stop key as live; timeshift generator polls it every 5s. redis_client = RedisClient.get_client() if not redis_client: # Deny the new stream if we cannot stop the old one. return False - stop_key = RedisKeys.client_stop(t['media_id'], t['client_id']) - redis_client.setex(stop_key, 60, "true") - logger.info(f"[stream limits][{requesting_client_id}] Set stop key for timeshift client {t['client_id']}") + stop_channel_ids = _set_timeshift_stop_keys( + redis_client, t['media_id'], t['client_id'], _STOP_REASON_LIMIT, + ) + from apps.timeshift.views import _close_active_upstream + for stop_channel_id in stop_channel_ids: + _close_active_upstream(stop_channel_id, t['client_id']) + logger.info( + f"[stream limits][{requesting_client_id}] Set stop key for " + f"timeshift client {t['client_id']} on {stop_channel_ids}", + ) else: connection_manager = MultiWorkerVODConnectionManager.get_instance() redis_client = connection_manager.redis_client @@ -101,32 +238,35 @@ def get_user_active_connections(user_id): connections = [] try: - # Grab live streams - for key in redis_client.scan_iter(match="live:channel:*:clients:*", count=1000): - parts = key.split(':') - if len(parts) >= 5: - channel_id = parts[2] - client_id = parts[4] + # Grab live and timeshift streams (same key layout, separate namespaces) + for pattern, conn_type in ( + ("live:channel:*:clients:*", "live"), + ("timeshift:channel:*:clients:*", "timeshift"), + ): + for key in redis_client.scan_iter(match=pattern, count=1000): + parts = key.split(':') + if len(parts) >= 5: + channel_id = parts[2] + client_id = parts[4] - client_user_id, connected_at = redis_client.hmget(key, 'user_id', 'connected_at') + client_user_id, connected_at = redis_client.hmget(key, 'user_id', 'connected_at') - logger.debug(f"[stream limits] user_id = {user_id}") - logger.debug(f"[stream limits] channel_id = {channel_id}") - logger.debug(f"[stream limits] client_id = {client_id}") + logger.debug(f"[stream limits] user_id = {user_id}") + logger.debug(f"[stream limits] channel_id = {channel_id}") + logger.debug(f"[stream limits] client_id = {client_id}") - if user_id is None or (client_user_id and int(client_user_id) == user_id): - try: - conn_type = 'timeshift' if channel_id.startswith('timeshift_') else 'live' - logger.debug(f"[stream limits] Found {conn_type.upper()} connection for user {user_id} on channel {channel_id} with client ID {client_id}") - connected_at = float(connected_at) if connected_at else 0 - connections.append({ - 'media_id': channel_id, - 'client_id': client_id, - 'connected_at': connected_at, - 'type': conn_type, - }) - except (ValueError, TypeError): - pass + if user_id is None or (client_user_id and int(client_user_id) == user_id): + try: + logger.debug(f"[stream limits] Found {conn_type.upper()} connection for user {user_id} on channel {channel_id} with client ID {client_id}") + connected_at = float(connected_at) if connected_at else 0 + connections.append({ + 'media_id': channel_id, + 'client_id': client_id, + 'connected_at': connected_at, + 'type': conn_type, + }) + except (ValueError, TypeError): + pass # Grab VOD for key in redis_client.scan_iter(match="vod_persistent_connection:*", count=1000): @@ -187,6 +327,10 @@ def check_user_stream_limits(user, client_id, media_id=None): # session_id. Each distinct client/session still consumes its own slot. if media_id and client_id: media_id_str = str(media_id) + req_channel = None + channel_match = re.match(r"^(\d+)_", media_id_str) + if channel_match: + req_channel = channel_match.group(1) for conn in active_connections: if conn.get('type') != 'timeshift': continue @@ -198,6 +342,17 @@ def check_user_stream_limits(user, client_id, media_id=None): f"[stream limits][{client_id}] Same timeshift session probe for {media_id} allowed" ) return True + if req_channel: + parsed_conn = parse_stats_channel_id(conn_media_id) + if parsed_conn: + conn_channel = str(parsed_conn["channel_id"]) + else: + conn_channel = conn_media_id.split("_", 1)[0] + if conn_channel == req_channel: + logger.debug( + f"[stream limits][{client_id}] Same timeshift channel {req_channel} allowed" + ) + return True if user_stream_count >= user.stream_limit: if user_limit_settings.get("terminate_on_limit_exceeded", True) == False: diff --git a/apps/timeshift/api_urls.py b/apps/timeshift/api_urls.py new file mode 100644 index 00000000..afcecddb --- /dev/null +++ b/apps/timeshift/api_urls.py @@ -0,0 +1,18 @@ +from django.urls import path + +from . import api_views + +app_name = "catchup" + +urlpatterns = [ + path( + "sessions/", + api_views.CatchupSessionCreateAPIView.as_view(), + name="catchup-session-create", + ), + path( + "sessions//", + api_views.CatchupSessionDestroyAPIView.as_view(), + name="catchup-session-destroy", + ), +] diff --git a/apps/timeshift/api_views.py b/apps/timeshift/api_views.py new file mode 100644 index 00000000..59954a30 --- /dev/null +++ b/apps/timeshift/api_views.py @@ -0,0 +1,168 @@ +"""REST API for native catch-up playback session setup.""" + +from django.http import Http404 +from drf_spectacular.utils import extend_schema, inline_serializer +from rest_framework import serializers, status +from rest_framework.response import Response +from rest_framework.views import APIView + +from apps.accounts.permissions import IsStandardUser +from apps.channels.models import Channel +from apps.channels.utils import get_channel_catchup_streams +from dispatcharr.utils import network_access_allowed + +from .helpers import parse_catchup_timestamp +from .sessions import ( + HANDSHAKE_TTL_SECONDS, + SESSION_IDLE_TTL_SECONDS, + create_catchup_session, + delete_catchup_session, + user_owns_catchup_session, +) +from .views import _user_can_access_channel + + +class CatchupSessionCreateSerializer(serializers.Serializer): + channel_uuid = serializers.UUIDField( + help_text="Dispatcharr channel UUID to play catch-up from.", + ) + start = serializers.CharField( + help_text=( + "UTC programme start time (which archived show to play). " + "ISO-8601 (2026-07-09T14:00:00Z), Unix epoch, or XC wall-clock shapes." + ), + ) + + +class CatchupSessionResponseSerializer(serializers.Serializer): + session_id = serializers.CharField() + playback_url = serializers.CharField() + expires_at = serializers.IntegerField( + help_text="Unix timestamp. Handshake deadline: first playback GET within this window.", + ) + channel_uuid = serializers.UUIDField() + start = serializers.CharField() + + +class CatchupSessionCreateAPIView(APIView): + """Mint a server-side playback session for headerless video players.""" + + permission_classes = [IsStandardUser] + + @extend_schema( + description=( + "Create a catch-up playback session for a single archived programme.\n\n" + "Call this once per programme with a JWT or API key. The response " + "includes a ``playback_url`` for the video player **without** an " + "embedded access token.\n\n" + "**``start``** is the programme's broadcast start time in UTC " + "(from EPG ``start_time``). It selects *which* archived show to " + "fetch, not when the viewer pressed play.\n\n" + f"The player should open ``playback_url`` within " + f"**{HANDSHAKE_TTL_SECONDS} seconds** (see ``expires_at``). After the " + "first byte request, the session stays valid with a " + f"**{SESSION_IDLE_TTL_SECONDS // 60}-minute sliding idle window** " + "(refreshed on each range/seek request).\n\n" + "Start a **new** session for each different programme." + ), + request=CatchupSessionCreateSerializer, + responses={ + 201: CatchupSessionResponseSerializer, + 400: inline_serializer( + name="CatchupSessionCreateError", + fields={"error": serializers.CharField()}, + ), + 403: inline_serializer( + name="CatchupSessionForbidden", + fields={"error": serializers.CharField()}, + ), + 404: inline_serializer( + name="CatchupSessionNotFound", + fields={"error": serializers.CharField()}, + ), + }, + tags=["catchup"], + ) + def post(self, request): + user = request.user + if not network_access_allowed(request, "STREAMS", user): + return Response({"error": "Forbidden"}, status=status.HTTP_403_FORBIDDEN) + + body = CatchupSessionCreateSerializer(data=request.data) + body.is_valid(raise_exception=True) + + start = body.validated_data["start"].strip() + if parse_catchup_timestamp(start) is None: + return Response( + {"error": "Invalid start timestamp"}, + status=status.HTTP_400_BAD_REQUEST, + ) + + channel_uuid = body.validated_data["channel_uuid"] + try: + channel = Channel.objects.get(uuid=channel_uuid) + except Channel.DoesNotExist: + raise Http404("Channel not found") from None + + if not _user_can_access_channel(user, channel): + return Response({"error": "Access denied"}, status=status.HTTP_403_FORBIDDEN) + + if not channel.is_catchup: + return Response( + {"error": "Catch-up not supported for this channel"}, + status=status.HTTP_400_BAD_REQUEST, + ) + + if not get_channel_catchup_streams(channel): + return Response( + {"error": "Catch-up not supported for this channel"}, + status=status.HTTP_400_BAD_REQUEST, + ) + + try: + payload = create_catchup_session(user=user, channel=channel, start=start) + except RuntimeError: + return Response( + {"error": "Session service unavailable"}, + status=status.HTTP_503_SERVICE_UNAVAILABLE, + ) + + return Response( + CatchupSessionResponseSerializer(payload).data, + status=status.HTTP_201_CREATED, + ) + + +class CatchupSessionDestroyAPIView(APIView): + """Revoke a catch-up playback session early.""" + + permission_classes = [IsStandardUser] + + @extend_schema( + description=( + "Delete a catch-up session before it expires. Only the user who " + "created the session may revoke it. Returns 404 when the session " + "is missing or owned by another user." + ), + responses={ + 204: None, + 403: inline_serializer( + name="CatchupSessionDestroyForbidden", + fields={"error": serializers.CharField()}, + ), + 404: inline_serializer( + name="CatchupSessionDestroyNotFound", + fields={"error": serializers.CharField()}, + ), + }, + tags=["catchup"], + ) + def delete(self, request, session_id): + if not network_access_allowed(request, "STREAMS", request.user): + return Response({"error": "Forbidden"}, status=status.HTTP_403_FORBIDDEN) + + if not user_owns_catchup_session(session_id, request.user.id): + return Response({"error": "Session not found"}, status=status.HTTP_404_NOT_FOUND) + + delete_catchup_session(session_id) + return Response(status=status.HTTP_204_NO_CONTENT) diff --git a/apps/timeshift/helpers.py b/apps/timeshift/helpers.py index 98d10c36..7b8d5ac1 100644 --- a/apps/timeshift/helpers.py +++ b/apps/timeshift/helpers.py @@ -42,6 +42,7 @@ def normalize_catchup_timestamp_input(timestamp_str): - ``YYYY-MM-DD_HH-MM`` (XC underscore) - ``YYYY-MM-DD:HH:MM[:SS]`` (XC colon time in catch-up URLs) - ``YYYY-MM-DD HH:MM[:SS]`` (EPG / SQL datetime) + - ISO-8601 UTC (``2026-07-09T14:00:00Z`` or with offset) - Unix epoch seconds (10 digits) or milliseconds (13 digits) Returns: @@ -66,6 +67,15 @@ def normalize_catchup_timestamp_input(timestamp_str): return dt.replace(tzinfo=None).isoformat(timespec="seconds") return None + if "T" in value: + try: + dt = datetime.fromisoformat(value.replace("Z", "+00:00")) + if dt.tzinfo is not None: + dt = dt.astimezone(timezone.utc).replace(tzinfo=None) + return dt.isoformat(timespec="seconds") + except ValueError: + return None + match = _CATCHUP_WALL_CLOCK_RE.match(value) if not match: return None @@ -173,6 +183,85 @@ def get_programme_duration(channel, timestamp_str): return DEFAULT_DURATION_MINUTES +def get_programme_info(channel, timestamp_str): + """Return EPG metadata for the programme airing at *timestamp_str*.""" + try: + dt = parse_catchup_timestamp(timestamp_str) + if dt is None: + return None + dt = dt.replace(tzinfo=timezone.utc) + if not channel or not getattr(channel, "epg_data", None): + return None + + programme = channel.epg_data.programs.filter( + start_time__lte=dt, end_time__gt=dt + ).first() + if not programme: + return None + + duration_seconds = (programme.end_time - programme.start_time).total_seconds() + return { + "title": programme.title, + "sub_title": programme.sub_title or "", + "description": programme.description or "", + "start_time": programme.start_time.isoformat(), + "end_time": programme.end_time.isoformat(), + "duration_secs": int(duration_seconds), + } + except Exception: + return None + + +def get_catchup_programmes_for_sessions(sessions): + """Resolve EPG metadata for catch-up stats cards (batch, on demand). + + Each session dict needs ``session_id``, ``channel_uuid``, and + ``programme_start``. Called by the frontend when active sessions or seek + position change, not on every stats poll. + """ + from apps.channels.models import Channel + + if not sessions: + return [] + + valid = [ + s for s in sessions + if s.get("channel_uuid") and s.get("programme_start") and s.get("session_id") + ] + if not valid: + return [] + + valid = valid[:50] + uuids = list({str(s["channel_uuid"]) for s in valid}) + channels_by_uuid = { + str(ch.uuid): ch + for ch in Channel.objects.filter(uuid__in=uuids).select_related("epg_data") + } + + results = [] + for session in valid: + channel_uuid = str(session["channel_uuid"]) + programme_start = session["programme_start"] + channel = channels_by_uuid.get(channel_uuid) + info = get_programme_info(channel, programme_start) if channel else None + entry = { + "session_id": session["session_id"], + "channel_uuid": channel_uuid, + "programme_start": programme_start, + } + if info: + entry.update({ + "title": info["title"], + "sub_title": info.get("sub_title", ""), + "description": info.get("description", ""), + "start_time": info["start_time"], + "end_time": info["end_time"], + "duration_secs": info["duration_secs"], + }) + results.append(entry) + return results + + def build_timeshift_url_format_a(creds, stream_id, timestamp, duration_minutes): """QUERY layout: ``/streaming/timeshift.php?username=...&start=...``""" return ( diff --git a/apps/timeshift/redis_keys.py b/apps/timeshift/redis_keys.py new file mode 100644 index 00000000..9ab927b7 --- /dev/null +++ b/apps/timeshift/redis_keys.py @@ -0,0 +1,96 @@ +"""Redis key patterns and logical ids for catch-up (timeshift). + +All catch-up state lives under the ``timeshift:`` prefix so live-proxy +janitors and live stats scans never see it. +""" + +import re +import secrets + +# Logical id: stats channel key segment ``timeshift:channel:{id}:...`` +_STATS_CHANNEL_RE = re.compile(r"^(\d+)_(.+)$") + + +class TimeshiftRedisKeys: + @staticmethod + def channel_metadata(channel_id): + return f"timeshift:channel:{channel_id}:metadata" + + @staticmethod + def clients(channel_id): + return f"timeshift:channel:{channel_id}:clients" + + @staticmethod + def client_metadata(channel_id, client_id): + return f"timeshift:channel:{channel_id}:clients:{client_id}" + + @staticmethod + def client_stop(channel_id, client_id): + return f"timeshift:channel:{channel_id}:client:{client_id}:stop" + + @staticmethod + def pool(session_id): + return f"timeshift:pool:{session_id}" + + @staticmethod + def pool_lock(session_id): + return f"timeshift:pool:{session_id}:lock" + + @staticmethod + def pool_superseded(session_id): + return f"timeshift:pool:{session_id}:superseded" + + @staticmethod + def pool_scan_pattern(): + return "timeshift:pool:*" + + @staticmethod + def api_session(session_id): + return f"timeshift:session:{session_id}" + + @staticmethod + def stats_grace(stats_channel_id, client_id): + return f"timeshift:grace:{stats_channel_id}:{client_id}" + + @staticmethod + def stream_generation(virtual_channel_id, client_id): + return f"timeshift:stream_gen:{virtual_channel_id}:{client_id}" + + @staticmethod + def stream_generation_scan_pattern(client_id): + return f"timeshift:stream_gen:*:{client_id}" + + @staticmethod + def format_cache(account_id): + return f"timeshift:format_idx:{account_id}" + + +def mint_session_id(): + """Opaque per-viewer session id (URL query param and pool key suffix).""" + return secrets.token_urlsafe(16) + + +def stats_channel_id(channel_id, session_id): + """Stable stats/redis channel id for one viewer on a channel.""" + return f"{channel_id}_{session_id}" + + +def programme_media_id(channel_id, safe_ts): + """Catch-up position identity (channel + programme timestamp).""" + return f"{channel_id}_{safe_ts}" + + +def virtual_channel_id(channel_id, safe_ts, stream_id_value): + """Programme virtual id for stop keys and stream generation.""" + return f"{channel_id}_{safe_ts}_{stream_id_value}" + + +def parse_stats_channel_id(stats_channel_id): + """Split a stats channel id into numeric channel id and session id.""" + match = _STATS_CHANNEL_RE.match(str(stats_channel_id or "")) + if not match: + return None + return { + "channel_id": int(match.group(1)), + "session_id": match.group(2), + } diff --git a/apps/timeshift/sessions.py b/apps/timeshift/sessions.py new file mode 100644 index 00000000..1836e29e --- /dev/null +++ b/apps/timeshift/sessions.py @@ -0,0 +1,197 @@ +"""Redis-backed catch-up playback sessions for native API clients. + +A session is minted by ``POST /api/catchup/sessions/`` (JWT/API key). The +returned ``session_id`` lets a headerless video player call +``GET /proxy/catchup/{uuid}?session_id=...`` without embedding a JWT in the URL. + +Lifecycle: + * **Handshake**: unused sessions expire ``HANDSHAKE_TTL_SECONDS`` after POST. + * **Playback**: the first GET extends TTL to ``SESSION_IDLE_TTL_SECONDS``; + each subsequent GET (or active-stream heartbeat) refreshes that sliding + window. Pausing longer than this without a new request requires minting + a new session. + * **End of viewing**: when the client disconnects for real (not a seek within + the same session), the playback layer deletes the record after a short grace + window so stale ``session_id`` values cannot be replayed until TTL expiry. + * **User resolution**: prefer ``timeshift:pool:{session_id}.user_id`` while + the provider pool entry exists; fall back to the API session record when the + pool is idle/expired (pause gaps between HTTP range requests). +""" + +import logging +import time + +from apps.accounts.models import User +from apps.channels.models import Channel +from apps.timeshift.redis_keys import TimeshiftRedisKeys, mint_session_id +from core.utils import RedisClient + +logger = logging.getLogger(__name__) + +HANDSHAKE_TTL_SECONDS = 60 +# Max idle pause between range/seek requests (refreshed on each playback GET). +SESSION_IDLE_TTL_SECONDS = 10 * 60 + + +def mint_catchup_session_id(): + """Backward-compatible alias for :func:`mint_session_id`.""" + return mint_session_id() + + +def create_catchup_session(*, user, channel, start): + """Persist a new playback session and return metadata for the API response.""" + redis_client = RedisClient.get_client() + if redis_client is None: + raise RuntimeError("Redis unavailable") + + session_id = mint_session_id() + now = int(time.time()) + key = TimeshiftRedisKeys.api_session(session_id) + redis_client.hset( + key, + mapping={ + "user_id": str(user.id), + "channel_uuid": str(channel.uuid), + "channel_id": str(channel.id), + "start": str(start), + "created_at": str(now), + }, + ) + redis_client.expire(key, HANDSHAKE_TTL_SECONDS) + + handshake_expires_at = now + HANDSHAKE_TTL_SECONDS + playback_url = f"/proxy/catchup/{channel.uuid}?session_id={session_id}" + + return { + "session_id": session_id, + "playback_url": playback_url, + "expires_at": handshake_expires_at, + "channel_uuid": str(channel.uuid), + "start": str(start), + } + + +def get_catchup_session(session_id): + """Return session fields as a dict, or None if missing.""" + redis_client = RedisClient.get_client() + if redis_client is None or not session_id: + return None + try: + data = redis_client.hgetall(TimeshiftRedisKeys.api_session(session_id)) + except Exception as exc: + logger.warning("Catchup session read failed for %s: %s", session_id, exc) + return None + if not data: + return None + return data + + +def touch_catchup_session(session_id, *, redis_client=None): + """Extend sliding idle TTL after a playback request uses the session.""" + if redis_client is None: + redis_client = RedisClient.get_client() + if redis_client is None or not session_id: + return False + key = TimeshiftRedisKeys.api_session(session_id) + try: + if not redis_client.exists(key): + return False + redis_client.expire(key, SESSION_IDLE_TTL_SECONDS) + return True + except Exception as exc: + logger.warning("Catchup session touch failed for %s: %s", session_id, exc) + return False + + +def delete_catchup_session(session_id, *, redis_client=None): + if not session_id: + return False + if redis_client is None: + redis_client = RedisClient.get_client() + if redis_client is None: + return False + try: + deleted = bool(redis_client.delete(TimeshiftRedisKeys.api_session(session_id))) + if deleted: + logger.debug("Catchup session deleted: %s", session_id) + return deleted + except Exception as exc: + logger.warning("Catchup session delete failed for %s: %s", session_id, exc) + return False + + +def catchup_session_exists(session_id, *, redis_client=None): + """Return True when *session_id* has an API session record.""" + if not session_id: + return False + if redis_client is None: + redis_client = RedisClient.get_client() + if redis_client is None: + return False + try: + return bool(redis_client.exists(TimeshiftRedisKeys.api_session(session_id))) + except Exception: + return False + + +def _user_id_from_pool(session_id): + redis_client = RedisClient.get_client() + if redis_client is None or not session_id: + return None + try: + data = redis_client.hgetall(TimeshiftRedisKeys.pool(session_id)) + except Exception: + return None + if not data: + return None + uid = data.get("user_id") + if not uid: + return None + try: + return int(uid) + except (TypeError, ValueError): + return None + + +def resolve_catchup_playback(session_id, channel_uuid): + """Resolve user and programme start for a tokenless playback request. + + Returns: + ``(user, start)`` on success, or ``None`` if the session is invalid, + expired, or bound to a different channel. + """ + record = get_catchup_session(session_id) + if not record: + return None + + if str(record.get("channel_uuid") or "") != str(channel_uuid): + return None + + touch_catchup_session(session_id) + + user_id = _user_id_from_pool(session_id) + if user_id is None: + try: + user_id = int(record.get("user_id") or "") + except (TypeError, ValueError): + return None + + user = User.objects.filter(id=user_id, is_active=True).first() + if user is None: + return None + + start = record.get("start") + if not start: + return None + + return user, str(start) + + +def user_owns_catchup_session(session_id, user_id): + record = get_catchup_session(session_id) + if not record: + return False + try: + return int(record.get("user_id") or "") == int(user_id) + except (TypeError, ValueError): + return False diff --git a/apps/timeshift/stats.py b/apps/timeshift/stats.py new file mode 100644 index 00000000..e3177add --- /dev/null +++ b/apps/timeshift/stats.py @@ -0,0 +1,480 @@ +"""Build catch-up (timeshift) connection stats for the admin UI.""" + +import logging +import re +import time + +from django.db import close_old_connections + +from datetime import timezone as dt_timezone + +from apps.channels.models import Channel +from apps.m3u.models import M3UAccountProfile +from apps.proxy.live_proxy.constants import ChannelMetadataField +from apps.timeshift.redis_keys import TimeshiftRedisKeys, parse_stats_channel_id +from apps.timeshift.helpers import parse_catchup_timestamp + +logger = logging.getLogger(__name__) +_STREAM_STATS_TO_METADATA = { + "video_codec": ChannelMetadataField.VIDEO_CODEC, + "resolution": ChannelMetadataField.RESOLUTION, + "source_fps": ChannelMetadataField.SOURCE_FPS, + "pixel_format": ChannelMetadataField.PIXEL_FORMAT, + "video_bitrate": ChannelMetadataField.VIDEO_BITRATE, + "audio_codec": ChannelMetadataField.AUDIO_CODEC, + "sample_rate": ChannelMetadataField.SAMPLE_RATE, + "audio_channels": ChannelMetadataField.AUDIO_CHANNELS, + "audio_bitrate": ChannelMetadataField.AUDIO_BITRATE, + "stream_type": ChannelMetadataField.STREAM_TYPE, +} + +# Redis metadata → stats API / UI display keys (matches live stats cards). +_METADATA_TO_DISPLAY = { + ChannelMetadataField.RESOLUTION: "resolution", + ChannelMetadataField.SOURCE_FPS: "source_fps", + ChannelMetadataField.VIDEO_CODEC: "video_codec", + ChannelMetadataField.AUDIO_CODEC: "audio_codec", + ChannelMetadataField.AUDIO_CHANNELS: "audio_channels", + ChannelMetadataField.STREAM_TYPE: "stream_type", +} + + +def stream_stats_to_metadata_fields(stream_stats): + """Map a ``Stream.stream_stats`` dict to Redis channel metadata fields.""" + if not stream_stats: + return {} + out = {} + for src_key, field_name in _STREAM_STATS_TO_METADATA.items(): + value = stream_stats.get(src_key) + if value is not None and value != "": + out[field_name] = str(value) + if out: + out[ChannelMetadataField.STREAM_INFO_UPDATED] = str(time.time()) + return out + + +def stream_info_from_metadata(metadata): + """Extract display-oriented stream stats from Redis channel metadata.""" + if not metadata: + return {} + out = {} + for field_name, display_key in _METADATA_TO_DISPLAY.items(): + value = metadata.get(field_name) + if value is not None and value != "": + out[display_key] = _decode_value(value) + return out + + +def seed_stream_stats_metadata( + redis_client, + metadata_key, + metadata_payload, + *, + stats_stream_id, + stream_stats=None, +): + """Copy stream stats into catch-up Redis metadata when the upstream stream changes. + + ``stream_stats`` should come from the already-loaded ``catchup_stream`` row + when available. A DB lookup is only attempted when the pool reuses a session + whose metadata was evicted and no in-memory stats were passed. + """ + if stats_stream_id is None or redis_client is None: + return + sid = str(stats_stream_id) + try: + existing = redis_client.hget(metadata_key, ChannelMetadataField.STREAM_ID) + if isinstance(existing, bytes): + existing = existing.decode() + except Exception: + existing = None + + if existing == sid: + return + + stats = stream_stats + if not stats: + try: + from apps.channels.models import Stream + + row = Stream.objects.filter(id=int(sid)).values("stream_stats").first() + stats = row.get("stream_stats") if row else None + except Exception as exc: + logger.debug("Timeshift stream stats lookup failed for %s: %s", sid, exc) + stats = None + + fields = stream_stats_to_metadata_fields(stats) + if not fields: + return + metadata_payload.update(fields) + metadata_payload[ChannelMetadataField.STREAM_ID] = sid + + +def _decode_value(value): + if isinstance(value, bytes): + return value.decode() + return value + + +def _decode_hash(data): + if not data: + return {} + return {_decode_value(k): _decode_value(v) for k, v in data.items()} + + +def compute_playback_base_from_byte_range(range_start, content_length, duration_secs): + """Map a byte offset into an approximate programme position in seconds.""" + if range_start is None or range_start <= 0: + return None + try: + range_start = int(range_start) + content_length = int(content_length) + duration_secs = float(duration_secs) + except (TypeError, ValueError): + return None + if content_length <= 0 or duration_secs <= 0: + return None + ratio = min(1.0, max(0.0, range_start / content_length)) + return ratio * duration_secs + + +def resolve_stats_playback_fields( + *, + timestamp_utc, + existing_programme_start, + existing_position_anchor, + existing_playback_base, + range_start, + representation_length, + programme_duration_secs, + now, +): + """Return ``(playback_base_secs, position_anchor_at)`` for stats registration. + + ``playback_base_secs`` is set for byte-range seeks (VLC); ``None`` means + derive position from the URL programme timestamp instead. + """ + if isinstance(existing_programme_start, bytes): + existing_programme_start = existing_programme_start.decode() + if isinstance(existing_position_anchor, bytes): + existing_position_anchor = existing_position_anchor.decode() + if isinstance(existing_playback_base, bytes): + existing_playback_base = existing_playback_base.decode() + + programme_changed = ( + existing_programme_start is not None + and existing_programme_start != timestamp_utc + ) + byte_base = compute_playback_base_from_byte_range( + range_start, representation_length, programme_duration_secs, + ) + + if programme_changed: + return None, now + + if byte_base is not None: + return byte_base, now + + if range_start == 0: + return None, now + + if existing_programme_start == timestamp_utc and existing_position_anchor: + existing_base = None + if existing_playback_base is not None: + try: + existing_base = float(existing_playback_base) + except (TypeError, ValueError): + existing_base = None + return existing_base, existing_position_anchor + + return None, now + + +def compute_playback_position_secs( + programme_start_url, + epg_start_iso, + position_anchor_at, + current_time, + duration_secs=None, + playback_base_secs=None, +): + """Best-effort catch-up play position in seconds within the programme. + + IPTV clients seek by opening a new catch-up URL at the target time, so the + URL timestamp (``programme_start_url``) is the requested position. Offset it + from the programme's EPG start, then advance by wall-clock since the current + stream opened (``position_anchor_at``). + + Native players (VLC) often keep the programme URL fixed and seek via + ``Range: bytes=…``; in that case ``playback_base_secs`` carries the mapped + position at the anchor instead of the URL timestamp offset. + """ + elapsed_since_anchor = 0.0 + if position_anchor_at: + try: + elapsed_since_anchor = max(0.0, current_time - float(position_anchor_at)) + except (TypeError, ValueError): + elapsed_since_anchor = 0.0 + + if playback_base_secs is not None: + try: + position = float(playback_base_secs) + elapsed_since_anchor + except (TypeError, ValueError): + position = elapsed_since_anchor + else: + if not programme_start_url or not epg_start_iso: + return None + url_dt = parse_catchup_timestamp(programme_start_url) + if url_dt is None: + return None + try: + from datetime import datetime + + epg_dt = datetime.fromisoformat(epg_start_iso) + except (TypeError, ValueError): + return None + if epg_dt.tzinfo is not None: + epg_dt = epg_dt.astimezone(dt_timezone.utc).replace(tzinfo=None) + url_offset = (url_dt - epg_dt).total_seconds() + position = url_offset + elapsed_since_anchor + + if position < 0: + position = 0.0 + if duration_secs: + position = min(position, float(duration_secs)) + return position + + +def find_stats_channel_for_session(redis_client, session_id): + """Locate the Redis stats channel id for a catch-up session.""" + if not redis_client or not session_id: + return None + pattern = f"timeshift:channel:*:clients:{session_id}" + cursor = 0 + while True: + cursor, keys = redis_client.scan(cursor, match=pattern, count=100) + for key in keys: + key_str = _decode_value(key) + match = re.match(r"timeshift:channel:(.+):clients:.+", key_str) + if match: + return match.group(1) + if cursor == 0: + break + return None + + +def build_timeshift_stats_data(redis_client): + """Build catch-up stats payload from Redis session metadata.""" + empty = { + "timeshift_sessions": [], + "total_connections": 0, + "timestamp": time.time(), + } + if redis_client is None: + return empty + + try: + current_time = time.time() + connections = [] + cursor = 0 + metadata_pattern = "timeshift:channel:*:metadata" + + while True: + cursor, keys = redis_client.scan( + cursor, match=metadata_pattern, count=100, + ) + for key in keys: + key_str = _decode_value(key) + match = re.search(r"timeshift:channel:(.+):metadata", key_str) + if not match: + continue + stats_channel_id = match.group(1) + parsed_stats = parse_stats_channel_id(stats_channel_id) + if not parsed_stats: + continue + + metadata = _decode_hash(redis_client.hgetall(key)) + + client_set_key = TimeshiftRedisKeys.clients(stats_channel_id) + client_ids = redis_client.smembers(client_set_key) or [] + if not client_ids: + continue + + init_time = float(metadata.get(ChannelMetadataField.INIT_TIME, 0) or 0) + uptime = current_time - init_time if init_time > 0 else 0 + total_bytes = int(metadata.get(ChannelMetadataField.TOTAL_BYTES, 0) or 0) + avg_bitrate_kbps = 0 + if uptime > 0 and total_bytes > 0: + avg_bitrate_kbps = (total_bytes * 8) / uptime / 1000 + + logo_id = metadata.get(ChannelMetadataField.LOGO_ID) + m3u_profile_id = metadata.get(ChannelMetadataField.M3U_PROFILE) + channel_name = metadata.get( + ChannelMetadataField.CHANNEL_NAME, "Catch-up", + ) + channel_id_raw = metadata.get(ChannelMetadataField.CHANNEL_ID) + if not channel_id_raw: + channel_id_raw = str(parsed_stats["channel_id"]) + channel_uuid = metadata.get(ChannelMetadataField.CHANNEL_UUID, "") + stream_info = stream_info_from_metadata(metadata) + + for raw_client_id in client_ids: + client_id = _decode_value(raw_client_id) + client_key = TimeshiftRedisKeys.client_metadata( + stats_channel_id, client_id, + ) + client_data = _decode_hash(redis_client.hgetall(client_key)) + if not client_data: + continue + + programme_start = client_data.get("programme_start") + if not programme_start: + continue + + try: + channel_id = int(channel_id_raw) + except (TypeError, ValueError): + continue + + playback_base_raw = client_data.get("playback_base_secs") + playback_base_secs = None + if playback_base_raw is not None and playback_base_raw != "": + try: + playback_base_secs = float(playback_base_raw) + except (TypeError, ValueError): + playback_base_secs = None + + connected_at = client_data.get("connected_at") + duration = 0 + if connected_at: + try: + duration = int(current_time - float(connected_at)) + except (TypeError, ValueError): + duration = 0 + + connections.append({ + "stats_channel_id": stats_channel_id, + "session_id": parsed_stats["session_id"], + "client_id": client_id, + "channel_id": channel_id, + "channel_uuid": channel_uuid, + "channel_name": channel_name, + "logo_id": int(logo_id) if logo_id else None, + "programme_start": programme_start, + "position_anchor_at": client_data.get("position_anchor_at"), + "playback_base_secs": playback_base_secs, + "m3u_profile_id": int(m3u_profile_id) if m3u_profile_id else None, + "ip_address": client_data.get("ip_address", "Unknown"), + "user_agent": client_data.get("user_agent", "unknown"), + "user_id": client_data.get("user_id", "0"), + "username": client_data.get("username", "unknown"), + "connected_at": float(connected_at) if connected_at else None, + "duration": duration, + "bytes_streamed": total_bytes, + "avg_bitrate_kbps": round(avg_bitrate_kbps, 2), + "uptime": uptime, + **stream_info, + }) + + if cursor == 0: + break + + channel_ids = {conn["channel_id"] for conn in connections if conn.get("channel_id")} + channels_by_id = {} + if channel_ids: + for channel in Channel.objects.select_related("logo", "epg_data").filter( + id__in=channel_ids, + ): + channels_by_id[channel.id] = channel + + profile_ids = { + conn["m3u_profile_id"] + for conn in connections + if conn.get("m3u_profile_id") + } + profiles_by_id = {} + if profile_ids: + for profile in M3UAccountProfile.objects.select_related("m3u_account").filter( + id__in=profile_ids, + ): + profiles_by_id[profile.id] = profile + + session_stats = {} + for conn in connections: + session_key = conn["session_id"] + channel = channels_by_id.get(conn["channel_id"]) + + if channel: + conn["channel_name"] = channel.name + if channel.logo_id: + conn["logo_id"] = channel.logo_id + if channel.logo: + conn["logo_url"] = channel.logo.url + + profile = profiles_by_id.get(conn.get("m3u_profile_id")) + if profile: + conn["m3u_profile"] = { + "profile_name": profile.name, + "account_name": profile.m3u_account.name, + "account_id": profile.m3u_account.id, + "m3u_profile_id": profile.id, + } + + position_anchor_at = conn.get("position_anchor_at") + try: + position_anchor_at = ( + float(position_anchor_at) if position_anchor_at else None + ) + except (TypeError, ValueError): + position_anchor_at = None + + playback_base_secs = conn.get("playback_base_secs") + + if session_key not in session_stats: + session_stats[session_key] = { + "session_id": session_key, + "stats_channel_id": conn["stats_channel_id"], + "channel_id": conn["channel_id"], + "channel_uuid": conn.get("channel_uuid", ""), + "channel_name": conn["channel_name"], + "logo_id": conn.get("logo_id"), + "logo_url": conn.get("logo_url"), + "programme_start": conn["programme_start"], + "position_anchor_at": position_anchor_at, + "playback_base_secs": playback_base_secs, + "resolution": conn.get("resolution"), + "source_fps": conn.get("source_fps"), + "video_codec": conn.get("video_codec"), + "audio_codec": conn.get("audio_codec"), + "audio_channels": conn.get("audio_channels"), + "stream_type": conn.get("stream_type"), + "connection_count": 0, + "connections": [], + } + + session_stats[session_key]["connection_count"] += 1 + session_stats[session_key]["connections"].append({ + "client_id": conn["client_id"], + "session_id": conn["session_id"], + "ip_address": conn["ip_address"], + "user_agent": conn["user_agent"], + "user_id": conn["user_id"], + "username": conn["username"], + "connected_at": conn["connected_at"], + "duration": conn["duration"], + "bytes_streamed": conn["bytes_streamed"], + "avg_bitrate_kbps": conn["avg_bitrate_kbps"], + "m3u_profile": conn.get("m3u_profile", {}), + "m3u_profile_id": conn.get("m3u_profile_id"), + }) + + return { + "timeshift_sessions": list(session_stats.values()), + "total_connections": len(connections), + "timestamp": current_time, + } + except Exception as exc: + logger.error("Error building timeshift stats: %s", exc, exc_info=True) + return empty + finally: + close_old_connections() diff --git a/apps/timeshift/stats_views.py b/apps/timeshift/stats_views.py new file mode 100644 index 00000000..a748f444 --- /dev/null +++ b/apps/timeshift/stats_views.py @@ -0,0 +1,78 @@ +"""Admin API views for catch-up connection statistics.""" + +import json +import logging + +from django.http import JsonResponse +from django.views.decorators.csrf import csrf_exempt +from rest_framework.decorators import api_view, permission_classes + +from apps.accounts.permissions import IsAdmin +from apps.proxy.utils import stop_timeshift_client +from apps.timeshift.helpers import get_catchup_programmes_for_sessions +from apps.timeshift.stats import ( + build_timeshift_stats_data, + find_stats_channel_for_session, +) +from core.utils import RedisClient + +logger = logging.getLogger(__name__) + + +@api_view(["GET"]) +@permission_classes([IsAdmin]) +def timeshift_stats(request): + """Return active catch-up viewer sessions for the stats page.""" + redis_client = RedisClient.get_client() + if not redis_client: + return JsonResponse({"error": "Redis not available"}, status=500) + return JsonResponse(build_timeshift_stats_data(redis_client)) + + +@api_view(["POST"]) +@permission_classes([IsAdmin]) +def catchup_programmes(request): + """Return EPG metadata for active catch-up sessions (batch).""" + sessions = request.data.get("sessions") + if sessions is None: + return JsonResponse({"error": "sessions is required"}, status=400) + if not isinstance(sessions, list): + return JsonResponse({"error": "sessions must be an array"}, status=400) + return JsonResponse({ + "sessions": get_catchup_programmes_for_sessions(sessions), + }) + + +@csrf_exempt +@api_view(["POST"]) +@permission_classes([IsAdmin]) +def stop_timeshift_session(request): + """Stop a catch-up viewer by session id (one session = one stats channel).""" + try: + data = json.loads(request.body) + except json.JSONDecodeError: + return JsonResponse({"error": "Invalid JSON"}, status=400) + + session_id = data.get("session_id") + if not session_id: + return JsonResponse({"error": "No session_id provided"}, status=400) + + redis_client = RedisClient.get_client() + if not redis_client: + return JsonResponse({"error": "Redis not available"}, status=500) + + stats_channel_id = find_stats_channel_for_session(redis_client, session_id) + if not stats_channel_id: + return JsonResponse({"error": "Connection not found"}, status=404) + + result = stop_timeshift_client(redis_client, stats_channel_id, session_id) + if result.get("status") != "success": + return JsonResponse( + {"error": result.get("message", "Stop failed")}, + status=500, + ) + + return JsonResponse({ + "message": "Timeshift session stop signal sent", + "session_id": session_id, + }) diff --git a/apps/timeshift/tests/test_helpers.py b/apps/timeshift/tests/test_helpers.py index 4bd7e0e3..068bf1d0 100644 --- a/apps/timeshift/tests/test_helpers.py +++ b/apps/timeshift/tests/test_helpers.py @@ -62,6 +62,18 @@ class TimestampFormatTests(TestCase): "2026-06-23T04:00:00", ) + def test_normalize_iso8601_utc(self): + self.assertEqual( + normalize_catchup_timestamp_input("2026-06-23T04:00:00Z"), + "2026-06-23T04:00:00", + ) + + def test_normalize_iso8601_with_offset(self): + self.assertEqual( + normalize_catchup_timestamp_input("2026-06-23T06:00:00+02:00"), + "2026-06-23T04:00:00", + ) + def test_normalize_rejects_garbage(self): self.assertIsNone(normalize_catchup_timestamp_input("garbage")) self.assertIsNone(normalize_catchup_timestamp_input("")) diff --git a/apps/timeshift/tests/test_sessions.py b/apps/timeshift/tests/test_sessions.py new file mode 100644 index 00000000..0a6e8060 --- /dev/null +++ b/apps/timeshift/tests/test_sessions.py @@ -0,0 +1,318 @@ +"""Tests for catch-up playback session API and Redis helpers.""" + +import time +import uuid +from unittest.mock import MagicMock, patch + +from django.http import HttpResponse +from django.test import TestCase, override_settings +from rest_framework.test import APIClient + +from apps.accounts.models import User +from apps.channels.models import Channel, ChannelStream, Stream +from apps.m3u.models import M3UAccount +from apps.timeshift import sessions, views +from apps.timeshift.redis_keys import TimeshiftRedisKeys +from apps.timeshift.tests.test_views import _proxy_url +from rest_framework.test import APIRequestFactory, force_authenticate + + +class FakeRedisSessionStore: + """Minimal Redis stand-in for session module tests.""" + + def __init__(self): + self.store = {} + self.ttl = {} + + def hset(self, key, mapping=None, **kwargs): + if mapping is None: + mapping = kwargs + else: + mapping = {**mapping, **kwargs} + bucket = self.store.setdefault(key, {}) + bucket.update({k: str(v) for k, v in mapping.items()}) + + def hgetall(self, key): + return dict(self.store.get(key, {})) + + def expire(self, key, seconds): + self.ttl[key] = seconds + + def exists(self, key): + return key in self.store and bool(self.store[key]) + + def delete(self, key): + self.store.pop(key, None) + self.ttl.pop(key, None) + return 1 + + +@override_settings( + CACHES={ + "default": { + "BACKEND": "django.core.cache.backends.locmem.LocMemCache", + "LOCATION": "catchup-session-tests", + } + }, + REST_FRAMEWORK={ + "DEFAULT_AUTHENTICATION_CLASSES": [ + "rest_framework_simplejwt.authentication.JWTAuthentication", + "apps.accounts.authentication.ApiKeyAuthentication", + ], + "DEFAULT_PERMISSION_CLASSES": [ + "apps.accounts.permissions.IsAdmin", + ], + }, +) +class CatchupSessionApiTests(TestCase): + @classmethod + def setUpTestData(cls): + cls.user = User.objects.create( + username="catchup-session-user", + user_level=User.UserLevel.STANDARD, + ) + cls.other = User.objects.create( + username="catchup-session-other", + user_level=User.UserLevel.STANDARD, + ) + cls.account = M3UAccount.objects.create( + name="catchup-session-acct", + server_url="http://example.test", + account_type="XC", + is_active=True, + ) + cls.channel = Channel.objects.create( + name="Catchup Session Channel", + is_catchup=True, + catchup_days=7, + ) + cls.stream = Stream.objects.create( + name="catchup-session-stream", + url="http://example.test/live", + m3u_account=cls.account, + is_catchup=True, + catchup_days=7, + custom_properties={"stream_id": "111"}, + ) + ChannelStream.objects.create( + channel=cls.channel, stream=cls.stream, order=0, + ) + + def setUp(self): + self.client = APIClient() + self.client.force_authenticate(user=self.user) + self.redis = FakeRedisSessionStore() + + def _create_url(self): + return "/api/catchup/sessions/" + + @patch.object(sessions.RedisClient, "get_client") + @patch("apps.timeshift.api_views.network_access_allowed", return_value=True) + def test_post_creates_session_without_start_in_playback_url(self, _net, redis_mock): + redis_mock.return_value = self.redis + response = self.client.post( + self._create_url(), + { + "channel_uuid": str(self.channel.uuid), + "start": "2026-06-08T17:00:00Z", + }, + format="json", + ) + self.assertEqual(response.status_code, 201) + data = response.json() + self.assertGreater(len(data["session_id"]), 8) + self.assertIn(f"session_id={data['session_id']}", data["playback_url"]) + self.assertNotIn("start=", data["playback_url"]) + self.assertEqual(data["channel_uuid"], str(self.channel.uuid)) + self.assertEqual(data["start"], "2026-06-08T17:00:00Z") + self.assertGreater(data["expires_at"], int(time.time())) + + @patch.object(sessions.RedisClient, "get_client") + @patch("apps.timeshift.api_views.network_access_allowed", return_value=True) + def test_post_rejects_non_catchup_channel(self, _net, redis_mock): + redis_mock.return_value = self.redis + plain = Channel.objects.create(name="no-catchup") + response = self.client.post( + self._create_url(), + {"channel_uuid": str(plain.uuid), "start": "2026-06-08T17:00:00Z"}, + format="json", + ) + self.assertEqual(response.status_code, 400) + + @patch.object(sessions.RedisClient, "get_client") + @patch("apps.timeshift.api_views.network_access_allowed", return_value=True) + def test_delete_revokes_own_session(self, _net, redis_mock): + redis_mock.return_value = self.redis + created = self.client.post( + self._create_url(), + { + "channel_uuid": str(self.channel.uuid), + "start": "2026-06-08T17:00:00Z", + }, + format="json", + ) + session_id = created.json()["session_id"] + deleted = self.client.delete(f"/api/catchup/sessions/{session_id}/") + self.assertEqual(deleted.status_code, 204) + self.assertFalse(sessions.get_catchup_session(session_id)) + + @patch.object(sessions.RedisClient, "get_client") + @patch("apps.timeshift.api_views.network_access_allowed", return_value=True) + def test_delete_rejects_other_users_session(self, _net, redis_mock): + redis_mock.return_value = self.redis + created = self.client.post( + self._create_url(), + { + "channel_uuid": str(self.channel.uuid), + "start": "2026-06-08T17:00:00Z", + }, + format="json", + ) + session_id = created.json()["session_id"] + self.client.force_authenticate(user=self.other) + deleted = self.client.delete(f"/api/catchup/sessions/{session_id}/") + self.assertEqual(deleted.status_code, 404) + + +class CatchupSessionResolveTests(TestCase): + @classmethod + def setUpTestData(cls): + cls.user = User.objects.create( + username="catchup-resolve-user", + user_level=User.UserLevel.STANDARD, + ) + cls.channel = Channel.objects.create( + name="resolve-channel", + is_catchup=True, + ) + + def setUp(self): + self.redis = FakeRedisSessionStore() + + @patch.object(sessions.RedisClient, "get_client") + def test_resolve_prefers_pool_user_id(self, redis_mock): + redis_mock.return_value = self.redis + session_id = sessions.mint_catchup_session_id() + self.redis.hset( + TimeshiftRedisKeys.api_session(session_id), + mapping={ + "user_id": "999", + "channel_uuid": str(self.channel.uuid), + "channel_id": str(self.channel.id), + "start": "2026-06-08T17:00:00Z", + "created_at": "1", + }, + ) + self.redis.hset( + TimeshiftRedisKeys.pool(session_id), + mapping={"user_id": str(self.user.id)}, + ) + resolved = sessions.resolve_catchup_playback(session_id, self.channel.uuid) + self.assertIsNotNone(resolved) + self.assertEqual(resolved[0].id, self.user.id) + self.assertEqual(resolved[1], "2026-06-08T17:00:00Z") + self.assertEqual( + self.redis.ttl[TimeshiftRedisKeys.api_session(session_id)], + sessions.SESSION_IDLE_TTL_SECONDS, + ) + + @patch.object(sessions.RedisClient, "get_client") + def test_resolve_rejects_wrong_channel(self, redis_mock): + redis_mock.return_value = self.redis + session_id = sessions.mint_catchup_session_id() + self.redis.hset( + TimeshiftRedisKeys.api_session(session_id), + mapping={ + "user_id": str(self.user.id), + "channel_uuid": str(self.channel.uuid), + "channel_id": str(self.channel.id), + "start": "2026-06-08T17:00:00Z", + "created_at": "1", + }, + ) + other_uuid = uuid.uuid4() + self.assertIsNone( + sessions.resolve_catchup_playback(session_id, other_uuid), + ) + + +class CatchupProxySessionAuthTests(TestCase): + """Playback via API session without JWT.""" + + def setUp(self): + self.factory = APIRequestFactory() + self.channel_uuid = uuid.uuid4() + + @patch.object(views, "resolve_catchup_playback") + @patch.object(views, "network_access_allowed", return_value=True) + @patch.object(views, "_serve_catchup", return_value=HttpResponse("ok")) + @patch.object(views, "_user_can_access_channel", return_value=True) + @patch.object(views, "Channel") + def test_session_auth_without_jwt( + self, channel_cls, _access, serve, _net, resolve_mock, + ): + user = MagicMock(id=42, is_authenticated=False) + resolve_mock.return_value = (user, "2026-06-08T17:00:00Z") + channel_cls.objects.get.return_value = MagicMock( + id=8, uuid=self.channel_uuid, + ) + request = self.factory.get( + f"/proxy/catchup/{self.channel_uuid}?session_id=test", + ) + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 200) + serve.assert_called_once() + _args, kwargs = serve.call_args + self.assertEqual(_args[3], "2026-06-08T17:00:00Z") + + @patch.object(views, "resolve_catchup_playback", return_value=None) + @patch.object(views, "network_access_allowed", return_value=True) + def test_expired_session_without_jwt_returns_401(self, _net, _resolve): + request = self.factory.get( + f"/proxy/catchup/{self.channel_uuid}?session_id=gone", + ) + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 401) + + @patch.object(views, "resolve_catchup_playback") + @patch.object(views, "network_access_allowed", return_value=True) + def test_mismatched_jwt_and_session_returns_403(self, _net, resolve_mock): + resolve_mock.return_value = (MagicMock(id=1), "2026-06-08T17:00:00Z") + request = self.factory.get( + f"/proxy/catchup/{self.channel_uuid}?session_id=test", + ) + other = MagicMock(id=2, is_authenticated=True) + force_authenticate(request, user=other) + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 403) + + @patch.object(views, "network_access_allowed", return_value=True) + @patch.object(views, "_serve_catchup", return_value=HttpResponse("ok")) + @patch.object(views, "_user_can_access_channel", return_value=True) + @patch.object(views, "Channel") + def test_legacy_jwt_start_still_works(self, channel_cls, _access, serve, _net): + user = MagicMock(id=1, is_authenticated=True) + channel_cls.objects.get.return_value = MagicMock( + id=8, uuid=self.channel_uuid, + ) + request = self.factory.get( + f"/proxy/catchup/{self.channel_uuid}?start=2026-06-08T17:00:00Z", + ) + force_authenticate(request, user=user) + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 200) + serve.assert_called_once() + + def test_xc_path_unchanged(self): + request = self.factory.get(_proxy_url()) + with patch.object(views, "_authenticate_user", return_value=MagicMock(id=1)), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "_serve_catchup", return_value=HttpResponse("ok")) as serve: + channel_cls.objects.get.return_value = MagicMock(id=8) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-00", "8.ts", + ) + self.assertEqual(response.status_code, 200) + serve.assert_called_once() diff --git a/apps/timeshift/tests/test_stats.py b/apps/timeshift/tests/test_stats.py new file mode 100644 index 00000000..5a5f9e0b --- /dev/null +++ b/apps/timeshift/tests/test_stats.py @@ -0,0 +1,347 @@ +"""Tests for catch-up stats API and builders.""" + +import json +import time +from datetime import datetime, timedelta, timezone as dt_timezone +from unittest.mock import MagicMock, patch + +from django.test import TestCase +from rest_framework.test import APIRequestFactory, force_authenticate + +from apps.accounts.models import User +from apps.proxy.live_proxy.constants import ChannelMetadataField, ChannelState +from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, parse_stats_channel_id +from apps.timeshift import stats_views +from apps.timeshift.helpers import get_programme_info +from apps.timeshift.stats import ( + build_timeshift_stats_data, + compute_playback_base_from_byte_range, + compute_playback_position_secs, + find_stats_channel_for_session, + resolve_stats_playback_fields, + seed_stream_stats_metadata, + stream_stats_to_metadata_fields, +) +from apps.timeshift.tests.test_views import _FakeRedis + +TEST_SESSION_ID = "testsession1" +STATS_CHANNEL_ID = f"8_{TEST_SESSION_ID}" + + +class TimeshiftHelperParsingTests(TestCase): + def test_parse_stats_channel_id(self): + parsed = parse_stats_channel_id(STATS_CHANNEL_ID) + self.assertEqual(parsed["channel_id"], 8) + self.assertEqual(parsed["session_id"], TEST_SESSION_ID) + + +class GetProgrammeInfoTests(TestCase): + def _channel_with_programme(self, title="Evening News", sub_title="Local Edition", minutes=45): + start = datetime(2026, 6, 8, 17, 0, tzinfo=dt_timezone.utc) + programme = MagicMock( + title=title, + sub_title=sub_title, + description="Details", + start_time=start, + end_time=start + timedelta(minutes=minutes), + ) + channel = MagicMock() + channel.epg_data.programs.filter.return_value.first.return_value = programme + return channel + + def test_get_programme_info_resolves_title(self): + info = get_programme_info(self._channel_with_programme(), "2026-06-08:17-00") + self.assertEqual(info["title"], "Evening News") + self.assertEqual(info["sub_title"], "Local Edition") + self.assertEqual(info["duration_secs"], 45 * 60) + + +class ComputePlaybackPositionTests(TestCase): + EPG_START = "2026-07-10T14:00:00+00:00" + + def test_byte_range_base_plus_elapsed(self): + pos = compute_playback_position_secs( + "2026-07-10:14-00", + self.EPG_START, + position_anchor_at=1000.0, + current_time=1030.0, + duration_secs=3600, + playback_base_secs=1800.0, + ) + self.assertAlmostEqual(pos, 1830.0) + + def test_url_offset_plus_elapsed(self): + # Seeked to 14:19 (19 min into a 14:00 programme), stream opened 30s ago. + pos = compute_playback_position_secs( + "2026-07-10:14-19", + self.EPG_START, + position_anchor_at=1000.0, + current_time=1030.0, + duration_secs=3600, + ) + self.assertAlmostEqual(pos, 19 * 60 + 30) + + def test_capped_at_duration(self): + pos = compute_playback_position_secs( + "2026-07-10:14-19", + self.EPG_START, + position_anchor_at=1000.0, + current_time=100000.0, + duration_secs=1800, + ) + self.assertEqual(pos, 1800) + + def test_missing_epg_returns_none(self): + self.assertIsNone( + compute_playback_position_secs( + "2026-07-10:14-19", None, 1000.0, 1030.0, + ) + ) + + def test_no_anchor_uses_url_offset_only(self): + pos = compute_playback_position_secs( + "2026-07-10:14-05", + self.EPG_START, + position_anchor_at=None, + current_time=1030.0, + duration_secs=3600, + ) + self.assertAlmostEqual(pos, 5 * 60) + + +class ByteRangePlaybackTests(TestCase): + def test_compute_playback_base_from_byte_range(self): + base = compute_playback_base_from_byte_range( + 506786520, 833563944, 3900, + ) + self.assertAlmostEqual(base, 2371.0, delta=2.0) + + def test_resolve_byte_seek_reanchors(self): + base, anchor = resolve_stats_playback_fields( + timestamp_utc="2026-07-09:20:00:00", + existing_programme_start="2026-07-09:20:00:00", + existing_position_anchor="1000.0", + existing_playback_base=None, + range_start=506786520, + representation_length=833563944, + programme_duration_secs=3900, + now="2000.0", + ) + self.assertAlmostEqual(base, 2371.0, delta=2.0) + self.assertEqual(anchor, "2000.0") + + def test_resolve_same_programme_preserves_without_range(self): + base, anchor = resolve_stats_playback_fields( + timestamp_utc="2026-06-08:17-00", + existing_programme_start="2026-06-08:17-00", + existing_position_anchor="1000.0", + existing_playback_base="900.0", + range_start=None, + representation_length=833563944, + programme_duration_secs=3900, + now="2000.0", + ) + self.assertAlmostEqual(base, 900.0) + self.assertEqual(anchor, "1000.0") + + +class TimeshiftStreamStatsTests(TestCase): + def test_stream_stats_to_metadata_fields(self): + fields = stream_stats_to_metadata_fields({ + "resolution": "1920x1080", + "source_fps": 30, + "video_codec": "h264", + }) + self.assertEqual(fields[ChannelMetadataField.RESOLUTION], "1920x1080") + self.assertEqual(fields[ChannelMetadataField.SOURCE_FPS], "30") + self.assertEqual(fields[ChannelMetadataField.VIDEO_CODEC], "h264") + self.assertIn(ChannelMetadataField.STREAM_INFO_UPDATED, fields) + + def test_seed_stream_stats_metadata_skips_when_stream_unchanged(self): + redis = _FakeRedis() + metadata_key = RedisKeys.channel_metadata(STATS_CHANNEL_ID) + redis.hset(metadata_key, ChannelMetadataField.STREAM_ID, "7") + payload = {} + seed_stream_stats_metadata( + redis, metadata_key, payload, + stats_stream_id=7, + stream_stats={"resolution": "1920x1080"}, + ) + self.assertEqual(payload, {}) + + def test_seed_stream_stats_metadata_applies_new_stream(self): + redis = _FakeRedis() + metadata_key = RedisKeys.channel_metadata(STATS_CHANNEL_ID) + payload = {} + seed_stream_stats_metadata( + redis, metadata_key, payload, + stats_stream_id=7, + stream_stats={"resolution": "1280x720", "video_codec": "h264"}, + ) + self.assertEqual(payload[ChannelMetadataField.STREAM_ID], "7") + self.assertEqual(payload[ChannelMetadataField.RESOLUTION], "1280x720") + + +class BuildTimeshiftStatsDataTests(TestCase): + def setUp(self): + self.channel_id = 42 + self.session_id = TEST_SESSION_ID + self.stats_channel_id = f"{self.channel_id}_{self.session_id}" + self.redis = _FakeRedis() + now = time.time() + metadata_key = RedisKeys.channel_metadata(self.stats_channel_id) + client_set_key = RedisKeys.clients(self.stats_channel_id) + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.session_id) + + self.redis.hset( + metadata_key, + mapping={ + ChannelMetadataField.STATE: ChannelState.ACTIVE, + ChannelMetadataField.CHANNEL_ID: str(self.channel_id), + ChannelMetadataField.CHANNEL_UUID: "00000000-0000-0000-0000-000000000042", + ChannelMetadataField.CHANNEL_NAME: "Catch-up Stats Channel", + ChannelMetadataField.INIT_TIME: str(now - 120), + ChannelMetadataField.TOTAL_BYTES: "1048576", + ChannelMetadataField.LOGO_ID: "0", + ChannelMetadataField.RESOLUTION: "1920x1080", + ChannelMetadataField.SOURCE_FPS: "29.97", + ChannelMetadataField.VIDEO_CODEC: "h264", + }, + ) + self.redis.sadd(client_set_key, self.session_id) + self.anchor_at = now - 30 + self.redis.hset( + client_key, + mapping={ + "user_agent": "VLC/3.0.0", + "ip_address": "10.0.0.5", + "connected_at": str(now - 60), + "user_id": "1", + "username": "viewer", + "programme_vid": f"8_{self.channel_id}_2026-06-08-17-00_111", + "programme_start": "2026-06-08:17-15", + "position_anchor_at": str(self.anchor_at), + }, + ) + + @patch("apps.timeshift.stats.Channel") + def test_build_timeshift_stats_data_redis_only_programme_fields( + self, mock_channel_model, + ): + channel = MagicMock() + channel.id = self.channel_id + channel.name = "Catch-up Stats Channel" + channel.uuid = "00000000-0000-0000-0000-000000000042" + channel.logo_id = None + channel.logo = None + mock_channel_model.objects.select_related.return_value.filter.return_value = [ + channel, + ] + payload = build_timeshift_stats_data(self.redis) + self.assertEqual(payload["total_connections"], 1) + self.assertEqual(len(payload["timeshift_sessions"]), 1) + session = payload["timeshift_sessions"][0] + self.assertEqual(session["session_id"], self.session_id) + self.assertEqual(session["programme_start"], "2026-06-08:17-15") + self.assertAlmostEqual(session["position_anchor_at"], self.anchor_at, places=3) + self.assertNotIn("programme_title", session) + self.assertNotIn("playback_position_secs", session) + self.assertEqual(session["channel_name"], "Catch-up Stats Channel") + self.assertEqual(session["resolution"], "1920x1080") + self.assertEqual(session["connections"][0]["ip_address"], "10.0.0.5") + + def test_find_stats_channel_for_session(self): + found = find_stats_channel_for_session(self.redis, self.session_id) + self.assertEqual(found, self.stats_channel_id) + + @patch("apps.timeshift.stats.Channel") + def test_skips_clients_missing_required_metadata(self, mock_channel_model): + mock_channel_model.objects.select_related.return_value.filter.return_value = [] + orphan_key = RedisKeys.client_metadata(self.stats_channel_id, "orphan") + self.redis.hset(orphan_key, mapping={"user_agent": "VLC"}) + self.redis.sadd(RedisKeys.clients(self.stats_channel_id), "orphan") + payload = build_timeshift_stats_data(self.redis) + self.assertEqual(payload["total_connections"], 1) + + +class TimeshiftStatsApiTests(TestCase): + @classmethod + def setUpTestData(cls): + cls.admin = User.objects.create( + username="catchup-stats-admin", + user_level=User.UserLevel.ADMIN, + ) + + def setUp(self): + self.factory = APIRequestFactory() + self.redis = _FakeRedis() + self.session_id = TEST_SESSION_ID + self.stats_channel_id = f"8_{self.session_id}" + + @patch("apps.timeshift.stats_views.RedisClient.get_client") + @patch("apps.timeshift.stats_views.build_timeshift_stats_data") + def test_timeshift_stats_requires_admin(self, mock_build, redis_mock): + mock_build.return_value = { + "timeshift_sessions": [], + "total_connections": 0, + "timestamp": 1, + } + redis_mock.return_value = self.redis + + request = self.factory.get("/proxy/catchup/stats/") + force_authenticate(request, user=self.admin) + response = stats_views.timeshift_stats(request) + self.assertEqual(response.status_code, 200) + self.assertIn("timeshift_sessions", json.loads(response.content)) + + @patch("apps.timeshift.stats_views.stop_timeshift_client") + @patch("apps.timeshift.stats_views.RedisClient.get_client") + def test_stop_timeshift_session(self, redis_mock, stop_mock): + redis_mock.return_value = self.redis + stop_mock.return_value = {"status": "success"} + + metadata_key = RedisKeys.channel_metadata(self.stats_channel_id) + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.session_id) + self.redis.hset(metadata_key, mapping={ChannelMetadataField.STATE: ChannelState.ACTIVE}) + self.redis.sadd(RedisKeys.clients(self.stats_channel_id), self.session_id) + self.redis.hset( + client_key, + mapping={ + "programme_start": "2026-06-08:17-00", + }, + ) + + request = self.factory.post( + "/proxy/catchup/stop_client/", + {"session_id": self.session_id}, + format="json", + ) + force_authenticate(request, user=self.admin) + response = stats_views.stop_timeshift_session(request) + self.assertEqual(response.status_code, 200) + stop_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.session_id, + ) + + @patch("apps.timeshift.stats_views.get_catchup_programmes_for_sessions") + def test_catchup_programmes_batch(self, programmes_mock): + programmes_mock.return_value = [{ + "session_id": self.session_id, + "title": "Evening News", + "start_time": "2026-06-08T17:00:00+00:00", + }] + request = self.factory.post( + "/proxy/catchup/programs/", + {"sessions": [{ + "session_id": self.session_id, + "channel_uuid": "00000000-0000-0000-0000-000000000099", + "programme_start": "2026-06-08:17-00", + }]}, + format="json", + ) + force_authenticate(request, user=self.admin) + response = stats_views.catchup_programmes(request) + self.assertEqual(response.status_code, 200) + body = json.loads(response.content) + self.assertEqual(body["sessions"][0]["title"], "Evening News") + programmes_mock.assert_called_once() diff --git a/apps/timeshift/tests/test_views.py b/apps/timeshift/tests/test_views.py index 5b91d52e..95f0aa92 100644 --- a/apps/timeshift/tests/test_views.py +++ b/apps/timeshift/tests/test_views.py @@ -4,14 +4,18 @@ import fnmatch import time from unittest.mock import MagicMock, patch +import requests +from django.http import HttpResponse from django.test import RequestFactory, TestCase, override_settings +from rest_framework.test import APIRequestFactory, force_authenticate from apps.timeshift import views +from apps.timeshift.redis_keys import TimeshiftRedisKeys from apps.proxy.utils import check_user_stream_limits as _check_user_stream_limits from apps.proxy.utils import find_ts_sync as _find_ts_sync -TEST_SESSION_ID = "timeshift_testsession1" -TEST_MEDIA_ID = "timeshift_8_2026-06-08-17-00" +TEST_SESSION_ID = "testsession1" +TEST_MEDIA_ID = "8_2026-06-08-17-00" def _proxy_url(session_id=TEST_SESSION_ID): @@ -41,13 +45,14 @@ def _seed_pool_session( account_id=1, profile_id=31, stream_id="111", + dispatcharr_stream_id=1, provider_timestamp="2026-06-08:19-00", provider_tz_name=provider_tz_name, ) if serving_range is not None: - redis.hset(f"timeshift_pool:{session_id}", "serving_range", serving_range) + redis.hset(TimeshiftRedisKeys.pool(session_id), "serving_range", serving_range) if busy is not None: - redis.hset(f"timeshift_pool:{session_id}", "busy", busy) + redis.hset(TimeshiftRedisKeys.pool(session_id), "busy", busy) class FindTsSyncTests(TestCase): @@ -111,15 +116,18 @@ class StreamFromProviderStatusMappingTests(TestCase): "http://example.test/timeshift/u/p/60/2026-05-12:17-00/1.ts", ], user_agent="test-agent", + client_user_agent="test-client-agent", range_header=None, - virtual_channel_id="timeshift_1_2026-05-12-17-00_1", - client_id="timeshift_test123", + virtual_channel_id="1_2026-05-12-17-00_1", + client_id="test123", client_ip="127.0.0.1", user=None, channel_display_name="Test", timestamp_utc="2026-05-12:17-00", channel_logo_id=None, m3u_profile_id=None, + channel_id=1, + channel_uuid="00000000-0000-0000-0000-000000000001", debug=False, ) @@ -255,7 +263,7 @@ class StreamFromProviderStatusMappingTests(TestCase): # locmem cache: isolates this test from the shared Redis-backed django # cache (which persists across runs and parallel test sessions). from django.core.cache import cache as django_cache - django_cache.delete(views._FORMAT_CACHE_KEY.format(999)) + django_cache.delete(TimeshiftRedisKeys.format_cache(999)) # First request: candidate index 1 wins after index 0 returns 404. mocked_open.side_effect = [ @@ -339,6 +347,36 @@ class StreamFromProviderStatusMappingTests(TestCase): self.assertEqual(response.status_code, 206) self.assertEqual(mocked_open.call_count, 1) + @patch.object(views, "_iter_upstream_with_stop") + @patch.object(views, "_open_upstream") + def test_partial_206_with_sync_in_peek_not_trimmed( + self, mocked_open, mock_iter, + ): + # Near-EOF range probes often land on a TS sync byte mid-buffer. The + # partial-response path must win over sync trimming or Content-Length + # will not match bytes actually streamed. + preamble = b"\x00" * 80 + peek = preamble + _make_ts_payload(1024 - len(preamble)) + self.assertGreater(_find_ts_sync(peek), 0) + resp = _fake_upstream(206, body=peek) + resp.raw = MagicMock() + resp.raw.read = MagicMock(return_value=peek) + resp.headers["Content-Length"] = str(len(peek) + 500) + resp.headers["Content-Range"] = "bytes 1000-1599/2000" + mocked_open.return_value = resp + mock_iter.return_value = iter([]) + kwargs = dict(self.kwargs, range_header="bytes=1000-") + with patch.object(views, "RedisClient"), \ + patch.object(views, "_register_stats_client"), \ + patch.object(views, "_unregister_stats_client"): + response = views._stream_from_provider(**kwargs) + list(response.streaming_content) + self.assertEqual(response.status_code, 206) + mock_iter.assert_called_once() + self.assertEqual(mock_iter.call_args.kwargs.get("peek_data"), peek) + self.assertEqual(response["Content-Length"], str(len(peek) + 500)) + self.assertEqual(response["Content-Range"], "bytes 1000-1599/2000") + @patch.object(views, "_open_upstream") def test_partial_206_html_error_still_rejected(self, mocked_open): # The mid-packet allowance is gated on content type: a 206 whose body @@ -416,6 +454,7 @@ def _make_catchup_stream(provider_tz="Europe/Brussels", *, account_id=9, m3u_account.profiles.filter.return_value = [profile, *extra_profiles] stream = MagicMock() stream.m3u_account = m3u_account + stream.m3u_account_id = account_id stream.custom_properties = {"stream_id": stream_id} if stream_id else {} return stream @@ -436,9 +475,11 @@ class _FakeRedis: def __init__(self): self.store = {} + self.ttl = {} def setex(self, key, ttl, value): self.store[key] = str(value) + self.ttl[key] = ttl def set(self, key, value): self.store[key] = str(value) @@ -447,7 +488,22 @@ class _FakeRedis: return self.store.get(key) def delete(self, *keys): - return sum(1 for k in keys if self.store.pop(k, None) is not None) + removed = 0 + for key in keys: + if self.store.pop(key, None) is not None: + self.ttl.pop(key, None) + removed += 1 + return removed + + def eval(self, script, numkeys, *keys_and_args): + if numkeys != 1 or len(keys_and_args) != 2: + raise NotImplementedError("FakeRedis eval only supports claim script") + key, token = keys_and_args + current = self.store.get(key) + if current is not None and str(current) == str(token): + self.store.pop(key, None) + return 1 + return 0 def exists(self, key): return 1 if key in self.store else 0 @@ -482,6 +538,16 @@ class _FakeRedis: hash_value[field] = str(new_value) return new_value + def incr(self, key): + current = self.store.get(key, 0) + try: + current = int(current) + except (TypeError, ValueError): + current = 0 + new_value = current + 1 + self.store[key] = str(new_value) + return new_value + def hget(self, key, field): hash_value = self.store.get(key) return hash_value.get(field) if isinstance(hash_value, dict) else None @@ -493,7 +559,10 @@ class _FakeRedis: return sum(1 for f in fields if hash_value.pop(f, None) is not None) def expire(self, key, ttl): - return 1 if key in self.store else 0 + if key in self.store: + self.ttl[key] = ttl + return 1 + return 0 # --- set surface for the idle-session pool --- def sadd(self, key, *members): @@ -560,13 +629,49 @@ class _FakeRedisPipeline: def delete(self, key): self._ops.append(("delete", key)) + def hset(self, key, field=None, value=None, mapping=None, **kwargs): + self._ops.append(("hset", key, field, value, mapping, kwargs)) + + def sadd(self, key, member): + self._ops.append(("sadd", key, member)) + + def expire(self, key, ttl): + self._ops.append(("expire", key, ttl)) + + def setex(self, key, ttl, value): + self._ops.append(("setex", key, ttl, value)) + + def hdel(self, key, *fields): + self._ops.append(("hdel", key, fields)) + + def hincrby(self, key, field, amount=1): + self._ops.append(("hincrby", key, field, amount)) + def execute(self): results = [] - for op, key in self._ops: - if op == "get": - results.append(self._redis.get(key)) - else: - results.append(self._redis.delete(key)) + for op in self._ops: + if op[0] == "get": + results.append(self._redis.get(op[1])) + elif op[0] == "delete": + results.append(self._redis.delete(op[1])) + elif op[0] == "hset": + _, key, field, value, mapping, kwargs = op + results.append(self._redis.hset(key, field, value, mapping=mapping, **kwargs)) + elif op[0] == "sadd": + _, key, member = op + results.append(self._redis.sadd(key, member)) + elif op[0] == "expire": + _, key, ttl = op + results.append(self._redis.expire(key, ttl)) + elif op[0] == "setex": + _, key, ttl, value = op + results.append(self._redis.setex(key, ttl, value)) + elif op[0] == "hdel": + _, key, fields = op + results.append(self._redis.hdel(key, *fields)) + elif op[0] == "hincrby": + _, key, field, amount = op + results.append(self._redis.hincrby(key, field, amount)) self._ops = [] return results @@ -935,15 +1040,18 @@ class StreamFromProviderDecisiveEdgeTests(TestCase): "http://example.test/streaming/timeshift.php?stream=1&start=2026-05-12_17-00", ], user_agent="test-agent", + client_user_agent="test-client-agent", range_header=None, - virtual_channel_id="timeshift_1_2026-05-12-17-00_1", - client_id="timeshift_test456", + virtual_channel_id="1_2026-05-12-17-00_1", + client_id="test456", client_ip="127.0.0.1", user=None, channel_display_name="Test", timestamp_utc="2026-05-12:17-00", channel_logo_id=None, m3u_profile_id=None, + channel_id=1, + channel_uuid="00000000-0000-0000-0000-000000000001", debug=False, ) @@ -1075,10 +1183,10 @@ class TimeshiftSlotPoolTests(_ProxyLoopTestMixin, TestCase): reserves its own slot so concurrent provider connections stay capped by max_streams.""" - POOL_KEY = f"timeshift_pool:{TEST_SESSION_ID}" + POOL_KEY = f"timeshift:pool:{TEST_SESSION_ID}" def _pool_entry_ids(self): - return [k for k in self.fake_redis.store if k.startswith("timeshift_pool:")] + return [k for k in self.fake_redis.store if k.startswith("timeshift:pool:")] def test_reserve_called_with_default_profile_before_upstream(self): streams = [_make_catchup_stream(account_id=1, stream_id="111", profile_id=31)] @@ -1243,7 +1351,7 @@ class TimeshiftPoolReleaseTests(TestCase): self.session_id = TEST_SESSION_ID def _pool_key(self): - return f"timeshift_pool:{self.session_id}" + return f"timeshift:pool:{self.session_id}" def test_release_callback_frees_slot_exactly_once(self): _seed_pool_session(self.redis, session_id=self.session_id) @@ -1262,6 +1370,26 @@ class TimeshiftPoolReleaseTests(TestCase): release_mock.assert_called_once_with(31, self.redis) self.assertFalse(self.redis.exists(self._pool_key())) + def test_refresh_pool_ttl_extends_busy_session(self): + _seed_pool_session(self.redis, session_id=self.session_id, busy="1") + self.redis.hset(self._pool_key(), "last_activity", "1.0") + views._refresh_pool_session_ttl(self.redis, self.session_id) + self.assertGreater( + float(self.redis.hget(self._pool_key(), "last_activity")), 1.0, + ) + + def test_refresh_pool_ttl_extends_idle_session(self): + _seed_pool_session(self.redis, session_id=self.session_id, busy="0") + self.redis.hset(self._pool_key(), "last_activity", "1.0") + views._refresh_pool_session_ttl(self.redis, self.session_id) + self.assertGreater( + float(self.redis.hget(self._pool_key(), "last_activity")), 1.0, + ) + + def test_refresh_pool_ttl_skips_missing_entry(self): + views._refresh_pool_session_ttl(self.redis, self.session_id) + self.assertNotIn(self._pool_key(), self.redis.store) + def test_release_without_redis_is_noop(self): release = views._make_release_once(None, self.session_id, 31) with patch.object(views, "release_profile_slot") as release_mock: @@ -1317,10 +1445,36 @@ class TimeshiftTakeoverTests(TestCase): "type": conn_type, } - def test_displaces_other_positions_on_same_channel(self): + def test_same_session_programme_hop_leaves_stats_to_pool(self): connections = [ - self._conn("timeshift_8_2026-06-08-17-00_111", "timeshift_old1"), - self._conn("timeshift_9_2026-06-08-17-00_222", "timeshift_other"), + self._conn("8_same", "same"), + ] + with patch.object(views, "get_user_active_connections", + return_value=connections), \ + patch.object(views, "_unregister_stats_client") as unregister_mock: + views._terminate_previous_timeshift_sessions( + self.redis, self.user, 8, "8_2026-06-08-17-30", + "same", + ) + unregister_mock.assert_not_called() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + stop_key = RedisKeys.client_stop( + "8_2026-06-08-17-00_111", "same", + ) + self.assertNotIn(stop_key, self.redis.store) + + def test_displaces_other_positions_on_same_channel(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + old_client = "old1" + stats_channel_id = views.stats_channel_id(8, old_client) + programme_vid = "8_2026-06-08-17-00_111" + client_key = RedisKeys.client_metadata(stats_channel_id, old_client) + self.redis.hset(client_key, "programme_vid", programme_vid) + + connections = [ + self._conn(stats_channel_id, old_client), + self._conn("9_2026-06-08-17-00_222", "other"), self._conn("42", "live_client", conn_type="live"), ] with patch.object(views, "get_user_active_connections", @@ -1328,23 +1482,20 @@ class TimeshiftTakeoverTests(TestCase): patch.object(views, "release_profile_slot") as release_mock, \ patch.object(views, "_unregister_stats_client") as unregister_mock: views._terminate_previous_timeshift_sessions( - self.redis, self.user, 8, "timeshift_8_2026-06-09-20-00", "timeshift_current", + self.redis, self.user, 8, "8_2026-06-09-20-00", "current", ) conns_mock.assert_called_once_with(5) # Takeover defers slot release to the displaced generator's stop path; # it only drops stats and signals the stop key. release_mock.assert_not_called() unregister_mock.assert_called_once_with( - self.redis, "timeshift_8_2026-06-08-17-00_111", "timeshift_old1" - ) - from apps.proxy.live_proxy.redis_keys import RedisKeys - stop_key = RedisKeys.client_stop( - "timeshift_8_2026-06-08-17-00_111", "timeshift_old1" + self.redis, stats_channel_id, old_client, ) + stop_key = RedisKeys.client_stop(programme_vid, old_client) self.assertIn(stop_key, self.redis.store) # Channel 9's session untouched: no stop key set for it. other_stop = RedisKeys.client_stop( - "timeshift_9_2026-06-08-17-00_222", "timeshift_other" + "9_2026-06-08-17-00_222", "other" ) self.assertNotIn(other_stop, self.redis.store) @@ -1352,15 +1503,15 @@ class TimeshiftTakeoverTests(TestCase): # Concurrent range/probe requests of the SAME playback must not # displace one another. connections = [ - self._conn("timeshift_8_2026-06-08-17-00_111", "timeshift_sibling"), + self._conn("8_2026-06-08-17-00_111", "sibling"), ] with patch.object(views, "get_user_active_connections", return_value=connections), \ patch.object(views, "release_profile_slot") as release_mock, \ patch.object(views, "_unregister_stats_client") as unregister_mock: views._terminate_previous_timeshift_sessions( - self.redis, self.user, 8, "timeshift_8_2026-06-08-17-00", - "timeshift_sibling", + self.redis, self.user, 8, "8_2026-06-08-17-00", + "sibling", ) release_mock.assert_not_called() unregister_mock.assert_not_called() @@ -1369,15 +1520,15 @@ class TimeshiftTakeoverTests(TestCase): # Channel 8 must not displace channel 80/81 sessions (prefix ends # with an underscore). connections = [ - self._conn("timeshift_80_2026-06-08-17-00_111", "timeshift_c80"), + self._conn("80_2026-06-08-17-00_111", "c80"), ] with patch.object(views, "get_user_active_connections", return_value=connections), \ patch.object(views, "release_profile_slot") as release_mock, \ patch.object(views, "_unregister_stats_client") as unregister_mock: views._terminate_previous_timeshift_sessions( - self.redis, self.user, 8, "timeshift_8_2026-06-08-17-00", - "timeshift_new", + self.redis, self.user, 8, "8_2026-06-08-17-00", + "new", ) release_mock.assert_not_called() unregister_mock.assert_not_called() @@ -1385,10 +1536,10 @@ class TimeshiftTakeoverTests(TestCase): def test_noop_without_redis_or_user(self): with patch.object(views, "get_user_active_connections") as conns_mock: views._terminate_previous_timeshift_sessions( - None, self.user, 8, "timeshift_8_ts", "timeshift_s" + None, self.user, 8, "8_ts", "s" ) views._terminate_previous_timeshift_sessions( - self.redis, None, 8, "timeshift_8_ts", "timeshift_s" + self.redis, None, 8, "8_ts", "s" ) conns_mock.assert_not_called() @@ -1432,7 +1583,7 @@ class TimeshiftSessionReuseTests(TestCase): self.user = MagicMock(id=5) def _pool_key(self): - return f"timeshift_pool:{self.SESSION}" + return f"timeshift:pool:{self.SESSION}" def _make_idle_entry(self): _seed_pool_session(self.redis, session_id=self.SESSION) @@ -1487,7 +1638,7 @@ class TimeshiftSessionReuseTests(TestCase): reserve_mock.assert_not_called() def test_foreign_session_id_redirects_instead_of_reusing_pool(self): - victim_session = "timeshift_victim_session" + victim_session = "victim_session" _seed_pool_session(self.redis, session_id=victim_session, user_id=99) request = self.factory.get(_proxy_url(victim_session)) attacker = MagicMock(id=5) @@ -1508,65 +1659,112 @@ class TimeshiftSessionReuseTests(TestCase): request, "u", "p", "8", "2026-06-08:17-00", "8.ts", ) self.assertEqual(response.status_code, 301) - self.assertIn("session_id=timeshift_", response["Location"]) + self.assertIn("session_id=", response["Location"]) self.assertNotIn(victim_session, response["Location"]) acquire_mock.assert_not_called() attempt_mock.assert_not_called() - def test_find_matching_idle_session_requires_ip_and_user_agent(self): + def test_find_matching_pool_session_idle_requires_ip_and_user_agent(self): _seed_pool_session( - self.redis, session_id="timeshift_other", + self.redis, session_id="other", user_id=5, client_ip="1.2.3.4", client_user_agent="test-agent", ) with patch.object(views, "release_profile_slot"): - views._release_pool_session(self.redis, "timeshift_other", 31) - matched = views._find_matching_idle_session( + views._release_pool_session(self.redis, "other", 31) + matched = views._find_matching_pool_session( self.redis, media_id=TEST_MEDIA_ID, user_id=5, client_ip="1.2.3.4", client_user_agent="test-agent", + include_busy=False, ) - self.assertEqual(matched, "timeshift_other") + self.assertEqual(matched, "other") - def test_find_matching_idle_session_rejects_ip_only_partial_fingerprint(self): + def test_find_matching_pool_session_idle_rejects_ip_only_partial_fingerprint(self): _seed_pool_session( - self.redis, session_id="timeshift_other", + self.redis, session_id="other", user_id=5, client_ip="1.2.3.4", client_user_agent="other-agent", ) with patch.object(views, "release_profile_slot"): - views._release_pool_session(self.redis, "timeshift_other", 31) - matched = views._find_matching_idle_session( + views._release_pool_session(self.redis, "other", 31) + matched = views._find_matching_pool_session( self.redis, media_id=TEST_MEDIA_ID, user_id=5, client_ip="1.2.3.4", client_user_agent="test-agent", + include_busy=False, ) self.assertIsNone(matched) - def test_find_matching_idle_session_rejects_different_user(self): + def test_find_matching_pool_session_idle_rejects_different_user(self): _seed_pool_session( - self.redis, session_id="timeshift_other", + self.redis, session_id="other", user_id=99, client_ip="1.2.3.4", client_user_agent="test-agent", ) with patch.object(views, "release_profile_slot"): - views._release_pool_session(self.redis, "timeshift_other", 31) - matched = views._find_matching_idle_session( + views._release_pool_session(self.redis, "other", 31) + matched = views._find_matching_pool_session( self.redis, media_id=TEST_MEDIA_ID, user_id=5, client_ip="1.2.3.4", client_user_agent="test-agent", + include_busy=False, ) self.assertIsNone(matched) + def test_find_matching_pool_session_finds_busy_pool_on_same_channel(self): + _seed_pool_session( + self.redis, session_id="busy", + media_id="8_2026-06-08-17-00", + user_id=5, client_ip="1.2.3.4", client_user_agent="test-agent", + busy="1", + ) + matched = views._find_matching_pool_session( + self.redis, + media_id="8_2026-06-08-17-30", + channel_id=8, + user_id=5, + client_ip="1.2.3.4", + client_user_agent="test-agent", + include_busy=True, + ) + self.assertEqual(matched, "busy") + + def test_find_matching_pool_session_finds_busy_pool(self): + _seed_pool_session( + self.redis, session_id="busy", + user_id=5, client_ip="1.2.3.4", client_user_agent="test-agent", + busy="1", + ) + matched = views._find_matching_pool_session( + self.redis, + media_id=TEST_MEDIA_ID, + user_id=5, + client_ip="1.2.3.4", + client_user_agent="test-agent", + include_busy=True, + ) + self.assertEqual(matched, "busy") + self.assertIsNone( + views._find_matching_pool_session( + self.redis, + media_id=TEST_MEDIA_ID, + user_id=5, + client_ip="1.2.3.4", + client_user_agent="test-agent", + include_busy=False, + ) + ) + def test_legacy_pool_entry_exists_helper_removed(self): self.assertFalse(hasattr(views, "_pool_entry_exists")) def test_new_session_uses_single_hgetall_before_pool_create(self): redis = _FakeRedis() - request = self.factory.get(_proxy_url("timeshift_newsession1")) + request = self.factory.get(_proxy_url("newsession1")) with patch.object(views, "_authenticate_user", return_value=MagicMock(id=5)), \ patch.object(views, "network_access_allowed", return_value=True), \ patch.object(views, "Channel") as channel_cls, \ @@ -1575,7 +1773,7 @@ class TimeshiftSessionReuseTests(TestCase): return_value=[_make_catchup_stream()]), \ patch.object(views, "get_programme_duration", return_value=40), \ patch.object(views, "check_user_stream_limits", return_value=True), \ - patch.object(views, "_find_matching_idle_session", return_value=None), \ + patch.object(views, "_find_matching_pool_session", return_value=None), \ patch.object(views, "_attempt_timeshift_stream", return_value=MagicMock(status_code=200)), \ patch.object(views, "RedisClient") as redis_cls, \ @@ -1589,7 +1787,7 @@ class TimeshiftSessionReuseTests(TestCase): views.timeshift_proxy( request, "u", "p", "8", "2026-06-08:17-00", "8.ts", ) - pool_key = "timeshift_pool:timeshift_newsession1" + pool_key = "timeshift:pool:newsession1" self.assertEqual( sum(1 for c in hgetall_mock.call_args_list if c.args == (pool_key,)), 1, @@ -1672,6 +1870,7 @@ class TimeshiftSessionReuseTests(TestCase): duration_minutes=40, client_id=self.SESSION, client_ip="1.2.3.4", + client_user_agent="test-agent", range_header=None, channel_logo_id=None, user=self.user, @@ -1687,7 +1886,7 @@ class TimeshiftSessionReuseTests(TestCase): # serve the REQUESTED timestamp. _seed_pool_session(self.redis, session_id=self.SESSION) response, ok, attempt_mock = self._call_reused_session( - "2026-06-08:17-30", "timeshift_8_2026-06-08-17-30", + "2026-06-08:17-30", "8_2026-06-08-17-30", ) self.assertIs(response, ok) # June -> CEST: 17:30 UTC reaches the provider as 19:30 local — never @@ -1703,11 +1902,11 @@ class TimeshiftSessionReuseTests(TestCase): # being served. _seed_pool_session(self.redis, session_id=self.SESSION) self._call_reused_session( - "2026-06-08:17-30", "timeshift_8_2026-06-08-17-30", + "2026-06-08:17-30", "8_2026-06-08-17-30", ) self.assertEqual( self.redis.hget(self._pool_key(), "media_id"), - "timeshift_8_2026-06-08-17-30", + "8_2026-06-08-17-30", ) self.assertEqual( self.redis.hget(self._pool_key(), "provider_timestamp"), @@ -1723,7 +1922,7 @@ class TimeshiftSessionReuseTests(TestCase): "content_length": "2000000000", "serving_range": "start", }) self._call_reused_session( - "2026-06-08:17-30", "timeshift_8_2026-06-08-17-30", + "2026-06-08:17-30", "8_2026-06-08-17-30", ) self.assertIsNone(self.redis.hget(self._pool_key(), "content_length")) self.assertIsNone(self.redis.hget(self._pool_key(), "serving_range")) @@ -1733,7 +1932,7 @@ class TimeshiftSessionReuseTests(TestCase): "content_length": "1000000", "serving_range": "range", }) self._call_reused_session( - "2026-06-08:17-30", "timeshift_8_2026-06-08-17-30", + "2026-06-08:17-30", "8_2026-06-08-17-30", ) self.assertEqual( self.redis.hget(self._pool_key(), "content_length"), "1000000", @@ -1744,7 +1943,7 @@ class TimeshiftSessionReuseTests(TestCase): # recreate a partial TTL-less hash that 503-wedges the session_id. views._update_pool_position( self.redis, self.SESSION, - media_id="timeshift_8_2026-06-08-17-30", + media_id="8_2026-06-08-17-30", provider_timestamp="2026-06-08:19-30", ) self.assertFalse(self.redis.exists(self._pool_key())) @@ -1772,12 +1971,13 @@ class TimeshiftSessionReuseTests(TestCase): }, profile=profile, channel=self.channel, - media_id="timeshift_8_2026-06-08-17-30", + media_id="8_2026-06-08-17-30", safe_ts="2026-06-08-17-30", timestamp="2026-06-08:17-30", duration_minutes=40, client_id=self.SESSION, client_ip="1.2.3.4", + client_user_agent="test-agent", range_header=None, channel_logo_id=None, user=self.user, @@ -1840,7 +2040,7 @@ class TimeshiftSessionReuseTests(TestCase): ) self.assertEqual( self.redis.hget(self._pool_key(), "media_id"), - "timeshift_8_2026-06-08-17-30", + "8_2026-06-08-17-30", ) @@ -1867,7 +2067,74 @@ class TimeshiftSessionRedirectTests(TestCase): request, "u", "p", "8", "2026-06-08:17-00", "8.ts", ) self.assertEqual(response.status_code, 301) - self.assertIn("session_id=timeshift_", response["Location"]) + self.assertIn("session_id=", response["Location"]) + + def test_missing_session_id_redirects_to_existing_busy_pool(self): + existing = "existingbusy1" + redis = _FakeRedis() + _seed_pool_session( + redis, + session_id=existing, + user_id=1, + client_ip="127.0.0.1", + client_user_agent="vlc-test", + busy="1", + ) + request = self.factory.get( + _proxy_url(session_id=None), + HTTP_USER_AGENT="vlc-test", + REMOTE_ADDR="127.0.0.1", + ) + with patch.object(views, "_authenticate_user", return_value=MagicMock(id=1)), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "get_channel_catchup_streams", + return_value=[_make_catchup_stream()]), \ + patch.object(views, "get_programme_duration", return_value=40), \ + patch.object(views, "parse_catchup_timestamp", return_value=True), \ + patch.object(views, "RedisClient") as redis_cls: + redis_cls.get_client.return_value = redis + channel_cls.objects.get.return_value = MagicMock(id=8) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-00", "8.ts", + ) + self.assertEqual(response.status_code, 301) + self.assertIn(f"session_id={existing}", response["Location"]) + + def test_missing_session_id_redirects_to_busy_pool_on_channel_hop(self): + existing = "channelhop1" + redis = _FakeRedis() + _seed_pool_session( + redis, + session_id=existing, + media_id="8_2026-06-08-17-00", + user_id=1, + client_ip="127.0.0.1", + client_user_agent="vlc-test", + busy="1", + ) + request = self.factory.get( + "/timeshift/u/p/8/2026-06-08:17-30/8.ts", + HTTP_USER_AGENT="vlc-test", + REMOTE_ADDR="127.0.0.1", + ) + with patch.object(views, "_authenticate_user", return_value=MagicMock(id=1)), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "get_channel_catchup_streams", + return_value=[_make_catchup_stream()]), \ + patch.object(views, "get_programme_duration", return_value=40), \ + patch.object(views, "parse_catchup_timestamp", return_value=True), \ + patch.object(views, "RedisClient") as redis_cls: + redis_cls.get_client.return_value = redis + channel_cls.objects.get.return_value = MagicMock(id=8) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-30", "8.ts", + ) + self.assertEqual(response.status_code, 301) + self.assertIn(f"session_id={existing}", response["Location"]) def test_redirect_preserves_existing_query_params(self): request = self.factory.get( @@ -1889,7 +2156,7 @@ class TimeshiftSessionRedirectTests(TestCase): ) self.assertEqual(response.status_code, 301) location = response["Location"] - self.assertIn("session_id=timeshift_", location) + self.assertIn("session_id=", location) self.assertIn("foo=bar", location) self.assertIn("baz=1", location) @@ -1963,7 +2230,7 @@ class TimeshiftStreamLimitExemptionTests(TestCase): def test_different_session_same_programme_counts_against_limit(self): connections = [{ "media_id": f"{self.MEDIA}_111", - "client_id": "timeshift_other_session", + "client_id": "other_session", "connected_at": 0.0, "type": "timeshift", }] @@ -1976,15 +2243,67 @@ class TimeshiftStreamLimitExemptionTests(TestCase): ) self.assertFalse(allowed) + def test_same_session_probe_allowed_via_stable_stats_channel(self): + # Programme hop: the active connection is tracked under the stable + # stats channel id (timeshift_{channel}_{client_id}) from an OLDER + # programme, while this request is for a NEW programme timestamp on + # the same channel. Same client/channel must still be exempt. + connections = [{ + "media_id": f"8_{TEST_SESSION_ID}", + "client_id": TEST_SESSION_ID, + "connected_at": 0.0, + "type": "timeshift", + }] + with patch("apps.proxy.utils.get_user_active_connections", + return_value=connections), \ + patch("apps.proxy.utils.CoreSettings.get_user_limits_settings", + return_value=self._limits_settings()): + allowed = _check_user_stream_limits( + self.user, TEST_SESSION_ID, media_id="8_2026-06-08-17-30", + ) + self.assertTrue(allowed) + + def test_different_channel_stable_stats_key_not_exempt(self): + connections = [{ + "media_id": f"9_{TEST_SESSION_ID}", + "client_id": TEST_SESSION_ID, + "connected_at": 0.0, + "type": "timeshift", + }] + with patch("apps.proxy.utils.get_user_active_connections", + return_value=connections), \ + patch("apps.proxy.utils.CoreSettings.get_user_limits_settings", + return_value=self._limits_settings(ignore_same_channel=False)): + allowed = _check_user_stream_limits( + self.user, TEST_SESSION_ID, media_id="8_2026-06-08-17-30", + ) + self.assertFalse(allowed) + + +class TimeshiftPoolIdleTtlTests(TestCase): + """The idle pool entry (fingerprint recovery) must outlive the stats + client entry, or a reconnect within that window mints a NEW session + (pool forgot it) while takeover logic still displaces the "old" one + (stats hadn't forgotten it yet) -- contradictory outcomes for what is + really the same viewer reconnecting after a pause.""" + + def test_pool_idle_ttl_covers_stats_client_ttl(self): + self.assertGreaterEqual(views._POOL_IDLE_TTL, views.CLIENT_TTL_SECONDS) + + def test_pool_idle_ttl_covers_disconnect_grace(self): + self.assertGreaterEqual( + views._POOL_IDLE_TTL, views._STATS_DISCONNECT_GRACE_SECONDS, + ) + class FakeRedisScanTests(TestCase): """FakeRedis SCAN matches redis-py glob semantics used by the pool scanner.""" def setUp(self): self.redis = _FakeRedis() - self.redis.store["timeshift_pool:timeshift_a"] = {"busy": "0"} - self.redis.store["timeshift_pool:timeshift_b"] = {"busy": "0"} - self.redis.store["timeshift_pool:other_c"] = {"busy": "0"} + self.redis.store["timeshift:pool:a"] = {"busy": "0"} + self.redis.store["timeshift:pool:b"] = {"busy": "0"} + self.redis.store["timeshift_pool_legacy:other_c"] = {"busy": "0"} self.redis.store["vod_persistent_connection:x"] = {} def test_scan_glob_filters_pool_keys(self): @@ -1992,14 +2311,14 @@ class FakeRedisScanTests(TestCase): seen = [] while True: cursor, keys = self.redis.scan( - cursor, match="timeshift_pool:timeshift_*", count=1, + cursor, match=TimeshiftRedisKeys.pool_scan_pattern(), count=1, ) seen.extend(keys) if cursor == 0: break self.assertEqual( seen, - ["timeshift_pool:timeshift_a", "timeshift_pool:timeshift_b"], + ["timeshift:pool:a", "timeshift:pool:b"], ) @@ -2045,6 +2364,1352 @@ class TimeshiftRangeClassificationTests(TestCase): def test_small_nonzero_range_is_displacing(self): self.assertTrue(views._should_displace_busy_playback("bytes=1000-")) + def test_programme_change_does_not_displace_busy_pool(self): + self.assertFalse( + views._should_displace_busy_pool( + None, None, None, + pool_media_id="8_2026-06-08-17-00", + media_id="8_2026-06-08-17-30", + ) + ) + + def test_programme_change_preempt_after_startup_window(self): + redis = _FakeRedis() + _seed_pool_session( + redis, session_id=TEST_SESSION_ID, + media_id="8_2026-06-08-17-00", + ) + redis.hset( + views._pool_key(TEST_SESSION_ID), "last_activity", "1.0", + ) + with patch.object(views.time, "time", return_value=10.0): + self.assertTrue( + views._should_preempt_for_programme_change( + redis, TEST_SESSION_ID, + "8_2026-06-08-17-00", + "8_2026-06-08-17-30", + ), + ) + + def test_parallel_programme_probe_does_not_preempt(self): + redis = _FakeRedis() + _seed_pool_session( + redis, session_id=TEST_SESSION_ID, + media_id="8_2026-06-08-17-00", + ) + redis.hset( + views._pool_key(TEST_SESSION_ID), + "last_activity", + str(views.time.time()), + ) + self.assertFalse( + views._should_preempt_for_programme_change( + redis, TEST_SESSION_ID, + "8_2026-06-08-17-00", + "8_2026-06-08-17-30", + ), + ) + + +class TimeshiftStatsClientTests(TestCase): + def setUp(self): + self.redis = _FakeRedis() + self.virtual_channel_id = f"{TEST_MEDIA_ID}_111" + self.stats_channel_id = views.stats_channel_id(8, TEST_SESSION_ID) + self.client_id = TEST_SESSION_ID + self.user = MagicMock(id=5, username="viewer") + + def test_register_stats_preserves_connected_at_on_reregister(self): + from apps.proxy.live_proxy.constants import ChannelMetadataField + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + metadata_key = RedisKeys.channel_metadata(self.stats_channel_id) + self.redis.hset(client_key, "connected_at", "1000.0") + self.redis.hset(metadata_key, ChannelMetadataField.INIT_TIME, "1000.0") + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + ) + + self.assertEqual(self.redis.hget(client_key, "connected_at"), "1000.0") + self.assertEqual( + self.redis.hget(metadata_key, ChannelMetadataField.INIT_TIME), "1000.0", + ) + self.assertEqual( + self.redis.hget(client_key, "programme_vid"), self.virtual_channel_id, + ) + self.assertEqual( + self.redis.hget(metadata_key, ChannelMetadataField.CHANNEL_ID), "8", + ) + self.assertEqual( + self.redis.hget(metadata_key, ChannelMetadataField.CHANNEL_UUID), + "00000000-0000-0000-0000-000000000008", + ) + self.assertIsNone(self.redis.hget(client_key, "channel_id")) + + def test_register_stats_emits_update_for_new_client(self): + with patch.object(views, "_trigger_timeshift_stats_update") as trigger_mock: + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + emit_stats_update=True, + ) + trigger_mock.assert_called_once_with(self.redis) + + def test_register_stats_skips_update_for_same_client_reregister(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "connected_at", "1000.0") + self.redis.hset(client_key, "programme_vid", self.virtual_channel_id) + with patch.object(views, "_trigger_timeshift_stats_update") as trigger_mock: + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + range_start=500_000_000, + representation_length=1_000_000_000, + programme_duration_secs=3600, + emit_stats_update=True, + ) + trigger_mock.assert_not_called() + + def test_register_stats_emits_update_for_programme_change(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + new_programme_vid = "8_2026-06-08-17-30_111" + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "connected_at", "1000.0") + self.redis.hset(client_key, "programme_vid", self.virtual_channel_id) + with patch.object(views, "_trigger_timeshift_stats_update") as trigger_mock: + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-30", + primary_url="http://example.test/timeshift.ts", + programme_vid=new_programme_vid, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + emit_stats_update=True, + ) + trigger_mock.assert_called_once_with(self.redis) + + def test_register_stats_updates_programme_on_hop(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + new_programme_vid = "8_2026-06-08-17-30_111" + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "connected_at", "1000.0") + self.redis.hset(client_key, "programme_vid", self.virtual_channel_id) + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-30", + primary_url="http://example.test/timeshift.ts", + programme_vid=new_programme_vid, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + ) + + self.assertEqual(self.redis.hget(client_key, "connected_at"), "1000.0") + self.assertEqual(self.redis.hget(client_key, "programme_vid"), new_programme_vid) + + def test_register_stats_cleans_old_programme_stream_generation_on_hop(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + old_programme_vid = "8_2026-06-08-17-00_111" + new_programme_vid = "8_2026-06-08-17-30_111" + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "programme_vid", old_programme_vid) + old_gen = views._stream_generation_key(old_programme_vid, self.client_id) + self.redis.incr(old_gen) + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-30", + primary_url="http://example.test/timeshift.ts", + programme_vid=new_programme_vid, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + ) + + self.assertNotIn(old_gen, self.redis.store) + + def test_register_stats_preserves_position_anchor_on_same_programme(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "programme_start", "2026-06-08:17-00") + self.redis.hset(client_key, "position_anchor_at", "1000.0") + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + ) + self.assertEqual(self.redis.hget(client_key, "position_anchor_at"), "1000.0") + + def test_register_stats_byte_range_seek_sets_playback_base(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "programme_start", "2026-06-08:17-00") + self.redis.hset(client_key, "position_anchor_at", "1000.0") + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + range_start=500_000_000, + representation_length=1_000_000_000, + programme_duration_secs=3600, + ) + self.assertAlmostEqual( + float(self.redis.hget(client_key, "playback_base_secs")), + 1800.0, + delta=1.0, + ) + self.assertNotEqual(self.redis.hget(client_key, "position_anchor_at"), "1000.0") + + def test_register_stats_reanchors_position_on_seek(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata(self.stats_channel_id, self.client_id) + self.redis.hset(client_key, "programme_start", "2026-06-08:17-00") + self.redis.hset(client_key, "position_anchor_at", "1000.0") + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-19", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + ) + self.assertNotEqual(self.redis.hget(client_key, "position_anchor_at"), "1000.0") + + def test_register_stats_seeds_stream_stats_from_memory(self): + from apps.proxy.live_proxy.constants import ChannelMetadataField + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + metadata_key = RedisKeys.channel_metadata(self.stats_channel_id) + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + stats_stream_id=42, + stream_stats={ + "resolution": "1920x1080", + "source_fps": 29.97, + "video_codec": "h264", + "audio_codec": "aac", + }, + ) + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.RESOLUTION), "1920x1080") + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.SOURCE_FPS), "29.97") + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.STREAM_ID), "42") + + def test_register_stats_skips_stream_stats_when_stream_unchanged(self): + from apps.proxy.live_proxy.constants import ChannelMetadataField + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + metadata_key = RedisKeys.channel_metadata(self.stats_channel_id) + self.redis.hset(metadata_key, mapping={ + ChannelMetadataField.STREAM_ID: "42", + ChannelMetadataField.RESOLUTION: "1280x720", + }) + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + stats_stream_id=42, + stream_stats={"resolution": "1920x1080"}, + ) + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.RESOLUTION), "1280x720") + + def test_register_stats_updates_stream_stats_on_failover_stream_change(self): + from apps.proxy.live_proxy.constants import ChannelMetadataField + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + metadata_key = RedisKeys.channel_metadata(self.stats_channel_id) + self.redis.hset(metadata_key, mapping={ + ChannelMetadataField.STREAM_ID: "42", + ChannelMetadataField.RESOLUTION: "1280x720", + }) + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + stats_stream_id=99, + stream_stats={"resolution": "1920x1080", "video_codec": "hevc"}, + ) + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.STREAM_ID), "99") + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.RESOLUTION), "1920x1080") + self.assertEqual(self.redis.hget(metadata_key, ChannelMetadataField.VIDEO_CODEC), "hevc") + + @patch.object(views, "_open_upstream") + def test_stream_keeps_stats_when_stopped_for_seek(self, mocked_open): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + ts = _make_ts_payload(2048) + upstream = MagicMock() + upstream.status_code = 206 + upstream.headers = { + "Content-Type": "video/mp2t", + "Content-Range": "bytes 0-2047/10000", + } + upstream.raw.read = MagicMock(side_effect=[ts, ts, ts, b""]) + upstream.close = MagicMock() + mocked_open.return_value = upstream + + release_cb = MagicMock() + with patch.object(views, "_unregister_stats_client") as unregister_mock, \ + patch.object(views, "_schedule_stats_disconnect_grace") as grace_mock: + response = views._stream_from_provider( + candidate_urls=["http://example.test/timeshift.ts"], + user_agent="provider-agent", + client_user_agent="vlc", + range_header="bytes=0-", + virtual_channel_id=self.virtual_channel_id, + stats_channel_id=self.stats_channel_id, + client_id=self.client_id, + client_ip="1.2.3.4", + user=self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + channel_logo_id=None, + m3u_profile_id=31, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + debug=False, + redis_client=self.redis, + release_cb=release_cb, + ) + + self.assertEqual(response["Content-Range"], "bytes 0-2047/10000") + self.assertEqual(response["Content-Length"], "2048") + self.assertEqual(response["Accept-Ranges"], "bytes") + + iterator = iter(response.streaming_content) + next(iterator) + + stop_key = RedisKeys.client_stop(self.virtual_channel_id, self.client_id) + self.redis.setex(stop_key, 60, views._STOP_REASON_REUSE) + + for _ in iterator: + pass + response.close() + + unregister_mock.assert_not_called() + grace_mock.assert_not_called() + release_cb.assert_called_once() + + @patch.object(views, "_open_upstream") + def test_stream_schedules_grace_on_plain_client_disconnect(self, mocked_open): + # Known client disconnect (no preempt stop key): defer stats removal so + # a reconnect within the grace window keeps connected_at continuity. + ts = _make_ts_payload(65536) + upstream = MagicMock() + upstream.status_code = 206 + upstream.headers = { + "Content-Type": "video/mp2t", + "Content-Range": "bytes 0-65535/100000", + } + upstream.raw.read = MagicMock(side_effect=[ts, ConnectionError("reset by peer")]) + upstream.close = MagicMock() + mocked_open.return_value = upstream + + release_cb = MagicMock() + with patch.object(views, "_unregister_stats_client") as unregister_mock, \ + patch.object(views, "_schedule_stats_disconnect_grace") as grace_mock: + response = views._stream_from_provider( + candidate_urls=["http://example.test/timeshift.ts"], + user_agent="provider-agent", + client_user_agent="vlc", + range_header="bytes=0-", + virtual_channel_id=self.virtual_channel_id, + stats_channel_id=self.stats_channel_id, + client_id=self.client_id, + client_ip="1.2.3.4", + user=self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + channel_logo_id=None, + m3u_profile_id=31, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + debug=False, + redis_client=self.redis, + release_cb=release_cb, + ) + + for _ in response.streaming_content: + pass + response.close() + + unregister_mock.assert_not_called() + grace_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.client_id, + ) + release_cb.assert_called_once_with( + mark_pool_idle=True, release_profile=True, + ) + + @patch.object(views, "_open_upstream") + def test_stopped_for_reuse_handoff_keeps_profile_slot(self, mocked_open): + """After a scrub preempt, the displaced stream keeps the profile slot.""" + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + _seed_pool_session(self.redis, session_id=self.client_id) + ts = _make_ts_payload(65536) + upstream = MagicMock() + upstream.status_code = 206 + upstream.headers = { + "Content-Type": "video/mp2t", + "Content-Range": "bytes 0-65535/100000", + } + upstream.raw.read = MagicMock(side_effect=[ts, b""]) + upstream.close = MagicMock() + mocked_open.return_value = upstream + + release_cb = views._make_release_once(self.redis, self.client_id, 31) + with patch.object(views, "_schedule_stats_disconnect_grace") as grace_mock, \ + patch.object(views, "release_profile_slot") as release_mock: + response = views._stream_from_provider( + candidate_urls=["http://example.test/timeshift.ts"], + user_agent="provider-agent", + client_user_agent="vlc", + range_header="bytes=0-", + virtual_channel_id=self.virtual_channel_id, + stats_channel_id=self.stats_channel_id, + client_id=self.client_id, + client_ip="1.2.3.4", + user=self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + channel_logo_id=None, + m3u_profile_id=31, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + debug=False, + redis_client=self.redis, + release_cb=release_cb, + ) + + iterator = iter(response.streaming_content) + next(iterator) + stop_key = RedisKeys.client_stop(self.virtual_channel_id, self.client_id) + self.redis.setex(stop_key, 60, "1") + for _ in iterator: + pass + response.close() + + grace_mock.assert_not_called() + release_mock.assert_not_called() + self.assertEqual( + self.redis.hget(f"timeshift:pool:{self.client_id}", "busy"), + "1", + ) + + def test_handoff_reacquire_skips_profile_reserve(self): + _seed_pool_session(self.redis, session_id=TEST_SESSION_ID, busy="1") + profile = MagicMock(id=31) + with patch.object(views.M3UAccountProfile.objects, "get", return_value=profile), \ + patch.object(views, "reserve_profile_slot") as reserve_mock: + acquired = views._acquire_idle_pool_session( + self.redis, TEST_SESSION_ID, handoff=True, + ) + self.assertIsNotNone(acquired) + reserve_mock.assert_not_called() + self.assertEqual( + self.redis.hget(f"timeshift:pool:{TEST_SESSION_ID}", "busy"), + "1", + ) + + def test_handoff_release_keeps_pool_busy(self): + _seed_pool_session(self.redis, session_id=TEST_SESSION_ID, busy="1") + with patch.object(views, "release_profile_slot") as release_mock: + views._release_pool_session( + self.redis, TEST_SESSION_ID, 31, + mark_pool_idle=False, release_profile=False, + ) + release_mock.assert_not_called() + self.assertEqual( + self.redis.hget(f"timeshift:pool:{TEST_SESSION_ID}", "busy"), + "1", + ) + + @patch.object(views, "_open_upstream") + def test_disconnect_after_seek_preempt_schedules_grace(self, mocked_open): + """The replacement stream after a seek must still arm disconnect cleanup.""" + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + ts = _make_ts_payload(65536) + upstream = MagicMock() + upstream.status_code = 206 + upstream.headers = { + "Content-Type": "video/mp2t", + "Content-Range": "bytes 0-65535/100000", + } + upstream.raw.read = MagicMock(side_effect=[ts, ConnectionError("reset by peer")]) + upstream.close = MagicMock() + mocked_open.return_value = upstream + + release_cb = MagicMock() + gen_key = views._stream_generation_key( + self.virtual_channel_id, self.client_id, + ) + self.redis.set(gen_key, "1") + stop_key = RedisKeys.client_stop(self.virtual_channel_id, self.client_id) + self.redis.setex(stop_key, 60, "1") + + with patch.object(views, "_schedule_stats_disconnect_grace") as grace_mock: + response = views._stream_from_provider( + candidate_urls=["http://example.test/timeshift.ts"], + user_agent="provider-agent", + client_user_agent="vlc", + range_header="bytes=1000-", + virtual_channel_id=self.virtual_channel_id, + stats_channel_id=self.stats_channel_id, + client_id=self.client_id, + client_ip="1.2.3.4", + user=self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + channel_logo_id=None, + m3u_profile_id=31, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + debug=False, + redis_client=self.redis, + release_cb=release_cb, + ) + + for _ in response.streaming_content: + pass + response.close() + + self.assertNotIn(stop_key, self.redis.store) + grace_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.client_id, + ) + release_cb.assert_called_once_with( + mark_pool_idle=True, release_profile=True, + ) + + def test_create_pool_session_clears_superseded_marker(self): + self.redis.set(views._superseded_pool_key(TEST_SESSION_ID), "1") + created = views._create_pool_session( + self.redis, + session_id=TEST_SESSION_ID, + media_id=TEST_MEDIA_ID, + user_id=5, + client_ip="1.2.3.4", + client_user_agent="test-agent", + account_id=1, + profile_id=31, + stream_id="111", + dispatcharr_stream_id=10, + provider_timestamp="2026-06-08:19-00", + ) + self.assertTrue(created) + self.assertNotIn( + views._superseded_pool_key(TEST_SESSION_ID), self.redis.store, + ) + + @patch.object(views, "_open_upstream") + def test_stream_disconnect_cleans_programme_stream_generation(self, mocked_open): + ts = _make_ts_payload(65536) + upstream = MagicMock() + upstream.status_code = 206 + upstream.headers = { + "Content-Type": "video/mp2t", + "Content-Range": "bytes 0-65535/100000", + } + upstream.raw.read = MagicMock(side_effect=[ts, b""]) + upstream.close = MagicMock() + mocked_open.return_value = upstream + + gen_key = views._stream_generation_key( + self.virtual_channel_id, self.client_id, + ) + release_cb = MagicMock() + with patch.object(views, "_schedule_stats_disconnect_grace"): + response = views._stream_from_provider( + candidate_urls=["http://example.test/timeshift.ts"], + user_agent="provider-agent", + client_user_agent="vlc", + range_header="bytes=0-", + virtual_channel_id=self.virtual_channel_id, + stats_channel_id=self.stats_channel_id, + client_id=self.client_id, + client_ip="1.2.3.4", + user=self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + channel_logo_id=None, + m3u_profile_id=31, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + debug=False, + redis_client=self.redis, + release_cb=release_cb, + ) + for _ in response.streaming_content: + pass + response.close() + + self.assertNotIn(gen_key, self.redis.store) + + @patch.object(views, "_open_upstream") + def test_stream_skips_grace_when_newer_generation_active(self, mocked_open): + # Seek race: the replacement stream bumps generation before the old + # generator's finally runs (VLC resets TCP without a polled stop key). + ts = _make_ts_payload(2048) + upstream = MagicMock() + upstream.status_code = 206 + upstream.headers = { + "Content-Type": "video/mp2t", + "Content-Range": "bytes 0-2047/10000", + } + upstream.raw.read = MagicMock(side_effect=[ts, b""]) + upstream.close = MagicMock() + mocked_open.return_value = upstream + + release_cb = MagicMock() + gen_key = views._stream_generation_key( + self.virtual_channel_id, self.client_id, + ) + + with patch.object(views, "_unregister_stats_client") as unregister_mock, \ + patch.object(views, "_schedule_stats_disconnect_grace") as grace_mock: + response = views._stream_from_provider( + candidate_urls=["http://example.test/timeshift.ts"], + user_agent="provider-agent", + client_user_agent="vlc", + range_header="bytes=0-", + virtual_channel_id=self.virtual_channel_id, + stats_channel_id=self.stats_channel_id, + client_id=self.client_id, + client_ip="1.2.3.4", + user=self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + channel_logo_id=None, + m3u_profile_id=31, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + debug=False, + redis_client=self.redis, + release_cb=release_cb, + ) + + iterator = iter(response.streaming_content) + next(iterator) + self.redis.incr(gen_key) + for _ in iterator: + pass + response.close() + + unregister_mock.assert_not_called() + grace_mock.assert_not_called() + release_cb.assert_called_once() + + def test_register_cancels_pending_stats_grace(self): + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + self.redis.setex(grace_key, 10, "pending-token") + + views._register_stats_client( + self.redis, + self.stats_channel_id, + self.client_id, + "1.2.3.4", + "vlc", + self.user, + channel_display_name="A&E", + timestamp_utc="2026-06-08:17-00", + primary_url="http://example.test/timeshift.ts", + programme_vid=self.virtual_channel_id, + channel_id=8, + channel_uuid="00000000-0000-0000-0000-000000000008", + ) + + self.assertNotIn(grace_key, self.redis.store) + + def test_grace_unregister_runs_when_not_cancelled(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + + with patch.object(views, "_unregister_stats_client") as unregister_mock, \ + patch.object(views, "_trigger_timeshift_stats_update") as trigger_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=2000.0, + ) + + unregister_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.client_id, + ) + trigger_mock.assert_called_once_with(self.redis) + self.assertNotIn(grace_key, self.redis.store) + + def test_grace_unregister_skipped_when_reconnect_cancelled(self): + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + self.redis.setex(grace_key, 10, "new-token") + + with patch.object(views, "_unregister_stats_client") as unregister_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, "old-token", + disconnected_at=1000.0, + ) + + unregister_mock.assert_not_called() + + def test_grace_unregister_skipped_when_client_reconnected(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "2000.0") + + with patch.object(views, "_unregister_stats_client") as unregister_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=1000.0, + ) + + unregister_mock.assert_not_called() + + def test_grace_unregister_deletes_api_catchup_session(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + session_key = TimeshiftRedisKeys.api_session(self.client_id) + self.redis.hset(session_key, "user_id", "5") + + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=2000.0, + ) + + self.assertNotIn(session_key, self.redis.store) + + def test_grace_unregister_cleans_stream_generation(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + old_vid = "8_2026-06-08-17-00_111" + new_vid = "8_2026-06-08-17-30_111" + self.redis.hset(client_key, "programme_vid", new_vid) + old_gen = views._stream_generation_key(old_vid, self.client_id) + new_gen = views._stream_generation_key(new_vid, self.client_id) + self.redis.incr(old_gen) + self.redis.incr(new_gen) + + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=2000.0, + ) + + self.assertNotIn(old_gen, self.redis.store) + self.assertNotIn(new_gen, self.redis.store) + + def test_grace_unregister_discards_pool_for_api_session(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + self.redis.hset(TimeshiftRedisKeys.api_session(self.client_id), "user_id", "5") + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, mapping={"busy": "0", "profile_id": "31"}) + + with patch.object(views, "release_profile_slot") as release_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=2000.0, + ) + + self.assertNotIn(pool_key, self.redis.store) + release_mock.assert_not_called() + + def test_grace_unregister_keeps_idle_pool_for_xc_session(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, "busy", "0") + + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=2000.0, + ) + + self.assertIn(pool_key, self.redis.store) + self.assertEqual( + self.redis.ttl.get(pool_key), views._POOL_IDLE_TTL, + ) + + def test_allocate_stream_generation_sets_ttl(self): + with patch.object(self.redis, "expire", wraps=self.redis.expire) as expire_mock: + views._allocate_stream_generation( + self.redis, self.virtual_channel_id, self.client_id, + ) + gen_key = views._stream_generation_key( + self.virtual_channel_id, self.client_id, + ) + expire_mock.assert_called_once_with(gen_key, views._POOL_ENTRY_TTL) + + def test_grace_unregister_runs_when_last_active_equals_disconnect(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + + with patch.object(views, "_unregister_stats_client") as unregister_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=1000.0, + ) + + unregister_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.client_id, + ) + + def test_should_schedule_grace_skips_startup_probe(self): + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, "busy", "0") + self.assertFalse( + views._should_schedule_stats_disconnect_grace( + 996, 0.5, stopped_for_reuse=False, + ), + ) + self.assertFalse( + views._should_schedule_stats_disconnect_grace( + 525220, 0.9, + stopped_for_reuse=False, + redis_client=self.redis, + client_id=self.client_id, + ), + ) + self.assertTrue( + views._should_schedule_stats_disconnect_grace( + 996, 3.0, stopped_for_reuse=False, + ), + ) + self.assertTrue( + views._should_schedule_stats_disconnect_grace( + 120000, 0.5, stopped_for_reuse=False, + ), + ) + self.assertTrue( + views._should_schedule_stats_disconnect_grace( + 120000, 3.0, + stopped_for_reuse=False, + redis_client=self.redis, + client_id=self.client_id, + ), + ) + self.assertFalse( + views._should_schedule_stats_disconnect_grace( + 120000, 0.5, stopped_for_reuse=True, + ), + ) + + def test_touch_stats_on_session_request_cancels_grace(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + self.redis.setex(grace_key, 10, "pending-token") + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, mapping={ + "last_active": "1000.0", + "channel_id": "8", + "programme_start": "2026-06-08:17-00", + }) + + views._touch_stats_on_session_request( + self.redis, 8, self.client_id, + ) + + self.assertNotIn(grace_key, self.redis.store) + last_active = float(self.redis.hget(client_key, "last_active")) + self.assertGreater(last_active, 1000.0) + + def test_grace_unregister_skipped_when_client_reconnected_during_settle(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "2000.0") + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, mapping={"busy": "0", "last_activity": "2000.0"}) + + with patch.object(views, "_unregister_stats_client") as unregister_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=1000.0, + ) + + unregister_mock.assert_not_called() + + def test_stats_client_reconnected_ignores_idle_pool_release_timestamp(self): + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, mapping={"busy": "0", "last_activity": "2000.0"}) + self.assertFalse( + views._stats_client_reconnected( + self.redis, self.stats_channel_id, self.client_id, + disconnected_at=1000.0, + ), + ) + + def test_grace_unregister_runs_after_idle_pool_release_timestamp(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, mapping={"busy": "0", "last_activity": "2000.0"}) + session_key = TimeshiftRedisKeys.api_session(self.client_id) + self.redis.hset(session_key, "user_id", "5") + + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=1000.0, + ) + + self.assertNotIn(client_key, self.redis.store) + self.assertNotIn(session_key, self.redis.store) + self.assertNotIn(pool_key, self.redis.store) + + def test_scheduled_grace_unregisters_after_settle_when_pool_only_released(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, mapping={"busy": "0", "last_activity": "2000.0"}) + + with patch.object(views, "_spawn_background_task", lambda func: func()), \ + patch.object(views.time, "sleep", lambda _s: None), \ + patch.object(views, "_unregister_stats_client") as unregister_mock: + views._schedule_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, + ) + + unregister_mock.assert_called_once_with( + self.redis, self.stats_channel_id, self.client_id, + ) + + def test_schedule_grace_stores_deadline_on_client_metadata(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + + with patch.object(views, "_spawn_background_task", lambda func: None): + views._schedule_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, + ) + + self.assertIn(views._STATS_GRACE_DEADLINE_FIELD, self.redis.store[client_key]) + self.assertIn( + views._STATS_GRACE_DISCONNECTED_AT_FIELD, self.redis.store[client_key], + ) + + def test_complete_expired_stats_grace_runs_after_deadline(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, mapping={ + "last_active": "0.5", + views._STATS_GRACE_DEADLINE_FIELD: "1.0", + views._STATS_GRACE_DISCONNECTED_AT_FIELD: "1.0", + }) + session_key = TimeshiftRedisKeys.api_session(self.client_id) + self.redis.hset(session_key, "user_id", "5") + + views._complete_expired_stats_grace( + self.redis, self.stats_channel_id, self.client_id, + ) + + self.assertNotIn(client_key, self.redis.store) + self.assertNotIn(session_key, self.redis.store) + self.assertNotIn(grace_key, self.redis.store) + + def test_heartbeat_refreshes_idle_pool_and_catchup_session(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + from apps.timeshift.sessions import SESSION_IDLE_TTL_SECONDS + + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, mapping={ + "last_active": "1000.0", + "programme_vid": self.virtual_channel_id, + }) + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, mapping={"busy": "0", "last_activity": "1.0"}) + session_key = TimeshiftRedisKeys.api_session(self.client_id) + self.redis.hset(session_key, "user_id", "5") + self.redis.expire(session_key, 60) + gen_key = views._stream_generation_key( + self.virtual_channel_id, self.client_id, + ) + self.redis.set(gen_key, "1") + self.redis.expire(gen_key, 60) + + views._heartbeat_stats_client( + self.redis, self.stats_channel_id, self.client_id, + bytes_delta=1024, + pool_session_id=self.client_id, + programme_vid=self.virtual_channel_id, + ) + + self.assertGreater( + float(self.redis.hget(pool_key, "last_activity")), 1.0, + ) + self.assertEqual( + self.redis.ttl.get(session_key), SESSION_IDLE_TTL_SECONDS, + ) + self.assertEqual( + self.redis.ttl.get(gen_key), views._POOL_ENTRY_TTL, + ) + + def test_grace_unregister_skipped_when_pool_session_busy(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + grace_key = views._stats_disconnect_grace_key( + self.stats_channel_id, self.client_id, + ) + token = "grace-token" + self.redis.setex(grace_key, 10, token) + client_key = RedisKeys.client_metadata( + self.stats_channel_id, self.client_id, + ) + self.redis.hset(client_key, "last_active", "1000.0") + pool_key = views._pool_key(self.client_id) + self.redis.hset(pool_key, "busy", "1") + + with patch.object(views, "_unregister_stats_client") as unregister_mock: + views._run_stats_disconnect_grace( + self.redis, self.stats_channel_id, self.client_id, token, + disconnected_at=2000.0, + ) + + unregister_mock.assert_not_called() + self.assertNotIn(grace_key, self.redis.store) + + +class TimeshiftUpstreamStopTests(TestCase): + def test_iter_upstream_with_stop_honors_stop_before_read(self): + redis = _FakeRedis() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + upstream = MagicMock() + upstream.raw.read = MagicMock(side_effect=[b"a", b"b", b"c", b""]) + + stop_key = RedisKeys.client_stop("1_test_111", "client_1") + chunks = list( + views._iter_upstream_with_stop( + upstream, 1, redis, stop_key, stream_generation=1, peek_data=b"p", + ) + ) + self.assertEqual(chunks, [b"p", b"a", b"b", b"c"]) + self.assertEqual(upstream.raw.read.call_count, 4) + + redis.setex(stop_key, 60, "1") + upstream.raw.read.reset_mock() + chunks = list( + views._iter_upstream_with_stop( + upstream, 1, redis, stop_key, stream_generation=1, peek_data=b"p", + ) + ) + self.assertEqual(chunks, []) + upstream.raw.read.assert_not_called() + upstream.close.assert_called() + + def test_iter_upstream_with_stop_honors_admin_stop(self): + redis = _FakeRedis() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + upstream = MagicMock() + upstream.raw.read = MagicMock(return_value=b"chunk") + + stop_key = RedisKeys.client_stop("1_test_111", "client_1") + redis.setex(stop_key, 60, views._STOP_REASON_ADMIN) + chunks = list( + views._iter_upstream_with_stop( + upstream, 1, redis, stop_key, stream_generation=1, + ) + ) + self.assertEqual(chunks, []) + upstream.raw.read.assert_not_called() + upstream.close.assert_called_once() + + def test_iter_upstream_with_stop_honors_limit_stop(self): + redis = _FakeRedis() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + upstream = MagicMock() + upstream.raw.read = MagicMock(return_value=b"chunk") + + stop_key = RedisKeys.client_stop("1_test_111", "client_1") + redis.setex(stop_key, 60, views._STOP_REASON_LIMIT) + chunks = list( + views._iter_upstream_with_stop( + upstream, 1, redis, stop_key, stream_generation=99, + ) + ) + self.assertEqual(chunks, []) + upstream.raw.read.assert_not_called() + upstream.close.assert_called_once() + + def test_successor_stream_ignores_preempt_stop_for_prior_generation(self): + redis = _FakeRedis() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + upstream = MagicMock() + upstream.raw.read = MagicMock(side_effect=[b"a", b"b", b""]) + + stop_key = RedisKeys.client_stop("1_test_111", "client_1") + redis.setex(stop_key, 60, "1") + + chunks = list( + views._iter_upstream_with_stop( + upstream, 1, redis, stop_key, stream_generation=2, peek_data=b"p", + ) + ) + self.assertEqual(chunks, [b"p", b"a", b"b"]) + upstream.close.assert_not_called() + + def test_iter_upstream_with_stop_retries_after_read_timeout(self): + redis = _FakeRedis() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + upstream = MagicMock() + upstream.raw.read = MagicMock( + side_effect=[ + requests.exceptions.ReadTimeout("timed out"), + b"abc", + b"", + ], + ) + + stop_key = RedisKeys.client_stop("1_test_111", "client_1") + chunks = list( + views._iter_upstream_with_stop( + upstream, 3, redis, stop_key, stream_generation=1, + ) + ) + self.assertEqual(chunks, [b"abc"]) + self.assertEqual(upstream.raw.read.call_count, 3) + + def test_iter_upstream_with_stop_closes_on_upstream_inactivity(self): + redis = _FakeRedis() + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + upstream = MagicMock() + upstream.raw.read = MagicMock( + side_effect=requests.exceptions.ReadTimeout("timed out"), + ) + stop_key = RedisKeys.client_stop("1_test_111", "client_1") + times = iter([100.0, 100.0, 100.0, 111.0]) + + with patch.object(views.time, "time", side_effect=lambda: next(times)): + chunks = list( + views._iter_upstream_with_stop( + upstream, 3, redis, stop_key, stream_generation=1, + inactivity_timeout=10, + ) + ) + + self.assertEqual(chunks, []) + upstream.close.assert_called() + class TimeshiftScrubPreemptTests(TestCase): """Scrub/range requests must stop the in-flight stream and reuse the pooled @@ -2064,24 +3729,33 @@ class TimeshiftScrubPreemptTests(TestCase): } def test_preempt_stops_sibling_clients_of_same_playback(self): + from apps.timeshift.redis_keys import TimeshiftRedisKeys as RedisKeys, TimeshiftRedisKeys + + stats_channel_id = views.stats_channel_id(8, TEST_SESSION_ID) + programme_vid = f"{TEST_MEDIA_ID}_111" + client_key = RedisKeys.client_metadata(stats_channel_id, TEST_SESSION_ID) + self.redis.hset(client_key, "programme_vid", programme_vid) + connections = [ - self._conn(f"{TEST_MEDIA_ID}_111", TEST_SESSION_ID), - self._conn("timeshift_9_2026-06-08-17-00_222", "timeshift_other"), + self._conn(stats_channel_id, TEST_SESSION_ID), + self._conn("9_2026-06-08-17-00_222", "other"), ] with patch.object(views, "get_user_active_connections", return_value=connections), \ patch.object(views, "_unregister_stats_client") as unregister_mock: + views._allocate_stream_generation(self.redis, programme_vid, TEST_SESSION_ID) views._preempt_playback_streams(self.redis, TEST_SESSION_ID, self.user) - unregister_mock.assert_called_once_with( - self.redis, f"{TEST_MEDIA_ID}_111", TEST_SESSION_ID, - ) - from apps.proxy.live_proxy.redis_keys import RedisKeys - stop_key = RedisKeys.client_stop(f"{TEST_MEDIA_ID}_111", TEST_SESSION_ID) + unregister_mock.assert_not_called() + stop_key = RedisKeys.client_stop(programme_vid, TEST_SESSION_ID) self.assertIn(stop_key, self.redis.store) + stop_value = self.redis.get(stop_key) + if isinstance(stop_value, bytes): + stop_value = stop_value.decode() + self.assertEqual(stop_value, "1") def test_preempt_leaves_other_playbacks_alone(self): connections = [ - self._conn("timeshift_8_2026-06-09-20-00_111", "timeshift_other_pos"), + self._conn("8_2026-06-09-20-00_111", "other_pos"), ] with patch.object(views, "get_user_active_connections", return_value=connections), \ @@ -2089,13 +3763,66 @@ class TimeshiftScrubPreemptTests(TestCase): views._preempt_playback_streams(self.redis, TEST_SESSION_ID, self.user) unregister_mock.assert_not_called() - def test_busy_pool_returns_503_instead_of_second_provider_connection(self): + def test_fresh_session_id_adopts_busy_pool_for_preempt(self): + existing = TEST_SESSION_ID + _seed_pool_session( + self.redis, + session_id=existing, + user_id=5, + client_ip="1.2.3.4", + client_user_agent="test-agent", + busy="1", + ) + request = self.factory.get( + _proxy_url("brandnewsession"), + HTTP_RANGE="bytes=1000-", + HTTP_USER_AGENT="test-agent", + REMOTE_ADDR="1.2.3.4", + ) + streams = [_make_catchup_stream(account_id=1, stream_id="111", profile_id=31)] + ok = MagicMock(status_code=206) + profile = MagicMock(id=31) + descriptor = {"account_id": "1", "stream_id": "111", "profile_id": "31"} + with patch.object(views, "_authenticate_user", return_value=self.user), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "get_channel_catchup_streams", return_value=streams), \ + patch.object(views, "get_programme_duration", return_value=40), \ + patch.object(views, "check_user_stream_limits", return_value=True), \ + patch.object(views, "RedisClient") as redis_cls, \ + patch.object(views, "reserve_profile_slot", return_value=(True, 1, None)), \ + patch.object(views, "release_profile_slot"), \ + patch.object(views, "get_transformed_credentials", side_effect=_fake_creds), \ + patch.object(views, "get_user_active_connections", return_value=[]), \ + patch.object( + views, "_try_reacquire_idle_pool", + return_value=(descriptor, profile), + ) as reacquire_mock, \ + patch.object(views, "_stream_reused_session", return_value=ok): + redis_cls.get_client.return_value = self.redis + channel_cls.objects.get.return_value = MagicMock( + id=8, name="Test", logo_id=None, + ) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-00", "8.ts", + ) + self.assertIs(response, ok) + reacquire_mock.assert_called_once_with( + self.redis, existing, user_id=5, user=self.user, + wait_seconds=views._POOL_SCRUB_WAIT_SECONDS, + ) + + def test_busy_pool_reuses_slot_after_scrub_preempt(self): _seed_pool_session(self.redis, session_id=TEST_SESSION_ID) request = self.factory.get( _proxy_url(TEST_SESSION_ID), HTTP_RANGE="bytes=1000-", ) streams = [_make_catchup_stream(account_id=1, stream_id="111", profile_id=31)] + ok = MagicMock(status_code=206) + profile = MagicMock(id=31) + descriptor = {"account_id": "1", "stream_id": "111", "profile_id": "31"} with patch.object(views, "_authenticate_user", return_value=MagicMock(id=5)), \ patch.object(views, "network_access_allowed", return_value=True), \ patch.object(views, "Channel") as channel_cls, \ @@ -2108,9 +3835,13 @@ class TimeshiftScrubPreemptTests(TestCase): patch.object(views, "release_profile_slot"), \ patch.object(views, "get_transformed_credentials", side_effect=_fake_creds), \ patch.object(views, "get_user_active_connections", return_value=[]), \ - patch.object(views, "_preempt_playback_streams") as preempt_mock, \ - patch.object(views, "_wait_for_idle_pool_session", return_value=None), \ - patch.object(views, "_attempt_timeshift_stream") as attempt_mock: + patch.object( + views, "_try_reacquire_idle_pool", + return_value=(descriptor, profile), + ) as reacquire_mock, \ + patch.object(views, "_stream_reused_session", return_value=ok) as reuse_mock, \ + patch.object(views, "_attempt_timeshift_stream", return_value=ok) as attempt_mock, \ + patch.object(views, "_force_abandon_busy_pool") as abandon_mock: redis_cls.get_client.return_value = self.redis channel_cls.objects.get.return_value = MagicMock( id=8, name="Test", logo_id=None, @@ -2118,13 +3849,60 @@ class TimeshiftScrubPreemptTests(TestCase): response = views.timeshift_proxy( request, "u", "p", "8", "2026-06-08:17-00", "8.ts", ) - self.assertEqual(response.status_code, 503) - preempt_mock.assert_called_once() + self.assertIs(response, ok) + reacquire_mock.assert_called_once() + reuse_mock.assert_called_once() + attempt_mock.assert_not_called() + abandon_mock.assert_not_called() + self.assertTrue(self.redis.exists(f"timeshift:pool:{TEST_SESSION_ID}")) + + def test_stale_idle_pool_scrub_reuses_without_reserve(self): + """Stale busy=0 while a stream is active must not reserve again.""" + _seed_pool_session( + self.redis, session_id=TEST_SESSION_ID, busy="0", serving_range="start", + ) + stats_channel_id = views.stats_channel_id(8, TEST_SESSION_ID) + request = self.factory.get( + _proxy_url(TEST_SESSION_ID), + HTTP_RANGE="bytes=1000-", + ) + streams = [_make_catchup_stream(account_id=1, stream_id="111", profile_id=31)] + ok = MagicMock(status_code=206) + profile = MagicMock(id=31) + descriptor = {"account_id": "1", "stream_id": "111", "profile_id": "31"} + active_conn = self._conn(stats_channel_id, TEST_SESSION_ID) + with patch.object(views, "_authenticate_user", return_value=MagicMock(id=5)), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "get_channel_catchup_streams", return_value=streams), \ + patch.object(views, "get_programme_duration", return_value=40), \ + patch.object(views, "check_user_stream_limits", return_value=True), \ + patch.object(views, "RedisClient") as redis_cls, \ + patch.object(views, "reserve_profile_slot", return_value=(True, 1, None)) as reserve_mock, \ + patch.object(views, "release_profile_slot"), \ + patch.object(views, "get_transformed_credentials", side_effect=_fake_creds), \ + patch.object(views, "get_user_active_connections", return_value=[active_conn]), \ + patch.object( + views, "_try_reacquire_idle_pool", + return_value=(descriptor, profile), + ) as reacquire_mock, \ + patch.object(views, "_stream_reused_session", return_value=ok), \ + patch.object(views, "_attempt_timeshift_stream", return_value=ok) as attempt_mock: + redis_cls.get_client.return_value = self.redis + channel_cls.objects.get.return_value = MagicMock( + id=8, name="Test", logo_id=None, + ) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-00", "8.ts", + ) + self.assertIs(response, ok) + reacquire_mock.assert_called_once() + reserve_mock.assert_not_called() attempt_mock.assert_not_called() - self.assertEqual(len(self._pool_entry_ids()), 1) def _pool_entry_ids(self): - return [k for k in self.redis.store if k.startswith("timeshift_pool:")] + return [k for k in self.redis.store if k.startswith("timeshift:pool:")] def test_startup_bytes_zero_deferred_without_preempt(self): _seed_pool_session( @@ -2192,6 +3970,89 @@ class TimeshiftScrubPreemptTests(TestCase): preempt_mock.assert_not_called() attempt_mock.assert_not_called() + def test_scrub_opens_failover_when_pool_still_busy(self): + _seed_pool_session(self.redis, session_id=TEST_SESSION_ID) + request = self.factory.get( + _proxy_url(TEST_SESSION_ID), + HTTP_RANGE="bytes=1000-", + ) + streams = [_make_catchup_stream(account_id=1, stream_id="111", profile_id=31)] + ok = MagicMock(status_code=206) + with patch.object(views, "_authenticate_user", return_value=MagicMock(id=5)), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "get_channel_catchup_streams", return_value=streams), \ + patch.object(views, "get_programme_duration", return_value=40), \ + patch.object(views, "check_user_stream_limits", return_value=True), \ + patch.object(views, "RedisClient") as redis_cls, \ + patch.object(views, "reserve_profile_slot", return_value=(True, 1, None)), \ + patch.object(views, "release_profile_slot"), \ + patch.object(views, "get_transformed_credentials", side_effect=_fake_creds), \ + patch.object(views, "get_user_active_connections", return_value=[]), \ + patch.object(views, "_try_reacquire_idle_pool", return_value=None) as reacquire_mock, \ + patch.object(views, "_attempt_timeshift_stream", return_value=ok) as attempt_mock: + redis_cls.get_client.return_value = self.redis + channel_cls.objects.get.return_value = MagicMock( + id=8, name="Test", logo_id=None, + ) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-00", "8.ts", + ) + self.assertIs(response, ok) + reacquire_mock.assert_called_once() + attempt_mock.assert_called_once() + + def test_preempt_closes_registered_upstream(self): + stats_channel_id = views.stats_channel_id(8, TEST_SESSION_ID) + upstream = MagicMock() + views._register_active_upstream(stats_channel_id, TEST_SESSION_ID, upstream) + connections = [{ + "media_id": stats_channel_id, + "client_id": TEST_SESSION_ID, + "type": "timeshift", + }] + with patch.object(views, "get_user_active_connections", return_value=connections), \ + patch.object(views, "_set_client_stop") as stop_mock: + views._preempt_playback_streams(self.redis, TEST_SESSION_ID, MagicMock(id=5)) + stop_mock.assert_called_once() + upstream.close.assert_called_once() + + def test_superseded_displaced_release_is_noop(self): + _seed_pool_session(self.redis, session_id=TEST_SESSION_ID) + self.redis.set(views._superseded_pool_key(TEST_SESSION_ID), "1") + with patch.object(views, "release_profile_slot") as release_mock: + views._release_pool_session( + self.redis, TEST_SESSION_ID, 31, release_profile=False, + ) + release_mock.assert_not_called() + self.assertEqual( + self.redis.hget(f"timeshift:pool:{TEST_SESSION_ID}", "busy"), + "1", + ) + + def test_force_abandon_marks_superseded_and_discards_pool(self): + _seed_pool_session(self.redis, session_id=TEST_SESSION_ID) + with patch.object(views, "release_profile_slot") as release_mock: + views._force_abandon_busy_pool(self.redis, TEST_SESSION_ID, 31) + self.assertIn( + views._superseded_pool_key(TEST_SESSION_ID), self.redis.store, + ) + self.assertNotIn(f"timeshift:pool:{TEST_SESSION_ID}", self.redis.store) + release_mock.assert_called_once_with(31, self.redis) + + def test_superseded_final_release_clears_marker_and_frees_profile(self): + _seed_pool_session(self.redis, session_id=TEST_SESSION_ID) + self.redis.set(views._superseded_pool_key(TEST_SESSION_ID), "1") + with patch.object(views, "release_profile_slot") as release_mock: + views._release_pool_session( + self.redis, TEST_SESSION_ID, 31, release_profile=True, + ) + release_mock.assert_called_once_with(31, self.redis) + self.assertNotIn( + views._superseded_pool_key(TEST_SESSION_ID), self.redis.store, + ) + def test_create_pool_session_rejects_duplicate_entry(self): first = views._create_pool_session( self.redis, @@ -2203,6 +4064,7 @@ class TimeshiftScrubPreemptTests(TestCase): account_id=1, profile_id=31, stream_id="111", + dispatcharr_stream_id=10, provider_timestamp="2026", ) second = views._create_pool_session( @@ -2215,11 +4077,12 @@ class TimeshiftScrubPreemptTests(TestCase): account_id=2, profile_id=41, stream_id="222", + dispatcharr_stream_id=20, provider_timestamp="2026", ) self.assertTrue(first) self.assertFalse(second) - self.assertTrue(self.redis.exists(f"timeshift_pool:{TEST_SESSION_ID}")) + self.assertTrue(self.redis.exists(f"timeshift:pool:{TEST_SESSION_ID}")) def test_create_pool_session_stores_provider_tz_name(self): views._create_pool_session( @@ -2232,13 +4095,18 @@ class TimeshiftScrubPreemptTests(TestCase): account_id=1, profile_id=31, stream_id="111", + dispatcharr_stream_id=10, provider_timestamp="2026-06-08:19-00", provider_tz_name="Europe/Brussels", ) self.assertEqual( - self.redis.hget(f"timeshift_pool:{TEST_SESSION_ID}", "provider_tz_name"), + self.redis.hget(f"timeshift:pool:{TEST_SESSION_ID}", "provider_tz_name"), "Europe/Brussels", ) + self.assertEqual( + self.redis.hget(f"timeshift:pool:{TEST_SESSION_ID}", "dispatcharr_stream_id"), + "10", + ) def test_scrub_reuses_idle_pool_without_opening_failover(self): _seed_pool_session(self.redis, session_id=TEST_SESSION_ID) @@ -2285,6 +4153,134 @@ class TimeshiftScrubPreemptTests(TestCase): reserve_mock.assert_called_once_with(profile, self.redis) +class CatchupProxyTests(TestCase): + """Native ``/proxy/catchup/{uuid}`` entry point.""" + + def setUp(self): + self.factory = APIRequestFactory() + self.user = MagicMock(id=1, user_level=10, is_authenticated=True) + self.channel_uuid = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + + def test_requires_authentication(self): + request = self.factory.get( + f"/proxy/catchup/{self.channel_uuid}?start=2026-06-08T17:00:00Z", + ) + with patch.object(views, "network_access_allowed", return_value=True): + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 401) + + def test_missing_start_returns_400(self): + request = self.factory.get(f"/proxy/catchup/{self.channel_uuid}") + force_authenticate(request, user=self.user) + channel = MagicMock(id=8, uuid=self.channel_uuid) + with patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True): + channel_cls.objects.get.return_value = channel + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 400) + self.assertIn(b"Missing start", response.content) + + def test_missing_session_id_redirects(self): + request = self.factory.get( + f"/proxy/catchup/{self.channel_uuid}?start=2026-06-08T17:00:00Z", + ) + force_authenticate(request, user=self.user) + channel = MagicMock(id=8, uuid=self.channel_uuid) + with patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "get_channel_catchup_streams", + return_value=[_make_catchup_stream()]), \ + patch.object(views, "get_programme_duration", return_value=40), \ + patch.object(views, "parse_catchup_timestamp", return_value=True), \ + patch.object(views, "RedisClient") as redis_cls: + redis_cls.get_client.return_value = _FakeRedis() + channel_cls.objects.get.return_value = channel + response = views.catchup_proxy(request, self.channel_uuid) + self.assertEqual(response.status_code, 301) + self.assertIn("session_id=", response["Location"]) + self.assertIn("start=", response["Location"]) + + def test_xc_entry_delegates_to_serve_catchup(self): + request = self.factory.get(_proxy_url()) + with patch.object(views, "_authenticate_user", return_value=MagicMock(id=1)), \ + patch.object(views, "network_access_allowed", return_value=True), \ + patch.object(views, "Channel") as channel_cls, \ + patch.object(views, "_user_can_access_channel", return_value=True), \ + patch.object(views, "_serve_catchup", return_value=HttpResponse("ok")) as serve: + channel_cls.objects.get.return_value = MagicMock(id=8) + response = views.timeshift_proxy( + request, "u", "p", "8", "2026-06-08:17-00", "8.ts", + ) + self.assertEqual(response.status_code, 200) + serve.assert_called_once() + + +class TimeshiftDownstreamLengthHeaderTests(TestCase): + def test_open_ended_range_passthrough_upstream_headers(self): + headers = views._build_downstream_length_headers( + range_header="bytes=1000-", + status_code=206, + representation_length=10000, + upstream_content_range="bytes 1000-2047/10000", + upstream_content_length="1048", + ) + self.assertEqual(headers["Content-Range"], "bytes 1000-2047/10000") + self.assertEqual(headers["Content-Length"], "1048") + self.assertEqual(headers["Accept-Ranges"], "bytes") + + def test_closed_range_passthrough_upstream_headers(self): + headers = views._build_downstream_length_headers( + range_header="bytes=1000-4999", + status_code=206, + representation_length=10000, + upstream_content_range="bytes 1000-4999/10000", + upstream_content_length="4000", + ) + self.assertEqual(headers["Content-Range"], "bytes 1000-4999/10000") + self.assertEqual(headers["Content-Length"], "4000") + + def test_206_synthesizes_range_when_upstream_omits_it(self): + headers = views._build_downstream_length_headers( + range_header="bytes=1000-", + status_code=206, + representation_length=10000, + upstream_content_range=None, + upstream_content_length="1048", + ) + self.assertEqual(headers["Content-Range"], "bytes 1000-9999/10000") + self.assertEqual(headers["Content-Length"], "1048") + + def test_full_file_response_sets_content_length_when_not_streaming(self): + headers = views._build_downstream_length_headers( + range_header=None, + status_code=200, + representation_length=10000, + upstream_content_range=None, + upstream_content_length="10000", + ) + self.assertEqual(headers["Content-Length"], "10000") + self.assertNotIn("Content-Range", headers) + + def test_streaming_full_file_omits_content_length(self): + headers = views._build_downstream_length_headers( + range_header=None, + status_code=200, + representation_length=10000, + upstream_content_range=None, + upstream_content_length="10000", + streaming=True, + ) + self.assertNotIn("Content-Length", headers) + self.assertEqual(headers["Accept-Ranges"], "bytes") + + def test_passthrough_includes_accept_ranges(self): + response = views._passthrough_response(416, "bytes */10000") + self.assertEqual(response["Content-Range"], "bytes */10000") + self.assertEqual(response["Accept-Ranges"], "bytes") + + class RollupSelfHealDbTests(TestCase): """Catch-up flag consistency after stream removal. diff --git a/apps/timeshift/urls.py b/apps/timeshift/urls.py new file mode 100644 index 00000000..056c9a68 --- /dev/null +++ b/apps/timeshift/urls.py @@ -0,0 +1,12 @@ +from django.urls import path + +from . import stats_views, views + +app_name = "catchup" + +urlpatterns = [ + path("stats/", stats_views.timeshift_stats, name="catchup_stats"), + path("programs/", stats_views.catchup_programmes, name="catchup_programmes"), + path("stop_client/", stats_views.stop_timeshift_session, name="catchup_stop_client"), + path("", views.catchup_proxy, name="catchup"), +] diff --git a/apps/timeshift/views.py b/apps/timeshift/views.py index b34cf379..84156287 100644 --- a/apps/timeshift/views.py +++ b/apps/timeshift/views.py @@ -1,9 +1,10 @@ -"""XC catch-up (timeshift) proxy with multi-provider failover.""" +"""Catch-up (timeshift) proxy with multi-provider failover.""" import hmac -import itertools +import json import logging import secrets +import threading import time from urllib.parse import urlencode @@ -16,9 +17,16 @@ from django.http import ( HttpResponseBadRequest, HttpResponseForbidden, HttpResponseNotFound, + JsonResponse, StreamingHttpResponse, ) +from rest_framework.decorators import api_view, authentication_classes, permission_classes +from rest_framework.permissions import AllowAny +from rest_framework_simplejwt.authentication import JWTAuthentication +from drf_spectacular.types import OpenApiTypes +from drf_spectacular.utils import OpenApiParameter, extend_schema +from apps.accounts.authentication import ApiKeyAuthentication, QueryParamJWTAuthentication from apps.accounts.models import User from apps.channels.models import Channel from apps.channels.utils import get_channel_catchup_streams @@ -27,14 +35,23 @@ from apps.m3u.models import M3UAccount, M3UAccountProfile from apps.m3u.tasks import get_transformed_credentials from apps.proxy.live_proxy.config_helper import ConfigHelper from apps.proxy.live_proxy.constants import ChannelMetadataField, ChannelState -from apps.proxy.live_proxy.redis_keys import RedisKeys +from apps.timeshift.redis_keys import ( + TimeshiftRedisKeys, + mint_session_id, + programme_media_id, + stats_channel_id as make_stats_channel_id, + virtual_channel_id as make_virtual_channel_id, +) + +stats_channel_id = make_stats_channel_id from apps.proxy.live_proxy.utils import get_client_ip from apps.proxy.utils import ( + _timeshift_stop_channel_id, check_user_stream_limits, find_ts_sync, get_user_active_connections, ) -from core.utils import RedisClient +from core.utils import RedisClient, _is_gevent_monkey_patched from dispatcharr.utils import network_access_allowed from .helpers import ( @@ -44,11 +61,32 @@ from .helpers import ( get_programme_duration, parse_catchup_timestamp, ) +from .sessions import catchup_session_exists, delete_catchup_session, resolve_catchup_playback +from .stats import resolve_stats_playback_fields, seed_stream_stats_metadata logger = logging.getLogger(__name__) CLIENT_TTL_SECONDS = 60 +_STATS_DISCONNECT_GRACE_SECONDS = 5 # VOD-style grace before dropping stats on disconnect. +_STATS_RECONNECT_SETTLE_SECONDS = 1.5 # Brief pause before arming grace (XC reconnects fast). +_STATS_GRACE_MIN_YIELDED_BYTES = 65536 # Ignore parallel startup probes that die immediately. +_STATS_GRACE_MIN_ELAPSED_SECONDS = 2.0 +# Atomically drop the grace key only when its value still matches *token*. +_CLAIM_STATS_GRACE_LUA = """ +if redis.call('get', KEYS[1]) == ARGV[1] then + return redis.call('del', KEYS[1]) +else + return 0 +end +""" _MATCH_SCORE_THRESHOLD = 8 # client_ip (5) + client_user_agent (3) +_STOP_REASON_REUSE = "reuse" +_STOP_REASON_HOP = "hop" +_STOP_REASON_ADMIN = "admin" +_STOP_REASON_LIMIT = "limit" +_TERMINAL_STOP_REASONS = frozenset({ + _STOP_REASON_HOP, _STOP_REASON_ADMIN, _STOP_REASON_LIMIT, +}) def _finalize_timeshift_response(response): @@ -84,7 +122,125 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) if not _user_can_access_channel(user, channel): return _finalize_timeshift_response(HttpResponseForbidden("Access denied")) - # Shape helpers pass through on parse failure; reject bad input before upstream. + return _serve_catchup(request, user, channel, timestamp) + + +@extend_schema( + description=( + "Stream catch-up (TV archive) MPEG-TS for a channel.\n\n" + "**Recommended (native apps):** call ``POST /api/catchup/sessions/`` " + "with a JWT or API key to obtain a ``playback_url`` containing " + "``session_id`` only. Pass that URL to the video player without an " + "``?token=`` required. The session binds the programme ``start`` " + "time server-side.\n\n" + "**Legacy / direct auth:** supply ``start`` (programme UTC start " + "time) plus ``Authorization: Bearer``, ``X-API-Key``, or " + "``?token=``. Optionally include a client ``session_id`` for " + "provider pooling.\n\n" + "If ``session_id`` is omitted on a direct-auth request, the server " + "responds with **301** adding a server-minted ``session_id``." + ), + parameters=[ + OpenApiParameter( + name="session_id", + type=OpenApiTypes.STR, + location=OpenApiParameter.QUERY, + required=False, + description=( + "Playback session from ``POST /api/catchup/sessions/``. " + "When present, JWT is optional. The session authenticates " + "the player. Reuse for all range/seek requests during the " + "same programme." + ), + ), + OpenApiParameter( + name="start", + type=OpenApiTypes.STR, + location=OpenApiParameter.QUERY, + required=False, + description=( + "Programme start time in UTC. Required for direct-auth " + "playback (no API session). Ignored when ``session_id`` was " + "issued by ``POST /api/catchup/sessions/``." + ), + ), + OpenApiParameter( + name="token", + type=OpenApiTypes.STR, + location=OpenApiParameter.QUERY, + required=False, + description=( + "JWT access token when Authorization headers are unavailable. " + "Not needed when using an API playback session." + ), + ), + ], + responses={ + 200: {"description": "MPEG-TS stream (may be partial content for Range requests)."}, + 301: { + "description": ( + "Redirect to the same URL with a server-minted ``session_id`` " + "when the client did not supply one (direct-auth flow only)." + ), + }, + 401: {"description": "Missing or expired authentication / session."}, + 403: {"description": "Access denied or session/channel mismatch."}, + }, + tags=["proxy"], +) +@api_view(["GET"]) +@authentication_classes([JWTAuthentication, ApiKeyAuthentication, QueryParamJWTAuthentication]) +@permission_classes([AllowAny]) +def catchup_proxy(request, channel_id): + """Native API catch-up playback for a channel.""" + if not network_access_allowed(request, "STREAMS"): + return JsonResponse({"error": "Forbidden"}, status=403) + + auth_user = ( + request.user + if getattr(request, "user", None) and request.user.is_authenticated + else None + ) + + session_id = request.GET.get("session_id") + timestamp = request.GET.get("start") + user = auth_user + + if session_id: + resolved = resolve_catchup_playback(session_id, channel_id) + if resolved is None: + if auth_user is None: + return JsonResponse( + {"error": "Invalid or expired playback session"}, + status=401, + ) + else: + session_user, bound_start = resolved + if auth_user is not None and auth_user.id != session_user.id: + return _finalize_timeshift_response(HttpResponseForbidden("Access denied")) + user = session_user + timestamp = bound_start + + if user is None: + return JsonResponse({"error": "Authentication required"}, status=401) + + try: + channel = Channel.objects.get(uuid=channel_id) + except Channel.DoesNotExist: + close_old_connections() + raise Http404("Channel not found") from None + + if not _user_can_access_channel(user, channel): + return _finalize_timeshift_response(HttpResponseForbidden("Access denied")) + + if not timestamp: + return _finalize_timeshift_response(HttpResponseBadRequest("Missing start parameter")) + + return _serve_catchup(request, user, channel, timestamp) + + +def _serve_catchup(request, user, channel, timestamp): + """Shared catch-up proxy logic for XC and native API entry points.""" if parse_catchup_timestamp(timestamp) is None: return _finalize_timeshift_response(HttpResponseBadRequest("Invalid timestamp")) @@ -107,13 +263,28 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) redis_client = RedisClient.get_client() - # Content identity (channel + catch-up position). Provider slot sharing is - # scoped per client session; never assume all requests for the same - # programme belong to one viewer. - media_id = f"timeshift_{channel.id}_{safe_ts}" + # One provider slot per session_id, not per programme. + media_id = programme_media_id(channel.id, safe_ts) session_id = request.GET.get("session_id") if not session_id: + matched = _find_matching_pool_session( + redis_client, + media_id=media_id, + channel_id=channel.id, + user_id=user.id, + client_ip=client_ip, + client_user_agent=client_user_agent, + include_busy=True, + ) + if matched: + logger.debug( + "Timeshift session redirect: reusing %s for %s", + matched, request.path, + ) + return _finalize_timeshift_response( + _redirect_with_session(request, matched), + ) logger.debug("Timeshift session redirect: %s (new session)", request.path) return _finalize_timeshift_response(_redirect_with_new_session(request)) @@ -128,19 +299,20 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) effective_session_id = session_id client_id = session_id - # Reuse an idle pool owned by this session, or fingerprint-match a prior - # idle session from the same client (VOD-style) before opening upstream. + # Reuse idle pool or fingerprint-match when XC drops ?session_id= on seek. if not session_entry: - matched = _find_matching_idle_session( + matched = _find_matching_pool_session( redis_client, media_id=media_id, + channel_id=channel.id, user_id=user.id, client_ip=client_ip, client_user_agent=client_user_agent, + include_busy=True, ) if matched: logger.info( - "Timeshift fingerprint matched idle session %s for %s", + "Timeshift fingerprint matched session %s for %s", matched, session_id, ) effective_session_id = matched @@ -170,6 +342,8 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) if not check_user_stream_limits(user, client_id, media_id=media_id): return _finalize_timeshift_response(HttpResponseForbidden("Stream limit exceeded")) + _touch_stats_on_session_request(redis_client, channel.id, effective_session_id) + if effective_session_id == session_id: pool = _snapshot_from_entry(session_entry) else: @@ -178,24 +352,64 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) pool_busy = pool["busy"] if pool else False pool_content_length = pool["content_length"] if pool else None busy_serving_range = pool["serving_range"] if pool else None + pool_media_id = None + if pool and pool.get("entry"): + pool_media_id = pool["entry"].get("media_id") + elif session_entry: + pool_media_id = session_entry.get("media_id") + scrub_displacement = ( + pool_exists + and _should_displace_busy_pool( + range_header, + pool_content_length, + busy_serving_range, + pool_media_id=pool_media_id, + media_id=media_id, + ) + ) + # Pool may look idle while the replacement upstream is still active. + if ( + scrub_displacement + and not pool_busy + and _session_has_active_timeshift_stream(user, effective_session_id) + ): + pool_busy = True acquired = None if pool_exists: - if pool_busy: - if _should_displace_busy_playback( - range_header, pool_content_length, busy_serving_range, - ): - _preempt_playback_streams(redis_client, effective_session_id, user) - acquired = _wait_for_idle_pool_session( - redis_client, - effective_session_id, - user_id=user.id, - wait_seconds=_POOL_PREEMPT_WAIT_SECONDS, - ) - else: + if not pool_busy: acquired = _acquire_idle_pool_session( redis_client, effective_session_id, user_id=user.id, ) + elif _should_preempt_for_programme_change( + redis_client, effective_session_id, pool_media_id, media_id, + ): + acquired = _try_reacquire_idle_pool( + redis_client, effective_session_id, + user_id=user.id, user=user, + ) + elif scrub_displacement: + acquired = _try_reacquire_idle_pool( + redis_client, effective_session_id, + user_id=user.id, user=user, + wait_seconds=_POOL_SCRUB_WAIT_SECONDS, + ) + if acquired is None: + logger.warning( + "Timeshift: session %s still busy after scrub preempt, opening failover", + effective_session_id, + ) + abandoned_profile_id = None + abandoned_entry = pool["entry"] if pool and pool.get("entry") else session_entry + if abandoned_entry: + abandoned_profile_id = abandoned_entry.get("profile_id") + _force_abandon_busy_pool( + redis_client, + effective_session_id, + abandoned_profile_id, + ) + pool_exists = False + pool_busy = False last_response = None decisive_accounts = set() @@ -214,6 +428,7 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) duration_minutes=duration_minutes, client_id=client_id, client_ip=client_ip, + client_user_agent=client_user_agent, range_header=range_header, channel_logo_id=channel_logo_id, user=user, @@ -231,19 +446,10 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) except (ValueError, TypeError): pass - if pool_exists and pool_busy and not _should_displace_busy_playback( - range_header, pool_content_length, busy_serving_range, - ): + if pool_exists and pool_busy and acquired is None: logger.debug( - "Timeshift: deferring non-displacing request for session %s range=%s", - effective_session_id, range_header or "(none)", - ) - return _finalize_timeshift_response(HttpResponse("Stream slot busy", status=503)) - - if pool_exists and pool_busy: - logger.warning( - "Timeshift: session %s did not become idle in time", - effective_session_id, + "Timeshift: deferring busy session %s range=%s media=%s pool_media=%s", + effective_session_id, range_header or "(none)", media_id, pool_media_id, ) return _finalize_timeshift_response(HttpResponse("Stream slot busy", status=503)) @@ -310,6 +516,7 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) account_id=m3u_account.id, profile_id=reserved_profile.id, stream_id=stream_id_value, + dispatcharr_stream_id=catchup_stream.id, provider_timestamp=provider_timestamp, provider_tz_name=provider_tz_name, ): @@ -344,6 +551,7 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) timestamp=timestamp, client_id=client_id, client_ip=client_ip, + client_user_agent=client_user_agent, range_header=range_header, channel_logo_id=channel_logo_id, user=user, @@ -351,6 +559,8 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) debug=debug, release_cb=release_cb, pool_session_id=effective_session_id, + stats_stream_id=catchup_stream.id, + stream_stats=catchup_stream.stream_stats, ) except Exception: _discard_pool_session(redis_client, effective_session_id, reserved_profile.id) @@ -361,9 +571,7 @@ def timeshift_proxy(request, username, password, stream_id, timestamp, duration) return _finalize_timeshift_response(response) if getattr(response, "timeshift_passthrough", False) is True: - # Terminal range answer (e.g. 416 past EOF): the upstream session is - # healthy, so free the slot but keep the entry idle for the next - # probe, and return verbatim without failing over to other accounts. + # 416 etc.: release slot, keep idle pool, no failover. release_cb() return _finalize_timeshift_response(response) @@ -423,36 +631,482 @@ def _user_can_access_channel(user, channel): ) -# Per-client session pool (keyed by session_id from the 301 redirect). Each -# viewer gets their own provider slot even when watching the same catch-up -# programme. Idle sessions can be fingerprint-matched (VOD-style) when a client -# returns without its prior session_id. -_POOL_KEY = "timeshift_pool:{session_id}" -_POOL_LOCK_KEY = "timeshift_pool_lock:{session_id}" -_POOL_ENTRY_TTL = 6 * 3600 -_POOL_IDLE_TTL = 30 +# Per-client pool (session_id from 301 redirect). Fingerprint-match when ?session_id= is dropped. +_POOL_ENTRY_TTL = 10 * 60 # Refreshed on playback GET and active-stream heartbeats. +_STREAM_READ_INACTIVITY_SECONDS = CLIENT_TTL_SECONDS # Shorter than pool TTL; data should flow. +_POOL_IDLE_TTL = CLIENT_TTL_SECONDS # Must match CLIENT_TTL_SECONDS (stats + fingerprint agree). _POOL_WAIT_SECONDS = 1.0 -_POOL_PREEMPT_WAIT_SECONDS = 5.0 +# Brief wait after scrub preempt before failover. +_POOL_SCRUB_WAIT_SECONDS = 2.0 _POOL_POLL_INTERVAL = 0.05 _EOF_PROBE_TAIL_BYTES = 512_000 _EOF_PROBE_UNKNOWN_LENGTH_MIN = 100_000_000 def _pool_key(session_id): - return _POOL_KEY.format(session_id=session_id) + return TimeshiftRedisKeys.pool(session_id) + + +def _stats_disconnect_grace_key(stats_channel_id, client_id): + return TimeshiftRedisKeys.stats_grace(stats_channel_id, client_id) + + +_STATS_GRACE_DEADLINE_FIELD = "stats_grace_deadline" +_STATS_GRACE_DISCONNECTED_AT_FIELD = "stats_grace_disconnected_at" + + +def _clear_stats_grace_schedule(redis_client, stats_channel_id, client_id): + if redis_client is None: + return + try: + redis_client.delete(_stats_disconnect_grace_key(stats_channel_id, client_id)) + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + redis_client.hdel( + client_key, + _STATS_GRACE_DEADLINE_FIELD, + _STATS_GRACE_DISCONNECTED_AT_FIELD, + ) + except Exception as exc: + logger.debug("Timeshift stats grace cancel failed: %s", exc) + + +def _cancel_stats_disconnect_grace(redis_client, stats_channel_id, client_id): + _clear_stats_grace_schedule(redis_client, stats_channel_id, client_id) + + +def _pool_session_exists(redis_client, session_id): + if redis_client is None or not session_id: + return False + try: + return bool(redis_client.exists(_pool_key(session_id))) + except Exception: + return False + + +def _pool_session_busy(redis_client, session_id): + if redis_client is None or not session_id: + return False + try: + busy = redis_client.hget(_pool_key(session_id), "busy") + except Exception: + return False + if isinstance(busy, bytes): + busy = busy.decode() + return str(busy) == "1" + + +def _should_schedule_stats_disconnect_grace( + total_yielded, + elapsed_secs, + *, + stopped_for_reuse, + redis_client=None, + client_id=None, +): + if stopped_for_reuse: + return False + if _is_timeshift_startup_probe(total_yielded, elapsed_secs): + return False + # XC duplicate requests often return 503 and reset the first connection + # within ~1s while the idle pool entry waits for the retry. + if ( + elapsed_secs < _STATS_GRACE_MIN_ELAPSED_SECONDS + and redis_client is not None + and client_id + and _pool_session_exists(redis_client, client_id) + ): + return False + return True + + +def _is_timeshift_startup_probe(total_yielded, elapsed_secs): + """XC often probes several programme URLs in parallel; losers die quickly.""" + return ( + total_yielded < _STATS_GRACE_MIN_YIELDED_BYTES + and elapsed_secs < _STATS_GRACE_MIN_ELAPSED_SECONDS + ) + + +def _stats_client_reconnected( + redis_client, stats_channel_id, client_id, *, disconnected_at, +): + """Return True when a new playback request superseded this disconnect.""" + if _pool_session_busy(redis_client, client_id): + return True + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + try: + last_active = redis_client.hget(client_key, "last_active") + if last_active: + if isinstance(last_active, bytes): + last_active = last_active.decode() + # Strict ``>``: the disconnect path heartbeats stats immediately + # before arming grace, so equality means no reconnect yet. + if float(last_active) > float(disconnected_at): + return True + except (TypeError, ValueError): + pass + return False + + +def _grace_token_claimed(redis_client, grace_key, token): + if redis_client is None: + return False + try: + current = redis_client.get(grace_key) + if isinstance(current, bytes): + current = current.decode() + return current == token + except Exception: + return False + + +def _complete_expired_stats_grace(redis_client, stats_channel_id, client_id): + """Run disconnect cleanup when the grace deadline has already passed. + + uWSGI workers often cannot keep a daemon thread alive after the streaming + response closes, so heartbeats and reconnect touches also drive this path. + """ + if redis_client is None or not client_id: + return + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + try: + deadline_raw = redis_client.hget(client_key, _STATS_GRACE_DEADLINE_FIELD) + if not deadline_raw: + return + if isinstance(deadline_raw, bytes): + deadline_raw = deadline_raw.decode() + if time.time() < float(deadline_raw): + return + grace_key = _stats_disconnect_grace_key(stats_channel_id, client_id) + token = redis_client.get(grace_key) + if token is None: + redis_client.hdel( + client_key, + _STATS_GRACE_DEADLINE_FIELD, + _STATS_GRACE_DISCONNECTED_AT_FIELD, + ) + return + if isinstance(token, bytes): + token = token.decode() + disconnected_raw = redis_client.hget( + client_key, _STATS_GRACE_DISCONNECTED_AT_FIELD, + ) + if disconnected_raw is None: + disconnected_at = ( + float(deadline_raw) + - _STATS_RECONNECT_SETTLE_SECONDS + - _STATS_DISCONNECT_GRACE_SECONDS + ) + else: + if isinstance(disconnected_raw, bytes): + disconnected_raw = disconnected_raw.decode() + disconnected_at = float(disconnected_raw) + _run_stats_disconnect_grace( + redis_client, stats_channel_id, client_id, token, + disconnected_at=disconnected_at, + ) + except Exception as exc: + logger.debug("Timeshift stats grace finalize failed: %s", exc) + + +def _touch_stats_on_session_request(redis_client, channel_id, session_id): + """Keep an in-flight stats card alive while XC reconnects the same session.""" + if redis_client is None or not session_id: + return + stats_channel_id = make_stats_channel_id(channel_id, session_id) + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, session_id) + try: + if not redis_client.exists(client_key): + return + _cancel_stats_disconnect_grace(redis_client, stats_channel_id, session_id) + _complete_expired_stats_grace(redis_client, stats_channel_id, session_id) + now = str(time.time()) + client_set_key = TimeshiftRedisKeys.clients(stats_channel_id) + metadata_key = TimeshiftRedisKeys.channel_metadata(stats_channel_id) + pipe = redis_client.pipeline(transaction=False) + pipe.hset(client_key, "last_active", now) + pipe.expire(client_key, CLIENT_TTL_SECONDS) + pipe.expire(client_set_key, CLIENT_TTL_SECONDS) + pipe.expire(metadata_key, CLIENT_TTL_SECONDS) + pipe.execute() + programme_vid = redis_client.hget(client_key, "programme_vid") + if isinstance(programme_vid, bytes): + programme_vid = programme_vid.decode() + _refresh_active_session_redis_ttl( + redis_client, session_id, programme_vid, + ) + except Exception as exc: + logger.debug("Timeshift stats touch failed: %s", exc) + + +def _run_stats_disconnect_grace( + redis_client, stats_channel_id, client_id, token, *, disconnected_at, +): + """Drop stats after the grace window unless a reconnect cancelled *token*.""" + if redis_client is None: + return + grace_key = _stats_disconnect_grace_key(stats_channel_id, client_id) + try: + claimed = redis_client.eval( + _CLAIM_STATS_GRACE_LUA, 1, grace_key, token, + ) + if not claimed: + return + if _pool_session_busy(redis_client, client_id): + return + if _stats_client_reconnected( + redis_client, stats_channel_id, client_id, + disconnected_at=disconnected_at, + ): + return + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + programme_vid = redis_client.hget(client_key, "programme_vid") + if isinstance(programme_vid, bytes): + programme_vid = programme_vid.decode() + _unregister_stats_client(redis_client, stats_channel_id, client_id) + _cleanup_all_stream_generations(redis_client, client_id) + api_session_ended = _finalize_playback_session_auth(redis_client, client_id) + if api_session_ended: + _discard_pool_session(redis_client, client_id, None) + else: + # Keep idle pool fingerprint for XC players that drop ?session_id=. + key = _pool_key(client_id) + try: + if redis_client.exists(key): + redis_client.expire(key, _POOL_IDLE_TTL) + except Exception as exc: + logger.debug("Timeshift pool idle TTL refresh failed: %s", exc) + try: + redis_client.delete(_superseded_pool_key(client_id)) + except Exception: + pass + _trigger_timeshift_stats_update(redis_client) + except Exception as exc: + logger.warning("Timeshift delayed stats unregister failed: %s", exc) + + +def _finalize_playback_session_auth(redis_client, session_id): + """Drop tokenless API session auth once viewing has genuinely ended. + + Returns ``True`` when an API session record existed and was deleted. + """ + if redis_client is None or not session_id: + return False + if not catchup_session_exists(session_id, redis_client=redis_client): + return False + delete_catchup_session(session_id, redis_client=redis_client) + return True + + +def _spawn_background_task(func): + """Run *func* on a gevent greenlet (uWSGI) or a non-daemon thread. + + Gated on monkey-patching, not gevent importability: in unpatched + processes (Celery prefork, tests) a spawned greenlet may never be + scheduled because nothing yields to the hub. + """ + if _is_gevent_monkey_patched(): + import gevent + gevent.spawn(func) + return + thread = threading.Thread(target=func, daemon=False) + thread.start() + + +def _schedule_stats_disconnect_grace(redis_client, stats_channel_id, client_id): + if redis_client is None: + return + token = secrets.token_hex(8) + grace_key = _stats_disconnect_grace_key(stats_channel_id, client_id) + disconnected_at = time.time() + grace_deadline = ( + disconnected_at + + _STATS_RECONNECT_SETTLE_SECONDS + + _STATS_DISCONNECT_GRACE_SECONDS + ) + ttl = int( + _STATS_RECONNECT_SETTLE_SECONDS + _STATS_DISCONNECT_GRACE_SECONDS + 2, + ) + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + try: + pipe = redis_client.pipeline(transaction=False) + pipe.setex(grace_key, ttl, token) + pipe.hset( + client_key, + mapping={ + _STATS_GRACE_DEADLINE_FIELD: str(grace_deadline), + _STATS_GRACE_DISCONNECTED_AT_FIELD: str(disconnected_at), + }, + ) + pipe.execute() + except Exception as exc: + logger.warning("Timeshift stats grace schedule failed: %s", exc) + return + + def _delayed_unregister(): + time.sleep(_STATS_RECONNECT_SETTLE_SECONDS) + if not _grace_token_claimed(redis_client, grace_key, token): + return + if _stats_client_reconnected( + redis_client, stats_channel_id, client_id, + disconnected_at=disconnected_at, + ): + try: + redis_client.delete(grace_key) + except Exception: + pass + return + time.sleep(_STATS_DISCONNECT_GRACE_SECONDS) + _run_stats_disconnect_grace( + redis_client, stats_channel_id, client_id, token, + disconnected_at=disconnected_at, + ) + + _spawn_background_task(_delayed_unregister) + + +def _trigger_timeshift_stats_update(redis_client): + """Push catch-up stats to websocket listeners after real connection changes.""" + if redis_client is None: + return + + def _send(): + try: + from apps.timeshift.stats import build_timeshift_stats_data + from core.utils import send_websocket_update + + stats = build_timeshift_stats_data(redis_client) + send_websocket_update( + "updates", + "update", + { + "success": True, + "type": "timeshift_stats", + "stats": json.dumps(stats), + }, + ) + except Exception as exc: + logger.debug("Failed to trigger timeshift stats update: %s", exc) + + _spawn_background_task(_send) def _parse_range_start(range_header): """Return the byte offset from a ``Range: bytes=START-`` header, or None.""" + parsed = _parse_client_range(range_header) + if parsed is None: + return None + return parsed[0] + + +def _parse_client_range(range_header): + """Return ``(start, end)`` from a client Range header; ``end`` may be None.""" if not range_header or not range_header.startswith("bytes="): return None - start_part = range_header[6:].split("-", 1)[0] - if not start_part: + range_part = range_header[6:] + if "-" not in range_part: return None + start_str, end_str = range_part.split("-", 1) try: - return int(start_part) + start = int(start_str) if start_str else 0 except (TypeError, ValueError): return None + if not end_str: + return start, None + try: + return start, int(end_str) + except (TypeError, ValueError): + return None + + +def _parse_content_range_header(content_range): + """Parse ``Content-Range: bytes START-END/TOTAL``.""" + if not content_range or not content_range.startswith("bytes "): + return None + body = content_range[6:] + if "/" not in body: + return None + range_part, total_part = body.rsplit("/", 1) + if "-" not in range_part: + return None + start_str, end_str = range_part.split("-", 1) + try: + start = int(start_str) if start_str else 0 + end = int(end_str) if end_str else None + total = None if total_part == "*" else int(total_part) + except (TypeError, ValueError): + return None + return {"start": start, "end": end, "total": total} + + +def _extract_representation_length(upstream_response): + """Return the full archived file size from upstream response headers.""" + if upstream_response is None: + return None + parsed = _parse_content_range_header( + upstream_response.headers.get("Content-Range", ""), + ) + if parsed and parsed.get("total") is not None: + return parsed["total"] + content_length = upstream_response.headers.get("Content-Length") + if content_length: + try: + return int(content_length) + except (TypeError, ValueError): + return None + return None + + +def _build_downstream_length_headers( + *, + range_header, + status_code, + representation_length, + upstream_content_range, + upstream_content_length, + streaming=False, +): + """Build RFC 7230/7233 length headers for the downstream IPTV client.""" + headers = {"Accept-Ranges": "bytes"} + parsed_upstream = ( + _parse_content_range_header(upstream_content_range) + if upstream_content_range else None + ) + + if representation_length is None and parsed_upstream: + representation_length = parsed_upstream.get("total") + + if status_code == 206: + # Trust upstream partial headers; peek bytes are forwarded verbatim. + if upstream_content_range: + headers["Content-Range"] = upstream_content_range + elif range_header and representation_length is not None: + client_range = _parse_client_range(range_header) + if client_range: + start, end = client_range + if end is None: + end = representation_length - 1 + else: + end = min(end, representation_length - 1) + headers["Content-Range"] = ( + f"bytes {start}-{end}/{representation_length}" + ) + if upstream_content_length: + headers["Content-Length"] = str(upstream_content_length) + elif parsed_upstream and parsed_upstream.get("end") is not None: + up_start = parsed_upstream["start"] + up_end = parsed_upstream["end"] + headers["Content-Length"] = str(up_end - up_start + 1) + return headers + + # Omit Content-Length on streaming full-file responses (seeks may preempt). + if not streaming: + if representation_length is not None: + headers["Content-Length"] = str(representation_length) + elif upstream_content_length: + headers["Content-Length"] = str(upstream_content_length) + + return headers def _is_near_eof_probe(range_header, content_length=None): @@ -470,6 +1124,56 @@ def _is_near_eof_probe(range_header, content_length=None): return start >= _EOF_PROBE_UNKNOWN_LENGTH_MIN +def _should_displace_busy_pool( + range_header, + content_length, + busy_serving_range, + *, + pool_media_id, + media_id, +): + """True when a busy pooled slot should be recycled for this request.""" + if pool_media_id is not None and str(pool_media_id) != str(media_id): + # Programme hops use _should_preempt_for_programme_change, not displacement. + return False + return _should_displace_busy_playback( + range_header, content_length, busy_serving_range, + ) + + +def _should_preempt_for_programme_change( + redis_client, session_id, pool_media_id, media_id, +): + """True when the viewer moved to a different programme on the same session.""" + if pool_media_id is None or str(pool_media_id) == str(media_id): + return False + try: + last_activity = float( + _get_pool_entry(redis_client, session_id).get("last_activity") or 0 + ) + except (TypeError, ValueError): + last_activity = 0 + # Ignore parallel startup probes within ~2s. + return (time.time() - last_activity) >= _STATS_GRACE_MIN_ELAPSED_SECONDS + + +def _try_reacquire_idle_pool( + redis_client, session_id, *, user_id, user, + wait_seconds=_POOL_WAIT_SECONDS, +): + """Stop the in-flight stream and wait to reuse the same provider slot.""" + _preempt_playback_streams(redis_client, session_id, user) + acquired = _acquire_idle_pool_session( + redis_client, session_id, user_id=user_id, handoff=True, + ) + if acquired is not None: + return acquired + return _wait_for_idle_pool_session( + redis_client, session_id, user_id=user_id, wait_seconds=wait_seconds, + handoff=True, + ) + + def _should_displace_busy_playback( range_header, content_length=None, busy_serving_range=None, ): @@ -497,18 +1201,17 @@ def _score_pool_fingerprint(entry, client_ip, client_user_agent): return score -def _mint_timeshift_session_id(): - return f"timeshift_{secrets.token_urlsafe(16)}" - - -def _redirect_with_new_session(request): - session_id = _mint_timeshift_session_id() +def _redirect_with_session(request, session_id): query_params = {k: request.GET.getlist(k) for k in request.GET} query_params["session_id"] = [session_id] redirect_url = f"{request.path}?{urlencode(query_params, doseq=True)}" return HttpResponse(status=301, headers={"Location": redirect_url}) +def _redirect_with_new_session(request): + return _redirect_with_session(request, mint_session_id()) + + def _pool_entry_owned_by_user(entry, user_id): """True when *entry* is unclaimed or owned by *user_id*.""" if not entry or not entry.get("profile_id"): @@ -519,27 +1222,40 @@ def _pool_entry_owned_by_user(entry, user_id): return str(owner) == str(user_id) -def _find_matching_idle_session( +def _find_matching_pool_session( redis_client, *, media_id, user_id, client_ip, client_user_agent, + channel_id=None, include_busy=False, ): - """Find an idle pooled session that likely belongs to the same client.""" + """Find a pooled session for the same viewer. + + Prefers an exact *media_id* match, then any in-flight or idle session on + the same channel so XC programme hops can recover a dropped ``session_id``. + """ if redis_client is None: return None + channel_prefix = f"{channel_id}_" if channel_id is not None else None matches = [] try: cursor = 0 while True: cursor, keys = redis_client.scan( - cursor, match="timeshift_pool:timeshift_*", count=100, + cursor, match=TimeshiftRedisKeys.pool_scan_pattern(), count=100, ) for key in keys: try: data = redis_client.hgetall(key) - if not data or data.get("busy") == "1": + if not data: + continue + if not include_busy and data.get("busy") == "1": continue if str(data.get("user_id") or "") != str(user_id): continue - if str(data.get("media_id") or "") != str(media_id): + stored_media = str(data.get("media_id") or "") + if stored_media == str(media_id): + exact_match = True + elif channel_prefix and stored_media.startswith(channel_prefix): + exact_match = False + else: continue session_id = key.rsplit(":", 1)[-1] score = _score_pool_fingerprint( @@ -547,7 +1263,9 @@ def _find_matching_idle_session( ) if score >= _MATCH_SCORE_THRESHOLD: last_activity = float(data.get("last_activity") or "0") - matches.append((session_id, score, last_activity)) + matches.append( + (session_id, score, last_activity, exact_match), + ) except Exception as exc: logger.debug("Timeshift pool scan skip %s: %s", key, exc) if cursor == 0: @@ -558,11 +1276,12 @@ def _find_matching_idle_session( if not matches: return None - matches.sort(key=lambda item: (item[1], item[2]), reverse=True) + matches.sort(key=lambda item: (item[3], item[1], item[2]), reverse=True) best = matches[0][0] logger.debug( - "Timeshift idle match: session=%s score=%s media=%s", - best, matches[0][1], media_id, + "Timeshift %s match: session=%s score=%s media=%s exact=%s", + "pool" if include_busy else "idle", + best, matches[0][1], media_id, matches[0][3], ) return best @@ -610,16 +1329,7 @@ def _store_pool_serving_range(redis_client, session_id, range_header): def _update_pool_position(redis_client, session_id, *, media_id, provider_timestamp): - """Move a reused session's descriptor to the position actually served. - - A seek keeps the session (provider-slot continuity) but changes the - catch-up position; the descriptor must follow it so later fingerprint - matches and same-channel displacement compare against the position the - session is really on. When the position actually moves, the byte state of - the PREVIOUS position's file (content_length, serving_range) is dropped — - keeping it would feed the near-EOF/displacement heuristics another - programme's size; the next successful open repopulates both. - """ + """Move a reused session's descriptor to the position actually served.""" if redis_client is None or not session_id: return key = _pool_key(session_id) @@ -644,13 +1354,8 @@ def _update_pool_position(redis_client, session_id, *, media_id, provider_timest def _store_pool_content_length(redis_client, session_id, upstream_response): if redis_client is None or not session_id or upstream_response is None: return - content_length = upstream_response.headers.get("Content-Length") - content_range = upstream_response.headers.get("Content-Range", "") - if content_range and "/" in content_range: - total = content_range.rsplit("/", 1)[-1] - if total != "*": - content_length = total - if not content_length: + content_length = _extract_representation_length(upstream_response) + if content_length is None: return try: redis_client.hset( @@ -662,14 +1367,20 @@ def _store_pool_content_length(redis_client, session_id, upstream_response): def _pool_lock(redis_client, session_id): return redis_client.lock( - _POOL_LOCK_KEY.format(session_id=session_id), + TimeshiftRedisKeys.pool_lock(session_id), timeout=10, blocking_timeout=5, ) -def _acquire_idle_pool_session(redis_client, session_id, *, user_id=None): - """Re-reserve an idle session's profile slot and mark it busy.""" +def _acquire_idle_pool_session( + redis_client, session_id, *, user_id=None, handoff=False, +): + """Re-reserve an idle session's profile slot and mark it busy. + + When *handoff* is True the displaced stream kept its profile reservation; + only flip the pool entry back to busy. + """ if redis_client is None or not session_id: return None key = _pool_key(session_id) @@ -680,16 +1391,17 @@ def _acquire_idle_pool_session(redis_client, session_id, *, user_id=None): return None if user_id is not None and not _pool_entry_owned_by_user(data, user_id): return None - if data.get("busy") == "1": + if data.get("busy") == "1" and not handoff: return None try: profile = M3UAccountProfile.objects.get(id=int(data["profile_id"])) except M3UAccountProfile.DoesNotExist: redis_client.delete(key) return None - reserved, _count, _reason = reserve_profile_slot(profile, redis_client) - if not reserved: - return None + if not handoff: + reserved, _count, _reason = reserve_profile_slot(profile, redis_client) + if not reserved: + return None redis_client.hset(key, mapping={ "busy": "1", "last_activity": str(time.time()), @@ -703,13 +1415,14 @@ def _acquire_idle_pool_session(redis_client, session_id, *, user_id=None): def _wait_for_idle_pool_session( redis_client, session_id, *, user_id=None, wait_seconds=_POOL_WAIT_SECONDS, + handoff=False, ): if redis_client is None or not session_id: return None deadline = time.time() + wait_seconds while True: acquired = _acquire_idle_pool_session( - redis_client, session_id, user_id=user_id, + redis_client, session_id, user_id=user_id, handoff=handoff, ) if acquired is not None: return acquired @@ -731,6 +1444,7 @@ def _create_pool_session( account_id, profile_id, stream_id, + dispatcharr_stream_id, provider_timestamp, provider_tz_name=None, ): @@ -751,29 +1465,170 @@ def _create_pool_session( "account_id": str(account_id), "profile_id": str(profile_id), "stream_id": str(stream_id), + "dispatcharr_stream_id": str(dispatcharr_stream_id), "provider_timestamp": str(provider_timestamp), "provider_tz_name": str(provider_tz_name or ""), "busy": "1", "last_activity": now, }) redis_client.expire(key, _POOL_ENTRY_TTL) + try: + redis_client.delete(_superseded_pool_key(session_id)) + except Exception: + pass return True except Exception as exc: logger.warning("Timeshift pool create failed for %s: %s", session_id, exc) return False -def _release_pool_session(redis_client, session_id, profile_id): +def _superseded_pool_key(session_id): + return TimeshiftRedisKeys.pool_superseded(session_id) + + +_active_upstream_lock = threading.Lock() +_active_upstreams = {} + + +def _active_upstream_key(virtual_channel_id, client_id): + return f"{virtual_channel_id}:{client_id}" + + +def _register_active_upstream(virtual_channel_id, client_id, upstream): + if not virtual_channel_id or not client_id or upstream is None: + return + key = _active_upstream_key(virtual_channel_id, client_id) + with _active_upstream_lock: + _active_upstreams[key] = upstream + + +def _unregister_active_upstream(virtual_channel_id, client_id): + if not virtual_channel_id or not client_id: + return + key = _active_upstream_key(virtual_channel_id, client_id) + with _active_upstream_lock: + _active_upstreams.pop(key, None) + + +def _close_active_upstream(virtual_channel_id, client_id): + """Close an in-worker upstream socket so a scrub preempt unblocks immediately.""" + if not virtual_channel_id or not client_id: + return + key = _active_upstream_key(virtual_channel_id, client_id) + with _active_upstream_lock: + upstream = _active_upstreams.pop(key, None) + if upstream is None: + return + try: + upstream.close() + except Exception: + pass + + +def _force_abandon_busy_pool(redis_client, session_id, profile_id): + """Drop a busy pool after a scrub preempt times out. + + The superseded marker prevents the displaced stream's ``release_cb`` from + double-releasing the provider profile slot. + """ + if redis_client is None or not session_id: + return + try: + redis_client.setex(_superseded_pool_key(session_id), 120, "1") + except Exception as exc: + logger.warning("Timeshift supersede mark failed for %s: %s", session_id, exc) + _discard_pool_session(redis_client, session_id, profile_id) + + +def _iter_upstream_with_stop( + upstream, + chunk_size, + redis_client, + stop_key, + stream_generation, + peek_data=None, + *, + inactivity_timeout=_STREAM_READ_INACTIVITY_SECONDS, +): + """Yield upstream bytes, polling the stop key before each blocking read.""" + last_data_at = time.time() + if peek_data: + should_stop, _ = _stream_stop_requested( + redis_client, stop_key, stream_generation, + ) + if should_stop: + try: + upstream.close() + except Exception: + pass + return + last_data_at = time.time() + yield peek_data + raw = upstream.raw + while True: + if ( + inactivity_timeout is not None + and time.time() - last_data_at >= inactivity_timeout + ): + logger.info( + "Timeshift upstream inactive for %ss, closing stream", + inactivity_timeout, + ) + try: + upstream.close() + except Exception: + pass + break + should_stop, _ = _stream_stop_requested( + redis_client, stop_key, stream_generation, + ) + if should_stop: + try: + upstream.close() + except Exception: + pass + break + try: + chunk = raw.read(chunk_size) + except requests.exceptions.ReadTimeout: + # Per-chunk read timeout: loop back to check stop (live-proxy pattern). + continue + except Exception: + break + if not chunk: + break + last_data_at = time.time() + yield chunk + + +def _release_pool_session( + redis_client, session_id, profile_id, *, + mark_pool_idle=True, release_profile=True, +): if redis_client is None: return - if profile_id is not None: + superseded = False + try: + superseded = bool(redis_client.exists(_superseded_pool_key(session_id))) + except Exception: + pass + if superseded and not release_profile: + # Displaced stream after a scrub failover: abandon already released. + return + if superseded and release_profile: + # Replacement stream is ending; stale marker must not block release. + try: + redis_client.delete(_superseded_pool_key(session_id)) + except Exception: + pass + if release_profile and profile_id is not None: try: release_profile_slot(int(profile_id), redis_client) except Exception as exc: logger.warning( "Timeshift slot release failed for profile %s: %s", profile_id, exc ) - if not session_id: + if not mark_pool_idle or not session_id: return key = _pool_key(session_id) try: @@ -788,6 +1643,45 @@ def _release_pool_session(redis_client, session_id, profile_id): logger.warning("Timeshift pool release failed for %s: %s", session_id, exc) +def _refresh_pool_session_ttl(redis_client, session_id): + """Extend pool metadata TTL while a session is actively streaming.""" + if redis_client is None or not session_id: + return + key = _pool_key(session_id) + try: + if not redis_client.exists(key): + return + busy = redis_client.hget(key, "busy") + if isinstance(busy, bytes): + busy = busy.decode() + ttl = _POOL_ENTRY_TTL if str(busy) == "1" else _POOL_IDLE_TTL + pipe = redis_client.pipeline(transaction=False) + pipe.hset(key, "last_activity", str(time.time())) + pipe.expire(key, ttl) + pipe.execute() + except Exception as exc: + logger.debug("Timeshift pool TTL refresh failed: %s", exc) + + +def _refresh_active_session_redis_ttl( + redis_client, session_id, programme_vid=None, +): + """Keep pool, stream generation, and API session auth alive during playback.""" + if redis_client is None or not session_id: + return + _refresh_pool_session_ttl(redis_client, session_id) + if programme_vid: + gen_key = _stream_generation_key(programme_vid, session_id) + try: + if redis_client.exists(gen_key): + redis_client.expire(gen_key, _POOL_ENTRY_TTL) + except Exception as exc: + logger.debug("Timeshift stream generation TTL refresh failed: %s", exc) + if catchup_session_exists(session_id, redis_client=redis_client): + from .sessions import touch_catchup_session + touch_catchup_session(session_id, redis_client=redis_client) + + def _discard_pool_session(redis_client, session_id, profile_id): if redis_client is None: return @@ -810,15 +1704,131 @@ def _discard_pool_session(redis_client, session_id, profile_id): def _make_release_once(redis_client, session_id, profile_id): state = {"done": False} - def _release(): + def _release(*, mark_pool_idle=True, release_profile=True): if state["done"]: return state["done"] = True - _release_pool_session(redis_client, session_id, profile_id) + _release_pool_session( + redis_client, session_id, profile_id, + mark_pool_idle=mark_pool_idle, + release_profile=release_profile, + ) return _release +def _stream_generation_key(virtual_channel_id, client_id): + return TimeshiftRedisKeys.stream_generation(virtual_channel_id, client_id) + + +def _current_stream_generation(redis_client, virtual_channel_id, client_id): + if redis_client is None: + return 0 + try: + gen = redis_client.get(_stream_generation_key(virtual_channel_id, client_id)) + return int(gen) if gen else 0 + except (TypeError, ValueError): + return 0 + + +def _allocate_stream_generation(redis_client, virtual_channel_id, client_id): + if redis_client is None: + return 1 + key = _stream_generation_key(virtual_channel_id, client_id) + try: + generation = int(redis_client.incr(key)) + redis_client.expire(key, _POOL_ENTRY_TTL) # Refreshed on each seek/new stream. + try: + redis_client.delete( + TimeshiftRedisKeys.client_stop(virtual_channel_id, client_id), + ) + redis_client.delete(_superseded_pool_key(client_id)) + except Exception: + pass + return generation + except Exception: + return 1 + + +def _cleanup_stream_generation(redis_client, virtual_channel_id, client_id): + """Delete the per-programme generation counter once a viewer is gone.""" + if redis_client is None or not virtual_channel_id or not client_id: + return + try: + redis_client.delete(_stream_generation_key(virtual_channel_id, client_id)) + except Exception as exc: + logger.debug("Timeshift stream generation cleanup failed: %s", exc) + + +def _cleanup_all_stream_generations(redis_client, client_id): + """Drop every programme-scoped generation counter for one viewer.""" + if redis_client is None or not client_id: + return + pattern = TimeshiftRedisKeys.stream_generation_scan_pattern(client_id) + try: + cursor = 0 + while True: + cursor, keys = redis_client.scan(cursor, match=pattern, count=100) + if keys: + redis_client.delete(*keys) + if cursor == 0: + break + except Exception as exc: + logger.debug("Timeshift stream generation scan cleanup failed: %s", exc) + + +def _set_client_stop(redis_client, virtual_channel_id, client_id, reason): + if redis_client is None: + return + stop_key = TimeshiftRedisKeys.client_stop(virtual_channel_id, client_id) + if reason == _STOP_REASON_REUSE: + try: + gen = redis_client.get(_stream_generation_key(virtual_channel_id, client_id)) + cancel_through = int(gen) if gen else 1 + except (TypeError, ValueError): + cancel_through = 1 + redis_client.setex(stop_key, 60, str(cancel_through)) + else: + redis_client.setex(stop_key, 60, reason) + + +def _stream_stop_requested(redis_client, stop_key, stream_generation): + """Return ``(should_stop, stopped_for_reuse)`` for this stream generation.""" + if redis_client is None or not stop_key: + return False, False + try: + value = redis_client.get(stop_key) + except Exception: + return False, False + if value is None: + return False, False + if isinstance(value, bytes): + value = value.decode() + if value == _STOP_REASON_REUSE: + return True, True + if value.isdigit(): + cancel_through = int(value) + return stream_generation <= cancel_through, True + if value in _TERMINAL_STOP_REASONS: + return True, False + return False, False + + +def _session_has_active_timeshift_stream(user, session_id): + """True when *session_id* still has a registered timeshift stats client.""" + if user is None or not session_id: + return False + try: + for conn in get_user_active_connections(user.id): + if conn.get("type") != "timeshift": + continue + if conn.get("client_id") == session_id: + return True + except Exception: + return False + return False + + def _preempt_playback_streams(redis_client, session_id, user): """Stop in-flight streams for this client session only.""" if redis_client is None or not session_id or user is None: @@ -831,13 +1841,17 @@ def _preempt_playback_streams(redis_client, session_id, user): continue conn_media_id = str(conn.get("media_id") or "") old_client_id = conn.get("client_id") + stop_target = _timeshift_stop_channel_id( + redis_client, conn_media_id, old_client_id, + ) logger.debug( "Timeshift preempt: stopping client %s on %s for reuse", - old_client_id, conn_media_id, + old_client_id, stop_target, ) - _unregister_stats_client(redis_client, conn_media_id, old_client_id) - stop_key = RedisKeys.client_stop(conn_media_id, old_client_id) - redis_client.setex(stop_key, 60, "true") + _set_client_stop( + redis_client, stop_target, old_client_id, _STOP_REASON_REUSE, + ) + _close_active_upstream(stop_target, old_client_id) except Exception as exc: logger.warning("Timeshift preempt failed: %s", exc) @@ -848,26 +1862,38 @@ def _terminate_previous_timeshift_sessions( """Displace this user's other catch-up positions on the same channel.""" if redis_client is None or user is None: return - prefix = f"timeshift_{channel_id}_" + channel_prefix = f"{channel_id}_" + displaced = False try: for conn in get_user_active_connections(user.id): if conn.get("type") != "timeshift": continue - if conn.get("client_id") == current_session_id: - continue conn_media_id = str(conn.get("media_id") or "") - if not conn_media_id.startswith(prefix): + old_client_id = conn.get("client_id") + if conn.get("client_id") == current_session_id: + # Programme hops for the same session are handled by pool + # displacement; stats stay on the stable per-session channel id. + continue + if not conn_media_id.startswith(channel_prefix): continue if conn_media_id.startswith(f"{current_media_id}_") or conn_media_id == current_media_id: continue - old_client_id = conn.get("client_id") + stats_channel_id = make_stats_channel_id(channel_id, old_client_id) logger.info( "Timeshift takeover: displacing session %s on %s", - old_client_id, conn_media_id, + old_client_id, stats_channel_id, ) - _unregister_stats_client(redis_client, conn_media_id, old_client_id) - stop_key = RedisKeys.client_stop(conn_media_id, old_client_id) - redis_client.setex(stop_key, 60, "true") + stop_target = _timeshift_stop_channel_id( + redis_client, stats_channel_id, old_client_id, + fallback=conn_media_id, + ) + _unregister_stats_client(redis_client, stats_channel_id, old_client_id) + displaced = True + _set_client_stop( + redis_client, stop_target, old_client_id, _STOP_REASON_HOP, + ) + if displaced: + _trigger_timeshift_stats_update(redis_client) except Exception as exc: logger.warning("Timeshift takeover check failed: %s", exc) @@ -885,6 +1911,7 @@ def _attempt_timeshift_stream( timestamp, client_id, client_ip, + client_user_agent, range_header, channel_logo_id, user, @@ -892,6 +1919,8 @@ def _attempt_timeshift_stream( debug, release_cb=None, pool_session_id=None, + stats_stream_id=None, + stream_stats=None, ): """Build the provider URL set for one (account, profile, stream) and stream it.""" server_url, xc_username, xc_password = get_transformed_credentials( @@ -907,7 +1936,10 @@ def _attempt_timeshift_stream( except AttributeError: user_agent = "" - virtual_channel_id = f"timeshift_{channel.id}_{safe_ts}_{stream_id_value}" + virtual_channel_id = make_virtual_channel_id( + channel.id, safe_ts, stream_id_value, + ) + stats_channel_id = make_stats_channel_id(channel.id, client_id) if debug: logger.debug( @@ -923,8 +1955,10 @@ def _attempt_timeshift_stream( user_agent=user_agent, range_header=range_header, virtual_channel_id=virtual_channel_id, + stats_channel_id=stats_channel_id, client_id=client_id, client_ip=client_ip, + client_user_agent=client_user_agent, user=user, channel_display_name=channel.name, timestamp_utc=timestamp, @@ -935,6 +1969,11 @@ def _attempt_timeshift_stream( redis_client=redis_client, release_cb=release_cb, pool_session_id=pool_session_id, + channel_id=channel.id, + channel_uuid=channel.uuid, + stats_stream_id=stats_stream_id, + stream_stats=stream_stats, + duration_minutes=duration_minutes, ) @@ -951,6 +1990,7 @@ def _stream_reused_session( duration_minutes, client_id, client_ip, + client_user_agent, range_header, channel_logo_id, user, @@ -963,14 +2003,7 @@ def _stream_reused_session( _discard_pool_session(redis_client, session_id, profile.id) return None - # The stored provider_timestamp is the position this session was CREATED - # for. Clients (TiviMate) keep the ?session_id= query when they rebuild - # the seek URL with a new start, so a reused session must serve the - # REQUESTED position, never replay the stored one — otherwise every - # timestamp-jump FF/RW snaps back to the session's original anchor. - # Provider zone is cached on the pool entry at session creation (account - # property from server_info); fall back to the reserved profile for legacy - # entries created before that field existed. + # Serve the requested position, not the pool entry's original anchor. provider_tz_name = descriptor.get("provider_tz_name") or None if provider_tz_name: provider_tz_name = str(provider_tz_name) @@ -988,6 +2021,15 @@ def _stream_reused_session( ) release_cb = _make_release_once(redis_client, session_id, profile.id) + raw_stream_id = descriptor.get("dispatcharr_stream_id") + if isinstance(raw_stream_id, bytes): + raw_stream_id = raw_stream_id.decode() + stats_stream_id = None + if raw_stream_id: + try: + stats_stream_id = int(raw_stream_id) + except (TypeError, ValueError): + stats_stream_id = None try: response = _attempt_timeshift_stream( m3u_account=m3u_account, @@ -1001,6 +2043,7 @@ def _stream_reused_session( timestamp=timestamp, client_id=client_id, client_ip=client_ip, + client_user_agent=client_user_agent, range_header=range_header, channel_logo_id=channel_logo_id, user=user, @@ -1008,6 +2051,7 @@ def _stream_reused_session( debug=debug, release_cb=release_cb, pool_session_id=session_id, + stats_stream_id=stats_stream_id, ) except Exception: _discard_pool_session(redis_client, session_id, profile.id) @@ -1025,7 +2069,7 @@ def _stream_reused_session( class _SlotReleasingStream: - """Iterator wrapper that releases the pool slot when WSGI closes the response.""" + """Iterator wrapper that closes the generator when WSGI closes the response.""" def __init__(self, generator, on_close): self._generator = generator @@ -1041,15 +2085,15 @@ class _SlotReleasingStream: try: self._generator.close() finally: - self._on_close() + self._on_close() # Backup if generator finally never ran. def _register_stats_client( redis_client, - virtual_channel_id, + stats_channel_id, client_id, client_ip, - user_agent, + client_user_agent, user, *, channel_display_name, @@ -1057,55 +2101,124 @@ def _register_stats_client( primary_url, channel_logo_id=None, m3u_profile_id=None, + programme_vid=None, + channel_id, + channel_uuid, + stats_stream_id=None, + stream_stats=None, + range_start=None, + representation_length=None, + programme_duration_secs=None, + emit_stats_update=False, ): """Write Redis keys so catch-up viewers appear on ``/stats``.""" if redis_client is None: return - client_set_key = RedisKeys.clients(virtual_channel_id) - client_key = RedisKeys.client_metadata(virtual_channel_id, client_id) - metadata_key = RedisKeys.channel_metadata(virtual_channel_id) + _cancel_stats_disconnect_grace(redis_client, stats_channel_id, client_id) + client_set_key = TimeshiftRedisKeys.clients(stats_channel_id) + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + metadata_key = TimeshiftRedisKeys.channel_metadata(stats_channel_id) now = str(time.time()) + try: + existing_connected_at = redis_client.hget(client_key, "connected_at") + existing_init_time = redis_client.hget( + metadata_key, ChannelMetadataField.INIT_TIME, + ) + existing_programme_start = redis_client.hget(client_key, "programme_start") + existing_position_anchor = redis_client.hget(client_key, "position_anchor_at") + existing_playback_base = redis_client.hget(client_key, "playback_base_secs") + existing_programme_vid = redis_client.hget(client_key, "programme_vid") + except Exception: + existing_connected_at = None + existing_init_time = None + existing_programme_start = None + existing_position_anchor = None + existing_playback_base = None + existing_programme_vid = None + if isinstance(existing_programme_vid, bytes): + existing_programme_vid = existing_programme_vid.decode() + notify_stats_update = existing_connected_at is None + if ( + programme_vid + and existing_programme_vid + and programme_vid != existing_programme_vid + ): + notify_stats_update = True + _cleanup_stream_generation(redis_client, existing_programme_vid, client_id) + playback_base_secs, position_anchor_at = resolve_stats_playback_fields( + timestamp_utc=timestamp_utc, + existing_programme_start=existing_programme_start, + existing_position_anchor=existing_position_anchor, + existing_playback_base=existing_playback_base, + range_start=range_start, + representation_length=representation_length, + programme_duration_secs=programme_duration_secs, + now=now, + ) client_payload = { - "user_agent": user_agent or "unknown", + "user_agent": client_user_agent or "unknown", "ip_address": client_ip, - "connected_at": now, + "connected_at": existing_connected_at or now, "last_active": now, "user_id": str(user.id) if user is not None else "0", "username": user.username if user is not None else "unknown", + "programme_start": timestamp_utc, + "position_anchor_at": position_anchor_at, } + if playback_base_secs is not None: + client_payload["playback_base_secs"] = str(playback_base_secs) + if programme_vid: + client_payload["programme_vid"] = programme_vid metadata_payload = { ChannelMetadataField.STATE: ChannelState.ACTIVE, - ChannelMetadataField.INIT_TIME: now, - ChannelMetadataField.OWNER: "timeshift", + ChannelMetadataField.INIT_TIME: existing_init_time or now, + ChannelMetadataField.CHANNEL_ID: str(channel_id), + ChannelMetadataField.CHANNEL_UUID: str(channel_uuid), ChannelMetadataField.CHANNEL_NAME: channel_display_name or "Timeshift", ChannelMetadataField.STREAM_NAME: f"Catch-up @ {timestamp_utc} UTC" if timestamp_utc else "Catch-up", ChannelMetadataField.URL: _redact_url(primary_url) if primary_url else "", - ChannelMetadataField.IS_TIMESHIFT: "1", } if channel_logo_id is not None: metadata_payload[ChannelMetadataField.LOGO_ID] = str(channel_logo_id) if m3u_profile_id is not None: metadata_payload[ChannelMetadataField.M3U_PROFILE] = str(m3u_profile_id) + seed_stream_stats_metadata( + redis_client, + metadata_key, + metadata_payload, + stats_stream_id=stats_stream_id, + stream_stats=stream_stats, + ) try: pipe = redis_client.pipeline(transaction=False) pipe.hset(client_key, mapping=client_payload) + if playback_base_secs is None: + pipe.hdel(client_key, "playback_base_secs") pipe.expire(client_key, CLIENT_TTL_SECONDS) pipe.sadd(client_set_key, client_id) pipe.expire(client_set_key, CLIENT_TTL_SECONDS) pipe.hset(metadata_key, mapping=metadata_payload) pipe.expire(metadata_key, CLIENT_TTL_SECONDS) pipe.execute() + if emit_stats_update and notify_stats_update: + _trigger_timeshift_stats_update(redis_client) except Exception as exc: logger.warning("Timeshift stats register failed: %s", exc) -def _heartbeat_stats_client(redis_client, virtual_channel_id, client_id, bytes_delta=0): +def _heartbeat_stats_client( + redis_client, stats_channel_id, client_id, bytes_delta=0, *, + pool_session_id=None, programme_vid=None, +): if redis_client is None: return - client_set_key = RedisKeys.clients(virtual_channel_id) - client_key = RedisKeys.client_metadata(virtual_channel_id, client_id) - metadata_key = RedisKeys.channel_metadata(virtual_channel_id) + client_set_key = TimeshiftRedisKeys.clients(stats_channel_id) + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + metadata_key = TimeshiftRedisKeys.channel_metadata(stats_channel_id) try: + if not redis_client.exists(client_key): + return + _complete_expired_stats_grace(redis_client, stats_channel_id, client_id) pipe = redis_client.pipeline(transaction=False) pipe.hset(client_key, "last_active", str(time.time())) pipe.expire(client_key, CLIENT_TTL_SECONDS) @@ -1114,16 +2227,23 @@ def _heartbeat_stats_client(redis_client, virtual_channel_id, client_id, bytes_d pipe.hincrby(metadata_key, ChannelMetadataField.TOTAL_BYTES, bytes_delta) pipe.expire(metadata_key, CLIENT_TTL_SECONDS) pipe.execute() + if programme_vid is None: + programme_vid = redis_client.hget(client_key, "programme_vid") + if isinstance(programme_vid, bytes): + programme_vid = programme_vid.decode() + _refresh_active_session_redis_ttl( + redis_client, pool_session_id or client_id, programme_vid, + ) except Exception as exc: logger.debug("Timeshift stats heartbeat failed: %s", exc) -def _unregister_stats_client(redis_client, virtual_channel_id, client_id): +def _unregister_stats_client(redis_client, stats_channel_id, client_id): if redis_client is None: return - client_set_key = RedisKeys.clients(virtual_channel_id) - client_key = RedisKeys.client_metadata(virtual_channel_id, client_id) - metadata_key = RedisKeys.channel_metadata(virtual_channel_id) + client_set_key = TimeshiftRedisKeys.clients(stats_channel_id) + client_key = TimeshiftRedisKeys.client_metadata(stats_channel_id, client_id) + metadata_key = TimeshiftRedisKeys.channel_metadata(stats_channel_id) try: redis_client.srem(client_set_key, client_id) redis_client.delete(client_key) @@ -1146,11 +2266,13 @@ def _open_upstream(url, user_agent, range_header): url, headers=headers, stream=True, - timeout=ConfigHelper.connection_timeout(), + timeout=( + ConfigHelper.connection_timeout(), + ConfigHelper.chunk_timeout(), + ), ) -_FORMAT_CACHE_KEY = "timeshift:format_idx:{}" _FORMAT_CACHE_TTL = 3600 # 1 hour @@ -1158,13 +2280,13 @@ def _get_cached_format_index(account_id): """Index of the URL shape that last worked for this account, or None.""" if account_id is None: return None - return cache.get(_FORMAT_CACHE_KEY.format(account_id)) + return cache.get(TimeshiftRedisKeys.format_cache(account_id)) def _set_cached_format_index(account_id, index): if account_id is None: return - cache.set(_FORMAT_CACHE_KEY.format(account_id), index, _FORMAT_CACHE_TTL) + cache.set(TimeshiftRedisKeys.format_cache(account_id), index, _FORMAT_CACHE_TTL) def _passthrough_response(status, content_range=None): @@ -1176,6 +2298,7 @@ def _passthrough_response(status, content_range=None): response = HttpResponse(status=status) if content_range: response["Content-Range"] = content_range + response["Accept-Ranges"] = "bytes" response.timeshift_passthrough = True return response @@ -1186,8 +2309,10 @@ def _stream_from_provider( user_agent, range_header, virtual_channel_id, + stats_channel_id=None, client_id, client_ip, + client_user_agent, user, channel_display_name, timestamp_utc, @@ -1198,6 +2323,11 @@ def _stream_from_provider( redis_client=None, release_cb=None, pool_session_id=None, + channel_id=None, + channel_uuid=None, + stats_stream_id=None, + stream_stats=None, + duration_minutes=None, ): """Try each upstream URL until one returns streamable MPEG-TS. @@ -1207,7 +2337,9 @@ def _stream_from_provider( """ chunk_size = max(ConfigHelper.chunk_size(), 262144) if release_cb is None: - release_cb = lambda: None # noqa: E731 + release_cb = lambda **_kwargs: None # noqa: E731 + if stats_channel_id is None: + stats_channel_id = virtual_channel_id cached_index = _get_cached_format_index(account_id) if cached_index is not None and 0 <= cached_index < len(candidate_urls): @@ -1257,23 +2389,20 @@ def _stream_from_provider( return _finalize_timeshift_response(_passthrough_response(416, content_range)) if response.status_code in (200, 206): peek = response.raw.read(1024) - sync_offset = find_ts_sync(peek) if peek else -1 - if sync_offset >= 0: - response._peek_data = peek[sync_offset:] - upstream = response - winning_index = orig_idx - break - # A 206 to a Range request legitimately starts mid-packet, so the - # sync byte rarely lands at offset 0. Trust the partial status and - # content type rather than the sync probe; only a full 200 carrying - # a PHP/HTML error page must be rejected here. content_type = response.headers.get("Content-Type", "") + # 206 may start mid-packet; accept before sync probe trims peek bytes. is_partial = response.status_code == 206 and bool(range_header) if is_partial and peek and "html" not in content_type and "json" not in content_type: response._peek_data = peek upstream = response winning_index = orig_idx break + sync_offset = find_ts_sync(peek) if peek else -1 + if sync_offset >= 0: + response._peek_data = peek[sync_offset:] + upstream = response + winning_index = orig_idx + break snippet = peek[:200].decode("utf-8", errors="replace") if peek else "(empty)" logger.warning( "Timeshift upstream returned %d but no TS sync in first %d " @@ -1316,64 +2445,117 @@ def _stream_from_provider( _store_pool_content_length(redis_client, pool_session_id, upstream) _store_pool_serving_range(redis_client, pool_session_id, range_header) + representation_length = _extract_representation_length(upstream) + if representation_length is None and redis_client and pool_session_id: + cached_length = redis_client.hget( + _pool_key(pool_session_id), "content_length", + ) + if cached_length is not None: + try: + if isinstance(cached_length, bytes): + cached_length = cached_length.decode() + representation_length = int(cached_length) + except (TypeError, ValueError): + representation_length = None + + client_length_headers = _build_downstream_length_headers( + range_header=range_header, + status_code=status, + representation_length=representation_length, + upstream_content_range=content_range or None, + upstream_content_length=upstream.headers.get("Content-Length"), + streaming=True, + ) + + programme_duration_secs = None + if duration_minutes: + programme_duration_secs = float(duration_minutes) * 60.0 + _register_stats_client( redis_client, - virtual_channel_id, + stats_channel_id, client_id, client_ip, - user_agent, + client_user_agent, user, channel_display_name=channel_display_name, timestamp_utc=timestamp_utc, primary_url=last_url, channel_logo_id=channel_logo_id, m3u_profile_id=m3u_profile_id, + programme_vid=virtual_channel_id, + channel_id=channel_id, + channel_uuid=channel_uuid, + stats_stream_id=stats_stream_id, + stream_stats=stream_stats, + range_start=_parse_range_start(range_header), + representation_length=representation_length, + programme_duration_secs=programme_duration_secs, + emit_stats_update=True, ) peek_data = getattr(upstream, "_peek_data", None) - chunks_iter = upstream.iter_content(chunk_size=chunk_size) - if peek_data: - chunks_iter = itertools.chain([peek_data], chunks_iter) + _register_active_upstream(virtual_channel_id, client_id, upstream) session_closed = {"done": False} - def _finish_session(*, close_upstream=False): + def _finish_session( + *, close_upstream=False, release_slot=True, + mark_pool_idle=True, release_profile=True, + ): if session_closed["done"]: return session_closed["done"] = True + _unregister_active_upstream(virtual_channel_id, client_id) if close_upstream: try: upstream.close() except Exception: pass - _unregister_stats_client(redis_client, virtual_channel_id, client_id) - release_cb() + if release_slot: + release_cb( + mark_pool_idle=mark_pool_idle, + release_profile=release_profile, + ) def stream_generator(): last_heartbeat = time.time() bytes_since_heartbeat = 0 total_yielded = 0 loop_start = time.time() - stop_key = RedisKeys.client_stop(virtual_channel_id, client_id) + stop_key = TimeshiftRedisKeys.client_stop(virtual_channel_id, client_id) + stream_generation = _allocate_stream_generation( + redis_client, virtual_channel_id, client_id, + ) stream_started_logged = False + stopped_for_reuse = False try: - for data in chunks_iter: + for data in _iter_upstream_with_stop( + upstream, chunk_size, redis_client, stop_key, + stream_generation, peek_data=peek_data, + ): if not data: continue if debug and not stream_started_logged: stream_started_logged = True logger.debug( - "Timeshift stream started: client=%s vid=%s range=%s status=%d", - client_id, virtual_channel_id, range_header or "(none)", status, + "Timeshift stream started: client=%s vid=%s range=%s status=%d gen=%d", + client_id, virtual_channel_id, range_header or "(none)", + status, stream_generation, ) yield data bytes_since_heartbeat += len(data) total_yielded += len(data) now = time.time() - if redis_client and redis_client.exists(stop_key): + should_stop, is_reuse = _stream_stop_requested( + redis_client, stop_key, stream_generation, + ) + if should_stop: logger.info("Timeshift client %s received stop signal", client_id) - redis_client.delete(stop_key) + stopped_for_reuse = is_reuse + if not is_reuse: + redis_client.delete(stop_key) break # Refresh stats every 5 seconds. if now - last_heartbeat >= 5: @@ -1387,8 +2569,10 @@ def _stream_from_provider( total_yielded, elapsed, mbps, ) _heartbeat_stats_client( - redis_client, virtual_channel_id, client_id, + redis_client, stats_channel_id, client_id, bytes_delta=bytes_since_heartbeat, + pool_session_id=pool_session_id or client_id, + programme_vid=virtual_channel_id, ) last_heartbeat = now bytes_since_heartbeat = 0 @@ -1400,8 +2584,10 @@ def _stream_from_provider( elapsed = time.time() - loop_start if bytes_since_heartbeat > 0: _heartbeat_stats_client( - redis_client, virtual_channel_id, client_id, + redis_client, stats_channel_id, client_id, bytes_delta=bytes_since_heartbeat, + pool_session_id=pool_session_id or client_id, + programme_vid=virtual_channel_id, ) if debug and total_yielded > 0: mbps = (total_yielded * 8) / elapsed / 1_000_000 if elapsed > 0 else 0 @@ -1409,18 +2595,61 @@ def _stream_from_provider( "Timeshift disconnect: vid=%s client=%s yielded=%d bytes in %.1fs (%.2f Mbps avg)", virtual_channel_id, client_id, total_yielded, elapsed, mbps, ) - _finish_session(close_upstream=True) + if redis_client and redis_client.exists(stop_key): + should_stop, is_reuse = _stream_stop_requested( + redis_client, stop_key, stream_generation, + ) + if should_stop: + if not stopped_for_reuse: + stopped_for_reuse = is_reuse + if not is_reuse: + redis_client.delete(stop_key) + if ( + not stopped_for_reuse + and redis_client + and stream_generation + < _current_stream_generation( + redis_client, virtual_channel_id, client_id, + ) + ): + # Newer stream generation took over (seek handoff). + stopped_for_reuse = True + if not stopped_for_reuse: + if not _is_timeshift_startup_probe(total_yielded, elapsed): + _cleanup_stream_generation( + redis_client, virtual_channel_id, client_id, + ) + if _should_schedule_stats_disconnect_grace( + total_yielded, + elapsed, + stopped_for_reuse=stopped_for_reuse, + redis_client=redis_client, + client_id=client_id, + ): + _schedule_stats_disconnect_grace( + redis_client, stats_channel_id, client_id, + ) + _finish_session(close_upstream=True, mark_pool_idle=True) + else: + # Seek handoff: replacement stream keeps pool busy and profile slot. + _finish_session( + close_upstream=True, + mark_pool_idle=False, + release_profile=False, + ) - stream_iter = _SlotReleasingStream(stream_generator(), _finish_session) + def _finish_session_backup(): + _finish_session(close_upstream=True) + + stream_iter = _SlotReleasingStream(stream_generator(), _finish_session_backup) response = StreamingHttpResponse( stream_iter, content_type=content_type, status=status, ) response["X-Accel-Buffering"] = "no" # avoid nginx throttling the stream - if content_range: - response["Content-Range"] = content_range - response["Accept-Ranges"] = "bytes" + for header_name, header_value in client_length_headers.items(): + response[header_name] = header_value return _finalize_timeshift_response(response) diff --git a/core/tasks.py b/core/tasks.py index eefd1697..92de0581 100644 --- a/core/tasks.py +++ b/core/tasks.py @@ -7,7 +7,7 @@ import re import time import os from core.utils import RedisClient, send_websocket_update, acquire_task_lock, release_task_lock -from apps.proxy.live_proxy.channel_status import ChannelStatus +from apps.proxy.live_proxy.channel_status import build_live_channel_stats_data from apps.m3u.models import M3UAccount from apps.epg.models import EPGSource from apps.m3u.tasks import refresh_single_m3u_account @@ -425,43 +425,22 @@ def fetch_channel_stats(): redis_client = RedisClient.get_client() try: - # Basic info for all channels - channel_pattern = "live:channel:*:metadata" - all_channels = [] - - # Extract channel IDs from keys - cursor = 0 - while True: - cursor, keys = redis_client.scan(cursor, match=channel_pattern) - for key in keys: - channel_id_match = re.search(r"live:channel:(.*):metadata", key) - if channel_id_match: - ch_id = channel_id_match.group(1) - channel_info = ChannelStatus.get_basic_channel_info(ch_id) - if channel_info: - all_channels.append(channel_info) - - if cursor == 0: - break - - send_websocket_update( - "updates", - "update", - { - "success": True, - "type": "channel_stats", - "stats": json.dumps({'channels': all_channels, 'count': len(all_channels)}) - }, - collect_garbage=True - ) - - # Explicitly clean up large data structures - all_channels = None - + live_stats = build_live_channel_stats_data(redis_client) except Exception as e: logger.error(f"Error in channel_status: {e}", exc_info=True) return + send_websocket_update( + "updates", + "update", + { + "success": True, + "type": "channel_stats", + "stats": json.dumps(live_stats), + }, + collect_garbage=True + ) + @shared_task def rehash_streams(keys): """ diff --git a/frontend/src/WebSocket.jsx b/frontend/src/WebSocket.jsx index 89b1a1ef..e050e912 100644 --- a/frontend/src/WebSocket.jsx +++ b/frontend/src/WebSocket.jsx @@ -320,6 +320,10 @@ export const WebsocketProvider = ({ children }) => { setVodStats(JSON.parse(parsedEvent.data.stats)); break; + case 'timeshift_stats': + setTimeshiftStats(JSON.parse(parsedEvent.data.stats)); + break; + case 'vod_started': case 'vod_stopped': { const { content_name, client_ip, user_id } = parsedEvent.data; @@ -1055,6 +1059,7 @@ export const WebsocketProvider = ({ children }) => { const setChannelStats = useChannelsStore((s) => s.setChannelStats); const setVodStats = useChannelsStore((s) => s.setVodStats); + const setTimeshiftStats = useChannelsStore((s) => s.setTimeshiftStats); const fetchPlaylists = usePlaylistsStore((s) => s.fetchPlaylists); const setRefreshProgress = usePlaylistsStore((s) => s.setRefreshProgress); const setProfilePreview = usePlaylistsStore((s) => s.setProfilePreview); diff --git a/frontend/src/api.js b/frontend/src/api.js index 1c826465..50db0165 100644 --- a/frontend/src/api.js +++ b/frontend/src/api.js @@ -2483,6 +2483,56 @@ export default class API { } } + static async getTimeshiftStats() { + try { + const response = await request(`${host}/proxy/catchup/stats/`); + + return response; + } catch (e) { + errorNotification('Failed to retrieve catch-up stats', e); + } + } + + static async getAllConnectionStats() { + try { + const response = await request(`${host}/proxy/stats/`); + + return response; + } catch (e) { + errorNotification('Failed to retrieve connection stats', e); + throw e; + } + } + + static async getCatchupPrograms(sessions) { + try { + const response = await request(`${host}/proxy/catchup/programs/`, { + method: 'POST', + body: { sessions }, + }); + + return response; + } catch (e) { + console.error('Failed to retrieve catch-up programmes', e); + return { sessions: [] }; + } + } + + static async stopTimeshiftSession(sessionId) { + try { + const response = await request(`${host}/proxy/catchup/stop_client/`, { + method: 'POST', + body: { + session_id: sessionId, + }, + }); + + return response; + } catch (e) { + errorNotification('Failed to stop catch-up session', e); + } + } + static async stopVODClient(clientId) { try { const response = await request(`${host}/proxy/vod/stop_client/`, { diff --git a/frontend/src/components/ProgramPreview.jsx b/frontend/src/components/ProgramPreview.jsx index 7da27ca2..5163c0a5 100644 --- a/frontend/src/components/ProgramPreview.jsx +++ b/frontend/src/components/ProgramPreview.jsx @@ -21,7 +21,70 @@ const formatProgramTime = (seconds) => { return `${minutes}:${secs.toString().padStart(2, '0')}`; }; -const ProgramPreview = ({ program, loading, fetched, label = 'Now Playing:' }) => { +const formatAirWindow = (startTime, endTime) => { + if (!startTime || !endTime) { + return null; + } + const start = new Date(startTime); + const end = new Date(endTime); + const dateOpts = { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' }; + const endOpts = { hour: '2-digit', minute: '2-digit' }; + return `${start.toLocaleString(undefined, dateOpts)} to ${end.toLocaleTimeString(undefined, endOpts)}`; +}; + +const getTimelineState = ({ + timelineMode, + program, + playbackElapsedSeconds, + now, +}) => { + const startTime = program.start_time ? new Date(program.start_time) : null; + const endTime = program.end_time ? new Date(program.end_time) : null; + if (!startTime || !endTime) { + return { hasValidTime: false }; + } + + const totalDuration = (endTime - startTime) / 1000; + if (totalDuration <= 0) { + return { hasValidTime: false }; + } + + if (timelineMode === 'catchup') { + const watched = Math.max(0, Math.floor(playbackElapsedSeconds ?? 0)); + const cappedWatched = Math.min(watched, totalDuration); + const remaining = Math.max(0, totalDuration - cappedWatched); + return { + hasValidTime: true, + elapsed: cappedWatched, + remaining, + percentage: Math.min(100, (cappedWatched / totalDuration) * 100), + elapsedLabel: 'watched', + remainingLabel: 'remaining', + airWindow: formatAirWindow(program.start_time, program.end_time), + }; + } + + const elapsed = (now - startTime) / 1000; + const remaining = (endTime - now) / 1000; + return { + hasValidTime: true, + elapsed, + remaining, + percentage: Math.min(100, Math.max(0, (elapsed / totalDuration) * 100)), + elapsedLabel: 'elapsed', + remainingLabel: 'remaining', + airWindow: null, + }; +}; + +const ProgramPreview = ({ + program, + loading, + fetched, + label = 'Now Playing:', + timelineMode = 'live', + playbackElapsedSeconds = 0, +}) => { const [isExpanded, setIsExpanded] = useState(false); if (loading) { @@ -46,21 +109,12 @@ const ProgramPreview = ({ program, loading, fetched, label = 'Now Playing:' }) = return null; } - const now = new Date(); - const startTime = program.start_time ? new Date(program.start_time) : null; - const endTime = program.end_time ? new Date(program.end_time) : null; - - let elapsed = 0, remaining = 0, percentage = 0; - let hasValidTime = false; - if (startTime && endTime) { - const totalDuration = (endTime - startTime) / 1000; - if (totalDuration > 0) { - hasValidTime = true; - elapsed = (now - startTime) / 1000; - remaining = (endTime - now) / 1000; - percentage = Math.min(100, Math.max(0, (elapsed / totalDuration) * 100)); - } - } + const timeline = getTimelineState({ + timelineMode, + program, + playbackElapsedSeconds, + now: new Date(), + }); return ( <> @@ -92,18 +146,23 @@ const ProgramPreview = ({ program, loading, fetched, label = 'Now Playing:' }) = )} - {isExpanded && hasValidTime && ( + {isExpanded && timeline.hasValidTime && ( + {timeline.airWindow && ( + + Aired {timeline.airWindow} + + )} - {formatProgramTime(elapsed)} elapsed + {formatProgramTime(timeline.elapsed)} {timeline.elapsedLabel} - {formatProgramTime(remaining)} remaining + {formatProgramTime(timeline.remaining)} {timeline.remainingLabel} { expect(screen.getByText(/elapsed/)).toBeTruthy(); expect(screen.queryByText('null')).toBeNull(); }); + + it('uses session-based progress for catch-up programmes', () => { + const program = { + title: 'Archived Show', + description: 'Catch-up description', + start_time: '2026-06-08T17:00:00+00:00', + end_time: '2026-06-08T18:00:00+00:00', + }; + render( + + ); + + expect(screen.getByText('Watching:')).toBeTruthy(); + + const expandButton = screen.getByTestId('chevron-right').closest('button'); + fireEvent.click(expandButton); + + expect(screen.getByText(/Aired /)).toBeTruthy(); + expect(screen.getByText('15:00 watched')).toBeTruthy(); + expect(screen.getByText('45:00 remaining')).toBeTruthy(); + expect(screen.getByTestId('progress').getAttribute('data-value')).toBe('25'); + }); }); describe('formatProgramTime', () => { diff --git a/frontend/src/components/cards/StreamConnectionCard.jsx b/frontend/src/components/cards/StreamConnectionCard.jsx index ae85b410..d05478bd 100644 --- a/frontend/src/components/cards/StreamConnectionCard.jsx +++ b/frontend/src/components/cards/StreamConnectionCard.jsx @@ -22,7 +22,6 @@ import { Gauge, HardDriveDownload, HardDriveUpload, - Rewind, SquareX, Timer, Users, @@ -665,18 +664,6 @@ const StreamConnectionCard = ({ {/* Add stream information badges */} - {channel.is_timeshift && ( - - } - > - TIMESHIFT - - - )} {channel.resolution && ( diff --git a/frontend/src/components/cards/TimeshiftConnectionCard.jsx b/frontend/src/components/cards/TimeshiftConnectionCard.jsx new file mode 100644 index 00000000..88736af4 --- /dev/null +++ b/frontend/src/components/cards/TimeshiftConnectionCard.jsx @@ -0,0 +1,405 @@ +import React, { useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import logo from '../../images/logo.png'; +import { + ActionIcon, + Badge, + Box, + Card, + Center, + Group, + Stack, + Text, + Tooltip, +} from '@mantine/core'; +import { + ChevronDown, + HardDriveUpload, + History, + SquareX, + Timer, +} from 'lucide-react'; +import ProgramPreview from '../ProgramPreview.jsx'; +import { + calculateConnectionDuration, + calculateConnectionStartTime, + computeCatchupPlaybackSeconds, + getConnectionDurationSeconds, +} from '../../utils/cards/TimeshiftConnectionCardUtils.js'; +import { useDateTimeFormat } from '../../utils/dateTimeUtils.js'; +import { getLogoUrl } from '../../utils/cards/StreamConnectionCardUtils.js'; +import useUsersStore from '../../store/users.jsx'; + +const ClientDetails = ({ connection, connectionStartTime }) => ( + + {connection.user_agent && connection.user_agent !== 'Unknown' && ( + + + User Agent: + + + {connection.user_agent.length > 100 + ? `${connection.user_agent.substring(0, 100)}...` + : connection.user_agent} + + + )} + + + + Session ID: + + + {connection.session_id || connection.client_id || 'Unknown'} + + + + {connection.connected_at && ( + + + Connected: + + {connectionStartTime} + + )} + + {connection.duration > 0 && ( + + + Watch Duration: + + + {calculateConnectionDuration(connection)} + + + )} + + {connection.bytes_streamed > 0 && ( + + + Data Sent: + + + {(connection.bytes_streamed / (1024 * 1024)).toFixed(1)} MB + + + )} + + {connection.avg_bitrate_kbps > 0 && ( + + + Avg Bitrate: + + + {connection.avg_bitrate_kbps > 1000 + ? `${(connection.avg_bitrate_kbps / 1000).toFixed(2)} Mbps` + : `${connection.avg_bitrate_kbps.toFixed(0)} Kbps`} + + + )} + +); + +const TimeshiftConnectionCard = ({ + timeshiftSession, + currentProgram, + stopTimeshiftSession, + logos, +}) => { + const { fullDateTimeFormat } = useDateTimeFormat(); + const [isClientExpanded, setIsClientExpanded] = useState(false); + const users = useUsersStore((s) => s.users); + const [, setUpdateTrigger] = useState(0); + + const usersMap = useMemo(() => { + const map = {}; + users.forEach((u) => { + map[String(u.id)] = u.username; + }); + return map; + }, [users]); + + useEffect(() => { + const interval = setInterval(() => { + setUpdateTrigger((prev) => prev + 1); + }, 1000); + return () => clearInterval(interval); + }, []); + + const connection = + timeshiftSession.individual_connection || + (timeshiftSession.connections && timeshiftSession.connections[0]); + + const logoUrl = + timeshiftSession.logo_url || + getLogoUrl(timeshiftSession.logo_id, logos) || + logo; + + const programmePreview = useMemo(() => { + if (currentProgram?.title) { + return { + title: currentProgram.title, + description: currentProgram.description, + start_time: currentProgram.start_time, + end_time: currentProgram.end_time, + duration_secs: currentProgram.duration_secs, + }; + } + if (timeshiftSession.programme_start) { + return { + title: `Catch-up @ ${timeshiftSession.programme_start}`, + }; + } + return null; + }, [currentProgram, timeshiftSession.programme_start]); + + const m3uProfileName = + connection?.m3u_profile?.profile_name || + connection?.m3u_profile?.account_name || + null; + + // Play position from URL timestamp + EPG window + stream-open anchor. + const playbackBaseRef = useRef({ base: null, receivedAtMs: 0 }); + useEffect(() => { + const computed = computeCatchupPlaybackSeconds({ + programmeStart: timeshiftSession.programme_start, + programStartTime: programmePreview?.start_time, + programDurationSecs: programmePreview?.duration_secs, + positionAnchorAt: timeshiftSession.position_anchor_at, + playbackBaseSecs: timeshiftSession.playback_base_secs, + nowMs: Date.now(), + }); + if (computed != null) { + playbackBaseRef.current = { + base: computed, + receivedAtMs: Date.now(), + }; + } + }, [ + timeshiftSession.programme_start, + timeshiftSession.position_anchor_at, + timeshiftSession.playback_base_secs, + programmePreview?.start_time, + programmePreview?.duration_secs, + ]); + + const { base: playbackBase, receivedAtMs: playbackReceivedAtMs } = + playbackBaseRef.current; + const playbackElapsedSeconds = + playbackBase != null + ? playbackBase + (Date.now() - playbackReceivedAtMs) / 1000 + : getConnectionDurationSeconds(connection); + + const getConnectionStartTime = useCallback( + (conn) => calculateConnectionStartTime(conn, fullDateTimeFormat), + [fullDateTimeFormat] + ); + + return ( + + + + + channel logo + + + + {connection && ( + +
+ + {calculateConnectionDuration(connection)} +
+ + )} + {connection && stopTimeshiftSession && ( +
+ + + stopTimeshiftSession( + connection.session_id || connection.client_id, + ) + } + > + + + +
+ )} + + + + {m3uProfileName && ( + + + + + {m3uProfileName} + + + + )} + + + {timeshiftSession.channel_name || 'Catch-up'} + + + + + + + {programmePreview && ( + + + + )} + + + {timeshiftSession.resolution && ( + + + {timeshiftSession.resolution} + + + )} + {timeshiftSession.source_fps && ( + + + {timeshiftSession.source_fps} FPS + + + )} + {timeshiftSession.video_codec && ( + + + {timeshiftSession.video_codec.toUpperCase()} + + + )} + {timeshiftSession.audio_codec && ( + + + {timeshiftSession.audio_codec.toUpperCase()} + + + )} + {timeshiftSession.audio_channels && ( + + + {timeshiftSession.audio_channels} + + + )} + {timeshiftSession.stream_type && ( + + + {timeshiftSession.stream_type.toUpperCase()} + + + )} + + + {connection && ( + + setIsClientExpanded(!isClientExpanded)} + > + + + Client IP: + + + {connection.ip_address || 'Unknown IP'} + + {usersMap[String(connection.user_id)] && ( + <> + + User: + + + {usersMap[String(connection.user_id)]} + + + )} + + + + + {isClientExpanded ? 'Hide Details' : 'Show Details'} + + + + + + {isClientExpanded && ( + + )} + + )} + + + ); +}; + +export default TimeshiftConnectionCard; diff --git a/frontend/src/pages/Stats.jsx b/frontend/src/pages/Stats.jsx index 86c7f1c3..38320fee 100644 --- a/frontend/src/pages/Stats.jsx +++ b/frontend/src/pages/Stats.jsx @@ -23,19 +23,23 @@ import useLocalStorage from '../hooks/useLocalStorage'; import SystemEvents from '../components/SystemEvents'; import ErrorBoundary from '../components/ErrorBoundary.jsx'; import { - fetchActiveChannelStats, + fetchAllConnectionStats, getClientStats, + getCatchupPrograms, getCombinedConnections, getCurrentPrograms, getStatsByChannelId, - getVODStats, stopChannel, stopClient, + stopTimeshiftSession, stopVODClient, } from '../utils/pages/StatsUtils.js'; const VodConnectionCard = React.lazy( () => import('../components/cards/VodConnectionCard.jsx') ); +const TimeshiftConnectionCard = React.lazy( + () => import('../components/cards/TimeshiftConnectionCard.jsx') +); const StreamConnectionCard = React.lazy( () => import('../components/cards/StreamConnectionCard.jsx') ); @@ -46,7 +50,9 @@ const Connections = ({ channelsByUUID, channels, handleStopVODClient, + handleStopTimeshiftSession, currentPrograms, + catchupPrograms, }) => { const logos = useLogosStore((s) => s.logos); @@ -88,6 +94,16 @@ const Connections = ({ stopVODClient={handleStopVODClient} /> ); + } else if (connection.type === 'timeshift') { + return ( + + ); } return null; })} @@ -101,17 +117,20 @@ const StatsPage = () => { const setChannelStats = useChannelsStore((s) => s.setChannelStats); const vodConnections = useChannelsStore((s) => s.activeVodConnections); const setVodStats = useChannelsStore((s) => s.setVodStats); + const timeshiftSessions = useChannelsStore((s) => s.activeTimeshiftSessions); + const setTimeshiftStats = useChannelsStore((s) => s.setTimeshiftStats); const streamProfiles = useStreamProfilesStore((s) => s.profiles); const [clients, setClients] = useState([]); const [channelHistory, setChannelHistory] = useState({}); const [isPollingActive, setIsPollingActive] = useState(false); const [currentPrograms, setCurrentPrograms] = useState({}); + const [catchupPrograms, setCatchupPrograms] = useState({}); const [channels, setChannels] = useState({}); // id -> channel const [channelsByUUID, setChannelsByUUID] = useState({}); // uuid -> id // Compute needed channel UUIDs from the current active channels. - // Stream previews use a non-UUID hash as channel_id — filter those out. + // Stream previews use a non-UUID hash as channel_id; filter those out. const UUID_REGEX = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; const neededUUIDs = useMemo( @@ -168,55 +187,51 @@ const StatsPage = () => { (total, vodContent) => total + (vodContent.connections?.length || 0), 0 ); + const timeshiftConnectionsCount = timeshiftSessions.reduce( + (total, session) => total + (session.connections?.length || 0), + 0 + ); const handleStopVODClient = async (clientId) => { await stopVODClient(clientId); - // Refresh VOD stats after stopping to update the UI - fetchVODStats(); + fetchAllStats(); }; - // Function to fetch channel stats from API - const fetchChannelStats = useCallback(async () => { - try { - const response = await fetchActiveChannelStats(); - if (response) { - setChannelStats(response); - } else { - console.log('API response was empty or null'); - } - } catch (error) { - console.error('Error fetching channel stats:', error); - console.error('Error details:', { - message: error.message, - status: error.status, - body: error.body, - }); - } - }, [setChannelStats]); + const handleStopTimeshiftSession = async (sessionId) => { + await stopTimeshiftSession(sessionId); + fetchAllStats(); + }; - const fetchVODStats = useCallback(async () => { + const fetchAllStats = useCallback(async () => { try { - const response = await getVODStats(); + const response = await fetchAllConnectionStats(); if (response) { - setVodStats(response); + if (response.live) { + setChannelStats(response.live); + } + if (response.vod) { + setVodStats(response.vod); + } + if (response.catchup) { + setTimeshiftStats(response.catchup); + } } else { - console.log('VOD API response was empty or null'); + console.log('Combined stats API response was empty or null'); } } catch (error) { - console.error('Error fetching VOD stats:', error); + console.error('Error fetching connection stats:', error); console.error('Error details:', { message: error.message, status: error.status, body: error.body, }); } - }, [setVodStats]); + }, [setChannelStats, setVodStats, setTimeshiftStats]); // Always fetch once on mount, regardless of polling interval setting useEffect(() => { - fetchChannelStats(); - fetchVODStats(); - }, [fetchChannelStats, fetchVODStats]); + fetchAllStats(); + }, [fetchAllStats]); // Set up polling for stats when on stats page useEffect(() => { @@ -226,8 +241,7 @@ const StatsPage = () => { setIsPollingActive(true); const interval = setInterval(() => { - fetchChannelStats(); - fetchVODStats(); + fetchAllStats(); }, refreshInterval); return () => { @@ -237,7 +251,7 @@ const StatsPage = () => { } else { setIsPollingActive(false); } - }, [refreshInterval, fetchChannelStats, fetchVODStats]); + }, [refreshInterval, fetchAllStats]); useEffect(() => { console.log('Processing channel stats:', channelStats); @@ -328,10 +342,76 @@ const StatsPage = () => { }; }, [activeChannelIds]); // Only re-run when active channel set changes - // Combine active streams and VOD connections into a single mixed list + // Programme metadata for catch-up cards: fetch when sessions or seek + // position (programme_start) change, then reschedule at programme end. + const timeshiftProgrammeKey = useMemo(() => { + return timeshiftSessions + .map((session) => `${session.session_id}:${session.programme_start || ''}`) + .sort() + .join(','); + }, [timeshiftSessions]); + + const timeshiftSessionsRef = useRef(timeshiftSessions); + useEffect(() => { + timeshiftSessionsRef.current = timeshiftSessions; + }, [timeshiftSessions]); + + useEffect(() => { + if (!timeshiftProgrammeKey) { + setCatchupPrograms({}); + return; + } + + let timer = null; + + const fetchPrograms = async () => { + const sessions = timeshiftSessionsRef.current.map((session) => ({ + session_id: session.session_id, + channel_uuid: session.channel_uuid, + programme_start: session.programme_start, + })); + const programs = await getCatchupPrograms(sessions); + setCatchupPrograms(programs); + + if (programs && Object.keys(programs).length > 0) { + const now = new Date(); + let nearestEndTime = null; + + Object.values(programs).forEach((program) => { + if (program && program.end_time) { + const endTime = new Date(program.end_time); + if ( + endTime > now && + (!nearestEndTime || endTime < nearestEndTime) + ) { + nearestEndTime = endTime; + } + } + }); + + if (nearestEndTime) { + const timeUntilChange = nearestEndTime.getTime() - now.getTime(); + const fetchDelay = Math.max(timeUntilChange + 5000, 0); + timer = setTimeout(fetchPrograms, fetchDelay); + } + } + }; + + fetchPrograms(); + + return () => { + if (timer) clearTimeout(timer); + }; + }, [timeshiftProgrammeKey]); + + // Combine active streams, VOD, and catch-up into a single mixed list const combinedConnections = useMemo(() => { - return getCombinedConnections(channelHistory, vodConnections); - }, [channelHistory, vodConnections]); + return getCombinedConnections( + channelHistory, + vodConnections, + timeshiftSessions + ); + }, [channelHistory, vodConnections, timeshiftSessions]); return ( <> @@ -348,6 +428,11 @@ const StatsPage = () => { {vodConnectionsCount !== 1 ? 'VOD connections' : 'VOD connection'} + {' • '} + {timeshiftConnectionsCount}{' '} + {timeshiftConnectionsCount !== 1 + ? 'catch-up sessions' + : 'catch-up session'} Refresh Interval (seconds): @@ -374,10 +459,7 @@ const StatsPage = () => {