diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e09e1fa..86fe5205 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,16 +17,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Catch-up Stats UI.** Dedicated catch-up connection cards show channel logo, programme preview (watched/remaining timeline), playback position, bitrate, and a stop button. Stats refresh in real time via WebSocket (`timeshift_stats`) with desktop notifications when catch-up sessions start or end. - **Catch-up flags** (`tv_archive`, `tv_archive_duration`) denormalized at import time for zero-cost output queries; end-of-refresh SQL rollup updates channels linked to the refreshed account and self-heals stale flags on those channels only (e.g. after catch-up streams are removed or an account is deactivated). - **Strictly-UTC API surface, automatic per-provider timezone.** `server_info.timezone` is always `UTC` and the XC EPG `start`/`end` strings are emitted in UTC; the proxy converts the requested instant to the serving provider's own reported timezone (`server_info.timezone` captured on account refresh) at request time. No timezone to configure. - - **Multi-provider failover.** Catch-up walks the channel's catch-up streams in order (like live playback): if one provider cannot serve the archive the next is tried, each attempt with its own account context (credentials, reported timezone, user-agent). Accounts that fail with an auth/ban-class status are not retried via their other streams. + - **Multi-provider failover.** Catch-up walks the channel's catch-up streams in order (like live playback): if one provider cannot serve the archive the next is tried, each attempt with its own account context (credentials, reported timezone, user-agent). Accounts that fail with an auth/ban-class status are not retried via their other streams. Attempts prefer streams whose `catchup_days` cover the requested programme age, then fall back to shorter archives in channel order. - **Provider pool accounting.** Catch-up reserves a provider profile slot (`connection_pool`) before connecting upstream — same contract as live and VOD — and releases it when the session ends. When the default profile is at capacity it walks the account's alternate profiles with their own resolved credentials, and answers `503` when every profile is full. - **Per-client session pool.** The first request without `session_id` and no fingerprint match receives a `301` redirect to the same URL with a stable `?session_id=` query parameter; that ID becomes the client identity for stats, stop keys, and pool coordination. Reconnects that omit `session_id` but match an existing pool entry (same user, IP, and user-agent) are served immediately without a redirect, matching IPTV client fast-forward behaviour. Each viewer gets their own Redis-backed pool entry even when watching the same programme; idle sessions can be reclaimed via fingerprint matching. Plain GET restarts stream from byte 0 with upstream `Content-Length` when known (provider-faithful). Real scrubs within a session stop the in-flight stream through the standard stop-key mechanism; parallel startup probes (full-file, open-ended, and EOF tail `Range` requests) are deferred with `503` instead of opening extra upstream connections. Parallel HTTP probes from the same `session_id` for the same programme do not count as extra streams toward the user limit; distinct sessions still each consume a slot. - - **`xmltv_prev_days_override` under `Settings → EPG`** (default `0` = auto-detect). Verbose timeshift logging follows the standard logger DEBUG level. - - **EPG XMLTV `prev_days` auto-detection** from the provider's largest `tv_archive_duration` (capped at 30 days), overridable via setting or `?prev_days=` URL parameter. + - Verbose timeshift logging follows the standard logger DEBUG level. + - **EPG XMLTV `prev_days` auto-detection** from the provider's largest `tv_archive_duration` (capped at 30 days), overridable via per-user `epg_prev_days` or `?prev_days=` URL parameter. - **Byte path streams directly via `iter_content` + generator yield** (same pattern as the VOD proxy), with an upfront MPEG-TS sync peek that strips any PHP-warning preamble before bytes reach strict demuxers. Throughput stays comfortably above the typical 5 Mbps FHD bitrate so clients don't buffer. - **Catch-up indicators in Channels and Streams tables.** Channels and streams with catch-up enabled show a grey history icon beside the name (tooltip includes archive days when known). Expanded channel stream rows show a catch-up badge. Channel and stream API serializers expose `is_catchup` and `catchup_days` for the UI. The Channels and Streams table filter menus include **Only Catch-up** to narrow each list to catch-up entries. - **Combined connection stats API.** `GET /proxy/stats/` (admin) returns live, VOD, and catch-up connection stats in one JSON response (`live`, `vod`, `catchup`, `timestamp`). The Stats page polls this endpoint instead of separate live and VOD stats requests. - **Frontend unit tests extended to table components.** `ChannelsTable`, `StreamsTable`, `M3UsTable`, `EPGsTable`, `LogosTable`, `CustomTable`, and related header/editable subcomponents now have Vitest + Testing Library suites. Table business logic was extracted into `frontend/src/utils/tables/*` (with shared `M3uTableUtils` for M3U/EPG sort headers) and covered by dedicated util tests. `dateTimeUtils.formatDuration` gained a `human` precision mode for readable content lengths. — Thanks [@nick4810](https://github.com/nick4810) -- **Frontend unit tests extended to plugin, backup, and settings components.** `AboutModal`, `PluginDetailPanel`, `BackupManager`, `AvailablePluginCard`, `ConnectionSecurityPanel`, and `UserLimitsForm` were refactored with business logic moved into `frontend/src/utils/components/*` and `PluginsUtils`; `pluginUtils` (version comparison, compatibility labels, install/downgrade detection) now lives under `utils/components/`. Vitest + Testing Library suites were added for those components plus `PluginWarnings` and `EpgSettingsForm`. — Thanks [@nick4810](https://github.com/nick4810) +- **Frontend unit tests extended to plugin, backup, and settings components.** `AboutModal`, `PluginDetailPanel`, `BackupManager`, `AvailablePluginCard`, `ConnectionSecurityPanel`, and `UserLimitsForm` were refactored with business logic moved into `frontend/src/utils/components/*` and `PluginsUtils`; `pluginUtils` (version comparison, compatibility labels, install/downgrade detection) now lives under `utils/components/`. Vitest + Testing Library suites were added for those components plus `PluginWarnings`. — Thanks [@nick4810](https://github.com/nick4810) ### Changed diff --git a/apps/channels/tests/test_catchup_utils.py b/apps/channels/tests/test_catchup_utils.py index 1020aa65..31ea7e02 100644 --- a/apps/channels/tests/test_catchup_utils.py +++ b/apps/channels/tests/test_catchup_utils.py @@ -44,13 +44,6 @@ class ResolveXcEpgPrevDaysTests(TestCase): self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 3) mock_compute.assert_not_called() - @patch("apps.channels.utils.compute_provider_archive_days_capped", return_value=14) - @patch("core.models.CoreSettings.get_xmltv_prev_days_override", return_value=7) - def test_epg_settings_override_prevents_auto_detect(self, _override, mock_compute): - request = self.factory.get("/xmltv.php") - self.assertEqual(resolve_xc_epg_prev_days(request, self.user), 7) - mock_compute.assert_not_called() - class CatchupRollupActiveAccountTests(TestCase): """Denormalized catch-up flags ignore disabled M3U accounts.""" diff --git a/apps/channels/utils.py b/apps/channels/utils.py index 70b9eecd..e5f9b08c 100644 --- a/apps/channels/utils.py +++ b/apps/channels/utils.py @@ -67,8 +67,7 @@ def resolve_xc_epg_prev_days(request, user, *, auto_detect_fallback=True): Resolution order: 1. URL ``?prev_days=`` (explicit; 0 means no past programmes) 2. ``user.custom_properties.epg_prev_days`` - 3. ``CoreSettings.epg_settings.xmltv_prev_days_override`` when > 0 - 4. Auto-detect (only when *auto_detect_fallback* is True) + 3. Auto-detect (only when *auto_detect_fallback* is True) """ user_custom = (user.custom_properties or {}) if user else {} url_prev = request.GET.get("prev_days") @@ -85,14 +84,6 @@ def resolve_xc_epg_prev_days(request, user, *, auto_detect_fallback=True): except (ValueError, TypeError): return 0 - from core.models import CoreSettings - - try: - override = int(CoreSettings.get_xmltv_prev_days_override() or 0) - except (TypeError, ValueError): - override = 0 - if override > 0: - return max(0, min(override, MAX_AUTO_PREV_DAYS)) if auto_detect_fallback: return compute_provider_archive_days_capped() return 0 diff --git a/apps/timeshift/helpers.py b/apps/timeshift/helpers.py index 207c2bc8..30e34222 100644 --- a/apps/timeshift/helpers.py +++ b/apps/timeshift/helpers.py @@ -1,6 +1,7 @@ """URL builders and timestamp helpers for XC catch-up.""" import logging +import math import re from collections import namedtuple from datetime import datetime, timezone @@ -339,3 +340,51 @@ def format_timestamp_as_underscore(timestamp): def format_timestamp_as_sql_datetime(timestamp): """Reshape to ``YYYY-MM-DD HH:MM:SS`` without timezone conversion.""" return _reshape_timestamp(timestamp, "%Y-%m-%d %H:%M:%S", "SQL") + + +def programme_age_days(timestamp_str, *, now=None): + """Archive depth in whole days needed to cover a catch-up start timestamp. + + Returns: + ``None`` if the timestamp cannot be parsed, ``0`` if start is at/after + *now*, otherwise ``ceil(elapsed / 86400)`` (at least 1). + """ + dt = parse_catchup_timestamp(timestamp_str) + if dt is None: + return None + if now is None: + now = datetime.now(timezone.utc).replace(tzinfo=None) + elif getattr(now, "tzinfo", None) is not None: + now = now.astimezone(timezone.utc).replace(tzinfo=None) + + elapsed = (now - dt).total_seconds() + if elapsed <= 0: + return 0 + return max(1, math.ceil(elapsed / 86400.0)) + + +def order_catchup_streams_for_timestamp(streams, timestamp_str, *, now=None): + """Prefer streams whose ``catchup_days`` cover the programme age. + + Relative channel order is preserved within the preferred and fallback + groups. Streams with unknown/zero ``catchup_days`` stay preferred so + incomplete provider metadata does not skip a possible server. Unparseable + timestamps leave the input order unchanged. + """ + age = programme_age_days(timestamp_str, now=now) + if age is None: + return list(streams) + + preferred = [] + fallback = [] + for stream in streams: + raw = getattr(stream, "catchup_days", None) + try: + days = int(raw) if raw is not None else 0 + except (TypeError, ValueError): + days = 0 + if days <= 0 or days >= age: + preferred.append(stream) + else: + fallback.append(stream) + return preferred + fallback diff --git a/apps/timeshift/tests/test_helpers.py b/apps/timeshift/tests/test_helpers.py index 882e01c8..5a4deed7 100644 --- a/apps/timeshift/tests/test_helpers.py +++ b/apps/timeshift/tests/test_helpers.py @@ -15,7 +15,9 @@ from apps.timeshift.helpers import ( format_timestamp_as_sql_datetime, format_timestamp_as_underscore, normalize_catchup_timestamp_input, + order_catchup_streams_for_timestamp, parse_catchup_timestamp, + programme_age_days, ) @@ -335,3 +337,78 @@ class GetProgrammeDurationTests(TestCase): from unittest.mock import MagicMock from apps.timeshift.helpers import get_programme_duration self.assertEqual(get_programme_duration(MagicMock(), "garbage"), 120) + + +class ProgrammeAgeAndStreamOrderTests(TestCase): + """Archive-age helpers used to prefer deep catch-up providers first.""" + + def test_programme_age_days_ceil(self): + now = datetime(2026, 7, 16, 12, 0, 0) + self.assertEqual( + programme_age_days("2026-07-12:12-00", now=now), + 4, + ) + self.assertEqual( + programme_age_days("2026-07-15:12-00", now=now), + 1, + ) + self.assertEqual( + programme_age_days("2026-07-16:12-00", now=now), + 0, + ) + + def test_programme_age_days_unparseable(self): + self.assertIsNone(programme_age_days("garbage")) + + def test_order_prefers_covering_streams_then_fallback(self): + from types import SimpleNamespace + + now = datetime(2026, 7, 16, 12, 0, 0) + streams = [ + SimpleNamespace(catchup_days=2, name="p1"), + SimpleNamespace(catchup_days=1, name="p2"), + SimpleNamespace(catchup_days=5, name="p3"), + ] + ordered = order_catchup_streams_for_timestamp( + streams, "2026-07-12:12-00", now=now + ) + self.assertEqual([s.name for s in ordered], ["p3", "p1", "p2"]) + + def test_order_keeps_channel_order_within_groups(self): + from types import SimpleNamespace + + now = datetime(2026, 7, 16, 12, 0, 0) + streams = [ + SimpleNamespace(catchup_days=7, name="a"), + SimpleNamespace(catchup_days=2, name="b"), + SimpleNamespace(catchup_days=14, name="c"), + SimpleNamespace(catchup_days=1, name="d"), + ] + ordered = order_catchup_streams_for_timestamp( + streams, "2026-07-12:12-00", now=now + ) + self.assertEqual([s.name for s in ordered], ["a", "c", "b", "d"]) + + def test_unknown_catchup_days_stay_preferred(self): + from types import SimpleNamespace + + now = datetime(2026, 7, 16, 12, 0, 0) + streams = [ + SimpleNamespace(catchup_days=2, name="short"), + SimpleNamespace(catchup_days=0, name="unknown"), + SimpleNamespace(catchup_days=5, name="deep"), + ] + ordered = order_catchup_streams_for_timestamp( + streams, "2026-07-12:12-00", now=now + ) + self.assertEqual([s.name for s in ordered], ["unknown", "deep", "short"]) + + def test_unparseable_timestamp_preserves_order(self): + from types import SimpleNamespace + + streams = [ + SimpleNamespace(catchup_days=2, name="a"), + SimpleNamespace(catchup_days=5, name="b"), + ] + ordered = order_catchup_streams_for_timestamp(streams, "garbage") + self.assertEqual([s.name for s in ordered], ["a", "b"]) diff --git a/apps/timeshift/tests/test_views.py b/apps/timeshift/tests/test_views.py index 04302613..408eedea 100644 --- a/apps/timeshift/tests/test_views.py +++ b/apps/timeshift/tests/test_views.py @@ -531,12 +531,12 @@ class RedactUrlTests(TestCase): def _make_catchup_stream(provider_tz="Europe/Brussels", *, account_id=9, stream_id="22372", account_type="XC", profile_id=31, - extra_profiles=()): + extra_profiles=(), catchup_days=0): """Build a mocked catch-up Stream with its own provider context. The default (tz-bearing) profile leads the active-profile list the view walks; ``extra_profiles`` appends alternate (non-default) profiles for - capacity-walk tests. + capacity-walk tests. ``catchup_days`` defaults to 0 (unknown archive depth). """ profile = MagicMock() profile.id = profile_id @@ -550,6 +550,7 @@ def _make_catchup_stream(provider_tz="Europe/Brussels", *, account_id=9, stream.m3u_account = m3u_account stream.m3u_account_id = account_id stream.custom_properties = {"stream_id": stream_id} if stream_id else {} + stream.catchup_days = catchup_days return stream @@ -1022,6 +1023,38 @@ class TimeshiftProxyFailoverTests(TestCase): self.assertIs(response, passthrough) self.assertEqual(stream_mock.call_count, 1) + @patch("apps.timeshift.helpers.programme_age_days", return_value=4) + def test_prefers_stream_with_sufficient_catchup_days(self, _age): + """Deep archive is tried first when earlier streams are too short.""" + streams = [ + _make_catchup_stream(account_id=1, stream_id="111", catchup_days=2), + _make_catchup_stream(account_id=2, stream_id="222", catchup_days=1), + _make_catchup_stream(account_id=3, stream_id="333", catchup_days=5), + ] + ok = MagicMock(status_code=200) + response, stream_mock, build_mock, _ = self._call(streams, [ok]) + self.assertIs(response, ok) + self.assertEqual(stream_mock.call_count, 1) + self.assertEqual(stream_mock.call_args.kwargs["account_id"], 3) + self.assertEqual(build_mock.call_args.args[1], "333") + + @patch("apps.timeshift.helpers.programme_age_days", return_value=4) + def test_falls_back_to_shorter_archives_when_preferred_fail(self, _age): + streams = [ + _make_catchup_stream(account_id=1, stream_id="111", catchup_days=2), + _make_catchup_stream(account_id=2, stream_id="222", catchup_days=1), + _make_catchup_stream(account_id=3, stream_id="333", catchup_days=5), + ] + ok = MagicMock(status_code=200) + response, stream_mock, _, _ = self._call( + streams, [MagicMock(status_code=404), ok] + ) + self.assertIs(response, ok) + self.assertEqual( + [c.kwargs["account_id"] for c in stream_mock.call_args_list], + [3, 1], + ) + class _ProxyLoopTestMixin: """Shared driver for tests exercising the failover loop end to end - diff --git a/apps/timeshift/views.py b/apps/timeshift/views.py index c83c6100..120a168a 100644 --- a/apps/timeshift/views.py +++ b/apps/timeshift/views.py @@ -59,6 +59,7 @@ from .helpers import ( build_timeshift_candidate_urls, convert_timestamp_to_provider_tz, get_programme_duration, + order_catchup_streams_for_timestamp, parse_catchup_timestamp, ) from .sessions import catchup_session_exists, delete_catchup_session, resolve_catchup_playback @@ -298,6 +299,8 @@ def _serve_catchup(request, user, channel, timestamp): HttpResponseBadRequest("Timeshift not supported for this channel") ) + catchup_streams = order_catchup_streams_for_timestamp(catchup_streams, timestamp) + debug = logger.isEnabledFor(logging.DEBUG) # EPG duration lookup stays in UTC; provider TZ conversion is per-attempt below. diff --git a/core/models.py b/core/models.py index 676ef02d..ad9677c6 100644 --- a/core/models.py +++ b/core/models.py @@ -280,18 +280,8 @@ class CoreSettings(models.Model): "epg_match_ignore_prefixes": [], "epg_match_ignore_suffixes": [], "epg_match_ignore_custom": [], - # XC catch-up: forced XMLTV lookback (0 = auto-detect, capped at 30). - "xmltv_prev_days_override": 0, }) - @classmethod - def get_xmltv_prev_days_override(cls): - """Global XC XMLTV prev_days default (0 = auto-detect from provider archives).""" - try: - return int(cls.get_epg_settings().get("xmltv_prev_days_override", 0) or 0) - except (TypeError, ValueError): - return 0 - @classmethod def _safe_string_list(cls, value): """Return a list of strings, filtering out non-list or non-string values.""" diff --git a/frontend/src/components/forms/settings/EpgSettingsForm.jsx b/frontend/src/components/forms/settings/EpgSettingsForm.jsx deleted file mode 100644 index 5af4f796..00000000 --- a/frontend/src/components/forms/settings/EpgSettingsForm.jsx +++ /dev/null @@ -1,74 +0,0 @@ -import useSettingsStore from '../../../store/settings.jsx'; -import React, { useEffect, useState } from 'react'; -import { useForm } from '@mantine/form'; -import { Alert, Button, Flex, NumberInput, Stack, Text } from '@mantine/core'; -import { EPG_SETTINGS_OPTIONS } from '../../../constants.js'; -import { - getChangedSettings, - parseSettings, - saveChangedSettings, -} from '../../../utils/pages/SettingsUtils.js'; -import { getEpgSettingsFormInitialValues } from '../../../utils/forms/settings/EpgSettingsFormUtils.js'; - -const EpgSettingsForm = React.memo(({ active }) => { - const settings = useSettingsStore((s) => s.settings); - const [saved, setSaved] = useState(false); - - const form = useForm({ - mode: 'controlled', - initialValues: getEpgSettingsFormInitialValues(), - }); - - useEffect(() => { - if (!active) setSaved(false); - }, [active]); - - useEffect(() => { - if (settings) { - const parsed = parseSettings(settings); - form.setFieldValue( - 'xmltv_prev_days_override', - parsed.xmltv_prev_days_override ?? 0 - ); - } - }, [settings]); - - const onSubmit = async () => { - setSaved(false); - const changedSettings = getChangedSettings(form.getValues(), settings); - try { - await saveChangedSettings(settings, changedSettings); - setSaved(true); - } catch (error) { - console.error('Error saving EPG settings:', error); - } - }; - - const prevDaysConfig = EPG_SETTINGS_OPTIONS.xmltv_prev_days_override; - - return ( -
- ); -}); - -export default EpgSettingsForm; diff --git a/frontend/src/components/forms/settings/__tests__/EpgSettingsForm.test.jsx b/frontend/src/components/forms/settings/__tests__/EpgSettingsForm.test.jsx deleted file mode 100644 index 88bac9fb..00000000 --- a/frontend/src/components/forms/settings/__tests__/EpgSettingsForm.test.jsx +++ /dev/null @@ -1,417 +0,0 @@ -import { render, screen, fireEvent, waitFor } from '@testing-library/react'; -import { describe, it, expect, vi, beforeEach } from 'vitest'; -import EpgSettingsForm from '../EpgSettingsForm'; - -// ── Store mock ───────────────────────────────────────────────────────────────── -vi.mock('../../../../store/settings.jsx', () => ({ default: vi.fn() })); - -// ── @mantine/form mock ──────────────────────────────────────────────────────── -vi.mock('@mantine/form', () => ({ useForm: vi.fn() })); - -// ── @mantine/core mock ──────────────────────────────────────────────────────── -vi.mock('@mantine/core', () => ({ - Alert: ({ title, color, variant }) => ( -