- Replace per-entry ProgramData DB query with single batch fetch
- Pass pre-loaded EPGData objects to find_current_program_for_tvg_id
to avoid redundant select_related re-fetch
- Delete Redis programme index before XMLTV download so stale byte
offsets are never used during the refresh window
When selecting an EPG channel in the channel form, if that EPG entry's
programs had never been parsed, the preview showed "No current program".
This was because parse_programs_for_tvg_id requires a linked channel.
- Add force parameter to parse_programs_for_tvg_id to bypass channel check
- Auto-trigger parsing from current-programs API when no programs exist
- Check Redis task lock to prevent duplicate parse tasks from concurrent requests
- Frontend retries up to 3 times with 10s delay to pick up parsed results
- Skip parsing for dummy EPG sources
- Extended CurrentProgramsAPIView to accept epg_data_ids parameter
- Added getCurrentProgramForEpg() API method in frontend
- Added UI in Channel form showing current program with tooltip
- Shows loading state, no program state, and program details
- Includes expandable description, progress bar with elapsed/remaining time
- Matches stats page 'Now Playing' feature behavior
- Added support for customizable title and description matching modes in series rules.
- Implemented a preview feature for series rules to show matching upcoming programs.
- Created a new SeriesRuleEditorModal for editing series rules with improved UI.
- Refactored query parsing logic into reusable functions for better maintainability.
- Updated API to handle new parameters for series rule creation and evaluation.
- Enhanced the ProgramRecordingModal and SeriesRecordingModal to integrate the new rule editor.
- Added pagination for program search results in the API.
- Fix test user: set user_level=1 (STANDARD), REMOTE_ADDR="127.0.0.1" to
satisfy IsStandardUser + network_access_allowed checks; move user creation
to setUpTestData (1 DB write per class instead of per test)
- Fix parenthesis parser: group_result was computed then silently discarded;
rewrite parse_expression to recursively compose Q objects correctly so
queries like "(Newcastle OR NEW) AND (Villa OR AST)" work as documented
- Fix field selection performance: resolve allowed fields before serialization
and pass as context; get_channels/get_streams short-circuit when not requested
- Move import re to top of file; remove mid-file alias regex_module
- Convert ProgramSearchAPIView standalone APIView into @action(detail=False)
on ProgramViewSet so the router registers programs/search/ before
programs/{pk}/ automatically, eliminating the URL ordering dependency
- Scope permission to the action directly via permission_classes=[IsStandardUser]
rather than adding "search" to the global permission_classes_by_action dict
- Add test_title_parenthetical_grouping to cover the fixed parser path
- Tighten existing test assertions (case-insensitive, whole-word, description
AND, page_size cap) to catch regressions rather than passing vacuously
Co-authored-by: Copilot <copilot@github.com>
- Performance: `xc_get_epg` now uses `select_related('epg_data__epg_source')` on all three channel fetch paths. Previously each request triggered 2 additional queries to resolve `channel.epg_data` and `channel.epg_data.epg_source`.
- Performance: `generate_m3u` now uses `prefetch_related` for streams when `?direct=true` is requested, eliminating N+1 stream queries (one per channel) on that code path.
- Performance: `EPGGridAPIView` (`apps/epg/api_views.py`) now uses `select_related('epg_data__epg_source')` on the `channels_with_custom_dummy` queryset, eliminating 2 extra queries per channel (for `epg_data` and `epg_source`) in the dummy EPG generation loop.
- EPG refresh tasks (`refresh_epg_data`) were being killed mid-transaction on large EPG sources. The `soft_time_limit=1700s` introduced in v0.21.0 raised `SoftTimeLimitExceeded`, a subclass of `Exception`, which was swallowed by the existing `except Exception` handler in `parse_programs_for_source`, leaving the database in a partial state with no logged error. The underlying cause is that the HTML entity preprocessing step added in v0.21.0 adds 2–10+ minutes on multi-gigabyte EPG files, pushing total task duration past the 1700s limit for large sources. `soft_time_limit` has been removed from `refresh_epg_data` and `time_limit` raised to 14400s (4 hours) as a true last-resort ceiling; the existing `TaskLockRenewer` daemon thread continues to renew the Redis lock every 120s for legitimately long-running tasks.
- HTML entity preprocessing (`_resolve_html_entities`) now performs a fast binary pre-scan before any file read or write. The pre-scan reads the file in 4 MB chunks without text decoding and exits immediately on the first non-XML named entity found. For EPG sources that contain no HTML entities — the common case — the full line-by-line read+rewrite pass (which takes 2–10+ minutes on a 2.5 GB file) is skipped entirely.
Previously only UnicodeDecodeError/UnicodeEncodeError were caught. Unexpected errors like disk full or permission denied would propagate
and crash the EPG refresh, which would have succeeded before the preprocessing step existed. Now all exceptions are caught and logged, leaving the original file untouched for lxml to handle as before.
lxml 6.0.2 with recover=True silently drops HTML named entities (e.g. é, î) that are not valid XML, causing channel names and program data to lose characters during EPG ingestion.
- Add preprocessing step in fetch_xmltv() that converts HTML named entities to Unicode while preserving XML-predefined entities
- Detect encoding from XML declaration, fall back to UTF-8 for unknown codecs, skip files that fail to decode cleanly
- Guard against entities resolving to XML-special characters and html.unescape partial matching edge cases
- Add 28 unit tests covering entity resolution, encoding detection, and file-level processing
Replace DRF ModelSerializer with .values() + plain dict construction in
EPGGridAPIView, eliminating per-row serializer instantiation, field
object creation, and OrderedDict overhead across ~12k programs.
Pre-compute season/episode at import time instead of on every API request:
- Extract S/E from onscreen episode-num inline in extract_custom_properties()
immediately after storing the raw value, rather than in a separate pass
- Add missing description-based S/E fallback to parse_programs_for_tvg_id(),
bringing it in line with parse_programs_for_source()
Remove runtime extract_season_episode() call from ProgramDataSerializer and
EPGGridAPIView — values are now guaranteed to be in custom_properties after
import, so season/episode resolution is a plain dict lookup at request time.
Also remove unnecessary select_related("epg") JOIN and .count() query from
the grid endpoint.
Replace expanded inline cards with a ProgramDetailModal for viewing full program
details. Add real-time progress bars to currently-airing programs using direct DOM manipulation. Extract and display season/episode info from EPG data with a 3-tier fallback (custom_properties → onscreen_episode → description text).
Backend:
- Add ProgramDetailSerializer for rich EPG data (categories, credits, ratings, images, external IDs) served on the detail endpoint
- Add infer_is_live() for status badges (live, new, premiere, finale)
- Add shared S/E extraction utilities in apps/epg/utils.py
- Optimize querysets with select_related("epg")
Frontend:
- ProgramDetailModal with conditional field rendering and image fallback chain
- Progress bars via scaleX transform updated every second
- Parallel data fetching with Promise.all for channels + programs
- Browser tab visibility recovery for progress bars
- Now-line with triangle marker and sub-millisecond precision
- Card layout: title, season/episode + subtitle, time + badges
- Shared formatSeasonEpisode() and external URL helpers
Add is_new, is_live, is_premiere, and is_finale fields to the EPG program API and render them as colored badges in the TV guide.
- Compact card: S/E, LIVE, and NEW badges on the title row
- Expanded card: all badges (S/E, LIVE, NEW, PREMIERE, FINALE) on the subtitle row, now moved directly after title for consistency
- PREMIERE/FINALE detected from premiere_text (case-insensitive)
- PREMIERE/FINALE only shown in expanded view to conserve space
- Badge order: S/E → LIVE → NEW → PREMIERE → FINALE
- Consolidate serializer season/episode parsing into a single to_representation
pass: one custom_properties read, regex runs at most once per program
- Compact card: S/E badge inline with title; secondary row shows sub_title with
description fallback
- Expanded card: sub_title and S/E badge share a row above the description
- Fix secondary text color for legibility on dark backgrounds (#a0aec0 → #e2e8f0)
- Fix expanded card overflow (height:100% → flex:1 1 0) so action buttons stay visible
- Bump expanded card height 180 → 200px, description max-height 80 → 100px
Display sub_title (episode name) and season/episode badge on guide program cards. Extract season/episode from custom_properties with fallback parsing of onscreen_episode format (e.g. "S12 E6").
- Add season/episode SerializerMethodFields to ProgramDataSerializer
- Parse onscreen_episode string as fallback when xmltv_ns data absent
- Display sub_title with S/E badge on guide cards
- Show description for programs without sub_title (movies)
- Add formatSeasonEpisode utility with full test coverage
- Add 15 backend serializer tests and 12 frontend utility tests
Fulfills #1065
Large M3U downloads that exceed the 300s Redis lock TTL caused Celery Beat
to re-queue duplicate tasks, creating overlapping downloads that never complete.
Files changed:
- core/utils.py: Add TaskLockRenewer class — a daemon thread that periodically renews Redis lock TTL (every 120s) to prevent expiry during long-running tasks.
- apps/m3u/tasks.py: Apply TaskLockRenewer to refresh_single_m3u_account and refresh_m3u_groups; add HTTP timeout (30s connect, 60s read) to M3U download (the only download path missing one); stream M3U download to disk instead of accumulating in memory; add Celery task time limits (1 hour hard limit).
- apps/epg/tasks.py: Apply TaskLockRenewer to refresh_epg_data and parse_programs_for_tvg_id; add Celery task time limits (30 min / 1 hour).
Verified with a throttled test server: locks renewed at T+120s, T+240s, T+360s; 50,000 streams processed with no duplicate tasks.
- Add proper datetime validation in API with 400 error responses
- Fix test timestamp format (was creating invalid +00:00Z format)
- Time filters now properly validate and return errors on invalid input
- All 26 tests passing with correct filtering behavior
