Previously only UnicodeDecodeError/UnicodeEncodeError were caught. Unexpected errors like disk full or permission denied would propagate
and crash the EPG refresh, which would have succeeded before the preprocessing step existed. Now all exceptions are caught and logged, leaving the original file untouched for lxml to handle as before.
lxml 6.0.2 with recover=True silently drops HTML named entities (e.g. é, î) that are not valid XML, causing channel names and program data to lose characters during EPG ingestion.
- Add preprocessing step in fetch_xmltv() that converts HTML named entities to Unicode while preserving XML-predefined entities
- Detect encoding from XML declaration, fall back to UTF-8 for unknown codecs, skip files that fail to decode cleanly
- Guard against entities resolving to XML-special characters and html.unescape partial matching edge cases
- Add 28 unit tests covering entity resolution, encoding detection, and file-level processing
Replace DRF ModelSerializer with .values() + plain dict construction in
EPGGridAPIView, eliminating per-row serializer instantiation, field
object creation, and OrderedDict overhead across ~12k programs.
Pre-compute season/episode at import time instead of on every API request:
- Extract S/E from onscreen episode-num inline in extract_custom_properties()
immediately after storing the raw value, rather than in a separate pass
- Add missing description-based S/E fallback to parse_programs_for_tvg_id(),
bringing it in line with parse_programs_for_source()
Remove runtime extract_season_episode() call from ProgramDataSerializer and
EPGGridAPIView — values are now guaranteed to be in custom_properties after
import, so season/episode resolution is a plain dict lookup at request time.
Also remove unnecessary select_related("epg") JOIN and .count() query from
the grid endpoint.
Replace expanded inline cards with a ProgramDetailModal for viewing full program
details. Add real-time progress bars to currently-airing programs using direct DOM manipulation. Extract and display season/episode info from EPG data with a 3-tier fallback (custom_properties → onscreen_episode → description text).
Backend:
- Add ProgramDetailSerializer for rich EPG data (categories, credits, ratings, images, external IDs) served on the detail endpoint
- Add infer_is_live() for status badges (live, new, premiere, finale)
- Add shared S/E extraction utilities in apps/epg/utils.py
- Optimize querysets with select_related("epg")
Frontend:
- ProgramDetailModal with conditional field rendering and image fallback chain
- Progress bars via scaleX transform updated every second
- Parallel data fetching with Promise.all for channels + programs
- Browser tab visibility recovery for progress bars
- Now-line with triangle marker and sub-millisecond precision
- Card layout: title, season/episode + subtitle, time + badges
- Shared formatSeasonEpisode() and external URL helpers
Add is_new, is_live, is_premiere, and is_finale fields to the EPG program API and render them as colored badges in the TV guide.
- Compact card: S/E, LIVE, and NEW badges on the title row
- Expanded card: all badges (S/E, LIVE, NEW, PREMIERE, FINALE) on the subtitle row, now moved directly after title for consistency
- PREMIERE/FINALE detected from premiere_text (case-insensitive)
- PREMIERE/FINALE only shown in expanded view to conserve space
- Badge order: S/E → LIVE → NEW → PREMIERE → FINALE
- Consolidate serializer season/episode parsing into a single to_representation
pass: one custom_properties read, regex runs at most once per program
- Compact card: S/E badge inline with title; secondary row shows sub_title with
description fallback
- Expanded card: sub_title and S/E badge share a row above the description
- Fix secondary text color for legibility on dark backgrounds (#a0aec0 → #e2e8f0)
- Fix expanded card overflow (height:100% → flex:1 1 0) so action buttons stay visible
- Bump expanded card height 180 → 200px, description max-height 80 → 100px
Display sub_title (episode name) and season/episode badge on guide program cards. Extract season/episode from custom_properties with fallback parsing of onscreen_episode format (e.g. "S12 E6").
- Add season/episode SerializerMethodFields to ProgramDataSerializer
- Parse onscreen_episode string as fallback when xmltv_ns data absent
- Display sub_title with S/E badge on guide cards
- Show description for programs without sub_title (movies)
- Add formatSeasonEpisode utility with full test coverage
- Add 15 backend serializer tests and 12 frontend utility tests
Fulfills #1065
Large M3U downloads that exceed the 300s Redis lock TTL caused Celery Beat
to re-queue duplicate tasks, creating overlapping downloads that never complete.
Files changed:
- core/utils.py: Add TaskLockRenewer class — a daemon thread that periodically renews Redis lock TTL (every 120s) to prevent expiry during long-running tasks.
- apps/m3u/tasks.py: Apply TaskLockRenewer to refresh_single_m3u_account and refresh_m3u_groups; add HTTP timeout (30s connect, 60s read) to M3U download (the only download path missing one); stream M3U download to disk instead of accumulating in memory; add Celery task time limits (1 hour hard limit).
- apps/epg/tasks.py: Apply TaskLockRenewer to refresh_epg_data and parse_programs_for_tvg_id; add Celery task time limits (30 min / 1 hour).
Verified with a throttled test server: locks renewed at T+120s, T+240s, T+360s; 50,000 streams processed with no duplicate tasks.
Add cron expression support for M3U and EPG refreshes with interactive
builder modal and quick reference examples. Refactor backup scheduling
to use shared ScheduleInput component for consistency.
- New CronBuilder modal with preset buttons and custom field editors
- Info popover with common cron expression examples
- Shared ScheduleInput component for interval/cron toggle
- Native Bearer token authentication support in Swagger UI - users can now enter just the JWT token and the "Bearer " prefix is automatically added
- Modern OpenAPI 3.0 specification compliance
- Better auto-generation of request/response schemas
- Improved documentation accuracy with serializer introspection
- Currently playing program title displayed with live broadcast indicator (green Radio icon)
- Expandable program descriptions via chevron button
- Efficient POST-based API endpoint (`/api/epg/current-programs/`) supporting batch channel queries or fetching all channels
- Smart scheduling that fetches new program data 5 seconds after current program ends
- Only polls when active channel list changes, not on stats refresh
- Channel preview button: Added preview functionality to active stream cards on stats page
Dramatically improve EPG refresh performance by parsing the XML file once
per source instead of once per channel. The new implementation:
- Pre-filters to only process EPG entries mapped to actual channels
- Parses the entire XML file in a single pass
- Uses O(1) set lookups to skip unmapped channel programmes
- Skips non-mapped channels entirely with minimal overhead
For EPG sources with many channels but few mapped (e.g., 10,000 channels
with 100 mapped to channels), this provides approximately:
- 99% reduction in file open operations
- 99% reduction in XML file scans
- Proportional reduction in CPU and I/O overhead
The parse_programs_for_tvg_id() function is retained for single-channel
use cases (e.g., when a new channel is mapped via signals).
Fixes inefficient repeated file parsing that was occurring with large
EPG sources.
System Event Logging:
- Add SystemEvent model with 15 event types tracking channel operations, client connections, M3U/EPG activities, and buffering events
- Log detailed metrics for M3U/EPG refresh operations (streams/programs created/updated/deleted)
- Track M3U/EPG downloads with client information (IP address, user agent, profile, channel count)
- Record channel lifecycle events (start, stop, reconnect) with stream and client details
- Monitor client connections/disconnections and buffering events with stream metadata
Event Viewer UI:
- Add SystemEvents component with real-time updates via WebSocket
- Implement pagination, filtering by event type, and configurable auto-refresh
- Display events with color-coded badges and type-specific icons
- Integrate event viewer into Stats page with modal display
- Add event management settings (retention period, refresh rate)
M3U/EPG Endpoint Optimizations:
- Implement content caching with 5-minute TTL to reduce duplicate processing
- Add client-based event deduplication (2-second window) using IP and user agent hashing
- Support HEAD requests for efficient preflight checks
- Cache streamed EPG responses while maintaining streaming behavior for first request
This enhancement introduces a powerful custom dummy EPG system that allows users to generate EPG programs on-demand by parsing channel or stream names using configurable regex patterns.
Key Features:
- Custom Pattern Matching: Define regex patterns to extract information from channel/stream names (teams, leagues, times, dates, etc.)
- Flexible Name Source: Choose to parse either the channel name or a specific stream name (by index)
- Timezone-Aware Scheduling: Automatic DST handling using pytz timezone names (e.g., 'US/Eastern', 'Europe/London')
- Time Format Support: Parse both 12-hour (AM/PM) and 24-hour time formats
- Date Parsing: Extract dates from names with flexible month/day/year patterns
- Custom Templates: Format EPG titles and descriptions using captured groups with {placeholder} syntax
- Upcoming/Ended Customization: Define custom titles and descriptions for programs before and after scheduled events
- Live Preview: Test patterns and templates in real-time with sample input
- Smart Program Generation: Automatically creates "Upcoming" and "Ended" programs around scheduled events
Use Cases:
- Sports channels with event details in stream names (e.g., "NHL 01: Bruins VS Leafs @ 8:00PM ET")
- Movie channels with genre/title/year information
- Racing events with driver/track/series details
- Any scenario where EPG data is embedded in channel/stream naming conventions
Technical Implementation:
- Backend: Pattern matching engine with timezone conversion and program scheduling logic
- Frontend: Interactive form with validation, pattern testing, and visual group preview
- Name Source Options: Parse from channel name or selectable stream index (1-based)
- Fallback Behavior: Uses standard dummy EPG if patterns don't match
- Custom Properties: Stores all configuration in EPGSource.custom_properties JSON field
Configuration Options:
- Title Pattern: Extract primary information (required)
- Time Pattern: Extract hour/minute/AM-PM (optional)
- Date Pattern: Extract month/day/year (optional)
- Timezone: Event timezone with automatic DST support
- Program Duration: Length of generated programs in minutes
- Title Template: Format EPG title using captured groups
- Description Template: Format EPG description using captured groups
- Upcoming Title Template: Custom title for programs before event starts (optional)
- Upcoming Description Template: Custom description for programs before event starts (optional)
- Ended Title Template: Custom title for programs after event ends (optional)
- Ended Description Template: Custom description for programs after event ends (optional)
- Name Source: Channel name or stream name
- Stream Index: Which stream to use when parsing stream names (1, 2, 3, etc.)
Closes#293
