- Moved the `programme_index` from the `EPGSource` model to a new `EPGSourceIndex` table, ensuring that the large JSON blob is only loaded when explicitly accessed, thus improving query performance and memory efficiency.
- Updated related queries and API views to utilize the new structure, including adjustments to EPG generation and import logic to prevent unnecessary data loading.
- Enhanced memory management in the EPG grid endpoint to reduce worker RSS during response handling.
- Implemented robust DB connection management in `refresh_epg_data` to handle transient errors, ensuring reliable status updates for EPG sources.
- Added retry logic for database queries and improved error handling to prevent task failures from unhandled exceptions.
- Updated EPG source status handling to ensure accurate reporting of errors and terminal states during refresh operations.
- Introduced a temporary staging table for efficient batch processing of EPG program inserts, reducing memory and I/O contention during updates.
- Enhanced the `parse_programs_for_source` function to stream parsed rows into the staging table before swapping them atomically into the main program data.
- Added unit tests to validate the new staging and swapping logic, ensuring existing programs are preserved during failures and that batch processing works as intended.
- Replaced individual EPG program parse tasks with a centralized dispatch function to streamline guide refresh for newly assigned EPG IDs.
- Implemented batching for guide fetches when multiple EPGs are mapped, reducing redundant API calls and improving efficiency.
- Updated related utility functions to support the new fetching strategy and added tests to ensure correct behavior under various scenarios.
- Implemented a targeted guide fetch for Schedules Direct when mapping a channel, ensuring immediate data availability without redundant API calls.
- Updated EPG signals to queue guide fetches for newly mapped channels, streamlining the process and improving user experience.
- Refactored existing logic to skip unnecessary program refreshes for dummy sources and ensure proper handling of Schedules Direct data.
- Added unit tests to validate the new fetching behavior and ensure robustness of the integration.
- Added functions to compute schedule changes based on MD5 hashes and backfill missing dates for mapped stations.
- Enhanced handling of orphaned ProgramData records for unmapped EPG entries during schedule fetch.
- Introduced unit tests to validate MD5 comparison, backfill logic, and metadata fetching requirements.
- Addressed a bug where the EPG channel list did not refresh after a source finished parsing channels due to the `epg_refresh` WebSocket handler using a stale snapshot. The handler now correctly reads the current store state and ensures `fetchEPGData()` is called upon completion of channel parsing.
- Updated tests to verify that the frontend receives the correct parsing completion signals.
Introduce a new poster style selection feature in the EPG settings, allowing users to choose their preferred artwork style. Update the logic for fetching and displaying program artwork from Schedules Direct, ensuring that the selected style is honored with appropriate fallbacks. Set default poster art to 'primary'. Add comprehensive tests to validate the poster selection functionality across various scenarios.
- Rename api_key to password (RemoveField + AddField) for clarity and reusability
- Move sd_changes_remaining/sd_changes_reset_at to custom_properties JSONField
instead of nullable columns on all EPGSource rows
- Replace sd_program_md5 column on ProgramData with SDProgramMD5 relation table
keyed by epg_source + program_id, fixing the key mismatch bug in program
MD5 delta detection
- Fix total_changed counter: .items() -> .values() so len() returns date count
not tuple length
- Add missing SD lineup management endpoints to api_views.py
(sd_lineups GET/POST/DELETE, sd_lineups_search POST)
- Regenerate migration 0023_schedules_direct using Django to ensure correct
index names and schema
- Add EPGSource username field and SDScheduleMD5 model for MD5 delta sync
- Implement full Schedules Direct fetch with SHA1 auth, token caching,
20-day schedule window, and MD5-based delta sync to minimize API usage
- Add lineup manager UI allowing users to search and manage SD lineups
- Add stations-only fetch on source creation for immediate EPG matching
- Enforce 2-hour minimum refresh interval to respect SD rate limits,
with first-fetch bypass when no prior full refresh exists
- Add real-time progress updates via synchronous Redis WebSocket delivery,
bypassing gevent.spawn for reliable delivery from Celery prefork workers
- Refactor EPGsTable status cell into standalone EPGStatusCell component
with direct Zustand subscription for live progress bar updates
- Guard XMLTV parse trigger for channels linked to SD EPG sources
- Add 15 backend tests covering auth, credentials, signals, and serializers
Migrations: 0023 (username field), 0024 (api_key help_text), 0025 (SD fields + SDScheduleMD5)
- Fix test user: set user_level=1 (STANDARD), REMOTE_ADDR="127.0.0.1" to
satisfy IsStandardUser + network_access_allowed checks; move user creation
to setUpTestData (1 DB write per class instead of per test)
- Fix parenthesis parser: group_result was computed then silently discarded;
rewrite parse_expression to recursively compose Q objects correctly so
queries like "(Newcastle OR NEW) AND (Villa OR AST)" work as documented
- Fix field selection performance: resolve allowed fields before serialization
and pass as context; get_channels/get_streams short-circuit when not requested
- Move import re to top of file; remove mid-file alias regex_module
- Convert ProgramSearchAPIView standalone APIView into @action(detail=False)
on ProgramViewSet so the router registers programs/search/ before
programs/{pk}/ automatically, eliminating the URL ordering dependency
- Scope permission to the action directly via permission_classes=[IsStandardUser]
rather than adding "search" to the global permission_classes_by_action dict
- Add test_title_parenthetical_grouping to cover the fixed parser path
- Tighten existing test assertions (case-insensitive, whole-word, description
AND, page_size cap) to catch regressions rather than passing vacuously
Co-authored-by: Copilot <copilot@github.com>
lxml 6.0.2 with recover=True silently drops HTML named entities (e.g. é, î) that are not valid XML, causing channel names and program data to lose characters during EPG ingestion.
- Add preprocessing step in fetch_xmltv() that converts HTML named entities to Unicode while preserving XML-predefined entities
- Detect encoding from XML declaration, fall back to UTF-8 for unknown codecs, skip files that fail to decode cleanly
- Guard against entities resolving to XML-special characters and html.unescape partial matching edge cases
- Add 28 unit tests covering entity resolution, encoding detection, and file-level processing
Replace expanded inline cards with a ProgramDetailModal for viewing full program
details. Add real-time progress bars to currently-airing programs using direct DOM manipulation. Extract and display season/episode info from EPG data with a 3-tier fallback (custom_properties → onscreen_episode → description text).
Backend:
- Add ProgramDetailSerializer for rich EPG data (categories, credits, ratings, images, external IDs) served on the detail endpoint
- Add infer_is_live() for status badges (live, new, premiere, finale)
- Add shared S/E extraction utilities in apps/epg/utils.py
- Optimize querysets with select_related("epg")
Frontend:
- ProgramDetailModal with conditional field rendering and image fallback chain
- Progress bars via scaleX transform updated every second
- Parallel data fetching with Promise.all for channels + programs
- Browser tab visibility recovery for progress bars
- Now-line with triangle marker and sub-millisecond precision
- Card layout: title, season/episode + subtitle, time + badges
- Shared formatSeasonEpisode() and external URL helpers
Add is_new, is_live, is_premiere, and is_finale fields to the EPG program API and render them as colored badges in the TV guide.
- Compact card: S/E, LIVE, and NEW badges on the title row
- Expanded card: all badges (S/E, LIVE, NEW, PREMIERE, FINALE) on the subtitle row, now moved directly after title for consistency
- PREMIERE/FINALE detected from premiere_text (case-insensitive)
- PREMIERE/FINALE only shown in expanded view to conserve space
- Badge order: S/E → LIVE → NEW → PREMIERE → FINALE
Display sub_title (episode name) and season/episode badge on guide program cards. Extract season/episode from custom_properties with fallback parsing of onscreen_episode format (e.g. "S12 E6").
- Add season/episode SerializerMethodFields to ProgramDataSerializer
- Parse onscreen_episode string as fallback when xmltv_ns data absent
- Display sub_title with S/E badge on guide cards
- Show description for programs without sub_title (movies)
- Add formatSeasonEpisode utility with full test coverage
- Add 15 backend serializer tests and 12 frontend utility tests
Fulfills #1065
