- Moved the `programme_index` from the `EPGSource` model to a new `EPGSourceIndex` table, ensuring that the large JSON blob is only loaded when explicitly accessed, thus improving query performance and memory efficiency.
- Updated related queries and API views to utilize the new structure, including adjustments to EPG generation and import logic to prevent unnecessary data loading.
- Enhanced memory management in the EPG grid endpoint to reduce worker RSS during response handling.
- Implemented robust DB connection management in `refresh_epg_data` to handle transient errors, ensuring reliable status updates for EPG sources.
- Added retry logic for database queries and improved error handling to prevent task failures from unhandled exceptions.
- Updated EPG source status handling to ensure accurate reporting of errors and terminal states during refresh operations.
- Introduced a temporary staging table for efficient batch processing of EPG program inserts, reducing memory and I/O contention during updates.
- Enhanced the `parse_programs_for_source` function to stream parsed rows into the staging table before swapping them atomically into the main program data.
- Added unit tests to validate the new staging and swapping logic, ensuring existing programs are preserved during failures and that batch processing works as intended.
- Replaced individual EPG program parse tasks with a centralized dispatch function to streamline guide refresh for newly assigned EPG IDs.
- Implemented batching for guide fetches when multiple EPGs are mapped, reducing redundant API calls and improving efficiency.
- Updated related utility functions to support the new fetching strategy and added tests to ensure correct behavior under various scenarios.
- Implemented a targeted guide fetch for Schedules Direct when mapping a channel, ensuring immediate data availability without redundant API calls.
- Updated EPG signals to queue guide fetches for newly mapped channels, streamlining the process and improving user experience.
- Refactored existing logic to skip unnecessary program refreshes for dummy sources and ensure proper handling of Schedules Direct data.
- Added unit tests to validate the new fetching behavior and ensure robustness of the integration.
- Added functions to compute schedule changes based on MD5 hashes and backfill missing dates for mapped stations.
- Enhanced handling of orphaned ProgramData records for unmapped EPG entries during schedule fetch.
- Introduced unit tests to validate MD5 comparison, backfill logic, and metadata fetching requirements.
- Introduced `dispatcharr_user_agent`, `dispatcharr_dvr_user_agent`, and `dispatcharr_http_headers` functions in `core.utils` to standardize User-Agent strings and HTTP headers for outbound requests.
- Updated various components, including `LogoViewSet`, `EPGSourceViewSet`, and VOD proxy views, to utilize the new header functions, ensuring consistent User-Agent usage across the application.
- Enhanced the handling of Schedules Direct API requests by including proper User-Agent headers, addressing previous API compliance issues.
- Addressed a bug where the EPG channel list did not refresh after a source finished parsing channels due to the `epg_refresh` WebSocket handler using a stale snapshot. The handler now correctly reads the current store state and ensures `fetchEPGData()` is called upon completion of channel parsing.
- Updated tests to verify that the frontend receives the correct parsing completion signals.
Add support for automatically applying channel logos from EPG data during refresh. Introduce a new toggle in the UI for enabling/disabling this feature, and update the backend to handle both channel IDs and EPG source IDs for logo application. Enhance the logo application task to process large libraries efficiently in chunks, improving performance and memory usage. Update changelog to reflect these changes.
Remove the _sd_send_ws_sync function and replace its usage with send_epg_update in the fetch_schedules_direct function. This change simplifies the code by ensuring all WebSocket updates are sent through a single function, improving maintainability and consistency in handling EPG updates.
Introduce a new poster style selection feature in the EPG settings, allowing users to choose their preferred artwork style. Update the logic for fetching and displaying program artwork from Schedules Direct, ensuring that the selected style is honored with appropriate fallbacks. Set default poster art to 'primary'. Add comprehensive tests to validate the poster selection functionality across various scenarios.
Add a new task to fetch program artwork from Schedules Direct and auto-apply logos to channels based on EPG data. Enhance the existing logic to handle poster backfill and update program records with fetched artwork. Readd the WebSocket handler to manage IP lookup events and adjust the EPGsTable component to display relevant credentials based on the source type.
Update the variable name in the EPG settings to better reflect its purpose. Adjust related logic in the UI to ensure consistent handling of the auto-apply feature for SD logos.
Build absolute poster proxy URLs via build_absolute_uri_with_port so
program detail images resolve correctly in dev and behind reverse
proxies. Backfill sd_icon for programs when poster fetch is enabled
but a delta refresh skips program metadata. Add a dev-mode frontend
fallback for relative image paths and correct the poster caching
description in the SD settings UI.
Add User-Agent header to API requests in ProgramViewSet for better tracking. Update cast processing logic in fetch_schedules_direct to align with XMLTV standards, ensuring proper categorization of main cast and guest stars.
- Replace per-entry ProgramData DB query with single batch fetch
- Pass pre-loaded EPGData objects to find_current_program_for_tvg_id
to avoid redundant select_related re-fetch
- Delete Redis programme index before XMLTV download so stale byte
offsets are never used during the refresh window
When selecting an EPG channel in the channel form, if that EPG entry's
programs had never been parsed, the preview showed "No current program".
This was because parse_programs_for_tvg_id requires a linked channel.
- Add force parameter to parse_programs_for_tvg_id to bypass channel check
- Auto-trigger parsing from current-programs API when no programs exist
- Check Redis task lock to prevent duplicate parse tasks from concurrent requests
- Frontend retries up to 3 times with 10s delay to pick up parsed results
- Skip parsing for dummy EPG sources
- Rename api_key to password (RemoveField + AddField) for clarity and reusability
- Move sd_changes_remaining/sd_changes_reset_at to custom_properties JSONField
instead of nullable columns on all EPGSource rows
- Replace sd_program_md5 column on ProgramData with SDProgramMD5 relation table
keyed by epg_source + program_id, fixing the key mismatch bug in program
MD5 delta detection
- Fix total_changed counter: .items() -> .values() so len() returns date count
not tuple length
- Add missing SD lineup management endpoints to api_views.py
(sd_lineups GET/POST/DELETE, sd_lineups_search POST)
- Regenerate migration 0023_schedules_direct using Django to ensure correct
index names and schema
- Add EPGSource username field and SDScheduleMD5 model for MD5 delta sync
- Implement full Schedules Direct fetch with SHA1 auth, token caching,
20-day schedule window, and MD5-based delta sync to minimize API usage
- Add lineup manager UI allowing users to search and manage SD lineups
- Add stations-only fetch on source creation for immediate EPG matching
- Enforce 2-hour minimum refresh interval to respect SD rate limits,
with first-fetch bypass when no prior full refresh exists
- Add real-time progress updates via synchronous Redis WebSocket delivery,
bypassing gevent.spawn for reliable delivery from Celery prefork workers
- Refactor EPGsTable status cell into standalone EPGStatusCell component
with direct Zustand subscription for live progress bar updates
- Guard XMLTV parse trigger for channels linked to SD EPG sources
- Add 15 backend tests covering auth, credentials, signals, and serializers
Migrations: 0023 (username field), 0024 (api_key help_text), 0025 (SD fields + SDScheduleMD5)
- EPG refresh tasks (`refresh_epg_data`) were being killed mid-transaction on large EPG sources. The `soft_time_limit=1700s` introduced in v0.21.0 raised `SoftTimeLimitExceeded`, a subclass of `Exception`, which was swallowed by the existing `except Exception` handler in `parse_programs_for_source`, leaving the database in a partial state with no logged error. The underlying cause is that the HTML entity preprocessing step added in v0.21.0 adds 2–10+ minutes on multi-gigabyte EPG files, pushing total task duration past the 1700s limit for large sources. `soft_time_limit` has been removed from `refresh_epg_data` and `time_limit` raised to 14400s (4 hours) as a true last-resort ceiling; the existing `TaskLockRenewer` daemon thread continues to renew the Redis lock every 120s for legitimately long-running tasks.
- HTML entity preprocessing (`_resolve_html_entities`) now performs a fast binary pre-scan before any file read or write. The pre-scan reads the file in 4 MB chunks without text decoding and exits immediately on the first non-XML named entity found. For EPG sources that contain no HTML entities — the common case — the full line-by-line read+rewrite pass (which takes 2–10+ minutes on a 2.5 GB file) is skipped entirely.
Previously only UnicodeDecodeError/UnicodeEncodeError were caught. Unexpected errors like disk full or permission denied would propagate
and crash the EPG refresh, which would have succeeded before the preprocessing step existed. Now all exceptions are caught and logged, leaving the original file untouched for lxml to handle as before.
lxml 6.0.2 with recover=True silently drops HTML named entities (e.g. é, î) that are not valid XML, causing channel names and program data to lose characters during EPG ingestion.
- Add preprocessing step in fetch_xmltv() that converts HTML named entities to Unicode while preserving XML-predefined entities
- Detect encoding from XML declaration, fall back to UTF-8 for unknown codecs, skip files that fail to decode cleanly
- Guard against entities resolving to XML-special characters and html.unescape partial matching edge cases
- Add 28 unit tests covering entity resolution, encoding detection, and file-level processing
Replace DRF ModelSerializer with .values() + plain dict construction in
EPGGridAPIView, eliminating per-row serializer instantiation, field
object creation, and OrderedDict overhead across ~12k programs.
Pre-compute season/episode at import time instead of on every API request:
- Extract S/E from onscreen episode-num inline in extract_custom_properties()
immediately after storing the raw value, rather than in a separate pass
- Add missing description-based S/E fallback to parse_programs_for_tvg_id(),
bringing it in line with parse_programs_for_source()
Remove runtime extract_season_episode() call from ProgramDataSerializer and
EPGGridAPIView — values are now guaranteed to be in custom_properties after
import, so season/episode resolution is a plain dict lookup at request time.
Also remove unnecessary select_related("epg") JOIN and .count() query from
the grid endpoint.
