This update introduces Redis caching for grouped settings in the CoreSettings model, improving performance by reducing database queries. A new method, `get_network_access_settings`, is added to streamline access to network settings. Additionally, the `setting_flag_enabled` function is introduced to handle boolean settings more robustly. The cache is invalidated on save and delete operations, ensuring data consistency. Tests have been added to validate the caching behavior and the new settings functionality.
- Introduced an optional `duration` parameter for catch-up sessions, allowing clients to specify the programme length in minutes. This value is preferred over EPG-derived durations and includes a buffer for provider lag.
- Updated the API views and serializers to accept and process the new `duration` field.
- Enhanced the catch-up proxy implementation to utilize the client-supplied duration, improving playback accuracy.
- Added tests to validate the new duration handling and ensure proper integration with existing functionality.
DB sessions were never actually pinned to UTC. Three stacked failures:
1. The connection_created receiver (_force_utc0 in core/apps.py) was a
nested closure connected with Django's default weak reference. It was
garbage-collected as soon as CoreConfig.ready() returned, so
SET TIME ZONE 'UTC0' fired into a dead weakref and never executed -
in any version since it landed (verified: the signal's receivers list
on a live 0.27.2 shows a dead weakref).
2. Sessions were UTC on older stacks anyway because Django's own
init_connection_state configured the timezone. Since native
psycopg-pool support (Django 5.1+), that path is gated on
`not self.pool` - and the geventpool mixin's `pool` property is
always truthy, so Django silently skips timezone (and role)
configuration for this backend on every connection. This is what
actually regressed at the psycopg2->3 / Django upgrade: the masking
layer disappeared, not the (already dead) signal.
3. psycopg3 logs "unknown PostgreSQL timezone: 'UTC0'" because the
POSIX spec is unresolvable in Python zoneinfo (cosmetic, but it
means 'UTC0' buys nothing on psycopg3).
Net effect: every session ran at the server-default timezone (verified
live: SHOW TimeZone through the pool returns 'Etc/UTC'). Deployments
whose Postgres default is non-UTC (e.g. /etc/localtime bind-mounts,
the original issue 651 report) get the EPG offset corruption back.
Fix: pin the GUC in the libpq startup packet in the pool backend's
get_connection_params():
- covers every connection the gevent pool creates; no signal, GC, or
Django-flow dependency (same pattern as the client_encoding pin the
pool already applies)
- startup-packet GUCs are the session default: they survive ROLLBACK,
and RESET TimeZone returns to UTC rather than the server default
- 'UTC' resolves cleanly in psycopg3's zoneinfo lookup, and PostgreSQL
resolves it against its own bundled tzdata, not host-mounted files
The dead signal is removed. Regression tests exercise the session
timezone through the real pool backend explicitly (the test runner's
default engine is the vanilla Django backend), and assert the
RESET-returns-to-UTC session-default property.
Tested: live 0.27.2 AIO (psycopg 3.3.4, PostgreSQL 17) - unpatched
backend+pool session shows 'Etc/UTC'; patched shows 'UTC' incl. after
RESET and ROLLBACK. Full A/B against the dev image with the server
default forced to Europe/Zurich: stock renders a 12:00Z source
programme as start="20260715140000 +0000" in /output/epg; patched
renders it correctly; the regression tests fail on stock and pass
when patched.
Clients that build catch-up requests in QUERY layout (e.g. Open-TV /
Fred TV: /streaming/timeshift.php?username=...&stream=...&start=...)
had no matching urlpattern, so the request silently fell through to
the frontend's <path:unused_path> catch-all and got served index.html
(200 OK, wrong content) instead of reaching the proxy — no error, no
log line, the client just fails to play.
The PATH-style layout (timeshift/<user>/<pass>/<stream_id>/<ts>/<dur>)
already worked; QUERY-style autodetection already existed for outgoing
provider requests (helpers.build_timeshift_url_format_a/_b) but was
never mirrored to the incoming route.
Split timeshift_proxy into a shared _timeshift_proxy_impl plus two thin
entry points (PATH-style timeshift_proxy, new QUERY-style
timeshift_proxy_query) so both incoming layouts are recognized.
Reported against the predecessor plugin as dispatcharr_timeshift#10;
reproduced identically against dispatcharr:dev (confirmed via nginx
access log: a QUERY-style request returned 200 with a response body
exactly matching the size of frontend/dist/index.html).
Updated the import mechanism for celery_app to avoid issues with lazy-loading, ensuring the default Celery app is correctly configured in production environments. Removed the __getattr__ function for celery_app, simplifying the module structure.
- Added `dispatcharr.settings_test` for isolated testing, automatically creating an empty PostgreSQL test database and ensuring transaction isolation during tests.
- Updated `manage.py` to switch to the test settings when running tests.
- Enhanced the CONTRIBUTING.md file with detailed instructions on running the backend test suite and handling Celery tasks in tests.
- Refactored test cases to use static methods for `worker_id` in `test_process_label.py` and adjusted assertions in `test_user_preferences.py` for better clarity and correctness.
- Implemented `ensure_custom_properties_dict()` to normalize custom properties across various models and serializers, addressing issues with legacy JSON-encoded strings.
- Updated M3U account and channel group models to ensure custom properties are consistently stored as dictionaries during save operations.
- Enhanced Celery task management by ensuring old DB connections are closed before and after tasks, improving reliability and preventing errors during account refresh operations. (Fixes#1338)
- Added a new function `_is_uwsgi_worker` to accurately identify when the application is running within a uWSGI worker.
- Updated `get_process_role` to label processes as "uwsgi" based on the new function or command-line arguments, improving process role accuracy.
- Expanded unit tests to cover various scenarios for uWSGI role detection, ensuring robust validation of the new logic.
- Introduced `lookup_by_name_year` function to limit database scans for movies and series without TMDB/IMDB IDs, enhancing performance by scoping lookups to current batch names.
- Updated `process_movie_batch` and `process_series_batch` to utilize the new lookup function, reducing memory usage and improving efficiency.
- Registered `refresh_vod_content` and `batch_refresh_series_episodes` for post-task memory cleanup in Celery, ensuring better resource management during VOD content refreshes.
- Introduced a new module to determine the process role based on command-line arguments, enhancing PostgreSQL connection labeling for better monitoring.
- Updated the database connection parameters to include the application name derived from the process role.
- Added unit tests to validate the process role detection logic for different scenarios, including Celery and uWSGI.
- Introduced a temporary staging table for efficient batch processing of EPG program inserts, reducing memory and I/O contention during updates.
- Enhanced the `parse_programs_for_source` function to stream parsed rows into the staging table before swapping them atomically into the main program data.
- Added unit tests to validate the new staging and swapping logic, ensuring existing programs are preserved during failures and that batch processing works as intended.
- Introduced DATABASE_POOL_CONN_MAX_LIFETIME to manage pooled connection lifecycle, improving performance and resource management.
- Updated database engine path to use 'dispatcharr.db.backends.postgresql_psycopg3' for consistency.
- Enhanced logging configuration for geventpool connection lifecycle tracking.
- Moved matching logic to a dedicated module for better organization and testability.
- Made single-channel auto-matching asynchronous, allowing for larger EPG libraries without hitting HTTP timeouts.
- Enhanced memory management and throughput during EPG matching, including optimizations for fuzzy matching and bulk processing.
- Fixed various reliability issues in the auto-matching process, ensuring accurate channel assignments and improved UI feedback.
- Updated API views and frontend components to reflect changes in the matching process and provide real-time notifications.
- Added tests for EPG matching functionality and name normalization.
- Single-channel and selected-channel auto-match always run, even when the channel already has EPG assigned; match-all (no channel IDs) still only processes channels without EPG.
- timeshift_proxy(): drop the resolve_channel_by_provider_stream_id
fallback. The client only ever sends Dispatcharr's internal Channel.id,
so resolve by id and return 404 on miss. Remove the dead `channel = None`
sentinel and the now-unused import.
- Delete the now-orphaned resolve_channel_by_provider_stream_id helper and
its local Stream import from apps/channels/utils.py (zero remaining refs).
- Import django.core.cache.cache at module top-level instead of inside
_get_cached_format_index / _set_cached_format_index.
- Fix the stale dispatcharr/urls.py comment: the "duration" slot carries
Channel.id, not the provider stream_id.
- Genericize provider host references in code comments.
- Switch engine to postgresql_psycopg3 (PR had wrong engine name,
and we're migrating from psycopg2 to psycopg3 as part of this work)
- Set CONN_MAX_AGE=0 (required by geventpool; PR had None)
- Reduce MAX_CONNS 20→8, REUSE_CONNS 10→3 to avoid exhausting
pg max_connections across 4 uWSGI workers + Celery
- Add pool=False to explicitly disable Django's native psycopg3 pool
Adds native catch-up/timeshift replay for Xtream Codes providers through
the same HTTPStreamReader transport pipeline as live TV.
Timeshift proxy (apps/timeshift/):
- URL cascade: 3 candidate timestamp formats per provider, per-account
format cache for fast-forward seek performance
- MPEG-TS preamble stripping (shared with HTTPStreamReader)
- Stats integration: timeshift viewers appear on /stats with TIMESHIFT badge
- Auth via hmac.compare_digest on XC password
Catchup detection — denormalized for zero-cost output queries:
- Stream.is_catchup + Stream.catchup_days populated at XC import time
- Channel.has_catchup + Channel.catchup_days + Channel.catchup_provider_stream_id
rolled up via ChannelStream post_save signal (UI path) and explicit SQL
after bulk_create (import path)
- _xc_channel_entry() reads denormalized fields instead of per-channel
custom_properties JSON introspection (eliminates N+1 queries)
- Migration 0038 backfills existing data via raw SQL
XC API enhancements:
- server_info.timezone + start/end + time_now use configured timezone
(triple consistency rule — fixes wrong-programme-plays bug)
- Dynamic has_archive flag + auto prev_days for catch-up channels
- XMLTV timestamps rewritten to local timezone for catch-up clients
HTTPStreamReader extended (apps/proxy/live_proxy/input/http_streamer.py):
- 1 MB pipe buffer via fcntl F_SETPIPE_SZ (eliminates producer/consumer
ping-pong that halved throughput)
- Pre-opened response= for URL cascade workflows
- strip_ts_preamble= for XC servers emitting PHP warnings before TS
- find_ts_sync() as shared utility
- Builds on upstream O_NONBLOCK + select() write loop
Provider stream_id lookup order:
- stream_xc() and xc_get_epg() try internal Channel.id first, fall back
to provider stream_id only when needed (avoids unconditional query on
every request)
Also includes:
- VOD provider cascade in stream_vod() — iterates all M3U relations by
priority when first provider is at capacity
- Defensive null-safety: custom_sid: None → "" in get_live_streams,
get_vod_streams, get_vod_info, get_series_info (fixes iPlayTV crash on
JSON null for string fields)
- Timeshift settings UI (timezone selector, debug toggle)
- StreamConnectionCard violet TIMESHIFT badge
- Orphan cleanup skips timeshift_* virtual channels
- Enable gevent cooperative multitasking in all uWSGI worker configs
(gevent-early-monkey-patch + import dispatcharr.gevent_patch)
- Rewrite WebSocket group sends to bypass asyncio in gevent workers:
_gevent_ws_send() replicates the channels_redis 4.x wire format
directly via synchronous Redis so send_websocket_update() and
_send_async() no longer fail silently after epoll is patched out
- Fix PostgreSQL connection exhaustion: CONN_MAX_AGE=0 + explicit
close_old_connections() in stream manager and cleanup watchdog loops
- Fix stream proxy race: register client before the connect-wait loop
so the cleanup watchdog never sees zero clients on a live channel
- Channel list/logo/profile queryset optimisations: conditional DISTINCT,
EXISTS semi-joins for filter-options, channel_count annotation to
eliminate N+1 in LogoSerializer, prefetched memberships in
ChannelProfileSerializer
- JsonResponse for channel ID list and summary endpoints
Plugins live in /data/plugins/<slug>/plugin.py, outside INSTALLED_APPS,
so celery's autodiscover_tasks() never imports them. Any plugin using
module-level @shared_task for cron-scheduled work therefore has its
task unregistered with the worker after every worker restart, until
something else (a connect/utils.trigger_event call, etc.) lazily
imports the plugin module via PluginManager.discover_plugins.
In the meantime, beat fires the periodic task on time and the worker
rejects it:
ERROR celery.worker.consumer.consumer Received unregistered task of
type 'telegram_alerts.send_daily_report'.
PeriodicTask.last_run_at advances anyway, so the failure is silent at
the default INFO log level.
Fix: hook celery's worker_ready signal in dispatcharr/celery.py to
call PluginManager.discover_plugins(sync_db=False) on every worker
boot. Importing the plugin modules runs their @shared_task decorators,
registering the tasks before beat starts firing.
Exception handling is intentionally broad: if one plugin's plugin.py
has an import error, the worker must still come up — discovery failure
should not be a fatal error for the whole worker process.
Tests in tests/test_celery_plugin_discovery.py verify:
- the handler calls discover_plugins(sync_db=False)
- the handler swallows PluginManager.get() exceptions
- the handler swallows discover_plugins() exceptions
- the handler is actually connected to the worker_ready signal
Confirmed the test fails against current dev with the exact
'cannot import name discover_plugins_on_worker_ready' that motivates
this change, and passes with the fix.
- Set `DEFAULT_PERMISSION_CLASSES` to `Authenticated` in the DRF configuration.
- Explicitly marked the HDHomeRun discovery endpoints (`DiscoverAPIView`, `LineupAPIView`, `LineupStatusAPIView`, `HDHRDeviceXMLAPIView`) and the version endpoint with `permission_classes = [AllowAny]` to document their intentionally public access now that the global default is `Authenticated`.
