This update addresses issues related to geventpool database connection leaks by consistently invoking `close_old_connections()` in `finally` blocks within the backup scheduler, plugin repository refresh, and core notification synchronization processes. These changes ensure that idle connections are properly released, preventing resource exhaustion. HLS proxy server has been disabled (hasn't been used). Additionally, the proxy server cleanup logic has been enhanced to safely handle the absence of proxy instances. Tests have been updated to validate these improvements and ensure stability in high-load scenarios.
This update introduces a new `is_catchup_enabled` function to determine if catch-up is allowed for users based on their custom properties and system settings. The `UserViewSet` is modified to restrict admin-managed properties, including catch-up access. Additionally, various views and tests are updated to incorporate catch-up checks, ensuring that users without access receive appropriate error responses. The frontend is enhanced with a catch-up toggle in user and system settings forms, allowing for better management of catch-up capabilities.
- Implemented a new `LockedSpectacularAPIView` to handle schema generation, ensuring it no longer fails due to concurrent gevent requests. The schema builds are now single-flight per process and cached in Django's cache, allowing all workers to share the result.
- Updated the `urls.py` to use the new view for schema generation, enhancing stability and performance.
- Added `dispatcharr.settings_test` for isolated testing, automatically creating an empty PostgreSQL test database and ensuring transaction isolation during tests.
- Updated `manage.py` to switch to the test settings when running tests.
- Enhanced the CONTRIBUTING.md file with detailed instructions on running the backend test suite and handling Celery tasks in tests.
- Refactored test cases to use static methods for `worker_id` in `test_process_label.py` and adjusted assertions in `test_user_preferences.py` for better clarity and correctness.
- Implemented `ensure_custom_properties_dict()` to normalize custom properties across various models and serializers, addressing issues with legacy JSON-encoded strings.
- Updated M3U account and channel group models to ensure custom properties are consistently stored as dictionaries during save operations.
- Enhanced Celery task management by ensuring old DB connections are closed before and after tasks, improving reliability and preventing errors during account refresh operations. (Fixes#1338)
- Added a new function `_is_uwsgi_worker` to accurately identify when the application is running within a uWSGI worker.
- Updated `get_process_role` to label processes as "uwsgi" based on the new function or command-line arguments, improving process role accuracy.
- Expanded unit tests to cover various scenarios for uWSGI role detection, ensuring robust validation of the new logic.
- Introduced a new module to determine the process role based on command-line arguments, enhancing PostgreSQL connection labeling for better monitoring.
- Updated the database connection parameters to include the application name derived from the process role.
- Added unit tests to validate the process role detection logic for different scenarios, including Celery and uWSGI.
- Improved database connection management by ensuring `close_old_connections()` is called in various methods to prevent connection leaks.
- Updated event dispatching to run asynchronously in gevent, preventing blocking during live-proxy and streaming paths.
- Introduced DATABASE_POOL_CONN_MAX_LIFETIME to manage pooled connection lifecycle, improving performance and resource management.
- Updated database engine path to use 'dispatcharr.db.backends.postgresql_psycopg3' for consistency.
- Enhanced logging configuration for geventpool connection lifecycle tracking.
Remove the _sd_send_ws_sync function and replace its usage with send_epg_update in the fetch_schedules_direct function. This change simplifies the code by ensuring all WebSocket updates are sent through a single function, improving maintainability and consistency in handling EPG updates.
Plugins live in /data/plugins/<slug>/plugin.py, outside INSTALLED_APPS,
so celery's autodiscover_tasks() never imports them. Any plugin using
module-level @shared_task for cron-scheduled work therefore has its
task unregistered with the worker after every worker restart, until
something else (a connect/utils.trigger_event call, etc.) lazily
imports the plugin module via PluginManager.discover_plugins.
In the meantime, beat fires the periodic task on time and the worker
rejects it:
ERROR celery.worker.consumer.consumer Received unregistered task of
type 'telegram_alerts.send_daily_report'.
PeriodicTask.last_run_at advances anyway, so the failure is silent at
the default INFO log level.
Fix: hook celery's worker_ready signal in dispatcharr/celery.py to
call PluginManager.discover_plugins(sync_db=False) on every worker
boot. Importing the plugin modules runs their @shared_task decorators,
registering the tasks before beat starts firing.
Exception handling is intentionally broad: if one plugin's plugin.py
has an import error, the worker must still come up — discovery failure
should not be a fatal error for the whole worker process.
Tests in tests/test_celery_plugin_discovery.py verify:
- the handler calls discover_plugins(sync_db=False)
- the handler swallows PluginManager.get() exceptions
- the handler swallows discover_plugins() exceptions
- the handler is actually connected to the worker_ready signal
Confirmed the test fails against current dev with the exact
'cannot import name discover_plugins_on_worker_ready' that motivates
this change, and passes with the fix.
- Move modular Redis wait from uWSGI exec-pre to entrypoint (exec-pre runs under 'su -' which strips Docker env vars, so DISPATCHARR_ENV and REDIS_HOST were never available)
- Selective flush in modular mode: clears stale app state (stream locks, proxy metadata) while preserving Celery broker/result keys
- AIO mode unchanged: full flushdb via uWSGI exec-pre
- Update unit tests for both flush paths
- Fix Postgres version check failing with restricted DB users (use $POSTGRES_DB instead of hardcoded 'postgres')
- Fix DVR recording broken in modular mode (respect DISPATCHARR_PORT instead of hardcoding 9191)
- Remove flushdb() from wait_for_redis.py to prevent Redis data loss on container restart
- Add DISPATCHARR_PORT to celery environment in docker-compose.yml
- Add depends_on health conditions for proper service startup ordering
- Add extra_hosts for host.docker.internal resolution on Linux
- Harden celery entrypoint with timeouts for JWT wait (120s) and migration wait (300s)
- Replace fragile showmigrations grep with migrate --check
- Add unit tests for DVR port resolution and flushdb removal regression
