Commit graph

20 commits

Author SHA1 Message Date
SergeantPanda
46695588f7 fix(m3u): enhance custom properties handling and DB connection management
- Implemented `ensure_custom_properties_dict()` to normalize custom properties across various models and serializers, addressing issues with legacy JSON-encoded strings.
- Updated M3U account and channel group models to ensure custom properties are consistently stored as dictionaries during save operations.
- Enhanced Celery task management by ensuring old DB connections are closed before and after tasks, improving reliability and preventing errors during account refresh operations. (Fixes #1338)
2026-06-18 14:59:09 -05:00
SergeantPanda
5d424adbe8 perf(vod): optimize VOD batch processing and memory management
- Introduced `lookup_by_name_year` function to limit database scans for movies and series without TMDB/IMDB IDs, enhancing performance by scoping lookups to current batch names.
- Updated `process_movie_batch` and `process_series_batch` to utilize the new lookup function, reducing memory usage and improving efficiency.
- Registered `refresh_vod_content` and `batch_refresh_series_episodes` for post-task memory cleanup in Celery, ensuring better resource management during VOD content refreshes.
2026-06-16 14:45:24 -05:00
SergeantPanda
34c938b1ce feat(epg): implement staging and batch processing for EPG program updates
- Introduced a temporary staging table for efficient batch processing of EPG program inserts, reducing memory and I/O contention during updates.
- Enhanced the `parse_programs_for_source` function to stream parsed rows into the staging table before swapping them atomically into the main program data.
- Added unit tests to validate the new staging and swapping logic, ensuring existing programs are preserved during failures and that batch processing works as intended.
2026-06-15 19:50:50 -05:00
SergeantPanda
7086f41d64 feat(epg): overhaul EPG auto-matching logic and improve performance
- Moved matching logic to a dedicated module for better organization and testability.
- Made single-channel auto-matching asynchronous, allowing for larger EPG libraries without hitting HTTP timeouts.
- Enhanced memory management and throughput during EPG matching, including optimizations for fuzzy matching and bulk processing.
- Fixed various reliability issues in the auto-matching process, ensuring accurate channel assignments and improved UI feedback.
- Updated API views and frontend components to reflect changes in the matching process and provide real-time notifications.
- Added tests for EPG matching functionality and name normalization.
- Single-channel and selected-channel auto-match always run, even when the channel already has EPG assigned; match-all (no channel IDs) still only processes channels without EPG.
2026-06-08 18:12:29 -05:00
SergeantPanda
e61539fc80 fix: Update plugin discovery comments and improve weakref handling in tests. Update changelog. 2026-05-13 17:02:34 -05:00
R3XCHRIS
f38d3a4f17 fix: register plugin @shared_tasks on worker startup (#1244)
Plugins live in /data/plugins/<slug>/plugin.py, outside INSTALLED_APPS,
so celery's autodiscover_tasks() never imports them. Any plugin using
module-level @shared_task for cron-scheduled work therefore has its
task unregistered with the worker after every worker restart, until
something else (a connect/utils.trigger_event call, etc.) lazily
imports the plugin module via PluginManager.discover_plugins.

In the meantime, beat fires the periodic task on time and the worker
rejects it:

  ERROR celery.worker.consumer.consumer Received unregistered task of
  type 'telegram_alerts.send_daily_report'.

PeriodicTask.last_run_at advances anyway, so the failure is silent at
the default INFO log level.

Fix: hook celery's worker_ready signal in dispatcharr/celery.py to
call PluginManager.discover_plugins(sync_db=False) on every worker
boot. Importing the plugin modules runs their @shared_task decorators,
registering the tasks before beat starts firing.

Exception handling is intentionally broad: if one plugin's plugin.py
has an import error, the worker must still come up — discovery failure
should not be a fatal error for the whole worker process.

Tests in tests/test_celery_plugin_discovery.py verify:
  - the handler calls discover_plugins(sync_db=False)
  - the handler swallows PluginManager.get() exceptions
  - the handler swallows discover_plugins() exceptions
  - the handler is actually connected to the worker_ready signal
Confirmed the test fails against current dev with the exact
'cannot import name discover_plugins_on_worker_ready' that motivates
this change, and passes with the fix.
2026-05-13 17:00:31 +01:00
SergeantPanda
1e5298a8d3 Bug Fix: Implement Redis locks in run_recording and worker_ready to prevent duplicate ffmpeg processes and ensure single execution across workers 2026-04-30 10:15:03 -05:00
SergeantPanda
c956672587 Enhancement: Route long-running DVR recordings to a dedicated queue. 2026-04-26 09:34:20 -05:00
SergeantPanda
76271fc9fd Bug Fix: Version update notification persisting after upgrading to the notified version (e.g. "v0.20.2 available" shown while already running v0.20.2). Root cause: check_for_version_update.delay() was called from AppConfig.ready(), which fires inside Celery prefork pool subprocesses before the broker connection is established, causing the dispatch to fail silently with no log output. Fixed by moving the startup dispatch to the worker_ready signal in celery.py (consistent with the existing recover_recordings_on_startup pattern), and deleting the stale version-{current_version} notification at the top of the production check path so it is cleared even when GitHub is unreachable. A WebSocket update is sent immediately on deletion so the frontend badge clears without waiting for the API response. 2026-03-08 13:38:18 -05:00
None
9d08305dcb fix(dvr): fix recording modal crash, harden recovery pipeline, improve tests
Frontend:
- RecordingDetailsModal: fix ReferenceError crash (editing state used before declaration in useMemo — TDZ in production bundles broke all tile clicks)
- RecordingDetailsModal: fix focus-stealing on description edit by calling Movie()/Series() as plain functions instead of <Movie />/<Series />(inline component refs change every render, causing unmount/remount)
- RecordingDetailsModal: suppress sub_title in modal title after custom save
- ErrorBoundary: log caught errors and show message for easier debugging
- guideUtils: filter terminal recordings from mapRecordingsByProgramId so Guide red dots clear after recording finishes or rule is deleted
- RecordingDetailsModalUtils.test: add missing end_time to test fixtures (filterByUpcoming now uses end_time instead of start_time)

Backend:
- tasks.py: output file collision avoidance (check .mkv/.ts before writing)
- tasks.py: pre-restart TS segment concatenation (direct concat→MKV remux)
- tasks.py: recovery pipeline — recover crashed "recording" status, remux expired recordings, revoke stale tasks, deterministic task IDs, early lock release, 10-min lock TTL for large remux operations
- tasks.py: poster resolver skips external API when title matches channel name
- celery.py: trigger recover_recordings_on_startup via worker_ready signal

Tests:
- Renamed test_dvr_fixes.py → test_recording_pipeline.py (clarity)
- Renamed test_dvr_retry.py → test_db_retry.py (clarity), fixed mock paths for close_old_connections (resolved 3 test failures), replaced synthetic test
- test_recording_stop_cancel: replaced brittle time.sleep(0.5) with deterministic thread-capture pattern
2026-03-04 10:21:03 -06:00
SergeantPanda
9b2ebf169b Better database connection cleanup. 2025-10-16 14:22:19 -05:00
SergeantPanda
ea81cfb1af Add auto channel sync settings to ChannelGroupM3UAccount and update related components
- Introduced `auto_channel_sync` and `auto_sync_channel_start` fields in the ChannelGroupM3UAccount model.
- Added API endpoint to update M3U group settings.
- Updated M3UGroupFilter component to manage auto sync settings.
- Enhanced M3URefreshNotification and M3U components for better user guidance.
- Created a Celery task for automatic channel synchronization after M3U refresh.
2025-07-13 15:59:25 -05:00
SergeantPanda
f87ab4b071 Rolled back some earlier memory omptimizations that were causing issues with extremely large m3us. 2025-05-22 21:52:28 -05:00
SergeantPanda
7c809931d7 Rewored celery memory cleanup logic. 2025-05-18 20:57:37 -05:00
SergeantPanda
1174e2e0c7 EPG processing enhancements. Celery memory management. 2025-05-17 16:42:37 -05:00
SergeantPanda
ee8cef5aa9 Cleaned up logging. 2025-05-10 14:13:43 -05:00
SergeantPanda
67aca64420 Properly set EV for all profiles so uWSGI daemons can see it. 2025-05-10 13:25:03 -05:00
SergeantPanda
24fba3c2b1 Change some celery tasks from info to debug. 2025-05-10 09:58:57 -05:00
SergeantPanda
d3615e1a66 Huge overhaul of logging. More standardized and we are now capturing logs from celery task and sening to console.
Also adds a new environmental variable: DISPATCHARR_LOG_LEVEL, log levels available: TRACE, DEBUG, INFO, WARNING, ERROR, CRITICAL
2025-05-10 09:29:06 -05:00
Dispatcharr
8edb743ebd Initial commit 2025-02-18 11:14:09 -06:00