Commit graph

167 commits

Author SHA1 Message Date
SergeantPanda
c081354150 feat(websocket): enhance connection handling with cached IP lookup 2026-05-31 19:28:19 -05:00
Seth Van Niekerk
16cb9edcbd
blur public IP in sidebar, add ip lookup toggle and env var, make lookup non-blocking 2026-05-30 13:07:07 -04:00
SergeantPanda
a2656184e8 perf/fix: migrate DB driver to psycopg3, fix pool settings
- Switch engine to postgresql_psycopg3 (PR had wrong engine name,
  and we're migrating from psycopg2 to psycopg3 as part of this work)
- Set CONN_MAX_AGE=0 (required by geventpool; PR had None)
- Reduce MAX_CONNS 20→8, REUSE_CONNS 10→3 to avoid exhausting
  pg max_connections across 4 uWSGI workers + Celery
- Add pool=False to explicitly disable Django's native psycopg3 pool
2026-05-28 19:31:50 -05:00
SergeantPanda
80b72d8cfa Enhancement: Update psycopg2 to 3. 2026-05-28 19:27:52 -05:00
Jonathan Caicedo
94c8bd36bf perf: replace per-request DB connections with geventpool 2026-05-25 11:12:46 -04:00
SergeantPanda
108bd52e7f perf: replace local cache with django-redis for improved EPG caching
Some checks failed
Base Image Build / prepare (push) Has been cancelled
CI Pipeline / prepare (push) Has been cancelled
Build and Push Multi-Arch Docker Image / build-and-push (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Has been cancelled
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
CI Pipeline / create-manifest (push) Has been cancelled
2026-05-17 20:23:57 -05:00
SergeantPanda
bdfad0f314 fix: clarify comments in gevent_patch.py regarding blocking behavior in send_websocket_update 2026-05-16 10:59:09 -05:00
SergeantPanda
c33d456743 Enable gevent in uWSGI workers; fix WS delivery and DB exhaustion
- Enable gevent cooperative multitasking in all uWSGI worker configs
  (gevent-early-monkey-patch + import dispatcharr.gevent_patch)
- Rewrite WebSocket group sends to bypass asyncio in gevent workers:
  _gevent_ws_send() replicates the channels_redis 4.x wire format
  directly via synchronous Redis so send_websocket_update() and
  _send_async() no longer fail silently after epoll is patched out
- Fix PostgreSQL connection exhaustion: CONN_MAX_AGE=0 + explicit
  close_old_connections() in stream manager and cleanup watchdog loops
- Fix stream proxy race: register client before the connect-wait loop
  so the cleanup watchdog never sees zero clients on a live channel
- Channel list/logo/profile queryset optimisations: conditional DISTINCT,
  EXISTS semi-joins for filter-options, channel_count annotation to
  eliminate N+1 in LogoSerializer, prefetched memberships in
  ChannelProfileSerializer
- JsonResponse for channel ID list and summary endpoints
2026-05-14 18:55:09 -05:00
SergeantPanda
e61539fc80 fix: Update plugin discovery comments and improve weakref handling in tests. Update changelog. 2026-05-13 17:02:34 -05:00
R3XCHRIS
f38d3a4f17 fix: register plugin @shared_tasks on worker startup (#1244)
Plugins live in /data/plugins/<slug>/plugin.py, outside INSTALLED_APPS,
so celery's autodiscover_tasks() never imports them. Any plugin using
module-level @shared_task for cron-scheduled work therefore has its
task unregistered with the worker after every worker restart, until
something else (a connect/utils.trigger_event call, etc.) lazily
imports the plugin module via PluginManager.discover_plugins.

In the meantime, beat fires the periodic task on time and the worker
rejects it:

  ERROR celery.worker.consumer.consumer Received unregistered task of
  type 'telegram_alerts.send_daily_report'.

PeriodicTask.last_run_at advances anyway, so the failure is silent at
the default INFO log level.

Fix: hook celery's worker_ready signal in dispatcharr/celery.py to
call PluginManager.discover_plugins(sync_db=False) on every worker
boot. Importing the plugin modules runs their @shared_task decorators,
registering the tasks before beat starts firing.

Exception handling is intentionally broad: if one plugin's plugin.py
has an import error, the worker must still come up — discovery failure
should not be a fatal error for the whole worker process.

Tests in tests/test_celery_plugin_discovery.py verify:
  - the handler calls discover_plugins(sync_db=False)
  - the handler swallows PluginManager.get() exceptions
  - the handler swallows discover_plugins() exceptions
  - the handler is actually connected to the worker_ready signal
Confirmed the test fails against current dev with the exact
'cannot import name discover_plugins_on_worker_ready' that motivates
this change, and passes with the fix.
2026-05-13 17:00:31 +01:00
SergeantPanda
84fbbfa3c4 refactor: Clean up get_client_ip() function in utils.py
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-05-09 09:48:53 -05:00
SergeantPanda
dc649ffd88 Enhancement: Initial fmp4 support and major ts_proxy refactor 2026-05-08 17:46:47 -05:00
SergeantPanda
883390308b
Merge pull request #1214 from sv-dispatcharr/feat/restrict-xc-user-access
restrict XC access per user by IP & CIDR allowlist
2026-05-03 15:41:37 -05:00
Seth Van Niekerk
a01814069a
code review & pivot 2026-05-01 09:10:37 -04:00
SergeantPanda
1e5298a8d3 Bug Fix: Implement Redis locks in run_recording and worker_ready to prevent duplicate ffmpeg processes and ensure single execution across workers 2026-04-30 10:15:03 -05:00
Seth Van Niekerk
8894d51877
restrict XC access per user by configurable IP allowlist 2026-04-28 16:35:32 -04:00
SergeantPanda
c956672587 Enhancement: Route long-running DVR recordings to a dedicated queue. 2026-04-26 09:34:20 -05:00
SergeantPanda
e861fca092 Enhancement: Enhanced Swagger UI authorization dialog: registered a custom OpenApiAuthenticationExtension for ApiKeyAuthentication so drf-spectacular now generates an ApiKeyAuth (apiKey) entry alongside jwtAuth. Both entries include descriptive text linking to the relevant endpoints (/api/accounts/token/, /api/accounts/api-keys/generate/, /api/accounts/api-keys/revoke/). 2026-04-10 18:29:13 -05:00
SergeantPanda
66ee67da30 security: Change from default of "Authenticated" to "IsAdmin" 2026-04-10 10:46:16 -05:00
SergeantPanda
04a684b359 security: Removed CORS_ALLOW_CREDENTIALS = True from CORS configuration. Dispatcharr authenticates via JWT Authorization headers and API keys — not cookies — so credentials are never sent cross-origin by browsers. The setting was also redundant: browsers reject Access-Control-Allow-Credentials: true when Access-Control-Allow-Origin is a wildcard (*), so it had no effect in practice. 2026-04-10 08:34:13 -05:00
SergeantPanda
b535f28ac5 security: Added rate limiting to the login endpoint (POST /api/accounts/token/) using DRF's built-in throttling. A LoginRateThrottle (3 requests/minute per IP, sliding window) is applied to the TokenObtainPairView. Repeated failed attempts from the same IP receive 429 Too Many Requests. 2026-04-09 21:46:50 -05:00
SergeantPanda
47427d4b0f security:
- Set `DEFAULT_PERMISSION_CLASSES` to `Authenticated` in the DRF configuration.
- Explicitly marked the HDHomeRun discovery endpoints (`DiscoverAPIView`, `LineupAPIView`, `LineupStatusAPIView`, `HDHRDeviceXMLAPIView`) and the version endpoint with `permission_classes = [AllowAny]` to document their intentionally public access now that the global default is `Authenticated`.
2026-04-09 21:36:51 -05:00
SergeantPanda
086cc74959 Bug Fix: M3U profile URL rewriting now uses the regex module instead of re across all URL transform code paths (url_utils.transform_url, core/views.py, vod_proxy/_transform_url, tasks.get_transformed_credentials, and the WebSocket live-preview handler in consumers.py). The regex module natively accepts JavaScript/PCRE-style named capture groups ((?<name>...)) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a re.error. As a further improvement, regex also supports variable-length lookbehind assertions (e.g. (?<=a+)), which re rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling regex.sub: $<name>\g<name> and $1/$2/… → \1/\2/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
2026-03-29 17:55:08 -05:00
dekzter
90a600b48f merged in dev 2026-03-26 12:17:43 -04:00
dekzter
a19be96ced fleshed out user limits and termination logic 2026-03-25 17:33:26 -04:00
None
b30a24e2fb feat: add TLS connection support for Redis and PostgreSQL (#950)
- Add 10 env vars for Redis/PostgreSQL TLS (REDIS_SSL, REDIS_SSL_VERIFY, REDIS_SSL_CA_CERT, REDIS_SSL_CERT, REDIS_SSL_KEY, POSTGRES_SSL, POSTGRES_SSL_MODE, POSTGRES_SSL_CA_CERT, POSTGRES_SSL_CERT, POSTGRES_SSL_KEY)
- Propagate SSL params to all Redis connection points: RedisClient, stream view, PubSub, client manager, Celery broker/result backend, Django Channels, and pre-Django startup scripts
- Add Celery SSL config via URL query string params (required by Kombu's internal URL parsing)
- Add PostgreSQL sslmode and certificate options to DATABASES config
- Add startup validation: cert file existence, URL scheme conflict detection, TLS status logging
- Add TLS error hints to Redis connection failure messages
- Add Connection Security panel in System Settings (modular mode only) showing encryption, verification, and mTLS status
- Add PG client key permission fix in web and celery entrypoints for Docker Desktop compatibility (0777 to 0600)
- Add TLS env var passthrough in entrypoint for su - login shells
- Document TLS env vars in modular docker-compose.yml
- Change env_mode from dev/prod to actual DISPATCHARR_ENV value for modular mode detection
2026-03-21 17:24:09 -05:00
SergeantPanda
1d4603dadd Enhancement: Optimize profile expiration handling and update task naming for clarity 2026-03-15 12:42:34 -05:00
SergeantPanda
b22f9e8e07 Enhancement: Account expiration tracking and notifications for M3U profiles 2026-03-14 18:38:59 -05:00
dekzter
f69a462253 merged in dev 2026-03-13 08:22:44 -04:00
SergeantPanda
76271fc9fd Bug Fix: Version update notification persisting after upgrading to the notified version (e.g. "v0.20.2 available" shown while already running v0.20.2). Root cause: check_for_version_update.delay() was called from AppConfig.ready(), which fires inside Celery prefork pool subprocesses before the broker connection is established, causing the dispatch to fail silently with no log output. Fixed by moving the startup dispatch to the worker_ready signal in celery.py (consistent with the existing recover_recordings_on_startup pattern), and deleting the stale version-{current_version} notification at the top of the production check path so it is cleared even when GitHub is unreachable. A WebSocket update is sent immediately on deletion so the frontend badge clears without waiting for the API response. 2026-03-08 13:38:18 -05:00
SergeantPanda
bf87c44a72 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/CodeBormen/1063 2026-03-05 16:48:34 -06:00
None
ce9f7ccbf9 fix(m3u): fix Celery worker memory leak during M3U/XC refresh
Restructure cleanup in refresh_single_m3u_account so del statements
run before gc.collect(), enabling cyclic garbage collection. Add worker_max_memory_per_child as a safety net against memory fragmentation.

Closes #1012, Closes #1053
2026-03-05 13:47:20 -06:00
None
9d08305dcb fix(dvr): fix recording modal crash, harden recovery pipeline, improve tests
Frontend:
- RecordingDetailsModal: fix ReferenceError crash (editing state used before declaration in useMemo — TDZ in production bundles broke all tile clicks)
- RecordingDetailsModal: fix focus-stealing on description edit by calling Movie()/Series() as plain functions instead of <Movie />/<Series />(inline component refs change every render, causing unmount/remount)
- RecordingDetailsModal: suppress sub_title in modal title after custom save
- ErrorBoundary: log caught errors and show message for easier debugging
- guideUtils: filter terminal recordings from mapRecordingsByProgramId so Guide red dots clear after recording finishes or rule is deleted
- RecordingDetailsModalUtils.test: add missing end_time to test fixtures (filterByUpcoming now uses end_time instead of start_time)

Backend:
- tasks.py: output file collision avoidance (check .mkv/.ts before writing)
- tasks.py: pre-restart TS segment concatenation (direct concat→MKV remux)
- tasks.py: recovery pipeline — recover crashed "recording" status, remux expired recordings, revoke stale tasks, deterministic task IDs, early lock release, 10-min lock TTL for large remux operations
- tasks.py: poster resolver skips external API when title matches channel name
- celery.py: trigger recover_recordings_on_startup via worker_ready signal

Tests:
- Renamed test_dvr_fixes.py → test_recording_pipeline.py (clarity)
- Renamed test_dvr_retry.py → test_db_retry.py (clarity), fixed mock paths for close_old_connections (resolved 3 test failures), replaced synthetic test
- test_recording_stop_cancel: replaced brittle time.sleep(0.5) with deterministic thread-capture pattern
2026-03-04 10:21:03 -06:00
SergeantPanda
d487bb5671
Revert "fix(wsgi): ensure gevent monkey-patching before Django import chain" 2026-03-03 15:33:55 -06:00
SergeantPanda
a8ba2ade18 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/endoze/1026 2026-03-03 11:52:12 -06:00
Endoze
9b6c793b11
fix(wsgi): ensure gevent monkey-patching before Django import chain
Celery's Redis connection pool was silently failing in modular
deployment mode because Django's setup imports celery.py before uWSGI's
gevent plugin reliably patches stdlib sockets. Moving patch_all() to the
top of wsgi.py guarantees all broker connections are gevent-aware
regardless of uWSGI initialization order.
2026-02-26 10:23:50 -05:00
dekzter
6f4665e6eb merged in dev 2026-02-26 08:20:20 -05:00
dekzter
7dc01a821b
Merge pull request #1019 from Dispatcharr/integrations-fix
pass all details into event triggers, updated custom script default path
2026-02-24 12:42:29 -05:00
dekzter
9470600474 pass all details into event triggers, updated custom script default path 2026-02-24 12:13:21 -05:00
dekzter
aece3367ce backend work for api key authentication 2026-02-21 13:45:07 -05:00
dekzter
55a5180ee1 hardening of script handling with configurable variables - making script execution more secure 2026-02-15 07:37:00 -05:00
dekzter
bc72ea7310 merged in dev 2026-02-11 19:51:21 -05:00
SergeantPanda
1df802341d
Merge pull request #937 from CodeBormen:Feature/Support-External-Redis-Auth
feat: Add Redis authentication support for modular deployment
2026-02-09 17:29:38 -06:00
dekzter
24f812dc4d initial connect feature 2026-02-08 09:29:22 -05:00
SergeantPanda
de81dc6c80 Enhancement: Refactored app initialization to prevent redundant execution across multiple worker processes. Created dispatcharr.app_initialization utility module with should_skip_initialization() function that prevents custom initialization tasks (backup scheduler sync, developer notifications sync) from running during management commands, in worker processes, or in development servers. This significantly reduces startup overhead in multi-worker deployments (e.g., uWSGI with 10 workers now syncs the scheduler once instead of 10 times). Applied to both CoreConfig and BackupsConfig apps. 2026-02-05 13:48:01 -06:00
SergeantPanda
e26c1908c5 Bug Fix: Automatic backups not enabled by default on new installations: Added backups app to INSTALLED_APPS and implemented automatic scheduler initialization in BackupsConfig.ready(). The backup scheduler now properly syncs the periodic task on startup, ensuring automatic daily backups are enabled and scheduled immediately on fresh database creation without requiring manual user intervention. 2026-02-04 16:56:11 -06:00
None
e217960500 feat: Add Redis authentication support for modular deployment
Add support for authentication when connecting to external Redis instances in modular deployment mode. This enables secure Redis deployments using either password-only authentication (Redis <6) or
username + password authentication (Redis 6+ ACL).

Changes:
- Add REDIS_PASSWORD and REDIS_USER environment variables
- Implement URL encoding for special characters in passwords
- Update all Redis connection points to support auth
- Add comprehensive documentation and examples to docker-compose.yml
- Maintains full backward compatibility (empty defaults = no auth)

All authentication mechanisms have been fully tested including pasword-only authentication, Redis 6+ ACL authentication with username + password, volume-mounted configuration files, and special character handling.

Existing deployments are not effected, authentication support is entirely opt-in using docker-compose.

Also, acknowledging the fact that the docker-compose file for modular deployments has been getting out of hand with auth support, the docker-compose.yml file was re-formatted for better visibility in configuration. This seemed like a better way to go than mandating a .env file.
2026-02-03 21:42:06 -06:00
SergeantPanda
b01eb9585c feat: add system notifications and update checks
Real-time notifications for system events and alerts
Per-user notification management and dismissal
Update check on startup and every 24 hours to notify users of available versions
Notification center UI component
Automatic cleanup of expired notifications
2026-02-03 09:24:02 -06:00
SergeantPanda
3c556494c1 Enhancement: Updated default network access settings for M3U and EPG endpoints to only allow local/private networks by default (127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, ::1/128, fc00::/7, fe80::/10). This improves security by preventing public internet access to these endpoints unless explicitly configured. Other endpoints (Streams, XC API, UI) remain open by default. 2026-01-31 20:26:45 -06:00
SergeantPanda
bd1e0e8f07 Bug fix: Admin URL Conflict with XC Streams: Updated nginx configuration to only redirect exact /admin and /admin/ paths to login in production, preventing interference with stream URLs that use "admin" as a username (e.g., /admin/password/stream_id now properly routes to stream handling instead of being redirected). 2026-01-27 14:36:57 -06:00