Commit graph

90 commits

Author SHA1 Message Date
SergeantPanda
1289e95c93 Merge branch 'dev' of https://github.com/Dispatcharr/Dispatcharr into pr/JCBird1012/1157 2026-04-14 15:57:48 -05:00
SergeantPanda
d5e6ee0a0d Enhancement: Use Unix socket if AIO and POSTGRES_HOST is set to loopback. 2026-04-14 15:51:40 -05:00
SergeantPanda
6ad1207a0b fix: graceful container shutdown and cleanup improvements
- Explicitly stop uwsgi, celery, daphne, and redis-server on SIGTERM;
  these are spawned as uwsgi attach-daemons and were not tracked in
  pids[], causing exit 137 on docker stop
- Replace fixed sleep 3 with a polling loop (8 s ceiling) that exits as
  soon as all processes have stopped
- Use pg_ctl stop -m immediate as the postgres force-stop fallback
  instead of SIGKILL to avoid data corruption
- Add _cleanup_done guard to prevent double invocation of cleanup() when
  the trap fires and the explicit call at script end both execute
- Register process names at startup in pid_names[] so crash diagnostics
  show meaningful names after a process has exited
- Suppress the unexpected-exit diagnostic block on normal docker stop
- Guard all pgrep-based pids+=() against empty values (nginx, vite,
  postgres) to prevent empty PID entries corrupting the pids[] array
2026-04-13 20:21:01 -05:00
Shokkstokk
49db83260d fix: graceful container shutdown - exit 137 on docker stop 2026-04-10 12:36:35 -04:00
SergeantPanda
6f8ca243af fix: ensure PGSSL* environment variables are explicitly set by stripping inherited values
fix: use psycopg2.sql for safe database drop and create commands

fix: Also check POSTGRES_SSL is true before skipping default POSTGRES_PASSWORD

fix: update usage comment to reflect correct script path

fix: improve cleanup logic to conditionally remove certificate directory
2026-03-31 09:51:09 -05:00
None
763f42cfff fix: pass TLS parameters to all external connection points in modular mode
- Add setup_pg_ssl_env() to export libpq PGSSL* env vars; all child psql/pg_dump/pg_isready commands inherit TLS automatically

- Move TLS client key permission fix before external PG connections (was running after, defeating the fix)

- Enhance key fix to trigger on ownership mismatch (root-owned 0600 key unreadable by app user)

- Extract key fix to shared script (docker/init/00-fix-pg-ssl-key.sh) sourced by both entrypoints

- Default POSTGRES_PASSWORD to empty in modular mode (cert-only auth sends no spurious password)

- Propagate TLS OPTIONS to backup pg_dump/pg_restore subprocess calls via _get_pg_env()

- Pass SSL kwargs to dropdb management command's psycopg2.connect()

- Add REDIS_SSL_PARAMS to VOD proxy Redis connections missed in original TLS PR

- Add 8-scenario TLS integration test suite (PG mTLS, Redis TLS, verify-full, key fix, Celery, regression)

- Add 6 unit tests for _get_pg_env() and dropdb TLS parameter passing
2026-03-30 20:19:27 -05:00
Jonathan Caicedo
6591d1dac7 perf: use PostgreSQL Unix socket for AIO 2026-03-29 16:13:50 -04:00
SergeantPanda
9da6f7794c fix: ensure PGID defaults to PUID when setting ownership for PostgreSQL SSL key (matching entrypoint.celery behavior. 2026-03-23 18:26:52 -05:00
None
b30a24e2fb feat: add TLS connection support for Redis and PostgreSQL (#950)
- Add 10 env vars for Redis/PostgreSQL TLS (REDIS_SSL, REDIS_SSL_VERIFY, REDIS_SSL_CA_CERT, REDIS_SSL_CERT, REDIS_SSL_KEY, POSTGRES_SSL, POSTGRES_SSL_MODE, POSTGRES_SSL_CA_CERT, POSTGRES_SSL_CERT, POSTGRES_SSL_KEY)
- Propagate SSL params to all Redis connection points: RedisClient, stream view, PubSub, client manager, Celery broker/result backend, Django Channels, and pre-Django startup scripts
- Add Celery SSL config via URL query string params (required by Kombu's internal URL parsing)
- Add PostgreSQL sslmode and certificate options to DATABASES config
- Add startup validation: cert file existence, URL scheme conflict detection, TLS status logging
- Add TLS error hints to Redis connection failure messages
- Add Connection Security panel in System Settings (modular mode only) showing encryption, verification, and mTLS status
- Add PG client key permission fix in web and celery entrypoints for Docker Desktop compatibility (0777 to 0600)
- Add TLS env var passthrough in entrypoint for su - login shells
- Document TLS env vars in modular docker-compose.yml
- Change env_mode from dev/prod to actual DISPATCHARR_ENV value for modular mode detection
2026-03-21 17:24:09 -05:00
SergeantPanda
03e1061716 Bug Fix: Fix postgres lock file permissions before starting and upgrading Postgres. 2026-03-13 14:59:46 -05:00
None
29525336ab replace migration port and correct variable quoting for guarding
- replaced $_pg_tmp_port with $POSTRGES_PORT in migration logic. Works because "pg_ctl stop -w" is run before entrypoint.sh restarts postgres. Verified that this does not cause a race condition.

- Correcting unquoted environment variables to prevent inappropriate variables from causing execution inconsistencies (still unquoted in comments and prints)
2026-03-12 20:25:54 -05:00
None
52ed0fc15a fix: run PostgreSQL as PUID/PGID user to resolve ownership conflicts (#1078)
PostgreSQL now runs as the PUID/PGID user ($POSTGRES_USER) instead of the internal postgres system user (UID 102). This fixes container startup failures when PUID/PGID is set, caused by chown permission errors on restricted filesystems (NFS root_squash, CIFS) and UID collisions with the postgres system user.

Changes:
- Run all PostgreSQL operations (initdb, pg_ctl, psql) as $POSTGRES_USER
- Auto-detect PUID/PGID from existing data owner when not explicitly set
- Validate PUID/PGID (reject zero, non-numeric values) before startup
- Migrate existing data ownership with sentinel-based skip optimization
- Use trust auth for local Unix sockets, md5 for network connections
- Add promote_app_role() and ensure_app_database() as idempotent startup guarantees that handle fresh installs, upgrades, and PUID changes
- Preserve postgres role as superuser for rollback compatibility
- Centralize /data/db ownership in 02-postgres.sh (sentinel-aware)
- Add integration test suite (20 scenarios) covering fresh installs, upgrades, restarts, PUID changes, UID collisions, bind mounts, modular mode, PG major upgrades, and end-to-end web UI verification
2026-03-11 23:36:32 -05:00
None
bb2c3c228d Fix stale env vars in profile.d on container restart
- Rewrite /etc/profile.d/dispatcharr.sh on every startup instead of only on first run, so container restarts with changed env vars pick up new values
- Quote exported values to prevent breakage from special characters in POSTGRES_PASSWORD or other vars
- Update /etc/environment entries instead of skipping if already present
2026-03-06 15:27:19 -06:00
None
4f41c287ac Fix Redis flush and wait_for_redis in modular mode
- Move modular Redis wait from uWSGI exec-pre to entrypoint (exec-pre runs under 'su -' which strips Docker env vars, so DISPATCHARR_ENV and REDIS_HOST were never available)
- Selective flush in modular mode: clears stale app state (stream locks, proxy metadata) while preserving Celery broker/result keys
- AIO mode unchanged: full flushdb via uWSGI exec-pre
- Update unit tests for both flush paths
2026-03-06 15:17:32 -06:00
None
39646f0b01 Fix modular mode PostgreSQL/Redis connection checks
Replace raw Python AF_INET socket checks with pg_isready (PostgreSQL) and socket.create_connection (Redis) in modular deployment mode

The raw socket check may fail in some Docker environments depending on networking and DNS configurations, causing indefinite hangs. Replaced database check with native pg_isready
and memstore checck with AF_UNSPEC in the case IPv6 is enabled.

Fixes #952
2026-02-10 23:00:35 -06:00
None
e217960500 feat: Add Redis authentication support for modular deployment
Add support for authentication when connecting to external Redis instances in modular deployment mode. This enables secure Redis deployments using either password-only authentication (Redis <6) or
username + password authentication (Redis 6+ ACL).

Changes:
- Add REDIS_PASSWORD and REDIS_USER environment variables
- Implement URL encoding for special characters in passwords
- Update all Redis connection points to support auth
- Add comprehensive documentation and examples to docker-compose.yml
- Maintains full backward compatibility (empty defaults = no auth)

All authentication mechanisms have been fully tested including pasword-only authentication, Redis 6+ ACL authentication with username + password, volume-mounted configuration files, and special character handling.

Existing deployments are not effected, authentication support is entirely opt-in using docker-compose.

Also, acknowledging the fact that the docker-compose file for modular deployments has been getting out of hand with auth support, the docker-compose.yml file was re-formatted for better visibility in configuration. This seemed like a better way to go than mandating a .env file.
2026-02-03 21:42:06 -06:00
SergeantPanda
fb8c831f1b
Merge pull request #881 from CodeBormen/fix-modular-compose-deployment
feat: Add DISPATCHARR_ENV=modular support for multi-container deployment
2026-02-03 09:49:07 -06:00
None
8eef088950 Add PostgreSQL version check for modular deployments
Implements automatic version validation for external databases in modular mode to ensure compatibility.

Changes:
- Added check_external_postgres_version() in 02-postgres.sh
- Integrated version check in entrypoint.sh after database connection
- Enforces minimum PostgreSQL version matching DispatcharrBase
- Allows newer versions with forward compatibility notice
- Rejects older versions with clear upgrade instructions

The version requirement automatically scales when DispatcharrBase is updated, requiring no manual maintenance. Only applies to modular deployments using external databases; AIO deployments are unaffected.

Tested and verified correct behavior with PostgreSQL v17, v18, and v16
2026-01-31 20:45:39 -06:00
SergeantPanda
bdcc865b9d Bug Fix: Fixed NumPy baseline detection in Docker entrypoint. Now properly detects when NumPy crashes on import due to CPU baseline incompatibility and installs legacy NumPy version. Previously, if NumPy failed to import, the script would skip legacy installation assuming it was already compatible.
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Base Image Build / prepare (push) Has been cancelled
Base Image Build / docker (amd64, ubuntu-24.04) (push) Has been cancelled
Base Image Build / docker (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Base Image Build / create-manifest (push) Has been cancelled
2026-01-30 16:35:56 -06:00
SergeantPanda
256d19a2cc Move legacy numpy install to after init-dev. UV replaces. 2026-01-30 12:45:11 -06:00
SergeantPanda
a3ed1d9628 Refactor for uv transition. 2026-01-30 12:12:42 -06:00
Matt Grutza
47b591bacb Add configurable POSTGRES_PORT and REDIS_PORT for external databases
Make database ports configurable for users with external PostgreSQL/Redis.

- Add POSTGRES_PORT and REDIS_PORT to docker-compose.yml (web and celery)
- Remove hardcoded CELERY_BROKER_URL (Django builds it from components)
- Add REDIS_PORT export to entrypoint.sh and profile variables
- Add Redis connection check for modular mode (matches PostgreSQL pattern)

Users can now specify custom ports for external databases:
  [defaults]
  POSTGRES_PORT=5432
  REDIS_PORT=6379
2026-01-23 21:28:53 -06:00
Matt Grutza
557e192d4c Add UTF8 encoding check for external databases
Extend ensure_utf8_encoding() to support both internal and external PostgreSQL.

- Modular mode: Uses TCP connection with PGPASSWORD authentication
- AIO mode: Uses Unix socket as postgres user
- Explicitly set database owner during recreation (fixes missing --owner flag)
- Conversion logic (dump/drop/recreate/restore) works for both AIO and modular modes
- Skip internal PostgreSQL setup entirely when DISPATCHARR_ENV=modular
2026-01-23 19:15:27 -06:00
Matt Grutza
236b2307e1 Add health check and fix dangerous checks
- docker-compose: added service health checks

- dockerfile: added if logic for sed command so missing files do not cause build fail

- entrypoint.celery: change migration detection to not continue until ALL migrations are applied

- entrypoint: trim django key and change external postgres check to python in case postgres binaries are missing
2026-01-21 17:40:36 -06:00
Matt Grutza
3706e63bed feat: Add DISPATCHARR_ENV=modular support for multi-container deployments
- docker/entrypoint.sh: Conditional PostgreSQL init/startup for modular mode
- docker/entrypoint.celery.sh: New dedicated entrypoint for celery container - prevents race condition in modular deployment
- docker/uwsgi.modular.ini: New uWSGI config without Redis/Celery daemons
- docker/Dockerfile: Add line ending fixes and chmod for entrypoints - adds cross-platform support for image builds
- docker/docker-compose.yml: Configure modular mode with DISPATCHARR_ENV - uses new celery entrypoint and adds depends_on to prevent race condition
2026-01-21 11:14:28 -06:00
SergeantPanda
38fa0fe99d Bug Fix: Fixed NumPy baseline detection in Docker entrypoint. Now calls numpy.show_config() directly with case-insensitive grep instead of incorrectly wrapping the output.
Some checks are pending
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
2026-01-14 17:10:06 -06:00
SergeantPanda
6d5d16d667 Enhancement: Add check for existing NumPy baseline support before reinstalling legacy NumPy to avoid unnecessary installations. 2026-01-12 12:29:54 -06:00
SergeantPanda
518c93c398 Enhance Docker setup for legacy NumPy support and streamline installation process 2026-01-06 14:07:37 -06:00
sethwv-alt
b157159b87
Fix root-owned __pycache__ by running Django commands as non-root user 2025-12-31 12:16:19 -05:00
SergeantPanda
e2736babaa Reset umask after creating secret file. 2025-12-07 20:04:58 -06:00
SergeantPanda
2155229d7f Fix uwsgi command path in entrypoint script 2025-12-07 19:40:32 -06:00
dekzter
d0edc3fa07 remove permission lines to see if this resolves lack of django secret key in environment profile.d 2025-12-07 07:54:30 -05:00
dekzter
f3a901cb3a Security Fix - generate JWT on application init 2025-12-06 13:40:10 -05:00
SergeantPanda
0a6f9eb8e1 Bug fix: Fixes uWSGI not getting environmental variables passed to it and LXC not being able to access daemons launched by uWSGI.
Fixes #575, #576, #577
2025-10-22 15:31:12 -05:00
SergeantPanda
73bb1ecd2d Trying a different approach to fix LXC issues. 2025-10-22 08:26:15 -05:00
SergeantPanda
645c1ec9df Testing redis potential fix. Launch redis in entrypoint before uWSGI 2025-10-21 21:47:15 -05:00
SergeantPanda
119b222428 Enhancement: Allow setting both Celery and UWSGI nice levels. Default nice levels are UWSGI=0, Celery=5. Added documentation to the compose files on how to use it. Using nice levels for UWSGI allows us to have streaming run at a high priority. 2025-10-21 10:11:53 -05:00
SergeantPanda
7cbdb61f2c Enhancement: Ensure root's .bashrc sources Dispatcharr profile scripts for interactive non-login shells. This will help when running root commands from the container cli 2025-10-18 19:08:02 -05:00
SergeantPanda
f9a9d5d336 Convert postgres to UTF8 from ASCII 2025-08-02 19:02:57 -05:00
SergeantPanda
108a992643 Detect mismatched Postgres version and automatically run pg_upgrade 2025-07-31 14:53:55 -05:00
SergeantPanda
c3d1600c07 Additional logging. 2025-06-27 20:22:49 -05:00
SergeantPanda
44a79d2a8a Log UWSGI if debug is set (DISPATCHARR_DEBUG=true) 2025-05-14 18:49:46 -05:00
SergeantPanda
cff68625e0 Move hardware detection to end of entrypoint so it's easier to find. 2025-05-13 12:44:32 -05:00
SergeantPanda
0953e044b7 Huge improvement to hardware acceleration script. Renamed for accuracy. 2025-05-12 19:46:23 -05:00
SergeantPanda
ade8a625bc Refactor environment variable handling in entrypoint.sh for consistency and efficiency 2025-05-12 13:57:22 -05:00
SergeantPanda
da9a78c875 Only write LIBVA_DRIVER_NAME to dispatcharr.sh if it's set by the user. 2025-05-12 13:16:15 -05:00
SergeantPanda
87798f4434 Only set LIBVA_DRIVER_NAME if user has it specified. Fixes but where FFmpeg wasn't auto detecting the correct driver for VAAPI and QSV. 2025-05-12 12:52:11 -05:00
SergeantPanda
8dd3bd6878 Use variables in export. 2025-05-12 09:10:38 -05:00
SergeantPanda
b85a5be023 Add LD_LIBRARY_PATH to dispatch user and global. 2025-05-12 08:44:07 -05:00
SergeantPanda
3a06b12f2c Hopefully fixes QSV hardware acceleration. 2025-05-11 12:45:57 -05:00