enhancement(tests): introduce isolated backend test settings and improve test documentation

- Added `dispatcharr.settings_test` for isolated testing, automatically creating an empty PostgreSQL test database and ensuring transaction isolation during tests.
- Updated `manage.py` to switch to the test settings when running tests.
- Enhanced the CONTRIBUTING.md file with detailed instructions on running the backend test suite and handling Celery tasks in tests.
- Refactored test cases to use static methods for `worker_id` in `test_process_label.py` and adjusted assertions in `test_user_preferences.py` for better clarity and correctness.
This commit is contained in:
SergeantPanda 2026-06-23 20:54:12 -05:00
parent 7b6adf62d7
commit 53fa1e42a0
6 changed files with 85 additions and 6 deletions

View file

@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **Isolated backend test settings (`dispatcharr.settings_test`).** `python manage.py test` now switches to this module automatically (via `manage.py`). It creates an empty PostgreSQL `test_<dbname>` database (same engine as production), uses the standard Postgres backend instead of geventpool so `TestCase` transactions isolate correctly, and leaves Celery tasks queued (no eager `post_save` signal runs during tests). Set `TEST_USE_SQLITE=1` for an in-memory SQLite fallback when Postgres is unavailable.
### Performance
- **`get_vod_streams` and `get_series` XC API endpoints are faster and no longer exhaust Docker `/dev/shm`.** Large libraries (e.g. 125k movies) previously ran one wide `DISTINCT ON` query with parallel workers, which could fail with `could not resize shared memory segment … No space left on device` on the default 64MB container shm. Both endpoints now dedupe on narrow relation rows first (`SET LOCAL max_parallel_workers_per_gather = 0`), then fetch display columns via `.values()` (no ORM model instantiation per row). Redundant `category` and `logo` joins were dropped in favor of FK ids; alphabetical sort runs in SQL. Typical full-library response time drops from ~2328s to ~810s with stable shm usage.

View file

@ -144,7 +144,19 @@ Untested code is significantly less likely to be merged.
- Use Django's `TestCase` for unit/integration tests.
- Test files live at `apps/<app>/tests/`.
- Run the test suite with: `uv run python manage.py test`
- Run the backend test suite with:
```bash
python manage.py test
```
`manage.py` automatically uses `dispatcharr.settings_test`, which creates an empty PostgreSQL database `test_<dbname>` (same engine as production), runs migrations, and rolls back each test in a transaction. Your live VOD/channels data is not used.
Optional: `TEST_USE_SQLITE=1` for machines without Postgres (some PostgreSQL-only tests skip automatically).
Tests that exercise Celery task bodies should use `@override_settings(CELERY_TASK_ALWAYS_EAGER=True)` locally. Global eager mode is off because `post_save` signals on M3U/EPG models call `.delay()` and would break `TestCase` transaction isolation.
- Do **not** override with `--settings=dispatcharr.settings` on a live instance.
### Frontend

View file

@ -0,0 +1,58 @@
"""
Django settings for running the backend test suite in isolation.
Always use this module instead of dispatcharr.settings when running tests:
python manage.py test
`manage.py` selects this module automatically for the ``test`` command.
Django creates a separate empty database (``test_<POSTGRES_DB>``) and runs
migrations your live data under /data/db is not used.
Why not dispatcharr.settings?
- Production/AIO points at the live ``dispatcharr`` database.
- django-db-geventpool breaks TestCase transaction isolation on pooled connections.
SQLite (``TEST_USE_SQLITE=1``) is an optional fallback for machines without
Postgres; production and CI should use the default Postgres test database.
"""
import os
from dispatcharr.settings import * # noqa: F401,F403
# Fast password hashing for tests.
PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
# Do NOT run Celery tasks inline during tests. post_save signals on M3UAccount and
# EPGSource call .delay(); eager mode runs them inside TestCase transactions and
# closes/poisons the DB connection for subsequent queries in the same test.
CELERY_TASK_ALWAYS_EAGER = False
CELERY_TASK_EAGER_PROPAGATES = False
_use_sqlite = os.environ.get("TEST_USE_SQLITE", "").lower() in ("1", "true", "yes")
if _use_sqlite:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": ":memory:",
}
}
else:
# Default: PostgreSQL with Django-managed test_dispatcharr (matches production).
# Uses the standard backend (not geventpool) so TestCase transactions isolate.
_pg_name = os.environ.get("POSTGRES_DB", "dispatcharr")
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": _pg_name,
"USER": os.environ.get("POSTGRES_USER", "dispatch"),
"PASSWORD": os.environ.get("POSTGRES_PASSWORD", "secret"),
"HOST": os.environ.get("POSTGRES_HOST", "localhost"),
"PORT": int(os.environ.get("POSTGRES_PORT", 5432)),
"TEST": {
"NAME": "test_" + _pg_name,
},
}
}

View file

@ -5,7 +5,12 @@ import sys
def main():
"""Run administrative tasks."""
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'dispatcharr.settings')
# Use isolated test DB settings for `manage.py test` (empty test_<dbname>).
# Override with --settings=... on the command line if needed.
if len(sys.argv) > 1 and sys.argv[1] == "test":
os.environ["DJANGO_SETTINGS_MODULE"] = "dispatcharr.settings_test"
else:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "dispatcharr.settings")
try:
from django.core.management import execute_from_command_line
except ImportError as exc:

View file

@ -17,13 +17,13 @@ class ProcessLabelTests(SimpleTestCase):
self.assertEqual(role, "uwsgi")
def test_uwsgi_labeled_when_worker_module_present(self):
fake_uwsgi = type("uwsgi", (), {"worker_id": lambda: 2})()
fake_uwsgi = type("uwsgi", (), {"worker_id": staticmethod(lambda: 2)})()
with patch.dict("sys.modules", {"uwsgi": fake_uwsgi}):
role = get_process_role(["/dispatcharrpy/bin/python", "-c", "pass"])
self.assertEqual(role, "uwsgi")
def test_uwsgi_master_not_labeled_as_uwsgi(self):
fake_uwsgi = type("uwsgi", (), {"worker_id": lambda: 0})()
fake_uwsgi = type("uwsgi", (), {"worker_id": staticmethod(lambda: 0)})()
with patch.dict("sys.modules", {"uwsgi": fake_uwsgi}):
role = get_process_role(["/dispatcharrpy/bin/python", "-c", "pass"])
self.assertEqual(role, "django")

View file

@ -128,13 +128,13 @@ class UserPreferencesAPITests(TestCase):
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
def test_patch_me_cannot_escalate_privileges(self):
"""Test PATCH /me/ rejects attempts to change user_level or is_staff"""
"""PATCH /me/ ignores privilege fields; they are stripped before save."""
original_level = self.user.user_level
data = {"user_level": 99, "is_staff": True, "is_superuser": True}
response = self.client.patch(self.me_url, data, format="json")
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.user.refresh_from_db()
self.assertEqual(self.user.user_level, original_level)