From db6755a009e587c21c0f368ac9d2701a2ec677ff Mon Sep 17 00:00:00 2001 From: SergeantPanda Date: Thu, 30 Apr 2026 14:28:26 -0500 Subject: [PATCH] Enhancement: Add database index on Stream.name to improve query performance and reduce disk spill during sorting (Fixes #1209) --- CHANGELOG.md | 2 +- .../migrations/0036_alter_stream_name.py | 18 ++++++++++++++++++ apps/channels/models.py | 2 +- 3 files changed, 20 insertions(+), 2 deletions(-) create mode 100644 apps/channels/migrations/0036_alter_stream_name.py diff --git a/CHANGELOG.md b/CHANGELOG.md index e98d8e20..3df32e44 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,8 +61,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Eliminated a redundant `Stream` DB lookup on stream switches. `get_stream_info_for_switch()` already fetches the `Stream` object to build the URL; it now includes `stream_name` in its return dict. `change_stream_url()` captures that value and passes it through to `_update_channel_metadata()`, which skips its own `Stream.objects.filter()` when the name is already known. - **Dedicated thread-pool Celery worker for DVR recordings**. `run_recording` is long-running and almost entirely I/O-bound: it loops in short ticks polling FFmpeg, the DB, and Redis for the full duration of the recording. Running it on the default prefork worker pool (`--autoscale=6,1`) meant at most 6 concurrent recordings, and only if no other background work was running. M3U refreshes, EPG parsing, channel matching, comskip, etc. all competed for the same 6 slots, so if every worker was busy when a recording's start time arrived, the task would queue in Redis and FFmpeg would not start until a worker freed up, causing the user to miss the beginning of the show. - A second Celery worker is now started alongside the default one, configured with `--pool=threads --concurrency=20` and bound to a dedicated `dvr` queue. Threads fit this workload because every blocking call in `run_recording` releases the GIL (sleep, subprocess wait, file / DB / Redis I/O) and FFmpeg itself runs in a separate OS process. A `task_routes` entry in `dispatcharr/celery.py` routes `apps.channels.tasks.run_recording` to the `dvr` queue regardless of how it is dispatched (Celery Beat `PeriodicTask`, `.delay()`, etc.). The default prefork worker now subscribes only to the `celery` queue (`-Q celery -n default@%h --autoscale=6,1`), so recordings can no longer starve background tasks and background tasks can no longer delay recordings. - - Net result: up to 20 concurrent recordings, zero startup delay regardless of EPG / M3U refresh activity, and a memory cost of roughly 80 to 120 MB for the second always-on worker process. Each additional concurrent recording costs nearly nothing on top because all 20 threads share that single process. Applied to AIO (`docker/uwsgi.ini`), dev (`docker/uwsgi.dev.ini`), debug (`docker/uwsgi.debug.ini`), and the modular celery container (`docker/entrypoint.celery.sh`). +- **Added database index on `Stream.name`**. The stream list default sort is `ORDER BY name`, but the column had no index, causing full table scans that spilled to disk (observed at ~38 MB temp file / ~800 ms) on large stream libraries. `db_index=True` is now set on the field, letting PostgreSQL satisfy the sort via an index scan with no disk spill. (Fixes #1209) ### Changed diff --git a/apps/channels/migrations/0036_alter_stream_name.py b/apps/channels/migrations/0036_alter_stream_name.py new file mode 100644 index 00000000..a008cb60 --- /dev/null +++ b/apps/channels/migrations/0036_alter_stream_name.py @@ -0,0 +1,18 @@ +# Generated by Django 6.0.4 on 2026-04-30 19:22 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('dispatcharr_channels', '0035_alter_channel_name_alter_stream_name'), + ] + + operations = [ + migrations.AlterField( + model_name='stream', + name='name', + field=models.CharField(db_index=True, default='Default Stream', max_length=512), + ), + ] diff --git a/apps/channels/models.py b/apps/channels/models.py index 1c989855..c0de0fe1 100644 --- a/apps/channels/models.py +++ b/apps/channels/models.py @@ -56,7 +56,7 @@ class Stream(models.Model): Represents a single stream (e.g. from an M3U source or custom URL). """ - name = models.CharField(max_length=512, default="Default Stream") + name = models.CharField(max_length=512, default="Default Stream", db_index=True) url = models.URLField(max_length=4096, blank=True, null=True) m3u_account = models.ForeignKey( M3UAccount,