Enhancement: Add database index on Stream.name to improve query performance and reduce disk spill during sorting (Fixes #1209)

This commit is contained in:
SergeantPanda 2026-04-30 14:28:26 -05:00
parent 3801dd7aab
commit db6755a009
3 changed files with 20 additions and 2 deletions

View file

@ -61,8 +61,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Eliminated a redundant `Stream` DB lookup on stream switches. `get_stream_info_for_switch()` already fetches the `Stream` object to build the URL; it now includes `stream_name` in its return dict. `change_stream_url()` captures that value and passes it through to `_update_channel_metadata()`, which skips its own `Stream.objects.filter()` when the name is already known.
- **Dedicated thread-pool Celery worker for DVR recordings**. `run_recording` is long-running and almost entirely I/O-bound: it loops in short ticks polling FFmpeg, the DB, and Redis for the full duration of the recording. Running it on the default prefork worker pool (`--autoscale=6,1`) meant at most 6 concurrent recordings, and only if no other background work was running. M3U refreshes, EPG parsing, channel matching, comskip, etc. all competed for the same 6 slots, so if every worker was busy when a recording's start time arrived, the task would queue in Redis and FFmpeg would not start until a worker freed up, causing the user to miss the beginning of the show.
- A second Celery worker is now started alongside the default one, configured with `--pool=threads --concurrency=20` and bound to a dedicated `dvr` queue. Threads fit this workload because every blocking call in `run_recording` releases the GIL (sleep, subprocess wait, file / DB / Redis I/O) and FFmpeg itself runs in a separate OS process. A `task_routes` entry in `dispatcharr/celery.py` routes `apps.channels.tasks.run_recording` to the `dvr` queue regardless of how it is dispatched (Celery Beat `PeriodicTask`, `.delay()`, etc.). The default prefork worker now subscribes only to the `celery` queue (`-Q celery -n default@%h --autoscale=6,1`), so recordings can no longer starve background tasks and background tasks can no longer delay recordings.
- Net result: up to 20 concurrent recordings, zero startup delay regardless of EPG / M3U refresh activity, and a memory cost of roughly 80 to 120 MB for the second always-on worker process. Each additional concurrent recording costs nearly nothing on top because all 20 threads share that single process. Applied to AIO (`docker/uwsgi.ini`), dev (`docker/uwsgi.dev.ini`), debug (`docker/uwsgi.debug.ini`), and the modular celery container (`docker/entrypoint.celery.sh`).
- **Added database index on `Stream.name`**. The stream list default sort is `ORDER BY name`, but the column had no index, causing full table scans that spilled to disk (observed at ~38 MB temp file / ~800 ms) on large stream libraries. `db_index=True` is now set on the field, letting PostgreSQL satisfy the sort via an index scan with no disk spill. (Fixes #1209)
### Changed

View file

@ -0,0 +1,18 @@
# Generated by Django 6.0.4 on 2026-04-30 19:22
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('dispatcharr_channels', '0035_alter_channel_name_alter_stream_name'),
]
operations = [
migrations.AlterField(
model_name='stream',
name='name',
field=models.CharField(db_index=True, default='Default Stream', max_length=512),
),
]

View file

@ -56,7 +56,7 @@ class Stream(models.Model):
Represents a single stream (e.g. from an M3U source or custom URL).
"""
name = models.CharField(max_length=512, default="Default Stream")
name = models.CharField(max_length=512, default="Default Stream", db_index=True)
url = models.URLField(max_length=4096, blank=True, null=True)
m3u_account = models.ForeignKey(
M3UAccount,