refactor(http headers): centralize User-Agent construction and streamline API requests

- Introduced `dispatcharr_user_agent`, `dispatcharr_dvr_user_agent`, and `dispatcharr_http_headers` functions in `core.utils` to standardize User-Agent strings and HTTP headers for outbound requests.
- Updated various components, including `LogoViewSet`, `EPGSourceViewSet`, and VOD proxy views, to utilize the new header functions, ensuring consistent User-Agent usage across the application.
- Enhanced the handling of Schedules Direct API requests by including proper User-Agent headers, addressing previous API compliance issues.
This commit is contained in:
SergeantPanda 2026-06-11 09:45:35 -05:00
parent 5f9fa4c11f
commit 843940f552
9 changed files with 81 additions and 37 deletions

View file

@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Live XC upstream URLs use current credentials.** `_resolve_live_stream_url()` builds `/live/{user}/{pass}/{stream_id}.ts` from transformed account credentials and the stream's provider `stream_id`, so playback stays aligned with the active login after credential or profile changes instead of reusing a stale `stream.url` from sync.
- **Channel stream switches check pooled capacity.** `get_stream_info_for_switch()` uses `profile_available_for_channel_switch()` when targeting a specific stream, and releases a reserved slot if URL assembly fails after `get_stream()` allocated one.
- **Live proxy init reads Redis assignment once.** After `generate_stream_url()` reserves a slot, the stream handler reads `channel_stream` / `stream_profile` from Redis instead of calling `get_stream()` again, avoiding double INCR under concurrent release.
- **Centralized Dispatcharr User-Agent construction in `core.utils`.** Outbound HTTP calls (Schedules Direct API, update checks, logo/VOD fallbacks, DVR recording clients) now use `dispatcharr_user_agent()`, `dispatcharr_dvr_user_agent()`, and `dispatcharr_http_headers()` instead of ad-hoc `Dispatcharr/{version}` strings and stale `Dispatcharr/1.0` fallbacks.
- **EPG auto-match overhaul** — matching logic moved to `apps/channels/epg_matching.py`; Celery tasks in `tasks.py` are thin wrappers.
- Single-channel auto-match is now asynchronous: the API returns `202 Accepted` and pushes the result over WebSocket (`single_channel_epg_match`), so large EPG libraries no longer hit the previous 30-second HTTP timeout.
- Progress, bulk completion, and single-channel results use `send_websocket_update` instead of `async_to_sync(channel_layer.group_send)`, so notifications work reliably under gevent-patched uWSGI and Celery workers.
@ -46,6 +47,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Wrong channel assignments from global ML similarity; ML validation now checks the fuzzy best match (or top fuzzy candidates as a last resort) instead of scoring the entire catalog.
- Channel form auto-match spinner could stick after errors or early task exits; all single-channel outcomes now push a WebSocket result, and the UI clears loading state after a 3-minute timeout.
- Bulk auto-match completion no longer calls `batch-set-epg` from the WebSocket handler, which had been re-applying every match and queueing redundant `parse_programs_for_tvg_id` tasks even when assignments were unchanged.
- **Schedules Direct lineup search country dropdown.** The country list was fetched directly from the SD API in the browser, which failed due to CORS and silently fell back to a 14-country hardcoded list. Countries are now included in the `GET sd-lineups` response (server-side fetch with proper User-Agent) so the full SD country list populates correctly. — Thanks [@sethwv](https://github.com/sethwv)
- **Schedules Direct program poster proxy omitted User-Agent on image requests.** The poster endpoint authenticated with SD correctly but fetched images with only the `token` header, which violates SD's API requirements (error 1003). Image requests now include the standard `Dispatcharr/{version}` User-Agent via `dispatcharr_http_headers()`.
## [0.26.0] - 2026-06-07

View file

@ -2800,8 +2800,9 @@ class LogoViewSet(viewsets.ModelViewSet):
user_agent_obj = UserAgent.objects.get(id=int(default_user_agent_id))
user_agent = user_agent_obj.user_agent
except (CoreSettings.DoesNotExist, UserAgent.DoesNotExist, ValueError):
# Fallback to hardcoded if default not found
user_agent = 'Dispatcharr/1.0'
# Fallback if default not found
from core.utils import dispatcharr_user_agent
user_agent = dispatcharr_user_agent()
# Hard total timeout (connect + full download) prevents a slow
# server dripping bytes from holding a greenlet indefinitely.

View file

@ -1181,12 +1181,13 @@ def _dvr_ffmpeg_retry_backoff_seconds(retry_index):
def _dvr_build_ffmpeg_cmd(stream_url, recording_id, hls_m3u8, hls_seg_pattern, hls_start_number):
"""Build the FFmpeg command for DVR HLS segment recording."""
from core.utils import dispatcharr_dvr_user_agent
return [
"ffmpeg", "-y",
"-reconnect", "1",
"-reconnect_streamed", "1",
"-reconnect_delay_max", "5",
"-user_agent", f"Dispatcharr-DVR/recording-{recording_id}",
"-user_agent", dispatcharr_dvr_user_agent(recording_id),
# Regenerate monotonic PTS to handle erratic/discontinuous timestamps
# from IPTV sources.
"-fflags", "+genpts",

View file

@ -128,7 +128,7 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
import hashlib
import requests as http_requests
from apps.epg.tasks import SD_BASE_URL
from version import __version__ as dispatcharr_version
from core.utils import dispatcharr_http_headers
username = (source.username or '').strip()
password = (source.password or '').strip()
@ -143,7 +143,7 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
auth_response = http_requests.post(
f"{SD_BASE_URL}/token",
json={'username': username, 'password': sha1_password},
headers={'Content-Type': 'application/json', 'User-Agent': f'Dispatcharr/{dispatcharr_version}'},
headers=dispatcharr_http_headers(),
timeout=15,
)
auth_response.raise_for_status()
@ -231,12 +231,12 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
"""Fetch the SD country list (token not required; User-Agent is)."""
import requests as http_requests
from apps.epg.tasks import SD_BASE_URL
from version import __version__ as dispatcharr_version
from core.utils import dispatcharr_http_headers
try:
resp = http_requests.get(
f"{SD_BASE_URL}/available/countries",
headers={'User-Agent': f'Dispatcharr/{dispatcharr_version}'},
headers=dispatcharr_http_headers(content_type=None),
timeout=15,
)
resp.raise_for_status()
@ -254,7 +254,7 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
"""
import requests as http_requests
from apps.epg.tasks import SD_BASE_URL
from version import __version__ as dispatcharr_version
from core.utils import dispatcharr_http_headers
source = self.get_object()
if source.source_type != 'schedules_direct':
@ -267,11 +267,7 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
if error:
return error
headers = {
'Content-Type': 'application/json',
'User-Agent': f'Dispatcharr/{dispatcharr_version}',
'token': token,
}
headers = dispatcharr_http_headers(token=token)
if request.method == "GET":
countries = self._fetch_sd_countries()
@ -420,7 +416,7 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
"""
import requests as http_requests
from apps.epg.tasks import SD_BASE_URL
from version import __version__ as dispatcharr_version
from core.utils import dispatcharr_http_headers
source = self.get_object()
if source.source_type != 'schedules_direct':
@ -441,11 +437,7 @@ class EPGSourceViewSet(viewsets.ModelViewSet):
if error:
return error
headers = {
'Content-Type': 'application/json',
'User-Agent': f'Dispatcharr/{dispatcharr_version}',
'token': token,
}
headers = dispatcharr_http_headers(token=token)
try:
resp = http_requests.get(
@ -514,6 +506,7 @@ class ProgramViewSet(viewsets.ModelViewSet):
"""Proxy endpoint for SD program poster images. Nginx caches the response."""
import requests as http_requests
from apps.epg.tasks import SD_BASE_URL
from core.utils import dispatcharr_http_headers
program = self.get_object()
poster_sd_url = (program.custom_properties or {}).get('sd_icon')
@ -537,14 +530,10 @@ class ProgramViewSet(viewsets.ModelViewSet):
if not token:
sha1_password = hashlib.sha1(source.password.encode('utf-8')).hexdigest()
try:
from version import __version__ as dispatcharr_version
auth_resp = http_requests.post(
f"{SD_BASE_URL}/token",
json={'username': source.username, 'password': sha1_password},
headers={
'Content-Type': 'application/json',
'User-Agent': f'Dispatcharr/{dispatcharr_version}',
},
headers=dispatcharr_http_headers(),
timeout=10,
)
auth_data = auth_resp.json()
@ -570,7 +559,7 @@ class ProgramViewSet(viewsets.ModelViewSet):
try:
img_resp = http_requests.get(
poster_sd_url,
headers={'token': token},
headers=dispatcharr_http_headers(token=token, content_type=None),
timeout=15,
allow_redirects=True,
)

View file

@ -2173,17 +2173,10 @@ def fetch_schedules_direct(source, stations_only=False, force=False):
# SD API spec requires the User-Agent to identify the application and version.
# SergeantPanda confirmed Dispatcharr should identify itself properly.
# -------------------------------------------------------------------------
from version import __version__ as dispatcharr_version
sd_user_agent = f"Dispatcharr/{dispatcharr_version}"
from core.utils import dispatcharr_http_headers
def _sd_headers(token=None):
h = {
'Content-Type': 'application/json',
'User-Agent': sd_user_agent,
}
if token:
h['token'] = token
return h
return dispatcharr_http_headers(token=token)
def _sd_post_refresh_tasks(mapped_epg_ids, program_metadata, today):
"""Poster fetch, logo auto-apply, and pruning — runs even when schedules are unchanged."""

View file

@ -23,6 +23,7 @@ from rest_framework_simplejwt.authentication import JWTAuthentication
from apps.accounts.authentication import ApiKeyAuthentication, QueryParamJWTAuthentication
from apps.proxy.utils import check_user_stream_limits
from dispatcharr.utils import network_access_allowed
from core.utils import dispatcharr_user_agent
logger = logging.getLogger(__name__)
@ -704,7 +705,7 @@ def head_vod(request, content_type, content_id, session_id=None, profile_id=None
# Use M3U account's user agent as primary, client user agent as fallback
m3u_user_agent = m3u_account.get_user_agent().user_agent if m3u_account.get_user_agent() else None
headers = {
'User-Agent': m3u_user_agent or client_user_agent or 'Dispatcharr/1.0',
'User-Agent': m3u_user_agent or client_user_agent or dispatcharr_user_agent(),
'Accept': '*/*',
'Range': 'bytes=0-1' # Request only first 2 bytes
}

View file

@ -796,11 +796,11 @@ def check_for_version_update():
from packaging import version as pkg_version
from version import __version__, __timestamp__
from core.models import SystemNotification
from core.utils import send_websocket_notification
from core.utils import dispatcharr_http_headers, send_websocket_notification
try:
is_dev_build = __timestamp__ is not None
DISPATCHARR_HEADERS = {'User-Agent': f'Dispatcharr/{__version__}'}
DISPATCHARR_HEADERS = dispatcharr_http_headers(content_type=None)
if is_dev_build:
# Check Docker Hub for newer dev builds

View file

@ -6,6 +6,35 @@ from apps.epg.models import EPGSource
from core.models import CoreSettings, DVR_SETTINGS_KEY, EPG_SETTINGS_KEY
class DispatcharrUserAgentTests(TestCase):
@patch('version.__version__', '1.2.3')
def test_dispatcharr_user_agent(self):
from core.utils import dispatcharr_user_agent
self.assertEqual(dispatcharr_user_agent(), 'Dispatcharr/1.2.3')
def test_dispatcharr_dvr_user_agent(self):
from core.utils import dispatcharr_dvr_user_agent
self.assertEqual(dispatcharr_dvr_user_agent(42), 'Dispatcharr-DVR/recording-42')
@patch('version.__version__', '1.2.3')
def test_dispatcharr_http_headers_with_token(self):
from core.utils import dispatcharr_http_headers
headers = dispatcharr_http_headers(token='tok123')
self.assertEqual(headers, {
'User-Agent': 'Dispatcharr/1.2.3',
'Content-Type': 'application/json',
'token': 'tok123',
})
@patch('version.__version__', '1.2.3')
def test_dispatcharr_http_headers_without_content_type(self):
from core.utils import dispatcharr_http_headers
self.assertEqual(
dispatcharr_http_headers(content_type=None),
{'User-Agent': 'Dispatcharr/1.2.3'},
)
class ProgrammeIndexRebuildTests(TestCase):
def test_startup_rebuild_does_not_lock_out_queued_build_task(self):
EPGSource.objects.update(

View file

@ -21,6 +21,33 @@ logger = logging.getLogger(__name__)
# Import the command detector
from .command_utils import is_management_command
def dispatcharr_user_agent():
"""Return the standard Dispatcharr User-Agent string (Dispatcharr/{version})."""
from version import __version__
return f'Dispatcharr/{__version__}'
def dispatcharr_dvr_user_agent(recording_id):
"""Return the User-Agent string used by DVR FFmpeg clients for a recording."""
return f'Dispatcharr-DVR/recording-{recording_id}'
def dispatcharr_http_headers(*, token=None, content_type='application/json'):
"""
Build HTTP headers for outbound Dispatcharr requests.
content_type=None omits Content-Type (e.g. simple GET proxies).
token is included when authenticating with Schedules Direct.
"""
headers = {'User-Agent': dispatcharr_user_agent()}
if content_type:
headers['Content-Type'] = content_type
if token:
headers['token'] = token
return headers
def natural_sort_key(text):
"""
Convert a string into a list of string and number chunks for natural sorting.