mirror of
https://github.com/Dispatcharr/Dispatcharr.git
synced 2026-07-17 16:50:53 +00:00
Bug Fix: M3U profile URL rewriting now uses the regex module instead of re across all URL transform code paths (url_utils.transform_url, core/views.py, vod_proxy/_transform_url, tasks.get_transformed_credentials, and the WebSocket live-preview handler in consumers.py). The regex module natively accepts JavaScript/PCRE-style named capture groups ((?<name>...)) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a re.error. As a further improvement, regex also supports variable-length lookbehind assertions (e.g. (?<=a+)), which re rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling regex.sub: $<name> → \g<name> and $1/$2/… → \1/\2/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled
This commit is contained in:
parent
9579248485
commit
086cc74959
7 changed files with 29 additions and 18 deletions
|
|
@ -59,6 +59,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
### Fixed
|
||||
|
||||
- M3U profile URL rewriting now uses the `regex` module instead of `re` across all URL transform code paths (`url_utils.transform_url`, `core/views.py`, `vod_proxy/_transform_url`, `tasks.get_transformed_credentials`, and the WebSocket live-preview handler in `consumers.py`). The `regex` module natively accepts JavaScript/PCRE-style named capture groups (`(?<name>...)`) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a `re.error`. As a further improvement, `regex` also supports variable-length lookbehind assertions (e.g. `(?<=a+)`), which `re` rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling `regex.sub`: `$<name>` → `\g<name>` and `$1`/`$2`/… → `\1`/`\2`/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
|
||||
- Web UI stream preview (`FloatingVideo`) was calling `mpegts.createPlayer()` with all `Config` options (e.g. `enableWorker`, `liveSync`, `headers`) merged into the first `MediaDataSource` argument. mpegts.js only reads `Config` from the optional second argument; unrecognised fields in the first are silently ignored. As a result all player configuration was effectively the library defaults — worker offloading was disabled, latency management had no effect, and the `Authorization: Bearer` header (required for user identification) was never sent. Fixed by splitting into the correct two-argument call. Both `liveBufferLatencyChasing` and `liveSync` have been disabled, eliminating playback-rate fluctuations that caused audible stuttering on live streams. SourceBuffer cleanup thresholds were also relaxed from 10s/5s to 120s/60s to prevent frequent SourceBuffer pauses.
|
||||
- HTML named entities in XMLTV EPG files are now resolved to Unicode characters before lxml parsing. Some EPG providers (particularly French and other European sources) use HTML named entities like `é`, `î`, `ü` in channel names, program titles, and metadata. These are not valid XML entities — lxml 6.0.2 with `recover=True` silently drops them, causing characters to go missing (e.g., "Chaîne Télé" becomes "Chane Tl"). A preprocessing step in `fetch_xmltv()` now resolves HTML named entities while preserving the 5 XML-predefined entities, detecting encoding from the XML declaration (falling back to UTF-8), and processing line-by-line to a temp file before atomically replacing the original. (Closes #1095) — Thanks [@CodeBormen](https://github.com/CodeBormen)
|
||||
- Duplicate recordings created when EPG sources refresh and re-evaluate series rules (Fixes #940) — Thanks [@CodeBormen](https://github.com/CodeBormen):
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
# apps/m3u/tasks.py
|
||||
import logging
|
||||
import re
|
||||
import regex
|
||||
import requests
|
||||
import os
|
||||
import gc
|
||||
|
|
@ -2399,11 +2400,13 @@ def get_transformed_credentials(account, profile=None):
|
|||
# Apply profile-specific transformations if profile is provided
|
||||
if profile and profile.search_pattern and profile.replace_pattern:
|
||||
try:
|
||||
# Handle backreferences in the replacement pattern
|
||||
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', profile.replace_pattern)
|
||||
# Handle backreferences: convert JS-style $<name> -> \g<name>, $1 -> \1
|
||||
# regex module accepts JS-style (?<name>...) named groups natively
|
||||
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', profile.replace_pattern)
|
||||
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
|
||||
|
||||
# Apply transformation to the complete URL
|
||||
transformed_complete_url = re.sub(profile.search_pattern, safe_replace_pattern, complete_url)
|
||||
transformed_complete_url = regex.sub(profile.search_pattern, safe_replace_pattern, complete_url)
|
||||
logger.info(f"Transformed complete URL: {complete_url} -> {transformed_complete_url}")
|
||||
|
||||
# Extract components from the transformed URL
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ Utilities for handling stream URLs and transformations.
|
|||
"""
|
||||
|
||||
import logging
|
||||
import re
|
||||
import regex
|
||||
from typing import Optional, Tuple, List
|
||||
from django.shortcuts import get_object_or_404
|
||||
from apps.channels.models import Channel, Stream
|
||||
|
|
@ -146,13 +146,14 @@ def transform_url(input_url: str, search_pattern: str, replace_pattern: str) ->
|
|||
logger.debug(f" base URL: {input_url}")
|
||||
logger.debug(f" search: {search_pattern}")
|
||||
|
||||
# Handle backreferences in the replacement pattern
|
||||
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', replace_pattern)
|
||||
# Convert JS-style backreferences in replace pattern: $<name> -> \g<name>, $1 -> \1
|
||||
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', replace_pattern)
|
||||
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
|
||||
logger.debug(f" replace: {replace_pattern}")
|
||||
logger.debug(f" safe replace: {safe_replace_pattern}")
|
||||
|
||||
# Apply the transformation
|
||||
stream_url = re.sub(search_pattern, safe_replace_pattern, input_url)
|
||||
# Apply the transformation (regex module accepts JS-style (?<name>...) natively)
|
||||
stream_url = regex.sub(search_pattern, safe_replace_pattern, input_url)
|
||||
logger.info(f"Generated stream url: {stream_url}")
|
||||
|
||||
return stream_url
|
||||
|
|
|
|||
|
|
@ -270,17 +270,20 @@ def _get_m3u_profile(m3u_account, profile_id, session_id=None):
|
|||
def _transform_url(original_url, m3u_profile):
|
||||
"""Transform URL based on M3U profile settings"""
|
||||
try:
|
||||
import re
|
||||
import regex
|
||||
|
||||
if not original_url:
|
||||
return None
|
||||
|
||||
search_pattern = m3u_profile.search_pattern
|
||||
replace_pattern = m3u_profile.replace_pattern
|
||||
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', replace_pattern)
|
||||
# Convert JS-style backreferences in replace: $<name> -> \g<name>, $1 -> \1
|
||||
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', replace_pattern)
|
||||
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
|
||||
|
||||
if search_pattern and replace_pattern:
|
||||
transformed_url = re.sub(search_pattern, safe_replace_pattern, original_url)
|
||||
# regex module accepts JS-style (?<name>...) named groups natively
|
||||
transformed_url = regex.sub(search_pattern, safe_replace_pattern, original_url)
|
||||
return transformed_url
|
||||
|
||||
return original_url
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ from shlex import split as shlex_split
|
|||
import sys
|
||||
import subprocess
|
||||
import logging
|
||||
import re
|
||||
import regex
|
||||
import redis
|
||||
|
||||
from django.conf import settings
|
||||
|
|
@ -132,10 +132,13 @@ def stream_view(request, channel_uuid):
|
|||
# Prepare the pattern replacement.
|
||||
logger.debug("Executing the following pattern replacement:")
|
||||
logger.debug(f" search: {active_profile.search_pattern}")
|
||||
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', active_profile.replace_pattern)
|
||||
# Convert JS-style backreferences in replace: $<name> -> \g<name>, $1 -> \1
|
||||
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', active_profile.replace_pattern)
|
||||
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
|
||||
logger.debug(f" replace: {active_profile.replace_pattern}")
|
||||
logger.debug(f" safe replace: {safe_replace_pattern}")
|
||||
stream_url = re.sub(active_profile.search_pattern, safe_replace_pattern, input_url)
|
||||
# regex module accepts JS-style (?<name>...) named groups natively
|
||||
stream_url = regex.sub(active_profile.search_pattern, safe_replace_pattern, input_url)
|
||||
logger.debug(f"Generated stream url: {stream_url}")
|
||||
|
||||
# Get the stream profile set on the channel.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import json
|
||||
from channels.generic.websocket import AsyncWebsocketConsumer
|
||||
import re, logging
|
||||
import regex, logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
|
@ -54,9 +54,9 @@ class MyWebSocketConsumer(AsyncWebsocketConsumer):
|
|||
|
||||
# Apply the transformation using the replace_with_mark function
|
||||
try:
|
||||
search_preview = re.sub(data["search"], replace_with_mark, data["url"])
|
||||
search_preview = regex.sub(data["search"], replace_with_mark, data["url"])
|
||||
except Exception as e:
|
||||
search_preview = data["search"]
|
||||
search_preview = data["url"]
|
||||
logger.error(f"Failed to generate replace preview: {e}")
|
||||
|
||||
result = transform_url(data["url"], data["search"], data["replace"])
|
||||
|
|
|
|||
|
|
@ -290,7 +290,7 @@ const RegexFormAndView = ({ profile = null, m3u, isOpen, onClose }) => {
|
|||
setXcMode(mode);
|
||||
};
|
||||
|
||||
// Local regex testing for immediate visual feedback
|
||||
// Local regex for the live demo preview
|
||||
const getHighlightedSearchText = () => {
|
||||
if (!searchPattern || !sampleInput) return sampleInput;
|
||||
try {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue