Bug Fix: M3U profile URL rewriting now uses the regex module instead of re across all URL transform code paths (url_utils.transform_url, core/views.py, vod_proxy/_transform_url, tasks.get_transformed_credentials, and the WebSocket live-preview handler in consumers.py). The regex module natively accepts JavaScript/PCRE-style named capture groups ((?<name>...)) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a re.error. As a further improvement, regex also supports variable-length lookbehind assertions (e.g. (?<=a+)), which re rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling regex.sub: $<name>\g<name> and $1/$2/… → \1/\2/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
Some checks failed
CI Pipeline / prepare (push) Waiting to run
CI Pipeline / docker (amd64, ubuntu-24.04) (push) Blocked by required conditions
CI Pipeline / docker (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
CI Pipeline / create-manifest (push) Blocked by required conditions
Build and Push Multi-Arch Docker Image / build-and-push (push) Waiting to run
Frontend Tests / test (push) Has been cancelled

This commit is contained in:
SergeantPanda 2026-03-29 17:55:08 -05:00
parent 9579248485
commit 086cc74959
7 changed files with 29 additions and 18 deletions

View file

@ -59,6 +59,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- M3U profile URL rewriting now uses the `regex` module instead of `re` across all URL transform code paths (`url_utils.transform_url`, `core/views.py`, `vod_proxy/_transform_url`, `tasks.get_transformed_credentials`, and the WebSocket live-preview handler in `consumers.py`). The `regex` module natively accepts JavaScript/PCRE-style named capture groups (`(?<name>...)`) without any conversion, eliminating the root cause of patterns that matched in the frontend live preview but failed on the backend with a `re.error`. As a further improvement, `regex` also supports variable-length lookbehind assertions (e.g. `(?<=a+)`), which `re` rejects with an error; patterns using these will now work correctly on the backend as well. Replace-pattern JS tokens are still normalised before calling `regex.sub`: `$<name>``\g<name>` and `$1`/`$2`/… → `\1`/`\2`/… (Python replacement syntax). Also fixed a bug in the WebSocket preview handler where a pattern error was incorrectly returning the search pattern string as the preview output instead of the original URL. (Fixes #1005)
- Web UI stream preview (`FloatingVideo`) was calling `mpegts.createPlayer()` with all `Config` options (e.g. `enableWorker`, `liveSync`, `headers`) merged into the first `MediaDataSource` argument. mpegts.js only reads `Config` from the optional second argument; unrecognised fields in the first are silently ignored. As a result all player configuration was effectively the library defaults — worker offloading was disabled, latency management had no effect, and the `Authorization: Bearer` header (required for user identification) was never sent. Fixed by splitting into the correct two-argument call. Both `liveBufferLatencyChasing` and `liveSync` have been disabled, eliminating playback-rate fluctuations that caused audible stuttering on live streams. SourceBuffer cleanup thresholds were also relaxed from 10s/5s to 120s/60s to prevent frequent SourceBuffer pauses.
- HTML named entities in XMLTV EPG files are now resolved to Unicode characters before lxml parsing. Some EPG providers (particularly French and other European sources) use HTML named entities like `&eacute;`, `&icirc;`, `&uuml;` in channel names, program titles, and metadata. These are not valid XML entities — lxml 6.0.2 with `recover=True` silently drops them, causing characters to go missing (e.g., "Chaîne Télé" becomes "Chane Tl"). A preprocessing step in `fetch_xmltv()` now resolves HTML named entities while preserving the 5 XML-predefined entities, detecting encoding from the XML declaration (falling back to UTF-8), and processing line-by-line to a temp file before atomically replacing the original. (Closes #1095) — Thanks [@CodeBormen](https://github.com/CodeBormen)
- Duplicate recordings created when EPG sources refresh and re-evaluate series rules (Fixes #940) — Thanks [@CodeBormen](https://github.com/CodeBormen):

View file

@ -1,6 +1,7 @@
# apps/m3u/tasks.py
import logging
import re
import regex
import requests
import os
import gc
@ -2399,11 +2400,13 @@ def get_transformed_credentials(account, profile=None):
# Apply profile-specific transformations if profile is provided
if profile and profile.search_pattern and profile.replace_pattern:
try:
# Handle backreferences in the replacement pattern
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', profile.replace_pattern)
# Handle backreferences: convert JS-style $<name> -> \g<name>, $1 -> \1
# regex module accepts JS-style (?<name>...) named groups natively
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', profile.replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
# Apply transformation to the complete URL
transformed_complete_url = re.sub(profile.search_pattern, safe_replace_pattern, complete_url)
transformed_complete_url = regex.sub(profile.search_pattern, safe_replace_pattern, complete_url)
logger.info(f"Transformed complete URL: {complete_url} -> {transformed_complete_url}")
# Extract components from the transformed URL

View file

@ -3,7 +3,7 @@ Utilities for handling stream URLs and transformations.
"""
import logging
import re
import regex
from typing import Optional, Tuple, List
from django.shortcuts import get_object_or_404
from apps.channels.models import Channel, Stream
@ -146,13 +146,14 @@ def transform_url(input_url: str, search_pattern: str, replace_pattern: str) ->
logger.debug(f" base URL: {input_url}")
logger.debug(f" search: {search_pattern}")
# Handle backreferences in the replacement pattern
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', replace_pattern)
# Convert JS-style backreferences in replace pattern: $<name> -> \g<name>, $1 -> \1
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
logger.debug(f" replace: {replace_pattern}")
logger.debug(f" safe replace: {safe_replace_pattern}")
# Apply the transformation
stream_url = re.sub(search_pattern, safe_replace_pattern, input_url)
# Apply the transformation (regex module accepts JS-style (?<name>...) natively)
stream_url = regex.sub(search_pattern, safe_replace_pattern, input_url)
logger.info(f"Generated stream url: {stream_url}")
return stream_url

View file

@ -270,17 +270,20 @@ def _get_m3u_profile(m3u_account, profile_id, session_id=None):
def _transform_url(original_url, m3u_profile):
"""Transform URL based on M3U profile settings"""
try:
import re
import regex
if not original_url:
return None
search_pattern = m3u_profile.search_pattern
replace_pattern = m3u_profile.replace_pattern
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', replace_pattern)
# Convert JS-style backreferences in replace: $<name> -> \g<name>, $1 -> \1
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
if search_pattern and replace_pattern:
transformed_url = re.sub(search_pattern, safe_replace_pattern, original_url)
# regex module accepts JS-style (?<name>...) named groups natively
transformed_url = regex.sub(search_pattern, safe_replace_pattern, original_url)
return transformed_url
return original_url

View file

@ -4,7 +4,7 @@ from shlex import split as shlex_split
import sys
import subprocess
import logging
import re
import regex
import redis
from django.conf import settings
@ -132,10 +132,13 @@ def stream_view(request, channel_uuid):
# Prepare the pattern replacement.
logger.debug("Executing the following pattern replacement:")
logger.debug(f" search: {active_profile.search_pattern}")
safe_replace_pattern = re.sub(r'\$(\d+)', r'\\\1', active_profile.replace_pattern)
# Convert JS-style backreferences in replace: $<name> -> \g<name>, $1 -> \1
safe_replace_pattern = regex.sub(r'\$<([^>]+)>', r'\\g<\1>', active_profile.replace_pattern)
safe_replace_pattern = regex.sub(r'\$(\d+)', r'\\\1', safe_replace_pattern)
logger.debug(f" replace: {active_profile.replace_pattern}")
logger.debug(f" safe replace: {safe_replace_pattern}")
stream_url = re.sub(active_profile.search_pattern, safe_replace_pattern, input_url)
# regex module accepts JS-style (?<name>...) named groups natively
stream_url = regex.sub(active_profile.search_pattern, safe_replace_pattern, input_url)
logger.debug(f"Generated stream url: {stream_url}")
# Get the stream profile set on the channel.

View file

@ -1,6 +1,6 @@
import json
from channels.generic.websocket import AsyncWebsocketConsumer
import re, logging
import regex, logging
logger = logging.getLogger(__name__)
@ -54,9 +54,9 @@ class MyWebSocketConsumer(AsyncWebsocketConsumer):
# Apply the transformation using the replace_with_mark function
try:
search_preview = re.sub(data["search"], replace_with_mark, data["url"])
search_preview = regex.sub(data["search"], replace_with_mark, data["url"])
except Exception as e:
search_preview = data["search"]
search_preview = data["url"]
logger.error(f"Failed to generate replace preview: {e}")
result = transform_url(data["url"], data["search"], data["replace"])

View file

@ -290,7 +290,7 @@ const RegexFormAndView = ({ profile = null, m3u, isOpen, onClose }) => {
setXcMode(mode);
};
// Local regex testing for immediate visual feedback
// Local regex for the live demo preview
const getHighlightedSearchText = () => {
if (!searchPattern || !sampleInput) return sampleInput;
try {