Application log timestamps are pinned to UTC no matter what the operator
configures, while nginx in the same container honors TZ - one container,
two timezones (issue 1439). Two mechanisms force this:
1. docker/entrypoint.sh launches the app tree via `su - dispatch`,
which strips the environment; only whitelisted vars are re-exported
through /etc/environment and /etc/profile.d, and TZ is not among
them, so no app process ever sees the operator's declared zone.
2. Whitelisting TZ alone cannot fix Django-managed processes anyway:
Django's settings loader unconditionally re-stamps os.environ["TZ"]
= TIME_ZONE ("UTC") and calls time.tzset() in every uwsgi, celery
and management-command process. Stdlib logging's %(asctime)s renders
via time.localtime(), so it can never show local time in a Django
process regardless of the environment.
The fix, keeping TIME_ZONE = "UTC" / USE_TZ untouched (DB truth and
every wire surface - XMLTV offsets, XC server_info/time_now/EPG
timestamps - stay UTC by design):
- entrypoint: normalize a DISPATCHARR_TIME_ZONE bootstrap var from the
standard TZ env and whitelist it. TZ itself is deliberately NOT
whitelisted: pam_env would hand it to every su- child including
initdb/postgres, silently flipping the database server timezone on
fresh installs - the precondition for the EPG-offset corruption
class tracked in issue 651, and (with an invalid zone) a source of
psycopg "unknown PostgreSQL timezone" warnings emitted while the
connection lock is held. Verified empirically: with TZ whitelisted,
a fresh install ran its sessions at the container zone; without it,
the server default stays UTC.
- settings: capture DISPATCHARR_DISPLAY_TZ at module import, before
Django's tzset re-stamp, as the pre-database display default.
- logging: the verbose formatter renders timestamps with
datetime/ZoneInfo from an in-process cached zone - never
time.localtime(), and never a database query at emit time. An
emit-time query can self-deadlock: a log record that originates
inside a psycopg call (e.g. its timezone warning) would re-enter the
same non-reentrant connection lock the ORM query then needs. The
cache is refreshed out-of-band from provably safe contexts instead:
request_started, celery task_prerun, and a CoreSettings post_save
receiver (immediate in the process that saves the UI setting). The
zone resolves to the UI's System > Time Zone setting - already
canonical for DVR rules, celery crontabs and backup naming - with
the env capture as the value before the first refresh, and any
database error or invalid stored zone keeps the previous value.
- first-boot seed: the fresh-install default that the settings
consolidation migration writes for the system time zone now derives
from the same env capture (validated against zoneinfo, falling back
to UTC) instead of a hardcoded "UTC", so a new install honors the
declared container timezone from first boot. Existing installs are
untouched: the migration only manufactures a default when no stored
value exists, and installs that already ran it never re-run it.
Known residuals: uWSGI's native request log renders its own C-level
ftime outside Python logging and keeps UTC, as do celery's internal
worker_log_format lines; idle workers adopt a changed UI zone on their
next request/task. All documented rather than patched.
Updated the session handling mechanism to improve user experience during reconnects. The first request without a `session_id` now receives a `301` redirect with a minted `session_id`, while reconnects that omit `session_id` but match an existing pool entry are served immediately without a redirect. Additionally, refined playback logic for plain GET requests to restart from byte 0, aligning with provider behavior. Updated tests to reflect these changes and ensure proper session reuse and playback anchoring.
- Added entry for extended frontend unit tests covering plugin, backup, and settings components.
- Updated Backup Manager to ensure the "Created" column refreshes when date/time format preferences change.
Updated the stream switching mechanism in the proxy server to ensure that the `stream_id` is persisted correctly and that the requesting worker waits for confirmation from the owning worker in multi-worker deployments. Improved error handling for stream switch failures, returning appropriate HTTP status codes (502/504) based on the confirmation status. Additionally, refined the metadata update process to handle cases where the requested URL is already in use, ensuring a successful response without unnecessary operations. This change addresses issues with stale metadata and enhances the robustness of the stream switching feature. (Fixes#1412)
Updated the session matching logic to improve handling of fresh sessions. Introduced a new parameter, `fresh_session`, to skip adopting idle exact-media pools when a new session ID is provided. This prevents unnecessary reconnections to previously abandoned programme slots. Added tests to verify that fresh sessions correctly skip idle matches while retaining busy exact-media sessions and allowing channel hops. This change enhances the robustness of the timeshift feature and improves user experience during session transitions.
Add end-to-end catch-up support for XC clients and native apps: provider
proxy with failover and per-viewer session pooling, REST session minting
for tokenless playback URLs, catch-up admin stats, combined connection
stats, and Stats UI with dedicated cards plus websocket updates.
Includes Redis namespace consolidation under timeshift:* (dropping legacy
timeshift_ id prefixes), dedicated catch-up stop by session_id, and
cleaner channel/client metadata split for stats keys.
Closes#133
Updated the EPG generation logic to ensure that channels without a channel number do not cause crashes in the XMLTV export. Implemented a deferred assignment mechanism for channels with null numbers, allowing them to receive valid integers during processing. Added tests to verify that the XMLTV output remains stable and valid even when channels are unnumbered.
Implemented logic to abort the VOD refresh process when the provider returns no categories, preserving existing group selections. This change prevents unintended deletions of category relations and ensures that accounts retain their current settings during transient provider outages or API issues. Updated logging to reflect the abort condition for better traceability.
Updated the handling of channels in the live streams setup to ensure that channels without a channel number do not cause crashes. Implemented logic to assign a valid number to these channels during processing. Added tests to verify that the system correctly assigns numbers to channels with null values, ensuring stability in the live streams feature.
Added a helper function to close old database connections in the `timeshift_proxy` view before returning any HTTP responses. This change prevents potential database connection leaks and ensures that connections are properly managed during the request lifecycle. Updated tests to verify that connections are closed appropriately in various response scenarios.
Also stores provider_tz_name with session pool in redis to avoid an extra database call.
Enhanced the `/output/epg` cold rebuild process to prevent freezing of the gevent uWSGI worker. The `_stream_build` function now yields control to the gevent hub after processing each cached chunk, allowing other requests to be handled concurrently. This change improves responsiveness during CPU-bound XMLTV generation. Additionally, introduced a new utility function, `_cooperative_yield`, to facilitate yielding in CPU-bound loops.
Self-review follow-ups on the position fix:
- When the position actually moves, drop content_length/serving_range from
the pool entry — they describe the PREVIOUS position's file, and keeping
them would feed the near-EOF/displacement heuristics another programme's
size (a metadata probe near the new file's EOF could displace live
playback, or a genuine scrub could be misjudged as a probe). The next
successful open repopulates both.
- Guard the update under the pool lock and skip it when the entry has
vanished (Redis restart/eviction) — a bare HSET would resurrect a
partial, TTL-less hash that answers every later request for that
session_id with 503.
- Align the reuse-path timezone lookup with the fresh path
(is_active=True on the default-profile filter).
Three more tests: byte state dropped on move / kept on same-position
update, vanished entry never resurrected, tz fallback to the reserved
profile when no active default exists.
Clients (TiviMate) keep the ?session_id= query when they rebuild the seek
URL with a new start timestamp, so every timestamp-jump FF/RW landed on
the reused session's STORED provider_timestamp — playback snapped back to
the position the session was created for (the rewind anchor), 100%
reproducible: two requests with different start values on one session
returned byte-identical streams.
The reuse path now always recomputes the provider timestamp from the
REQUESTED one (the provider zone is a property of the account — read from
the default profile's server_info, same as the fresh path) and moves the
pool descriptor (media_id + provider_timestamp) to the position actually
served, so fingerprint matching and same-channel displacement keep
comparing against reality. Slot continuity is unchanged.
Regression tests: reused session serves the requested timestamp (unit),
descriptor follows the seek (unit), and an end-to-end timestamp-jump with
the same session_id reaches the new position through timeshift_proxy.
Revised the import mechanism for the test discovery module to load it dynamically by file path, avoiding eager loading of the `dispatcharr` package. This change ensures that the script can run in a bare Python environment before the application virtual environment is set up, improving compatibility with CI processes.
Updated the import mechanism for celery_app to avoid issues with lazy-loading, ensuring the default Celery app is correctly configured in production environments. Removed the __getattr__ function for celery_app, simplifying the module structure.
Modified the file type validation in scan_and_process_files() to recognize .xz files alongside .xml, .gz, and .zip. Updated logging messages to reflect the inclusion of .xz in the skipped file notifications.
Extends _open_m3u_text_source() and fetch_m3u_lines() to treat an
uploaded .xz playlist the same way as the existing .m3u.gz path: it is
streamed lazily via lzma.open() rather than loaded fully into memory
(unlike the .zip path, which must read archive members). Uses stdlib
lzma, no new dependency.
Companion to the EPG xz support added for Dispatcharr/Dispatcharr#1414 -
the M3U upload path has the same gzip/zip dispatch structure and would
otherwise hit the same gap for an xz-compressed playlist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds .xz to the set of compressed formats detect_file_format(),
extract_compressed_file(), fetch_xmltv(), and EPGSource.get_cache_file()
recognize, mirroring the existing gzip handling. Uses stdlib lzma, no new
dependency. Detection works via LZMA magic bytes (fd 37 7a 58 5a 00),
the .xz extension (including compound extensions like .xml.xz), and the
application/x-xz mimetype fallback.
Widened the magic-byte read in get_cache_file() from 4 to 6 bytes since
the xz signature is 6 bytes (the previous 4-byte read was sufficient
only for the 2-byte gzip/zip signatures it originally supported). That
widening silently broke the bare-<tv> raw-XML detection: the
fixed-length slice comparison header[:5] == b'<tv>' can never match a
5-byte slice of a 6-byte header against a 4-byte literal. Replaced the
fixed-length slice comparisons with header.startswith(...), which is
correct regardless of how many bytes are read - this also fixes the
adjacent <?xml declaration check, which was silently dead before this
change (a 4-byte read could never equal the 5-byte b'<?xml' literal
either).
FixesDispatcharr/Dispatcharr#1414
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
