Commit graph

51 commits

Author SHA1 Message Date
Kieran
6ead29182d
[Enhancement] Auto-update yt-dlp (#589)
* Added a command for updating yt-dlp

* Added a yt-dlp update worker to run daily

* Added a new file that runs post-boot when the app is ready to serve requests; put yt-dlp updater in there

* Updated config to expose the current env globally; updated startup tasks to not run in test env

* Removes unneeded test code
2025-01-27 11:33:38 -08:00
Kieran
62214b80a6
[Enhancement] Run fast indexing on source creation and at higher priority (#583)
* Updated default job priorities for downloading queue

* Added the ability to set priority to various downloading helpers

* Sets sources to fast index on creation
2025-01-22 14:54:15 -08:00
Kieran
fb27988963
[Enhancement] Add Prometheus support (#556)
* Added prometheus to deps list

* WIP - screwing around with Prometheus and grafana

* Added basic prometheus config

* Updated docs in prom_ex module

* Updated README
2025-01-09 12:38:17 -08:00
Kieran
83b1fa23d0
[Housekeeping] Update config options to enhance NixOS compatibility (#493)
* Make more paths configurable

* Updated tzdata default path to use the specified 'extras' path
2024-12-10 14:19:34 -08:00
Kieran Eglin
2f83fe03d2
Updated default tzdata location to prevent needless pulls every time I push a new version 2024-10-30 10:18:03 -07:00
Kieran Eglin
3c10d78e78
Updated timezone logic to prevent boot crash 2024-10-30 10:14:04 -07:00
Kieran
47bb1cb98d
Updated timezone logic to not prevent booting (#441) 2024-10-28 09:15:45 -07:00
Kieran Eglin
e3f42c28d0
Opps - reversed the IPv6 logic 2024-08-27 20:25:53 -07:00
Kieran
612c760755
[Enhancement] Allow IPv6 (#372)
* Added option to enable IPv6 in runtime config

* Updated README
2024-08-27 20:08:53 -07:00
Kieran
14b8ecbe44
[Enhancement] Show error messages in-app for easier triage (#365)
* Added an improved error message screen with actionable details

* Added a basic 404 page

* fixed tests
2024-08-23 13:33:27 -07:00
Kieran Eglin
80209240d2
Updated logger to add dates in logfiles rather than just console output 2024-08-20 08:58:30 -07:00
Kieran
8e9f02c807
[Enhancement] Support specifying count of yt-dlp workers per-queue (#355)
* Updated certain Oban queues to have a configurable limit

* Updated README
2024-08-14 12:37:24 -07:00
Kieran
d5ae41cdab
[Performance] Asyncronously delete sources and media profiles (#277)
* [WIP] started on source deletion

* Removed unneeded test blocks

* Added marked_for_deletion_at to sources and media profiles

* Hooked up async deletion to media profiles as well
2024-07-22 12:26:22 -07:00
Kieran
7f1daf90ca
Add date to 'prod' logger (#305) 2024-07-11 10:19:14 -07:00
Kieran
b73a9a4a67
[Enhancement] Added support for subdirectory-style reverse proxies (#276)
* Updated routes and other fun stuff to support reverse proxies

* Updated README

* Linting
2024-05-30 12:58:51 -07:00
Kieran
bdcb49185a
[Enhancement] Adds in-app diagnostics page (#243)
* Added improved sidebar menuing

* Added new view for getting diagnostic data

* Changed default log level to debug

* Disabled false-positive static analysis
2024-05-15 12:27:57 -07:00
Kieran
a38ffbc55b
[Enhancement] Allow redownloading of files for existing media items (#239)
* Added ability to specify overwrite behaviour when downloading media

* Added helper for redownloading media items

* renamed media redownload worker to disambiguate it from similarly named methods

* Added new redownload option to source actions dropdown

* Refactored MediaQuery to use a __using__ macro

* docs
2024-05-13 14:25:39 -07:00
Kieran
77a9ff4d9f
Made tz data location user-configurable (#238) 2024-05-13 09:27:16 -07:00
Kieran
b5d2ba4191
[Housekeeping] Reduce Docker image size (#231)
* First pass at cutting down image size

* Made selfhosted docker work multiplatform

* Updated compile-time value to be runtime

* Fixed issue with tzdata file permissions

* Applied better fix for tz issue
2024-05-08 09:01:21 -07:00
Kieran
a1a568b7f7
[Enhancement] Add history table to homepage (And TZ support) (#229)
* Added basic history table to homepage

* Improved homepage history query

* Set table to use 24h time

* Added timezone support

* Updated README to reflect new TZ stuff
2024-05-07 12:16:57 -07:00
Kieran
112c6a4f14
[Enhancement] Custom media lifecycle scripts (#219)
* Namespaced notification modules under lifecycle

* Added a JSON encoder for all the main model types

* Added startup task to create user script file

* Hook up user script event to media download

* Hooked up media deletion user script

* Added jq to docker deps

* Updated README
2024-05-02 08:43:37 -07:00
Kieran
e841f39cf2 [Enhancement] Redownload new media after a delay (#173)
* Added redownload-related columns

* Added methods for fetching re-downloadable media items

* Filled out redownload worker + tests

* Added redownload worker to config.exs cron

* Added to UI and README
2024-04-10 17:54:45 -07:00
Kieran Eglin
a2bcd454c7 Added apprise to runtime 2024-04-09 19:07:49 -07:00
Kieran
26d457e656 [Enhancement] Add Apprise support (#170)
* [WIP] add settings sidebar entry and placeholder page

* [WIP] added placeholder UI and logic for settings form

* Added column and UI for apprise server

* Add some tests

* Added placeholder command runner for apprise

* [WIP] Adding apprise package

* Added apprise command runner

* Hooked up apprise notification module

* Ensured apprise was running in verbose mode

* Updated wording of apprise notification

* Added apprise to README
2024-04-09 17:45:39 -07:00
Kieran
4b12764f45
[Housekeeping] Upgraded Tailwind to 3.4.3 (#163)
* Upgraded Tailwind

* Enabled asset compression in prod
2024-04-03 14:57:45 -07:00
Kieran
79c61bca4f
[Enhancement] Delete media after "x" days (#160)
* [Enhancement] Adds ability to stop media from re-downloading (#159)

* Added column

* Added methods for ignoring media items from future download

* Added new deletion options to controller and UI

* Added controller actions and UI for editing a media item

* Added column to sources

* Added retention period to form

* [WIP] getting retention methods in place

* Hooked up retention worker

* Added column and UI to prevent automatic deletion

* Docs

* Removed unused backfill worker

* Added edit links to media item tabs on source view

* Clarified form wording

* Form wording (again)
2024-04-03 10:44:11 -07:00
Kieran
89f4634d1f
[Enhancement] Misc fixes 2024-03-29 (#150)
* Removes header to allow embedding in frame

* Escaped links in podcast RSS builder

* Turned homepage sections into clickable links

* Enabled live dashboard in all ENVs

* Ensured download worker always returns a 2-member tuple
2024-03-29 18:01:09 -07:00
Kieran
edb12b088f [Triage] Allow changing SQLite journal mode (#145)
* Adds ability to configure journal mode for SQLite

* docs

* docs
2024-03-28 20:10:27 -07:00
Kieran
320b25f5b6 RSS feed for sources (#110)
* Add media streaming (#108)

* [WIP] set up streaming route

* Added UUID to sources and media items

* Added media preview to MI show page

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* Comment

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* Generate rss feeds (#123)

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* docs

* Added unique index to UUID fields
2024-03-27 18:11:25 -07:00
Kieran
af86fede6c [Bugfix] Expose port in Dockerfile and force IPv4 (#129)
* Exposes port on built image

* Forced IPv4

* Added current version to sidebar
2024-03-26 17:54:20 -07:00
Kieran
fe641cfc49 Cookie auth (#116)
* Added new extras directory

* Automatically create cookie file on boot

* Added cookie file option to DL option builder

* Put cookie auth in correct spot

* Updated Docs

* Removed useless setter for config path
2024-03-24 20:23:25 -07:00
Kieran
3c522d8d6b Source title regex filtering (#98)
* Added a basic/advanced mode to source form

* Added regex field and UI to source form

* Implemented title regex filtering
2024-03-20 17:51:14 -07:00
Kieran
a135746c97 Improve episode-level compatability with media center apps (#86)
* Add media profile presets (#85)

* Added presets for output templates

* Added presets for the entire media profile form

* Append `-thumb` to thumbnails when downloading (#87)

* Appended -thumb to thumbnails when downloading

* Added code to compensate for yt-dlp bug

* Squash all the commits from the other branch bc I broke things (#88)
2024-03-14 18:30:08 -07:00
Kieran
33baa99aae Refactor modules into contexts (#78)
* [WIP] break out a few contexts, start refactoring fast index modules

* [WIP] more contexts, this time around slow indexing and downloads

* [WIP] got all tests passing

* [WIP] Added moduledocs

* Built a genserver to rename old jobs on boot

* Added a module naming check; moved things around

* Fixed specs
2024-03-12 17:54:55 -07:00
Kieran
c0a11e98d9 Updated check origin in release environment (#67) 2024-03-10 15:31:40 -07:00
Kieran
e1f8e94686 Fast indexing (#58)
* Made method to getting singular media details; Renamed other related method

* Takes a fun and flirty digression to remove abstractions around yt-dlp since I'm 100% committed to using it exclusively

* Removed commented test code

* Lays the groundwork for fast indexing

* Added module for working with youtube RSS feed

* Added methods to kick off indexing workers from RSS response

* Improve short detection (#59)

* Made media attribute-related yt-dlp calls return a struct

* Added shorts attribute to media items

* Added ability to discern a short from yt-dlp response

* Updated search to use new shorts attribute

* Fast index UI (#63)

* Added fast_index field and adds it to source form

* Added fast indexing to source changeset operations

* Added fast indexing worker and updated other modules to start using it

* Handled fast index worker on source update

* Add support modals (#65)

* Added fast indexing upgrade modal

* Improved modal on smaller screens

* Updated links to work again

* Added donation modal

* Reverted source fast index to 15 minutes

* Removed unneeded HTML attributes from old alpine approach
2024-03-10 14:36:34 -07:00
Kieran
b7beeaf9eb [Bugfix] Runtime log level fix (#56)
* Added log level to runtime config

* Started comparing against an atom instead of a string
2024-03-07 18:26:20 -08:00
Kieran
36daeff48c Added log level to runtime config (#55) 2024-03-07 18:07:00 -08:00
Kieran
82eb12b983 [Bugfix] Fix basic auth in deployed apps (#52)
* Renamed auth env vars and updated them to work in prod env

* Updated basic auth parsing to ignore empty strings
2024-03-04 18:29:26 -08:00
Kieran
90a0022d85 Rename video-related modules to be media-related (#50)
* Added test from before that I forgor

* Renamed video-related modules to media
2024-03-04 17:46:33 -08:00
Kieran
f55cdc80dd Streaming media item creation during indexing (#49)
* Implemented streaming during indexing

* Updated file watcher to enqueue download; refactored download worker methods

* Updated File Follower Server timeout
2024-03-04 17:14:02 -08:00
Kieran
b370c97bfb Added way to track media filesize (#48) 2024-03-02 19:19:10 -08:00
Kieran
c77047951f Improve metadata storage (#37)
* Updated config path to specify metadata storage path

* Removed metadata column from DB, replacing it with filepath columns

* Updated app to store compressed metadata; automatically download thumbnails

* Ensured metadata is deleted when other files are deleted

* Updated docs

* Added inets to application start so http calls work in prod
2024-02-24 13:40:05 -08:00
Kieran
610acb15f1 Added log rotation for prod config (#32) 2024-02-21 18:01:18 -08:00
Kieran
3efc533fb7 Added optional basic auth (#31) 2024-02-21 17:37:25 -08:00
Kieran
1f56db01a5 Prod packaging v1 (#23)
* Sets media directory for prod

* Improved logging; Added docker files for prod

* Improved fetching SECRET_KEY_BASE for selfhosted instances
2024-02-16 18:50:00 -08:00
Kieran
b81c8d64b3 Switch to sqlite (#22)
* Updated project to use sqlite

* Edited migrations directly because I fear no man

* Updated search functions and tests to work with sqlite

* Updated build tools to remove postgres
2024-02-16 17:11:41 -08:00
Kieran
c5fcb8fe12 Delete media items (#20)
* Added method for deleting media files and their content

* Adds controllers and methods for deleting media and files

* Improved tmpfile setup and teardown for tests

* Actually got tmpfile cleanup running once per suite run

* Finally fixed flash messages
2024-02-15 17:51:19 -08:00
Kieran
4dd9d837a3 Download video(s) (first iteration) (#5)
* Updated package.json (and made an excuse to make a branch)

* Video filepath parser (#6)

* Restructured files; Added parser placeholder

* More restructuring

* Added basic parser for hydrating template strings

* Improved docs

* More docs

* Initial implementation of media profiles (#7)

* [WIP] Added basic video download method

* [WIP] Very-WIP first steps at parsing options and downloading

* Made my options safe by default and removed special safe versions

* Ran html generator for mediaprofile model - leaving as-is for now

* Addressed a bunch of TODO comments

* Add "channel" type Media Source (#8)

* [WIP] Working on fetching channel metadata in yt-dlp backend

* Finished first draft of methods to do with querying channels

* Renamed CommandRunnerMock to have a more descriptive name

* Ran the phx generator for the channel model

* Renamed Downloader namespace to MediaClient

* [WIP] saving before attempting LiveView

* LiveView did not work out but here's a working controller how about

* Index a channel (#9)

* Ran a MediaItem generator; Reformatted to my liking

* [WIP] added basic index function

* setup oban

* Added basic Oban job for indexing

* Added in workers for indexing; hooked them into record creation flow

* Added a task model with a phx generator

* Tied together tasks with jobs and channels

* Download indexed videos (#10)

* Clarified documentation

* more comments

* [WIP] hooked up basic video downloading; starting work on metadata

* Added metadata model and parsing

Adding the metadata model made me realize that, in many cases, yt-dlp
returns undesired input in stdout, breaking parsing. In order to get
the metadata model working, I had to change the way in which the app
interacts with yt-dlp. Now, output is written as a file to disk which
is immediately re-read and returned.

* Added tests for video download worker

* Hooked up video downloading to the channel indexing pipeline

* Adds tasks for media items

* Updated video metadata parser to extract the title

* Ran linting
2024-01-30 20:15:59 -08:00
Kieran
89b1640eb6 Initial yt-dlp implementation (#3)
* Added basic runner for yt-dlp commands

* Updated formatting rules

* Added JSON runner; Added test script for CLI interactions

* Reworked yt-dlp runner; added proper test mocks
2024-01-19 22:26:32 -08:00