Mirrors/pinchflat
1
0
Fork 0
mirror of https://github.com/kieraneglin/pinchflat.git synced 2026-01-23 02:24:24 +00:00
Code Issues Projects Releases Packages Wiki Activity

[Enhancement] Support specifying count of yt-dlp workers per-queue (#355)

Browse source 
* Updated certain Oban queues to have a configurable limit

* Updated README
This commit is contained in:
Kieran 2024-08-14 12:37:24 -07:00 • committed by GitHub
parent fd20ac5d84
commit 8e9f02c807
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 33 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

8
.iex.exs
View file

@ -23,3 +23,11 @@ alias Pinchflat.Metadata.MetadataFileHelpers
alias Pinchflat.SlowIndexing.FileFollowerServer
Pinchflat.Release.check_file_permissions()
defmodule IexHelpers do
def restart do
:init.restart()
end
end
import IexHelpers

21
README.md
View file

@ -145,16 +145,17 @@ If you change this setting and it works well for you, please leave a comment on
### Environment variables
| Name | Required? | Default | Notes |
| --------------------- | --------- | ------------------------- | ---------------------------------------------------------------------------------------------- |
| TZ | No | `UTC` | Must follow IANA TZ format |
| LOG_LEVEL | No | `debug` | Can be set to `info` |
| BASIC_AUTH_USERNAME | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| BASIC_AUTH_PASSWORD | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| EXPOSE_FEED_ENDPOINTS | No | | See [RSS feed docs](https://github.com/kieraneglin/pinchflat/wiki/Podcast-RSS-Feeds) |
| JOURNAL_MODE | No | `wal` | Set to `delete` if your config directory is stored on a network share (not recommended) |
| TZ_DATA_DIR | No | `/etc/elixir_tzdata_data` | The container path where the timezone database is stored |
| BASE_ROUTE_PATH | No | `/` | The base path for route generation. Useful when running behind certain reverse proxies |
| Name | Required? | Default | Notes |
| --------------------------- | --------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `TZ` | No | `UTC` | Must follow IANA TZ format |
| `LOG_LEVEL` | No | `debug` | Can be set to `info` but `debug` is strongly recommended |
| `BASIC_AUTH_USERNAME` | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| `BASIC_AUTH_PASSWORD` | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| `EXPOSE_FEED_ENDPOINTS` | No | | See [RSS feed docs](https://github.com/kieraneglin/pinchflat/wiki/Podcast-RSS-Feeds) |
| `JOURNAL_MODE` | No | `wal` | Set to `delete` if your config directory is stored on a network share (not recommended) |
| `TZ_DATA_DIR` | No | `/etc/elixir_tzdata_data` | The container path where the timezone database is stored |
| `BASE_ROUTE_PATH` | No | `/` | The base path for route generation. Useful when running behind certain reverse proxies |
| `YT_DLP_WORKER_CONCURRENCY` | No | `2` | The number of concurrent workers that use `yt-dlp` _per queue_. Set to 1 if you're getting IP limited, otherwise don't touch it |
## EFF donations

10
config/config.exs
View file

@ -57,16 +57,6 @@ config :pinchflat, Oban,
{"0 1 * * *", Pinchflat.Downloading.MediaRetentionWorker},
{"0 2 * * *", Pinchflat.Downloading.MediaQualityUpgradeWorker}
]}
],
# TODO: consider making this an env var or something?
queues: [
default: 10,
fast_indexing: 6,
media_indexing: 2,
media_collection_indexing: 2,
media_fetching: 2,
local_data: 8,
remote_metadata: 4
]
# Configures the mailer

14
config/runtime.exs
View file

@ -40,6 +40,20 @@ config :pinchflat, Pinchflat.Repo,
Path.join([:code.priv_dir(:pinchflat), "repo", "extensions", "sqlean-linux-#{system_arch}", "sqlean"])
]
# Some users may want to increase the number of workers that use yt-dlp to improve speeds
# Others may want to decrease the number of these workers to lessen the chance of an IP ban
{yt_dlp_worker_count, _} = Integer.parse(System.get_env("YT_DLP_WORKER_CONCURRENCY", "2"))
config :pinchflat, Oban,
queues: [
default: 10,
fast_indexing: 6,
media_collection_indexing: yt_dlp_worker_count,
media_fetching: yt_dlp_worker_count,
remote_metadata: yt_dlp_worker_count,
local_data: 8
]
if config_env() == :prod do
config_path = "/config"
db_path = System.get_env("DATABASE_PATH", Path.join([config_path, "db", "pinchflat.db"]))