e0ed7e469c
* Publish an epub of the docs on Read the Docs (#1835) Read the Docs' built-in formats support (the existing formats: all in .readthedocs.yaml) only produces epub/PDF for Sphinx projects, and is a silent no-op for MkDocs ones. Instead, per RTD's documented build-customization path, generate the epub ourselves in a post_build job and place it in $READTHEDOCS_OUTPUT/epub/, which RTD then publishes on the project Downloads page and in the docs flyout menu. The epub itself is built by the new docs/build-epub.sh: it takes the committed, generated Markdown pages in docs/src in mkdocs.yml nav order, strips the HTML-only quicklinks header from each page, and runs pandoc (installed on RTD via build.apt_packages). Locally, `make -C docs epub` does the same for anyone with pandoc installed; nothing here is part of `make dev` or any default build path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix misrender --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2.9 KiB
Miller is a command-line tool for querying, shaping, and reformatting data files in various formats, including CSV, TSV, JSON, JSON Lines, YAML, and DCF.
The big picture: Even well into the 21st century, our world is full of text-formatted data such as CSV. Google CSV memes, for example. We need tooling to thrive in this world, nimbly manipulating data which is in CSVs. And we need tooling to move beyond CSV, to be able to pull data out and into other storage and processing systems. Miller is designed for both of these goals.
In several senses, Miller is more than one tool:
Format conversion: You can convert CSV files to JSON, or vice versa, or pretty-print your data horizontally or vertically to make it easier to read.
Data manipulation: With a few keystrokes, you can remove columns you don't care about -- or make new ones.
Pre-processing/post-processing vs standalone use: You can use Miller to clean data files and put them into standard formats, perhaps in preparation for loading them into a database or a hands-off data-processing pipeline. Or you can use it post-process and summarize database-query output. As well, you can use Miller to explore and analyze your data interactively.
Compact verbs vs programming language: For low-keystroking, you can do things like
mlr --csv sort -f name input.csv
mlr --json head -n 1 myfile.json
The sort, head, etc., are called verbs. They're analogs of familiar command-line tools like sort, head, and so on -- but they're aware of name-indexed, multi-line file formats like CSV, TSV, JSON, YAML, and DCF. In addition, though, using Miller's put verb, you can use programming-language statements for expressions like
mlr --csv put '$rate = $units / $seconds' input.csv
which allow you to express your own logic succinctly.
Multiple domains: People use Miller for data analysis, data science, software engineering, devops/system-administration, journalism, scientific research, and more.
In the following, you can see how CSV, TSV, tabular, JSON, and other file formats share a common theme which is lists of key-value-pairs. Miller embraces this common theme.
