mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-19 09:24:14 +00:00
59 lines
2.8 KiB
HTML
59 lines
2.8 KiB
HTML
<center><table border=1 width="60%"><tr><td><center>
|
|
<b>
|
|
Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV.
|
|
</b>
|
|
</center> </td></tr></table></center>
|
|
|
|
<p/> With Miller you get to use named fields without needing to count
|
|
positional indices. For example:
|
|
|
|
POKI_INCLUDE_ESCAPED(index-snippet-2.txt)HERE
|
|
|
|
<p/>This is something the Unix toolkit always could have done, and arguably
|
|
always should have done. It operates on key-value-pair data while the familiar
|
|
Unix tools operate on integer-indexed fields: if the natural data structure for
|
|
the latter is the array, then Miller’s natural data structure is the
|
|
insertion-ordered hash map. This encompasses a <b>variety of data formats</b>,
|
|
including but not limited to the familiar CSV. (Miller can handle
|
|
positionally-indexed data as a special case.)
|
|
|
|
<p/> Features:
|
|
<ul>
|
|
|
|
<li> I/O formats including <b>tabular pretty-printing</b> and <b>positionally indexed</b> (Unix-toolkit style)
|
|
<li> <b>Conversion</b> between formats
|
|
<li> <b>Format-aware processing</b>: e.g. CSV sort and tac keep header lines first
|
|
<li> High-throughput <b>performance</b> on par with the Unix toolkit
|
|
|
|
<li/> Miller is <b>pipe-friendly</b> and interoperates with Unix toolkit.
|
|
|
|
<li/> Miller is <b>streaming</b>: most operations need only a single record in
|
|
memory at a time, rather than ingesting all input before producing any output.
|
|
For those operations which require deeper retention (<tt>sort</tt>,
|
|
<tt>tac</tt>, <tt>stats1</tt>), Miller retains only as much data as needed.
|
|
This means that whenever functionally possible you can operate on files which
|
|
are larger than your system’s available RAM, and you can use Miller in
|
|
<tt>tail -f</tt> contexts.
|
|
|
|
<li/> It complements SQL <b>databases</b>: you can slice, dice, and reformat
|
|
data on the client side on its way into or out of a database. You can also reap
|
|
some of the benefits of databases for quick, setup-free one-off tasks when just
|
|
need to query some data in disk files in a hurry.
|
|
|
|
<li/> Likewise, you can use Miller’s text-reformatting strengths to
|
|
(among other examples) snarf and munge log-file data into CSV format and then
|
|
load that into R for further statistical and/or graphical processing.
|
|
|
|
<li/>Miller also goes beyond classic Unix tools by stepping into our modern,
|
|
<b>no-SQL</b> world: its essential record-heterogeneity property allows it to
|
|
operate on data where records with different schema (field names) are
|
|
interleaved.
|
|
|
|
<li/> Not unlike <a href="http://stedolan.github.io/jq/">jq</a> (for JSON),
|
|
Miller is written in modern C, and it has <b>zero runtime dependencies</b>. You
|
|
can download or compile a single binary, <tt>scp</tt> it to a faraway machine,
|
|
and expect it to work.
|
|
</ul>
|
|
|
|
<p>Releases and release notes:
|
|
<a href="https://github.com/johnkerl/miller/releases">https://github.com/johnkerl/miller/releases</a>.
|