mirror of
https://github.com/johnkerl/miller.git
synced 2026-08-02 04:22:59 +00:00
79 lines
3.5 KiB
HTML
79 lines
3.5 KiB
HTML
POKI_PUT_TOC_HERE
|
|
|
|
<h1>File-format awareness</h1>
|
|
|
|
Miller respects CSV headers. If you do <code>mlr --csv cat *.csv</code> then the header line is written once:
|
|
|
|
<table><tr>
|
|
<td>
|
|
POKI_RUN_COMMAND{{cat data/a.csv}}HERE
|
|
</td>
|
|
<td>
|
|
POKI_RUN_COMMAND{{cat data/b.csv}}HERE
|
|
</td>
|
|
<td>
|
|
POKI_RUN_COMMAND{{mlr --csv cat data/a.csv data/b.csv}}HERE
|
|
</td>
|
|
<td>
|
|
POKI_RUN_COMMAND{{mlr --csv sort -nr b data/a.csv data/b.csv}}HERE
|
|
</td>
|
|
</tr></table>
|
|
|
|
Likewise with <code>mlr sort</code>, <code>mlr tac</code>, and so on.
|
|
|
|
<h1>awk-like features: mlr filter and mlr put</h1>
|
|
|
|
<ul>
|
|
|
|
<li/> <code>mlr filter</code> includes/excludes records based on a filter
|
|
expression, e.g. <code>mlr filter '$count > 10'</code>.
|
|
|
|
<li/> <code>mlr put</code> adds a new field as a function of others, e.g. <code>mlr
|
|
put '$xy = $x * $y'</code> or <code>mlr put '$counter = NR'</code>.
|
|
|
|
<li/> The <code>$name</code> syntax is straight from <code>awk</code>’s <code>$1 $2
|
|
$3</code> (adapted to name-based indexing), as are the variables <code>FS</code>,
|
|
<code>OFS</code>, <code>RS</code>, <code>ORS</code>, <code>NF</code>, <code>NR</code>, and
|
|
<code>FILENAME</code>. The <code>ENV[...]</code> syntax is from Ruby.
|
|
|
|
<li/> While <code>awk</code> functions are record-based, Miller subcommands (or
|
|
<i>verbs</i>) are stream-based: each of them maps a stream of records into
|
|
another stream of records.
|
|
|
|
<li/> Like <code>awk</code>, Miller (as of v5.0.0) allows you to define new
|
|
functions within its <code>put</code> and <code>filter</code> expression language.
|
|
Further programmability comes from chaining with <code>then</code>.
|
|
|
|
<li/> As with <code>awk</code>, <code>$</code>-variables are stream variables and all
|
|
verbs (such as <code>cut</code>, <code>stats1</code>, <code>put</code>, etc.) as well as
|
|
<code>put</code>/<code>filter</code> statements operate on streams. This means that
|
|
you define actions to be done on each record and then stream your data through
|
|
those actions. The built-in variables <code>NF</code>, <code>NR</code>, etc. change
|
|
from one line to another, <code>$x</code> is a label for field <code>x</code> in the
|
|
current record, and the input to <code>sqrt($x)</code> changes from one record to
|
|
the next. The expression language for the <code>put</code> and <code>filter</code>
|
|
verbs additionally allows you to define <code>begin {...}</code> and
|
|
<code>end {...}</code> blocks for actions to be taken before and after records are
|
|
processed, respectively.
|
|
|
|
<li/> As with <code>awk</code>, Miller’s <code>put</code>/<code>filter</code>
|
|
language lets you set <code>@sum=0</code> before records are read, then update that
|
|
sum on each record, then print its value at the end. Unlike <code>awk</code>,
|
|
Miller makes syntactically explicit the difference between variables with
|
|
extent across all records (names starting with <code>@</code>, such as
|
|
<code>@sum</code>) and variables which are local to the current expression (names
|
|
starting without <code>@</code>, such as <code>sum</code>).
|
|
|
|
<li/> Miller can be faster than <code>awk</code>, <code>cut</code>, and so on,
|
|
depending on platform; see also POKI_PUT_LINK_FOR_PAGE(performance.html)HERE).
|
|
In particular, Miller’s DSL syntax is parsed into C control structures at
|
|
startup time, with the bulk data-stream processing all done in C.
|
|
|
|
</ul>
|
|
|
|
<h1>See also</h1>
|
|
|
|
<p/>See POKI_PUT_LINK_FOR_PAGE(reference.html)HERE for more on Miller’s
|
|
subcommands <code>cat</code>, <code>cut</code>, <code>head</code>, <code>sort</code>,
|
|
<code>tac</code>, <code>tail</code>, <code>top</code>, and <code>uniq</code>, as well as awk-like
|
|
<code>mlr filter</code> and <code>mlr put</code>.
|