mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-18 00:45:47 +00:00
79 lines
3.3 KiB
HTML
79 lines
3.3 KiB
HTML
POKI_PUT_TOC_HERE
|
|
|
|
<h1>File-format awareness</h1>
|
|
|
|
Miller respects CSV headers. If you do <tt>mlr --csv cat *.csv</tt> then the header line is written once:
|
|
|
|
<table><tr>
|
|
<td>
|
|
POKI_RUN_COMMAND{{cat data/a.csv}}HERE
|
|
</td>
|
|
<td>
|
|
POKI_RUN_COMMAND{{cat data/b.csv}}HERE
|
|
</td>
|
|
<td>
|
|
POKI_RUN_COMMAND{{mlr --csv cat data/a.csv data/b.csv}}HERE
|
|
</td>
|
|
<td>
|
|
POKI_RUN_COMMAND{{mlr --csv sort -nr b data/a.csv data/b.csv}}HERE
|
|
</td>
|
|
</tr></table>
|
|
|
|
Likewise with <tt>mlr sort</tt>, <tt>mlr tac</tt>, and so on.
|
|
|
|
<h1>awk-like features: mlr filter and mlr put</h1>
|
|
|
|
<ul>
|
|
|
|
<li/> <tt>mlr filter</tt> includes/excludes records based on a filter
|
|
expression, e.g. <tt>mlr filter '$count > 10'</tt>.
|
|
|
|
<li/> <tt>mlr put</tt> adds a new field as a function of others, e.g. <tt>mlr
|
|
put '$xy = $x * $y'</tt> or <tt>mlr put '$counter = NR'</tt>.
|
|
|
|
<li/> The <tt>$name</tt> syntax is straight from <tt>awk</tt>’s <tt>$1 $2
|
|
$3</tt> (adapted to name-based indexing), as are the variables <tt>FS</tt>,
|
|
<tt>OFS</tt>, <tt>RS</tt>, <tt>ORS</tt>, <tt>NF</tt>, <tt>NR</tt>, and
|
|
<tt>FILENAME</tt>. The <tt>ENV[...]</tt> syntax is from Ruby.
|
|
|
|
<li/> While <tt>awk</tt> functions are record-based, Miller subcommands (or
|
|
<i>verbs</i>) are stream-based: each of them maps a stream of records into
|
|
another stream of records.
|
|
|
|
<li/> Like <tt>awk</tt>, Miller (as of v5.0.0) allows you to define new
|
|
functions within its <tt>put</tt> and <tt>filter</tt> expression language.
|
|
Further programmability comes from chaining with <tt>then</tt>.
|
|
|
|
<li/> As with <tt>awk</tt>, <tt>$</tt>-variables are stream variables and all
|
|
verbs (such as <tt>cut</tt>, <tt>stats1</tt>, <tt>put</tt>, etc.) as well as
|
|
<tt>put</tt>/<tt>filter</tt> statements operate on streams. This means that
|
|
you define actions to be done on each record and then stream your data through
|
|
those actions. The built-in variables <tt>NF</tt>, <tt>NR</tt>, etc. change
|
|
from one line to another, <tt>$x</tt> is a label for field <tt>x</tt> in the
|
|
current record, and the input to <tt>sqrt($x)</tt> changes from one record to
|
|
the next. The expression language for the <tt>put</tt> and <tt>filter</tt>
|
|
verbs additionally allows you to define <tt>begin {...}</tt> and
|
|
<tt>end {...}</tt> blocks for actions to be taken before and after records are
|
|
processed, respectively.
|
|
|
|
<li/> As with <tt>awk</tt>, Miller’s <tt>put</tt>/<tt>filter</tt>
|
|
language lets you set <tt>@sum=0</tt> before records are read, then update that
|
|
sum on each record, then print its value at the end. Unlike <tt>awk</tt>,
|
|
Miller makes syntactically explicit the difference between variables with
|
|
extent across all records (names starting with <tt>@</tt>, such as
|
|
<tt>@sum</tt>) and variables which are local to the current expression (names
|
|
starting without <tt>@</tt>, such as <tt>sum</tt>).
|
|
|
|
<li/> Miller can be faster than <tt>awk</tt>, <tt>cut</tt>, and so on,
|
|
depending on platform; see also POKI_PUT_LINK_FOR_PAGE(performance.html)HERE).
|
|
In particular, Miller’s DSL syntax is parsed into C control structures at
|
|
startup time, with the bulk data-stream processing all done in C.
|
|
|
|
</ul>
|
|
|
|
<h1>See also</h1>
|
|
|
|
<p/>See POKI_PUT_LINK_FOR_PAGE(reference.html)HERE for more on Miller’s
|
|
subcommands <tt>cat</tt>, <tt>cut</tt>, <tt>head</tt>, <tt>sort</tt>,
|
|
<tt>tac</tt>, <tt>tail</tt>, <tt>top</tt>, and <tt>uniq</tt>, as well as awk-like
|
|
<tt>mlr filter</tt> and <tt>mlr put</tt>.
|