CSS neatens

This commit is contained in:
John Kerl 2020-02-02 09:45:19 -05:00
parent 973104ac76
commit 21067acde0
13 changed files with 140 additions and 140 deletions

View file

@ -81,7 +81,7 @@ pageTracker._trackPageview();
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="bodyToggler.toggle('body_section_toggle_csv_file_examples');" href="javascript:;">Toggle section visibility</button>
<div id="body_section_toggle_csv_file_examples" style="display: block">
<p/><boldmaroon> Sample CSV data file: </boldmaroon>
<p class="boldmaroon"> Sample CSV data file: </p>
<p/>
<div class="pokipanel">
@ -102,7 +102,7 @@ purple,square,0,91,72.3735,8.2430
</div>
<p/>
<p/><boldmaroon> <code>mlr cat</code> is like cat ...</boldmaroon>
<p class="boldmaroon"><code>mlr cat</code> is like cat ...</p>
<p/>
<div class="pokipanel">
@ -123,7 +123,7 @@ purple,square,0,91,72.3735,8.2430
</div>
<p/>
<p/><boldmaroon>... but it can also do format conversion (here, to pretty-printed tabular format): </boldmaroon>
<p class="boldmaroon">... but it can also do format conversion (here, to pretty-printed tabular format): </p>
<p/>
<div class="pokipanel">
@ -144,8 +144,8 @@ purple square 0 91 72.3735 8.2430
</div>
<p/>
<p/><boldmaroon> <code>mlr head</code> and <code>mlr tail</code> count records rather than lines. The CSV
header is included either way:</boldmaroon>
<p class="boldmaroon"> <code>mlr head</code> and <code>mlr tail</code> count records rather than lines. The CSV
header is included either way:</p>
<p/>
<div class="pokipanel">
@ -172,8 +172,8 @@ purple,square,0,91,72.3735,8.2430
</div>
<p/>
<p/><boldmaroon> Sort primarily alphabetically on one field, then secondarily
numerically descending on another field: </boldmaroon>
<p class="boldmaroon"> Sort primarily alphabetically on one field, then secondarily
numerically descending on another field: </p>
<p/>
<div class="pokipanel">
@ -194,7 +194,7 @@ yellow triangle 1 11 43.6498 9.8870
</div>
<p/>
<p/><boldmaroon> Use <code>cut</code> to retain only specified fields, in input-data order:</boldmaroon>
<p class="boldmaroon"> Use <code>cut</code> to retain only specified fields, in input-data order:</p>
<p/>
<div class="pokipanel">
@ -215,7 +215,7 @@ square 0
</div>
<p/>
<p/><boldmaroon> Use <code>cut -o</code> to retain only specified fields, in your specified order:</boldmaroon>
<p class="boldmaroon"> Use <code>cut -o</code> to retain only specified fields, in your specified order:</p>
<p/>
<div class="pokipanel">
@ -236,7 +236,7 @@ flag shape
</div>
<p/>
<p/><boldmaroon> Use <code>cut -x</code> to omit specified fields:</boldmaroon>
<p class="boldmaroon"> Use <code>cut -x</code> to omit specified fields:</p>
<p/>
<div class="pokipanel">
@ -257,7 +257,7 @@ purple 91 72.3735 8.2430
</div>
<p/>
<p/><boldmaroon> Use <code>filter</code> to retain specified records:</boldmaroon>
<p class="boldmaroon"> Use <code>filter</code> to retain specified records:</p>
<p/>
<div class="pokipanel">
@ -282,7 +282,7 @@ red circle 1 16 13.8103 2.9010
</div>
<p/>
<p/><boldmaroon> Use <code>put</code> to add/replace fields which are computed from other fields:</boldmaroon>
<p class="boldmaroon"> Use <code>put</code> to add/replace fields which are computed from other fields:</p>
<p/>
<div class="pokipanel">
@ -303,10 +303,10 @@ purple square 0 91 72.3735 8.2430 8.779995 purple_square
</div>
<p/>
<p/><boldmaroon> Even though Miller&rsquo;s main selling point is
<p class="boldmaroon"> Even though Miller&rsquo;s main selling point is
name-indexing, sometimes you really want to refer to a field name by its
positional index. Use <code>$[[3]]</code> to access the name of field 3 or
<code>$[[[3]]]</code> to access the value of field 3:</boldmaroon>
<code>$[[[3]]]</code> to access the value of field 3:</p>
<p/>
<div class="pokipanel">
@ -345,7 +345,7 @@ purple square NEW 91 72.3735 8.2430
</div>
<p/>
<p/><boldmaroon> JSON output:</boldmaroon>
<p class="boldmaroon"> JSON output:</p>
<p/>
<div class="pokipanel">
@ -365,7 +365,7 @@ $ mlr --icsv --ojson put '$ratio = $quantity/$rate; $shape = toupper($shape)' ex
</div>
<p/>
<p/><boldmaroon> JSON output with vertical-formatting flags:</boldmaroon>
<p class="boldmaroon"> JSON output with vertical-formatting flags:</p>
<p/>
<div class="pokipanel">
@ -393,11 +393,11 @@ $ mlr --icsv --ojson --jvstack --jlistwrap tail -n 2 example.csv
</div>
<p/>
<p/><boldmaroon> Use <code>then</code> to pipe commands together. Also, the
<p class="boldmaroon"> Use <code>then</code> to pipe commands together. Also, the
<code>-g</code> option for many Miller commands is for group-by: here, <code>head -n
1 -g shape</code> outputs the first record for each distinct value of the
<code>shape</code> field. This means we&rsquo;re finding the record with highest
<code>index</code> field for each distinct <code>shape</code> field:</boldmaroon>
<code>index</code> field for each distinct <code>shape</code> field:</p>
<p/>
<div class="pokipanel">
@ -411,9 +411,9 @@ purple triangle 0 65 80.1405 5.8240
</div>
<p/>
<p/><boldmaroon> Statistics can be computed with or without group-by field(s). Also, the first of these two
<p class="boldmaroon"> Statistics can be computed with or without group-by field(s). Also, the first of these two
examples uses <code>--oxtab</code> output format which is a nice alternative to <code>--opprint</code> when you
have lots of columns:</boldmaroon>
have lots of columns:</p>
<p/>
<div class="pokipanel">
@ -461,11 +461,11 @@ square purple 1 72.373500 72.373500 72.373500
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="bodyToggler.togglebodyToggler.toggle('body_section_toggle_printing_to_files');" href="javascript:;">Toggle section visibility</button>
<div id="body_section_toggle_printing_to_files" style="display: block">
<p/>Often we want to print output <boldmaroon>to the screen</boldmaroon>. Miller does this by default, as we&rsquo;ve
seen in the previous examples.
<p>Often we want to print output <span class="boldmaroon">to the screen</span>. Miller does this by default, as we&rsquo;ve
seen in the previous examples.</p>
<p/> Sometimes we want to print output to another file: <boldmaroon>just use '>
outputfilenamegoeshere'</boldmaroon> at the end of your command:
<p/> Sometimes we want to print output to another file: <span class="boldmaroon">just use '>
outputfilenamegoeshere'</span> at the end of your command:</p>
<table><tr><td>
<div class="pokipanel"> <pre>
@ -490,7 +490,7 @@ purple square 0 91 72.3735 8.2430
</pre> </div>
</td></tr></table>
<p/> Other times we just want our files to be changed in-place: <boldmaroon>just use 'mlr -I'</boldmaroon>.
<p/> Other times we just want our files to be changed in-place: <span class="boldmaroon">just use 'mlr -I'</span>.</p>
<table><tr><td>
<div class="pokipanel"> <pre>
@ -538,8 +538,8 @@ mlr -I --csv cut -x -f unwanted_column_name *.csv
<p/> If you like, you can first copy off your original data somewhere else, before doing in-place operations.
<p/><boldmaroon> Lastly, using <code>tee</code> within <code>put</code>, you can split your input data into separate files
per one or more field names:</boldmaroon>
<p class="boldmaroon"> Lastly, using <code>tee</code> within <code>put</code>, you can split your input data into separate files
per one or more field names:</p>
<p/>
<div class="pokipanel">
@ -616,7 +616,7 @@ shape=square,flag=0,index=36
</pre>
</div>
<p/>Data written this way are called <boldmaroon>DKVP</boldmaroon>, for <i>delimited key-value pairs</i>.
<p>Data written this way are called <span class="boldmaroon">DKVP</span>, for <i>delimited key-value pairs</i>.</p>
<p/>We&rsquo;ve also already seen other ways to write the same data:
@ -640,8 +640,8 @@ shape=square,flag=0,index=36 flag 1 "flag": 0,
</pre>
</div>
<p/><boldmaroon>Anything we can do with CSV input data, we can do with any
other format input data.</boldmaroon> And you can read from one format, do any
<p class="boldmaroon">Anything we can do with CSV input data, we can do with any
other format input data.</p> And you can read from one format, do any
record-processing, and output to the same format as the input, or to a
different output format.

View file

@ -353,7 +353,7 @@ other 21st-century compilers)
<a id="Optional_external_dependencies"/><h2>Optional external dependencies</h2>
This documentation pageset is built using <boldmaroon>Poki</boldmaroon>:
This documentation pageset is built using <span class="boldmaroon">Poki</span>:
<a href="http://johnkerl.org/poki/doc">docs here</a>,
<a href="https://github.com/johnkerl/poki">source code here</a>.
Note that <a href="http://johnkerl.org/miller/doc/index.html">http://johnkerl.org/miller/doc/index.html</a>

View file

@ -8,77 +8,77 @@ POKI_PUT_TOC_HERE
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="bodyToggler.toggle('body_section_toggle_csv_file_examples');" href="javascript:;">Toggle section visibility</button>
<div id="body_section_toggle_csv_file_examples" style="display: block">
<p/><boldmaroon> Sample CSV data file: </boldmaroon>
<p class="boldmaroon"> Sample CSV data file: </p>
POKI_RUN_COMMAND{{cat example.csv}}HERE
<p/><boldmaroon> <code>mlr cat</code> is like cat ...</boldmaroon>
<p class="boldmaroon"><code>mlr cat</code> is like cat ...</p>
POKI_RUN_COMMAND{{mlr --csv cat example.csv}}HERE
<p/><boldmaroon>... but it can also do format conversion (here, to pretty-printed tabular format): </boldmaroon>
<p class="boldmaroon">... but it can also do format conversion (here, to pretty-printed tabular format): </p>
POKI_RUN_COMMAND{{mlr --icsv --opprint cat example.csv}}HERE
<p/><boldmaroon> <code>mlr head</code> and <code>mlr tail</code> count records rather than lines. The CSV
header is included either way:</boldmaroon>
<p class="boldmaroon"> <code>mlr head</code> and <code>mlr tail</code> count records rather than lines. The CSV
header is included either way:</p>
POKI_RUN_COMMAND{{mlr --csv head -n 4 example.csv}}HERE
POKI_RUN_COMMAND{{mlr --csv tail -n 4 example.csv}}HERE
<p/><boldmaroon> Sort primarily alphabetically on one field, then secondarily
numerically descending on another field: </boldmaroon>
<p class="boldmaroon"> Sort primarily alphabetically on one field, then secondarily
numerically descending on another field: </p>
POKI_RUN_COMMAND{{mlr --icsv --opprint sort -f shape -nr index example.csv}}HERE
<p/><boldmaroon> Use <code>cut</code> to retain only specified fields, in input-data order:</boldmaroon>
<p class="boldmaroon"> Use <code>cut</code> to retain only specified fields, in input-data order:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint cut -f flag,shape example.csv}}HERE
<p/><boldmaroon> Use <code>cut -o</code> to retain only specified fields, in your specified order:</boldmaroon>
<p class="boldmaroon"> Use <code>cut -o</code> to retain only specified fields, in your specified order:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint cut -o -f flag,shape example.csv}}HERE
<p/><boldmaroon> Use <code>cut -x</code> to omit specified fields:</boldmaroon>
<p class="boldmaroon"> Use <code>cut -x</code> to omit specified fields:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint cut -x -f flag,shape example.csv}}HERE
<p/><boldmaroon> Use <code>filter</code> to retain specified records:</boldmaroon>
<p class="boldmaroon"> Use <code>filter</code> to retain specified records:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint filter '$color == "red"' example.csv}}HERE
POKI_RUN_COMMAND{{mlr --icsv --opprint filter '$color == "red" && $flag == 1' example.csv}}HERE
<p/><boldmaroon> Use <code>put</code> to add/replace fields which are computed from other fields:</boldmaroon>
<p class="boldmaroon"> Use <code>put</code> to add/replace fields which are computed from other fields:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint put '$ratio = $quantity / $rate; $color_shape = $color . "_" . $shape' example.csv}}HERE
<p/><boldmaroon> Even though Miller&rsquo;s main selling point is
<p class="boldmaroon"> Even though Miller&rsquo;s main selling point is
name-indexing, sometimes you really want to refer to a field name by its
positional index. Use <code>$[[3]]</code> to access the name of field 3 or
<code>$[[[3]]]</code> to access the value of field 3:</boldmaroon>
<code>$[[[3]]]</code> to access the value of field 3:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint put '$[[3]] = "NEW"' example.csv}}HERE
POKI_RUN_COMMAND{{mlr --icsv --opprint put '$[[[3]]] = "NEW"' example.csv}}HERE
<p/><boldmaroon> JSON output:</boldmaroon>
<p class="boldmaroon"> JSON output:</p>
POKI_RUN_COMMAND{{mlr --icsv --ojson put '$ratio = $quantity/$rate; $shape = toupper($shape)' example.csv}}HERE
<p/><boldmaroon> JSON output with vertical-formatting flags:</boldmaroon>
<p class="boldmaroon"> JSON output with vertical-formatting flags:</p>
POKI_RUN_COMMAND{{mlr --icsv --ojson --jvstack --jlistwrap tail -n 2 example.csv}}HERE
<p/><boldmaroon> Use <code>then</code> to pipe commands together. Also, the
<p class="boldmaroon"> Use <code>then</code> to pipe commands together. Also, the
<code>-g</code> option for many Miller commands is for group-by: here, <code>head -n
1 -g shape</code> outputs the first record for each distinct value of the
<code>shape</code> field. This means we&rsquo;re finding the record with highest
<code>index</code> field for each distinct <code>shape</code> field:</boldmaroon>
<code>index</code> field for each distinct <code>shape</code> field:</p>
POKI_RUN_COMMAND{{mlr --icsv --opprint sort -f shape -nr index then head -n 1 -g shape example.csv}}HERE
<p/><boldmaroon> Statistics can be computed with or without group-by field(s). Also, the first of these two
<p class="boldmaroon"> Statistics can be computed with or without group-by field(s). Also, the first of these two
examples uses <code>--oxtab</code> output format which is a nice alternative to <code>--opprint</code> when you
have lots of columns:</boldmaroon>
have lots of columns:</p>
POKI_RUN_COMMAND{{mlr --icsv --oxtab --from example.csv stats1 -a p0,p10,p25,p50,p75,p90,p99,p100 -f rate}}HERE
POKI_RUN_COMMAND{{mlr --icsv --opprint --from example.csv stats1 -a count,min,mean,max -f quantity -g shape}}HERE
@ -89,11 +89,11 @@ POKI_RUN_COMMAND{{mlr --icsv --opprint --from example.csv stats1 -a count,min,me
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="bodyToggler.togglebodyToggler.toggle('body_section_toggle_printing_to_files');" href="javascript:;">Toggle section visibility</button>
<div id="body_section_toggle_printing_to_files" style="display: block">
<p/>Often we want to print output <boldmaroon>to the screen</boldmaroon>. Miller does this by default, as we&rsquo;ve
seen in the previous examples.
<p>Often we want to print output <span class="boldmaroon">to the screen</span>. Miller does this by default, as we&rsquo;ve
seen in the previous examples.</p>
<p/> Sometimes we want to print output to another file: <boldmaroon>just use '>
outputfilenamegoeshere'</boldmaroon> at the end of your command:
<p/> Sometimes we want to print output to another file: <span class="boldmaroon">just use '>
outputfilenamegoeshere'</span> at the end of your command:</p>
<table><tr><td>
<div class="pokipanel"> <pre>
@ -118,7 +118,7 @@ purple square 0 91 72.3735 8.2430
</pre> </div>
</td></tr></table>
<p/> Other times we just want our files to be changed in-place: <boldmaroon>just use 'mlr -I'</boldmaroon>.
<p/> Other times we just want our files to be changed in-place: <span class="boldmaroon">just use 'mlr -I'</span>.</p>
<table><tr><td>
<div class="pokipanel"> <pre>
@ -166,8 +166,8 @@ mlr -I --csv cut -x -f unwanted_column_name *.csv
<p/> If you like, you can first copy off your original data somewhere else, before doing in-place operations.
<p/><boldmaroon> Lastly, using <code>tee</code> within <code>put</code>, you can split your input data into separate files
per one or more field names:</boldmaroon>
<p class="boldmaroon"> Lastly, using <code>tee</code> within <code>put</code>, you can split your input data into separate files
per one or more field names:</p>
POKI_RUN_COMMAND{{mlr --csv --from example.csv put -q 'tee > $shape.".csv", $*'}}HERE
<table><tr><td>
@ -207,7 +207,7 @@ shape=square,flag=0,index=36
</pre>
</div>
<p/>Data written this way are called <boldmaroon>DKVP</boldmaroon>, for <i>delimited key-value pairs</i>.
<p>Data written this way are called <span class="boldmaroon">DKVP</span>, for <i>delimited key-value pairs</i>.</p>
<p/>We&rsquo;ve also already seen other ways to write the same data:
@ -231,8 +231,8 @@ shape=square,flag=0,index=36 flag 1 "flag": 0,
</pre>
</div>
<p/><boldmaroon>Anything we can do with CSV input data, we can do with any
other format input data.</boldmaroon> And you can read from one format, do any
<p class="boldmaroon">Anything we can do with CSV input data, we can do with any
other format input data.</p> And you can read from one format, do any
record-processing, and output to the same format as the input, or to a
different output format.

View file

@ -241,7 +241,7 @@ other 21st-century compilers)
<h2>Optional external dependencies</h2>
This documentation pageset is built using <boldmaroon>Poki</boldmaroon>:
This documentation pageset is built using <span class="boldmaroon">Poki</span>:
<a href="http://johnkerl.org/poki/doc">docs here</a>,
<a href="https://github.com/johnkerl/poki">source code here</a>.
Note that <a href="http://johnkerl.org/miller/doc/index.html">http://johnkerl.org/miller/doc/index.html</a>

View file

@ -373,14 +373,14 @@ POKI_RUN_COMMAND{{mlr --csv join --ul -j id -f data/color-codes.csv then unspars
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="bodyToggler.toggle('body_section_toggle_xml_or_json');" href="javascript:;">Toggle section visibility</button>
<div id="body_section_toggle_xml_or_json" style="display: block">
<p/>Miller handles <boldmaroon>tabular data</boldmaroon>, which is a list of
<p/>Miller handles <span class="boldmaroon">tabular data</span>, which is a list of
records each having fields which are key-value pairs. Miller also doesn&rsquo;t
require that each record have the same field names (see also <a
href="record-heterogeneity.html">here</a>). Regardless, tabular data is a
<boldmaroon>non-recursive data structure</boldmaroon>.
<span class="boldmaroon">non-recursive data structure</span>.
<p/> XML, JSON, etc. are, by contrast, all <boldmaroon>recursive</boldmaroon>
or <boldmaroon>nested</boldmaroon> data structures. For example, in JSON
<p/> XML, JSON, etc. are, by contrast, all <span class="boldmaroon">recursive</span>
or <span class="boldmaroon">nested</span> data structures. For example, in JSON
you can represent a hash map whose values are lists of lists.
<p/>Now, you can put tabular data into these formats &mdash; since list-of-key-value-pairs

View file

@ -179,7 +179,7 @@ POKI_RUN_COMMAND{{mlr --nidx --fs ' ' --repifs cut -f 2,3 data/mydata.txt}}HERE
<p/>JSON is a format which supports arbitrarily deep nesting of
&ldquo;objects&rdquo; (hashmaps) and &ldquo;arrays&rdquo; (lists), while Miller
is a tool for handling <boldmaroon>tabular data</boldmaroon> only. This means
is a tool for handling <span class="boldmaroon">tabular data</span> only. This means
Miller cannot (and should not) handle arbitrary JSON. (Check out <a
href="http://stedolan.github.io/jq/">jq</a>.)
@ -187,8 +187,8 @@ href="http://stedolan.github.io/jq/">jq</a>.)
<h2>Single-level JSON objects</h2>
An <boldmaroon>array of single-level objects</boldmaroon> is, quite simply,
<boldmaroon>a table:</boldmaroon>
An <span class="boldmaroon">array of single-level objects</span> is, quite simply,
<span class="boldmaroon">a table:</span>
POKI_RUN_COMMAND{{mlr --json head -n 2 then cut -f color,shape data/json-example-1.json}}HERE
POKI_RUN_COMMAND{{mlr --json --jvstack head -n 2 then cut -f color,u,v data/json-example-1.json}}HERE
@ -196,7 +196,7 @@ POKI_RUN_COMMAND{{mlr --ijson --opprint stats1 -a mean,stddev,count -f u -g shap
<h2>Nested JSON objects</h2>
<p/>Additionally, Miller can <boldmaroon>tabularize nested objects by concatentating keys:</boldmaroon>
<p/>Additionally, Miller can <span class="boldmaroon">tabularize nested objects by concatentating keys:</span>
POKI_RUN_COMMAND{{mlr --json --jvstack head -n 2 data/json-example-2.json}}HERE
POKI_RUN_COMMAND{{mlr --ijson --opprint head -n 4 data/json-example-2.json}}HERE
@ -358,15 +358,15 @@ POKI_RUN_COMMAND{{mlr --usage-format-conversion-keystroke-saver-options}}HERE
<div id="body_section_toggle_autodetect_line_endings" style="display: block">
<p/> Default line endings (<code>--irs</code> and <code>--ors</code>) are <code>'auto'</code>
which means <boldmaroon>autodetect from the input file format</boldmaroon>, as
which means <span class="boldmaroon">autodetect from the input file format</span>, as
long as the input file(s) have lines ending in either LF (also known as
linefeed, <code>'\n'</code>, <code>0x0a</code>, Unix-style) or CRLF (also known as
carriage-return/linefeed pairs, <code>'\r\n'</code>, <code>0x0d 0x0a</code>, Windows
style).
<p/> <boldmaroon>If both IRS and ORS are auto (which is the default) then LF input will
<p/> <span class="boldmaroon">If both IRS and ORS are auto (which is the default) then LF input will
lead to LF output and CRLF input will lead to CRLF output, regardless of the
platform you&rsquo;re running on</boldmaroon>.
platform you&rsquo;re running on</span>.
<p/> The line-ending autodetector triggers on the first line ending detected in
the input stream. E.g. if you specify a CRLF-terminated file on the command

View file

@ -6,7 +6,7 @@ online-analytical-processing culture. Other key participants include
concept, Miller explicitly strives to imitate several existing tools:
<p/>
<boldmaroon>Unix toolkit</boldmaroon>: Intentional similarities as described in
<span class="boldmaroon">Unix toolkit</span>: Intentional similarities as described in
POKI_PUT_LINK_FOR_PAGE(feature-comparison.html)HERE.
<p/>Recipes abound for command-line data analysis using the Unix toolkit. Here are just a couple of my favorites:
@ -16,7 +16,7 @@ POKI_PUT_LINK_FOR_PAGE(feature-comparison.html)HERE.
<li/> <a href="https://github.com/dbohdan/structured-text-tools">https://github.com/dbohdan/structured-text-tools</a>
</ul>
<p/> <boldmaroon>RecordStream</boldmaroon>: Miller owes particular inspiration
<p/> <span class="boldmaroon">RecordStream</span>: Miller owes particular inspiration
to <a href="https://github.com/benbernard/RecordStream">RecordStream</a>. The
key difference is that RecordStream is a Perl-based tool for manipulating JSON
(including requiring it to separately manipulate other formats such as CSV into
@ -25,15 +25,15 @@ The similarities include the <code>sort</code>, <code>stats1</code> (analog of
RecordStream&rsquo;s <code>collate</code>), and <code>delta</code> operations, as well
as <code>filter</code> and <code>put</code>, and pretty-print formatting.
<p/> <boldmaroon>stats_m</boldmaroon>: A third source of lineage is my Python
<p/> <span class="boldmaroon">stats_m</span>: A third source of lineage is my Python
<a href="https://github.com/johnkerl/scripts-math/tree/master/stats">stats_m</a>
module. This includes simple single-pass algorithms which form Miller&rsquo;s
<code>stats1</code> and <code>stats2</code> subcommands.
<p/> <boldmaroon>SQL</boldmaroon>: Fourthly, Miller&rsquo;s <code>group-by</code> command
<p/> <span class="boldmaroon">SQL</span>: Fourthly, Miller&rsquo;s <code>group-by</code> command
name is from SQL, as is the term <code>aggregate</code>.
<p/> <boldmaroon>Added value</boldmaroon>:
<p/> <span class="boldmaroon">Added value</span>:
Miller&rsquo;s added values include:
<ul>
<li> Name-indexing, compared to the Unix toolkit&rsquo;s positional indexing.
@ -43,17 +43,17 @@ Miller&rsquo;s added values include:
<li> Various file formats, and on-the-fly format conversion.
</ul>
<p/><boldmaroon>jq</boldmaroon>: Miller does for name-indexed text what
<p/><span class="boldmaroon">jq</span>: Miller does for name-indexed text what
<a href="http://stedolan.github.io/jq/">jq</a> does for JSON. If you&rsquo;re
not already familiar with <code>jq</code>, please <a href="http://stedolan.github.io/jq/">check it out!</a>.
<p/><boldmaroon>What about similar tools?</boldmaroon>
<p/><span class="boldmaroon">What about similar tools?</span>
Here&rsquo;s a comprehensive list:
<a href="https://github.com/dbohdan/structured-text-tools">https://github.com/dbohdan/structured-text-tools</a>.
It doesn&rsquo;t mention <a href="https://github.com/turicas/rows">rows</a> so here&rsquo;s a plug for that as well.
As it turns out, I learned about most of these after writing Miller.
<p/><boldmaroon>What about DOTADIW?</boldmaroon> One of the key points of the
<p/><span class="boldmaroon">What about DOTADIW?</span> One of the key points of the
<a href="http://en.wikipedia.org/wiki/Unix_philosophy">Unix philosophy</a> is
that a tool should do one thing and do it well. Hence <code>sort</code> and
<code>cut</code> do just one thing. Why does Miller put <code>awk</code>-like
@ -70,7 +70,7 @@ be a tool which collects together format-aware record-stream processing into
one place, with good reuse of Miller-internal library code for its various
features.
<p/><boldmaroon>Why not use Perl/Python/Ruby etc.?</boldmaroon> Maybe you
<p/><span class="boldmaroon">Why not use Perl/Python/Ruby etc.?</span> Maybe you
should. With those tools you&rsquo;ll get far more expressive power, and
sufficiently quick turnaround time for small-to-medium-sized data. Using
Miller you&rsquo;ll get something less than a complete programming language,
@ -88,8 +88,8 @@ If I&rsquo;d gotten good enough performance from the latter I&rsquo;d have done
it without question and Miller would be far more flexible. But C won the
performance criteria by a landslide so we have Miller in C with a custom DSL.
<p/> <boldmaroon>No, really, why one more command-line data-manipulation
tool?</boldmaroon> I wrote Miller because I was frustrated with tools like
<p/> <span class="boldmaroon">No, really, why one more command-line data-manipulation
tool?</span> I wrote Miller because I was frustrated with tools like
<code>grep</code>, <code>sed</code>, and so on being <i>line-aware</i> without being
<i>format-aware</i>. The single most poignant example I can think of is seeing
people grep data lines out of their CSV files and sadly losing their header

View file

@ -13,15 +13,15 @@ why I felt it necessary to build Miller, etc. Here are some answers.
<p/> For background, I&rsquo;m a software engineer, with a heavy devops bent
and a non-trivial amount of data-engineering in my career.
<boldmaroon>Initially I wrote Miller mainly for myself:</boldmaroon> I&rsquo;m
<span class="boldmaroon">Initially I wrote Miller mainly for myself:</span> I&rsquo;m
coder-friendly (being a coder); I&rsquo;m Github-friendly; most of my data are
well-structured or easily structurable (TSV-formatted SQL-query output, CSV
files, log files, JSON data structures); I care about interoperability between
all the various formats Miller supports (I&rsquo;ve encountered them all); I do
all my work on Linux or OSX.
<p/> But now there&rsquo;s this neat little tool <boldmaroon>which seems to be
useful for people in various disciplines</boldmaroon>. I don&rsquo;t even know
<p/> But now there&rsquo;s this neat little tool <span class="boldmaroon">which seems to be
useful for people in various disciplines</span>. I don&rsquo;t even know
entirely <i>who</i>. I can click through Github starrers and read a bit about
what they seem to do, but not everyone&rsquo;s <i>on</i> Github (or stars
things). I&rsquo;ve gotten a lot of feature requests through Github &mdash; but
@ -50,7 +50,7 @@ it&rsquo;s for?
years of my career in the software industry I&rsquo;ve found myself, and
others, doing a lot of ad-hoc things which really were fundamentally the same
<i>except</i> for format. So the number one thing about Miller is doing common
things while supporting <boldmaroon>multiple formats</boldmaroon>: (a) ingest a
things while supporting <span class="boldmaroon">multiple formats</span>: (a) ingest a
list of records where a record is a list of key-value pairs (however
represented in the input files); (b) transform that stream of records; (c) emit
the transformed stream &mdash; either in the same format as input, or in a
@ -60,9 +60,9 @@ different format.
something only for a single file format, I didn&rsquo;t want to build something
only for one problem domain. In my work doing software engineering, devops,
data engineering, etc. I saw a lot of commonalities and I wanted to
<boldmaroon>solve as many problems simultaneously as possible</boldmaroon>.
<span class="boldmaroon">solve as many problems simultaneously as possible</span>.
<p/> Third: it had to be <boldmaroon>streaming</boldmaroon>. As time goes by
<p/> Third: it had to be <span class="boldmaroon">streaming</span>. As time goes by
and we (some of us, sometimes) have machines with tens or hundreds of GB of
RAM, it&rsquo;s maybe less important, but I&rsquo;m unhappy with tools which
ingest all data, then do stuff, then emit all data. One reason is to be able to
@ -71,7 +71,7 @@ input which trickles in, e.g. you have some process emitting data now and then
and you can pipe it to Miller and it will emit transformed records one at a
time.
<p/> Fourth: it had to be <boldmaroon>fast</boldmaroon>. This precludes all
<p/> Fourth: it had to be <span class="boldmaroon">fast</span>. This precludes all
sorts of very nice things written in Ruby, for example. I love Ruby as a very
expressive language, and I have several very useful little utility scripts
written in Ruby. But a few years ago I ported over some of my old
@ -83,8 +83,8 @@ in order to make it performant. I did simple experiments in several languages,
and nothing was as fast as C, so I used C: see also <a
href="whyc.html#C_vs._Go,_D,_Rust,_etc.;_C_is_fast">here</a>.
<p/> Fifth thing: I wanted Miller to be <boldmaroon>pipe-friendly and
interoperate with other command-line tools</boldmaroon>. Since the basic
<p/> Fifth thing: I wanted Miller to be <span class="boldmaroon">pipe-friendly and
interoperate with other command-line tools</span>. Since the basic
paradigm is ingest records, transform records, emit records &mdash; where the
input and output formats can be the same or different, and the transform can be
complex, or just pass-through &mdash; this means you can use it to transform
@ -93,17 +93,17 @@ data-cleaning/prep/formatting and do all the "real" work in R, you can. If you
just want a little glue script between other tools you can get that. And if you
want to do non-trivial data-reduction in Miller you can.
<p/> Sixth thing: Must have <boldmaroon>comprehensive documentation and
unit-test</boldmaroon>. Since Miller handles a lot of formats and solves a lot
<p/> Sixth thing: Must have <span class="boldmaroon">comprehensive documentation and
unit-test</span>. Since Miller handles a lot of formats and solves a lot
of problems, there&rsquo;s a lot to test and a lot to keep working correctly as
I add features or optimize. And I wanted it to be able to explain itself
&mdash; not only through web docs like the one you&rsquo;re reading but also
through <code>man mlr</code> and <code>mlr --help</code>, <code>mlr sort --help</code>,
etc.
<p/> Seventh thing: <boldmaroon>Must have a domain-specific
language</boldmaroon> (DSL) <boldmaroon>but also must let you do common things
without it</boldmaroon>. All those little verbs Miller has to help you
<p/> Seventh thing: <span class="boldmaroon">Must have a domain-specific
language</span> (DSL) <span class="boldmaroon">but also must let you do common things
without it</span>. All those little verbs Miller has to help you
<i>avoid</i> having to write for-loops are great. I use them for
keystroke-saving: <code>mlr stats1 -a mean,stddev,min,max -f quantity</code>, for
example, without you having to write for-loops or define accumulator variables.
@ -112,8 +112,8 @@ you want to: <code>mlr put '$distance = $rate * $time'</code> or anything else y
can think up. In Perl/AWK/etc. it&rsquo;s all DSL. In xsv et al. it&rsquo;s
all verbs. In Miller I like having the combination.
<p/> Eighth thing: It&rsquo;s an <boldmaroon>awful lot of fun to
write</boldmaroon>. In my experience I didn&rsquo;t find any tools which do
<p/> Eighth thing: It&rsquo;s an <span class="boldmaroon">awful lot of fun to
write</span>. In my experience I didn&rsquo;t find any tools which do
multi-format, streaming, efficient, multi-purpose, with DSL and non-DSL, so I
wrote one. But I don&rsquo;t guarantee it&rsquo;s unique in the world. It fills
a niche in the world (people use it) but it also fills a niche in my life.

View file

@ -14,10 +14,10 @@ body {
background-color: white;
}
maroon {
.maroon {
color: maroon;
}
boldmaroon {
.boldmaroon {
font-weight: bold;
color: maroon;
}

View file

@ -995,14 +995,14 @@ id,code,color
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="bodyToggler.toggle('body_section_toggle_xml_or_json');" href="javascript:;">Toggle section visibility</button>
<div id="body_section_toggle_xml_or_json" style="display: block">
<p/>Miller handles <boldmaroon>tabular data</boldmaroon>, which is a list of
<p/>Miller handles <span class="boldmaroon">tabular data</span>, which is a list of
records each having fields which are key-value pairs. Miller also doesn&rsquo;t
require that each record have the same field names (see also <a
href="record-heterogeneity.html">here</a>). Regardless, tabular data is a
<boldmaroon>non-recursive data structure</boldmaroon>.
<span class="boldmaroon">non-recursive data structure</span>.
<p/> XML, JSON, etc. are, by contrast, all <boldmaroon>recursive</boldmaroon>
or <boldmaroon>nested</boldmaroon> data structures. For example, in JSON
<p/> XML, JSON, etc. are, by contrast, all <span class="boldmaroon">recursive</span>
or <span class="boldmaroon">nested</span> data structures. For example, in JSON
you can represent a hash map whose values are lists of lists.
<p/>Now, you can put tabular data into these formats &mdash; since list-of-key-value-pairs

View file

@ -420,7 +420,7 @@ light
<p/>JSON is a format which supports arbitrarily deep nesting of
&ldquo;objects&rdquo; (hashmaps) and &ldquo;arrays&rdquo; (lists), while Miller
is a tool for handling <boldmaroon>tabular data</boldmaroon> only. This means
is a tool for handling <span class="boldmaroon">tabular data</span> only. This means
Miller cannot (and should not) handle arbitrary JSON. (Check out <a
href="http://stedolan.github.io/jq/">jq</a>.)
@ -428,8 +428,8 @@ href="http://stedolan.github.io/jq/">jq</a>.)
<a id="Single-level_JSON_objects"/><h2>Single-level JSON objects</h2>
An <boldmaroon>array of single-level objects</boldmaroon> is, quite simply,
<boldmaroon>a table:</boldmaroon>
An <span class="boldmaroon">array of single-level objects</span> is, quite simply,
<span class="boldmaroon">a table:</span>
<p/>
<div class="pokipanel">
@ -471,7 +471,7 @@ circle 0.366013 0.209094 3
<a id="Nested_JSON_objects"/><h2>Nested JSON objects</h2>
<p/>Additionally, Miller can <boldmaroon>tabularize nested objects by concatentating keys:</boldmaroon>
<p/>Additionally, Miller can <span class="boldmaroon">tabularize nested objects by concatentating keys:</span>
<p/>
<div class="pokipanel">
@ -878,15 +878,15 @@ output only.
<div id="body_section_toggle_autodetect_line_endings" style="display: block">
<p/> Default line endings (<code>--irs</code> and <code>--ors</code>) are <code>'auto'</code>
which means <boldmaroon>autodetect from the input file format</boldmaroon>, as
which means <span class="boldmaroon">autodetect from the input file format</span>, as
long as the input file(s) have lines ending in either LF (also known as
linefeed, <code>'\n'</code>, <code>0x0a</code>, Unix-style) or CRLF (also known as
carriage-return/linefeed pairs, <code>'\r\n'</code>, <code>0x0d 0x0a</code>, Windows
style).
<p/> <boldmaroon>If both IRS and ORS are auto (which is the default) then LF input will
<p/> <span class="boldmaroon">If both IRS and ORS are auto (which is the default) then LF input will
lead to LF output and CRLF input will lead to CRLF output, regardless of the
platform you&rsquo;re running on</boldmaroon>.
platform you&rsquo;re running on</span>.
<p/> The line-ending autodetector triggers on the first line ending detected in
the input stream. E.g. if you specify a CRLF-terminated file on the command

View file

@ -72,7 +72,7 @@ online-analytical-processing culture. Other key participants include
concept, Miller explicitly strives to imitate several existing tools:
<p/>
<boldmaroon>Unix toolkit</boldmaroon>: Intentional similarities as described in
<span class="boldmaroon">Unix toolkit</span>: Intentional similarities as described in
<a href="feature-comparison.html">Unix-toolkit context</a>.
<p/>Recipes abound for command-line data analysis using the Unix toolkit. Here are just a couple of my favorites:
@ -82,7 +82,7 @@ concept, Miller explicitly strives to imitate several existing tools:
<li/> <a href="https://github.com/dbohdan/structured-text-tools">https://github.com/dbohdan/structured-text-tools</a>
</ul>
<p/> <boldmaroon>RecordStream</boldmaroon>: Miller owes particular inspiration
<p/> <span class="boldmaroon">RecordStream</span>: Miller owes particular inspiration
to <a href="https://github.com/benbernard/RecordStream">RecordStream</a>. The
key difference is that RecordStream is a Perl-based tool for manipulating JSON
(including requiring it to separately manipulate other formats such as CSV into
@ -91,15 +91,15 @@ The similarities include the <code>sort</code>, <code>stats1</code> (analog of
RecordStream&rsquo;s <code>collate</code>), and <code>delta</code> operations, as well
as <code>filter</code> and <code>put</code>, and pretty-print formatting.
<p/> <boldmaroon>stats_m</boldmaroon>: A third source of lineage is my Python
<p/> <span class="boldmaroon">stats_m</span>: A third source of lineage is my Python
<a href="https://github.com/johnkerl/scripts-math/tree/master/stats">stats_m</a>
module. This includes simple single-pass algorithms which form Miller&rsquo;s
<code>stats1</code> and <code>stats2</code> subcommands.
<p/> <boldmaroon>SQL</boldmaroon>: Fourthly, Miller&rsquo;s <code>group-by</code> command
<p/> <span class="boldmaroon">SQL</span>: Fourthly, Miller&rsquo;s <code>group-by</code> command
name is from SQL, as is the term <code>aggregate</code>.
<p/> <boldmaroon>Added value</boldmaroon>:
<p/> <span class="boldmaroon">Added value</span>:
Miller&rsquo;s added values include:
<ul>
<li> Name-indexing, compared to the Unix toolkit&rsquo;s positional indexing.
@ -109,17 +109,17 @@ Miller&rsquo;s added values include:
<li> Various file formats, and on-the-fly format conversion.
</ul>
<p/><boldmaroon>jq</boldmaroon>: Miller does for name-indexed text what
<p/><span class="boldmaroon">jq</span>: Miller does for name-indexed text what
<a href="http://stedolan.github.io/jq/">jq</a> does for JSON. If you&rsquo;re
not already familiar with <code>jq</code>, please <a href="http://stedolan.github.io/jq/">check it out!</a>.
<p/><boldmaroon>What about similar tools?</boldmaroon>
<p/><span class="boldmaroon">What about similar tools?</span>
Here&rsquo;s a comprehensive list:
<a href="https://github.com/dbohdan/structured-text-tools">https://github.com/dbohdan/structured-text-tools</a>.
It doesn&rsquo;t mention <a href="https://github.com/turicas/rows">rows</a> so here&rsquo;s a plug for that as well.
As it turns out, I learned about most of these after writing Miller.
<p/><boldmaroon>What about DOTADIW?</boldmaroon> One of the key points of the
<p/><span class="boldmaroon">What about DOTADIW?</span> One of the key points of the
<a href="http://en.wikipedia.org/wiki/Unix_philosophy">Unix philosophy</a> is
that a tool should do one thing and do it well. Hence <code>sort</code> and
<code>cut</code> do just one thing. Why does Miller put <code>awk</code>-like
@ -136,7 +136,7 @@ be a tool which collects together format-aware record-stream processing into
one place, with good reuse of Miller-internal library code for its various
features.
<p/><boldmaroon>Why not use Perl/Python/Ruby etc.?</boldmaroon> Maybe you
<p/><span class="boldmaroon">Why not use Perl/Python/Ruby etc.?</span> Maybe you
should. With those tools you&rsquo;ll get far more expressive power, and
sufficiently quick turnaround time for small-to-medium-sized data. Using
Miller you&rsquo;ll get something less than a complete programming language,
@ -154,8 +154,8 @@ If I&rsquo;d gotten good enough performance from the latter I&rsquo;d have done
it without question and Miller would be far more flexible. But C won the
performance criteria by a landslide so we have Miller in C with a custom DSL.
<p/> <boldmaroon>No, really, why one more command-line data-manipulation
tool?</boldmaroon> I wrote Miller because I was frustrated with tools like
<p/> <span class="boldmaroon">No, really, why one more command-line data-manipulation
tool?</span> I wrote Miller because I was frustrated with tools like
<code>grep</code>, <code>sed</code>, and so on being <i>line-aware</i> without being
<i>format-aware</i>. The single most poignant example I can think of is seeing
people grep data lines out of their CSV files and sadly losing their header

View file

@ -84,15 +84,15 @@ why I felt it necessary to build Miller, etc. Here are some answers.
<p/> For background, I&rsquo;m a software engineer, with a heavy devops bent
and a non-trivial amount of data-engineering in my career.
<boldmaroon>Initially I wrote Miller mainly for myself:</boldmaroon> I&rsquo;m
<span class="boldmaroon">Initially I wrote Miller mainly for myself:</span> I&rsquo;m
coder-friendly (being a coder); I&rsquo;m Github-friendly; most of my data are
well-structured or easily structurable (TSV-formatted SQL-query output, CSV
files, log files, JSON data structures); I care about interoperability between
all the various formats Miller supports (I&rsquo;ve encountered them all); I do
all my work on Linux or OSX.
<p/> But now there&rsquo;s this neat little tool <boldmaroon>which seems to be
useful for people in various disciplines</boldmaroon>. I don&rsquo;t even know
<p/> But now there&rsquo;s this neat little tool <span class="boldmaroon">which seems to be
useful for people in various disciplines</span>. I don&rsquo;t even know
entirely <i>who</i>. I can click through Github starrers and read a bit about
what they seem to do, but not everyone&rsquo;s <i>on</i> Github (or stars
things). I&rsquo;ve gotten a lot of feature requests through Github &mdash; but
@ -121,7 +121,7 @@ it&rsquo;s for?
years of my career in the software industry I&rsquo;ve found myself, and
others, doing a lot of ad-hoc things which really were fundamentally the same
<i>except</i> for format. So the number one thing about Miller is doing common
things while supporting <boldmaroon>multiple formats</boldmaroon>: (a) ingest a
things while supporting <span class="boldmaroon">multiple formats</span>: (a) ingest a
list of records where a record is a list of key-value pairs (however
represented in the input files); (b) transform that stream of records; (c) emit
the transformed stream &mdash; either in the same format as input, or in a
@ -131,9 +131,9 @@ different format.
something only for a single file format, I didn&rsquo;t want to build something
only for one problem domain. In my work doing software engineering, devops,
data engineering, etc. I saw a lot of commonalities and I wanted to
<boldmaroon>solve as many problems simultaneously as possible</boldmaroon>.
<span class="boldmaroon">solve as many problems simultaneously as possible</span>.
<p/> Third: it had to be <boldmaroon>streaming</boldmaroon>. As time goes by
<p/> Third: it had to be <span class="boldmaroon">streaming</span>. As time goes by
and we (some of us, sometimes) have machines with tens or hundreds of GB of
RAM, it&rsquo;s maybe less important, but I&rsquo;m unhappy with tools which
ingest all data, then do stuff, then emit all data. One reason is to be able to
@ -142,7 +142,7 @@ input which trickles in, e.g. you have some process emitting data now and then
and you can pipe it to Miller and it will emit transformed records one at a
time.
<p/> Fourth: it had to be <boldmaroon>fast</boldmaroon>. This precludes all
<p/> Fourth: it had to be <span class="boldmaroon">fast</span>. This precludes all
sorts of very nice things written in Ruby, for example. I love Ruby as a very
expressive language, and I have several very useful little utility scripts
written in Ruby. But a few years ago I ported over some of my old
@ -154,8 +154,8 @@ in order to make it performant. I did simple experiments in several languages,
and nothing was as fast as C, so I used C: see also <a
href="whyc.html#C_vs._Go,_D,_Rust,_etc.;_C_is_fast">here</a>.
<p/> Fifth thing: I wanted Miller to be <boldmaroon>pipe-friendly and
interoperate with other command-line tools</boldmaroon>. Since the basic
<p/> Fifth thing: I wanted Miller to be <span class="boldmaroon">pipe-friendly and
interoperate with other command-line tools</span>. Since the basic
paradigm is ingest records, transform records, emit records &mdash; where the
input and output formats can be the same or different, and the transform can be
complex, or just pass-through &mdash; this means you can use it to transform
@ -164,17 +164,17 @@ data-cleaning/prep/formatting and do all the "real" work in R, you can. If you
just want a little glue script between other tools you can get that. And if you
want to do non-trivial data-reduction in Miller you can.
<p/> Sixth thing: Must have <boldmaroon>comprehensive documentation and
unit-test</boldmaroon>. Since Miller handles a lot of formats and solves a lot
<p/> Sixth thing: Must have <span class="boldmaroon">comprehensive documentation and
unit-test</span>. Since Miller handles a lot of formats and solves a lot
of problems, there&rsquo;s a lot to test and a lot to keep working correctly as
I add features or optimize. And I wanted it to be able to explain itself
&mdash; not only through web docs like the one you&rsquo;re reading but also
through <code>man mlr</code> and <code>mlr --help</code>, <code>mlr sort --help</code>,
etc.
<p/> Seventh thing: <boldmaroon>Must have a domain-specific
language</boldmaroon> (DSL) <boldmaroon>but also must let you do common things
without it</boldmaroon>. All those little verbs Miller has to help you
<p/> Seventh thing: <span class="boldmaroon">Must have a domain-specific
language</span> (DSL) <span class="boldmaroon">but also must let you do common things
without it</span>. All those little verbs Miller has to help you
<i>avoid</i> having to write for-loops are great. I use them for
keystroke-saving: <code>mlr stats1 -a mean,stddev,min,max -f quantity</code>, for
example, without you having to write for-loops or define accumulator variables.
@ -183,8 +183,8 @@ you want to: <code>mlr put '$distance = $rate * $time'</code> or anything else y
can think up. In Perl/AWK/etc. it&rsquo;s all DSL. In xsv et al. it&rsquo;s
all verbs. In Miller I like having the combination.
<p/> Eighth thing: It&rsquo;s an <boldmaroon>awful lot of fun to
write</boldmaroon>. In my experience I didn&rsquo;t find any tools which do
<p/> Eighth thing: It&rsquo;s an <span class="boldmaroon">awful lot of fun to
write</span>. In my experience I didn&rsquo;t find any tools which do
multi-format, streaming, efficient, multi-purpose, with DSL and non-DSL, so I
wrote one. But I don&rsquo;t guarantee it&rsquo;s unique in the world. It fills
a niche in the world (people use it) but it also fills a niche in my life.