mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-22 07:30:43 +00:00
oosvar doc
This commit is contained in:
John Kerl
parent
f29547e23e
commit
38391d16cb
3 changed files with 55 additions and 62 deletions
|
|
@ -10,8 +10,6 @@ TOP-OF-LIST SUMMARY
|
|||
|
||||
* rh/fedora/centos mlr-3.5.0
|
||||
|
||||
? were += et al. ever announced?!?
|
||||
|
||||
* reduce parameter-marshaling by pevalbag; measure perf delta (suspect negligible)
|
||||
* neaten/xxxes/valgrind/UT/doc for emit/unset/tri-split-null/+=etal
|
||||
* double-check oosvar+emit+tri-split-null by re-implementing as many mappers as possible using ossvars + emit
|
||||
|
|
@ -45,6 +43,7 @@ TOP-OF-LIST SUMMARY
|
|||
- unset/all
|
||||
- bare-boolean and pattern-action
|
||||
- put -q: equiv to putting 'filter false'
|
||||
- += et al. were ever announced -- fix intro release & note
|
||||
|
||||
================================================================
|
||||
~@ PRE4 @~
|
||||
|
|
|
|||
|
|
@ -766,27 +766,8 @@ only when you assign to them.
|
|||
In particular, if you have two or more <tt>put</tt> commands separated by <tt>then</tt>,
|
||||
each put will have its own set of out-of-stream variables:
|
||||
|
||||
<!--
|
||||
xxx begin-block at start of each file: use pattern/action with NR==0
|
||||
xxx += et al. ??
|
||||
xxx semicolon as stt sep
|
||||
xxx fix me
|
||||
PxKI_RUN_COMMAND{{mlr -x-csv cat data/a.csv}}HERE
|
||||
PxKI_RUN_COMMAND{{mlr -x-csv put '@sum += $a; end {emit @sum}' data/a.csv}}HERE
|
||||
PxKI_RUN_COMMAND{{mlr -x-csv put '@sum += $a; end {emit @sum}' then put '$a=10*$a; @sum += $a; end {emit @sum}' data/a.csv}}HERE
|
||||
|
||||
mlr put '@sum1 += $a; @a=$a;dump;emit all' then put '$a=10*$a; @sum2 += $a; @a=$a;dump;emit all' data/a.dkvp
|
||||
mlr put '@sum1 += $a; @a=$a;dump' then put '$a=10*$a; @sum2 += $a; @a=$a;dump' data/a.dkvp
|
||||
mlr put -q '@sum1 += $a; @a=$a;dump;emit all' then put '$a=10*$a; @sum2 += $a; @a=$a;dump;emit all' data/a.dkvp
|
||||
mlr put -q '@sum1 += $a; @a=$a;dump;emit all' then put -q '$a=10*$a; @sum2 += $a; @a=$a;dump;emit all' data/a.dkvp
|
||||
mlr put -q '@sum += $a; end {emit @sum}' then put -q '$a=10*$a; @sum += $a; end {emit @sum}' data/a.dkvp
|
||||
mlr put -q '@sum += $a; end {emit @sum}' then put -q '$a=10*$a; @sum2 += $a; end {emit @sum2}' data/a.dkvp
|
||||
mlr put -q '@sum += $a; end {emit @sum}' then put -q '$a=10*$a; @sum2 += $a; end {emit @sum2}' data/a.dkvp
|
||||
mlr put '@sum += $a; end {emit @sum}' then put '$a=10*$a; @sum2 += $a; end {emit @sum2}' data/a.dkvp
|
||||
mlr put '@sum += $a; end {emit @sum}' then put '$a=10*$a; @sum += $a; end {emit @sum}' data/a.dkvp
|
||||
|
||||
|
||||
-->
|
||||
POKI_RUN_COMMAND{{cat data/a.dkvp}}HERE
|
||||
POKI_RUN_COMMAND{{mlr put '@sum += $a; end {emit @sum}' then put 'isnotnull($a) {$a=10*$a; @sum += $a}; end {emit @sum}' data/a.dkvp}}HERE
|
||||
|
||||
<li/> These are read-write: you can do <tt>$sum=@sum</tt>, <tt>@sum=$sum</tt>,
|
||||
etc.
|
||||
|
|
@ -800,8 +781,6 @@ structure. Note that these aren’t output records which participate in
|
|||
<tt>then</tt>-chaining; rather, they’re just an immediate print to
|
||||
stdout.
|
||||
|
||||
<li/> xxx scope to single put; what if two plus
|
||||
|
||||
</ul>
|
||||
|
||||
<p/>Features of out-of-stream variables, and examples of their use, will be
|
||||
|
|
@ -834,22 +813,28 @@ POKI_RUN_COMMAND{{mlr put '$a =~ "([a-z]+)_([0-9]+)"; $b = "left_\1"; $c = "righ
|
|||
<p/>Miller supports an <tt>awk</tt>-like <tt>begin/end</tt> syntax. The
|
||||
statements in the <tt>begin</tt> block are executed before any input records
|
||||
are read; the statements in the <tt>end</tt> block are executed after the last
|
||||
input record is read. All other statements are, of course, executed on every
|
||||
input record. For example:
|
||||
input record is read. (If you want to execute some statement at the start of
|
||||
each file, not at the start of the first file as with <tt>begin</tt>, you might
|
||||
use a pattern/action block of the form <tt>NR == 0 { ... }</tt>.) All
|
||||
statements outside of <tt>begin</tt> or <tt>end</tt> are, of course, executed
|
||||
on every input record. For example:
|
||||
|
||||
POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-1.sh)HERE
|
||||
|
||||
<p/>Since uninitialized out-of-stream variables default to 0 for
|
||||
addition/substraction and 1 for multiplication when they appear on expression
|
||||
right-hand sides (as in <tt>awk</tt>), the above can be written more succinctly as
|
||||
right-hand sides (as in <tt>awk</tt>), the above can be written more succinctly
|
||||
as
|
||||
|
||||
POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-2.sh)HERE
|
||||
|
||||
<p/>The <b>put -q</b> option is a shorthand which suppresses printing of each output record, with only <tt>emit</tt> statements being output. So to get only summary outputs, one could write
|
||||
<p/>The <b>put -q</b> option is a shorthand which suppresses printing of each
|
||||
output record, with only <tt>emit</tt> statements being output. So to get only
|
||||
summary outputs, one could write
|
||||
|
||||
POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-3.sh)HERE
|
||||
|
||||
<p/>We can do similarly with other variables:
|
||||
<p/>We can do similarly with multiple out-of-stream variables:
|
||||
|
||||
POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-4.sh)HERE
|
||||
|
||||
|
|
@ -857,6 +842,8 @@ This is of course not much different than
|
|||
|
||||
POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-5.sh)HERE
|
||||
|
||||
<h3>Indexed out-of-stream variables for put</h3>
|
||||
|
||||
<p/>Using an index on the <tt>@count</tt> and <tt>@sum</tt> variables, we get the benefit of the
|
||||
<tt>-g</tt> (group-by) option which <tt>mlr stats1</tt> and various other Miller commands have:
|
||||
|
||||
|
|
@ -874,9 +861,9 @@ Miller.</b>
|
|||
|
||||
POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-8.sh)HERE
|
||||
|
||||
<p/>xxx to do:
|
||||
<h3>Emit statements for put</h3>
|
||||
|
||||
<p/> emit variants
|
||||
<p/>xxx to do: emit variants
|
||||
|
||||
<p/> xxx we’ve already seen above; define / exemplify ...
|
||||
|
||||
|
|
|
|||
|
|
@ -173,6 +173,8 @@ Miller commands were run with pretty-print-tabular output format.
|
|||
• <a href="#Out-of-stream_variables_for_put">Out-of-stream variables for put</a><br/>
|
||||
• <a href="#Pattern-action_blocks_for_put">Pattern-action blocks for put</a><br/>
|
||||
• <a href="#Begin/end_blocks_for_put">Begin/end blocks for put</a><br/>
|
||||
• <a href="#Indexed_out-of-stream_variables_for_put">Indexed out-of-stream variables for put</a><br/>
|
||||
• <a href="#Emit_statements_for_put">Emit statements for put</a><br/>
|
||||
• <a href="#More_variable_assignments_for_put">More variable assignments for put</a><br/>
|
||||
• <a href="#regularize">regularize</a><br/>
|
||||
• <a href="#rename">rename</a><br/>
|
||||
|
|
@ -2388,27 +2390,26 @@ only when you assign to them.
|
|||
In particular, if you have two or more <tt>put</tt> commands separated by <tt>then</tt>,
|
||||
each put will have its own set of out-of-stream variables:
|
||||
|
||||
<!--
|
||||
xxx begin-block at start of each file: use pattern/action with NR==0
|
||||
xxx += et al. ??
|
||||
xxx semicolon as stt sep
|
||||
xxx fix me
|
||||
PxKI_RUN_COMMAND{{mlr -x-csv cat data/a.csv}}HERE
|
||||
PxKI_RUN_COMMAND{{mlr -x-csv put '@sum += $a; end {emit @sum}' data/a.csv}}HERE
|
||||
PxKI_RUN_COMMAND{{mlr -x-csv put '@sum += $a; end {emit @sum}' then put '$a=10*$a; @sum += $a; end {emit @sum}' data/a.csv}}HERE
|
||||
|
||||
mlr put '@sum1 += $a; @a=$a;dump;emit all' then put '$a=10*$a; @sum2 += $a; @a=$a;dump;emit all' data/a.dkvp
|
||||
mlr put '@sum1 += $a; @a=$a;dump' then put '$a=10*$a; @sum2 += $a; @a=$a;dump' data/a.dkvp
|
||||
mlr put -q '@sum1 += $a; @a=$a;dump;emit all' then put '$a=10*$a; @sum2 += $a; @a=$a;dump;emit all' data/a.dkvp
|
||||
mlr put -q '@sum1 += $a; @a=$a;dump;emit all' then put -q '$a=10*$a; @sum2 += $a; @a=$a;dump;emit all' data/a.dkvp
|
||||
mlr put -q '@sum += $a; end {emit @sum}' then put -q '$a=10*$a; @sum += $a; end {emit @sum}' data/a.dkvp
|
||||
mlr put -q '@sum += $a; end {emit @sum}' then put -q '$a=10*$a; @sum2 += $a; end {emit @sum2}' data/a.dkvp
|
||||
mlr put -q '@sum += $a; end {emit @sum}' then put -q '$a=10*$a; @sum2 += $a; end {emit @sum2}' data/a.dkvp
|
||||
mlr put '@sum += $a; end {emit @sum}' then put '$a=10*$a; @sum2 += $a; end {emit @sum2}' data/a.dkvp
|
||||
mlr put '@sum += $a; end {emit @sum}' then put '$a=10*$a; @sum += $a; end {emit @sum}' data/a.dkvp
|
||||
|
||||
|
||||
-->
|
||||
<p/>
|
||||
<div class="pokipanel">
|
||||
<pre>
|
||||
$ cat data/a.dkvp
|
||||
a=1,b=2,c=3
|
||||
a=4,b=5,c=6
|
||||
</pre>
|
||||
</div>
|
||||
<p/>
|
||||
<p/>
|
||||
<div class="pokipanel">
|
||||
<pre>
|
||||
$ mlr put '@sum += $a; end {emit @sum}' then put 'isnotnull($a) {$a=10*$a; @sum += $a}; end {emit @sum}' data/a.dkvp
|
||||
a=10,b=2,c=3
|
||||
a=40,b=5,c=6
|
||||
sum=5
|
||||
sum=50
|
||||
</pre>
|
||||
</div>
|
||||
<p/>
|
||||
|
||||
<li/> These are read-write: you can do <tt>$sum=@sum</tt>, <tt>@sum=$sum</tt>,
|
||||
etc.
|
||||
|
|
@ -2422,8 +2423,6 @@ structure. Note that these aren’t output records which participate in
|
|||
<tt>then</tt>-chaining; rather, they’re just an immediate print to
|
||||
stdout.
|
||||
|
||||
<li/> xxx scope to single put; what if two plus
|
||||
|
||||
</ul>
|
||||
|
||||
<p/>Features of out-of-stream variables, and examples of their use, will be
|
||||
|
|
@ -2516,8 +2515,11 @@ a=xyz_789,b=left_xyz,c=right_789
|
|||
<p/>Miller supports an <tt>awk</tt>-like <tt>begin/end</tt> syntax. The
|
||||
statements in the <tt>begin</tt> block are executed before any input records
|
||||
are read; the statements in the <tt>end</tt> block are executed after the last
|
||||
input record is read. All other statements are, of course, executed on every
|
||||
input record. For example:
|
||||
input record is read. (If you want to execute some statement at the start of
|
||||
each file, not at the start of the first file as with <tt>begin</tt>, you might
|
||||
use a pattern/action block of the form <tt>NR == 0 { ... }</tt>.) All
|
||||
statements outside of <tt>begin</tt> or <tt>end</tt> are, of course, executed
|
||||
on every input record. For example:
|
||||
|
||||
<p/>
|
||||
<div class="pokipanel">
|
||||
|
|
@ -2544,7 +2546,8 @@ x_sum=4.536294
|
|||
|
||||
<p/>Since uninitialized out-of-stream variables default to 0 for
|
||||
addition/substraction and 1 for multiplication when they appear on expression
|
||||
right-hand sides (as in <tt>awk</tt>), the above can be written more succinctly as
|
||||
right-hand sides (as in <tt>awk</tt>), the above can be written more succinctly
|
||||
as
|
||||
|
||||
<p/>
|
||||
<div class="pokipanel">
|
||||
|
|
@ -2568,7 +2571,9 @@ x_sum=4.536294
|
|||
</div>
|
||||
<p/>
|
||||
|
||||
<p/>The <b>put -q</b> option is a shorthand which suppresses printing of each output record, with only <tt>emit</tt> statements being output. So to get only summary outputs, one could write
|
||||
<p/>The <b>put -q</b> option is a shorthand which suppresses printing of each
|
||||
output record, with only <tt>emit</tt> statements being output. So to get only
|
||||
summary outputs, one could write
|
||||
|
||||
<p/>
|
||||
<div class="pokipanel">
|
||||
|
|
@ -2582,7 +2587,7 @@ x_sum=4.536294
|
|||
</div>
|
||||
<p/>
|
||||
|
||||
<p/>We can do similarly with other variables:
|
||||
<p/>We can do similarly with multiple out-of-stream variables:
|
||||
|
||||
<p/>
|
||||
<div class="pokipanel">
|
||||
|
|
@ -2612,6 +2617,8 @@ x_count=10,x_sum=4.536294
|
|||
</div>
|
||||
<p/>
|
||||
|
||||
<a id="Indexed_out-of-stream_variables_for_put"/><h3>Indexed out-of-stream variables for put</h3>
|
||||
|
||||
<p/>Using an index on the <tt>@count</tt> and <tt>@sum</tt> variables, we get the benefit of the
|
||||
<tt>-g</tt> (group-by) option which <tt>mlr stats1</tt> and various other Miller commands have:
|
||||
|
||||
|
|
@ -2688,9 +2695,9 @@ num_total=5,num_positive=3
|
|||
</div>
|
||||
<p/>
|
||||
|
||||
<p/>xxx to do:
|
||||
<a id="Emit_statements_for_put"/><h3>Emit statements for put</h3>
|
||||
|
||||
<p/> emit variants
|
||||
<p/>xxx to do: emit variants
|
||||
|
||||
<p/> xxx we’ve already seen above; define / exemplify ...
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue