diff --git a/c/todo.txt b/c/todo.txt index 88be8bccd..256b174f2 100644 --- a/c/todo.txt +++ b/c/todo.txt @@ -10,8 +10,6 @@ TOP-OF-LIST SUMMARY * rh/fedora/centos mlr-3.5.0 -? were += et al. ever announced?!? - * reduce parameter-marshaling by pevalbag; measure perf delta (suspect negligible) * neaten/xxxes/valgrind/UT/doc for emit/unset/tri-split-null/+=etal * double-check oosvar+emit+tri-split-null by re-implementing as many mappers as possible using ossvars + emit @@ -45,6 +43,7 @@ TOP-OF-LIST SUMMARY - unset/all - bare-boolean and pattern-action - put -q: equiv to putting 'filter false' + - += et al. were ever announced -- fix intro release & note ================================================================ ~@ PRE4 @~ diff --git a/doc/content-for-reference.html b/doc/content-for-reference.html index 066ae82e2..e6cb43aa8 100644 --- a/doc/content-for-reference.html +++ b/doc/content-for-reference.html @@ -766,27 +766,8 @@ only when you assign to them. In particular, if you have two or more put commands separated by then, each put will have its own set of out-of-stream variables: - +POKI_RUN_COMMAND{{cat data/a.dkvp}}HERE +POKI_RUN_COMMAND{{mlr put '@sum += $a; end {emit @sum}' then put 'isnotnull($a) {$a=10*$a; @sum += $a}; end {emit @sum}' data/a.dkvp}}HERE
  • These are read-write: you can do $sum=@sum, @sum=$sum, etc. @@ -800,8 +781,6 @@ structure. Note that these aren’t output records which participate in then-chaining; rather, they’re just an immediate print to stdout. -
  • xxx scope to single put; what if two plus -

    Features of out-of-stream variables, and examples of their use, will be @@ -834,22 +813,28 @@ POKI_RUN_COMMAND{{mlr put '$a =~ "([a-z]+)_([0-9]+)"; $b = "left_\1"; $c = "righ

    Miller supports an awk-like begin/end syntax. The statements in the begin block are executed before any input records are read; the statements in the end block are executed after the last -input record is read. All other statements are, of course, executed on every -input record. For example: +input record is read. (If you want to execute some statement at the start of +each file, not at the start of the first file as with begin, you might +use a pattern/action block of the form NR == 0 { ... }.) All +statements outside of begin or end are, of course, executed +on every input record. For example: POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-1.sh)HERE

    Since uninitialized out-of-stream variables default to 0 for addition/substraction and 1 for multiplication when they appear on expression -right-hand sides (as in awk), the above can be written more succinctly as +right-hand sides (as in awk), the above can be written more succinctly +as POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-2.sh)HERE -

    The put -q option is a shorthand which suppresses printing of each output record, with only emit statements being output. So to get only summary outputs, one could write +

    The put -q option is a shorthand which suppresses printing of each +output record, with only emit statements being output. So to get only +summary outputs, one could write POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-3.sh)HERE -

    We can do similarly with other variables: +

    We can do similarly with multiple out-of-stream variables: POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-4.sh)HERE @@ -857,6 +842,8 @@ This is of course not much different than POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-5.sh)HERE +

    Indexed out-of-stream variables for put

    +

    Using an index on the @count and @sum variables, we get the benefit of the -g (group-by) option which mlr stats1 and various other Miller commands have: @@ -874,9 +861,9 @@ Miller. POKI_INCLUDE_AND_RUN_ESCAPED(data/begin-end-example-8.sh)HERE -

    xxx to do: +

    Emit statements for put

    -

    emit variants +

    xxx to do: emit variants

    xxx we’ve already seen above; define / exemplify ... diff --git a/doc/reference.html b/doc/reference.html index 3c75e7788..1df6f774d 100644 --- a/doc/reference.html +++ b/doc/reference.html @@ -173,6 +173,8 @@ Miller commands were run with pretty-print-tabular output format.         • Out-of-stream variables for put
            • Pattern-action blocks for put
            • Begin/end blocks for put
    +        • Indexed out-of-stream variables for put
    +        • Emit statements for put
            • More variable assignments for put
        • regularize
        • rename
    @@ -2388,27 +2390,26 @@ only when you assign to them. In particular, if you have two or more put commands separated by then, each put will have its own set of out-of-stream variables: - +

    +

    +
    +$ cat data/a.dkvp
+a=1,b=2,c=3
+a=4,b=5,c=6
+
    +
    +

    +

    +

    +
    +$ mlr put '@sum += $a; end {emit @sum}' then put 'isnotnull($a) {$a=10*$a; @sum += $a}; end {emit @sum}' data/a.dkvp
+a=10,b=2,c=3
+a=40,b=5,c=6
+sum=5
+sum=50
+
    +
    +

  • These are read-write: you can do $sum=@sum, @sum=$sum, etc. @@ -2422,8 +2423,6 @@ structure. Note that these aren’t output records which participate in then-chaining; rather, they’re just an immediate print to stdout. -
  • xxx scope to single put; what if two plus -

    Features of out-of-stream variables, and examples of their use, will be @@ -2516,8 +2515,11 @@ a=xyz_789,b=left_xyz,c=right_789

    Miller supports an awk-like begin/end syntax. The statements in the begin block are executed before any input records are read; the statements in the end block are executed after the last -input record is read. All other statements are, of course, executed on every -input record. For example: +input record is read. (If you want to execute some statement at the start of +each file, not at the start of the first file as with begin, you might +use a pattern/action block of the form NR == 0 { ... }.) All +statements outside of begin or end are, of course, executed +on every input record. For example:

    @@ -2544,7 +2546,8 @@ x_sum=4.536294

    Since uninitialized out-of-stream variables default to 0 for addition/substraction and 1 for multiplication when they appear on expression -right-hand sides (as in awk), the above can be written more succinctly as +right-hand sides (as in awk), the above can be written more succinctly +as

    @@ -2568,7 +2571,9 @@ x_sum=4.536294

    -

    The put -q option is a shorthand which suppresses printing of each output record, with only emit statements being output. So to get only summary outputs, one could write +

    The put -q option is a shorthand which suppresses printing of each +output record, with only emit statements being output. So to get only +summary outputs, one could write

    @@ -2582,7 +2587,7 @@ x_sum=4.536294

    -

    We can do similarly with other variables: +

    We can do similarly with multiple out-of-stream variables:

    @@ -2612,6 +2617,8 @@ x_count=10,x_sum=4.536294

    +

    Indexed out-of-stream variables for put

    +

    Using an index on the @count and @sum variables, we get the benefit of the -g (group-by) option which mlr stats1 and various other Miller commands have: @@ -2688,9 +2695,9 @@ num_total=5,num_positive=3

    -

    xxx to do: +

    Emit statements for put

    -

    emit variants +

    xxx to do: emit variants

    xxx we’ve already seen above; define / exemplify ...