- To do.
+ One of Miller’s strengths is its compact notation: for example, given input of the form
+
+POKI_RUN_COMMAND{{head -n 5 ../data/medium}}HERE
+
+you can simply do
+
+POKI_RUN_COMMAND{{mlr --oxtab stats1 -a sum -f x ../data/medium}}HERE
+
+or
+
+POKI_RUN_COMMAND{{mlr --opprint stats1 -a sum -f x -g b ../data/medium}}HERE
+
+rather than the more tedious
+
+POKI_INCLUDE_AND_RUN_ESCAPED(oosvar-example-sum.sh)HERE
+
+or
+
+POKI_INCLUDE_AND_RUN_ESCAPED(oosvar-example-sum-grouped.sh)HERE
+
+ The former (mlr stats1 et al.) has the advantages of being easier
+to type, being less error-prone to type, and running faster.
+
+ Nonetheless, out-of-stream variables (which I whimsically call
+oosvars), begin/end blocks, and emit statements give you the ability to
+implement logic — if you wish to do so — which isn’t present
+in other Miller verbs. (If you find yourself often using the same
+out-of-stream-variable logic over and over, please file a request at https://github.com/johnkerl/miller/issues
+to get it implemented directly in C as a Miller verb of its own.)
+
+ The following examples compute some things using oosvars which are already
+computable using Miller verbs, by way of providing food for thought.
+
+
+
+
Mean without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --opprint stats1 -a mean -f x data/medium}}HERE
+POKI_INCLUDE_AND_RUN_ESCAPED(data/mean-with-oosvars.sh)HERE
+
+
+
+
Keyed mean without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --opprint stats1 -a mean -f x -g a,b data/medium}}HERE
+POKI_INCLUDE_AND_RUN_ESCAPED(data/keyed-mean-with-oosvars.sh)HERE
+
+
+
+
Variance and standard deviation without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --oxtab stats1 -a count,sum,mean,var,stddev -f x data/medium}}HERE
+POKI_RUN_COMMAND{{cat variance.mlr}}HERE
+POKI_RUN_COMMAND{{mlr --oxtab put -q -f variance.mlr data/medium}}HERE
+
+You can also do this keyed, of course, imitating the keyed-mean example above.
+
+
+
+
Min/max without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --oxtab stats1 -a min,max -f x data/medium}}HERE
+
+POKI_RUN_COMMAND{{mlr --oxtab put -q '@x_min = min(@x_min, $x); @x_max = max(@x_max, $x); end{emitf @x_min, @x_max}' data/medium}}HERE
+
+
+
+
Keyed min/max without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --opprint stats1 -a min,max -f x -g a data/medium}}HERE
+POKI_INCLUDE_AND_RUN_ESCAPED(data/keyed-min-max-with-oosvars.sh)HERE
+
+
+
+
Delta without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --opprint step -a delta -f x data/small}}HERE
+
+POKI_RUN_COMMAND{{mlr --opprint put '$x_delta = is_present(@last) ? $x - @last : 0; @last = $x' data/small}}HERE
+
+
+
+
Keyed delta without/with oosvars
+
+
+
+POKI_RUN_COMMAND{{mlr --opprint step -a delta -f x -g a data/small}}HERE
+
+POKI_RUN_COMMAND{{mlr --opprint put '$x_delta = is_present(@last[$a]) ? $x - @last[$a] : 0; @last[$a]=$x' data/small}}HERE
+
+
-
- One of Miller’s strengths is its compact notation: for example, given input of the form
-
-POKI_RUN_COMMAND{{head -n 5 ../data/medium}}HERE
-
-you can simply do
-
-POKI_RUN_COMMAND{{mlr --oxtab stats1 -a sum -f x ../data/medium}}HERE
-
-or
-
-POKI_RUN_COMMAND{{mlr --opprint stats1 -a sum -f x -g b ../data/medium}}HERE
-
-rather than the more tedious
-
-POKI_INCLUDE_AND_RUN_ESCAPED(oosvar-example-sum.sh)HERE
-
-or
-
-POKI_INCLUDE_AND_RUN_ESCAPED(oosvar-example-sum-grouped.sh)HERE
-
- The former (mlr stats1 et al.) has the advantages of being easier
-to type, being less error-prone to type, and running faster.
-
- Nonetheless, out-of-stream variables (which I whimsically call
-oosvars), begin/end blocks, and emit statements give you the ability to
-implement logic — if you wish to do so — which isn’t present
-in other Miller verbs. (If you find yourself often using the same
-out-of-stream-variable logic over and over, please file a request at https://github.com/johnkerl/miller/issues
-to get it implemented directly in C as a Miller verb of its own.)
-
- The following examples compute some things using oosvars which are already
-computable using Miller verbs, by way of providing food for thought.
-
-
-
-
Mean without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --opprint stats1 -a mean -f x data/medium}}HERE
-POKI_INCLUDE_AND_RUN_ESCAPED(data/mean-with-oosvars.sh)HERE
-
-
-
-
Keyed mean without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --opprint stats1 -a mean -f x -g a,b data/medium}}HERE
-POKI_INCLUDE_AND_RUN_ESCAPED(data/keyed-mean-with-oosvars.sh)HERE
-
-
-
-
Variance and standard deviation without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --oxtab stats1 -a count,sum,mean,var,stddev -f x data/medium}}HERE
-POKI_RUN_COMMAND{{cat variance.mlr}}HERE
-POKI_RUN_COMMAND{{mlr --oxtab put -q -f variance.mlr data/medium}}HERE
-
-You can also do this keyed, of course, imitating the keyed-mean example above.
-
-
-
-
Min/max without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --oxtab stats1 -a min,max -f x data/medium}}HERE
-
-POKI_RUN_COMMAND{{mlr --oxtab put -q '@x_min = min(@x_min, $x); @x_max = max(@x_max, $x); end{emitf @x_min, @x_max}' data/medium}}HERE
-
-
-
-
Keyed min/max without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --opprint stats1 -a min,max -f x -g a data/medium}}HERE
-POKI_INCLUDE_AND_RUN_ESCAPED(data/keyed-min-max-with-oosvars.sh)HERE
-
-
-
-
Delta without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --opprint step -a delta -f x data/small}}HERE
-
-POKI_RUN_COMMAND{{mlr --opprint put '$x_delta = is_present(@last) ? $x - @last : 0; @last = $x' data/small}}HERE
-
-
-
-
Keyed delta without/with oosvars
-
-
-
-POKI_RUN_COMMAND{{mlr --opprint step -a delta -f x -g a data/small}}HERE
-
-POKI_RUN_COMMAND{{mlr --opprint put '$x_delta = is_present(@last[$a]) ? $x - @last[$a] : 0; @last[$a]=$x' data/small}}HERE
-
-
diff --git a/doc/content-for-originality.html b/doc/content-for-originality.html
index 9999210e7..b5be809f3 100644
--- a/doc/content-for-originality.html
+++ b/doc/content-for-originality.html
@@ -45,6 +45,12 @@ Miller’s added values include:
jq does for JSON. If you’re
not already familiar with jq, please check it out!.
+What about similar tools?
+Here’s a comprehensive list:
+https://github.com/dbohdan/structured-text-tools.
+It doesn’t mention rows so here’s a plug for that as well.
+As it turns out, I learned about most of these after writing Miller.
+
What about DOTADIW? One of the key points of the
Unix philosophy is
that a tool should do one thing and do it well. Hence sort and
diff --git a/doc/cookbook.html b/doc/cookbook.html
index c8069848e..95cd45a08 100644
--- a/doc/cookbook.html
+++ b/doc/cookbook.html
@@ -115,7 +115,6 @@ Miller commands were run with pretty-print-tabular output format.
• Cookbook part 1 • Cookbook part 2 • Cookbook part 3
- • Cookbook part 4 • Data-diving examples • Manpage • Reference
diff --git a/doc/cookbook2.html b/doc/cookbook2.html
index 7ccebe000..2142c20c4 100644
--- a/doc/cookbook2.html
+++ b/doc/cookbook2.html
@@ -115,7 +115,6 @@ Miller commands were run with pretty-print-tabular output format.
• Cookbook part 1 • Cookbook part 2 • Cookbook part 3
- • Cookbook part 4 • Data-diving examples • Manpage • Reference
diff --git a/doc/cookbook3.html b/doc/cookbook3.html
index f76212e42..9084a5450 100644
--- a/doc/cookbook3.html
+++ b/doc/cookbook3.html
@@ -115,7 +115,6 @@ Miller commands were run with pretty-print-tabular output format.
• Cookbook part 1 • Cookbook part 2 • Cookbook part 3
- • Cookbook part 4 • Data-diving examples • Manpage • Reference
@@ -188,12 +187,20 @@ Miller commands were run with pretty-print-tabular output format.
-Comparing verbs and DSL
+Stats with and without out-of-stream variables
+
+
+ The former (mlr stats1 et al.) has the advantages of being easier
+to type, being less error-prone to type, and running faster.
+
+ Nonetheless, out-of-stream variables (which I whimsically call
+oosvars), begin/end blocks, and emit statements give you the ability to
+implement logic — if you wish to do so — which isn’t present
+in other Miller verbs. (If you find yourself often using the same
+out-of-stream-variable logic over and over, please file a request at https://github.com/johnkerl/miller/issues
+to get it implemented directly in C as a Miller verb of its own.)
+
+ The following examples compute some things using oosvars which are already
+computable using Miller verbs, by way of providing food for thought.
+
+
+
+
Mean without/with oosvars
+
+
+
+
+
+
+$ mlr --opprint stats1 -a mean -f x data/medium
+x_mean
+0.498602
+
+$ mlr --opprint step -a delta -f x data/small
+a b i x y x_delta
+pan pan 1 0.3467901443380824 0.7268028627434533 0
+eks pan 2 0.7586799647899636 0.5221511083334797 0.411890
+wye wye 3 0.20460330576630303 0.33831852551664776 -0.554077
+eks wye 4 0.38139939387114097 0.13418874328430463 0.176796
+wye pan 5 0.5732889198020006 0.8636244699032729 0.191890
+
+
+
+
+
+
+
+$ mlr --opprint put '$x_delta = is_present(@last) ? $x - @last : 0; @last = $x' data/small
+a b i x y x_delta
+pan pan 1 0.3467901443380824 0.7268028627434533 0
+eks pan 2 0.7586799647899636 0.5221511083334797 0.411890
+wye wye 3 0.20460330576630303 0.33831852551664776 -0.554077
+eks wye 4 0.38139939387114097 0.13418874328430463 0.176796
+wye pan 5 0.5732889198020006 0.8636244699032729 0.191890
+
+
+
+
+
+
+
Keyed delta without/with oosvars
+
+
+
+
+
+
+$ mlr --opprint step -a delta -f x -g a data/small
+a b i x y x_delta
+pan pan 1 0.3467901443380824 0.7268028627434533 0
+eks pan 2 0.7586799647899636 0.5221511083334797 0
+wye wye 3 0.20460330576630303 0.33831852551664776 0
+eks wye 4 0.38139939387114097 0.13418874328430463 -0.377281
+wye pan 5 0.5732889198020006 0.8636244699032729 0.368686
+
+
+
+
+
+
+
+$ mlr --opprint put '$x_delta = is_present(@last[$a]) ? $x - @last[$a] : 0; @last[$a]=$x' data/small
+a b i x y x_delta
+pan pan 1 0.3467901443380824 0.7268028627434533 0
+eks pan 2 0.7586799647899636 0.5221511083334797 0
+wye wye 3 0.20460330576630303 0.33831852551664776 0
+eks wye 4 0.38139939387114097 0.13418874328430463 -0.377281
+wye pan 5 0.5732889198020006 0.8636244699032729 0.368686
+
-
-
- The former (mlr stats1 et al.) has the advantages of being easier
-to type, being less error-prone to type, and running faster.
-
- Nonetheless, out-of-stream variables (which I whimsically call
-oosvars), begin/end blocks, and emit statements give you the ability to
-implement logic — if you wish to do so — which isn’t present
-in other Miller verbs. (If you find yourself often using the same
-out-of-stream-variable logic over and over, please file a request at https://github.com/johnkerl/miller/issues
-to get it implemented directly in C as a Miller verb of its own.)
-
- The following examples compute some things using oosvars which are already
-computable using Miller verbs, by way of providing food for thought.
-
-
-
-
Mean without/with oosvars
-
-
-
-
-
-
-$ mlr --opprint stats1 -a mean -f x data/medium
-x_mean
-0.498602
-
-$ mlr --opprint step -a delta -f x data/small
-a b i x y x_delta
-pan pan 1 0.3467901443380824 0.7268028627434533 0
-eks pan 2 0.7586799647899636 0.5221511083334797 0.411890
-wye wye 3 0.20460330576630303 0.33831852551664776 -0.554077
-eks wye 4 0.38139939387114097 0.13418874328430463 0.176796
-wye pan 5 0.5732889198020006 0.8636244699032729 0.191890
-
-
-
-
-
-
-
-$ mlr --opprint put '$x_delta = is_present(@last) ? $x - @last : 0; @last = $x' data/small
-a b i x y x_delta
-pan pan 1 0.3467901443380824 0.7268028627434533 0
-eks pan 2 0.7586799647899636 0.5221511083334797 0.411890
-wye wye 3 0.20460330576630303 0.33831852551664776 -0.554077
-eks wye 4 0.38139939387114097 0.13418874328430463 0.176796
-wye pan 5 0.5732889198020006 0.8636244699032729 0.191890
-
-
-
-
-
-
-
Keyed delta without/with oosvars
-
-
-
-
-
-
-$ mlr --opprint step -a delta -f x -g a data/small
-a b i x y x_delta
-pan pan 1 0.3467901443380824 0.7268028627434533 0
-eks pan 2 0.7586799647899636 0.5221511083334797 0
-wye wye 3 0.20460330576630303 0.33831852551664776 0
-eks wye 4 0.38139939387114097 0.13418874328430463 -0.377281
-wye pan 5 0.5732889198020006 0.8636244699032729 0.368686
-
-
-
-
-
-
-
-$ mlr --opprint put '$x_delta = is_present(@last[$a]) ? $x - @last[$a] : 0; @last[$a]=$x' data/small
-a b i x y x_delta
-pan pan 1 0.3467901443380824 0.7268028627434533 0
-eks pan 2 0.7586799647899636 0.5221511083334797 0
-wye wye 3 0.20460330576630303 0.33831852551664776 0
-eks wye 4 0.38139939387114097 0.13418874328430463 -0.377281
-wye pan 5 0.5732889198020006 0.8636244699032729 0.368686
-
-$ mlr --opprint step -a ewma -d 0.1 -f x data/small
-a b i x y x_ewma_0.1
-pan pan 1 0.3467901443380824 0.7268028627434533 0.346790
-eks pan 2 0.7586799647899636 0.5221511083334797 0.387979
-wye wye 3 0.20460330576630303 0.33831852551664776 0.369642
-eks wye 4 0.38139939387114097 0.13418874328430463 0.370817
-wye pan 5 0.5732889198020006 0.8636244699032729 0.391064
-
-
-
-
-
-
-
-$ mlr --opprint put '
- begin{ @a=0.1 };
- $e = NR==1 ? $x : @a * $x + (1 - @a) * @e;
- @e=$e
-' data/small
-a b i x y e
-pan pan 1 0.3467901443380824 0.7268028627434533 0.346790
-eks pan 2 0.7586799647899636 0.5221511083334797 0.387979
-wye wye 3 0.20460330576630303 0.33831852551664776 0.369642
-eks wye 4 0.38139939387114097 0.13418874328430463 0.370817
-wye pan 5 0.5732889198020006 0.8636244699032729 0.391064
-