# Scripting with Miller
Suppose you are often doing something like
mlr --icsv --opprint \
filter '$quantity != 20' \
then count-distinct -f shape \
then fraction -f count \
example.csv
shape count count_fraction
triangle 3 0.3
square 4 0.4
circle 3 0.3
Typing this out can get a bit old, if the only thing that changes for you is the filename. Some options include:
* On Linux/Mac/etc you can make a script with `#!/bin/sh` which invokes Miller as part of the shell-script body.
* On Linux/Mac/etc you can make a script with `#!/usr/bin/env mlr -s` which invokes Miller.
* On any platform you can put the reusable part of your command line into a text file (say `myflags.txt`), then `mlr -s myflags-txt filename-which-varies.csv`.
Let's look at examples of each.
## Shell scripts
A shell-script option:
cat example-shell-script
#!/bin/bash
mlr --c2p \
filter '$quantity != 20' \
then count-distinct -f shape \
then fraction -f count \
-- "$@"
Key points here:
* Use `--` before `"$@"` at the end so that main-flags like `--json` won't be confused for options to the `fraction` verb.
* Use `"$@"` at the end which means "all remaining arguments to the script".
* Use `chmod +x example-shell-script` after you create one of these.
Then you can do
example-shell-script example.csv
shape count count_fraction
triangle 3 0.3
square 4 0.4
circle 3 0.3
cat example.csv | example-shell-script
shape count count_fraction
triangle 3 0.3
square 4 0.4
circle 3 0.3
example-shell-script --ojson example.csv
{
"shape": "triangle",
"count": 3,
"count_fraction": 0.3
}
{
"shape": "square",
"count": 4,
"count_fraction": 0.4
}
{
"shape": "circle",
"count": 3,
"count_fraction": 0.3
}
example-shell-script --ojson then filter '$count == 3' example.csv
{
"shape": "triangle",
"count": 3,
"count_fraction": 0.3
}
{
"shape": "circle",
"count": 3,
"count_fraction": 0.3
}
etc.
## Miller scripts
Here instead of putting `#!/bin/bash` on the first line, we can put `mlr` directly:
cat example-mlr-s-script
#!/usr/bin/env mlr -s
--c2p
filter '$quantity != 20'
then count-distinct -f shape
then fraction -f count
Points:
* This is largely the same as a shell script.
* Use `chmod +x example-mlr-s-script` after you create one of these.
* You leave off the initial `mlr` since that's present on line 1.
* You don't need all the backslashing for line-continuations.
* You don't need the explicit `--` or `"$@"`.
Then you can do
example-mlr-s-script example.csv
shape count count_fraction
triangle 3 0.3
square 4 0.4
circle 3 0.3
cat example.csv | example-mlr-s-script
shape count count_fraction
triangle 3 0.3
square 4 0.4
circle 3 0.3
example-mlr-s-script --ojson example.csv
{
"shape": "triangle",
"count": 3,
"count_fraction": 0.3
}
{
"shape": "square",
"count": 4,
"count_fraction": 0.4
}
{
"shape": "circle",
"count": 3,
"count_fraction": 0.3
}
example-mlr-s-script --ojson then filter '$count == 3' example.csv
{
"shape": "triangle",
"count": 3,
"count_fraction": 0.3
}
{
"shape": "circle",
"count": 3,
"count_fraction": 0.3
}
## Miller scripts on Windows
Both the previous options require executable mode with `chmod`, and a _shebang
line_ with `#!...`, which are unixisms.
One of the nice features of `mlr -s` is it can be done without a shebang line,
and this works fine on Windows. For example:
cat example-mlr-s-script-no-shebang
--c2p
filter '$quantity != 20'
then count-distinct -f shape
then fraction -f count
Points:
* Same as above, where the `#!` line isn't needed. (But you can include a `#!` line; `mlr -s` will simply see it as a comment line.).
* As above, you don't need all the backslashing for line-continuations.
* As above, you don't need the explicit `--` or `"$@"`.
Then you can do
mlr -s example-mlr-s-script-no-shebang example.csv
shape count count_fraction
triangle 3 0.3
square 4 0.4
circle 3 0.3
mlr -s example-mlr-s-script-no-shebang --ojson example.csv
{
"shape": "triangle",
"count": 3,
"count_fraction": 0.3
}
{
"shape": "square",
"count": 4,
"count_fraction": 0.4
}
{
"shape": "circle",
"count": 3,
"count_fraction": 0.3
}
mlr -s example-mlr-s-script-no-shebang --ojson then filter '$count == 3' example.csv
{
"shape": "triangle",
"count": 3,
"count_fraction": 0.3
}
{
"shape": "circle",
"count": 3,
"count_fraction": 0.3
}
and so on. See also [Miller on Windows](miller-on-windows.md).