miller/docs/src/repl.md.in

# The REPL

The Miller REPL (read-evaluate-print loop) is an interactive counterpart to record-processing using the `put`/`filter` language. (A REPL is anything that evaluates what you type into it -- like `python` with no arguments, or Ruby's `irb`, or `node` with no arguments, etc.)

Miller's REPL isn't a source-level debugger which lets you execute one source-code *statement* at a time -- however, it does let you operate on one *record* at a time. Further, it lets you use "immediate expressions", namely, you can interact with the [Miller programming language](miller-programming-language.md) without having to provide data from an input file.

GENMD-CARDIFY-HIGHLIGHT-ONE
mlr repl

[mlr] 1 + 2
3

[mlr] apply([1,2,3,4,5], func(e) {return e ** 3})
[1, 8, 27, 64, 125]

[mlr] :open example.csv

[mlr] :read

[mlr] $*
{
  "color": "yellow",
  "shape": "triangle",
  "flag": "true",
  "k": 1,
  "index": 11,
  "quantity": 43.6498,
  "rate": 9.8870
}

[mlr] :context
FILENAME="example.csv",FILENUM=1,NR=1,FNR=1
GENMD-EOF

## Using Miller without the REPL

Using `put` and `filter`, you can do the following as we've seen above:

* Specify input format (e.g. `--icsv`), output format (e.g. `--ojson`), etc. using command-line flags.
* Specify filenames on the command line.
* Define `begin {...}` blocks which are executed before the first record is read.
* Define `end {...}` blocks which are executed after the last record is read.
* Define user-defined functions/subroutines using `func` and `subr`.
* Specify statements to be executed on each record -- which are anything outside of `begin`/`end`/`func`/`subr`.
* Example:

GENMD-RUN-COMMAND
mlr --icsv --ojson --from example.csv head -n 2 \
  then put 'begin {print "HELLO"} $qr = $quantity / $rate; end {print "GOODBYE"}'
GENMD-EOF

## Using Miller with the REPL

Using the REPL, by contrast, you get interactive control over those same steps:

* Specify input format (e.g. `--icsv`), output format (e.g. `--ojson`), etc. using command-line flags.
* REPL-only statements (non-DSL statements) start with `:`, such as `:help` or `:quit`
  or `:open`.
* Specify filenames either on the command line or via `:open` at the Miller REPL.
* Read records one at a time using `:read`.
* Write the current record (maybe after you've modified it with things like `$z = $x + $y`)
  using `:write`. This goes to the terminal; you can use `:> {filename}` to make writes
  go to a file, or `:>> {filename}` to append.
* You can type `:reopen` to go back to the start of the same file(s) you specified
  with `:open`.
* Skip ahead using statements `:skip 10` or `:skip until NR == 100` or
  `:skip until $status_code != 200`.
* Similarly, but processing records rather than skipping past them, using
  `:process` rather than `:skip`. Like `:write`, these go to the screen;
  use `:> {filename}` or `:>> {filename}` to log to a file instead.
* Define `begin {...}` blocks; invoke them at will using `:begin`.
* Define `end {...}` blocks; invoke them at will using `:end`.
* Define user-defined functions/subroutines using `func`/`subr`; call them from other statements.
* Interactively specify statements to be executed immediately on the current record.
* Load any of the above from Miller-script files using `:load`.

The input "record" by default is the empty map but you can do things like
`$x=3`, or `unset $y`, or `$* = {"x": 3, "y": 4}` to populate it. Or, `:open
foo.dat` followed by `:read` to populate it from a data file.

Non-assignment expressions, such as `7` or `true`, operate as filter conditions
in the `put` DSL: they can be used to specify whether a record will or won't be
included in the output-record stream.  But here in the REPL, they are simply
printed to the terminal, e.g. if you type `1+2`, you will see `3`.

## Entering multi-line statements

* To enter multi-line statements, enter `<` on a line by itself, then the code (taking care
  for semicolons), then `>` on a line by itself. These will be executed immediately.
* If you enter `<<` on a line by itself, then the code, then `>>` on a line by
  itself, the statements will be remembered for executing on records with
  `:main`, as if you had done `:load` to load statements from a file.

## Examples

Use the REPL to look at arithmetic:

GENMD-CARDIFY-HIGHLIGHT-ONE
mlr repl

[mlr] 6/3
2

[mlr] 6/5
1.2

[mlr] typeof(6/3)
int

[mlr] typeof(6/5)
float
GENMD-EOF

Read the first record from a small file:

GENMD-CARDIFY-HIGHLIGHT-ONE
mlr repl

[mlr] :open foo.dat

[mlr] :read

[mlr] :context
FILENAME="foo.dat",FILENUM=1,NR=1,FNR=1

[mlr] $*
{
  "a": "eks",
  "b": "wye",
  "i": 4,
  "x": 0.38139939387114097,
  "y": 0.13418874328430463
}

[mlr] $z = $x + $i

[mlr] :write
a=eks,b=wye,i=4,x=0.38139939387114097,y=0.13418874328430463,z=4.381399393871141
GENMD-EOF

Skip until deep into a larger file, then inspect a record:

GENMD-CARDIFY-HIGHLIGHT-ONE
mlr repl --csv

[mlr] :open data/colored-shapes.csv
[mlr] :skip until NR == 10000
[mlr] :r
[mlr] $*
{
  "color": "yellow",
  "shape": "circle",
  "flag": 1,
  "i": 496422,
  "u": 0.6530503199545348,
  "v": 0.23908588907834516,
  "w": 0.4799125551304738,
  "x": 6.379888206335166
}
GENMD-EOF

## Parsing and operator precedence

You can invoke `mlr repl` with the `-v` or `-d` flags to show parse trees for expressions you enter.
(The `-v` and `-d` flags differ only in the format they use to present the parse trees.)
For example, if you have any questions about operator precedence, you can check the
[operator-precedence section](reference-dsl-operators.md#operator-precedence) -- or, you can
try it out and see for yourself:

GENMD-CARDIFY-HIGHLIGHT-ONE
mlr repl -v
Miller 6.0.0-rc1 REPL for darwin/amd64/go1.16.5
Docs: https://miller.readthedocs.io
Type ':h' or ':help' for online help; ':q' or ':quit' to quit.
[mlr] x = 1 * 2 + 3
* statement block
    * assignment "="
        * local variable "x"
        * operator "+"
            * operator "*"
                * int literal "1"
                * int literal "2"
            * int literal "3"
[mlr] x = 1 + 2 * 3
* statement block
    * assignment "="
        * local variable "x"
        * operator "+"
            * int literal "1"
            * operator "*"
                * int literal "2"
                * int literal "3"
GENMD-EOF

GENMD-CARDIFY-HIGHLIGHT-ONE
mlr repl -d
$ mrpl -d
Miller 6.0.0-rc1 REPL for darwin/amd64/go1.16.5
Docs: https://miller.readthedocs.io
Type ':h' or ':help' for online help; ':q' or ':quit' to quit.
[mlr] x = 1 * 2 + 3
(statement-block
    (=
        x
        (+
            (* 1 2)
            3
        )
    )
)
[mlr] x = 1 + 2 * 3
(statement-block
    (=
        x
        (+
            1
            (* 2 3)
        )
    )
)
GENMD-EOF

## History-editing

No command-line-history-editing feature is built in but **rlwrap mlr repl** is a
delight. You may need `brew install rlwrap`, `sudo apt-get install rlwrap`,
etc. depending on your platform.

Suggestion: `alias mrpl='rlwrap mlr repl'` in your shell's startup file.

## Customization

In your shell environment you can set the `MLR_REPL_PS1` and `MLR_REPL_PS2` environment variables if you like:

GENMD-CARDIFY
$ mlr repl -q
> 1+2
3
> <
... func f(x) {
...   return x**2
... }
... >
> f(99)
9801
GENMD-EOF

GENMD-CARDIFY
$ export MLR_REPL_PS1='\nMLR REPL> '
$ export MLR_REPL_PS2='  '
$ mlr repl -q

MLR REPL> 1+2
3

MLR REPL> <
  func f(x) {
    return x**2
  }
  >

MLR REPL> f(99)
9801
GENMD-EOF

See also `MLR_REPL_PS1_COLOR` on the [output-colorization page](output-colorization.md).

## Online help

After `mlr repl`, type `:help` to see more about your options. In particular, `:help examples`.