diff --git a/Makefile b/Makefile
index ec2b817cb..91a7187a0 100644
--- a/Makefile
+++ b/Makefile
@@ -92,19 +92,20 @@ staticcheck:
# * turns *.md.in into *.md (live code samples), using mlr from the $PATH
# * note the man/manpage.txt becomes some of the HTML content
# * turns *.md into docs/site HTML and CSS files
-dev:
- -make fmt
- make build
+dev: fmt build
make check
make -C man build
make -C docs/src forcebuild
make -C docs
@echo DONE
-docs: build
+docs: fmt build
make -C docs/src forcebuild
make -C docs
+man: fmt build
+ make -C man build
+
# ----------------------------------------------------------------
# Keystroke-savers
@@ -124,4 +125,4 @@ release_tarball: build check
# ================================================================
# Go does its own dependency management, outside of make.
-.PHONY: build mlr check unit_test regression_test bench fmt staticcheck dev docs
+.PHONY: build mlr check unit_test regression_test bench fmt staticcheck dev docs man
diff --git a/docs/src/features.md b/docs/src/features.md
index ae1222a3f..4cca80a80 100644
--- a/docs/src/features.md
+++ b/docs/src/features.md
@@ -17,7 +17,7 @@ Quick links:
# Features
Miller is like awk, sed, cut, join, and sort for **name-indexed data, such as
-CSV, TSV, JSON, and JSON Lines**. You get to work with your data using named
+CSV, TSV, JSON, JSON Lines, and YAML**. You get to work with your data using named
fields, without needing to count positional column indices.
This is something the Unix toolkit always could have done, and arguably always
@@ -25,7 +25,7 @@ should have done. It operates on key-value-pair data while the familiar Unix
tools operate on integer-indexed fields: if the natural data structure for the
latter is the array, then Miller's natural data structure is the
insertion-ordered hash map. This encompasses a **variety of data formats**,
-including but not limited to the familiar CSV, TSV, JSON, and JSON Lines.
+including but not limited to the familiar CSV, TSV, JSON, JSON Lines, and YAML.
(Miller can handle **positionally-indexed data** as a special case.)
* Miller is **multi-purpose**: it's useful for **data cleaning**, **data reduction**, **statistical reporting**, **devops**, **system administration**, **log-file processing**, **format conversion**, and **database-query post-processing**.
@@ -42,7 +42,7 @@ including but not limited to the familiar CSV, TSV, JSON, and JSON Lines.
* Miller is **pipe-friendly** and interoperates with the Unix toolkit
-* Miller's I/O formats include **tabular pretty-printing**, **positionally indexed** (Unix-toolkit style), CSV, JSON, and others
+* Miller's I/O formats include **tabular pretty-printing**, **positionally indexed** (Unix-toolkit style), CSV, JSON, YAML, and others
* Miller does **conversion** between formats
diff --git a/docs/src/features.md.in b/docs/src/features.md.in
index 13ea25bb2..b3d674f67 100644
--- a/docs/src/features.md.in
+++ b/docs/src/features.md.in
@@ -1,7 +1,7 @@
# Features
Miller is like awk, sed, cut, join, and sort for **name-indexed data, such as
-CSV, TSV, JSON, and JSON Lines**. You get to work with your data using named
+CSV, TSV, JSON, JSON Lines, and YAML**. You get to work with your data using named
fields, without needing to count positional column indices.
This is something the Unix toolkit always could have done, and arguably always
@@ -9,7 +9,7 @@ should have done. It operates on key-value-pair data while the familiar Unix
tools operate on integer-indexed fields: if the natural data structure for the
latter is the array, then Miller's natural data structure is the
insertion-ordered hash map. This encompasses a **variety of data formats**,
-including but not limited to the familiar CSV, TSV, JSON, and JSON Lines.
+including but not limited to the familiar CSV, TSV, JSON, JSON Lines, and YAML.
(Miller can handle **positionally-indexed data** as a special case.)
* Miller is **multi-purpose**: it's useful for **data cleaning**, **data reduction**, **statistical reporting**, **devops**, **system administration**, **log-file processing**, **format conversion**, and **database-query post-processing**.
@@ -26,7 +26,7 @@ including but not limited to the familiar CSV, TSV, JSON, and JSON Lines.
* Miller is **pipe-friendly** and interoperates with the Unix toolkit
-* Miller's I/O formats include **tabular pretty-printing**, **positionally indexed** (Unix-toolkit style), CSV, JSON, and others
+* Miller's I/O formats include **tabular pretty-printing**, **positionally indexed** (Unix-toolkit style), CSV, JSON, YAML, and others
* Miller does **conversion** between formats
diff --git a/docs/src/file-formats.md b/docs/src/file-formats.md
index 8a09dac54..c43cd27f7 100644
--- a/docs/src/file-formats.md
+++ b/docs/src/file-formats.md
@@ -17,7 +17,7 @@ Quick links:
# File formats
Miller handles name-indexed data using several formats: some you probably know
-by name, such as CSV, TSV, JSON, and JSON Lines -- and other formats you're likely already
+by name, such as CSV, TSV, JSON, JSON Lines, and YAML -- and other formats you're likely already
seeing and using in your structured data.
Additionally, Miller gives you the option to include comments within your data.
@@ -62,6 +62,16 @@ JSON Lines (sequence of one-line objects):
Record 1: "apple":"1", "bat":"2", "cog":"3"
Record 2: "dish:egg":"7", "dish:flint":"8", "garlic":""
+YAML (single document = object or array of objects; multiple documents with ---):
++---------------------+
+| apple: 1 |
+| bat: 2 | Record 1: "apple":"1", "bat":"2", "cog":"3"
+| cog: 3 |
+| --- |
+| dish: 7 | Record 2: "dish":"7", "egg":"8"
+| egg: 8 |
++---------------------+
+
PPRINT: pretty-printed tabular
+---------------------+
| apple bat cog |
@@ -408,6 +418,18 @@ won't reject your input data for lack of outermost `[...]`, nor will it reject y
of newlines. The difference is on _output_: using `--ojson`, you get outermost `[...]` and pretty-printed
records; using `--ojsonl`, you get no outermost `[...]`, and one line per record.
+## YAML
+
+Miller supports YAML as an I/O format in the same spirit as JSON: input can be a single YAML document
+that is an object (one record) or an array of objects (one record per element), or multiple YAML
+documents separated by `---` (each document is an object or array of objects). Output is either one
+YAML document per record (with `---` between documents) or a single YAML document that is an array
+of all records, controlled by the same kind of list-wrap behavior as JSON.
+
+Use `--yaml`, `--iyaml`, or `--oyaml` (or `-i yaml` / `-o yaml`). Nested structures and types are
+handled like JSON: nested objects become dotted keys when flattened; types are preserved through
+the stream.
+
## PPRINT: Pretty-printed tabular
Miller's pretty-print format is similar to CSV, but with column alignment. For example, compare
@@ -708,20 +730,21 @@ While you can do format conversion using `mlr --icsv --ojson cat myfile.csv`, th
FORMAT-CONVERSION KEYSTROKE-SAVER FLAGS
As keystroke-savers for format-conversion you may use the following.
-The letters c, t, j, l, d, n, x, p, and m refer to formats CSV, TSV, DKVP, NIDX,
-JSON, JSON Lines, XTAB, PPRINT, and markdown, respectively.
+The letters c, t, j, l, d, n, x, p, m, and y refer to formats CSV, TSV, JSON, JSON Lines,
+DKVP, NIDX, XTAB, PPRINT, markdown, and YAML, respectively.
-| In\out | CSV | TSV | JSON | JSONL | DKVP | NIDX | XTAB | PPRINT | Markdown |
-+----------+----------+----------+----------+-------+-------+-------+-------+--------+----------|
-| CSV | --c2c,-c | --c2t | --c2j | --c2l | --c2d | --c2n | --c2x | --c2p | --c2m |
-| TSV | --t2c | --t2t,-t | --t2j | --t2l | --t2d | --t2n | --t2x | --t2p | --t2m |
-| JSON | --j2c | --j2t | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p | --j2m |
-| JSONL | --l2c | --l2t | --l2j | --l2l | --l2d | --l2n | --l2x | --l2p | --l2m |
-| DKVP | --d2c | --d2t | --d2j | --d2l | --d2d | --d2n | --d2x | --d2p | --d2m |
-| NIDX | --n2c | --n2t | --n2j | --n2l | --n2d | --n2n | --n2x | --n2p | --n2m |
-| XTAB | --x2c | --x2t | --x2j | --x2l | --x2d | --x2n | --x2x | --x2p | --x2m |
-| PPRINT | --p2c | --p2t | --p2j | --p2l | --p2d | --p2n | --p2x | -p2p | --p2m |
-| Markdown | --m2c | --m2t | --m2j | --m2l | --m2d | --m2n | --m2x | --m2p | |
+| In\out | CSV | TSV | JSON | JSONL | DKVP | NIDX | XTAB | PPRINT | Markdown | YAML |
++----------+----------+----------+----------+-------+-------+-------+-------+--------+----------+--------+
+| CSV | --c2c,-c | --c2t | --c2j | --c2l | --c2d | --c2n | --c2x | --c2p | --c2m | --c2y |
+| TSV | --t2c | --t2t,-t | --t2j | --t2l | --t2d | --t2n | --t2x | --t2p | --t2m | --t2y |
+| JSON | --j2c | --j2t | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p | --j2m | --j2y |
+| JSONL | --l2c | --l2t | --l2j | --l2l | --l2d | --l2n | --l2x | --l2p | --l2m | --l2y |
+| DKVP | --d2c | --d2t | --d2j | --d2l | --d2d | --d2n | --d2x | --d2p | --d2m | --d2y |
+| NIDX | --n2c | --n2t | --n2j | --n2l | --n2d | --n2n | --n2x | --n2p | --n2m | --n2y |
+| XTAB | --x2c | --x2t | --x2j | --x2l | --x2d | --x2n | --x2x | --x2p | --x2m | --x2y |
+| PPRINT | --p2c | --p2t | --p2j | --p2l | --p2d | --p2n | --p2x | -p2p | --p2m | --p2y |
+| Markdown | --m2c | --m2t | --m2j | --m2l | --m2d | --m2n | --m2x | --m2p | | --m2y |
+| YAML | --y2c | --y2t | --y2j | --y2l | --y2d | --y2n | --y2x | --y2p | --y2m | --y2y |
-p Keystroke-saver for `--nidx --fs space --repifs`.
-T Keystroke-saver for `--nidx --fs tab`.
diff --git a/docs/src/file-formats.md.in b/docs/src/file-formats.md.in
index 2ed581b19..0ac5223eb 100644
--- a/docs/src/file-formats.md.in
+++ b/docs/src/file-formats.md.in
@@ -1,7 +1,7 @@
# File formats
Miller handles name-indexed data using several formats: some you probably know
-by name, such as CSV, TSV, JSON, and JSON Lines -- and other formats you're likely already
+by name, such as CSV, TSV, JSON, JSON Lines, and YAML -- and other formats you're likely already
seeing and using in your structured data.
Additionally, Miller gives you the option to include comments within your data.
@@ -168,6 +168,18 @@ won't reject your input data for lack of outermost `[...]`, nor will it reject y
of newlines. The difference is on _output_: using `--ojson`, you get outermost `[...]` and pretty-printed
records; using `--ojsonl`, you get no outermost `[...]`, and one line per record.
+## YAML
+
+Miller supports YAML as an I/O format in the same spirit as JSON: input can be a single YAML document
+that is an object (one record) or an array of objects (one record per element), or multiple YAML
+documents separated by `---` (each document is an object or array of objects). Output is either one
+YAML document per record (with `---` between documents) or a single YAML document that is an array
+of all records, controlled by the same kind of list-wrap behavior as JSON.
+
+Use `--yaml`, `--iyaml`, or `--oyaml` (or `-i yaml` / `-o yaml`). Nested structures and types are
+handled like JSON: nested objects become dotted keys when flattened; types are preserved through
+the stream.
+
## PPRINT: Pretty-printed tabular
Miller's pretty-print format is similar to CSV, but with column alignment. For example, compare
diff --git a/docs/src/glossary.md b/docs/src/glossary.md
index 774975c41..b9e5a5e22 100644
--- a/docs/src/glossary.md
+++ b/docs/src/glossary.md
@@ -448,6 +448,10 @@ record is an object written on a single line, without need to be wrapped an
outermost list. This format helps people interoperate with non-JSON-aware
tools in the [Unix toolkit](#unix-toolkit) which generally operate on lines.
+## YAML
+
+A human-readable [file format](file-formats.md#yaml) for structured data supported by Miller. Input can be a single document (object or array of objects) or multiple documents separated by `---`. Output can be one top-level array document or one document per record.
+
## key
The string index in a [map](#map). Also, the name of a field in a [record](#record).
@@ -464,7 +468,7 @@ trying to define a [local variable](#local-variable) `if = 3` will result in a p
A subsequence of a text file in between line-ending symbols such as the special linefeed character.
Tools in the [Unix toolkit](#unix-toolkit) generally operate on lines; Miller is designed to do
that (using the [NIDX format flags](file-formats.md#nidx-index-numbered-toolkit-style)), as well
-as non-line-oriented formats such as [CSV, TSV, JSON, and others](file-formats.md).
+as non-line-oriented formats such as [CSV, TSV, JSON, YAML, and others](file-formats.md).
## local variable
diff --git a/docs/src/glossary.md.in b/docs/src/glossary.md.in
index b8eb8f417..e47dc611b 100644
--- a/docs/src/glossary.md.in
+++ b/docs/src/glossary.md.in
@@ -432,6 +432,10 @@ record is an object written on a single line, without need to be wrapped an
outermost list. This format helps people interoperate with non-JSON-aware
tools in the [Unix toolkit](#unix-toolkit) which generally operate on lines.
+## YAML
+
+A human-readable [file format](file-formats.md#yaml) for structured data supported by Miller. Input can be a single document (object or array of objects) or multiple documents separated by `---`. Output can be one top-level array document or one document per record.
+
## key
The string index in a [map](#map). Also, the name of a field in a [record](#record).
@@ -448,7 +452,7 @@ trying to define a [local variable](#local-variable) `if = 3` will result in a p
A subsequence of a text file in between line-ending symbols such as the special linefeed character.
Tools in the [Unix toolkit](#unix-toolkit) generally operate on lines; Miller is designed to do
that (using the [NIDX format flags](file-formats.md#nidx-index-numbered-toolkit-style)), as well
-as non-line-oriented formats such as [CSV, TSV, JSON, and others](file-formats.md).
+as non-line-oriented formats such as [CSV, TSV, JSON, YAML, and others](file-formats.md).
## local variable
diff --git a/docs/src/index.md b/docs/src/index.md
index bcb69c8ed..e7d72c341 100644
--- a/docs/src/index.md
+++ b/docs/src/index.md
@@ -16,7 +16,7 @@ Quick links:
# Introduction
-**Miller is a command-line tool for querying, shaping, and reformatting data files in various formats, including CSV, TSV, JSON, and JSON Lines.**
+**Miller is a command-line tool for querying, shaping, and reformatting data files in various formats, including CSV, TSV, JSON, JSON Lines, and YAML.**
**The big picture:** Even well into the 21st century, our world is full of text-formatted data such as CSV. Google _CSV memes_, for example. We need tooling to _thrive in this world_, nimbly manipulating data which is in CSVs. And we need tooling to _move beyond CSV_, to be able to pull data out and into other storage and processing systems. Miller is designed for both of these goals.
@@ -39,7 +39,7 @@ pretty-print your data horizontally or vertically to make it easier to read.
mlr --json head -n 1 myfile.json
-The `sort`, `head`, etc., are called *verbs*. They're analogs of familiar command-line tools like `sort`, `head`, and so on -- but they're aware of name-indexed, multi-line file formats like CSV, TSV, and JSON. In addition, though, using Miller's `put` verb, you can use programming-language statements for expressions like
+The `sort`, `head`, etc., are called *verbs*. They're analogs of familiar command-line tools like `sort`, `head`, and so on -- but they're aware of name-indexed, multi-line file formats like CSV, TSV, JSON, and YAML. In addition, though, using Miller's `put` verb, you can use programming-language statements for expressions like
mlr --csv put '$rate = $units / $seconds' input.csv
diff --git a/docs/src/index.md.in b/docs/src/index.md.in
index 25073a3f1..6ade85f28 100644
--- a/docs/src/index.md.in
+++ b/docs/src/index.md.in
@@ -1,6 +1,6 @@
# Introduction
-**Miller is a command-line tool for querying, shaping, and reformatting data files in various formats, including CSV, TSV, JSON, and JSON Lines.**
+**Miller is a command-line tool for querying, shaping, and reformatting data files in various formats, including CSV, TSV, JSON, JSON Lines, and YAML.**
**The big picture:** Even well into the 21st century, our world is full of text-formatted data such as CSV. Google _CSV memes_, for example. We need tooling to _thrive in this world_, nimbly manipulating data which is in CSVs. And we need tooling to _move beyond CSV_, to be able to pull data out and into other storage and processing systems. Miller is designed for both of these goals.
@@ -23,7 +23,7 @@ GENMD-SHOW-COMMAND
mlr --json head -n 1 myfile.json
GENMD-EOF
-The `sort`, `head`, etc., are called *verbs*. They're analogs of familiar command-line tools like `sort`, `head`, and so on -- but they're aware of name-indexed, multi-line file formats like CSV, TSV, and JSON. In addition, though, using Miller's `put` verb, you can use programming-language statements for expressions like
+The `sort`, `head`, etc., are called *verbs*. They're analogs of familiar command-line tools like `sort`, `head`, and so on -- but they're aware of name-indexed, multi-line file formats like CSV, TSV, JSON, and YAML. In addition, though, using Miller's `put` verb, you can use programming-language statements for expressions like
GENMD-SHOW-COMMAND
mlr --csv put '$rate = $units / $seconds' input.csv
diff --git a/docs/src/manpage.md b/docs/src/manpage.md
index f14427fe8..b3f893c2d 100644
--- a/docs/src/manpage.md
+++ b/docs/src/manpage.md
@@ -94,6 +94,16 @@ This is simply a copy of what you should see on running `man mlr` at a command p
Record 1: "apple":"1", "bat":"2", "cog":"3"
Record 2: "dish:egg":"7", "dish:flint":"8", "garlic":""
+ YAML (single document = object or array of objects; multiple documents with ---):
+ +---------------------+
+ | apple: 1 |
+ | bat: 2 | Record 1: "apple":"1", "bat":"2", "cog":"3"
+ | cog: 3 |
+ | --- |
+ | dish: 7 | Record 2: "dish":"7", "egg":"8"
+ | egg: 8 |
+ +---------------------+
+
PPRINT: pretty-printed tabular
+---------------------+
| apple bat cog |
@@ -407,6 +417,7 @@ This is simply a copy of what you should see on running `man mlr` at a command p
--itsvlite Use TSV-lite format for input data.
--iusv or --iusvlite Use USV format for input data.
--ixtab Use XTAB format for input data.
+ --iyaml Use YAML format for input data.
--json or -j or --j2j Use JSON format for input and output data.
--jsonl or --l2l Use JSON Lines format for input and output data.
--nidx or --n2n Use NIDX format for input and output data.
@@ -423,12 +434,14 @@ This is simply a copy of what you should see on running `man mlr` at a command p
--otsvlite Use TSV-lite format for output data.
--ousv or --ousvlite Use USV format for output data.
--oxtab Use XTAB format for output data.
+ --oyaml Use YAML format for output data.
--pprint or --p2p Use PPRINT format for input and output data.
--tsv or -t or --t2t Use TSV format for input and output data.
--tsvlite Use TSV-lite format for input and output data.
--usv or --usvlite Use USV format for input and output data.
--xtab or --x2x Use XTAB format for input and output data.
--xvright Right-justify values for XTAB format.
+ --yaml or --y2y Use YAML format for input and output data.
-i {format name} Use format name for input data. For example: `-i csv`
is the same as `--icsv`.
-o {format name} Use format name for output data. For example: `-o
@@ -456,20 +469,21 @@ This is simply a copy of what you should see on running `man mlr` at a command p
1mFORMAT-CONVERSION KEYSTROKE-SAVER FLAGS0m
As keystroke-savers for format-conversion you may use the following.
- The letters c, t, j, l, d, n, x, p, and m refer to formats CSV, TSV, DKVP, NIDX,
- JSON, JSON Lines, XTAB, PPRINT, and markdown, respectively.
+ The letters c, t, j, l, d, n, x, p, m, and y refer to formats CSV, TSV, JSON, JSON Lines,
+ DKVP, NIDX, XTAB, PPRINT, markdown, and YAML, respectively.
- | In\out | CSV | TSV | JSON | JSONL | DKVP | NIDX | XTAB | PPRINT | Markdown |
- +----------+----------+----------+----------+-------+-------+-------+-------+--------+----------|
- | CSV | --c2c,-c | --c2t | --c2j | --c2l | --c2d | --c2n | --c2x | --c2p | --c2m |
- | TSV | --t2c | --t2t,-t | --t2j | --t2l | --t2d | --t2n | --t2x | --t2p | --t2m |
- | JSON | --j2c | --j2t | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p | --j2m |
- | JSONL | --l2c | --l2t | --l2j | --l2l | --l2d | --l2n | --l2x | --l2p | --l2m |
- | DKVP | --d2c | --d2t | --d2j | --d2l | --d2d | --d2n | --d2x | --d2p | --d2m |
- | NIDX | --n2c | --n2t | --n2j | --n2l | --n2d | --n2n | --n2x | --n2p | --n2m |
- | XTAB | --x2c | --x2t | --x2j | --x2l | --x2d | --x2n | --x2x | --x2p | --x2m |
- | PPRINT | --p2c | --p2t | --p2j | --p2l | --p2d | --p2n | --p2x | -p2p | --p2m |
- | Markdown | --m2c | --m2t | --m2j | --m2l | --m2d | --m2n | --m2x | --m2p | |
+ | In\out | CSV | TSV | JSON | JSONL | DKVP | NIDX | XTAB | PPRINT | Markdown | YAML |
+ +----------+----------+----------+----------+-------+-------+-------+-------+--------+----------+--------+
+ | CSV | --c2c,-c | --c2t | --c2j | --c2l | --c2d | --c2n | --c2x | --c2p | --c2m | --c2y |
+ | TSV | --t2c | --t2t,-t | --t2j | --t2l | --t2d | --t2n | --t2x | --t2p | --t2m | --t2y |
+ | JSON | --j2c | --j2t | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p | --j2m | --j2y |
+ | JSONL | --l2c | --l2t | --l2j | --l2l | --l2d | --l2n | --l2x | --l2p | --l2m | --l2y |
+ | DKVP | --d2c | --d2t | --d2j | --d2l | --d2d | --d2n | --d2x | --d2p | --d2m | --d2y |
+ | NIDX | --n2c | --n2t | --n2j | --n2l | --n2d | --n2n | --n2x | --n2p | --n2m | --n2y |
+ | XTAB | --x2c | --x2t | --x2j | --x2l | --x2d | --x2n | --x2x | --x2p | --x2m | --x2y |
+ | PPRINT | --p2c | --p2t | --p2j | --p2l | --p2d | --p2n | --p2x | -p2p | --p2m | --p2y |
+ | Markdown | --m2c | --m2t | --m2j | --m2l | --m2d | --m2n | --m2x | --m2p | | --m2y |
+ | YAML | --y2c | --y2t | --y2j | --y2l | --y2d | --y2n | --y2x | --y2p | --y2m | --y2y |
-p Keystroke-saver for `--nidx --fs space --repifs`.
-T Keystroke-saver for `--nidx --fs tab`.
@@ -488,6 +502,10 @@ This is simply a copy of what you should see on running `man mlr` at a command p
the default for JSON Lines output format.
--no-jvstack Put objects/arrays all on one line for JSON output.
This is the default for JSON Lines output format.
+ --no-yarray Do not wrap YAML output in a single array document;
+ emit one YAML document per record with `---` between.
+ --yarray or --ya Wrap YAML output in a single top-level array
+ document. This is the default for YAML output format.
1mLEGACY FLAGS0m
These are flags which don't do anything in the current Miller version.
@@ -827,6 +845,7 @@ This is simply a copy of what you should see on running `man mlr` at a command p
pprint " " N/A "\n"
tsv " " N/A "\n"
xtab "\n" " " "\n\n"
+ yaml N/A N/A N/A
--fs {string} Specify FS for input and output.
--ifs {string} Specify FS for input.
diff --git a/docs/src/manpage.txt b/docs/src/manpage.txt
index 39ff434e7..31faedd7d 100644
--- a/docs/src/manpage.txt
+++ b/docs/src/manpage.txt
@@ -73,6 +73,16 @@
Record 1: "apple":"1", "bat":"2", "cog":"3"
Record 2: "dish:egg":"7", "dish:flint":"8", "garlic":""
+ YAML (single document = object or array of objects; multiple documents with ---):
+ +---------------------+
+ | apple: 1 |
+ | bat: 2 | Record 1: "apple":"1", "bat":"2", "cog":"3"
+ | cog: 3 |
+ | --- |
+ | dish: 7 | Record 2: "dish":"7", "egg":"8"
+ | egg: 8 |
+ +---------------------+
+
PPRINT: pretty-printed tabular
+---------------------+
| apple bat cog |
@@ -386,6 +396,7 @@
--itsvlite Use TSV-lite format for input data.
--iusv or --iusvlite Use USV format for input data.
--ixtab Use XTAB format for input data.
+ --iyaml Use YAML format for input data.
--json or -j or --j2j Use JSON format for input and output data.
--jsonl or --l2l Use JSON Lines format for input and output data.
--nidx or --n2n Use NIDX format for input and output data.
@@ -402,12 +413,14 @@
--otsvlite Use TSV-lite format for output data.
--ousv or --ousvlite Use USV format for output data.
--oxtab Use XTAB format for output data.
+ --oyaml Use YAML format for output data.
--pprint or --p2p Use PPRINT format for input and output data.
--tsv or -t or --t2t Use TSV format for input and output data.
--tsvlite Use TSV-lite format for input and output data.
--usv or --usvlite Use USV format for input and output data.
--xtab or --x2x Use XTAB format for input and output data.
--xvright Right-justify values for XTAB format.
+ --yaml or --y2y Use YAML format for input and output data.
-i {format name} Use format name for input data. For example: `-i csv`
is the same as `--icsv`.
-o {format name} Use format name for output data. For example: `-o
@@ -435,20 +448,21 @@
1mFORMAT-CONVERSION KEYSTROKE-SAVER FLAGS0m
As keystroke-savers for format-conversion you may use the following.
- The letters c, t, j, l, d, n, x, p, and m refer to formats CSV, TSV, DKVP, NIDX,
- JSON, JSON Lines, XTAB, PPRINT, and markdown, respectively.
+ The letters c, t, j, l, d, n, x, p, m, and y refer to formats CSV, TSV, JSON, JSON Lines,
+ DKVP, NIDX, XTAB, PPRINT, markdown, and YAML, respectively.
- | In\out | CSV | TSV | JSON | JSONL | DKVP | NIDX | XTAB | PPRINT | Markdown |
- +----------+----------+----------+----------+-------+-------+-------+-------+--------+----------|
- | CSV | --c2c,-c | --c2t | --c2j | --c2l | --c2d | --c2n | --c2x | --c2p | --c2m |
- | TSV | --t2c | --t2t,-t | --t2j | --t2l | --t2d | --t2n | --t2x | --t2p | --t2m |
- | JSON | --j2c | --j2t | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p | --j2m |
- | JSONL | --l2c | --l2t | --l2j | --l2l | --l2d | --l2n | --l2x | --l2p | --l2m |
- | DKVP | --d2c | --d2t | --d2j | --d2l | --d2d | --d2n | --d2x | --d2p | --d2m |
- | NIDX | --n2c | --n2t | --n2j | --n2l | --n2d | --n2n | --n2x | --n2p | --n2m |
- | XTAB | --x2c | --x2t | --x2j | --x2l | --x2d | --x2n | --x2x | --x2p | --x2m |
- | PPRINT | --p2c | --p2t | --p2j | --p2l | --p2d | --p2n | --p2x | -p2p | --p2m |
- | Markdown | --m2c | --m2t | --m2j | --m2l | --m2d | --m2n | --m2x | --m2p | |
+ | In\out | CSV | TSV | JSON | JSONL | DKVP | NIDX | XTAB | PPRINT | Markdown | YAML |
+ +----------+----------+----------+----------+-------+-------+-------+-------+--------+----------+--------+
+ | CSV | --c2c,-c | --c2t | --c2j | --c2l | --c2d | --c2n | --c2x | --c2p | --c2m | --c2y |
+ | TSV | --t2c | --t2t,-t | --t2j | --t2l | --t2d | --t2n | --t2x | --t2p | --t2m | --t2y |
+ | JSON | --j2c | --j2t | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p | --j2m | --j2y |
+ | JSONL | --l2c | --l2t | --l2j | --l2l | --l2d | --l2n | --l2x | --l2p | --l2m | --l2y |
+ | DKVP | --d2c | --d2t | --d2j | --d2l | --d2d | --d2n | --d2x | --d2p | --d2m | --d2y |
+ | NIDX | --n2c | --n2t | --n2j | --n2l | --n2d | --n2n | --n2x | --n2p | --n2m | --n2y |
+ | XTAB | --x2c | --x2t | --x2j | --x2l | --x2d | --x2n | --x2x | --x2p | --x2m | --x2y |
+ | PPRINT | --p2c | --p2t | --p2j | --p2l | --p2d | --p2n | --p2x | -p2p | --p2m | --p2y |
+ | Markdown | --m2c | --m2t | --m2j | --m2l | --m2d | --m2n | --m2x | --m2p | | --m2y |
+ | YAML | --y2c | --y2t | --y2j | --y2l | --y2d | --y2n | --y2x | --y2p | --y2m | --y2y |
-p Keystroke-saver for `--nidx --fs space --repifs`.
-T Keystroke-saver for `--nidx --fs tab`.
@@ -467,6 +481,10 @@
the default for JSON Lines output format.
--no-jvstack Put objects/arrays all on one line for JSON output.
This is the default for JSON Lines output format.
+ --no-yarray Do not wrap YAML output in a single array document;
+ emit one YAML document per record with `---` between.
+ --yarray or --ya Wrap YAML output in a single top-level array
+ document. This is the default for YAML output format.
1mLEGACY FLAGS0m
These are flags which don't do anything in the current Miller version.
@@ -806,6 +824,7 @@
pprint " " N/A "\n"
tsv " " N/A "\n"
xtab "\n" " " "\n\n"
+ yaml N/A N/A N/A
--fs {string} Specify FS for input and output.
--ifs {string} Specify FS for input.
diff --git a/docs/src/mk-flag-info.rb b/docs/src/mk-flag-info.rb
index 3f54fe08a..cbe109817 100755
--- a/docs/src/mk-flag-info.rb
+++ b/docs/src/mk-flag-info.rb
@@ -15,19 +15,22 @@ for section_name in section_names
if section_name =~ /.*conversion.*keystroke-saver*/
# The markdown in this section looks a lot better when hand-crafted (thanks Nikos!).
puts <= recordsPerBatch {
+ readerChannel <- recordsAndContexts
+ recordsAndContexts = make([]*types.RecordAndContext, 0, recordsPerBatch)
+ }
+ } else if mlrval.IsArray() {
+ records := mlrval.GetArray()
+ if records == nil {
+ errorChannel <- fmt.Errorf("internal coding error in YAML record-reader")
+ return
+ }
+ for _, mv := range records {
+ if !mv.IsMap() {
+ errorChannel <- fmt.Errorf(
+ "valid but unmillerable YAML: expected map (object); got %s",
+ mv.GetTypeName(),
+ )
+ return
+ }
+ record := mv.GetMap()
+ if record == nil {
+ errorChannel <- fmt.Errorf("internal coding error in YAML record-reader")
+ return
+ }
+ context.UpdateForInputRecord()
+ recordsAndContexts = append(recordsAndContexts, types.NewRecordAndContext(record, context))
+ if int64(len(recordsAndContexts)) >= recordsPerBatch {
+ readerChannel <- recordsAndContexts
+ recordsAndContexts = make([]*types.RecordAndContext, 0, recordsPerBatch)
+ }
+ }
+ } else {
+ errorChannel <- fmt.Errorf(
+ "valid but unmillerable YAML: expected map (object); got %s",
+ mlrval.GetTypeName(),
+ )
+ return
+ }
+ }
+
+flush:
+ if len(recordsAndContexts) > 0 {
+ readerChannel <- recordsAndContexts
+ }
+}
diff --git a/pkg/mlrval/mlrval_yaml.go b/pkg/mlrval/mlrval_yaml.go
new file mode 100644
index 000000000..9410600bb
--- /dev/null
+++ b/pkg/mlrval/mlrval_yaml.go
@@ -0,0 +1,202 @@
+// ================================================================
+// YAML decode/encode for Mlrval and Mlrmap.
+// Converts between YAML native types (from gopkg.in/yaml.v3) and Miller's
+// record model. YAML maps become Mlrmap; keys are stringified (YAML allows
+// non-string keys). Used by the YAML record reader and writer.
+// ================================================================
+
+package mlrval
+
+import (
+ "fmt"
+ "io"
+ "sort"
+
+ "gopkg.in/yaml.v3"
+)
+
+// MlrvalDecodeFromYAML decodes one YAML document from the decoder into an
+// *Mlrval. Returns (nil, true, nil) on EOF. The decoded value can be a map
+// (one record), array (array of records when elements are maps), or scalar.
+func MlrvalDecodeFromYAML(decoder *yaml.Decoder) (*Mlrval, bool, error) {
+ var doc interface{}
+ err := decoder.Decode(&doc)
+ if err == io.EOF {
+ return nil, true, nil
+ }
+ if err != nil {
+ return nil, false, err
+ }
+ if doc == nil {
+ return NULL, false, nil
+ }
+ mv, err := mlrvalFromYAMLNative(doc)
+ if err != nil {
+ return nil, false, err
+ }
+ return mv, false, nil
+}
+
+// mlrvalFromYAMLNative converts a YAML-decoded value (map[interface{}]interface{},
+// []interface{}, or scalar) into *Mlrval.
+func mlrvalFromYAMLNative(v interface{}) (*Mlrval, error) {
+ if v == nil {
+ return NULL, nil
+ }
+ switch val := v.(type) {
+ case map[interface{}]interface{}:
+ return mlrvalFromYAMLMap(val)
+ case map[string]interface{}:
+ return mlrvalFromYAMLStringMap(val)
+ case []interface{}:
+ return mlrvalFromYAMLArray(val)
+ case string:
+ return FromString(val), nil
+ case bool:
+ return FromBool(val), nil
+ case int:
+ return FromInt(int64(val)), nil
+ case int64:
+ return FromInt(val), nil
+ case uint64:
+ if val <= 1<<63-1 {
+ return FromInt(int64(val)), nil
+ }
+ return FromFloat(float64(val)), nil
+ case float64:
+ return FromFloat(val), nil
+ case float32:
+ return FromFloat(float64(val)), nil
+ default:
+ return FromString(fmt.Sprint(val)), nil
+ }
+}
+
+func mlrvalFromYAMLMap(m map[interface{}]interface{}) (*Mlrval, error) {
+ keys := make([]string, 0, len(m))
+ for k := range m {
+ keys = append(keys, yamlKeyString(k))
+ }
+ sort.Strings(keys)
+ out := FromEmptyMap()
+ for _, keyStr := range keys {
+ var v interface{}
+ for k, val := range m {
+ if yamlKeyString(k) == keyStr {
+ v = val
+ break
+ }
+ }
+ valMv, err := mlrvalFromYAMLNative(v)
+ if err != nil {
+ return nil, err
+ }
+ out.MapPut(FromString(keyStr), valMv)
+ }
+ return out, nil
+}
+
+func mlrvalFromYAMLStringMap(m map[string]interface{}) (*Mlrval, error) {
+ keys := make([]string, 0, len(m))
+ for k := range m {
+ keys = append(keys, k)
+ }
+ sort.Strings(keys)
+ out := FromEmptyMap()
+ for _, k := range keys {
+ valMv, err := mlrvalFromYAMLNative(m[k])
+ if err != nil {
+ return nil, err
+ }
+ out.MapPut(FromString(k), valMv)
+ }
+ return out, nil
+}
+
+func yamlKeyString(k interface{}) string {
+ switch t := k.(type) {
+ case string:
+ return t
+ case int:
+ return fmt.Sprintf("%d", t)
+ case int64:
+ return fmt.Sprintf("%d", t)
+ case float64:
+ return fmt.Sprintf("%g", t)
+ default:
+ return fmt.Sprint(k)
+ }
+}
+
+func mlrvalFromYAMLArray(a []interface{}) (*Mlrval, error) {
+ out := FromEmptyArray()
+ for _, elem := range a {
+ mv, err := mlrvalFromYAMLNative(elem)
+ if err != nil {
+ return nil, err
+ }
+ out.ArrayAppend(mv)
+ }
+ return out, nil
+}
+
+// MlrmapToYAMLNative converts an Mlrmap to a Go value suitable for
+// yaml.Marshal: map[string]interface{} with nested maps/arrays.
+func MlrmapToYAMLNative(mlrmap *Mlrmap) (interface{}, error) {
+ if mlrmap == nil {
+ return nil, nil
+ }
+ out := make(map[string]interface{})
+ for pe := mlrmap.Head; pe != nil; pe = pe.Next {
+ v, err := mlrvalToYAMLNative(pe.Value)
+ if err != nil {
+ return nil, err
+ }
+ out[pe.Key] = v
+ }
+ return out, nil
+}
+
+// mlrvalToYAMLNative converts *Mlrval to a Go value for yaml.Marshal.
+func mlrvalToYAMLNative(mv *Mlrval) (interface{}, error) {
+ if mv == nil {
+ return nil, nil
+ }
+ switch mv.Type() {
+ case MT_ABSENT, MT_VOID, MT_NULL:
+ return nil, nil
+ case MT_STRING:
+ s, _ := mv.GetStringValue()
+ return s, nil
+ case MT_INT:
+ i, _ := mv.GetIntValue()
+ return i, nil
+ case MT_FLOAT:
+ f, _ := mv.GetFloatValue()
+ return f, nil
+ case MT_BOOL:
+ b, _ := mv.GetBoolValue()
+ return b, nil
+ case MT_ARRAY:
+ arr := mv.GetArray()
+ if arr == nil {
+ return []interface{}{}, nil
+ }
+ out := make([]interface{}, 0, len(arr))
+ for _, elem := range arr {
+ v, err := mlrvalToYAMLNative(elem)
+ if err != nil {
+ return nil, err
+ }
+ out = append(out, v)
+ }
+ return out, nil
+ case MT_MAP:
+ m := mv.GetMap()
+ return MlrmapToYAMLNative(m)
+ case MT_ERROR, MT_PENDING:
+ return mv.String(), nil
+ default:
+ return mv.String(), nil
+ }
+}
diff --git a/pkg/mlrval/mlrval_yaml_test.go b/pkg/mlrval/mlrval_yaml_test.go
new file mode 100644
index 000000000..0c40d58e3
--- /dev/null
+++ b/pkg/mlrval/mlrval_yaml_test.go
@@ -0,0 +1,196 @@
+// ================================================================
+// Tests for YAML decode/encode (MlrvalDecodeFromYAML, MlrmapToYAMLNative).
+// ================================================================
+
+package mlrval
+
+import (
+ "bytes"
+ "strings"
+ "testing"
+
+ "github.com/stretchr/testify/assert"
+
+ "gopkg.in/yaml.v3"
+)
+
+func TestMlrvalFromYAMLNativeScalars(t *testing.T) {
+ mv, err := mlrvalFromYAMLNative("hello")
+ assert.NoError(t, err)
+ assert.True(t, mv.IsString())
+ s, _ := mv.GetStringValue()
+ assert.Equal(t, "hello", s)
+
+ mv, err = mlrvalFromYAMLNative(42)
+ assert.NoError(t, err)
+ assert.True(t, mv.IsInt())
+ i, _ := mv.GetIntValue()
+ assert.Equal(t, int64(42), i)
+
+ mv, err = mlrvalFromYAMLNative(3.14)
+ assert.NoError(t, err)
+ assert.True(t, mv.IsFloat())
+ f, _ := mv.GetFloatValue()
+ assert.Equal(t, 3.14, f)
+
+ mv, err = mlrvalFromYAMLNative(true)
+ assert.NoError(t, err)
+ assert.True(t, mv.IsBool())
+ b, _ := mv.GetBoolValue()
+ assert.True(t, b)
+
+ mv, err = mlrvalFromYAMLNative(nil)
+ assert.NoError(t, err)
+ assert.True(t, mv.IsNull())
+}
+
+func TestMlrvalFromYAMLNativeMap(t *testing.T) {
+ // gopkg.in/yaml.v3 unmarshals into map[string]interface{}
+ m := map[string]interface{}{
+ "a": 1,
+ "b": "two",
+ "c": true,
+ }
+ mv, err := mlrvalFromYAMLNative(m)
+ assert.NoError(t, err)
+ assert.True(t, mv.IsMap())
+ rec := mv.GetMap()
+ assert.NotNil(t, rec)
+ assert.Equal(t, int64(3), rec.FieldCount)
+ av := rec.Get("a")
+ assert.True(t, av.IsInt())
+ ai, _ := av.GetIntValue()
+ assert.Equal(t, int64(1), ai)
+ bv := rec.Get("b")
+ assert.True(t, bv.IsString())
+ bs, _ := bv.GetStringValue()
+ assert.Equal(t, "two", bs)
+}
+
+func TestMlrvalDecodeFromYAML(t *testing.T) {
+ input := "a: 1\nb: 2\n"
+ decoder := yaml.NewDecoder(bytes.NewBufferString(input))
+ mv, eof, err := MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.False(t, eof)
+ assert.NotNil(t, mv)
+ assert.True(t, mv.IsMap())
+ rec := mv.GetMap()
+ assert.Equal(t, int64(2), rec.FieldCount)
+ v := rec.Get("a")
+ vi, _ := v.GetIntValue()
+ assert.Equal(t, int64(1), vi)
+ v = rec.Get("b")
+ vi, _ = v.GetIntValue()
+ assert.Equal(t, int64(2), vi)
+
+ // EOF on next decode
+ _, eof, err = MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.True(t, eof)
+}
+
+func TestMlrmapToYAMLNativeRoundTrip(t *testing.T) {
+ rec := NewMlrmapAsRecord()
+ rec.PutReference("x", FromInt(10))
+ rec.PutReference("y", FromString("hello"))
+ rec.PutReference("z", FromBool(true))
+
+ native, err := MlrmapToYAMLNative(rec)
+ assert.NoError(t, err)
+ assert.NotNil(t, native)
+
+ out, err := yaml.Marshal(native)
+ assert.NoError(t, err)
+ decoder := yaml.NewDecoder(bytes.NewReader(out))
+ mv, eof, err := MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.False(t, eof)
+ assert.True(t, mv.IsMap())
+ back := mv.GetMap()
+ xv := back.Get("x")
+ xi, _ := xv.GetIntValue()
+ assert.Equal(t, int64(10), xi)
+ yv := back.Get("y")
+ ys, _ := yv.GetStringValue()
+ assert.Equal(t, "hello", ys)
+ zv := back.Get("z")
+ zb, _ := zv.GetBoolValue()
+ assert.True(t, zb)
+}
+
+func TestMlrvalDecodeFromYAMLArrayOfMaps(t *testing.T) {
+ input := "- a: 1\n b: 2\n- a: 3\n b: 4\n"
+ decoder := yaml.NewDecoder(bytes.NewBufferString(input))
+ mv, eof, err := MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.False(t, eof)
+ assert.True(t, mv.IsArray())
+ arr := mv.GetArray()
+ assert.Len(t, arr, 2)
+ assert.True(t, arr[0].IsMap())
+ assert.True(t, arr[1].IsMap())
+ r0 := arr[0].GetMap()
+ av := r0.Get("a")
+ ai, _ := av.GetIntValue()
+ assert.Equal(t, int64(1), ai)
+ r1 := arr[1].GetMap()
+ av = r1.Get("a")
+ ai, _ = av.GetIntValue()
+ assert.Equal(t, int64(3), ai)
+}
+
+func TestMlrvalDecodeFromYAMLMultiDoc(t *testing.T) {
+ input := "a: 1\nb: 2\n---\nc: 3\nd: 4\n"
+ decoder := yaml.NewDecoder(bytes.NewBufferString(input))
+
+ mv1, eof, err := MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.False(t, eof)
+ assert.True(t, mv1.IsMap())
+ assert.Equal(t, int64(2), mv1.GetMap().FieldCount)
+
+ mv2, eof, err := MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.False(t, eof)
+ assert.True(t, mv2.IsMap())
+ assert.Equal(t, int64(2), mv2.GetMap().FieldCount)
+
+ _, eof, err = MlrvalDecodeFromYAML(decoder)
+ assert.NoError(t, err)
+ assert.True(t, eof)
+}
+
+func TestYAMLKeyStringNonStringKeys(t *testing.T) {
+ // map[interface{}]interface{} with int key
+ m := map[interface{}]interface{}{
+ 1: "one",
+ "two": 2,
+ }
+ mv, err := mlrvalFromYAMLMap(m)
+ assert.NoError(t, err)
+ assert.True(t, mv.IsMap())
+ rec := mv.GetMap()
+ v := rec.Get("1")
+ assert.True(t, v.IsString())
+ s, _ := v.GetStringValue()
+ assert.Equal(t, "one", s)
+ v = rec.Get("two")
+ assert.True(t, v.IsInt())
+}
+
+func TestMlrmapToYAMLNativeNested(t *testing.T) {
+ inner := NewMlrmapAsRecord()
+ inner.PutReference("p", FromInt(1))
+ inner.PutReference("q", FromInt(2))
+ rec := NewMlrmapAsRecord()
+ rec.PutReference("a", FromString("x"))
+ rec.PutReference("nested", FromMap(inner))
+
+ native, err := MlrmapToYAMLNative(rec)
+ assert.NoError(t, err)
+ out, err := yaml.Marshal(native)
+ assert.NoError(t, err)
+ assert.True(t, strings.Contains(string(out), "nested:"))
+ assert.True(t, strings.Contains(string(out), "p: 1"))
+}
diff --git a/pkg/output/record_writer_factory.go b/pkg/output/record_writer_factory.go
index bb6aba5fa..c7bb9ad02 100644
--- a/pkg/output/record_writer_factory.go
+++ b/pkg/output/record_writer_factory.go
@@ -18,6 +18,8 @@ func Create(writerOptions *cli.TWriterOptions) (IRecordWriter, error) {
return NewRecordWriterJSON(writerOptions)
case "jsonl":
return NewRecordWriterJSONLines(writerOptions)
+ case "yaml":
+ return NewRecordWriterYAML(writerOptions)
case "md":
return NewRecordWriterMarkdown(writerOptions)
case "markdown":
diff --git a/pkg/output/record_writer_yaml.go b/pkg/output/record_writer_yaml.go
new file mode 100644
index 000000000..4cf8b4d53
--- /dev/null
+++ b/pkg/output/record_writer_yaml.go
@@ -0,0 +1,91 @@
+package output
+
+import (
+ "bufio"
+ "fmt"
+ "os"
+
+ "gopkg.in/yaml.v3"
+
+ "github.com/johnkerl/miller/v6/pkg/cli"
+ "github.com/johnkerl/miller/v6/pkg/mlrval"
+ "github.com/johnkerl/miller/v6/pkg/types"
+)
+
+type RecordWriterYAML struct {
+ writerOptions *cli.TWriterOptions
+ bufferedRecords []interface{} // used when WrapYAMLOutputInOuterList is true
+ wroteAnyRecords bool // for multi-doc: emit "---\n" before 2nd and later docs
+}
+
+func NewRecordWriterYAML(writerOptions *cli.TWriterOptions) (*RecordWriterYAML, error) {
+ return &RecordWriterYAML{
+ writerOptions: writerOptions,
+ bufferedRecords: nil,
+ wroteAnyRecords: false,
+ }, nil
+}
+
+func (writer *RecordWriterYAML) Write(
+ outrec *mlrval.Mlrmap,
+ context *types.Context,
+ bufferedOutputStream *bufio.Writer,
+ outputIsStdout bool,
+) error {
+ if writer.writerOptions.WrapYAMLOutputInOuterList {
+ writer.writeWithListWrap(outrec, bufferedOutputStream)
+ } else {
+ writer.writeWithoutListWrap(outrec, bufferedOutputStream)
+ }
+ return nil
+}
+
+func (writer *RecordWriterYAML) writeWithListWrap(
+ outrec *mlrval.Mlrmap,
+ bufferedOutputStream *bufio.Writer,
+) {
+ if outrec != nil {
+ if writer.bufferedRecords == nil {
+ writer.bufferedRecords = make([]interface{}, 0)
+ }
+ native, err := mlrval.MlrmapToYAMLNative(outrec)
+ if err != nil {
+ fmt.Fprintln(os.Stderr, err)
+ os.Exit(1)
+ }
+ writer.bufferedRecords = append(writer.bufferedRecords, native)
+ } else {
+ // End of stream: emit single YAML document as array
+ out, err := yaml.Marshal(writer.bufferedRecords)
+ if err != nil {
+ fmt.Fprintln(os.Stderr, err)
+ os.Exit(1)
+ }
+ bufferedOutputStream.Write(out)
+ writer.bufferedRecords = nil
+ }
+}
+
+func (writer *RecordWriterYAML) writeWithoutListWrap(
+ outrec *mlrval.Mlrmap,
+ bufferedOutputStream *bufio.Writer,
+) {
+ if outrec == nil {
+ return
+ }
+ if writer.wroteAnyRecords {
+ bufferedOutputStream.WriteString("---\n")
+ }
+ native, err := mlrval.MlrmapToYAMLNative(outrec)
+ if err != nil {
+ fmt.Fprintln(os.Stderr, err)
+ os.Exit(1)
+ }
+ out, err := yaml.Marshal(native)
+ if err != nil {
+ fmt.Fprintln(os.Stderr, err)
+ os.Exit(1)
+ }
+ bufferedOutputStream.Write(out)
+ writer.wroteAnyRecords = true
+}
diff --git a/pkg/terminals/help/entry.go b/pkg/terminals/help/entry.go
index 47d3f6e9d..9d08e9f9c 100644
--- a/pkg/terminals/help/entry.go
+++ b/pkg/terminals/help/entry.go
@@ -401,6 +401,16 @@ JSON Lines (sequence of one-line objects):
Record 1: "apple":"1", "bat":"2", "cog":"3"
Record 2: "dish:egg":"7", "dish:flint":"8", "garlic":""
+YAML (single document = object or array of objects; multiple documents with ---):
++---------------------+
+| apple: 1 |
+| bat: 2 | Record 1: "apple":"1", "bat":"2", "cog":"3"
+| cog: 3 |
+| --- |
+| dish: 7 | Record 2: "dish":"7", "egg":"8"
+| egg: 8 |
++---------------------+
+
PPRINT: pretty-printed tabular
+---------------------+
| apple bat cog |
diff --git a/pkg/terminals/regtest/regtester.go b/pkg/terminals/regtest/regtester.go
index 5fea3a963..4734bf9ce 100644
--- a/pkg/terminals/regtest/regtester.go
+++ b/pkg/terminals/regtest/regtester.go
@@ -60,6 +60,7 @@ import (
"os"
"path/filepath"
"runtime"
+ "sort"
"strings"
"github.com/johnkerl/miller/v6/pkg/colorizer"
@@ -167,6 +168,7 @@ func (regtester *RegTester) Execute(
regtester.resetCounts()
lib.InternalCodingErrorIf(len(casePaths) == 0)
+ sort.Strings(casePaths)
if !regtester.plainMode {
fmt.Println("REGRESSION TEST:")
@@ -342,6 +344,7 @@ func (regtester *RegTester) hasCaseSubdirectories(
fmt.Printf("%s: %v\n", dirName, err)
os.Exit(1)
}
+ sort.Strings(names)
for _, name := range names {
path := dirName + string(filepath.Separator) + name
diff --git a/test/cases/io-format-conversion-keystroke-savers/c2y/cmd b/test/cases/io-format-conversion-keystroke-savers/c2y/cmd
new file mode 100644
index 000000000..a1c2f69a0
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/c2y/cmd
@@ -0,0 +1 @@
+mlr --c2y cat test/input/abixy.csv
diff --git a/test/cases/io-format-conversion-keystroke-savers/c2y/experr b/test/cases/io-format-conversion-keystroke-savers/c2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/c2y/expout b/test/cases/io-format-conversion-keystroke-savers/c2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/c2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/d2y/cmd b/test/cases/io-format-conversion-keystroke-savers/d2y/cmd
new file mode 100644
index 000000000..d1812dde3
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/d2y/cmd
@@ -0,0 +1 @@
+mlr --d2y cat test/input/abixy.dkvp
diff --git a/test/cases/io-format-conversion-keystroke-savers/d2y/experr b/test/cases/io-format-conversion-keystroke-savers/d2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/d2y/expout b/test/cases/io-format-conversion-keystroke-savers/d2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/d2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/j2y/cmd b/test/cases/io-format-conversion-keystroke-savers/j2y/cmd
new file mode 100644
index 000000000..5065a9881
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/j2y/cmd
@@ -0,0 +1 @@
+mlr --j2y cat test/input/abixy.json
diff --git a/test/cases/io-format-conversion-keystroke-savers/j2y/experr b/test/cases/io-format-conversion-keystroke-savers/j2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/j2y/expout b/test/cases/io-format-conversion-keystroke-savers/j2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/j2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/m2y/cmd b/test/cases/io-format-conversion-keystroke-savers/m2y/cmd
new file mode 100644
index 000000000..d9124b2c6
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/m2y/cmd
@@ -0,0 +1 @@
+mlr --m2y cat test/input/abixy.md
diff --git a/test/cases/io-format-conversion-keystroke-savers/m2y/experr b/test/cases/io-format-conversion-keystroke-savers/m2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/m2y/expout b/test/cases/io-format-conversion-keystroke-savers/m2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/m2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/n2y/cmd b/test/cases/io-format-conversion-keystroke-savers/n2y/cmd
new file mode 100644
index 000000000..c20d36ccb
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/n2y/cmd
@@ -0,0 +1 @@
+mlr --n2y cat test/input/abixy.nidx
diff --git a/test/cases/io-format-conversion-keystroke-savers/n2y/experr b/test/cases/io-format-conversion-keystroke-savers/n2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/n2y/expout b/test/cases/io-format-conversion-keystroke-savers/n2y/expout
new file mode 100644
index 000000000..11099d6b9
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/n2y/expout
@@ -0,0 +1,50 @@
+- "1": pan
+ "2": pan
+ "3": 1
+ "4": 0.3467901443380824
+ "5": 0.7268028627434533
+- "1": eks
+ "2": pan
+ "3": 2
+ "4": 0.7586799647899636
+ "5": 0.5221511083334797
+- "1": wye
+ "2": wye
+ "3": 3
+ "4": 0.20460330576630303
+ "5": 0.33831852551664776
+- "1": eks
+ "2": wye
+ "3": 4
+ "4": 0.38139939387114097
+ "5": 0.13418874328430463
+- "1": wye
+ "2": pan
+ "3": 5
+ "4": 0.5732889198020006
+ "5": 0.8636244699032729
+- "1": zee
+ "2": pan
+ "3": 6
+ "4": 0.5271261600918548
+ "5": 0.49322128674835697
+- "1": eks
+ "2": zee
+ "3": 7
+ "4": 0.6117840605678454
+ "5": 0.1878849191181694
+- "1": zee
+ "2": wye
+ "3": 8
+ "4": 0.5985540091064224
+ "5": 0.976181385699006
+- "1": hat
+ "2": wye
+ "3": 9
+ "4": 0.03144187646093577
+ "5": 0.7495507603507059
+- "1": pan
+ "2": wye
+ "3": 10
+ "4": 0.5026260055412137
+ "5": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/p2y/cmd b/test/cases/io-format-conversion-keystroke-savers/p2y/cmd
new file mode 100644
index 000000000..4a1eb07be
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/p2y/cmd
@@ -0,0 +1 @@
+mlr --p2y cat test/input/abixy.pprint
diff --git a/test/cases/io-format-conversion-keystroke-savers/p2y/experr b/test/cases/io-format-conversion-keystroke-savers/p2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/p2y/expout b/test/cases/io-format-conversion-keystroke-savers/p2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/p2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/t2y/cmd b/test/cases/io-format-conversion-keystroke-savers/t2y/cmd
new file mode 100644
index 000000000..736c608d6
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/t2y/cmd
@@ -0,0 +1 @@
+mlr --t2y cat test/input/abixy.tsv
diff --git a/test/cases/io-format-conversion-keystroke-savers/t2y/experr b/test/cases/io-format-conversion-keystroke-savers/t2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/t2y/expout b/test/cases/io-format-conversion-keystroke-savers/t2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/t2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/x2y/cmd b/test/cases/io-format-conversion-keystroke-savers/x2y/cmd
new file mode 100644
index 000000000..b8ee28797
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/x2y/cmd
@@ -0,0 +1 @@
+mlr --x2y cat test/input/abixy.xtab
diff --git a/test/cases/io-format-conversion-keystroke-savers/x2y/experr b/test/cases/io-format-conversion-keystroke-savers/x2y/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/x2y/expout b/test/cases/io-format-conversion-keystroke-savers/x2y/expout
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/x2y/expout
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2c/cmd b/test/cases/io-format-conversion-keystroke-savers/y2c/cmd
new file mode 100644
index 000000000..b61391cd9
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2c/cmd
@@ -0,0 +1 @@
+mlr --y2c cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2c/experr b/test/cases/io-format-conversion-keystroke-savers/y2c/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2c/expout b/test/cases/io-format-conversion-keystroke-savers/y2c/expout
new file mode 100644
index 000000000..37eac50e4
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2c/expout
@@ -0,0 +1,11 @@
+a,b,i,x,y
+pan,pan,1,0.34679014,0.72680286
+eks,pan,2,0.75867996,0.52215111
+wye,wye,3,0.20460331,0.33831853
+eks,wye,4,0.38139939,0.13418874
+wye,pan,5,0.57328892,0.86362447
+zee,pan,6,0.52712616,0.49322129
+eks,zee,7,0.61178406,0.18788492
+zee,wye,8,0.59855401,0.97618139
+hat,wye,9,0.03144188,0.74955076
+pan,wye,10,0.50262601,0.95261836
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2c/input b/test/cases/io-format-conversion-keystroke-savers/y2c/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2c/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2d/cmd b/test/cases/io-format-conversion-keystroke-savers/y2d/cmd
new file mode 100644
index 000000000..0acb4c02f
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2d/cmd
@@ -0,0 +1 @@
+mlr --y2d cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2d/experr b/test/cases/io-format-conversion-keystroke-savers/y2d/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2d/expout b/test/cases/io-format-conversion-keystroke-savers/y2d/expout
new file mode 100644
index 000000000..940df3d11
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2d/expout
@@ -0,0 +1,10 @@
+a=pan,b=pan,i=1,x=0.34679014,y=0.72680286
+a=eks,b=pan,i=2,x=0.75867996,y=0.52215111
+a=wye,b=wye,i=3,x=0.20460331,y=0.33831853
+a=eks,b=wye,i=4,x=0.38139939,y=0.13418874
+a=wye,b=pan,i=5,x=0.57328892,y=0.86362447
+a=zee,b=pan,i=6,x=0.52712616,y=0.49322129
+a=eks,b=zee,i=7,x=0.61178406,y=0.18788492
+a=zee,b=wye,i=8,x=0.59855401,y=0.97618139
+a=hat,b=wye,i=9,x=0.03144188,y=0.74955076
+a=pan,b=wye,i=10,x=0.50262601,y=0.95261836
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2d/input b/test/cases/io-format-conversion-keystroke-savers/y2d/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2d/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2j/cmd b/test/cases/io-format-conversion-keystroke-savers/y2j/cmd
new file mode 100644
index 000000000..09556834a
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2j/cmd
@@ -0,0 +1 @@
+mlr --y2j cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2j/experr b/test/cases/io-format-conversion-keystroke-savers/y2j/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2j/expout b/test/cases/io-format-conversion-keystroke-savers/y2j/expout
new file mode 100644
index 000000000..369f76808
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2j/expout
@@ -0,0 +1,10 @@
+[
+{
+ "a": 1,
+ "b": 2
+},
+{
+ "a": 3,
+ "b": 4
+}
+]
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2j/input b/test/cases/io-format-conversion-keystroke-savers/y2j/input
new file mode 100644
index 000000000..33d41a2a0
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2j/input
@@ -0,0 +1,5 @@
+a: 1
+b: 2
+---
+a: 3
+b: 4
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2m/cmd b/test/cases/io-format-conversion-keystroke-savers/y2m/cmd
new file mode 100644
index 000000000..6ffb0c6fe
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2m/cmd
@@ -0,0 +1 @@
+mlr --y2m cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2m/experr b/test/cases/io-format-conversion-keystroke-savers/y2m/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2m/expout b/test/cases/io-format-conversion-keystroke-savers/y2m/expout
new file mode 100644
index 000000000..3d77a0324
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2m/expout
@@ -0,0 +1,12 @@
+| a | b | i | x | y |
+| --- | --- | --- | --- | --- |
+| pan | pan | 1 | 0.34679014 | 0.72680286 |
+| eks | pan | 2 | 0.75867996 | 0.52215111 |
+| wye | wye | 3 | 0.20460331 | 0.33831853 |
+| eks | wye | 4 | 0.38139939 | 0.13418874 |
+| wye | pan | 5 | 0.57328892 | 0.86362447 |
+| zee | pan | 6 | 0.52712616 | 0.49322129 |
+| eks | zee | 7 | 0.61178406 | 0.18788492 |
+| zee | wye | 8 | 0.59855401 | 0.97618139 |
+| hat | wye | 9 | 0.03144188 | 0.74955076 |
+| pan | wye | 10 | 0.50262601 | 0.95261836 |
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2m/input b/test/cases/io-format-conversion-keystroke-savers/y2m/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2m/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2n/cmd b/test/cases/io-format-conversion-keystroke-savers/y2n/cmd
new file mode 100644
index 000000000..5decfdbd6
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2n/cmd
@@ -0,0 +1 @@
+mlr --y2n cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2n/experr b/test/cases/io-format-conversion-keystroke-savers/y2n/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2n/expout b/test/cases/io-format-conversion-keystroke-savers/y2n/expout
new file mode 100644
index 000000000..17f7e1ee6
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2n/expout
@@ -0,0 +1,10 @@
+pan pan 1 0.34679014 0.72680286
+eks pan 2 0.75867996 0.52215111
+wye wye 3 0.20460331 0.33831853
+eks wye 4 0.38139939 0.13418874
+wye pan 5 0.57328892 0.86362447
+zee pan 6 0.52712616 0.49322129
+eks zee 7 0.61178406 0.18788492
+zee wye 8 0.59855401 0.97618139
+hat wye 9 0.03144188 0.74955076
+pan wye 10 0.50262601 0.95261836
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2n/input b/test/cases/io-format-conversion-keystroke-savers/y2n/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2n/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2p/cmd b/test/cases/io-format-conversion-keystroke-savers/y2p/cmd
new file mode 100644
index 000000000..be9a17d42
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2p/cmd
@@ -0,0 +1 @@
+mlr --y2p cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2p/experr b/test/cases/io-format-conversion-keystroke-savers/y2p/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2p/expout b/test/cases/io-format-conversion-keystroke-savers/y2p/expout
new file mode 100644
index 000000000..b8ac13481
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2p/expout
@@ -0,0 +1,11 @@
+a b i x y
+pan pan 1 0.34679014 0.72680286
+eks pan 2 0.75867996 0.52215111
+wye wye 3 0.20460331 0.33831853
+eks wye 4 0.38139939 0.13418874
+wye pan 5 0.57328892 0.86362447
+zee pan 6 0.52712616 0.49322129
+eks zee 7 0.61178406 0.18788492
+zee wye 8 0.59855401 0.97618139
+hat wye 9 0.03144188 0.74955076
+pan wye 10 0.50262601 0.95261836
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2p/input b/test/cases/io-format-conversion-keystroke-savers/y2p/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2p/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2t/cmd b/test/cases/io-format-conversion-keystroke-savers/y2t/cmd
new file mode 100644
index 000000000..ab2433ef4
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2t/cmd
@@ -0,0 +1 @@
+mlr --y2t cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2t/experr b/test/cases/io-format-conversion-keystroke-savers/y2t/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2t/expout b/test/cases/io-format-conversion-keystroke-savers/y2t/expout
new file mode 100644
index 000000000..03ac8f384
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2t/expout
@@ -0,0 +1,11 @@
+a b i x y
+pan pan 1 0.34679014 0.72680286
+eks pan 2 0.75867996 0.52215111
+wye wye 3 0.20460331 0.33831853
+eks wye 4 0.38139939 0.13418874
+wye pan 5 0.57328892 0.86362447
+zee pan 6 0.52712616 0.49322129
+eks zee 7 0.61178406 0.18788492
+zee wye 8 0.59855401 0.97618139
+hat wye 9 0.03144188 0.74955076
+pan wye 10 0.50262601 0.95261836
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2t/input b/test/cases/io-format-conversion-keystroke-savers/y2t/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2t/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2x/cmd b/test/cases/io-format-conversion-keystroke-savers/y2x/cmd
new file mode 100644
index 000000000..432efdd8b
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2x/cmd
@@ -0,0 +1 @@
+mlr --y2x cat ${CASEDIR}/input
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2x/experr b/test/cases/io-format-conversion-keystroke-savers/y2x/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2x/expout b/test/cases/io-format-conversion-keystroke-savers/y2x/expout
new file mode 100644
index 000000000..9c955fb87
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2x/expout
@@ -0,0 +1,59 @@
+a pan
+b pan
+i 1
+x 0.34679014
+y 0.72680286
+
+a eks
+b pan
+i 2
+x 0.75867996
+y 0.52215111
+
+a wye
+b wye
+i 3
+x 0.20460331
+y 0.33831853
+
+a eks
+b wye
+i 4
+x 0.38139939
+y 0.13418874
+
+a wye
+b pan
+i 5
+x 0.57328892
+y 0.86362447
+
+a zee
+b pan
+i 6
+x 0.52712616
+y 0.49322129
+
+a eks
+b zee
+i 7
+x 0.61178406
+y 0.18788492
+
+a zee
+b wye
+i 8
+x 0.59855401
+y 0.97618139
+
+a hat
+b wye
+i 9
+x 0.03144188
+y 0.74955076
+
+a pan
+b wye
+i 10
+x 0.50262601
+y 0.95261836
diff --git a/test/cases/io-format-conversion-keystroke-savers/y2x/input b/test/cases/io-format-conversion-keystroke-savers/y2x/input
new file mode 100644
index 000000000..0fa69f851
--- /dev/null
+++ b/test/cases/io-format-conversion-keystroke-savers/y2x/input
@@ -0,0 +1,50 @@
+- a: pan
+ b: pan
+ i: 1
+ x: 0.3467901443380824
+ "y": 0.7268028627434533
+- a: eks
+ b: pan
+ i: 2
+ x: 0.7586799647899636
+ "y": 0.5221511083334797
+- a: wye
+ b: wye
+ i: 3
+ x: 0.20460330576630303
+ "y": 0.33831852551664776
+- a: eks
+ b: wye
+ i: 4
+ x: 0.38139939387114097
+ "y": 0.13418874328430463
+- a: wye
+ b: pan
+ i: 5
+ x: 0.5732889198020006
+ "y": 0.8636244699032729
+- a: zee
+ b: pan
+ i: 6
+ x: 0.5271261600918548
+ "y": 0.49322128674835697
+- a: eks
+ b: zee
+ i: 7
+ x: 0.6117840605678454
+ "y": 0.1878849191181694
+- a: zee
+ b: wye
+ i: 8
+ x: 0.5985540091064224
+ "y": 0.976181385699006
+- a: hat
+ b: wye
+ i: 9
+ x: 0.03144187646093577
+ "y": 0.7495507603507059
+- a: pan
+ b: wye
+ i: 10
+ x: 0.5026260055412137
+ "y": 0.9526183602969864
diff --git a/test/cases/io-yaml-io/0001/cmd b/test/cases/io-yaml-io/0001/cmd
new file mode 100644
index 000000000..6fd32a30b
--- /dev/null
+++ b/test/cases/io-yaml-io/0001/cmd
@@ -0,0 +1 @@
+mlr --iyaml --ocsv cat ${CASEDIR}/input
diff --git a/test/cases/io-yaml-io/0001/experr b/test/cases/io-yaml-io/0001/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-yaml-io/0001/expout b/test/cases/io-yaml-io/0001/expout
new file mode 100644
index 000000000..de7e0001c
--- /dev/null
+++ b/test/cases/io-yaml-io/0001/expout
@@ -0,0 +1,3 @@
+a,b,c
+1,2,hello
+10,20,world
diff --git a/test/cases/io-yaml-io/0001/input b/test/cases/io-yaml-io/0001/input
new file mode 100644
index 000000000..5d7c68867
--- /dev/null
+++ b/test/cases/io-yaml-io/0001/input
@@ -0,0 +1,7 @@
+a: 1
+b: 2
+c: hello
+---
+a: 10
+b: 20
+c: world
diff --git a/test/cases/io-yaml-io/0002/cmd b/test/cases/io-yaml-io/0002/cmd
new file mode 100644
index 000000000..31cb8b6be
--- /dev/null
+++ b/test/cases/io-yaml-io/0002/cmd
@@ -0,0 +1 @@
+mlr --iyaml --oyaml --no-yarray cat ${CASEDIR}/input
diff --git a/test/cases/io-yaml-io/0002/experr b/test/cases/io-yaml-io/0002/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-yaml-io/0002/expout b/test/cases/io-yaml-io/0002/expout
new file mode 100644
index 000000000..33d41a2a0
--- /dev/null
+++ b/test/cases/io-yaml-io/0002/expout
@@ -0,0 +1,5 @@
+a: 1
+b: 2
+---
+a: 3
+b: 4
diff --git a/test/cases/io-yaml-io/0002/input b/test/cases/io-yaml-io/0002/input
new file mode 100644
index 000000000..33d41a2a0
--- /dev/null
+++ b/test/cases/io-yaml-io/0002/input
@@ -0,0 +1,5 @@
+a: 1
+b: 2
+---
+a: 3
+b: 4