diff --git a/docs/src/flatten-unflatten.md b/docs/src/flatten-unflatten.md
index c0cb6fe6b..d876dea09 100644
--- a/docs/src/flatten-unflatten.md
+++ b/docs/src/flatten-unflatten.md
@@ -14,11 +14,11 @@ Quick links:
Release docs
-# Flatten/unflatten: converting between JSON and tabular formats
+# Flatten/unflatten: converting between JSON/YAML and tabular formats
Miller has long supported reading and writing multiple [file
-formats](file-formats.md) including CSV and JSON, as well as converting back
-and forth between them. Two things new in [Miller 6](new-in-miller-6-md),
+formats](file-formats.md) including CSV, JSON, and YAML, as well as converting
+back and forth between them. Two things new in [Miller 6](new-in-miller-6-md),
though, are that [arrays are now fully supported](reference-main-arrays.md),
and that [record values are typed](new-in-miller-6.md#improved-numeric-conversion)
throughout Miller's processing chain from input through [verbs](reference-verbs.md)
@@ -26,9 +26,10 @@ to output -- which includes improved handling for [maps](reference-main-maps.md)
[arrays](reference-main-arrays.md) as record values.
This raises the question, though, of how to handle maps and arrays as record values.
-For [JSON files](file-formats.md#json), this is easy -- JSON is a nested format where values
-can be maps or arrays, which can contain other maps or arrays, and so on, with the nesting
-happily indicated by curly braces:
+For [JSON](file-formats.md#json) or [YAML](file-formats.md#yaml) files, this is easy --
+both are nested formats where values can be maps or arrays, which can contain other maps
+or arrays, and so on, with the nesting happily indicated by curly braces (JSON) or
+indentation (YAML):
cat data/map-values.json
@@ -60,11 +61,11 @@ happily indicated by curly braces:
How can we represent these in CSV files?
-Miller's [non-JSON formats](file-formats.md), such as CSV, are all non-nested -- a
-cell in a CSV row can't contain another entire row. As we'll see in this
-section, there are two main ways to **flatten** nested data structures down to
-individual CSV cells -- either by _key-spreading_ (which is the default), or by
-_JSON-stringifying_:
+Miller's non-JSON/YAML [file formats](file-formats.md), such as CSV, are all
+non-nested -- a cell in a CSV row can't contain another entire row. As we'll
+see in this section, there are two main ways to **flatten** nested data
+structures down to individual CSV cells -- either by _key-spreading_ (which
+is the default), or by _JSON-stringifying_:
* **Key-spreading** is when the single map-valued field
`b={"x": 2, "y": 3}` spreads into multiple fields `b.x=2,b.y=3`;
@@ -73,7 +74,7 @@ _JSON-stringifying_:
Miller intends to provide intuitive default behavior for these conversions, while also
providing you with more control when you need it.
-## Converting maps between JSON and non-JSON
+## Converting maps between JSON/YAML and non-JSON/YAML
Let's first look at the default behavior with map-valued fields. Miller's
default behavior is to spread the map values into multiple keys -- using
@@ -152,7 +153,7 @@ a b.s.w b.s.x b.t.y b.t.z
6 7 8 9 10
-**Unflattening** is simply the reverse -- from non-JSON back to JSON:
+**Unflattening** is simply the reverse -- from non-JSON/YAML back to JSON or YAML:
cat data/map-values.json
@@ -199,7 +200,7 @@ a,b.x,b.y
]
-## Converting arrays between JSON and non-JSON
+## Converting arrays between JSON/YAML and non-JSON/YAML
If the input data contains arrays, these are also flattened similarly: the
[1-up array indices](reference-main-arrays.md#1-up-indexing) `1,2,3,...` become string keys
@@ -257,7 +258,7 @@ In the nested-data examples shown here, nested map values are shown containing
maps, and nested array values are shown containing arrays -- of course (even
though not shown here) nested map values can contain arrays, and vice versa.
-**Unflattening** arrays is, again, simply the reverse -- from non-JSON back to JSON:
+**Unflattening** arrays is, again, simply the reverse -- from non-JSON/YAML back to JSON or YAML:
cat data/array-values.json
@@ -352,7 +353,7 @@ a.1,a.3,a.5
An additional heuristic is that if a field name starts with a `.`, ends with
a `.`, or has two or more consecutive `.` characters, no attempt is made
-to unflatten it on conversion from non-JSON to JSON.
+to unflatten it on conversion from non-JSON/YAML to JSON or YAML.
cat data/flatten-dots.csv
@@ -403,13 +404,13 @@ unflattening (if the defaults aren't working for us in a particular situation),
let's first look a little into how they're implemented.
* There are two [verbs](reference-verbs.md) called [flatten](reference-verbs.md#flatten) and [unflatten](reference-verbs.md#unflatten).
-* When the output format is not JSON, if you've specified `mlr ... cat then sort ...` (some [chain](reference-main-then-chaining.md) of verbs) then Miller appends, in effect, `then flatten` to the end of the chain.
+* When the output format is not JSON or YAML, if you've specified `mlr ... cat then sort ...` (some [chain](reference-main-then-chaining.md) of verbs) then Miller appends, in effect, `then flatten` to the end of the chain.
* This behavior is on by default but it can be suppressed using the `--no-auto-flatten` [flag](reference-main-flag-list.md#flatten-unflatten-flags).
-* When the output format is JSON and the input format is not JSON, then (similarly) appends, in effect, `then unflatten` to the end of the chain.
+* When the output format is JSON or YAML and the input format is neither, then (similarly) Miller appends, in effect, `then unflatten` to the end of the chain.
* This behavior is on by default but it can be suppressed using the `--no-auto-unflatten` [flag](reference-main-flag-list.md#flatten-unflatten-flags).
Note in particular that auto-flatten happens even when the input format and the
-output format are both non-JSON, e.g. even for CSV-to-CSV processing. This is
+output format are both non-JSON/non-YAML, e.g. even for CSV-to-CSV processing. This is
because
[map](reference-main-maps.md)-valued/[array](reference-main-arrays.md)-valued
fields can be produced using [DSL statements](miller-programming-language.md):
diff --git a/docs/src/flatten-unflatten.md.in b/docs/src/flatten-unflatten.md.in
index 951ea1f58..f61d64aa3 100644
--- a/docs/src/flatten-unflatten.md.in
+++ b/docs/src/flatten-unflatten.md.in
@@ -1,8 +1,8 @@
-# Flatten/unflatten: converting between JSON and tabular formats
+# Flatten/unflatten: converting between JSON/YAML and tabular formats
Miller has long supported reading and writing multiple [file
-formats](file-formats.md) including CSV and JSON, as well as converting back
-and forth between them. Two things new in [Miller 6](new-in-miller-6-md),
+formats](file-formats.md) including CSV, JSON, and YAML, as well as converting
+back and forth between them. Two things new in [Miller 6](new-in-miller-6-md),
though, are that [arrays are now fully supported](reference-main-arrays.md),
and that [record values are typed](new-in-miller-6.md#improved-numeric-conversion)
throughout Miller's processing chain from input through [verbs](reference-verbs.md)
@@ -10,9 +10,10 @@ to output -- which includes improved handling for [maps](reference-main-maps.md)
[arrays](reference-main-arrays.md) as record values.
This raises the question, though, of how to handle maps and arrays as record values.
-For [JSON files](file-formats.md#json), this is easy -- JSON is a nested format where values
-can be maps or arrays, which can contain other maps or arrays, and so on, with the nesting
-happily indicated by curly braces:
+For [JSON](file-formats.md#json) or [YAML](file-formats.md#yaml) files, this is easy --
+both are nested formats where values can be maps or arrays, which can contain other maps
+or arrays, and so on, with the nesting happily indicated by curly braces (JSON) or
+indentation (YAML):
GENMD-RUN-COMMAND
cat data/map-values.json
@@ -24,11 +25,11 @@ GENMD-EOF
How can we represent these in CSV files?
-Miller's [non-JSON formats](file-formats.md), such as CSV, are all non-nested -- a
-cell in a CSV row can't contain another entire row. As we'll see in this
-section, there are two main ways to **flatten** nested data structures down to
-individual CSV cells -- either by _key-spreading_ (which is the default), or by
-_JSON-stringifying_:
+Miller's non-JSON/YAML [file formats](file-formats.md), such as CSV, are all
+non-nested -- a cell in a CSV row can't contain another entire row. As we'll
+see in this section, there are two main ways to **flatten** nested data
+structures down to individual CSV cells -- either by _key-spreading_ (which
+is the default), or by _JSON-stringifying_:
* **Key-spreading** is when the single map-valued field
`b={"x": 2, "y": 3}` spreads into multiple fields `b.x=2,b.y=3`;
@@ -37,7 +38,7 @@ _JSON-stringifying_:
Miller intends to provide intuitive default behavior for these conversions, while also
providing you with more control when you need it.
-## Converting maps between JSON and non-JSON
+## Converting maps between JSON/YAML and non-JSON/YAML
Let's first look at the default behavior with map-valued fields. Miller's
default behavior is to spread the map values into multiple keys -- using
@@ -76,7 +77,7 @@ GENMD-RUN-COMMAND
mlr --ijson --opprint cat data/map-values-nested.json
GENMD-EOF
-**Unflattening** is simply the reverse -- from non-JSON back to JSON:
+**Unflattening** is simply the reverse -- from non-JSON/YAML back to JSON or YAML:
GENMD-RUN-COMMAND
cat data/map-values.json
@@ -90,7 +91,7 @@ GENMD-RUN-COMMAND
mlr --ijson --ocsv cat data/map-values.json | mlr --icsv --ojson cat
GENMD-EOF
-## Converting arrays between JSON and non-JSON
+## Converting arrays between JSON/YAML and non-JSON/YAML
If the input data contains arrays, these are also flattened similarly: the
[1-up array indices](reference-main-arrays.md#1-up-indexing) `1,2,3,...` become string keys
@@ -118,7 +119,7 @@ In the nested-data examples shown here, nested map values are shown containing
maps, and nested array values are shown containing arrays -- of course (even
though not shown here) nested map values can contain arrays, and vice versa.
-**Unflattening** arrays is, again, simply the reverse -- from non-JSON back to JSON:
+**Unflattening** arrays is, again, simply the reverse -- from non-JSON/YAML back to JSON or YAML:
GENMD-RUN-COMMAND
cat data/array-values.json
@@ -160,7 +161,7 @@ GENMD-EOF
An additional heuristic is that if a field name starts with a `.`, ends with
a `.`, or has two or more consecutive `.` characters, no attempt is made
-to unflatten it on conversion from non-JSON to JSON.
+to unflatten it on conversion from non-JSON/YAML to JSON or YAML.
GENMD-RUN-COMMAND
cat data/flatten-dots.csv
@@ -181,13 +182,13 @@ unflattening (if the defaults aren't working for us in a particular situation),
let's first look a little into how they're implemented.
* There are two [verbs](reference-verbs.md) called [flatten](reference-verbs.md#flatten) and [unflatten](reference-verbs.md#unflatten).
-* When the output format is not JSON, if you've specified `mlr ... cat then sort ...` (some [chain](reference-main-then-chaining.md) of verbs) then Miller appends, in effect, `then flatten` to the end of the chain.
+* When the output format is not JSON or YAML, if you've specified `mlr ... cat then sort ...` (some [chain](reference-main-then-chaining.md) of verbs) then Miller appends, in effect, `then flatten` to the end of the chain.
* This behavior is on by default but it can be suppressed using the `--no-auto-flatten` [flag](reference-main-flag-list.md#flatten-unflatten-flags).
-* When the output format is JSON and the input format is not JSON, then (similarly) appends, in effect, `then unflatten` to the end of the chain.
+* When the output format is JSON or YAML and the input format is neither, then (similarly) Miller appends, in effect, `then unflatten` to the end of the chain.
* This behavior is on by default but it can be suppressed using the `--no-auto-unflatten` [flag](reference-main-flag-list.md#flatten-unflatten-flags).
Note in particular that auto-flatten happens even when the input format and the
-output format are both non-JSON, e.g. even for CSV-to-CSV processing. This is
+output format are both non-JSON/non-YAML, e.g. even for CSV-to-CSV processing. This is
because
[map](reference-main-maps.md)-valued/[array](reference-main-arrays.md)-valued
fields can be produced using [DSL statements](miller-programming-language.md):
diff --git a/docs/src/manpage.md b/docs/src/manpage.md
index 5c216d509..0b643ea59 100644
--- a/docs/src/manpage.md
+++ b/docs/src/manpage.md
@@ -479,7 +479,7 @@ This is simply a copy of what you should see on running `man mlr` at a command p
csv` is the same as `--ocsv`.
1mFLATTEN-UNFLATTEN FLAGS0m
- These flags control how Miller converts record values which are maps or arrays, when input is JSON and output is non-JSON (flattening) or input is non-JSON and output is JSON (unflattening).
+ These flags control how Miller converts record values which are maps or arrays, when input is JSON/YAML and output is not (flattening) or input is not JSON/YAML and output is JSON/YAML (unflattening).
See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatten-unflatten for more information.
@@ -487,16 +487,17 @@ This is simply a copy of what you should see on running `man mlr` at a command p
Separator for flattening multi-level JSON keys, e.g.
`{"a":{"b":3}}` becomes `a:b => 3` for non-JSON
formats. Defaults to `.`.
- --no-auto-flatten When output is non-JSON, suppress the default
+ --no-auto-flatten When output is not JSON or YAML, suppress the default
auto-flatten behavior. Default: if `$y = [7,8,9]`
then this flattens to `y.1=7,y.2=8,y.3=9`, and
similarly for maps. With `--no-auto-flatten`, instead
we get `$y=[1, 2, 3]`.
- --no-auto-unflatten When input is non-JSON and output is JSON, suppress
- the default auto-unflatten behavior. Default: if the
- input has `y.1=7,y.2=8,y.3=9` then this unflattens to
- `$y=[7,8,9]`. With `--no-auto-flatten`, instead we
- get `${y.1}=7,${y.2}=8,${y.3}=9`.
+ --no-auto-unflatten When input is not JSON or YAML and output is JSON or
+ YAML, suppress the default auto-unflatten behavior.
+ Default: if the input has `y.1=7,y.2=8,y.3=9` then
+ this unflattens to `$y=[7,8,9]`. With
+ `--no-auto-flatten`, instead we get
+ `${y.1}=7,${y.2}=8,${y.3}=9`.
1mFORMAT-CONVERSION KEYSTROKE-SAVER FLAGS0m
As keystroke-savers for format-conversion you may use the following.
@@ -4159,5 +4160,5 @@ This is simply a copy of what you should see on running `man mlr` at a command p
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
https://miller.readthedocs.io
- 2026-07-14 4mMILLER24m(1)
+ 2026-07-15 4mMILLER24m(1)
diff --git a/docs/src/manpage.txt b/docs/src/manpage.txt
index b235fea97..5cde2e3fb 100644
--- a/docs/src/manpage.txt
+++ b/docs/src/manpage.txt
@@ -458,7 +458,7 @@
csv` is the same as `--ocsv`.
1mFLATTEN-UNFLATTEN FLAGS0m
- These flags control how Miller converts record values which are maps or arrays, when input is JSON and output is non-JSON (flattening) or input is non-JSON and output is JSON (unflattening).
+ These flags control how Miller converts record values which are maps or arrays, when input is JSON/YAML and output is not (flattening) or input is not JSON/YAML and output is JSON/YAML (unflattening).
See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatten-unflatten for more information.
@@ -466,16 +466,17 @@
Separator for flattening multi-level JSON keys, e.g.
`{"a":{"b":3}}` becomes `a:b => 3` for non-JSON
formats. Defaults to `.`.
- --no-auto-flatten When output is non-JSON, suppress the default
+ --no-auto-flatten When output is not JSON or YAML, suppress the default
auto-flatten behavior. Default: if `$y = [7,8,9]`
then this flattens to `y.1=7,y.2=8,y.3=9`, and
similarly for maps. With `--no-auto-flatten`, instead
we get `$y=[1, 2, 3]`.
- --no-auto-unflatten When input is non-JSON and output is JSON, suppress
- the default auto-unflatten behavior. Default: if the
- input has `y.1=7,y.2=8,y.3=9` then this unflattens to
- `$y=[7,8,9]`. With `--no-auto-flatten`, instead we
- get `${y.1}=7,${y.2}=8,${y.3}=9`.
+ --no-auto-unflatten When input is not JSON or YAML and output is JSON or
+ YAML, suppress the default auto-unflatten behavior.
+ Default: if the input has `y.1=7,y.2=8,y.3=9` then
+ this unflattens to `$y=[7,8,9]`. With
+ `--no-auto-flatten`, instead we get
+ `${y.1}=7,${y.2}=8,${y.3}=9`.
1mFORMAT-CONVERSION KEYSTROKE-SAVER FLAGS0m
As keystroke-savers for format-conversion you may use the following.
@@ -4138,4 +4139,4 @@
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
https://miller.readthedocs.io
- 2026-07-14 4mMILLER24m(1)
+ 2026-07-15 4mMILLER24m(1)
diff --git a/docs/src/reference-main-flag-list.md b/docs/src/reference-main-flag-list.md
index d9b09cf0a..17b0de7df 100644
--- a/docs/src/reference-main-flag-list.md
+++ b/docs/src/reference-main-flag-list.md
@@ -210,7 +210,7 @@ are overridden in all cases by setting output format to `format2`.
## Flatten-unflatten flags
-These flags control how Miller converts record values which are maps or arrays, when input is JSON and output is non-JSON (flattening) or input is non-JSON and output is JSON (unflattening).
+These flags control how Miller converts record values which are maps or arrays, when input is JSON/YAML and output is not (flattening) or input is not JSON/YAML and output is JSON/YAML (unflattening).
See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatten-unflatten for more information.
@@ -218,8 +218,8 @@ See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatt
**Flags:**
* `--flatsep or --jflatsep {string}`: Separator for flattening multi-level JSON keys, e.g. `{"a":{"b":3}}` becomes `a:b => 3` for non-JSON formats. Defaults to `.`.
-* `--no-auto-flatten`: When output is non-JSON, suppress the default auto-flatten behavior. Default: if `$y = [7,8,9]` then this flattens to `y.1=7,y.2=8,y.3=9`, and similarly for maps. With `--no-auto-flatten`, instead we get `$y=[1, 2, 3]`.
-* `--no-auto-unflatten`: When input is non-JSON and output is JSON, suppress the default auto-unflatten behavior. Default: if the input has `y.1=7,y.2=8,y.3=9` then this unflattens to `$y=[7,8,9]`. With `--no-auto-flatten`, instead we get `${y.1}=7,${y.2}=8,${y.3}=9`.
+* `--no-auto-flatten`: When output is not JSON or YAML, suppress the default auto-flatten behavior. Default: if `$y = [7,8,9]` then this flattens to `y.1=7,y.2=8,y.3=9`, and similarly for maps. With `--no-auto-flatten`, instead we get `$y=[1, 2, 3]`.
+* `--no-auto-unflatten`: When input is not JSON or YAML and output is JSON or YAML, suppress the default auto-unflatten behavior. Default: if the input has `y.1=7,y.2=8,y.3=9` then this unflattens to `$y=[7,8,9]`. With `--no-auto-flatten`, instead we get `${y.1}=7,${y.2}=8,${y.3}=9`.
## Format-conversion keystroke-saver flags
diff --git a/man/manpage.txt b/man/manpage.txt
index b235fea97..5cde2e3fb 100644
--- a/man/manpage.txt
+++ b/man/manpage.txt
@@ -458,7 +458,7 @@
csv` is the same as `--ocsv`.
1mFLATTEN-UNFLATTEN FLAGS0m
- These flags control how Miller converts record values which are maps or arrays, when input is JSON and output is non-JSON (flattening) or input is non-JSON and output is JSON (unflattening).
+ These flags control how Miller converts record values which are maps or arrays, when input is JSON/YAML and output is not (flattening) or input is not JSON/YAML and output is JSON/YAML (unflattening).
See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatten-unflatten for more information.
@@ -466,16 +466,17 @@
Separator for flattening multi-level JSON keys, e.g.
`{"a":{"b":3}}` becomes `a:b => 3` for non-JSON
formats. Defaults to `.`.
- --no-auto-flatten When output is non-JSON, suppress the default
+ --no-auto-flatten When output is not JSON or YAML, suppress the default
auto-flatten behavior. Default: if `$y = [7,8,9]`
then this flattens to `y.1=7,y.2=8,y.3=9`, and
similarly for maps. With `--no-auto-flatten`, instead
we get `$y=[1, 2, 3]`.
- --no-auto-unflatten When input is non-JSON and output is JSON, suppress
- the default auto-unflatten behavior. Default: if the
- input has `y.1=7,y.2=8,y.3=9` then this unflattens to
- `$y=[7,8,9]`. With `--no-auto-flatten`, instead we
- get `${y.1}=7,${y.2}=8,${y.3}=9`.
+ --no-auto-unflatten When input is not JSON or YAML and output is JSON or
+ YAML, suppress the default auto-unflatten behavior.
+ Default: if the input has `y.1=7,y.2=8,y.3=9` then
+ this unflattens to `$y=[7,8,9]`. With
+ `--no-auto-flatten`, instead we get
+ `${y.1}=7,${y.2}=8,${y.3}=9`.
1mFORMAT-CONVERSION KEYSTROKE-SAVER FLAGS0m
As keystroke-savers for format-conversion you may use the following.
@@ -4138,4 +4139,4 @@
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
https://miller.readthedocs.io
- 2026-07-14 4mMILLER24m(1)
+ 2026-07-15 4mMILLER24m(1)
diff --git a/man/mlr.1 b/man/mlr.1
index 3c5b28c52..a7b882e2a 100644
--- a/man/mlr.1
+++ b/man/mlr.1
@@ -2,12 +2,12 @@
.\" Title: mlr
.\" Author: [see the "AUTHOR" section]
.\" Generator: ./mkman.rb
-.\" Date: 2026-07-14
+.\" Date: 2026-07-15
.\" Manual: \ \&
.\" Source: \ \&
.\" Language: English
.\"
-.TH "MILLER" "1" "2026-07-14" "\ \&" "\ \&"
+.TH "MILLER" "1" "2026-07-15" "\ \&" "\ \&"
.\" -----------------------------------------------------------------
.\" * Portability definitions
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -554,7 +554,7 @@ are overridden in all cases by setting output format to `format2`.
.RS 0
.\}
.nf
-These flags control how Miller converts record values which are maps or arrays, when input is JSON and output is non-JSON (flattening) or input is non-JSON and output is JSON (unflattening).
+These flags control how Miller converts record values which are maps or arrays, when input is JSON/YAML and output is not (flattening) or input is not JSON/YAML and output is JSON/YAML (unflattening).
See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatten-unflatten for more information.
@@ -562,16 +562,17 @@ See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatt
Separator for flattening multi-level JSON keys, e.g.
`{"a":{"b":3}}` becomes `a:b => 3` for non-JSON
formats. Defaults to `.`.
---no-auto-flatten When output is non-JSON, suppress the default
+--no-auto-flatten When output is not JSON or YAML, suppress the default
auto-flatten behavior. Default: if `$y = [7,8,9]`
then this flattens to `y.1=7,y.2=8,y.3=9`, and
similarly for maps. With `--no-auto-flatten`, instead
we get `$y=[1, 2, 3]`.
---no-auto-unflatten When input is non-JSON and output is JSON, suppress
- the default auto-unflatten behavior. Default: if the
- input has `y.1=7,y.2=8,y.3=9` then this unflattens to
- `$y=[7,8,9]`. With `--no-auto-flatten`, instead we
- get `${y.1}=7,${y.2}=8,${y.3}=9`.
+--no-auto-unflatten When input is not JSON or YAML and output is JSON or
+ YAML, suppress the default auto-unflatten behavior.
+ Default: if the input has `y.1=7,y.2=8,y.3=9` then
+ this unflattens to `$y=[7,8,9]`. With
+ `--no-auto-flatten`, instead we get
+ `${y.1}=7,${y.2}=8,${y.3}=9`.
.fi
.if n \{\
.RE
diff --git a/pkg/cli/flatten_unflatten.go b/pkg/cli/flatten_unflatten.go
index bd2593f2a..d256494bc 100644
--- a/pkg/cli/flatten_unflatten.go
+++ b/pkg/cli/flatten_unflatten.go
@@ -5,7 +5,7 @@ package cli
//
// PROBLEM TO BE SOLVED:
//
-// JSON has nested structures and CSV et al. do not. For example:
+// JSON and YAML have nested structures and CSV et al. do not. For example:
// {
// "req" : {
// "method": "GET",
@@ -22,7 +22,9 @@ package cli
//
// APPROACH:
//
-// Use the Principle of Least Surprise (POLS).
+// Use the Principle of Least Surprise (POLS). Below, "JSON" stands for any
+// format capable of representing nested structures natively -- currently
+// JSON, JSON Lines, and YAML.
//
// * If input is JSON and output is JSON:
// o Records can be nested from record-read
@@ -54,11 +56,24 @@ package cli
// flatten, don't undo that by putting an unflatten right after.
//
+// isNestable returns true for formats which can represent nested/array
+// structures natively, and thus don't need auto-flatten/auto-unflatten.
+func isNestable(format string) bool {
+ return format == "json" || format == "jsonl" || format == "yaml"
+}
+
func DecideFinalFlatten(writerOptions *TWriterOptions) bool {
ofmt := writerOptions.OutputFileFormat
if writerOptions.AutoFlatten {
- // Preserve nested/array structure for formats that support it.
- if ofmt != "json" && ofmt != "jsonl" && ofmt != "dcf" {
+ // JSON/YAML/JSON-Lines preserve nested/array structure natively, so
+ // they never need flattening.
+ //
+ // DCF is excluded for a different reason: it's not nestable, but it
+ // has its own hardcoded comma-list serialization for a fixed set of
+ // field names (Depends, Recommends, etc. -- see
+ // pkg/output/record_writer_dcf.go), which generic key-spreading
+ // flatten would clobber.
+ if !isNestable(ofmt) && ofmt != "dcf" {
return true
}
}
@@ -85,8 +100,8 @@ func DecideFinalUnflatten(
ofmt := options.WriterOptions.OutputFileFormat
if options.WriterOptions.AutoUnflatten {
- if ifmt != "json" {
- if ofmt == "json" {
+ if !isNestable(ifmt) {
+ if isNestable(ofmt) {
return true
}
}
diff --git a/pkg/cli/option_parse.go b/pkg/cli/option_parse.go
index 08b707ff7..5cf197194 100644
--- a/pkg/cli/option_parse.go
+++ b/pkg/cli/option_parse.go
@@ -3300,7 +3300,7 @@ var OutputColorizationFlagSection = FlagSection{
// FLATTEN/UNFLATTEN FLAGS
func FlattenUnflattenPrintInfo() {
- fmt.Println("These flags control how Miller converts record values which are maps or arrays, when input is JSON and output is non-JSON (flattening) or input is non-JSON and output is JSON (unflattening).")
+ fmt.Println("These flags control how Miller converts record values which are maps or arrays, when input is JSON/YAML and output is not (flattening) or input is not JSON/YAML and output is JSON/YAML (unflattening).")
fmt.Println()
fmt.Println("See the flatten/unflatten doc page https://miller.readthedocs.io/en/latest/flatten-unflatten for more information.")
}
@@ -3326,7 +3326,7 @@ var FlattenUnflattenFlagSection = FlagSection{
{
name: "--no-auto-flatten",
- help: "When output is non-JSON, suppress the default auto-flatten behavior. Default: if `$y = [7,8,9]` then this flattens to `y.1=7,y.2=8,y.3=9`, and similarly for maps. With `--no-auto-flatten`, instead we get `$y=[1, 2, 3]`.",
+ help: "When output is not JSON or YAML, suppress the default auto-flatten behavior. Default: if `$y = [7,8,9]` then this flattens to `y.1=7,y.2=8,y.3=9`, and similarly for maps. With `--no-auto-flatten`, instead we get `$y=[1, 2, 3]`.",
parser: func(args []string, argc int, pargi *int, options *TOptions) {
options.WriterOptions.AutoFlatten = false
*pargi += 1
@@ -3335,7 +3335,7 @@ var FlattenUnflattenFlagSection = FlagSection{
{
name: "--no-auto-unflatten",
- help: "When input is non-JSON and output is JSON, suppress the default auto-unflatten behavior. Default: if the input has `y.1=7,y.2=8,y.3=9` then this unflattens to `$y=[7,8,9]`. With `--no-auto-flatten`, instead we get `${y.1}=7,${y.2}=8,${y.3}=9`.",
+ help: "When input is not JSON or YAML and output is JSON or YAML, suppress the default auto-unflatten behavior. Default: if the input has `y.1=7,y.2=8,y.3=9` then this unflattens to `$y=[7,8,9]`. With `--no-auto-flatten`, instead we get `${y.1}=7,${y.2}=8,${y.3}=9`.",
parser: func(args []string, argc int, pargi *int, options *TOptions) {
options.WriterOptions.AutoUnflatten = false
*pargi += 1
diff --git a/test/cases/io-yaml-io/0005/cmd b/test/cases/io-yaml-io/0005/cmd
new file mode 100644
index 000000000..84ada0c8e
--- /dev/null
+++ b/test/cases/io-yaml-io/0005/cmd
@@ -0,0 +1 @@
+mlr --icsv --oyaml cat ${CASEDIR}/input
diff --git a/test/cases/io-yaml-io/0005/experr b/test/cases/io-yaml-io/0005/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-yaml-io/0005/expout b/test/cases/io-yaml-io/0005/expout
new file mode 100644
index 000000000..c40c130f9
--- /dev/null
+++ b/test/cases/io-yaml-io/0005/expout
@@ -0,0 +1,8 @@
+- a: 1
+ b:
+ - 2
+ - 3
+- a: 4
+ b:
+ - 5
+ - 6
diff --git a/test/cases/io-yaml-io/0005/input b/test/cases/io-yaml-io/0005/input
new file mode 100644
index 000000000..d17ad1a6d
--- /dev/null
+++ b/test/cases/io-yaml-io/0005/input
@@ -0,0 +1,3 @@
+a,b.1,b.2
+1,2,3
+4,5,6
diff --git a/test/cases/io-yaml-io/0006/cmd b/test/cases/io-yaml-io/0006/cmd
new file mode 100644
index 000000000..5c51bf61a
--- /dev/null
+++ b/test/cases/io-yaml-io/0006/cmd
@@ -0,0 +1 @@
+mlr --ijson --oyaml cat ${CASEDIR}/input
diff --git a/test/cases/io-yaml-io/0006/experr b/test/cases/io-yaml-io/0006/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-yaml-io/0006/expout b/test/cases/io-yaml-io/0006/expout
new file mode 100644
index 000000000..9aeb886e1
--- /dev/null
+++ b/test/cases/io-yaml-io/0006/expout
@@ -0,0 +1,4 @@
+- a: 1
+ req:
+ method: GET
+ path: api/check
diff --git a/test/cases/io-yaml-io/0006/input b/test/cases/io-yaml-io/0006/input
new file mode 100644
index 000000000..a9c61b51a
--- /dev/null
+++ b/test/cases/io-yaml-io/0006/input
@@ -0,0 +1,3 @@
+[
+{"a": 1, "req": {"method": "GET", "path": "api/check"}}
+]
diff --git a/test/cases/io-yaml-io/0007/cmd b/test/cases/io-yaml-io/0007/cmd
new file mode 100644
index 000000000..6fd32a30b
--- /dev/null
+++ b/test/cases/io-yaml-io/0007/cmd
@@ -0,0 +1 @@
+mlr --iyaml --ocsv cat ${CASEDIR}/input
diff --git a/test/cases/io-yaml-io/0007/experr b/test/cases/io-yaml-io/0007/experr
new file mode 100644
index 000000000..e69de29bb
diff --git a/test/cases/io-yaml-io/0007/expout b/test/cases/io-yaml-io/0007/expout
new file mode 100644
index 000000000..d17ad1a6d
--- /dev/null
+++ b/test/cases/io-yaml-io/0007/expout
@@ -0,0 +1,3 @@
+a,b.1,b.2
+1,2,3
+4,5,6
diff --git a/test/cases/io-yaml-io/0007/input b/test/cases/io-yaml-io/0007/input
new file mode 100644
index 000000000..c40c130f9
--- /dev/null
+++ b/test/cases/io-yaml-io/0007/input
@@ -0,0 +1,8 @@
+- a: 1
+ b:
+ - 2
+ - 3
+- a: 4
+ b:
+ - 5
+ - 6