Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-01-23 10:15:36 +00:00
Code Issues Projects Releases Wiki Activity

json-arrays-as-map doc

Browse source
This commit is contained in:
John Kerl 2017-03-17 09:30:37 -04:00
parent dafa792fb3
commit ee71cd22bc
2 changed files with 95 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

33
doc/content-for-file-formats.html
View file

@ -191,18 +191,31 @@ POKI_RUN_COMMAND{{mlr --json --jvstack head -n 1 then put '${values:uv} = ${valu
<h2>Arrays</h2>
<p/>Arrays aren&rsquo;t supported in Miller. The <tt>put</tt>/<tt>filter</tt>
DSL does support hashmaps, and an array could be read in as an integer-keyed
map: e.g. JSON <tt>["a","b","c"]</tt> could be read in as Miller hashmap
<tt>{1:"a",2:"b",3:"c"}</tt>. But then arrays and maps both become maps, with
the result that, once written back as output, they would differ from input.
<p/>Arrays aren&rsquo;t supported in Miller&rsquo;s
<tt>put</tt>/<tt>filter</tt> DSL. By default, JSON arrays are read in as
integer-keyed maps.
<p/>As a result, Miller by default fatals when it encounters arrays, which are
&ldquo;unmillerable&rdquo;. You can use <tt>mlr
--json-skip-arrays-on-input</tt>, if you like, which ignores arrays. To truly
handle JSON, please use a JSON-processing tool such as
<a href="http://stedolan.github.io/jq/">jq</a>.
<p/> Suppose you have arrays like this in our input data:
POKI_RUN_COMMAND{{cat data/json-example-3.json}}HERE
<p/> Then integer indices (starting from 0 and counting up) are used as map keys:
POKI_RUN_COMMAND{{mlr --ijson --oxtab cat data/json-example-3.json}}HERE
<p/> When the data are written back out as JSON, field names are re-expanded as above,
but what were arrays on input are now maps on output:
POKI_RUN_COMMAND{{mlr --json --jvstack cat data/json-example-3.json}}HERE
<p/> This is non-ideal, but it allows Miller (5.x release being latest as of
this writing) to handle JSON arrays at all.
<p/>You might also use <tt>mlr --json-skip-arrays-on-input</tt> or <tt>mlr
--json-fatal-arrays-on-input</tt>.
To truly handle JSON, please use a JSON-processing tool such as <a
href="http://stedolan.github.io/jq/">jq</a>.
<h2>Formatting JSON options</h2>

82
doc/file-formats.html
View file

@ -655,18 +655,80 @@ $ mlr --json --jvstack head -n 1 then put '${values:uv} = ${values:u} * ${values
<a id="Arrays"/><h2>Arrays</h2>
<p/>Arrays aren&rsquo;t supported in Miller. The <tt>put</tt>/<tt>filter</tt>
DSL does support hashmaps, and an array could be read in as an integer-keyed
map: e.g. JSON <tt>["a","b","c"]</tt> could be read in as Miller hashmap
<tt>{1:"a",2:"b",3:"c"}</tt>. But then arrays and maps both become maps, with
the result that, once written back as output, they would differ from input.
<p/>Arrays aren&rsquo;t supported in Miller&rsquo;s
<tt>put</tt>/<tt>filter</tt> DSL. By default, JSON arrays are read in as
integer-keyed maps.
<p/>As a result, Miller by default fatals when it encounters arrays, which are
&ldquo;unmillerable&rdquo;. You can use <tt>mlr
--json-skip-arrays-on-input</tt>, if you like, which ignores arrays. To truly
handle JSON, please use a JSON-processing tool such as
<a href="http://stedolan.github.io/jq/">jq</a>.
<p/> Suppose you have arrays like this in our input data:
<p/>
<div class="pokipanel">
<pre>
$ cat data/json-example-3.json
{
"label": "orange",
"values": [12.2, 13.8, 17.2]
}
{
"label": "purple",
"values": [27.0, 32.4]
}
</pre>
</div>
<p/>
<p/> Then integer indices (starting from 0 and counting up) are used as map keys:
<p/>
<div class="pokipanel">
<pre>
$ mlr --ijson --oxtab cat data/json-example-3.json
label orange
values:0 12.2
values:1 13.8
values:2 17.2
label purple
values:0 27.0
values:1 32.4
</pre>
</div>
<p/>
<p/> When the data are written back out as JSON, field names are re-expanded as above,
but what were arrays on input are now maps on output:
<p/>
<div class="pokipanel">
<pre>
$ mlr --json --jvstack cat data/json-example-3.json
{
"label": "orange",
"values": {
"0": 12.2,
"1": 13.8,
"2": 17.2
}
}
{
"label": "purple",
"values": {
"0": 27.0,
"1": 32.4
}
}
</pre>
</div>
<p/>
<p/> This is non-ideal, but it allows Miller (5.x release being latest as of
this writing) to handle JSON arrays at all.
<p/>You might also use <tt>mlr --json-skip-arrays-on-input</tt> or <tt>mlr
--json-fatal-arrays-on-input</tt>.
To truly handle JSON, please use a JSON-processing tool such as <a
href="http://stedolan.github.io/jq/">jq</a>.
<a id="Formatting_JSON_options"/><h2>Formatting JSON options</h2>