merge

docs6/docs/10min.md

@@ -87,7 +87,7 @@ purple,square,false,10,91,72.3735,8.2430
 {
   "color": "yellow",
   "shape": "circle",
-  "flag": true,
+  "flag": "true",
   "k": 9,
   "index": 87,
   "quantity": 63.5058,
@@ -96,7 +96,7 @@ purple,square,false,10,91,72.3735,8.2430
 {
   "color": "purple",
   "shape": "square",
-  "flag": false,
+  "flag": "false",
   "k": 10,
   "index": 91,
   "quantity": 72.3735,
@@ -212,14 +212,9 @@ red   square false 4 48    77.5542  7.4670
 red   square false 6 64    77.1991  9.5310
 </pre>
 
-<pre class="pre-highlight-in-pair">
+<pre class="pre-highlight-non-pair">
 <b>mlr --icsv --opprint filter '$color == "red" && $flag == true' example.csv</b>
 </pre>
-<pre class="pre-non-highlight-in-pair">
-color shape  flag k index quantity rate
-red   square true 2 15    79.2778  0.0130
-red   circle true 3 16    13.8103  2.9010
-</pre>
 
 You can use `put` to create new fields which are computed from other fields:
 

docs6/docs/cpu.md

@@ -39,6 +39,13 @@ In practice, profiling has shown that the input-reader uses the most CPU of all
 the above. This means CPUs running verbs may not be 100% utilized, since they
 are likely to be spending some of their time waiting for input data.
 
+Running Miller on a machine with more CPUs than active channels (as listed
+above) won't speed up a given invocation of Miller. However, of course, you'll
+be able to run more invocations of Miller at the same time if you like.
+
 You can set the Go-standard environment variable `GOMAXPROCS` if you like. If
 you don't, Miller will (as is standard for Go programs in Go 1.16 and above) up
 to all available CPUs.
+
+If you set `$GOMAXPROCS=1` in the environment, that's fine -- the Go runtime
+will multiplex different channel-handling goroutines onto the same CPU.

docs6/docs/cpu.md.in

@@ -25,6 +25,13 @@ In practice, profiling has shown that the input-reader uses the most CPU of all
 the above. This means CPUs running verbs may not be 100% utilized, since they
 are likely to be spending some of their time waiting for input data.
 
+Running Miller on a machine with more CPUs than active channels (as listed
+above) won't speed up a given invocation of Miller. However, of course, you'll
+be able to run more invocations of Miller at the same time if you like.
+
 You can set the Go-standard environment variable `GOMAXPROCS` if you like. If
 you don't, Miller will (as is standard for Go programs in Go 1.16 and above) up
 to all available CPUs.
+
+If you set `$GOMAXPROCS=1` in the environment, that's fine -- the Go runtime
+will multiplex different channel-handling goroutines onto the same CPU.

docs6/docs/data-cleaning-examples.md

@@ -58,9 +58,9 @@ A second option is to flag badly formatted data within the output stream:
 </pre>
 <pre class="pre-non-highlight-in-pair">
 name   reachable format_ok
-barney false     false
-betty  true      false
-fred   true      false
+barney false     true
+betty  true      true
+fred   true      true
 wilma  1         false
 </pre>
 
@@ -72,9 +72,6 @@ Or perhaps to flag badly formatted data outside the output stream:
 <b>' data/het-bool.csv</b>
 </pre>
 <pre class="pre-non-highlight-in-pair">
-Malformed at NR=1
-Malformed at NR=2
-Malformed at NR=3
 Malformed at NR=4
 name   reachable
 barney false
@@ -89,5 +86,5 @@ A third way is to abort the process on first instance of bad data:
 <b>mlr --csv put '$reachable = asserting_string($reachable)' data/het-bool.csv</b>
 </pre>
 <pre class="pre-non-highlight-in-pair">
-Miller: is_string type-assertion failed at NR=1 FNR=1 FILENAME=data/het-bool.csv
+Miller: is_string type-assertion failed at NR=4 FNR=4 FILENAME=data/het-bool.csv
 </pre>

docs6/docs/data/json-in-csv.csv

@@ -0,0 +1,3 @@
+id,blob
+100,"{""a"":1,""b"":[2,3,4]}"
+105,"{""a"":6,""b"":[7,8,9]}"

docs6/docs/data/type-infer.csv

@@ -0,0 +1,3 @@
+a,b,c
+1.2,3,true
+4,5.6,buongiorno

docs6/docs/file-formats.md

@@ -271,6 +271,13 @@ TODO: probably remove entirely
 
 * Use `--jflatsep yourseparatorhere` to specify the string used for key concatenation: this defaults to a single dot.
 
+### JSON-in-CSV
+
+It's quite common to have CSV data which contains stringified JSON as a column.
+See the [JSON parse and stringify
+section](reference-main-data-types.md#json-parse-and-stringify) for ways to
+decode these in Miller.
+
 ## PPRINT: Pretty-printed tabular
 
 Miller's pretty-print format is like CSV, but column-aligned.  For example, compare

docs6/docs/file-formats.md.in

@@ -118,6 +118,13 @@ TODO: probably remove entirely
 
 * Use `--jflatsep yourseparatorhere` to specify the string used for key concatenation: this defaults to a single dot.
 
+### JSON-in-CSV
+
+It's quite common to have CSV data which contains stringified JSON as a column.
+See the [JSON parse and stringify
+section](reference-main-data-types.md#json-parse-and-stringify) for ways to
+decode these in Miller.
+
 ## PPRINT: Pretty-printed tabular
 
 Miller's pretty-print format is like CSV, but column-aligned.  For example, compare

docs6/docs/javascripts/tables.js

@@ -0,0 +1,6 @@
+document$.subscribe(function() {
+  var tables = document.querySelectorAll("article table")
+  tables.forEach(function(table) {
+    new Tablesort(table)
+  })
+})

docs6/docs/keystroke-savers.md

@@ -34,7 +34,7 @@ red    square   true 2 15    79.2778  0.0130
 {
   "color": "yellow",
   "shape": "triangle",
-  "flag": true,
+  "flag": "true",
   "k": 1,
   "index": 11,
   "quantity": 43.6498,
@@ -43,7 +43,7 @@ red    square   true 2 15    79.2778  0.0130
 {
   "color": "red",
   "shape": "square",
-  "flag": true,
+  "flag": "true",
   "k": 2,
   "index": 15,
   "quantity": 79.2778,

docs6/docs/mk-func-info.rb

@@ -0,0 +1,114 @@
+#!/usr/bin/env ruby
+
+# ================================================================
+# Markdown autogen for built-in functions: table, or full details page.
+# Invoked by files in *.md.in.
+# ================================================================
+
+# ----------------------------------------------------------------
+# Displayname is for HTML rendered in the document body. Linkname is for
+# #-style links which don't do so well with special characters.
+
+LOOKUP = {
+  # name     display_name link_name
+'!'    => [ '\!',      'exclamation-point'        ],
+'!='   => [ '\!=',     'exclamation-point-equals' ],
+'!=~'  => [ '!=~',     'regnotmatch'              ],
+'%'    => [ '%',       'percent'                  ],
+'&&'   => [ '&&',      'logical-and'              ],
+'&'    => [ '&',       'bitwise-and'              ],
+'*'    => [ '\*',      'times'                    ],
+'**'   => [ '\**',     'exponentiation'           ],
+'+'    => [ '\+',      'plus'                     ],
+'-'    => [ '\-',      'minus'                    ],
+'.'    => [ '\.',      'dot'                      ],
+'.*'   => [ '\.\*',    'dot-times'                ],
+'.+'   => [ '\.\+',    'dot-plus'                 ],
+'.-'   => [ '\.\-',    'dot-minus'                ],
+'./'   => [ '\./',     'dot-slash'                ],
+'/'    => [ '/',       'slash'                    ],
+'//'   => [ '//',      'slash-slash'              ],
+':'    => [ '\:',      'colon'                    ],
+'<'    => [ '<',       'less-than'                ],
+'<<'   => [ '<<',      'lsh'                      ],
+'<='   => [ '<=',      'less-than-or-equals'      ],
+'='    => [ '=',       'equals'                   ],
+'=='   => [ '==',      'double-equals'            ],
+'=~'   => [ '=~',      'regmatch'                 ],
+'>'    => [ '>',       'greater-than'             ],
+'>='   => [ '>=',      'greater-than-or-equals'   ],
+'>>'   => [ '>>',      'srsh'                     ],
+'>>>'  => [ '>>>',     'ursh'                     ],
+'>>>=' => [ '>>>=',    'ursheq'                   ],
+'?'    => [ '?',       'question-mark'            ],
+'?.:'  => [ '?:',      'question-mark-colon'      ],
+'?:'   => [ '?:',      'question-mark-colon'      ],
+'??'   => [ '??',      'absent-coalesce'          ],
+'???'  => [ '???',     'absent-empty-coalesce'    ],
+'^'    => [ '^',       'bitwise-xor'              ],
+'^^'   => [ '^^',      'logical-xor'              ],
+'|'    => [ '\|',      'bitwise-or'               ],
+'||'   => [ '\|\|',    'logical-or'               ],
+'~'    => [ '~',       'bitwise-not'              ],
+}
+
+def name_to_display_name_and_link_name(name)
+  entry = LOOKUP[name]
+  if entry.nil?
+    return [name, name]
+  else
+    display_name, link_name = entry
+    return [display_name, link_name]
+  end
+end
+
+# ----------------------------------------------------------------
+def make_func_details
+  function_classes = `mlr help list-function-classes`.split
+
+  puts
+  puts "## Summary"
+  puts
+  for function_class in function_classes
+    class_link_name = "#"+"#{function_class}-functions"
+    class_display_name = "#{function_class.capitalize} functions"
+    #puts "* [#{display_name}](#{link_name})"
+
+    functions_in_class = `mlr help list-functions-in-class #{function_class}`.split
+    foo = []
+    for function_name in functions_in_class
+      display_name, link_name = name_to_display_name_and_link_name(function_name)
+      foo.append " [#{display_name}]("+"#"+"#{link_name})"
+    end
+    #puts "    * #{foo.join(',')}"
+    puts "* [**#{class_display_name}**](#{class_link_name}): #{foo.join(', ')}."
+
+  end
+  puts
+
+  for function_class in function_classes
+    display_name = "#{function_class.capitalize} functions"
+    puts "## #{display_name}"
+    puts
+
+    functions_in_class = `mlr help list-functions-in-class #{function_class}`.split
+
+    for function_name in functions_in_class
+      display_name, link_name = name_to_display_name_and_link_name(function_name)
+
+      puts
+      if display_name != link_name
+        puts "<a id=#{link_name} />"
+      end
+      puts "### #{display_name}"
+
+      puts '<pre class="pre-non-highlight-non-pair">'
+      system("mlr help function '#{function_name}'")
+      puts '</pre>'
+      puts
+    end
+  end
+end
+
+# ----------------------------------------------------------------
+make_func_details

docs6/docs/mk-func-table.rb

@@ -1,93 +0,0 @@
-#!/usr/bin/env ruby
-
-# Sphinx tables need to be precisely lined up, unlike Markdown tables which
-# are more flexible. For example this is fine Markdown:
-#
-#   | a | b | i | x | y |
-#   | --- | --- | --- | --- | --- |
-#   | pan | pan | 1 | 0.3467901443380824 | 0.7268028627434533 |
-#   | eks | pan | 2 | 0.7586799647899636 | 0.5221511083334797 |
-#   | wye | wye | 3 | 0.20460330576630303 | 0.33831852551664776 |
-#   | eks | wye | 4 | 0.38139939387114097 | 0.13418874328430463 |
-#
-# but Sphinx wants
-#
-#   +-----+-----+---+---------------------+---------------------+
-#   | a   | b   | i | x                   | y                   |
-#   +=====+=====+===+=====================+=====================+
-#   | pan | pan | 1 | 0.3467901443380824  | 0.7268028627434533  |
-#   +-----+-----+---+---------------------+---------------------+
-#   | eks | pan | 2 | 0.7586799647899636  | 0.5221511083334797  |
-#   +-----+-----+---+---------------------+---------------------+
-#   | wye | wye | 3 | 0.20460330576630303 | 0.33831852551664776 |
-#   +-----+-----+---+---------------------+---------------------+
-#   | eks | wye | 4 | 0.38139939387114097 | 0.13418874328430463 |
-#   +-----+-----+---+---------------------+---------------------+
-#
-# So we need to build up a list of tuples, then compute the max length down
-# each column, then write a fixed number of -/= per row.
-
-lines = `mlr --list-all-functions-as-table`
-
-# ----------------------------------------------------------------
-# Pass 1
-
-max_name_length = 1
-max_function_class_length = 1
-max_nargs_length = 1
-
-rows = []
-lines.split("\n").each do |line|
-  if line =~ /^? :.*/ # has an extra whitespace which throws off the naive split
-    ig, nore, function_class, nargs = line.split(/\s+/)
-    name = '? :'
-  else
-    name, function_class, nargs = line.split(/\s+/)
-  end
-
-  name = '``' + name + '``'
-
-  if max_name_length < name.length
-    max_name_length = name.length
-  end
-  if max_function_class_length < function_class.length
-    max_function_class_length = function_class.length
-  end
-  if max_nargs_length < nargs.length
-    max_nargs_length = nargs.length
-  end
-
-  # TODO: format name with page-internal link
-  row = [name, function_class, nargs]
-  rows.append(row)
-end
-
-# ----------------------------------------------------------------
-# Pass 2
-
-name_dashes = '-' * max_name_length
-function_class_dashes = '-' * max_function_class_length
-nargs_dashes = '-' * max_nargs_length
-
-name_equals = '=' * max_name_length
-function_class_equals = '=' * max_function_class_length
-nargs_equals = '=' * max_nargs_length
-
-dashes_line = "+-#{name_dashes}-+-#{function_class_dashes}-+-#{nargs_dashes}-+"
-equals_line = "+=#{name_equals}=+=#{function_class_equals}=+=#{nargs_equals}=+"
-
-counter = 0
-rows.each do |row|
-  name, function_class, nargs = row
-
-  counter = counter + 1
-  if counter == 1
-    puts dashes_line
-    puts "| #{name.ljust(max_name_length)} | #{function_class.ljust(max_function_class_length)} | #{nargs.ljust(max_nargs_length)} |"
-    puts equals_line
-  else
-    puts "| #{name.ljust(max_name_length)} | #{function_class.ljust(max_function_class_length)} | #{nargs.ljust(max_nargs_length)} |"
-    puts dashes_line
-  end
-end
-

docs6/docs/online-help.md

@@ -49,7 +49,10 @@ Type 'mlr help {topic}' for any of the following:
   mlr help function
   mlr help keyword
   mlr help list-functions
+  mlr help list-function-classes
+  mlr help list-functions-in-class
   mlr help list-functions-as-paragraph
+  mlr help list-functions-as-table
   mlr help list-keywords
   mlr help list-keywords-as-paragraph
   mlr help list-verbs
@@ -61,6 +64,7 @@ Type 'mlr help {topic}' for any of the following:
   mlr help separator-options
   mlr help type-arithmetic-info
   mlr help usage-functions
+  mlr help usage-functions-by-class
   mlr help usage-keywords
   mlr help usage-verbs
   mlr help verb
@@ -91,7 +95,10 @@ Type 'mlr help {topic}' for any of the following:
   mlr help function
   mlr help keyword
   mlr help list-functions
+  mlr help list-function-classes
+  mlr help list-functions-in-class
   mlr help list-functions-as-paragraph
+  mlr help list-functions-as-table
   mlr help list-keywords
   mlr help list-keywords-as-paragraph
   mlr help list-verbs
@@ -103,6 +110,7 @@ Type 'mlr help {topic}' for any of the following:
   mlr help separator-options
   mlr help type-arithmetic-info
   mlr help usage-functions
+  mlr help usage-functions-by-class
   mlr help usage-keywords
   mlr help usage-verbs
   mlr help verb
@@ -179,7 +187,7 @@ Given the name of a DSL function (from `mlr -f`) you can use `mlr help function`
 <b>mlr help function append</b>
 </pre>
 <pre class="pre-non-highlight-in-pair">
-append  (class=maps/arrays #args=2) Appends second argument to end of first argument, which must be an array.
+append  (class=collections #args=2) Appends second argument to end of first argument, which must be an array.
 </pre>
 
 <pre class="pre-highlight-in-pair">

docs6/docs/proofreads.txt

@@ -1,14 +1,21 @@
-----------------------------------------------------------------
-! merge 3 new 4 !
+TOP:
 
+c! seps \001 etc !
+  mlrc --iusv --oxtab cat regtest/input/example.usv
+  mlr  --iusv --oxtab cat regtest/input/example.usv
+C! repifs !! https://pkg.go.dev/regexp#Regexp.Split 2-for-1 -- get regexp as well ?
+
+E flatten/unflatten page
+? twi-dm re all-contribs: all-contributors.org
+C flags LUTs
+
+----------------------------------------------------------------
 ALL:
 * csv to csv,tsv throughout
-* rid of explicitly passing around os.Stdout in all various help functions, annoying
 * het.dkvp > het.json in more places
 * check each page for adequate h2 coverage
 
 * deduping pass!
-* 1-up/0-up -> one-up/zero-up everywhere
 
 * flags-doc
   C refactor cli-parser w/ LUT & help strings
@@ -19,15 +26,21 @@ ALL:
   https://squidfunk.github.io/mkdocs-material/customization/#extending-the-theme
 
 ----------------------------------------------------------------
-c! repifs !!
-c! seps \001 etc !
-C flags LUTs
-? twi-dm re all-contribs: all-contributors.org
 
 e fzf-ish w/ head -n 4, --from, up-arrow & append verb, then cat -- find & update the existing section
 
+? figure out flatsep vs iflatsep/oflatsep & have a story around the choice either way
+  - ctx ptr-split for invars
 E fill out flatten/unflatten page
-E fill out data-types page
+  echo a.b.c=3 | mlr --ojson cat
+  echo a.b.c=3 | mlr --ojson --no-jvstack cat | mlr --j2c --oflatsep : cat
+  echo a:b:c=3 | mlr --ojson --no-jvstack cat
+  echo a:b:c=3 | mlr --ojson --no-jvstack --oflatsep : cat
+  -> call it --flatsep
+
+E fill out strings page
+  e substr0/substr1/substr. slices & string-indexing new in m6.
+  c "abcde"[2:3] works but "abcde"[2] does not :(
 E fill out arrays page
   - xref to fla/unfla page
   - slices including semantics for out-of-range slices
@@ -37,8 +50,6 @@ E fill out maps page
 E make flags page
   C show-all-flags help -- fully alpha, or alpha-by-type; LUT refactor
 
-* functions: somewhere organize by type. olh/man6/docs6 ...
-
 * regex: more about what is / is not
   https://github.com/johnkerl/miller/issues/77#issuecomment-538553828
 
@@ -94,24 +105,12 @@ reference-verbs:
 E data/colored-shapes.dkvp (this page & throughout) a CSV file ...
 * ... I ONLY READ UP TO CUT & PAUSED ...
 
-reference-main-data-types:
-E field values are usually strings -> update
-E Field values are treated as numeric for the following: -> update
-? Miller's types for function processing are -> add JSON null; & check for others (array/map)
-E true/false -> add info about NaN and Inf et al.
-!! very much missing stuff!! where's the listing of mlrval types?!?
-  - reference-dsl-variables.md.in should link to it
-  - ditto programming-language.md.in
-c UTs for r"a" . r"b" and so on and so on
-
-reference-dsl-variables.md:
-? IFLATSEP & OFLATSEP -- ?
+reference-dsl-variables.md: FLATSEP
 
 reference-main-null-data:
 ? more variants of is_... ?
 ? Records with one or more empty sort-field values sort after records with all sort-field values present -> apparently not true for sort -nr
 e split out h2's
-! spell out JSON-null
 
 reference-main-arithmetic:
 ? test stats1/step -F flag
@@ -132,7 +131,6 @@ E User-defined subroutines -> non-factorial example -- maybe some useful aggrega
 l link to --load and --mload
 
 reference-dsl-builtin-functions:
-c mlr: option "--list-all-functions-as-table" not recognized. Please run "mlr --help" for usage information -> fix
 ! ... need to proofread entire list ...
 
 reference-dsl-output-statements:
@@ -157,5 +155,3 @@ E write up which file formats support which flags
 
 manpage:
 ? [NEEDS READ-THROUGH]
-
-mk-func-table.rb: port comments from sphinx to mkdocs

docs6/docs/reference-dsl-builtin-functions.md.in

@@ -1,13 +1,21 @@
 # DSL built-in functions
 
-## Summary
+Each function takes a specific number of arguments, as shown below, except for
+functions marked as variadic such as `min` and `max`. (The latter compute min
+and max of any number of arguments.) There is no notion of optional or
+default-on-absent arguments. All argument-passing is positional rather than by
+name; arguments are passed by value, not by reference.
 
-GENMD_RUN_CONTENT_GENERATOR(./mk-func-table.rb)
+At the command line, you can get a list of all functions using `mlr -f`, with
+details using `mlr -F`.  (Or, `mlr help usage-functions-by-class` to get
+details in the order shown on this page.) You can get detail for a given
+function using `mlr help function namegoeshere`, e.g.  `mlr help function
+gsub`.
 
-## List of functions
+Operators are listed here along with functions. In this case, the
+argument-count is the number of items involved in the infix operator, e.g. we
+say `x+y` so the details for the `+` operator say that its number of arguments
+is 2. Unary operators such as `!` and `~` show argument-count of 1; the ternary
+`? :` operator shows an argument-count of 3.
 
-Each function takes a specific number of arguments, as shown below, except for functions marked as variadic such as `min` and `max`. (The latter compute min and max of any number of numerical arguments.) There is no notion of optional or default-on-absent arguments. All argument-passing is positional rather than by name; arguments are passed by value, not by reference.
-
-You can get a list of all functions using **mlr -f**, with details using **mlr -F**.
-
-GENMD_RUN_CONTENT_GENERATOR(./mk-func-h2s.sh)
+GENMD_RUN_CONTENT_GENERATOR(./mk-func-info.rb)

docs6/docs/reference-dsl-differences.md

@@ -144,6 +144,28 @@ avoid this, use the dot operator for string-concatenation instead.
 
 Similarly, a final newline is printed for you; use [`printn`](reference-dsl-output-statements.md#print-statements) to avoid this.
 
+## String literals with double quotes only
+
+In some languages, like Ruby and Bash, string literals can be in single quotes or double quotes,
+where single quotes suppress `\n` converting to a newline character and double quotes allowing it:
+`'a\nb'` prints as the four characters `a`, `\`, `n`, and `b` on one line; `"a\nb"` prints as an
+`a` on one line and a `b` on another.
+
+In others, like Python and JavaScript, string literals can be in single quotes or double quotes,
+interchangeably -- so you can have `"don't"` or `'the "right" thing'` as you wish.
+
+In yet others, such as C/C++ and Java, string literals are in double auotes, like `"abc"`,
+while single quotes are for character literals like `'a'` or `'\n'`. In these, if `s` is a non-empty string,
+then `s[0]` is its first character.
+
+In the [Miller programming language](programming-language.md):
+
+* String literals are always in double quotes, like `"abc"`.
+* String-indexing/slicing always results in strings (even of length 1): `"abc"[1:1]` is the string `"a"`, and there is no notion in the Miller programming language of a character type.
+* The single-quote character plays no role whatsoever in the grammar of the Miller programming language.
+* Single quotes are reserved for wrapping expressions at the system command line. For example, in `mlr put '$message = "hello"' ...`, the [`put` verb](reference-dsl.md) gets the string `$message = "hello"`; the shell has consumed the outer single quotes by the time the Miller parser receives it.
+* Things are a little different on Windows, where `"""` sequences are sometimes necessary: see the [Miller on Windows page](miller-on-windows.md).
+
 ## Absent-null
 
 Miller has a somewhat novel flavor of null data called _absent_: if a record
@@ -194,3 +216,12 @@ Miller has a [key-value loop flavor](reference-dsl-control-structures.md#key-val
 ## Semantics for one-variable for-loops
 
 Miller also has a [single-variable loop flavor](reference-dsl-control-structures.md#single-variable-for-loops). If `x` is a map then `for (e in x) { ... }` binds `e` to successive map _keys_ (not values as in PHP). But if `x` is an array then `for e in x) { ... }` binds `e` to successive array _values_ (not indices).
+
+## JSON parse, stringify, decode, and encode
+
+Miller has the verbs
+[`json-parse`](reference-verbs.md#json-parse) and
+[`json-stringify`](reference-verbs.md#json-stringify), and the DSL functions
+[`json_parse`](reference-dsl-builtin-functions.md#json_parse) and
+[`json_stringify`](reference-dsl-builtin-functions.md#json_stringify).
+In some other lannguages these are called `json_decode` and `json_encode`.

docs6/docs/reference-dsl-differences.md.in

@@ -116,6 +116,28 @@ GENMD_EOF
 
 Similarly, a final newline is printed for you; use [`printn`](reference-dsl-output-statements.md#print-statements) to avoid this.
 
+## String literals with double quotes only
+
+In some languages, like Ruby and Bash, string literals can be in single quotes or double quotes,
+where single quotes suppress `\n` converting to a newline character and double quotes allowing it:
+`'a\nb'` prints as the four characters `a`, `\`, `n`, and `b` on one line; `"a\nb"` prints as an
+`a` on one line and a `b` on another.
+
+In others, like Python and JavaScript, string literals can be in single quotes or double quotes,
+interchangeably -- so you can have `"don't"` or `'the "right" thing'` as you wish.
+
+In yet others, such as C/C++ and Java, string literals are in double auotes, like `"abc"`,
+while single quotes are for character literals like `'a'` or `'\n'`. In these, if `s` is a non-empty string,
+then `s[0]` is its first character.
+
+In the [Miller programming language](programming-language.md):
+
+* String literals are always in double quotes, like `"abc"`.
+* String-indexing/slicing always results in strings (even of length 1): `"abc"[1:1]` is the string `"a"`, and there is no notion in the Miller programming language of a character type.
+* The single-quote character plays no role whatsoever in the grammar of the Miller programming language.
+* Single quotes are reserved for wrapping expressions at the system command line. For example, in `mlr put '$message = "hello"' ...`, the [`put` verb](reference-dsl.md) gets the string `$message = "hello"`; the shell has consumed the outer single quotes by the time the Miller parser receives it.
+* Things are a little different on Windows, where `"""` sequences are sometimes necessary: see the [Miller on Windows page](miller-on-windows.md).
+
 ## Absent-null
 
 Miller has a somewhat novel flavor of null data called _absent_: if a record
@@ -155,3 +177,12 @@ Miller has a [key-value loop flavor](reference-dsl-control-structures.md#key-val
 ## Semantics for one-variable for-loops
 
 Miller also has a [single-variable loop flavor](reference-dsl-control-structures.md#single-variable-for-loops). If `x` is a map then `for (e in x) { ... }` binds `e` to successive map _keys_ (not values as in PHP). But if `x` is an array then `for e in x) { ... }` binds `e` to successive array _values_ (not indices).
+
+## JSON parse, stringify, decode, and encode
+
+Miller has the verbs
+[`json-parse`](reference-verbs.md#json-parse) and
+[`json-stringify`](reference-verbs.md#json-stringify), and the DSL functions
+[`json_parse`](reference-dsl-builtin-functions.md#json_parse) and
+[`json_stringify`](reference-dsl-builtin-functions.md#json_stringify).
+In some other lannguages these are called `json_decode` and `json_encode`.

docs6/docs/reference-main-arrays.md

@@ -162,9 +162,9 @@ If you want to auto-extend an [array](reference-main-arrays.md), initialize it e
 
 Once an array is initialized, it can be extended by assigning to indices beyond
 its length.  If each write is one past the end of the array, the array will
-grow by one. (Memory management, handled for you, is a little more careful: not
-to worry, capacity is doubled so performance doesn't suffer a rellocate on
-every single extend.)
+grow by one. (Memory management, handled for you, is careful handled here in
+Miller: not to worry, capacity is doubled so performance doesn't suffer a
+rellocate on every single extend.)
 
 This is important in Miller so you can do things like `@records[NR] = $*` with
 a minimum of keystrokes without worrying about explicitly resizing arrays. In

docs6/docs/reference-main-arrays.md.in

@@ -108,9 +108,9 @@ GENMD_EOF
 
 Once an array is initialized, it can be extended by assigning to indices beyond
 its length.  If each write is one past the end of the array, the array will
-grow by one. (Memory management, handled for you, is a little more careful: not
-to worry, capacity is doubled so performance doesn't suffer a rellocate on
-every single extend.)
+grow by one. (Memory management, handled for you, is careful handled here in
+Miller: not to worry, capacity is doubled so performance doesn't suffer a
+rellocate on every single extend.)
 
 This is important in Miller so you can do things like `@records[NR] = $*` with
 a minimum of keystrokes without worrying about explicitly resizing arrays. In

docs6/docs/reference-main-data-types.md

@@ -16,62 +16,193 @@ Quick links:
 
 ## List of types
 
-Miller's types for function processing are:
+Miller's types are:
 
 * Scalars:
-    * **string**
-    * **float** (double-precision) and **int** (64-bit signed): see the [Arithmetic](reference-main-arithmetic.md) page
-    * **boolean**
+    * **string**: such as `"abcdefg"`, supporting concatenation, one-up indexing and slicing, and [library functions](reference-dsl-builtin-functions.md#string-functions). See the pages on [strings](reference-main-strings.md) and [regular expressions](reference-main-regular-expressions.md).
+    * **float** and **int**: such as `1.2` and `3`: double-precision and 64-bit signed, respectively. See the section on [arithmetic operators and math-related library functions](reference-dsl-builtin-functions.md#math-functions) as well as the [Arithmetic](reference-main-arithmetic.md) page.
+    * **boolean**: literals `true` and `false`; results of `==`, `<`, `>`, etc. See the section on [boolean operators](reference-dsl-builtin-functions.md#boolean-functions).
 * Collections:
-    * **map**: see the [Maps](reference-main-maps.md) page
-    * **array**: see the [Arrays](reference-main-arrays.md) page
+    * **map**: such as `{"a":1,"b":[2,3,4]}`, supporting key-indexing, preservation of insertion order, [library functions](reference-dsl-builtin-functions.md#collections-functions), etc. See the [Maps](reference-main-maps.md) page.
+    * **array**: such as `["a", 2, true]`, supporting one-up indexing and slicing, [library functions](reference-dsl-builtin-functions.md#collections-functions), etc. See the [Arrays](reference-main-arrays.md) page.
 * Nulls and error:
-    * **absent-null** (reads of unset right-hand sides, or fall-through non-explicit return values from user-defined functions) and **JSON-null**: see the [null-data page](reference-main-null-data.md)
-    * **error** (TODO: give an example)
+    * **absent-null**: Such as on reads of unset right-hand sides, or fall-through non-explicit return values from user-defined functions. See the [null-data page](reference-main-null-data.md).
+    * **JSON-null**: For `null` in JSON files; also used in [gapped auto-extend of arrays](reference-main-arrays.md#auto-extend-and-null-gaps). See the [null-data page](reference-main-null-data.md).
+    * **error** -- for various results which cannot be computed, often when the input to a [built-in function](reference-dsl-builtin-functions.md) is of the wrong type. For example, doing [strlen](reference-dsl-builtin-functions.md#strlen) or [substr](reference-dsl-builtin-functions.md#substr) on a non-string, [sec2gmt](reference-dsl-builtin-functions.md#sec2gmt) on a non-integer, etc.
 
-TODO: point to null-data page; arrays page; arithmetic page; what else?
+See also the list of [type-checking
+functions](reference-dsl-builtin-functions.md#type-checkin -functions) for the
+[Miller programming language](programming-language.md).
 
-## To be ported
+See also [Differences from other programming languages](reference-dsl-differences.md).
 
-Miller's input and output are all string-oriented: there is (as of August 2015 anyway) no support for binary record packing. In this sense, everything is a string in and out of Miller.  During processing, field names are always strings, even if they have names like "3"; field values can have various data types.  Field values' ability to be interpreted as a non-string type only has meaning when comparison or function operations are done on them.  And it is an error condition if Miller encounters non-numeric (or otherwise mistyped) data in a field in which it has been asked to do numeric (or otherwise type-specific) operations.
+## Type inference for literal and record data
 
-Field values are treated as numeric for the following:
+Miller's input and output are all text-oriented: all the
+[file formats supported by Miller](file-formats.md) are human-readable text,
+such as CSV, TSV, and JSON; binary formats such as
+[BSON](https://bsonspec.org/) and [Parquet](https://parquet.apache.org/) are
+not supported (as of mid-2021). In this sense, everything is a string in and out of
+Miller -- be it in data files, or in DSL expressions you key in.
 
-* Numeric sort: `mlr sort -n`, `mlr sort -nr`.
-* Statistics: `mlr histogram`, `mlr stats1`, `mlr stats2`.
-* Cross-record arithmetic: `mlr step`.
+In the [DSL](programming-language.md), `7` is an `int` and `8.9` is a float, as
+one would expect.  Likewise, on input from [data files](file-formats.md),
+string values representable as numbers, e.g. `1.2` or `3`, are treated as int
+or float, respectively. If a record has `x=1,y=2` then `mlr put '$z=$x+$y'`
+will produce `x=1,y=2,z=3`.
 
-For `mlr put` and `mlr filter`:
+Numbers retain their original string representation, so if `x` is `1.2` on one
+record and `1.200` on another, they'll print out that way on output (unless of
+course they've been modified during processing, e.g. `mlr put '$x = $x + 10`).
 
-* Miller's types for function processing are **empty-null** (empty string), **absent-null** (reads of unset right-hand sides, or fall-through non-explicit return values from user-defined functions), **error**, **string**, **float** (double-precision), **int** (64-bit signed), and **boolean**.
+Generally strings, numbers, and booleans don't mix; use type-casting like
+`string($x)` to convert. However, the dot (string-concatenation) operator has
+been special-cased: `mlr put '$z=$x.$y'` does not give an error, because the
+dot operator has been generalized to stringify non-strings
 
-* On input, string values representable as numbers, e.g. "3" or "3.1", are treated as int or float, respectively. If a record has `x=1,y=2` then `mlr put '$z=$x+$y'` will produce `x=1,y=2,z=3`, and `mlr put '$z=$x.$y'` does not give an error, but only because the dot operator has been generalized to stringify non-strings.  To coerce back to string for processing, use the `string` function: `mlr put '$z=string($x).string($y)'` will produce `x=1,y=2,z=12`.
+Examples:
 
-* On input, string values representable as boolean  (e.g. `"true"`, `"false"`) are *not* automatically treated as boolean.  (This is because `"true"` and `"false"` are ordinary words, and auto string-to-boolean on a column consisting of words would result in some strings mixed with some booleans.) Use the `boolean` function to coerce: e.g. giving the record `x=1,y=2,w=false` to `mlr put '$z=($x<$y) || boolean($w)'`.
+<pre class="pre-highlight-in-pair">
+<b>mlr --csv cat data/type-infer.csv</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+a,b,c
+1.2,3,true
+4,5.6,buongiorno
+</pre>
 
-* Functions take types as described in `mlr --help-all-functions`: for example, `log10` takes float input and produces float output, `gmt2sec` maps string to int, and `sec2gmt` maps int to string.
+<pre class="pre-highlight-in-pair">
+<b>mlr --icsv --oxtab --from data/type-infer.csv put '</b>
+<b>  $d = $a . $c;</b>
+<b>  $e = 7;</b>
+<b>  $f = 8.9;</b>
+<b>  $g = $e + $f;</b>
+<b>  $ta = typeof($a);</b>
+<b>  $tb = typeof($b);</b>
+<b>  $tc = typeof($c);</b>
+<b>  $td = typeof($d);</b>
+<b>  $te = typeof($e);</b>
+<b>  $tf = typeof($f);</b>
+<b>  $tg = typeof($g);</b>
+<b>' then reorder -f a,ta,b,tb,c,tc,d,td,e,te,f,tf,g,tg</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+a  1.2
+ta float
+b  3
+tb int
+c  true
+tc string
+d  1.2true
+td string
+e  7
+te int
+f  8.9
+tf float
+g  15.9
+tg float
 
-* All math functions described in `mlr --help-all-functions` take integer as well as float input.
+a  4
+ta int
+b  5.6
+tb float
+c  buongiorno
+tc string
+d  4buongiorno
+td string
+e  7
+te int
+f  8.9
+tf float
+g  15.9
+tg float
+</pre>
 
-## Escape sequences for string literals
+On input, string values representable as boolean  (e.g. `"true"`, `"false"`)
+are *not* automatically treated as boolean.  This is because `"true"` and
+`"false"` are ordinary words, and auto string-to-boolean on a column consisting
+of words would result in some strings mixed with some booleans. Use the
+`boolean` function to coerce: e.g. giving the record `x=1,y=2,w=false` to `mlr
+filter '$z=($x<$y) || boolean($w)'`.
 
-You can use the following backslash escapes for strings such as between the double quotes in contexts such as `mlr filter '$name =~ "..."'`, `mlr put '$name = $othername . "..."'`, `mlr put '$name = sub($name, "...", "...")`, etc.:
+The same is true for `inf`, `+inf`, `-inf`, `infinity`, `+infinity`,
+`-infinity`, `NaN`, and all upper-cased/lower-cased/mixed-case variants of
+those. These are valid IEEE floating-point numbers, but Miller treats these as
+strings. You can explicit force conversion: if `x=infinity` in a data file,
+then `typeof($x)` is `string` but `typeof(float($x))` is `float`.
 
-* `\a`: ASCII code 0x07 (alarm/bell)
-* `\b`: ASCII code 0x08 (backspace)
-* `\f`: ASCII code 0x0c (formfeed)
-* `\n`: ASCII code 0x0a (LF/linefeed/newline)
-* `\r`: ASCII code 0x0d (CR/carriage return)
-* `\t`: ASCII code 0x09 (tab)
-* `\v`: ASCII code 0x0b (vertical tab)
-* `\\`: backslash
-* `\"`: double quote
-* `\123`: Octal 123, etc. for `\000` up to `\377`
-* `\x7f`: Hexadecimal 7f, etc. for `\x00` up to `\xff`
+## JSON parse and stringify
 
-See also [https://en.wikipedia.org/wiki/Escape_sequences_in_C](https://en.wikipedia.org/wiki/Escape_sequences_in_C).
+If you have, say, a CSV file whose columns contain strings which are well-formatted JSON,
+they will not be auto-converted, but you can use the
+[`json-parse` verb](reference-verbs.md#json-parse)
+or the
+[`json_parse` DSL function](reference-dsl-builtin-functions.md#json_parse):
 
-These replacements apply only to strings you key in for the DSL expressions for `filter` and `put`: that is, if you type `\t` in a string literal for a `filter`/`put` expression, it will be turned into a tab character. If you want a backslash followed by a `t`, then please type `\\t`.
+<pre class="pre-highlight-in-pair">
+<b>mlr --csv --from data/json-in-csv.csv cat</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+id,blob
+100,"{""a"":1,""b"":[2,3,4]}"
+105,"{""a"":6,""b"":[7,8,9]}"
+</pre>
 
-However, these replacements are done automatically only for string literals within DSL expressions -- they are not done automatically to fields within your data stream.  If you wish to make these replacements, you can do (for example) `mlr put '$field = gsub($field, "\\t", "\t")'`. If you need to make such a replacement for all fields in your data, you should probably use the system `sed` command instead. 
+<pre class="pre-highlight-in-pair">
+<b>mlr --icsv --ojson --from data/json-in-csv.csv cat</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+{
+  "id": 100,
+  "blob": "{\"a\":1,\"b\":[2,3,4]}"
+}
+{
+  "id": 105,
+  "blob": "{\"a\":6,\"b\":[7,8,9]}"
+}
+</pre>
 
+<pre class="pre-highlight-in-pair">
+<b>mlr --icsv --ojson --from data/json-in-csv.csv json-parse -f blob</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+{
+  "id": 100,
+  "blob": {
+    "a": 1,
+    "b": [2, 3, 4]
+  }
+}
+{
+  "id": 105,
+  "blob": {
+    "a": 6,
+    "b": [7, 8, 9]
+  }
+}
+</pre>
+
+<pre class="pre-highlight-in-pair">
+<b>mlr --icsv --ojson --from data/json-in-csv.csv put '$blob = json_parse($blob)'</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+{
+  "id": 100,
+  "blob": {
+    "a": 1,
+    "b": [2, 3, 4]
+  }
+}
+{
+  "id": 105,
+  "blob": {
+    "a": 6,
+    "b": [7, 8, 9]
+  }
+}
+</pre>
+
+These have their respective operations to convert back to string: the
+[`json-stringify` verb](reference-verbs.md#json-stringify)
+and
+[`json_stringify` DSL function](reference-dsl-builtin-functions.md#json_stringify).

docs6/docs/reference-main-data-types.md.in

@@ -2,62 +2,110 @@
 
 ## List of types
 
-Miller's types for function processing are:
+Miller's types are:
 
 * Scalars:
-    * **string**
-    * **float** (double-precision) and **int** (64-bit signed): see the [Arithmetic](reference-main-arithmetic.md) page
-    * **boolean**
+    * **string**: such as `"abcdefg"`, supporting concatenation, one-up indexing and slicing, and [library functions](reference-dsl-builtin-functions.md#string-functions). See the pages on [strings](reference-main-strings.md) and [regular expressions](reference-main-regular-expressions.md).
+    * **float** and **int**: such as `1.2` and `3`: double-precision and 64-bit signed, respectively. See the section on [arithmetic operators and math-related library functions](reference-dsl-builtin-functions.md#math-functions) as well as the [Arithmetic](reference-main-arithmetic.md) page.
+    * **boolean**: literals `true` and `false`; results of `==`, `<`, `>`, etc. See the section on [boolean operators](reference-dsl-builtin-functions.md#boolean-functions).
 * Collections:
-    * **map**: see the [Maps](reference-main-maps.md) page
-    * **array**: see the [Arrays](reference-main-arrays.md) page
+    * **map**: such as `{"a":1,"b":[2,3,4]}`, supporting key-indexing, preservation of insertion order, [library functions](reference-dsl-builtin-functions.md#collections-functions), etc. See the [Maps](reference-main-maps.md) page.
+    * **array**: such as `["a", 2, true]`, supporting one-up indexing and slicing, [library functions](reference-dsl-builtin-functions.md#collections-functions), etc. See the [Arrays](reference-main-arrays.md) page.
 * Nulls and error:
-    * **absent-null** (reads of unset right-hand sides, or fall-through non-explicit return values from user-defined functions) and **JSON-null**: see the [null-data page](reference-main-null-data.md)
-    * **error** (TODO: give an example)
+    * **absent-null**: Such as on reads of unset right-hand sides, or fall-through non-explicit return values from user-defined functions. See the [null-data page](reference-main-null-data.md).
+    * **JSON-null**: For `null` in JSON files; also used in [gapped auto-extend of arrays](reference-main-arrays.md#auto-extend-and-null-gaps). See the [null-data page](reference-main-null-data.md).
+    * **error** -- for various results which cannot be computed, often when the input to a [built-in function](reference-dsl-builtin-functions.md) is of the wrong type. For example, doing [strlen](reference-dsl-builtin-functions.md#strlen) or [substr](reference-dsl-builtin-functions.md#substr) on a non-string, [sec2gmt](reference-dsl-builtin-functions.md#sec2gmt) on a non-integer, etc.
 
-TODO: point to null-data page; arrays page; arithmetic page; what else?
+See also the list of [type-checking
+functions](reference-dsl-builtin-functions.md#type-checkin -functions) for the
+[Miller programming language](programming-language.md).
 
-## To be ported
+See also [Differences from other programming languages](reference-dsl-differences.md).
 
-Miller's input and output are all string-oriented: there is (as of August 2015 anyway) no support for binary record packing. In this sense, everything is a string in and out of Miller.  During processing, field names are always strings, even if they have names like "3"; field values can have various data types.  Field values' ability to be interpreted as a non-string type only has meaning when comparison or function operations are done on them.  And it is an error condition if Miller encounters non-numeric (or otherwise mistyped) data in a field in which it has been asked to do numeric (or otherwise type-specific) operations.
+## Type inference for literal and record data
 
-Field values are treated as numeric for the following:
+Miller's input and output are all text-oriented: all the
+[file formats supported by Miller](file-formats.md) are human-readable text,
+such as CSV, TSV, and JSON; binary formats such as
+[BSON](https://bsonspec.org/) and [Parquet](https://parquet.apache.org/) are
+not supported (as of mid-2021). In this sense, everything is a string in and out of
+Miller -- be it in data files, or in DSL expressions you key in.
 
-* Numeric sort: `mlr sort -n`, `mlr sort -nr`.
-* Statistics: `mlr histogram`, `mlr stats1`, `mlr stats2`.
-* Cross-record arithmetic: `mlr step`.
+In the [DSL](programming-language.md), `7` is an `int` and `8.9` is a float, as
+one would expect.  Likewise, on input from [data files](file-formats.md),
+string values representable as numbers, e.g. `1.2` or `3`, are treated as int
+or float, respectively. If a record has `x=1,y=2` then `mlr put '$z=$x+$y'`
+will produce `x=1,y=2,z=3`.
 
-For `mlr put` and `mlr filter`:
+Numbers retain their original string representation, so if `x` is `1.2` on one
+record and `1.200` on another, they'll print out that way on output (unless of
+course they've been modified during processing, e.g. `mlr put '$x = $x + 10`).
 
-* Miller's types for function processing are **empty-null** (empty string), **absent-null** (reads of unset right-hand sides, or fall-through non-explicit return values from user-defined functions), **error**, **string**, **float** (double-precision), **int** (64-bit signed), and **boolean**.
+Generally strings, numbers, and booleans don't mix; use type-casting like
+`string($x)` to convert. However, the dot (string-concatenation) operator has
+been special-cased: `mlr put '$z=$x.$y'` does not give an error, because the
+dot operator has been generalized to stringify non-strings
 
-* On input, string values representable as numbers, e.g. "3" or "3.1", are treated as int or float, respectively. If a record has `x=1,y=2` then `mlr put '$z=$x+$y'` will produce `x=1,y=2,z=3`, and `mlr put '$z=$x.$y'` does not give an error, but only because the dot operator has been generalized to stringify non-strings.  To coerce back to string for processing, use the `string` function: `mlr put '$z=string($x).string($y)'` will produce `x=1,y=2,z=12`.
+Examples:
 
-* On input, string values representable as boolean  (e.g. `"true"`, `"false"`) are *not* automatically treated as boolean.  (This is because `"true"` and `"false"` are ordinary words, and auto string-to-boolean on a column consisting of words would result in some strings mixed with some booleans.) Use the `boolean` function to coerce: e.g. giving the record `x=1,y=2,w=false` to `mlr put '$z=($x<$y) || boolean($w)'`.
+GENMD_RUN_COMMAND
+mlr --csv cat data/type-infer.csv
+GENMD_EOF
 
-* Functions take types as described in `mlr --help-all-functions`: for example, `log10` takes float input and produces float output, `gmt2sec` maps string to int, and `sec2gmt` maps int to string.
+GENMD_RUN_COMMAND
+mlr --icsv --oxtab --from data/type-infer.csv put '
+  $d = $a . $c;
+  $e = 7;
+  $f = 8.9;
+  $g = $e + $f;
+  $ta = typeof($a);
+  $tb = typeof($b);
+  $tc = typeof($c);
+  $td = typeof($d);
+  $te = typeof($e);
+  $tf = typeof($f);
+  $tg = typeof($g);
+' then reorder -f a,ta,b,tb,c,tc,d,td,e,te,f,tf,g,tg
+GENMD_EOF
 
-* All math functions described in `mlr --help-all-functions` take integer as well as float input.
+On input, string values representable as boolean  (e.g. `"true"`, `"false"`)
+are *not* automatically treated as boolean.  This is because `"true"` and
+`"false"` are ordinary words, and auto string-to-boolean on a column consisting
+of words would result in some strings mixed with some booleans. Use the
+`boolean` function to coerce: e.g. giving the record `x=1,y=2,w=false` to `mlr
+filter '$z=($x<$y) || boolean($w)'`.
 
-## Escape sequences for string literals
+The same is true for `inf`, `+inf`, `-inf`, `infinity`, `+infinity`,
+`-infinity`, `NaN`, and all upper-cased/lower-cased/mixed-case variants of
+those. These are valid IEEE floating-point numbers, but Miller treats these as
+strings. You can explicit force conversion: if `x=infinity` in a data file,
+then `typeof($x)` is `string` but `typeof(float($x))` is `float`.
 
-You can use the following backslash escapes for strings such as between the double quotes in contexts such as `mlr filter '$name =~ "..."'`, `mlr put '$name = $othername . "..."'`, `mlr put '$name = sub($name, "...", "...")`, etc.:
+## JSON parse and stringify
 
-* `\a`: ASCII code 0x07 (alarm/bell)
-* `\b`: ASCII code 0x08 (backspace)
-* `\f`: ASCII code 0x0c (formfeed)
-* `\n`: ASCII code 0x0a (LF/linefeed/newline)
-* `\r`: ASCII code 0x0d (CR/carriage return)
-* `\t`: ASCII code 0x09 (tab)
-* `\v`: ASCII code 0x0b (vertical tab)
-* `\\`: backslash
-* `\"`: double quote
-* `\123`: Octal 123, etc. for `\000` up to `\377`
-* `\x7f`: Hexadecimal 7f, etc. for `\x00` up to `\xff`
+If you have, say, a CSV file whose columns contain strings which are well-formatted JSON,
+they will not be auto-converted, but you can use the
+[`json-parse` verb](reference-verbs.md#json-parse)
+or the
+[`json_parse` DSL function](reference-dsl-builtin-functions.md#json_parse):
 
-See also [https://en.wikipedia.org/wiki/Escape_sequences_in_C](https://en.wikipedia.org/wiki/Escape_sequences_in_C).
+GENMD_RUN_COMMAND
+mlr --csv --from data/json-in-csv.csv cat
+GENMD_EOF
 
-These replacements apply only to strings you key in for the DSL expressions for `filter` and `put`: that is, if you type `\t` in a string literal for a `filter`/`put` expression, it will be turned into a tab character. If you want a backslash followed by a `t`, then please type `\\t`.
+GENMD_RUN_COMMAND
+mlr --icsv --ojson --from data/json-in-csv.csv cat
+GENMD_EOF
 
-However, these replacements are done automatically only for string literals within DSL expressions -- they are not done automatically to fields within your data stream.  If you wish to make these replacements, you can do (for example) `mlr put '$field = gsub($field, "\\t", "\t")'`. If you need to make such a replacement for all fields in your data, you should probably use the system `sed` command instead. 
+GENMD_RUN_COMMAND
+mlr --icsv --ojson --from data/json-in-csv.csv json-parse -f blob
+GENMD_EOF
 
+GENMD_RUN_COMMAND
+mlr --icsv --ojson --from data/json-in-csv.csv put '$blob = json_parse($blob)'
+GENMD_EOF
+
+These have their respective operations to convert back to string: the
+[`json-stringify` verb](reference-verbs.md#json-stringify)
+and
+[`json_stringify` DSL function](reference-dsl-builtin-functions.md#json_stringify).

docs6/docs/reference-main-maps.md

@@ -82,7 +82,7 @@ The current record, accessible using `$*`, is a map.
 {
   "color": "yellow",
   "shape": "triangle",
-  "flag": true,
+  "flag": "true",
   "k": 1,
   "index": 11,
   "quantity": 43.6498,
@@ -92,7 +92,7 @@ Color is yellow
 {
   "color": "red",
   "shape": "square",
-  "flag": true,
+  "flag": "true",
   "k": 2,
   "index": 15,
   "quantity": 79.2778,

docs6/docs/reference-main-null-data.md

@@ -22,7 +22,7 @@ Miller has three kinds of null data:
 
 * **Absent (key not present)**: a field name is not present, e.g. input record is `x=1,y=2` and a `put` or `filter` expression refers to `$z`. Or, reading an out-of-stream variable which hasn't been assigned a value yet, e.g.  `mlr put -q '@sum += $x; end{emit @sum}'` or `mlr put -q '@sum[$a][$b] += $x; end{emit @sum, "a", "b"}'`.
 
-* **JSON null**: (TODO: describe)
+* **JSON null**: The main purpose of this is to support reading the `null` type in JSON files. The [Miller programming language](programming-language.md) has a `null` keyword as well, so you can also write the null type using `$x = null`. Addtionally, though, when you write past the end of an array, leaving gaps -- e.g. writing `a[12]` when the array `a` has length 10 -- JSON-null is used to fill the gaps. See also the [arrays page](reference-main-arrays.md#auto-extend-and-null-gaps).
 
 You can test these programatically using the functions `is_empty`/`is_not_empty`, `is_absent`/`is_present`, and `is_null`/`is_not_null`. For the last pair, note that null means either empty or absent.
 

docs6/docs/reference-main-null-data.md.in

@@ -8,7 +8,7 @@ Miller has three kinds of null data:
 
 * **Absent (key not present)**: a field name is not present, e.g. input record is `x=1,y=2` and a `put` or `filter` expression refers to `$z`. Or, reading an out-of-stream variable which hasn't been assigned a value yet, e.g.  `mlr put -q '@sum += $x; end{emit @sum}'` or `mlr put -q '@sum[$a][$b] += $x; end{emit @sum, "a", "b"}'`.
 
-* **JSON null**: (TODO: describe)
+* **JSON null**: The main purpose of this is to support reading the `null` type in JSON files. The [Miller programming language](programming-language.md) has a `null` keyword as well, so you can also write the null type using `$x = null`. Addtionally, though, when you write past the end of an array, leaving gaps -- e.g. writing `a[12]` when the array `a` has length 10 -- JSON-null is used to fill the gaps. See also the [arrays page](reference-main-arrays.md#auto-extend-and-null-gaps).
 
 You can test these programatically using the functions `is_empty`/`is_not_empty`, `is_absent`/`is_present`, and `is_null`/`is_not_null`. For the last pair, note that null means either empty or absent.
 

docs6/docs/reference-main-strings.md

@@ -0,0 +1,52 @@
+<!---  PLEASE DO NOT EDIT DIRECTLY. EDIT THE .md.in FILE PLEASE. --->
+<div>
+<span class="quicklinks">
+Quick links:
+&nbsp;
+<a class="quicklink" href="../reference-verbs/index.html">Verb list</a>
+&nbsp;
+<a class="quicklink" href="../reference-dsl-builtin-functions/index.html">Function list</a>
+&nbsp;
+<a class="quicklink" href="../glossary/index.html">Glossary</a>
+&nbsp;
+<a class="quicklink" href="https://github.com/johnkerl/miller" target="_blank">Repository ↗</a>
+</span>
+</div>
+# Strings
+
+TODO
+
+xxx concat
+
+xxx index and slice 1-up
+
+xxx lib functions
+
+always double-quote
+
+single-quote for shell; see windows page
+
+dot operator ...
+
+## Escape sequences for string literals
+
+You can use the following backslash escapes for strings such as between the double quotes in contexts such as `mlr filter '$name =~ "..."'`, `mlr put '$name = $othername . "..."'`, `mlr put '$name = sub($name, "...", "...")`, etc.:
+
+* `\a`: ASCII code 0x07 (alarm/bell)
+* `\b`: ASCII code 0x08 (backspace)
+* `\f`: ASCII code 0x0c (formfeed)
+* `\n`: ASCII code 0x0a (LF/linefeed/newline)
+* `\r`: ASCII code 0x0d (CR/carriage return)
+* `\t`: ASCII code 0x09 (tab)
+* `\v`: ASCII code 0x0b (vertical tab)
+* `\\`: backslash
+* `\"`: double quote
+* `\123`: Octal 123, etc. for `\000` up to `\377`
+* `\x7f`: Hexadecimal 7f, etc. for `\x00` up to `\xff`
+
+See also [https://en.wikipedia.org/wiki/Escape_sequences_in_C](https://en.wikipedia.org/wiki/Escape_sequences_in_C).
+
+These replacements apply only to strings you key in for the DSL expressions for `filter` and `put`: that is, if you type `\t` in a string literal for a `filter`/`put` expression, it will be turned into a tab character. If you want a backslash followed by a `t`, then please type `\\t`.
+
+However, these replacements are done automatically only for string literals within DSL expressions -- they are not done automatically to fields within your data stream.  If you wish to make these replacements, you can do (for example) `mlr put '$field = gsub($field, "\\t", "\t")'`. If you need to make such a replacement for all fields in your data, you should probably use the system `sed` command instead. 
+

docs6/docs/reference-main-strings.md.in

@@ -0,0 +1,38 @@
+# Strings
+
+TODO
+
+xxx concat
+
+xxx index and slice 1-up
+
+xxx lib functions
+
+always double-quote
+
+single-quote for shell; see windows page
+
+dot operator ...
+
+## Escape sequences for string literals
+
+You can use the following backslash escapes for strings such as between the double quotes in contexts such as `mlr filter '$name =~ "..."'`, `mlr put '$name = $othername . "..."'`, `mlr put '$name = sub($name, "...", "...")`, etc.:
+
+* `\a`: ASCII code 0x07 (alarm/bell)
+* `\b`: ASCII code 0x08 (backspace)
+* `\f`: ASCII code 0x0c (formfeed)
+* `\n`: ASCII code 0x0a (LF/linefeed/newline)
+* `\r`: ASCII code 0x0d (CR/carriage return)
+* `\t`: ASCII code 0x09 (tab)
+* `\v`: ASCII code 0x0b (vertical tab)
+* `\\`: backslash
+* `\"`: double quote
+* `\123`: Octal 123, etc. for `\000` up to `\377`
+* `\x7f`: Hexadecimal 7f, etc. for `\x00` up to `\xff`
+
+See also [https://en.wikipedia.org/wiki/Escape_sequences_in_C](https://en.wikipedia.org/wiki/Escape_sequences_in_C).
+
+These replacements apply only to strings you key in for the DSL expressions for `filter` and `put`: that is, if you type `\t` in a string literal for a `filter`/`put` expression, it will be turned into a tab character. If you want a backslash followed by a `t`, then please type `\\t`.
+
+However, these replacements are done automatically only for string literals within DSL expressions -- they are not done automatically to fields within your data stream.  If you wish to make these replacements, you can do (for example) `mlr put '$field = gsub($field, "\\t", "\t")'`. If you need to make such a replacement for all fields in your data, you should probably use the system `sed` command instead. 
+

docs6/docs/reference-verbs.md

@@ -1639,6 +1639,34 @@ left_a left_b left_c right_a right_b right_c
 1      4      5      1       4       5
 </pre>
 
+## json-parse
+
+<pre class="pre-highlight-in-pair">
+<b>mlr json-parse --help</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+Usage: mlr json-parse [options]
+Tries to convert string field values to parsed JSON, e.g. "[1,2,3]" -> [1,2,3].
+Options:
+-f {...} Comma-separated list of field names to json-parse (default all).
+-h|--help Show this message.
+</pre>
+
+## json-stringify
+
+<pre class="pre-highlight-in-pair">
+<b>mlr json-stringify --help</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+Usage: mlr json-stringify [options]
+Produces string field values from field-value data, e.g. [1,2,3] -> "[1,2,3]".
+Options:
+-f {...} Comma-separated list of field names to json-parse (default all).
+--jvstack Produce multi-line JSON output.
+--no-jvstack Produce single-line JSON output per record (default).
+-h|--help Show this message.
+</pre>
+
 ## label
 
 <pre class="pre-highlight-in-pair">

docs6/docs/reference-verbs.md.in

@@ -548,6 +548,18 @@ GENMD_RUN_COMMAND
 mlr --csvlite --opprint join -j "" --lp left_ --rp right_ -f data/self-join.csv data/self-join.csv
 GENMD_EOF
 
+## json-parse
+
+GENMD_RUN_COMMAND
+mlr json-parse --help
+GENMD_EOF
+
+## json-stringify
+
+GENMD_RUN_COMMAND
+mlr json-stringify --help
+GENMD_EOF
+
 ## label
 
 GENMD_RUN_COMMAND

docs6/docs/repl.md

@@ -48,7 +48,7 @@ HELLO
 {
   "color": "yellow",
   "shape": "triangle",
-  "flag": true,
+  "flag": "true",
   "k": 1,
   "index": 11,
   "quantity": 43.6498,
@@ -58,7 +58,7 @@ HELLO
 {
   "color": "red",
   "shape": "square",
-  "flag": true,
+  "flag": "true",
   "k": 2,
   "index": 15,
   "quantity": 79.2778,

docs6/docs/special-symbols-and-formatting.md

@@ -83,8 +83,8 @@ characters as delimiters -- here, control-A:
 <b>mlr --icsv --odkvp --ofs '\001'  cat commas.csv | cat -v</b>
 </pre>
 <pre class="pre-non-highlight-in-pair">
-Name=Xiao, Lin\001Role=administrator
-Name=Khavari, Darius\001Role=tester
+Name=Xiao, Lin^ARole=administrator
+Name=Khavari, Darius^ARole=tester
 </pre>
 
 ## How can I handle field names with special symbols in them?

docs6/mkdocs.yml

@@ -7,6 +7,9 @@ theme:
     text: Lato
     code: Lato Mono
 extra_css: ['extra.css']
+extra_javascript:
+  - https://cdnjs.cloudflare.com/ajax/libs/tablesort/5.2.1/tablesort.min.js
+  - javascripts/tables.js
 nav:
   - "Introduction": "index.md"
   - 'Getting started':
@@ -65,13 +68,16 @@ nav:
       - "Streaming processing, and memory usage": "streaming-and-memory.md"
       - "CPU/multicore usage": "cpu.md"
       - "Data types": "reference-main-data-types.md"
+      - "Strings": "reference-main-strings.md"
+      - "Regular expressions": "reference-main-regular-expressions.md"
       - "Arithmetic": "reference-main-arithmetic.md"
       - "Maps": "reference-main-maps.md"
       - "Arrays": "reference-main-arrays.md"
       - "Null/empty/absent data": "reference-main-null-data.md"
-      - "Regular expressions": "reference-main-regular-expressions.md"
       - "Flatten/unflatten: JSON vs. tabular formats": "flatten-unflatten.md"
       - "Compressed data": "reference-main-compressed-data.md"
+      - "Streaming processing, and memory usage": "streaming-and-memory.md"
+      - "CPU/multicore usage": "cpu.md"
       - "Miller environment variables": "reference-main-env-vars.md"
     - 'DSL reference':
       - "DSL overview": "reference-dsl.md"

go/regtest/cases/dsl-min-max-types/0001/expout

@@ -1,6 +1,6 @@
 {
   "n": 1,
-  "b": true,
+  "b": "true",
   "v": "",
   "s": "abc",
   "min": {
@@ -12,19 +12,19 @@
     },
     "b": {
       "n": 1,
-      "b": true,
-      "v": true,
-      "s": true
+      "b": "true",
+      "v": "",
+      "s": "abc"
     },
     "v": {
       "n": 1,
-      "b": true,
+      "b": "",
       "v": "",
       "s": ""
     },
     "s": {
       "n": 1,
-      "b": true,
+      "b": "abc",
       "v": "",
       "s": "abc"
     }

go/regtest/cases/dsl-min-max-types/0002/expout

@@ -1,30 +1,30 @@
 {
   "n": 1,
-  "b": true,
+  "b": "true",
   "v": "",
   "s": "abc",
   "max": {
     "n": {
       "n": 1,
-      "b": true,
+      "b": "true",
       "v": "",
       "s": "abc"
     },
     "b": {
-      "n": true,
-      "b": true,
-      "v": "",
-      "s": "abc"
+      "n": "true",
+      "b": "true",
+      "v": "true",
+      "s": "true"
     },
     "v": {
       "n": "",
-      "b": "",
+      "b": "true",
       "v": "",
       "s": "abc"
     },
     "s": {
       "n": "abc",
-      "b": "abc",
+      "b": "true",
       "v": "abc",
       "s": "abc"
     }

go/regtest/cases/dsl-min-max-types/0003/expout

@@ -1,12 +1,12 @@
 {
   "n": 1,
-  "b": true,
+  "b": "true",
   "v": "",
   "s": "abc",
   "le": {
     "n": {
       "n": true,
-      "b": false,
+      "b": true,
       "v": false,
       "s": true
     },
@@ -18,13 +18,13 @@
     },
     "v": {
       "n": true,
-      "b": false,
+      "b": true,
       "v": true,
       "s": true
     },
     "s": {
       "n": false,
-      "b": false,
+      "b": true,
       "v": false,
       "s": true
     }

go/regtest/cases/dsl-min-max-types/0004/expout

@@ -1,6 +1,6 @@
 {
   "n": 1,
-  "b": true,
+  "b": "true",
   "v": "",
   "s": "abc",
   "ge": {
@@ -11,10 +11,10 @@
       "s": false
     },
     "b": {
-      "n": false,
+      "n": true,
       "b": true,
-      "v": false,
-      "s": false
+      "v": true,
+      "s": true
     },
     "v": {
       "n": false,

go/regtest/cases/dsl-type-inference/0001/cmd

@@ -1 +1 @@
-mlr --xtab put       -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0002/cmd

@@ -1 +1 @@
-mlr --xtab filter    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab filter -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0003/cmd

@@ -1 +1 @@
-mlr --oxtab put    -f ${CASEDIR}/mlr ./${CASEDIR}/input
+mlr --oxtab put -f ${CASEDIR}/mlr ./${CASEDIR}/input

go/regtest/cases/dsl-type-inference/0004/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0005/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0006/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0007/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0008/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0009/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0012/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0015/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0016/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0017/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0018/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0019/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0020/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0021/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0024/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0027/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0028/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0029/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0030/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0031/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0032/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0033/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0036/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0039/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0040/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0041/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0042/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0043/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0044/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0045/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0048/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0051/cmd

@@ -1 +1 @@
-mlr --xtab put -F -f ${CASEDIR}/mlr  regtest/input/mixed-types.xtab
+mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0052/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0053/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0054/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0055/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0060/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0061/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0062/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0063/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0068/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0069/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0070/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0071/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0076/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0077/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0078/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0079/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0084/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0085/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0086/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0087/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0092/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0093/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0094/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0095/cmd

@@ -1 +1 @@
-mlr --xtab put    -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
+mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

go/regtest/cases/dsl-type-inference/0097/cmd

@@ -0,0 +1 @@
+mlr --idkvp --opprint put -f ${CASEDIR}/mlr ./${CASEDIR}/input

go/regtest/cases/dsl-type-inference/0097/expout

@@ -0,0 +1,33 @@
+a         ta
+12        int
+1.2       float
+inf       string
++inf      string
+-inf      string
+Inf       string
++Inf      string
+-Inf      string
+INF       string
++INF      string
+-INF      string
+infinity  string
++infinity string
+-infinity string
+Infinity  string
++Infinity string
+-Infinity string
+INFINITY  string
++INFINITY string
+-INFINITY string
+InFiNiTy  string
++InFiNiTy string
+-InFiNiTy string
+infiniti  string
++infiniti string
+-infiniti string
+true      string
+false     string
+True      string
+False     string
+TRUE      string
+FALSE     string

go/regtest/cases/dsl-type-inference/0097/input

@@ -0,0 +1,32 @@
+a=12
+a=1.2
+a=inf
+a=+inf
+a=-inf
+a=Inf
+a=+Inf
+a=-Inf
+a=INF
+a=+INF
+a=-INF
+a=infinity
+a=+infinity
+a=-infinity
+a=Infinity
+a=+Infinity
+a=-Infinity
+a=INFINITY
+a=+INFINITY
+a=-INFINITY
+a=InFiNiTy
+a=+InFiNiTy
+a=-InFiNiTy
+a=infiniti
+a=+infiniti
+a=-infiniti
+a=true
+a=false
+a=True
+a=False
+a=TRUE
+a=FALSE

go/regtest/cases/dsl-type-inference/0097/mlr

@@ -0,0 +1 @@
+$ta = typeof($a)