Use "\n" in replace for gsub and sub (#2042)

* Fix issue 1805

* Run `make dev`

docs/src/manpage.md

@@ -1333,6 +1333,8 @@ This is simply a copy of what you should see on running `man mlr` at a command p
        Usage: mlr gsub [options]
        Replaces old string with new string in specified field(s), with regex support
        for the old string and handling multiple matches, like the `gsub` DSL function.
+       The replacement string supports C-style backslash escapes such as \n, \t,
+       and \x1f. Write \\ to get a literal backslash.
        See also the `sub` and `ssub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
@@ -1980,7 +1982,10 @@ This is simply a copy of what you should see on running `man mlr` at a command p
    1mssub0m
        Usage: mlr ssub [options]
        Replaces old string with new string in specified field(s), without regex support for
-       the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.
+       the old string, like the `ssub` DSL function.
+       Both the search and replacement strings support C-style backslash escapes such
+       as \n, \t, and \x1f. Write \\ to get a literal backslash.
+       See also the `gsub` and `sub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
        -r {regex}  Regular expression for field names to convert.
@@ -2129,6 +2134,8 @@ This is simply a copy of what you should see on running `man mlr` at a command p
        Usage: mlr sub [options]
        Replaces old string with new string in specified field(s), with regex support
        for the old string and not handling multiple matches, like the `sub` DSL function.
+       The replacement string supports C-style backslash escapes such as \n, \t,
+       and \x1f. Write \\ to get a literal backslash.
        See also the `gsub` and `ssub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
@@ -3823,5 +3830,5 @@ This is simply a copy of what you should see on running `man mlr` at a command p
        MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
        https://miller.readthedocs.io
 
-                                  2026-04-08                         4mMILLER24m(1)
+                                  2026-04-19                         4mMILLER24m(1)
 </pre>

docs/src/manpage.txt

@@ -1312,6 +1312,8 @@
        Usage: mlr gsub [options]
        Replaces old string with new string in specified field(s), with regex support
        for the old string and handling multiple matches, like the `gsub` DSL function.
+       The replacement string supports C-style backslash escapes such as \n, \t,
+       and \x1f. Write \\ to get a literal backslash.
        See also the `sub` and `ssub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
@@ -1959,7 +1961,10 @@
    1mssub0m
        Usage: mlr ssub [options]
        Replaces old string with new string in specified field(s), without regex support for
-       the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.
+       the old string, like the `ssub` DSL function.
+       Both the search and replacement strings support C-style backslash escapes such
+       as \n, \t, and \x1f. Write \\ to get a literal backslash.
+       See also the `gsub` and `sub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
        -r {regex}  Regular expression for field names to convert.
@@ -2108,6 +2113,8 @@
        Usage: mlr sub [options]
        Replaces old string with new string in specified field(s), with regex support
        for the old string and not handling multiple matches, like the `sub` DSL function.
+       The replacement string supports C-style backslash escapes such as \n, \t,
+       and \x1f. Write \\ to get a literal backslash.
        See also the `gsub` and `ssub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
@@ -3802,4 +3809,4 @@
        MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
        https://miller.readthedocs.io
 
-                                  2026-04-08                         4mMILLER24m(1)
+                                  2026-04-19                         4mMILLER24m(1)

docs/src/reference-verbs.md

@@ -1461,6 +1461,8 @@ record_count resource
 Usage: mlr gsub [options]
 Replaces old string with new string in specified field(s), with regex support
 for the old string and handling multiple matches, like the `gsub` DSL function.
+The replacement string supports C-style backslash escapes such as \n, \t,
+and \x1f. Write \\ to get a literal backslash.
 See also the `sub` and `ssub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.
@@ -3219,7 +3221,10 @@ See also the "tee" DSL function which lets you do more ad-hoc customization.
 <pre class="pre-non-highlight-in-pair">
 Usage: mlr ssub [options]
 Replaces old string with new string in specified field(s), without regex support for
-the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.
+the old string, like the `ssub` DSL function.
+Both the search and replacement strings support C-style backslash escapes such
+as \n, \t, and \x1f. Write \\ to get a literal backslash.
+See also the `gsub` and `sub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.
 -r {regex}  Regular expression for field names to convert.
@@ -3725,6 +3730,8 @@ $ each 10 uptime | mlr -p step -a delta -f 11
 Usage: mlr sub [options]
 Replaces old string with new string in specified field(s), with regex support
 for the old string and not handling multiple matches, like the `sub` DSL function.
+The replacement string supports C-style backslash escapes such as \n, \t,
+and \x1f. Write \\ to get a literal backslash.
 See also the `gsub` and `ssub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.

man/manpage.txt

@@ -1312,6 +1312,8 @@
        Usage: mlr gsub [options]
        Replaces old string with new string in specified field(s), with regex support
        for the old string and handling multiple matches, like the `gsub` DSL function.
+       The replacement string supports C-style backslash escapes such as \n, \t,
+       and \x1f. Write \\ to get a literal backslash.
        See also the `sub` and `ssub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
@@ -1959,7 +1961,10 @@
    1mssub0m
        Usage: mlr ssub [options]
        Replaces old string with new string in specified field(s), without regex support for
-       the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.
+       the old string, like the `ssub` DSL function.
+       Both the search and replacement strings support C-style backslash escapes such
+       as \n, \t, and \x1f. Write \\ to get a literal backslash.
+       See also the `gsub` and `sub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
        -r {regex}  Regular expression for field names to convert.
@@ -2108,6 +2113,8 @@
        Usage: mlr sub [options]
        Replaces old string with new string in specified field(s), with regex support
        for the old string and not handling multiple matches, like the `sub` DSL function.
+       The replacement string supports C-style backslash escapes such as \n, \t,
+       and \x1f. Write \\ to get a literal backslash.
        See also the `gsub` and `ssub` verbs.
        Options:
        -f {a,b,c}  Field names to convert.
@@ -3802,4 +3809,4 @@
        MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
        https://miller.readthedocs.io
 
-                                  2026-04-08                         4mMILLER24m(1)
+                                  2026-04-19                         4mMILLER24m(1)

man/mlr.1

@@ -2,12 +2,12 @@
 .\"     Title: mlr
 .\"    Author: [see the "AUTHOR" section]
 .\" Generator: ./mkman.rb
-.\"      Date: 2026-04-08
+.\"      Date: 2026-04-19
 .\"    Manual: \ \&
 .\"    Source: \ \&
 .\"  Language: English
 .\"
-.TH "MILLER" "1" "2026-04-08" "\ \&" "\ \&"
+.TH "MILLER" "1" "2026-04-19" "\ \&" "\ \&"
 .\" -----------------------------------------------------------------
 .\" * Portability definitions
 .\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1631,6 +1631,8 @@ Options:
 Usage: mlr gsub [options]
 Replaces old string with new string in specified field(s), with regex support
 for the old string and handling multiple matches, like the `gsub` DSL function.
+The replacement string supports C-style backslash escapes such as \en, \et,
+and \ex1f. Write \e\e to get a literal backslash.
 See also the `sub` and `ssub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.
@@ -2464,7 +2466,10 @@ See also the "tee" DSL function which lets you do more ad-hoc customization.
 .nf
 Usage: mlr ssub [options]
 Replaces old string with new string in specified field(s), without regex support for
-the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.
+the old string, like the `ssub` DSL function.
+Both the search and replacement strings support C-style backslash escapes such
+as \en, \et, and \ex1f. Write \e\e to get a literal backslash.
+See also the `gsub` and `sub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.
 -r {regex}  Regular expression for field names to convert.
@@ -2637,6 +2642,8 @@ for more information on EWMA.
 Usage: mlr sub [options]
 Replaces old string with new string in specified field(s), with regex support
 for the old string and not handling multiple matches, like the `sub` DSL function.
+The replacement string supports C-style backslash escapes such as \en, \et,
+and \ex1f. Write \e\e to get a literal backslash.
 See also the `gsub` and `ssub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.

pkg/transformers/subs.go

@@ -44,6 +44,8 @@ func transformerSubUsage(
 	fmt.Fprintf(o, "Usage: %s %s [options]\n", "mlr", verbNameSub)
 	fmt.Fprintf(o, "Replaces old string with new string in specified field(s), with regex support\n")
 	fmt.Fprintf(o, "for the old string and not handling multiple matches, like the `sub` DSL function.\n")
+	fmt.Fprintf(o, "The replacement string supports C-style backslash escapes such as \\n, \\t,\n")
+	fmt.Fprintf(o, "and \\x1f. Write \\\\ to get a literal backslash.\n")
 	fmt.Fprintf(o, "See also the `gsub` and `ssub` verbs.\n")
 	fmt.Fprintf(o, "Options:\n")
 	fmt.Fprintf(o, "-f {a,b,c}  Field names to convert.\n")
@@ -58,6 +60,8 @@ func transformerGsubUsage(
 	fmt.Fprintf(o, "Usage: %s %s [options]\n", "mlr", verbNameGsub)
 	fmt.Fprintf(o, "Replaces old string with new string in specified field(s), with regex support\n")
 	fmt.Fprintf(o, "for the old string and handling multiple matches, like the `gsub` DSL function.\n")
+	fmt.Fprintf(o, "The replacement string supports C-style backslash escapes such as \\n, \\t,\n")
+	fmt.Fprintf(o, "and \\x1f. Write \\\\ to get a literal backslash.\n")
 	fmt.Fprintf(o, "See also the `sub` and `ssub` verbs.\n")
 	fmt.Fprintf(o, "Options:\n")
 	fmt.Fprintf(o, "-f {a,b,c}  Field names to convert.\n")
@@ -71,7 +75,10 @@ func transformerSsubUsage(
 ) {
 	fmt.Fprintf(o, "Usage: %s %s [options]\n", "mlr", verbNameSsub)
 	fmt.Fprintf(o, "Replaces old string with new string in specified field(s), without regex support for\n")
-	fmt.Fprintf(o, "the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.\n")
+	fmt.Fprintf(o, "the old string, like the `ssub` DSL function.\n")
+	fmt.Fprintf(o, "Both the search and replacement strings support C-style backslash escapes such\n")
+	fmt.Fprintf(o, "as \\n, \\t, and \\x1f. Write \\\\ to get a literal backslash.\n")
+	fmt.Fprintf(o, "See also the `gsub` and `sub` verbs.\n")
 	fmt.Fprintf(o, "Options:\n")
 	fmt.Fprintf(o, "-f {a,b,c}  Field names to convert.\n")
 	fmt.Fprintf(o, "-r {regex}  Regular expression for field names to convert.\n")
@@ -98,7 +105,7 @@ func transformerSubParseCLI(
 	opts *cli.TOptions,
 	doConstruct bool, // false for first pass of CLI-parse, true for second pass
 ) (RecordTransformer, error) {
-	return transformerSubsParseCLI(pargi, argc, args, opts, doConstruct, transformerSubUsage, NewTransformerSub)
+	return transformerSubsParseCLI(pargi, argc, args, opts, doConstruct, transformerSubUsage, NewTransformerSub, false)
 }
 
 func transformerGsubParseCLI(
@@ -108,7 +115,7 @@ func transformerGsubParseCLI(
 	opts *cli.TOptions,
 	doConstruct bool, // false for first pass of CLI-parse, true for second pass
 ) (RecordTransformer, error) {
-	return transformerSubsParseCLI(pargi, argc, args, opts, doConstruct, transformerGsubUsage, NewTransformerGsub)
+	return transformerSubsParseCLI(pargi, argc, args, opts, doConstruct, transformerGsubUsage, NewTransformerGsub, false)
 }
 
 func transformerSsubParseCLI(
@@ -118,10 +125,12 @@ func transformerSsubParseCLI(
 	opts *cli.TOptions,
 	doConstruct bool, // false for first pass of CLI-parse, true for second pass
 ) (RecordTransformer, error) {
-	return transformerSubsParseCLI(pargi, argc, args, opts, doConstruct, transformerSsubUsage, NewTransformerSsub)
+	return transformerSubsParseCLI(pargi, argc, args, opts, doConstruct, transformerSsubUsage, NewTransformerSsub, true)
 }
 
 // transformerSubsParseCLI is a shared CLI-parser for the sub, gsub, and ssub verbs.
+// When unbackslashOldText is true (ssub only), the search string is also unescaped;
+// for sub/gsub the search string is a regex and Go's regexp engine handles \n/\t/etc.
 func transformerSubsParseCLI(
 	pargi *int,
 	argc int,
@@ -130,6 +139,7 @@ func transformerSubsParseCLI(
 	doConstruct bool, // false for first pass of CLI-parse, true for second pass
 	usageFunc TransformerUsageFunc,
 	constructorFunc subConstructorFunc,
+	unbackslashOldText bool,
 ) (RecordTransformer, error) {
 
 	// Skip the verb name from the current spot in the mlr command line
@@ -192,6 +202,18 @@ func transformerSubsParseCLI(
 	oldText = args[argi]
 	newText = args[argi+1]
 
+	// Interpret C-style backslash escapes ("\n", "\t", "\x1f", etc.) in the
+	// replacement string the same way the DSL string-literal parser does, so
+	// that e.g. `mlr sub -a r "\n"` matches `sub($x, "r", "\n")` in the DSL.
+	// For sub/gsub the search string is a regex and Go's regexp engine already
+	// handles \n/\r/\t inside patterns; pre-unescaping would corrupt user-
+	// supplied regex metachars like \d or \s. For ssub the search string is a
+	// literal, so we unescape it too.
+	newText = lib.UnbackslashStringLiteral(newText)
+	if unbackslashOldText {
+		oldText = lib.UnbackslashStringLiteral(oldText)
+	}
+
 	argi += 2
 
 	*pargi = argi

test/cases/cli-help/0001/expout

@@ -389,6 +389,8 @@ gsub
 Usage: mlr gsub [options]
 Replaces old string with new string in specified field(s), with regex support
 for the old string and handling multiple matches, like the `gsub` DSL function.
+The replacement string supports C-style backslash escapes such as \n, \t,
+and \x1f. Write \\ to get a literal backslash.
 See also the `sub` and `ssub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.
@@ -1067,7 +1069,10 @@ See also the "tee" DSL function which lets you do more ad-hoc customization.
 ssub
 Usage: mlr ssub [options]
 Replaces old string with new string in specified field(s), without regex support for
-the old string, like the `ssub` DSL function. See also the `gsub` and `sub` verbs.
+the old string, like the `ssub` DSL function.
+Both the search and replacement strings support C-style backslash escapes such
+as \n, \t, and \x1f. Write \\ to get a literal backslash.
+See also the `gsub` and `sub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.
 -r {regex}  Regular expression for field names to convert.
@@ -1220,6 +1225,8 @@ sub
 Usage: mlr sub [options]
 Replaces old string with new string in specified field(s), with regex support
 for the old string and not handling multiple matches, like the `sub` DSL function.
+The replacement string supports C-style backslash escapes such as \n, \t,
+and \x1f. Write \\ to get a literal backslash.
 See also the `gsub` and `ssub` verbs.
 Options:
 -f {a,b,c}  Field names to convert.

test/cases/verb-sub-gsub-ssub/non-windows/gsub-escape-tab/cmd

@@ -0,0 +1 @@
+printf 'a=lorem\n' | mlr --ojson gsub -a '[lm]' '\t'

test/cases/verb-sub-gsub-ssub/non-windows/gsub-escape-tab/expout

@@ -0,0 +1,5 @@
+[
+{
+  "a": "\tore\t"
+}
+]

test/cases/verb-sub-gsub-ssub/non-windows/ssub-escape-tab/cmd

@@ -0,0 +1 @@
+printf 'a=X\tY\n' | mlr --ojson ssub -a '\t' '-'

test/cases/verb-sub-gsub-ssub/non-windows/ssub-escape-tab/expout

@@ -0,0 +1,5 @@
+[
+{
+  "a": "X-Y"
+}
+]

test/cases/verb-sub-gsub-ssub/non-windows/sub-escape-hex/cmd

@@ -0,0 +1 @@
+printf 'a=lorem\n' | mlr --ojson sub -a 'r' '\x41'

test/cases/verb-sub-gsub-ssub/non-windows/sub-escape-hex/expout

@@ -0,0 +1,5 @@
+[
+{
+  "a": "loAem"
+}
+]

test/cases/verb-sub-gsub-ssub/non-windows/sub-escape-literal-backslash/cmd

@@ -0,0 +1 @@
+printf 'a=lorem\n' | mlr --ojson sub -a 'r' '\\n'

test/cases/verb-sub-gsub-ssub/non-windows/sub-escape-literal-backslash/expout

@@ -0,0 +1,5 @@
+[
+{
+  "a": "lo\\nem"
+}
+]

test/cases/verb-sub-gsub-ssub/non-windows/sub-escape-newline/cmd

@@ -0,0 +1 @@
+printf 'a=lorem\n' | mlr --ocsv sub -a 'r' '\n'

test/cases/verb-sub-gsub-ssub/non-windows/sub-escape-newline/expout

@@ -0,0 +1,3 @@
+a
+"lo
+em"