From 5fedc65a337f865d0152ed3392a7b48bb8dc6a14 Mon Sep 17 00:00:00 2001
From: John Kerl <kerl.john.r@gmail.com>
Date: Mon, 6 Jul 2026 15:38:36 -0400
Subject: [PATCH] Document that NIDX/DKVP do not respect double quotes; point
 to CSV with --ifs (#369)

NIDX and DKVP split each line naively on the field separator, so a
separator inside a double-quoted string still starts a new field. For
RFC-4180-style quoted data with a non-comma separator, the right tool is
CSV format with --ifs set to that separator, plus --implicit-csv-header
when the file has no header line. Add a worked example of both to the
NIDX section of the file-formats page, as the FAQ entry requested in
issue #369.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 docs/src/data/semicolon-quoted.txt |  2 ++
 docs/src/file-formats.md           | 35 ++++++++++++++++++++++++++++++
 docs/src/file-formats.md.in        | 16 ++++++++++++++
 3 files changed, 53 insertions(+)
 create mode 100644 docs/src/data/semicolon-quoted.txt

diff --git a/docs/src/data/semicolon-quoted.txt b/docs/src/data/semicolon-quoted.txt
new file mode 100644
index 000000000..0888d79de
--- /dev/null
+++ b/docs/src/data/semicolon-quoted.txt
@@ -0,0 +1,2 @@
+a;b
+"text; text";"more text"
diff --git a/docs/src/file-formats.md b/docs/src/file-formats.md
index ef1eacd8b..7a38f0da5 100644
--- a/docs/src/file-formats.md
+++ b/docs/src/file-formats.md
@@ -836,6 +836,41 @@ the dawn's
 light
 </pre>
 
+Note that NIDX and DKVP split each line on the field separator without regard to double quotes: a separator inside a double-quoted string still starts a new field.
+
+<pre class="pre-highlight-in-pair">
+<b>cat data/semicolon-quoted.txt</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+a;b
+"text; text";"more text"
+</pre>
+
+<pre class="pre-highlight-in-pair">
+<b>mlr --inidx --ifs semicolon --oxtab cat data/semicolon-quoted.txt</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+1 a
+2 b
+
+1 "text
+2  text"
+3 "more text"
+</pre>
+
+If your data uses RFC-4180-style double-quoting to protect separators inside field values, use CSV format with `--ifs` set to your separator -- along with `--implicit-csv-header` if the file has no header line:
+
+<pre class="pre-highlight-in-pair">
+<b>mlr --icsv --ifs semicolon --implicit-csv-header --oxtab cat data/semicolon-quoted.txt</b>
+</pre>
+<pre class="pre-non-highlight-in-pair">
+1 a
+2 b
+
+1 text; text
+2 more text
+</pre>
+
 ## DCF (Debian control file)
 
 <pre class="pre-highlight-in-pair">
diff --git a/docs/src/file-formats.md.in b/docs/src/file-formats.md.in
index 90cfc846f..c739618e1 100644
--- a/docs/src/file-formats.md.in
+++ b/docs/src/file-formats.md.in
@@ -409,6 +409,22 @@ GENMD-RUN-COMMAND
 mlr --nidx --fs ' ' --repifs cut -f 2,3 data/mydata.txt
 GENMD-EOF
 
+Note that NIDX and DKVP split each line on the field separator without regard to double quotes: a separator inside a double-quoted string still starts a new field.
+
+GENMD-RUN-COMMAND
+cat data/semicolon-quoted.txt
+GENMD-EOF
+
+GENMD-RUN-COMMAND
+mlr --inidx --ifs semicolon --oxtab cat data/semicolon-quoted.txt
+GENMD-EOF
+
+If your data uses RFC-4180-style double-quoting to protect separators inside field values, use CSV format with `--ifs` set to your separator -- along with `--implicit-csv-header` if the file has no header line:
+
+GENMD-RUN-COMMAND
+mlr --icsv --ifs semicolon --implicit-csv-header --oxtab cat data/semicolon-quoted.txt
+GENMD-EOF
+
 ## DCF (Debian control file)
 
 GENMD-RUN-COMMAND