Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-17 16:38:54 +00:00
Code Issues Projects Releases Wiki Activity

Document that NIDX/DKVP do not respect double quotes; point to CSV with --ifs (#369)

Browse source 
NIDX and DKVP split each line naively on the field separator, so a
separator inside a double-quoted string still starts a new field. For
RFC-4180-style quoted data with a non-comma separator, the right tool is
CSV format with --ifs set to that separator, plus --implicit-csv-header
when the file has no header line. Add a worked example of both to the
NIDX section of the file-formats page, as the FAQ entry requested in
issue #369.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
John Kerl 2026-07-06 15:38:36 -04:00
parent a63ea57359
commit 5fedc65a33
3 changed files with 53 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/src/data/semicolon-quoted.txt Normal file
View file

@ -0,0 +1,2 @@
a;b
"text; text";"more text"

35
docs/src/file-formats.md
View file

@ -836,6 +836,41 @@ the dawn's
light
</pre>
Note that NIDX and DKVP split each line on the field separator without regard to double quotes: a separator inside a double-quoted string still starts a new field.
<pre class="pre-highlight-in-pair">
<b>cat data/semicolon-quoted.txt</b>
</pre>
<pre class="pre-non-highlight-in-pair">
a;b
"text; text";"more text"
</pre>
<pre class="pre-highlight-in-pair">
<b>mlr --inidx --ifs semicolon --oxtab cat data/semicolon-quoted.txt</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1 a
2 b
1 "text
2 text"
3 "more text"
</pre>
If your data uses RFC-4180-style double-quoting to protect separators inside field values, use CSV format with `--ifs` set to your separator -- along with `--implicit-csv-header` if the file has no header line:
<pre class="pre-highlight-in-pair">
<b>mlr --icsv --ifs semicolon --implicit-csv-header --oxtab cat data/semicolon-quoted.txt</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1 a
2 b
1 text; text
2 more text
</pre>
## DCF (Debian control file)
<pre class="pre-highlight-in-pair">

16
docs/src/file-formats.md.in
View file

@ -409,6 +409,22 @@ GENMD-RUN-COMMAND
mlr --nidx --fs ' ' --repifs cut -f 2,3 data/mydata.txt
GENMD-EOF
Note that NIDX and DKVP split each line on the field separator without regard to double quotes: a separator inside a double-quoted string still starts a new field.
GENMD-RUN-COMMAND
cat data/semicolon-quoted.txt
GENMD-EOF
GENMD-RUN-COMMAND
mlr --inidx --ifs semicolon --oxtab cat data/semicolon-quoted.txt
GENMD-EOF
If your data uses RFC-4180-style double-quoting to protect separators inside field values, use CSV format with `--ifs` set to your separator -- along with `--implicit-csv-header` if the file has no header line:
GENMD-RUN-COMMAND
mlr --icsv --ifs semicolon --implicit-csv-header --oxtab cat data/semicolon-quoted.txt
GENMD-EOF
## DCF (Debian control file)
GENMD-RUN-COMMAND