Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-01-23 02:14:13 +00:00
Code Issues Projects Releases Wiki Activity

Fix typo in online help for --no-jlistwrap (#1541)

Browse source 
* Add --no-auto-unsparsify flag

* Fix typo in online help for `--no-jlistwrap`

* Artifacts from `make dev`
This commit is contained in:
John Kerl 2024-04-11 08:12:45 -04:00 • committed by GitHub
parent 03b8cce048
commit e714738a7d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
16 changed files with 143 additions and 71 deletions
Download patch file Download diff file Expand all files Collapse all files

46
docs/src/data-diving-examples.md
View file

@ -160,11 +160,11 @@ CITRUS COUNTY 1332.9 79974.9 483785.1
<b> stats2 -a corr,linreg-ols,r2 -f tiv_2011,tiv_2012</b>
</pre>
<pre class="pre-non-highlight-in-pair">
tiv_2011_tiv_2012_corr 0.9730497632351701
tiv_2011_tiv_2012_ols_m 0.9835583980337732
tiv_2011_tiv_2012_ols_b 433854.6428968301
tiv_2011_tiv_2012_corr 0.9730497632351692
tiv_2011_tiv_2012_ols_m 0.9835583980337723
tiv_2011_tiv_2012_ols_b 433854.6428968317
tiv_2011_tiv_2012_ols_n 36634
tiv_2011_tiv_2012_r2 0.9468258417320204
tiv_2011_tiv_2012_r2 0.9468258417320189
</pre>
<pre class="pre-highlight-in-pair">
@ -322,7 +322,7 @@ Look at bivariate stats by color and shape. In particular, `u,v` pairwise correl
</pre>
<pre class="pre-non-highlight-in-pair">
u_v_corr w_x_corr
0.1334180491027861 -0.011319841199852926
0.1334180491027861 -0.011319841199866178
</pre>
<pre class="pre-highlight-in-pair">
@ -332,22 +332,22 @@ Look at bivariate stats by color and shape. In particular, `u,v` pairwise correl
</pre>
<pre class="pre-non-highlight-in-pair">
color shape u_v_corr w_x_corr
red circle 0.9807984401887242 -0.018565536587084836
orange square 0.17685855992752933 -0.07104431573805543
green circle 0.05764419437577257 0.011795729888018455
red square 0.0557447712489348 -0.0006801456507506415
yellow triangle 0.0445727377196281 0.024604310103079844
yellow square 0.0437917292729612 -0.044621972016306265
purple circle 0.03587354936895115 0.13411339541407613
blue square 0.03241153095761152 -0.05350764811965621
blue triangle 0.015356427073158612 -0.0006089997461408209
orange circle 0.010518953877704181 -0.1627939732927932
red triangle 0.00809782571528054 0.01248662135795501
purple triangle 0.005155190909099739 -0.04505790925621933
purple square -0.02568027696337717 0.057694296479293694
green square -0.025776073450284875 -0.0032651732520739014
orange triangle -0.030456661186085584 -0.13186999819263814
yellow circle -0.06477331572781515 0.0736944981970553
blue circle -0.1023476190192966 -0.030528539069839333
green triangle -0.10901825107358747 -0.04848782060162855
red circle 0.9807984401887236 -0.01856553658708754
orange square 0.17685855992752927 -0.07104431573806054
green circle 0.05764419437577255 0.01179572988801509
red square 0.05574477124893523 -0.0006801456507510942
yellow triangle 0.04457273771962798 0.024604310103081825
yellow square 0.04379172927296089 -0.04462197201631237
purple circle 0.03587354936895086 0.1341133954140899
blue square 0.03241153095761164 -0.053507648119643196
blue triangle 0.015356427073158766 -0.0006089997461435399
orange circle 0.010518953877704048 -0.16279397329279383
red triangle 0.00809782571528034 0.012486621357942596
purple triangle 0.005155190909099334 -0.045057909256220656
purple square -0.025680276963377404 0.05769429647930396
green square -0.0257760734502851 -0.003265173252087127
orange triangle -0.030456661186085785 -0.1318699981926352
yellow circle -0.06477331572781474 0.07369449819706045
blue circle -0.10234761901929677 -0.030528539069837757
green triangle -0.10901825107358765 -0.04848782060162929
</pre>

18
docs/src/manpage.md
View file

@ -19,7 +19,9 @@ Quick links:
This is simply a copy of what you should see on running `man mlr` at a command prompt, once Miller is installed on your system.
<pre class="pre-non-highlight-non-pair">
4mMILLER24m(1) 4mMILLER24m(1)
MILLER(1) MILLER(1)
1mNAME0m
Miller -- like awk, sed, cut, join, and sort for name-indexed data such
@ -338,6 +340,10 @@ This is simply a copy of what you should see on running `man mlr` at a command p
recreate missing headers.
--lazy-quotes Accepts quotes appearing in unquoted fields, and
non-doubled quotes appearing in quoted fields.
--no-auto-unsparsify For CSV/TSV output: if the record keys change from
one row to another, emit a blank line and a new
header line. This is non-compliant with RFC 4180 but
it helpful for heterogeneous data.
--no-implicit-csv-header or --no-implicit-tsv-header
Opposite of `--implicit-csv-header`. This is the
default anyway -- the main use is for the flags to
@ -468,8 +474,8 @@ This is simply a copy of what you should see on running `man mlr` at a command p
--jvstack Put one key-value pair per line for JSON output
(multi-line output). This is the default for JSON
output format.
--no-jlistwrap Wrap JSON output in outermost `[ ]`. This is the
default for JSON Lines output format.
--no-jlistwrap Do not wrap JSON output in outermost `[ ]`. This is
the default for JSON Lines output format.
--no-jvstack Put objects/arrays all on one line for JSON output.
This is the default for JSON Lines output format.
@ -809,7 +815,7 @@ This is simply a copy of what you should see on running `man mlr` at a command p
markdown " " N/A "\n"
nidx " " N/A "\n"
pprint " " N/A "\n"
tsv " " N/A "\n"
tsv " " N/A "\n"
xtab "\n" " " "\n\n"
--fs {string} Specify FS for input and output.
@ -3712,5 +3718,7 @@ This is simply a copy of what you should see on running `man mlr` at a command p
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
https://miller.readthedocs.io
2024-03-16 4mMILLER24m(1)
2024-04-11 MILLER(1)
</pre>

18
docs/src/manpage.txt
View file

@ -1,4 +1,6 @@
4mMILLER24m(1) 4mMILLER24m(1)
MILLER(1) MILLER(1)
1mNAME0m
Miller -- like awk, sed, cut, join, and sort for name-indexed data such
@ -317,6 +319,10 @@
recreate missing headers.
--lazy-quotes Accepts quotes appearing in unquoted fields, and
non-doubled quotes appearing in quoted fields.
--no-auto-unsparsify For CSV/TSV output: if the record keys change from
one row to another, emit a blank line and a new
header line. This is non-compliant with RFC 4180 but
it helpful for heterogeneous data.
--no-implicit-csv-header or --no-implicit-tsv-header
Opposite of `--implicit-csv-header`. This is the
default anyway -- the main use is for the flags to
@ -447,8 +453,8 @@
--jvstack Put one key-value pair per line for JSON output
(multi-line output). This is the default for JSON
output format.
--no-jlistwrap Wrap JSON output in outermost `[ ]`. This is the
default for JSON Lines output format.
--no-jlistwrap Do not wrap JSON output in outermost `[ ]`. This is
the default for JSON Lines output format.
--no-jvstack Put objects/arrays all on one line for JSON output.
This is the default for JSON Lines output format.
@ -788,7 +794,7 @@
markdown " " N/A "\n"
nidx " " N/A "\n"
pprint " " N/A "\n"
tsv " " N/A "\n"
tsv " " N/A "\n"
xtab "\n" " " "\n\n"
--fs {string} Specify FS for input and output.
@ -3691,4 +3697,6 @@
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
https://miller.readthedocs.io
2024-03-16 4mMILLER24m(1)
2024-04-11 MILLER(1)

3
docs/src/reference-main-flag-list.md
View file

@ -123,6 +123,7 @@ These are flags which are applicable to CSV format.
* `--headerless-csv-output or --ho or --headerless-tsv-output`: Print only CSV/TSV data lines; do not print CSV/TSV header lines.
* `--implicit-csv-header or --headerless-csv-input or --hi or --implicit-tsv-header`: Use 1,2,3,... as field labels, rather than from line 1 of input files. Tip: combine with `label` to recreate missing headers.
* `--lazy-quotes`: Accepts quotes appearing in unquoted fields, and non-doubled quotes appearing in quoted fields.
* `--no-auto-unsparsify`: For CSV/TSV output: if the record keys change from one row to another, emit a blank line and a new header line. This is non-compliant with RFC 4180 but it helpful for heterogeneous data.
* `--no-implicit-csv-header or --no-implicit-tsv-header`: Opposite of `--implicit-csv-header`. This is the default anyway -- the main use is for the flags to `mlr join` if you have main file(s) which are headerless but you want to join in on a file which does have a CSV/TSV header. Then you could use `mlr --csv --implicit-csv-header join --no-implicit-csv-header -l your-join-in-with-header.csv ... your-headerless.csv`.
* `--quote-all`: Force double-quoting of CSV fields.
* `-N`: Keystroke-saver for `--implicit-csv-header --headerless-csv-output`.
@ -234,7 +235,7 @@ These are flags which are applicable to JSON output format.
* `--jlistwrap or --jl`: Wrap JSON output in outermost `[ ]`. This is the default for JSON output format.
* `--jvquoteall`: Force all JSON values -- recursively into lists and object -- to string.
* `--jvstack`: Put one key-value pair per line for JSON output (multi-line output). This is the default for JSON output format.
* `--no-jlistwrap`: Wrap JSON output in outermost `[ ]`. This is the default for JSON Lines output format.
* `--no-jlistwrap`: Do not wrap JSON output in outermost `[ ]`. This is the default for JSON Lines output format.
* `--no-jvstack`: Put objects/arrays all on one line for JSON output. This is the default for JSON Lines output format.
## Legacy flags

38
docs/src/reference-verbs.md
View file

@ -3433,14 +3433,14 @@ fields, optionally categorized by one or more fields.
<b> data/medium</b>
</pre>
<pre class="pre-non-highlight-in-pair">
x_y_cov 0.00004257482082749404
x_y_corr 0.0005042001844473328
y_y_cov 0.08461122467974005
x_y_cov 0.000042574820827444476
x_y_corr 0.0005042001844467462
y_y_cov 0.08461122467974003
y_y_corr 1
x2_xy_cov 0.041883822817793716
x2_xy_corr 0.6301743420379936
x2_y2_cov -0.0003095372596253918
x2_y2_corr -0.003424908876111875
x2_xy_cov 0.04188382281779374
x2_xy_corr 0.630174342037994
x2_y2_cov -0.00030953725962542085
x2_y2_corr -0.0034249088761121966
</pre>
<pre class="pre-highlight-in-pair">
@ -3449,12 +3449,12 @@ x2_y2_corr -0.003424908876111875
<b> data/medium</b>
</pre>
<pre class="pre-non-highlight-in-pair">
a x_y_ols_m x_y_ols_b x_y_ols_n x_y_r2 y_y_ols_m y_y_ols_b y_y_ols_n y_y_r2 xy_y2_ols_m xy_y2_ols_b xy_y2_ols_n xy_y2_r2
pan 0.017025512736819345 0.500402892289764 2081 0.00028691820445815624 1 -0.00000000000000002890430283104539 2081 1 0.8781320866715664 0.11908230147563569 2081 0.4174982737731127
eks 0.04078049236855813 0.4814020796765104 1965 0.0016461239223448218 1 0.00000000000000017862676354313703 1965 1 0.897872861169018 0.1073405443361234 1965 0.4556322386425451
wye -0.03915349075204785 0.5255096523974457 1966 0.0015051268704373377 1 0.00000000000000004464425401127647 1966 1 0.8538317334220837 0.1267454301662969 1966 0.3899172181859931
zee 0.0027812364960401333 0.5043070448033061 2047 0.000007751652858787357 1 0.00000000000000004819404567023685 2047 1 0.8524439912011011 0.12401684308018947 2047 0.39356598090006495
hat -0.018620577041095272 0.5179005397264937 1941 0.00035200366460556604 1 -0.00000000000000003400445761787692 1941 1 0.8412305086345017 0.13557328318623207 1941 0.3687944261732266
a x_y_ols_m x_y_ols_b x_y_ols_n x_y_r2 y_y_ols_m y_y_ols_b y_y_ols_n y_y_r2 xy_y2_ols_m xy_y2_ols_b xy_y2_ols_n xy_y2_r2
pan 0.01702551273681908 0.5004028922897639 2081 0.00028691820445814767 1 0 2081 1 0.8781320866715662 0.11908230147563566 2081 0.41749827377311266
eks 0.0407804923685586 0.48140207967651016 1965 0.0016461239223448587 1 0 1965 1 0.8978728611690183 0.10734054433612333 1965 0.45563223864254526
wye -0.03915349075204814 0.5255096523974456 1966 0.0015051268704373607 1 0 1966 1 0.8538317334220835 0.1267454301662969 1966 0.38991721818599295
zee 0.0027812364960399147 0.5043070448033061 2047 0.000007751652858786137 1 0 2047 1 0.8524439912011013 0.12401684308018937 2047 0.39356598090006495
hat -0.018620577041095078 0.5179005397264935 1941 0.0003520036646055585 1 0 1941 1 0.8412305086345014 0.13557328318623216 1941 0.3687944261732265
</pre>
Here's an example simple line-fit. The `x` and `y`
@ -3540,11 +3540,11 @@ upsec_count_pca_quality 0.9999590846136102
donesec 92.33051350964094
color purple
upsec_count_pca_m -39.030097447953594
upsec_count_pca_b 979.9883413064917
upsec_count_pca_m -39.03009744795354
upsec_count_pca_b 979.9883413064914
upsec_count_pca_n 21
upsec_count_pca_quality 0.9999908956206317
donesec 25.108529196302943
donesec 25.10852919630297
</pre>
## step
@ -3821,9 +3821,9 @@ distinct_count 5 5 10000 10000 10000
mode pan wye 1 0.3467901443380824 0.7268028627434533
sum 0 0 50005000 4986.019681679581 5062.057444929905
mean - - 5000.5 0.49860196816795804 0.5062057444929905
stddev - - 2886.8956799071675 0.29029251511440074 0.2908800864269331
var - - 8334166.666666667 0.08426974433144457 0.08461122467974005
skewness - - 0 -0.0006899591185517494 -0.01784976012013298
stddev - - 2886.8956799071675 0.2902925151144007 0.290880086426933
var - - 8334166.666666667 0.08426974433144456 0.08461122467974003
skewness - - 0 -0.0006899591185521965 -0.017849760120133784
minlen 3 3 1 15 13
maxlen 3 3 5 22 22
min eks eks 1 0.00004509679127584487 0.00008818962627266114

4
docs/src/two-pass-algorithms.md
View file

@ -598,8 +598,8 @@ hat pan 0.4643355557376876
x_count 10000
x_sum 4986.019681679581
x_mean 0.49860196816795804
x_var 0.08426974433144457
x_stddev 0.29029251511440074
x_var 0.08426974433144456
x_stddev 0.2902925151144007
</pre>
<pre class="pre-highlight-in-pair">

18
man/manpage.txt
View file

@ -1,4 +1,6 @@
4mMILLER24m(1) 4mMILLER24m(1)
MILLER(1) MILLER(1)
1mNAME0m
Miller -- like awk, sed, cut, join, and sort for name-indexed data such
@ -317,6 +319,10 @@
recreate missing headers.
--lazy-quotes Accepts quotes appearing in unquoted fields, and
non-doubled quotes appearing in quoted fields.
--no-auto-unsparsify For CSV/TSV output: if the record keys change from
one row to another, emit a blank line and a new
header line. This is non-compliant with RFC 4180 but
it helpful for heterogeneous data.
--no-implicit-csv-header or --no-implicit-tsv-header
Opposite of `--implicit-csv-header`. This is the
default anyway -- the main use is for the flags to
@ -447,8 +453,8 @@
--jvstack Put one key-value pair per line for JSON output
(multi-line output). This is the default for JSON
output format.
--no-jlistwrap Wrap JSON output in outermost `[ ]`. This is the
default for JSON Lines output format.
--no-jlistwrap Do not wrap JSON output in outermost `[ ]`. This is
the default for JSON Lines output format.
--no-jvstack Put objects/arrays all on one line for JSON output.
This is the default for JSON Lines output format.
@ -788,7 +794,7 @@
markdown " " N/A "\n"
nidx " " N/A "\n"
pprint " " N/A "\n"
tsv " " N/A "\n"
tsv " " N/A "\n"
xtab "\n" " " "\n\n"
--fs {string} Specify FS for input and output.
@ -3691,4 +3697,6 @@
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
https://miller.readthedocs.io
2024-03-16 4mMILLER24m(1)
2024-04-11 MILLER(1)

12
man/mlr.1
View file

@ -2,12 +2,12 @@
.\" Title: mlr
.\" Author: [see the "AUTHOR" section]
.\" Generator: ./mkman.rb
.\" Date: 2024-03-16
.\" Date: 2024-04-11
.\" Manual: \ \&
.\" Source: \ \&
.\" Language: English
.\"
.TH "MILLER" "1" "2024-03-16" "\ \&" "\ \&"
.TH "MILLER" "1" "2024-04-11" "\ \&" "\ \&"
.\" -----------------------------------------------------------------
.\" * Portability definitions
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -390,6 +390,10 @@ These are flags which are applicable to CSV format.
recreate missing headers.
--lazy-quotes Accepts quotes appearing in unquoted fields, and
non-doubled quotes appearing in quoted fields.
--no-auto-unsparsify For CSV/TSV output: if the record keys change from
one row to another, emit a blank line and a new
header line. This is non-compliant with RFC 4180 but
it helpful for heterogeneous data.
--no-implicit-csv-header or --no-implicit-tsv-header
Opposite of `--implicit-csv-header`. This is the
default anyway -- the main use is for the flags to
@ -552,8 +556,8 @@ These are flags which are applicable to JSON output format.
--jvstack Put one key-value pair per line for JSON output
(multi-line output). This is the default for JSON
output format.
--no-jlistwrap Wrap JSON output in outermost `[ ]`. This is the
default for JSON Lines output format.
--no-jlistwrap Do not wrap JSON output in outermost `[ ]`. This is
the default for JSON Lines output format.
--no-jvstack Put objects/arrays all on one line for JSON output.
This is the default for JSON Lines output format.
.fi

11
pkg/cli/option_parse.go
View file

@ -460,7 +460,7 @@ var JSONOnlyFlagSection = FlagSection{
{
name: "--no-jlistwrap",
help: "Wrap JSON output in outermost `[ ]`. This is the default for JSON Lines output format.",
help: "Do not wrap JSON output in outermost `[ ]`. This is the default for JSON Lines output format.",
parser: func(args []string, argc int, pargi *int, options *TOptions) {
options.WriterOptions.WrapJSONOutputInOuterList = false
*pargi += 1
@ -2392,6 +2392,15 @@ var CSVTSVOnlyFlagSection = FlagSection{
},
},
{
name: "--no-auto-unsparsify",
help: "For CSV/TSV output: if the record keys change from one row to another, emit a blank line and a new header line. This is non-compliant with RFC 4180 but it helpful for heterogeneous data.",
parser: func(args []string, argc int, pargi *int, options *TOptions) {
options.WriterOptions.NoAutoUnsparsify = true
*pargi += 1
},
},
{
name: "--implicit-csv-header",
altNames: []string{"--headerless-csv-input", "--hi", "--implicit-tsv-header"},

34
pkg/cli/option_types.go
View file

@ -134,6 +134,40 @@ type TWriterOptions struct {
// (all but JSON) -- unless the user explicitly asks to suppress that.
AutoFlatten bool
// Default CSV/TSV:
// a=1,b=2,c=3
// a=4,b=5
// leads to
// a,b,c
// 1 2,3
// 4,5, <-- note trailing empty field
// and
// a=1,b=2,c=3
// d=4,e=5
// leads to
// fatal error
//
// With this flag:
// a=1,b=2,c=3
// a=4,b=5
// leads to
// a,b,c
// 1 2,3
//
// a,b
// 4,5
//
// and
// a=1,b=2,c=3
// d=4,e=5
// leads to
// a,b,c
// 1,2,3
//
// d,e
// 4,5
NoAutoUnsparsify bool
// For floating-point numbers: "" means use the Go default.
FPOFMT string

2
pkg/output/record_writer_csv.go
View file

@ -46,8 +46,8 @@ func (writer *RecordWriterCSV) Write(
bufferedOutputStream *bufio.Writer,
outputIsStdout bool,
) error {
// End of record stream: nothing special for this output format
if outrec == nil {
// End of record stream: nothing special for this output format
return nil
}

2
pkg/output/record_writer_csvlite.go
View file

@ -32,8 +32,8 @@ func (writer *RecordWriterCSVLite) Write(
bufferedOutputStream *bufio.Writer,
outputIsStdout bool,
) error {
// End of record stream: nothing special for this output format
if outrec == nil {
// End of record stream: nothing special for this output format
return nil
}

2
pkg/output/record_writer_dkvp.go
View file

@ -25,8 +25,8 @@ func (writer *RecordWriterDKVP) Write(
bufferedOutputStream *bufio.Writer,
outputIsStdout bool,
) error {
// End of record stream: nothing special for this output format
if outrec == nil {
// End of record stream: nothing special for this output format
return nil
}

2
pkg/output/record_writer_nidx.go
View file

@ -24,8 +24,8 @@ func (writer *RecordWriterNIDX) Write(
bufferedOutputStream *bufio.Writer,
outputIsStdout bool,
) error {
// End of record stream: nothing special for this output format
if outrec == nil {
// End of record stream: nothing special for this output format
return nil
}

2
pkg/output/record_writer_tsv.go
View file

@ -40,8 +40,8 @@ func (writer *RecordWriterTSV) Write(
bufferedOutputStream *bufio.Writer,
outputIsStdout bool,
) error {
// End of record stream: nothing special for this output format
if outrec == nil {
// End of record stream: nothing special for this output format
return nil
}

2
pkg/output/record_writer_xtab.go
View file

@ -48,8 +48,8 @@ func (writer *RecordWriterXTAB) Write(
bufferedOutputStream *bufio.Writer,
outputIsStdout bool,
) error {
// End of record stream: nothing special for this output format
if outrec == nil {
// End of record stream: nothing special for this output format
return nil
}