mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
Adds a sparkline(array|map) built-in that renders numeric values as a Unicode block-character string (▁▂▃▄▅▆▇█), plus a new `mlr sparkline` verb that summarizes each field's values in record order -- useful for eyeballing trends without external plotting tools. Also adds `-s` to `mlr histogram` to sparkline a field's binned counts (its distribution shape) rather than emitting one record per bin. This is a different chart from `mlr sparkline` (order-independent binning vs. record-order values), and the docs for each cross-reference the other to avoid conflating them. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
d9d72a3ad1
commit
a55327c7ef
34 changed files with 706 additions and 47 deletions
|
|
@ -64,7 +64,7 @@ one-line summary (here trimmed, and then counted, using Miller itself):
|
|||
<pre class="pre-non-highlight-in-pair">
|
||||
[
|
||||
{
|
||||
"count": 662
|
||||
"count": 664
|
||||
}
|
||||
]
|
||||
</pre>
|
||||
|
|
|
|||
|
|
@ -257,14 +257,14 @@ This is simply a copy of what you should see on running `man mlr` at a command p
|
|||
null_count os percentile percentiles pow qnorm reduce regextract
|
||||
regextract_or_else rightpad round roundm rstrip sec2dhms sec2gmt sec2gmtdate
|
||||
sec2hms sec2localdate sec2localtime select sgn sha1 sha256 sha512 sin sinh
|
||||
skewness sort sort_collection splita splitax splitkv splitkvx splitnv splitnvx
|
||||
sqrt ssub stat stddev strfntime strfntime_local strftime strftime_local string
|
||||
strip strlen strmatch strmatchx strpntime strpntime_local strptime
|
||||
strptime_local sub substr substr0 substr1 sum sum2 sum3 sum4 sysntime system
|
||||
systime systimeint tan tanh tolower toupper truncate typeof unflatten unformat
|
||||
unformatx upntime uptime urand urand32 urandelement urandint urandrange
|
||||
utf8_to_latin1 variance version ! != !=~ % & && * ** + - . .* .+ .- ./ / // <
|
||||
<< <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
skewness sort sort_collection sparkline splita splitax splitkv splitkvx
|
||||
splitnv splitnvx sqrt ssub stat stddev strfntime strfntime_local strftime
|
||||
strftime_local string strip strlen strmatch strmatchx strpntime
|
||||
strpntime_local strptime strptime_local sub substr substr0 substr1 sum sum2
|
||||
sum3 sum4 sysntime system systime systimeint tan tanh tolower toupper truncate
|
||||
typeof unflatten unformat unformatx upntime uptime urand urand32 urandelement
|
||||
urandint urandrange utf8_to_latin1 variance version ! != !=~ % & && * ** + - .
|
||||
.* .+ .- ./ / // < << <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
|
||||
1mCOMMENTS-IN-DATA FLAGS0m
|
||||
Miller lets you put comments in your data, such as
|
||||
|
|
@ -1462,7 +1462,11 @@ This is simply a copy of what you should see on running `man mlr` at a command p
|
|||
--auto Automatically computes limits, ignoring --lo and --hi. Holds all
|
||||
values in memory before producing any output.
|
||||
-o {prefix} Prefix for output field name. Default: no prefix.
|
||||
-s Print a one-line Unicode sparkline per field instead of per-bin
|
||||
counts.
|
||||
-h|--help Show this message.
|
||||
With -s, output is one record per value-field, with a sparkline field
|
||||
instead of one record per bin.
|
||||
|
||||
1mjson-parse0m
|
||||
Usage: mlr json-parse [options]
|
||||
|
|
@ -3284,6 +3288,12 @@ This is simply a copy of what you should see on running `man mlr` at a command p
|
|||
1msort_collection0m
|
||||
(class=stats #args=1) This is a helper function for the percentiles function; please see its online help for details.
|
||||
|
||||
1msparkline0m
|
||||
(class=stats #args=1) Returns a string of Unicode block characters (one of per element) representing the relative magnitudes of values in an array or map, for a compact ASCII/Unicode bar chart. Returns error for non-array/non-map types.
|
||||
Examples:
|
||||
sparkline([1,2,3,4,5,6,7,8]) is ""
|
||||
sparkline([3,3,3]) is ""
|
||||
|
||||
1msplita0m
|
||||
(class=conversion #args=2) Splits string into array with type inference. First argument is string to split; second is the separator to split on.
|
||||
Example:
|
||||
|
|
@ -4062,5 +4072,5 @@ This is simply a copy of what you should see on running `man mlr` at a command p
|
|||
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
|
||||
https://miller.readthedocs.io
|
||||
|
||||
2026-07-06 4mMILLER24m(1)
|
||||
2026-07-07 4mMILLER24m(1)
|
||||
</pre>
|
||||
|
|
|
|||
|
|
@ -209,8 +209,8 @@
|
|||
histogram json-parse json-stringify join label latin1-to-utf8 least-frequent
|
||||
merge-fields most-frequent nest nothing put regularize remove-empty-columns
|
||||
rename reorder repeat reshape sample sec2gmtdate sec2gmt seqgen shuffle
|
||||
skip-trivial-records sort sort-within-records sparsify split ssub stats1
|
||||
stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
skip-trivial-records sort sort-within-records sparkline sparsify split ssub
|
||||
stats1 stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
unflatten uniq unspace unsparsify
|
||||
|
||||
1mFUNCTION LIST0m
|
||||
|
|
@ -236,14 +236,14 @@
|
|||
null_count os percentile percentiles pow qnorm reduce regextract
|
||||
regextract_or_else rightpad round roundm rstrip sec2dhms sec2gmt sec2gmtdate
|
||||
sec2hms sec2localdate sec2localtime select sgn sha1 sha256 sha512 sin sinh
|
||||
skewness sort sort_collection splita splitax splitkv splitkvx splitnv splitnvx
|
||||
sqrt ssub stat stddev strfntime strfntime_local strftime strftime_local string
|
||||
strip strlen strmatch strmatchx strpntime strpntime_local strptime
|
||||
strptime_local sub substr substr0 substr1 sum sum2 sum3 sum4 sysntime system
|
||||
systime systimeint tan tanh tolower toupper truncate typeof unflatten unformat
|
||||
unformatx upntime uptime urand urand32 urandelement urandint urandrange
|
||||
utf8_to_latin1 variance version ! != !=~ % & && * ** + - . .* .+ .- ./ / // <
|
||||
<< <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
skewness sort sort_collection sparkline splita splitax splitkv splitkvx
|
||||
splitnv splitnvx sqrt ssub stat stddev strfntime strfntime_local strftime
|
||||
strftime_local string strip strlen strmatch strmatchx strpntime
|
||||
strpntime_local strptime strptime_local sub substr substr0 substr1 sum sum2
|
||||
sum3 sum4 sysntime system systime systimeint tan tanh tolower toupper truncate
|
||||
typeof unflatten unformat unformatx upntime uptime urand urand32 urandelement
|
||||
urandint urandrange utf8_to_latin1 variance version ! != !=~ % & && * ** + - .
|
||||
.* .+ .- ./ / // < << <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
|
||||
1mCOMMENTS-IN-DATA FLAGS0m
|
||||
Miller lets you put comments in your data, such as
|
||||
|
|
@ -1441,7 +1441,11 @@
|
|||
--auto Automatically computes limits, ignoring --lo and --hi. Holds all
|
||||
values in memory before producing any output.
|
||||
-o {prefix} Prefix for output field name. Default: no prefix.
|
||||
-s Print a one-line Unicode sparkline per field instead of per-bin
|
||||
counts.
|
||||
-h|--help Show this message.
|
||||
With -s, output is one record per value-field, with a sparkline field
|
||||
instead of one record per bin.
|
||||
|
||||
1mjson-parse0m
|
||||
Usage: mlr json-parse [options]
|
||||
|
|
@ -2062,6 +2066,16 @@
|
|||
-n Sort field names naturally (e.g. 2 before 12). Combines with -f/-r.
|
||||
-h|--help Show this message.
|
||||
|
||||
1msparkline0m
|
||||
Usage: mlr sparkline [options]
|
||||
Reduces numeric field(s), across all records in input order, to a compact
|
||||
Unicode sparkline -- one block character per record -- for visualizing
|
||||
trends. Emits one output record per field. Holds all records in memory
|
||||
before producing any output.
|
||||
Options:
|
||||
-f {a,b,c} Field names to sparkline.
|
||||
-h|--help Show this message.
|
||||
|
||||
1msparsify0m
|
||||
Usage: mlr sparsify [options]
|
||||
Unsets fields for which the key is the empty string (or, optionally, another
|
||||
|
|
@ -3263,6 +3277,12 @@
|
|||
1msort_collection0m
|
||||
(class=stats #args=1) This is a helper function for the percentiles function; please see its online help for details.
|
||||
|
||||
1msparkline0m
|
||||
(class=stats #args=1) Returns a string of Unicode block characters (one of per element) representing the relative magnitudes of values in an array or map, for a compact ASCII/Unicode bar chart. Returns error for non-array/non-map types.
|
||||
Examples:
|
||||
sparkline([1,2,3,4,5,6,7,8]) is ""
|
||||
sparkline([3,3,3]) is ""
|
||||
|
||||
1msplita0m
|
||||
(class=conversion #args=2) Splits string into array with type inference. First argument is string to split; second is the separator to split on.
|
||||
Example:
|
||||
|
|
@ -4041,4 +4061,4 @@
|
|||
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
|
||||
https://miller.readthedocs.io
|
||||
|
||||
2026-07-06 4mMILLER24m(1)
|
||||
2026-07-07 4mMILLER24m(1)
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ Operators are listed here along with functions. In this case, the argument count
|
|||
* [**Hashing functions**](#hashing-functions): [md5](#md5), [sha1](#sha1), [sha256](#sha256), [sha512](#sha512).
|
||||
* [**Higher-order-functions functions**](#higher-order-functions-functions): [any](#any), [apply](#apply), [every](#every), [fold](#fold), [reduce](#reduce), [select](#select), [sort](#sort).
|
||||
* [**Math functions**](#math-functions): [abs](#abs), [acos](#acos), [acosh](#acosh), [asin](#asin), [asinh](#asinh), [atan](#atan), [atan2](#atan2), [atanh](#atanh), [cbrt](#cbrt), [ceil](#ceil), [cos](#cos), [cosh](#cosh), [erf](#erf), [erfc](#erfc), [exp](#exp), [expm1](#expm1), [floor](#floor), [invqnorm](#invqnorm), [log](#log), [log10](#log10), [log1p](#log1p), [logifit](#logifit), [max](#max), [min](#min), [qnorm](#qnorm), [round](#round), [roundm](#roundm), [sgn](#sgn), [sin](#sin), [sinh](#sinh), [sqrt](#sqrt), [tan](#tan), [tanh](#tanh), [urand](#urand), [urand32](#urand32), [urandelement](#urandelement), [urandint](#urandint), [urandrange](#urandrange).
|
||||
* [**Stats functions**](#stats-functions): [antimode](#antimode), [count](#count), [distinct_count](#distinct_count), [kurtosis](#kurtosis), [maxlen](#maxlen), [mean](#mean), [meaneb](#meaneb), [median](#median), [minlen](#minlen), [mode](#mode), [null_count](#null_count), [percentile](#percentile), [percentiles](#percentiles), [skewness](#skewness), [sort_collection](#sort_collection), [stddev](#stddev), [sum](#sum), [sum2](#sum2), [sum3](#sum3), [sum4](#sum4), [variance](#variance).
|
||||
* [**Stats functions**](#stats-functions): [antimode](#antimode), [count](#count), [distinct_count](#distinct_count), [kurtosis](#kurtosis), [maxlen](#maxlen), [mean](#mean), [meaneb](#meaneb), [median](#median), [minlen](#minlen), [mode](#mode), [null_count](#null_count), [percentile](#percentile), [percentiles](#percentiles), [skewness](#skewness), [sort_collection](#sort_collection), [sparkline](#sparkline), [stddev](#stddev), [sum](#sum), [sum2](#sum2), [sum3](#sum3), [sum4](#sum4), [variance](#variance).
|
||||
* [**String functions**](#string-functions): [base64_decode](#base64_decode), [base64_encode](#base64_encode), [capitalize](#capitalize), [clean_whitespace](#clean_whitespace), [collapse_whitespace](#collapse_whitespace), [contains](#contains), [format](#format), [gssub](#gssub), [gsub](#gsub), [hex_decode](#hex_decode), [hex_encode](#hex_encode), [index](#index), [latin1_to_utf8](#latin1_to_utf8), [leftpad](#leftpad), [lstrip](#lstrip), [regextract](#regextract), [regextract_or_else](#regextract_or_else), [rightpad](#rightpad), [rstrip](#rstrip), [ssub](#ssub), [strip](#strip), [strlen](#strlen), [strmatch](#strmatch), [strmatchx](#strmatchx), [sub](#sub), [substr](#substr), [substr0](#substr0), [substr1](#substr1), [tolower](#tolower), [toupper](#toupper), [truncate](#truncate), [unformat](#unformat), [unformatx](#unformatx), [utf8_to_latin1](#utf8_to_latin1), [\.](#dot).
|
||||
* [**System functions**](#system-functions): [exec](#exec), [hostname](#hostname), [next](#next), [os](#os), [stat](#stat), [system](#system), [version](#version).
|
||||
* [**Time functions**](#time-functions): [dhms2fsec](#dhms2fsec), [dhms2sec](#dhms2sec), [fsec2dhms](#fsec2dhms), [fsec2hms](#fsec2hms), [gmt2localtime](#gmt2localtime), [gmt2nsec](#gmt2nsec), [gmt2sec](#gmt2sec), [hms2fsec](#hms2fsec), [hms2sec](#hms2sec), [localtime2gmt](#localtime2gmt), [localtime2nsec](#localtime2nsec), [localtime2sec](#localtime2sec), [nsec2gmt](#nsec2gmt), [nsec2gmtdate](#nsec2gmtdate), [nsec2localdate](#nsec2localdate), [nsec2localtime](#nsec2localtime), [sec2dhms](#sec2dhms), [sec2gmt](#sec2gmt), [sec2gmtdate](#sec2gmtdate), [sec2hms](#sec2hms), [sec2localdate](#sec2localdate), [sec2localtime](#sec2localtime), [strfntime](#strfntime), [strfntime_local](#strfntime_local), [strftime](#strftime), [strftime_local](#strftime_local), [strpntime](#strpntime), [strpntime_local](#strpntime_local), [strptime](#strptime), [strptime_local](#strptime_local), [sysntime](#sysntime), [systime](#systime), [systimeint](#systimeint), [upntime](#upntime), [uptime](#uptime).
|
||||
|
|
@ -1156,6 +1156,15 @@ sort_collection (class=stats #args=1) This is a helper function for the percent
|
|||
</pre>
|
||||
|
||||
|
||||
### sparkline
|
||||
<pre class="pre-non-highlight-non-pair">
|
||||
sparkline (class=stats #args=1) Returns a string of Unicode block characters (one of ▁▂▃▄▅▆▇█ per element) representing the relative magnitudes of values in an array or map, for a compact ASCII/Unicode bar chart. Returns error for non-array/non-map types.
|
||||
Examples:
|
||||
sparkline([1,2,3,4,5,6,7,8]) is "▁▂▃▄▅▆▇█"
|
||||
sparkline([3,3,3]) is "▁▁▁"
|
||||
</pre>
|
||||
|
||||
|
||||
### stddev
|
||||
<pre class="pre-non-highlight-non-pair">
|
||||
stddev (class=stats #args=1) Returns the sample standard deviation of values in an array or map. Returns empty string AKA void for array/map of length less than two; returns error for non-array/non-map types.
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ These fall into categories as follows:
|
|||
|
||||
* `awk`-like functionality: [filter](reference-verbs.md#filter), [put](reference-verbs.md#put), [sec2gmt](reference-verbs.md#sec2gmt), [sec2gmtdate](reference-verbs.md#sec2gmtdate), [step](reference-verbs.md#step), [tee](reference-verbs.md#tee).
|
||||
|
||||
* Statistically oriented: [bar](reference-verbs.md#bar), [bootstrap](reference-verbs.md#bootstrap), [decimate](reference-verbs.md#decimate), [histogram](reference-verbs.md#histogram), [least-frequent](reference-verbs.md#least-frequent), [most-frequent](reference-verbs.md#most-frequent), [sample](reference-verbs.md#sample), [shuffle](reference-verbs.md#shuffle), [stats1](reference-verbs.md#stats1), [stats2](reference-verbs.md#stats2).
|
||||
* Statistically oriented: [bar](reference-verbs.md#bar), [bootstrap](reference-verbs.md#bootstrap), [decimate](reference-verbs.md#decimate), [histogram](reference-verbs.md#histogram), [least-frequent](reference-verbs.md#least-frequent), [most-frequent](reference-verbs.md#most-frequent), [sample](reference-verbs.md#sample), [shuffle](reference-verbs.md#shuffle), [sparkline](reference-verbs.md#sparkline), [stats1](reference-verbs.md#stats1), [stats2](reference-verbs.md#stats2).
|
||||
|
||||
* Particularly oriented toward [Record Heterogeneity](record-heterogeneity.md), although all Miller commands can handle heterogeneous records: [group-by](reference-verbs.md#group-by), [group-like](reference-verbs.md#group-like), [having-fields](reference-verbs.md#having-fields).
|
||||
|
||||
|
|
@ -1787,7 +1787,11 @@ Options:
|
|||
--auto Automatically computes limits, ignoring --lo and --hi. Holds all
|
||||
values in memory before producing any output.
|
||||
-o {prefix} Prefix for output field name. Default: no prefix.
|
||||
-s Print a one-line Unicode sparkline per field instead of per-bin
|
||||
counts.
|
||||
-h|--help Show this message.
|
||||
With -s, output is one record per value-field, with a sparkline field
|
||||
instead of one record per bin.
|
||||
</pre>
|
||||
|
||||
This is just a histogram; there's not too much to say here. A note about binning, by example: Suppose you use `--lo 0.0 --hi 1.0 --nbins 10 -f x`. The input numbers less than 0 or greater than 1 aren't counted in any bin. Input numbers equal to 1 are counted in the last bin. That is, bin 0 has `0.0 < x < 0.1`, bin 1 has `0.1 < x < 0.2`, etc., but bin 9 has `0.9 < x < 1.0`.
|
||||
|
|
@ -1830,6 +1834,20 @@ my_bin_lo my_bin_hi my_x_count my_x2_count my_x3_count
|
|||
0.9 1 1013 507 341
|
||||
</pre>
|
||||
|
||||
Use `-s` for a compact one-line Unicode sparkline per field, rather than one output record per bin. Note this sparklines the *binned counts* -- i.e. the shape of the value distribution -- not the field's raw values in record order; for the latter, use [sparkline](reference-verbs.md#sparkline) instead:
|
||||
|
||||
<pre class="pre-highlight-in-pair">
|
||||
<b>mlr --opprint put '$x2=$x**2;$x3=$x2*$x' \</b>
|
||||
<b> then histogram -f x,x2,x3 --lo 0 --hi 1 --nbins 10 -s \</b>
|
||||
<b> data/medium</b>
|
||||
</pre>
|
||||
<pre class="pre-non-highlight-in-pair">
|
||||
field lo hi sparkline
|
||||
x 0 1 █▁▆▄▂▄▅▅▄▅
|
||||
x2 0 1 █▃▂▂▂▁▁▁▁▁
|
||||
x3 0 1 █▂▂▂▁▁▁▁▁▁
|
||||
</pre>
|
||||
|
||||
## join
|
||||
|
||||
<pre class="pre-highlight-in-pair">
|
||||
|
|
@ -3392,6 +3410,38 @@ a b c
|
|||
9 8 7
|
||||
</pre>
|
||||
|
||||
## sparkline
|
||||
|
||||
<pre class="pre-highlight-in-pair">
|
||||
<b>mlr sparkline --help</b>
|
||||
</pre>
|
||||
<pre class="pre-non-highlight-in-pair">
|
||||
Usage: mlr sparkline [options]
|
||||
Reduces numeric field(s), across all records in input order, to a compact
|
||||
Unicode sparkline -- one block character per record -- for visualizing
|
||||
trends. Emits one output record per field. Holds all records in memory
|
||||
before producing any output.
|
||||
Options:
|
||||
-f {a,b,c} Field names to sparkline.
|
||||
-h|--help Show this message.
|
||||
</pre>
|
||||
|
||||
This reduces one or more numeric fields, in input-record order, to a compact
|
||||
Unicode sparkline -- a quick way to eyeball a trend across records without
|
||||
plotting software. Contrast with [histogram](reference-verbs.md#histogram)
|
||||
`-s`, which sparklines the *distribution* of a field's values (binned by
|
||||
value, order-independent) rather than the field's values in record order.
|
||||
|
||||
<pre class="pre-highlight-in-pair">
|
||||
<b>mlr --icsv --opprint sparkline -f index,quantity,rate example.csv</b>
|
||||
</pre>
|
||||
<pre class="pre-non-highlight-in-pair">
|
||||
field n lo hi sparkline
|
||||
index 10 11 91 ▁▁▁▄▅▆▆▆██
|
||||
quantity 10 13.8103 81.229 ▄█▁████▆▆▇
|
||||
rate 10 0.013 9.887 █▁▃▆▇█▅▄▇▇
|
||||
</pre>
|
||||
|
||||
## sparsify
|
||||
|
||||
<pre class="pre-highlight-in-pair">
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ These fall into categories as follows:
|
|||
|
||||
* `awk`-like functionality: [filter](reference-verbs.md#filter), [put](reference-verbs.md#put), [sec2gmt](reference-verbs.md#sec2gmt), [sec2gmtdate](reference-verbs.md#sec2gmtdate), [step](reference-verbs.md#step), [tee](reference-verbs.md#tee).
|
||||
|
||||
* Statistically oriented: [bar](reference-verbs.md#bar), [bootstrap](reference-verbs.md#bootstrap), [decimate](reference-verbs.md#decimate), [histogram](reference-verbs.md#histogram), [least-frequent](reference-verbs.md#least-frequent), [most-frequent](reference-verbs.md#most-frequent), [sample](reference-verbs.md#sample), [shuffle](reference-verbs.md#shuffle), [stats1](reference-verbs.md#stats1), [stats2](reference-verbs.md#stats2).
|
||||
* Statistically oriented: [bar](reference-verbs.md#bar), [bootstrap](reference-verbs.md#bootstrap), [decimate](reference-verbs.md#decimate), [histogram](reference-verbs.md#histogram), [least-frequent](reference-verbs.md#least-frequent), [most-frequent](reference-verbs.md#most-frequent), [sample](reference-verbs.md#sample), [shuffle](reference-verbs.md#shuffle), [sparkline](reference-verbs.md#sparkline), [stats1](reference-verbs.md#stats1), [stats2](reference-verbs.md#stats2).
|
||||
|
||||
* Particularly oriented toward [Record Heterogeneity](record-heterogeneity.md), although all Miller commands can handle heterogeneous records: [group-by](reference-verbs.md#group-by), [group-like](reference-verbs.md#group-like), [having-fields](reference-verbs.md#having-fields).
|
||||
|
||||
|
|
@ -579,6 +579,14 @@ mlr --opprint put '$x2=$x**2;$x3=$x2*$x' \
|
|||
data/medium
|
||||
GENMD-EOF
|
||||
|
||||
Use `-s` for a compact one-line Unicode sparkline per field, rather than one output record per bin. Note this sparklines the *binned counts* -- i.e. the shape of the value distribution -- not the field's raw values in record order; for the latter, use [sparkline](reference-verbs.md#sparkline) instead:
|
||||
|
||||
GENMD-RUN-COMMAND
|
||||
mlr --opprint put '$x2=$x**2;$x3=$x2*$x' \
|
||||
then histogram -f x,x2,x3 --lo 0 --hi 1 --nbins 10 -s \
|
||||
data/medium
|
||||
GENMD-EOF
|
||||
|
||||
## join
|
||||
|
||||
GENMD-RUN-COMMAND
|
||||
|
|
@ -1017,6 +1025,22 @@ GENMD-RUN-COMMAND
|
|||
mlr --ijson --opprint sort-within-records data/sort-within-records.json
|
||||
GENMD-EOF
|
||||
|
||||
## sparkline
|
||||
|
||||
GENMD-RUN-COMMAND
|
||||
mlr sparkline --help
|
||||
GENMD-EOF
|
||||
|
||||
This reduces one or more numeric fields, in input-record order, to a compact
|
||||
Unicode sparkline -- a quick way to eyeball a trend across records without
|
||||
plotting software. Contrast with [histogram](reference-verbs.md#histogram)
|
||||
`-s`, which sparklines the *distribution* of a field's values (binned by
|
||||
value, order-independent) rather than the field's values in record order.
|
||||
|
||||
GENMD-RUN-COMMAND
|
||||
mlr --icsv --opprint sparkline -f index,quantity,rate example.csv
|
||||
GENMD-EOF
|
||||
|
||||
## sparsify
|
||||
|
||||
GENMD-RUN-COMMAND
|
||||
|
|
|
|||
|
|
@ -209,8 +209,8 @@
|
|||
histogram json-parse json-stringify join label latin1-to-utf8 least-frequent
|
||||
merge-fields most-frequent nest nothing put regularize remove-empty-columns
|
||||
rename reorder repeat reshape sample sec2gmtdate sec2gmt seqgen shuffle
|
||||
skip-trivial-records sort sort-within-records sparsify split ssub stats1
|
||||
stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
skip-trivial-records sort sort-within-records sparkline sparsify split ssub
|
||||
stats1 stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
unflatten uniq unspace unsparsify
|
||||
|
||||
1mFUNCTION LIST0m
|
||||
|
|
@ -236,14 +236,14 @@
|
|||
null_count os percentile percentiles pow qnorm reduce regextract
|
||||
regextract_or_else rightpad round roundm rstrip sec2dhms sec2gmt sec2gmtdate
|
||||
sec2hms sec2localdate sec2localtime select sgn sha1 sha256 sha512 sin sinh
|
||||
skewness sort sort_collection splita splitax splitkv splitkvx splitnv splitnvx
|
||||
sqrt ssub stat stddev strfntime strfntime_local strftime strftime_local string
|
||||
strip strlen strmatch strmatchx strpntime strpntime_local strptime
|
||||
strptime_local sub substr substr0 substr1 sum sum2 sum3 sum4 sysntime system
|
||||
systime systimeint tan tanh tolower toupper truncate typeof unflatten unformat
|
||||
unformatx upntime uptime urand urand32 urandelement urandint urandrange
|
||||
utf8_to_latin1 variance version ! != !=~ % & && * ** + - . .* .+ .- ./ / // <
|
||||
<< <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
skewness sort sort_collection sparkline splita splitax splitkv splitkvx
|
||||
splitnv splitnvx sqrt ssub stat stddev strfntime strfntime_local strftime
|
||||
strftime_local string strip strlen strmatch strmatchx strpntime
|
||||
strpntime_local strptime strptime_local sub substr substr0 substr1 sum sum2
|
||||
sum3 sum4 sysntime system systime systimeint tan tanh tolower toupper truncate
|
||||
typeof unflatten unformat unformatx upntime uptime urand urand32 urandelement
|
||||
urandint urandrange utf8_to_latin1 variance version ! != !=~ % & && * ** + - .
|
||||
.* .+ .- ./ / // < << <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
|
||||
1mCOMMENTS-IN-DATA FLAGS0m
|
||||
Miller lets you put comments in your data, such as
|
||||
|
|
@ -1441,7 +1441,11 @@
|
|||
--auto Automatically computes limits, ignoring --lo and --hi. Holds all
|
||||
values in memory before producing any output.
|
||||
-o {prefix} Prefix for output field name. Default: no prefix.
|
||||
-s Print a one-line Unicode sparkline per field instead of per-bin
|
||||
counts.
|
||||
-h|--help Show this message.
|
||||
With -s, output is one record per value-field, with a sparkline field
|
||||
instead of one record per bin.
|
||||
|
||||
1mjson-parse0m
|
||||
Usage: mlr json-parse [options]
|
||||
|
|
@ -2062,6 +2066,16 @@
|
|||
-n Sort field names naturally (e.g. 2 before 12). Combines with -f/-r.
|
||||
-h|--help Show this message.
|
||||
|
||||
1msparkline0m
|
||||
Usage: mlr sparkline [options]
|
||||
Reduces numeric field(s), across all records in input order, to a compact
|
||||
Unicode sparkline -- one block character per record -- for visualizing
|
||||
trends. Emits one output record per field. Holds all records in memory
|
||||
before producing any output.
|
||||
Options:
|
||||
-f {a,b,c} Field names to sparkline.
|
||||
-h|--help Show this message.
|
||||
|
||||
1msparsify0m
|
||||
Usage: mlr sparsify [options]
|
||||
Unsets fields for which the key is the empty string (or, optionally, another
|
||||
|
|
@ -3263,6 +3277,12 @@
|
|||
1msort_collection0m
|
||||
(class=stats #args=1) This is a helper function for the percentiles function; please see its online help for details.
|
||||
|
||||
1msparkline0m
|
||||
(class=stats #args=1) Returns a string of Unicode block characters (one of per element) representing the relative magnitudes of values in an array or map, for a compact ASCII/Unicode bar chart. Returns error for non-array/non-map types.
|
||||
Examples:
|
||||
sparkline([1,2,3,4,5,6,7,8]) is ""
|
||||
sparkline([3,3,3]) is ""
|
||||
|
||||
1msplita0m
|
||||
(class=conversion #args=2) Splits string into array with type inference. First argument is string to split; second is the separator to split on.
|
||||
Example:
|
||||
|
|
@ -4041,4 +4061,4 @@
|
|||
MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite
|
||||
https://miller.readthedocs.io
|
||||
|
||||
2026-07-06 4mMILLER24m(1)
|
||||
2026-07-07 4mMILLER24m(1)
|
||||
|
|
|
|||
56
man/mlr.1
56
man/mlr.1
|
|
@ -2,12 +2,12 @@
|
|||
.\" Title: mlr
|
||||
.\" Author: [see the "AUTHOR" section]
|
||||
.\" Generator: ./mkman.rb
|
||||
.\" Date: 2026-07-06
|
||||
.\" Date: 2026-07-07
|
||||
.\" Manual: \ \&
|
||||
.\" Source: \ \&
|
||||
.\" Language: English
|
||||
.\"
|
||||
.TH "MILLER" "1" "2026-07-06" "\ \&" "\ \&"
|
||||
.TH "MILLER" "1" "2026-07-07" "\ \&" "\ \&"
|
||||
.\" -----------------------------------------------------------------
|
||||
.\" * Portability definitions
|
||||
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -251,8 +251,8 @@ format-values fraction gap grep group-by group-like gsub having-fields head
|
|||
histogram json-parse json-stringify join label latin1-to-utf8 least-frequent
|
||||
merge-fields most-frequent nest nothing put regularize remove-empty-columns
|
||||
rename reorder repeat reshape sample sec2gmtdate sec2gmt seqgen shuffle
|
||||
skip-trivial-records sort sort-within-records sparsify split ssub stats1
|
||||
stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
skip-trivial-records sort sort-within-records sparkline sparsify split ssub
|
||||
stats1 stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
unflatten uniq unspace unsparsify
|
||||
.fi
|
||||
.if n \{\
|
||||
|
|
@ -284,14 +284,14 @@ minlen mmul mode msub next nsec2gmt nsec2gmtdate nsec2localdate nsec2localtime
|
|||
null_count os percentile percentiles pow qnorm reduce regextract
|
||||
regextract_or_else rightpad round roundm rstrip sec2dhms sec2gmt sec2gmtdate
|
||||
sec2hms sec2localdate sec2localtime select sgn sha1 sha256 sha512 sin sinh
|
||||
skewness sort sort_collection splita splitax splitkv splitkvx splitnv splitnvx
|
||||
sqrt ssub stat stddev strfntime strfntime_local strftime strftime_local string
|
||||
strip strlen strmatch strmatchx strpntime strpntime_local strptime
|
||||
strptime_local sub substr substr0 substr1 sum sum2 sum3 sum4 sysntime system
|
||||
systime systimeint tan tanh tolower toupper truncate typeof unflatten unformat
|
||||
unformatx upntime uptime urand urand32 urandelement urandint urandrange
|
||||
utf8_to_latin1 variance version ! != !=~ % & && * ** + - . .* .+ .- ./ / // <
|
||||
<< <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
skewness sort sort_collection sparkline splita splitax splitkv splitkvx
|
||||
splitnv splitnvx sqrt ssub stat stddev strfntime strfntime_local strftime
|
||||
strftime_local string strip strlen strmatch strmatchx strpntime
|
||||
strpntime_local strptime strptime_local sub substr substr0 substr1 sum sum2
|
||||
sum3 sum4 sysntime system systime systimeint tan tanh tolower toupper truncate
|
||||
typeof unflatten unformat unformatx upntime uptime urand urand32 urandelement
|
||||
urandint urandrange utf8_to_latin1 variance version ! != !=~ % & && * ** + - .
|
||||
\&.* .+ .- ./ / // < << <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~
|
||||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
|
|
@ -1791,7 +1791,11 @@ Options:
|
|||
--auto Automatically computes limits, ignoring --lo and --hi. Holds all
|
||||
values in memory before producing any output.
|
||||
-o {prefix} Prefix for output field name. Default: no prefix.
|
||||
-s Print a one-line Unicode sparkline per field instead of per-bin
|
||||
counts.
|
||||
-h|--help Show this message.
|
||||
With -s, output is one record per value-field, with a sparkline field
|
||||
instead of one record per bin.
|
||||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
|
|
@ -2564,6 +2568,22 @@ Options:
|
|||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
.SS "sparkline"
|
||||
.if n \{\
|
||||
.RS 0
|
||||
.\}
|
||||
.nf
|
||||
Usage: mlr sparkline [options]
|
||||
Reduces numeric field(s), across all records in input order, to a compact
|
||||
Unicode sparkline -- one block character per record -- for visualizing
|
||||
trends. Emits one output record per field. Holds all records in memory
|
||||
before producing any output.
|
||||
Options:
|
||||
-f {a,b,c} Field names to sparkline.
|
||||
-h|--help Show this message.
|
||||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
.SS "sparsify"
|
||||
.if n \{\
|
||||
.RS 0
|
||||
|
|
@ -4931,6 +4951,18 @@ Map without function: sort({"c":2,"a":3,"b":1}, "vnr") returns {"a":3,"c":2,"b":
|
|||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
.SS "sparkline"
|
||||
.if n \{\
|
||||
.RS 0
|
||||
.\}
|
||||
.nf
|
||||
(class=stats #args=1) Returns a string of Unicode block characters (one of ▁▂▃▄▅▆▇█ per element) representing the relative magnitudes of values in an array or map, for a compact ASCII/Unicode bar chart. Returns error for non-array/non-map types.
|
||||
Examples:
|
||||
sparkline([1,2,3,4,5,6,7,8]) is "▁▂▃▄▅▆▇█"
|
||||
sparkline([3,3,3]) is "▁▁▁"
|
||||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
.SS "splita"
|
||||
.if n \{\
|
||||
.RS 0
|
||||
|
|
|
|||
72
pkg/bifs/sparkline.go
Normal file
72
pkg/bifs/sparkline.go
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
package bifs
|
||||
|
||||
import (
|
||||
"github.com/johnkerl/miller/v6/pkg/mlrval"
|
||||
)
|
||||
|
||||
// sparklineTicks are the eighth-height Unicode block characters used to
|
||||
// render array/map values as a compact one-line bar chart.
|
||||
var sparklineTicks = []rune("▁▂▃▄▅▆▇█")
|
||||
|
||||
func BIF_sparkline(collection *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
ok, valueIfNot := check_collection(collection, "sparkline")
|
||||
if !ok {
|
||||
return valueIfNot
|
||||
}
|
||||
|
||||
var floatValues []float64
|
||||
if collection.IsArray() {
|
||||
array := collection.AcquireArrayValue()
|
||||
floatValues = make([]float64, 0, len(array))
|
||||
for _, element := range array {
|
||||
floatValue, isFloat := element.GetNumericToFloatValue()
|
||||
if !isFloat {
|
||||
return mlrval.FromNotNumericError("sparkline", element)
|
||||
}
|
||||
floatValues = append(floatValues, floatValue)
|
||||
}
|
||||
} else {
|
||||
m := collection.AcquireMapValue()
|
||||
floatValues = make([]float64, 0, m.FieldCount)
|
||||
for pe := m.Head; pe != nil; pe = pe.Next {
|
||||
floatValue, isFloat := pe.Value.GetNumericToFloatValue()
|
||||
if !isFloat {
|
||||
return mlrval.FromNotNumericError("sparkline", pe.Value)
|
||||
}
|
||||
floatValues = append(floatValues, floatValue)
|
||||
}
|
||||
}
|
||||
|
||||
if len(floatValues) == 0 {
|
||||
return mlrval.VOID
|
||||
}
|
||||
|
||||
lo := floatValues[0]
|
||||
hi := floatValues[0]
|
||||
for _, floatValue := range floatValues[1:] {
|
||||
if floatValue < lo {
|
||||
lo = floatValue
|
||||
}
|
||||
if floatValue > hi {
|
||||
hi = floatValue
|
||||
}
|
||||
}
|
||||
|
||||
numTicks := len(sparklineTicks)
|
||||
runes := make([]rune, len(floatValues))
|
||||
for i, floatValue := range floatValues {
|
||||
if hi == lo {
|
||||
runes[i] = sparklineTicks[0]
|
||||
continue
|
||||
}
|
||||
tickIndex := int(float64(numTicks-1)*(floatValue-lo)/(hi-lo) + 0.5)
|
||||
if tickIndex < 0 {
|
||||
tickIndex = 0
|
||||
} else if tickIndex >= numTicks {
|
||||
tickIndex = numTicks - 1
|
||||
}
|
||||
runes[i] = sparklineTicks[tickIndex]
|
||||
}
|
||||
|
||||
return mlrval.FromString(string(runes))
|
||||
}
|
||||
60
pkg/bifs/sparkline_test.go
Normal file
60
pkg/bifs/sparkline_test.go
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
package bifs
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
|
||||
"github.com/johnkerl/miller/v6/pkg/mlrval"
|
||||
)
|
||||
|
||||
func TestBIF_sparkline(t *testing.T) {
|
||||
// Needs array or map
|
||||
input := mlrval.FromInt(3)
|
||||
output := BIF_sparkline(input)
|
||||
assert.True(t, output.IsError())
|
||||
|
||||
// Non-numeric element
|
||||
input = mlrval.FromArray([]*mlrval.Mlrval{
|
||||
mlrval.FromInt(1),
|
||||
mlrval.FromString("abc"),
|
||||
})
|
||||
output = BIF_sparkline(input)
|
||||
assert.True(t, output.IsError())
|
||||
|
||||
// Empty array is void
|
||||
input = mlrval.FromArray([]*mlrval.Mlrval{})
|
||||
output = BIF_sparkline(input)
|
||||
assert.True(t, output.IsVoid())
|
||||
|
||||
// Ascending values span the full tick range, low to high
|
||||
input = mlrval.FromArray([]*mlrval.Mlrval{
|
||||
mlrval.FromInt(1),
|
||||
mlrval.FromInt(2),
|
||||
mlrval.FromInt(3),
|
||||
mlrval.FromInt(4),
|
||||
mlrval.FromInt(5),
|
||||
mlrval.FromInt(6),
|
||||
mlrval.FromInt(7),
|
||||
mlrval.FromInt(8),
|
||||
})
|
||||
output = BIF_sparkline(input)
|
||||
assert.True(t, mlrval.Equals(output, mlrval.FromString("▁▂▃▄▅▆▇█")))
|
||||
|
||||
// Same value throughout maps to the lowest tick
|
||||
input = mlrval.FromArray([]*mlrval.Mlrval{
|
||||
mlrval.FromInt(3),
|
||||
mlrval.FromInt(3),
|
||||
mlrval.FromInt(3),
|
||||
})
|
||||
output = BIF_sparkline(input)
|
||||
assert.True(t, mlrval.Equals(output, mlrval.FromString("▁▁▁")))
|
||||
|
||||
// Map input follows insertion order
|
||||
input = array_to_map_for_test(mlrval.FromArray([]*mlrval.Mlrval{
|
||||
mlrval.FromInt(1),
|
||||
mlrval.FromInt(8),
|
||||
}))
|
||||
output = BIF_sparkline(input)
|
||||
assert.True(t, mlrval.Equals(output, mlrval.FromString("▁█")))
|
||||
}
|
||||
|
|
@ -1097,6 +1097,17 @@ is normally distributed.`,
|
|||
},
|
||||
},
|
||||
|
||||
{
|
||||
name: "sparkline",
|
||||
class: FUNC_CLASS_STATS,
|
||||
help: `Returns a string of Unicode block characters (one of ▁▂▃▄▅▆▇█ per element) representing the relative magnitudes of values in an array or map, for a compact ASCII/Unicode bar chart. Returns error for non-array/non-map types.`,
|
||||
unaryFunc: bifs.BIF_sparkline,
|
||||
examples: []string{
|
||||
`sparkline([1,2,3,4,5,6,7,8]) is "▁▂▃▄▅▆▇█"`,
|
||||
`sparkline([3,3,3]) is "▁▁▁"`,
|
||||
},
|
||||
},
|
||||
|
||||
{
|
||||
name: "mode",
|
||||
class: FUNC_CLASS_STATS,
|
||||
|
|
|
|||
|
|
@ -62,6 +62,7 @@ var TRANSFORMER_LOOKUP_TABLE = []TransformerSetup{
|
|||
SkipTrivialRecordsSetup,
|
||||
SortSetup,
|
||||
SortWithinRecordsSetup,
|
||||
SparklineSetup,
|
||||
SparsifySetup,
|
||||
SplitSetup,
|
||||
SsubSetup,
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import (
|
|||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/johnkerl/miller/v6/pkg/bifs"
|
||||
"github.com/johnkerl/miller/v6/pkg/cli"
|
||||
"github.com/johnkerl/miller/v6/pkg/lib"
|
||||
"github.com/johnkerl/miller/v6/pkg/mlrval"
|
||||
|
|
@ -21,6 +22,7 @@ var histogramOptions = []OptionSpec{
|
|||
{Flag: "--nbins", Arg: "{n}", Type: "int", Desc: "Number of histogram bins. Defaults to 20."},
|
||||
{Flag: "--auto", Type: "bool", Desc: "Automatically computes limits, ignoring --lo and --hi. Holds all values in memory before producing any output."},
|
||||
{Flag: "-o", Arg: "{prefix}", Type: "string", Desc: "Prefix for output field name. Default: no prefix."},
|
||||
{Flag: "-s", Type: "bool", Desc: "Print a one-line Unicode sparkline per field instead of per-bin counts."},
|
||||
}
|
||||
|
||||
var HistogramSetup = TransformerSetup{
|
||||
|
|
@ -39,6 +41,8 @@ func transformerHistogramUsage(
|
|||
fmt.Fprintf(o, "Just a histogram. Input values < lo or > hi are not counted.\n")
|
||||
fmt.Fprintf(o, "Usage: %s %s [options]\n", argv0, verb)
|
||||
WriteVerbOptions(o, histogramOptions)
|
||||
fmt.Fprintf(o, "With -s, output is one record per value-field, with a sparkline field\n")
|
||||
fmt.Fprintf(o, "instead of one record per bin.\n")
|
||||
}
|
||||
|
||||
func transformerHistogramParseCLI(
|
||||
|
|
@ -61,6 +65,7 @@ func transformerHistogramParseCLI(
|
|||
hi := 0.0
|
||||
doAuto := false
|
||||
outputPrefix := ""
|
||||
doSparkline := false
|
||||
|
||||
var err error
|
||||
for argi < argc /* variable increment: 1 or 2 depending on flag */ {
|
||||
|
|
@ -111,6 +116,9 @@ func transformerHistogramParseCLI(
|
|||
return nil, err
|
||||
}
|
||||
|
||||
case "-s":
|
||||
doSparkline = true
|
||||
|
||||
default:
|
||||
return nil, cli.VerbErrorf(verb, "option \"%s\" not recognized", opt)
|
||||
}
|
||||
|
|
@ -140,6 +148,7 @@ func transformerHistogramParseCLI(
|
|||
hi,
|
||||
doAuto,
|
||||
outputPrefix,
|
||||
doSparkline,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
|
@ -160,6 +169,7 @@ type TransformerHistogram struct {
|
|||
countsByField map[string][]int64
|
||||
vectorsByFieldName map[string][]float64 // For auto-mode
|
||||
outputPrefix string
|
||||
doSparkline bool
|
||||
|
||||
recordTransformerFunc RecordTransformerFunc
|
||||
}
|
||||
|
|
@ -171,6 +181,7 @@ func NewTransformerHistogram(
|
|||
hi float64,
|
||||
doAuto bool,
|
||||
outputPrefix string,
|
||||
doSparkline bool,
|
||||
) (*TransformerHistogram, error) {
|
||||
|
||||
countsByField := make(map[string][]int64)
|
||||
|
|
@ -185,6 +196,7 @@ func NewTransformerHistogram(
|
|||
valueFieldNames: valueFieldNames,
|
||||
countsByField: countsByField,
|
||||
outputPrefix: outputPrefix,
|
||||
doSparkline: doSparkline,
|
||||
nbins: nbins,
|
||||
}
|
||||
|
||||
|
|
@ -256,10 +268,40 @@ func (tr *TransformerHistogram) ingestNonAuto(
|
|||
}
|
||||
}
|
||||
|
||||
// sparklineRecord summarizes a field's binned counts as a single record with
|
||||
// a Unicode-block sparkline field, rather than one record per bin.
|
||||
func (tr *TransformerHistogram) sparklineRecord(
|
||||
valueFieldName string,
|
||||
lo float64,
|
||||
hi float64,
|
||||
) *mlrval.Mlrmap {
|
||||
counts := tr.countsByField[valueFieldName]
|
||||
countMlrvals := make([]*mlrval.Mlrval, len(counts))
|
||||
for i, count := range counts {
|
||||
countMlrvals[i] = mlrval.FromInt(count)
|
||||
}
|
||||
sparkline := bifs.BIF_sparkline(mlrval.FromArray(countMlrvals))
|
||||
|
||||
outrec := mlrval.NewMlrmapAsRecord()
|
||||
outrec.PutReference(tr.outputPrefix+"field", mlrval.FromString(valueFieldName))
|
||||
outrec.PutReference(tr.outputPrefix+"lo", mlrval.FromFloat(lo))
|
||||
outrec.PutReference(tr.outputPrefix+"hi", mlrval.FromFloat(hi))
|
||||
outrec.PutReference(tr.outputPrefix+"sparkline", sparkline)
|
||||
return outrec
|
||||
}
|
||||
|
||||
func (tr *TransformerHistogram) emitNonAuto(
|
||||
endOfStreamContext *types.Context,
|
||||
outputRecordsAndContexts *[]*types.RecordAndContext, // list of *types.RecordAndContext
|
||||
) {
|
||||
if tr.doSparkline {
|
||||
for _, valueFieldName := range tr.valueFieldNames {
|
||||
outrec := tr.sparklineRecord(valueFieldName, tr.lo, tr.hi)
|
||||
*outputRecordsAndContexts = append(*outputRecordsAndContexts, types.NewRecordAndContext(outrec, endOfStreamContext))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
countFieldNames := make(map[string]string)
|
||||
for _, valueFieldName := range tr.valueFieldNames {
|
||||
countFieldNames[valueFieldName] = tr.outputPrefix + valueFieldName + "_count"
|
||||
|
|
@ -363,6 +405,14 @@ func (tr *TransformerHistogram) emitAuto(
|
|||
}
|
||||
}
|
||||
|
||||
if tr.doSparkline {
|
||||
for _, valueFieldName := range tr.valueFieldNames {
|
||||
outrec := tr.sparklineRecord(valueFieldName, lo, hi)
|
||||
*outputRecordsAndContexts = append(*outputRecordsAndContexts, types.NewRecordAndContext(outrec, endOfStreamContext))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Emission pass
|
||||
countFieldNames := make(map[string]string)
|
||||
for _, valueFieldName := range tr.valueFieldNames {
|
||||
|
|
|
|||
182
pkg/transformers/sparkline.go
Normal file
182
pkg/transformers/sparkline.go
Normal file
|
|
@ -0,0 +1,182 @@
|
|||
package transformers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/johnkerl/miller/v6/pkg/bifs"
|
||||
"github.com/johnkerl/miller/v6/pkg/cli"
|
||||
"github.com/johnkerl/miller/v6/pkg/mlrval"
|
||||
"github.com/johnkerl/miller/v6/pkg/types"
|
||||
)
|
||||
|
||||
const verbNameSparkline = "sparkline"
|
||||
|
||||
var sparklineOptions = []OptionSpec{
|
||||
{Flag: "-f", Arg: "{a,b,c}", Type: "csv-list", Desc: "Field names to sparkline."},
|
||||
}
|
||||
|
||||
var SparklineSetup = TransformerSetup{
|
||||
Verb: verbNameSparkline,
|
||||
UsageFunc: transformerSparklineUsage,
|
||||
ParseCLIFunc: transformerSparklineParseCLI,
|
||||
IgnoresInput: false,
|
||||
Options: sparklineOptions,
|
||||
}
|
||||
|
||||
func transformerSparklineUsage(
|
||||
o *os.File,
|
||||
) {
|
||||
fmt.Fprintf(o, "Usage: %s %s [options]\n", "mlr", verbNameSparkline)
|
||||
fmt.Fprintf(o, "Reduces numeric field(s), across all records in input order, to a compact\n")
|
||||
fmt.Fprintf(o, "Unicode sparkline -- one block character per record -- for visualizing\n")
|
||||
fmt.Fprintf(o, "trends. Emits one output record per field. Holds all records in memory\n")
|
||||
fmt.Fprintf(o, "before producing any output.\n")
|
||||
WriteVerbOptions(o, sparklineOptions)
|
||||
}
|
||||
|
||||
func transformerSparklineParseCLI(
|
||||
pargi *int,
|
||||
argc int,
|
||||
args []string,
|
||||
_ *cli.TOptions,
|
||||
doConstruct bool, // false for first pass of CLI-parse, true for second pass
|
||||
) (RecordTransformer, error) {
|
||||
|
||||
// Skip the verb name from the current spot in the mlr command line
|
||||
argi := *pargi
|
||||
verb := args[argi]
|
||||
argi++
|
||||
|
||||
// Parse local flags
|
||||
var err error
|
||||
var fieldNames []string = nil
|
||||
|
||||
for argi < argc /* variable increment: 1 or 2 depending on flag */ {
|
||||
opt := args[argi]
|
||||
if !strings.HasPrefix(opt, "-") {
|
||||
break // No more flag options to process
|
||||
}
|
||||
if args[argi] == "--" {
|
||||
break // All transformers must do this so main-flags can follow verb-flags
|
||||
}
|
||||
argi++
|
||||
|
||||
switch opt {
|
||||
case "-h", "--help":
|
||||
transformerSparklineUsage(os.Stdout)
|
||||
return nil, cli.ErrHelpRequested
|
||||
|
||||
case "-f":
|
||||
fieldNames, err = cli.VerbGetStringArrayArg(verb, opt, args, &argi, argc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
default:
|
||||
return nil, cli.VerbErrorf(verb, "option \"%s\" not recognized", opt)
|
||||
}
|
||||
}
|
||||
|
||||
if fieldNames == nil {
|
||||
return nil, cli.VerbErrorf(verb, "-f field names required")
|
||||
}
|
||||
|
||||
*pargi = argi
|
||||
if !doConstruct { // All transformers must do this for main command-line parsing
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
transformer, err := NewTransformerSparkline(fieldNames)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return transformer, nil
|
||||
}
|
||||
|
||||
type TransformerSparkline struct {
|
||||
fieldNames []string
|
||||
valuesByField map[string][]*mlrval.Mlrval
|
||||
}
|
||||
|
||||
func NewTransformerSparkline(
|
||||
fieldNames []string,
|
||||
) (*TransformerSparkline, error) {
|
||||
valuesByField := make(map[string][]*mlrval.Mlrval)
|
||||
for _, fieldName := range fieldNames {
|
||||
valuesByField[fieldName] = make([]*mlrval.Mlrval, 0)
|
||||
}
|
||||
return &TransformerSparkline{
|
||||
fieldNames: fieldNames,
|
||||
valuesByField: valuesByField,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (tr *TransformerSparkline) Transform(
|
||||
inrecAndContext *types.RecordAndContext,
|
||||
outputRecordsAndContexts *[]*types.RecordAndContext, // list of *types.RecordAndContext
|
||||
inputDownstreamDoneChannel <-chan bool,
|
||||
outputDownstreamDoneChannel chan<- bool,
|
||||
) {
|
||||
HandleDefaultDownstreamDone(inputDownstreamDoneChannel, outputDownstreamDoneChannel)
|
||||
|
||||
if !inrecAndContext.EndOfStream {
|
||||
inrec := inrecAndContext.Record
|
||||
for _, fieldName := range tr.fieldNames {
|
||||
mvalue := inrec.Get(fieldName)
|
||||
if mvalue != nil {
|
||||
tr.valuesByField[fieldName] = append(tr.valuesByField[fieldName], mvalue.Copy())
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Else, end of stream: emit one summary record per field.
|
||||
for _, fieldName := range tr.fieldNames {
|
||||
values := tr.valuesByField[fieldName]
|
||||
|
||||
outrec := mlrval.NewMlrmapAsRecord()
|
||||
outrec.PutReference("field", mlrval.FromString(fieldName))
|
||||
outrec.PutReference("n", mlrval.FromInt(int64(len(values))))
|
||||
|
||||
sparkline := bifs.BIF_sparkline(mlrval.FromArray(values))
|
||||
if !sparkline.IsError() {
|
||||
lo, hi, haveLoHi := floatRangeOf(values)
|
||||
if haveLoHi {
|
||||
outrec.PutReference("lo", mlrval.FromFloat(lo))
|
||||
outrec.PutReference("hi", mlrval.FromFloat(hi))
|
||||
}
|
||||
}
|
||||
outrec.PutReference("sparkline", sparkline)
|
||||
|
||||
*outputRecordsAndContexts = append(*outputRecordsAndContexts, types.NewRecordAndContext(outrec, &inrecAndContext.Context))
|
||||
}
|
||||
|
||||
*outputRecordsAndContexts = append(*outputRecordsAndContexts, inrecAndContext) // Emit the end-of-stream marker
|
||||
}
|
||||
|
||||
// floatRangeOf returns the min and max of the numeric values, and whether
|
||||
// any were found.
|
||||
func floatRangeOf(values []*mlrval.Mlrval) (lo float64, hi float64, haveLoHi bool) {
|
||||
for _, value := range values {
|
||||
floatValue, isFloat := value.GetNumericToFloatValue()
|
||||
if !isFloat {
|
||||
continue
|
||||
}
|
||||
if !haveLoHi {
|
||||
lo = floatValue
|
||||
hi = floatValue
|
||||
haveLoHi = true
|
||||
} else {
|
||||
if floatValue < lo {
|
||||
lo = floatValue
|
||||
}
|
||||
if floatValue > hi {
|
||||
hi = floatValue
|
||||
}
|
||||
}
|
||||
}
|
||||
return lo, hi, haveLoHi
|
||||
}
|
||||
|
|
@ -474,7 +474,11 @@ Options:
|
|||
--auto Automatically computes limits, ignoring --lo and --hi. Holds all
|
||||
values in memory before producing any output.
|
||||
-o {prefix} Prefix for output field name. Default: no prefix.
|
||||
-s Print a one-line Unicode sparkline per field instead of per-bin
|
||||
counts.
|
||||
-h|--help Show this message.
|
||||
With -s, output is one record per value-field, with a sparkline field
|
||||
instead of one record per bin.
|
||||
|
||||
================================================================
|
||||
json-parse
|
||||
|
|
@ -1120,6 +1124,17 @@ Options:
|
|||
-n Sort field names naturally (e.g. 2 before 12). Combines with -f/-r.
|
||||
-h|--help Show this message.
|
||||
|
||||
================================================================
|
||||
sparkline
|
||||
Usage: mlr sparkline [options]
|
||||
Reduces numeric field(s), across all records in input order, to a compact
|
||||
Unicode sparkline -- one block character per record -- for visualizing
|
||||
trends. Emits one output record per field. Holds all records in memory
|
||||
before producing any output.
|
||||
Options:
|
||||
-f {a,b,c} Field names to sparkline.
|
||||
-h|--help Show this message.
|
||||
|
||||
================================================================
|
||||
sparsify
|
||||
Usage: mlr sparsify [options]
|
||||
|
|
|
|||
1
test/cases/dsl-stats/sparkline/various/cmd
Normal file
1
test/cases/dsl-stats/sparkline/various/cmd
Normal file
|
|
@ -0,0 +1 @@
|
|||
mlr -n --ofmtf 6 --xtab put -f ${CASEDIR}/mlr
|
||||
0
test/cases/dsl-stats/sparkline/various/experr
Normal file
0
test/cases/dsl-stats/sparkline/various/experr
Normal file
22
test/cases/dsl-stats/sparkline/various/expout
Normal file
22
test/cases/dsl-stats/sparkline/various/expout
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
sparkline_0 (error)
|
||||
sparkline_0_type error
|
||||
sparkline_null (error)
|
||||
sparkline_null_type error
|
||||
sparkline_empty_array
|
||||
sparkline_empty_array_type empty
|
||||
sparkline_array_1 ▁
|
||||
sparkline_array_1_type string
|
||||
sparkline_array_flat ▁▁▁
|
||||
sparkline_array_flat_type string
|
||||
sparkline_array_ascending ▁▂▃▄▅▆▇█
|
||||
sparkline_array_ascending_type string
|
||||
sparkline_array_descending █▇▆▅▄▃▂▁
|
||||
sparkline_array_descending_type string
|
||||
sparkline_array_nonnumeric (error)
|
||||
sparkline_array_nonnumeric_type error
|
||||
sparkline_empty_map
|
||||
sparkline_empty_map_type empty
|
||||
sparkline_map_1 ▁
|
||||
sparkline_map_1_type string
|
||||
sparkline_map_ascending ▁▄█
|
||||
sparkline_map_ascending_type string
|
||||
27
test/cases/dsl-stats/sparkline/various/mlr
Normal file
27
test/cases/dsl-stats/sparkline/various/mlr
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
end {
|
||||
outputs = {};
|
||||
|
||||
outputs["sparkline_0"] = sparkline(0);
|
||||
outputs["sparkline_null"] = sparkline(null);
|
||||
outputs["sparkline_nonesuch"] = sparkline(nonesuch);
|
||||
|
||||
outputs["sparkline_empty_array"] = sparkline([]);
|
||||
outputs["sparkline_array_1"] = sparkline([7]);
|
||||
outputs["sparkline_array_flat"] = sparkline([3,3,3]);
|
||||
outputs["sparkline_array_ascending"] = sparkline([1,2,3,4,5,6,7,8]);
|
||||
outputs["sparkline_array_descending"] = sparkline([8,7,6,5,4,3,2,1]);
|
||||
outputs["sparkline_array_nonnumeric"] = sparkline([1,"abc"]);
|
||||
|
||||
outputs["sparkline_empty_map"] = sparkline({});
|
||||
outputs["sparkline_map_1"] = sparkline({ "a" : 7 });
|
||||
outputs["sparkline_map_ascending"] = sparkline({ "a" : 1, "b" : 4, "c" : 8 });
|
||||
|
||||
typed_outputs = {};
|
||||
|
||||
for (k, v in outputs) {
|
||||
typed_outputs[k] = v;
|
||||
typed_outputs[k."_type"] = typeof(v);
|
||||
}
|
||||
|
||||
emit typed_outputs;
|
||||
}
|
||||
1
test/cases/verb-histogram/0008/cmd
Normal file
1
test/cases/verb-histogram/0008/cmd
Normal file
|
|
@ -0,0 +1 @@
|
|||
mlr --icsv --opprint histogram --nbins 9 --auto -f x,y -s test/input/histo2.csv
|
||||
0
test/cases/verb-histogram/0008/experr
Normal file
0
test/cases/verb-histogram/0008/experr
Normal file
3
test/cases/verb-histogram/0008/expout
Normal file
3
test/cases/verb-histogram/0008/expout
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
field lo hi sparkline
|
||||
x 20.00000000 300.00000000 ▁▁▅▁▁█▁▁▅
|
||||
y 20.00000000 300.00000000 █▃▁▁▁▁▁▁▁
|
||||
1
test/cases/verb-sparkline/0001/cmd
Normal file
1
test/cases/verb-sparkline/0001/cmd
Normal file
|
|
@ -0,0 +1 @@
|
|||
mlr --ojson sparkline -f i,x,y test/input/abixy
|
||||
0
test/cases/verb-sparkline/0001/experr
Normal file
0
test/cases/verb-sparkline/0001/experr
Normal file
23
test/cases/verb-sparkline/0001/expout
Normal file
23
test/cases/verb-sparkline/0001/expout
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
[
|
||||
{
|
||||
"field": "i",
|
||||
"n": 10,
|
||||
"lo": 1.00000000,
|
||||
"hi": 10.00000000,
|
||||
"sparkline": "▁▂▃▃▄▅▆▆▇█"
|
||||
},
|
||||
{
|
||||
"field": "x",
|
||||
"n": 10,
|
||||
"lo": 0.03144188,
|
||||
"hi": 0.75867996,
|
||||
"sparkline": "▄█▃▄▆▆▇▆▁▆"
|
||||
},
|
||||
{
|
||||
"field": "y",
|
||||
"n": 10,
|
||||
"lo": 0.13418874,
|
||||
"hi": 0.97618139,
|
||||
"sparkline": "▆▄▃▁▇▄▁█▆█"
|
||||
}
|
||||
]
|
||||
1
test/cases/verb-sparkline/0002/cmd
Normal file
1
test/cases/verb-sparkline/0002/cmd
Normal file
|
|
@ -0,0 +1 @@
|
|||
mlr --opprint sparkline -f i test/input/abixy
|
||||
0
test/cases/verb-sparkline/0002/experr
Normal file
0
test/cases/verb-sparkline/0002/experr
Normal file
2
test/cases/verb-sparkline/0002/expout
Normal file
2
test/cases/verb-sparkline/0002/expout
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
field n lo hi sparkline
|
||||
i 10 1.00000000 10.00000000 ▁▂▃▃▄▅▆▆▇█
|
||||
1
test/cases/verb-sparkline/0003/cmd
Normal file
1
test/cases/verb-sparkline/0003/cmd
Normal file
|
|
@ -0,0 +1 @@
|
|||
mlr sparkline --help
|
||||
0
test/cases/verb-sparkline/0003/experr
Normal file
0
test/cases/verb-sparkline/0003/experr
Normal file
8
test/cases/verb-sparkline/0003/expout
Normal file
8
test/cases/verb-sparkline/0003/expout
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
Usage: mlr sparkline [options]
|
||||
Reduces numeric field(s), across all records in input order, to a compact
|
||||
Unicode sparkline -- one block character per record -- for visualizing
|
||||
trends. Emits one output record per field. Holds all records in memory
|
||||
before producing any output.
|
||||
Options:
|
||||
-f {a,b,c} Field names to sparkline.
|
||||
-h|--help Show this message.
|
||||
1
test/cases/verb-sparkline/0004/cmd
Normal file
1
test/cases/verb-sparkline/0004/cmd
Normal file
|
|
@ -0,0 +1 @@
|
|||
mlr --ojson sparkline -f nonesuch,a test/input/abixy
|
||||
0
test/cases/verb-sparkline/0004/experr
Normal file
0
test/cases/verb-sparkline/0004/experr
Normal file
12
test/cases/verb-sparkline/0004/expout
Normal file
12
test/cases/verb-sparkline/0004/expout
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
[
|
||||
{
|
||||
"field": "nonesuch",
|
||||
"n": 0,
|
||||
"sparkline": ""
|
||||
},
|
||||
{
|
||||
"field": "a",
|
||||
"n": 10,
|
||||
"sparkline": (error)
|
||||
}
|
||||
]
|
||||
Loading…
Add table
Add a link
Reference in a new issue