mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
Add mlr rank verb (#2178)
* stats1: add rank accumulator (#383) Adds `mlr stats1 -a rank` for standard competition ranking (1,2,2,4,...) on pre-sorted data, most useful with -s for a rank on every record. * stats1: make rank order-independent by default, add --rank-sorted opt-in fast path The rank accumulator previously only compared each value to the immediately preceding record, silently giving wrong ranks for non-adjacent duplicates (e.g. unsorted input, or interleaved -g groups). Default to correctly computing standard competition rank from all values seen so far (order-independent, buffers values, same approach as percentile accumulators). Add --rank-sorted for callers who can promise sorted input and want the previous O(1)-space streaming behavior instead. * Revert "stats1: make rank order-independent by default, add --rank-sorted opt-in fast path" This reverts commitaa45a591fe. * Revert "stats1: add rank accumulator (#383)" This reverts commit96deed048a. * Add mlr rank verb (#383) Reverts the earlier stats1 -a rank / --rank-sorted approach: stats1 is a reduce verb (many records -> one summary record per group) and rank is a per-record annotator, so it never fit cleanly there -- it needed stats1's -s iterative-stats escape hatch just to be useful, plus a bolted-on sorted/unsorted split. mlr rank is a dedicated verb modeled on mlr fraction: -f fields to rank, -g optional group-by, output field <f>_rank. By default it's a two-pass algorithm (buffers input, like fraction does) giving standard competition rank (1,2,2,4,...) that's correct regardless of input order. --sorted opts into a single-pass, O(1)-space streaming alternative for callers who can promise pre-sorted input (e.g. via 'mlr sort' first). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Regenerate docs/man pages after merging main (sparkline verb) --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
a55327c7ef
commit
f9afae491f
36 changed files with 680 additions and 23 deletions
|
|
@ -207,11 +207,11 @@
|
|||
count-similar cut decimate describe fill-down fill-empty filter flatten
|
||||
format-values fraction gap grep group-by group-like gsub having-fields head
|
||||
histogram json-parse json-stringify join label latin1-to-utf8 least-frequent
|
||||
merge-fields most-frequent nest nothing put regularize remove-empty-columns
|
||||
rename reorder repeat reshape sample sec2gmtdate sec2gmt seqgen shuffle
|
||||
skip-trivial-records sort sort-within-records sparkline sparsify split ssub
|
||||
stats1 stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
unflatten uniq unspace unsparsify
|
||||
merge-fields most-frequent nest nothing put rank regularize
|
||||
remove-empty-columns rename reorder repeat reshape sample sec2gmtdate sec2gmt
|
||||
seqgen shuffle skip-trivial-records sort sort-within-records sparkline
|
||||
sparsify split ssub stats1 stats2 step sub summary surv tac tail tee template
|
||||
top utf8-to-latin1 unflatten uniq unspace unsparsify
|
||||
|
||||
1mFUNCTION LIST0m
|
||||
abs acos acosh antimode any append apply arrayify asin asinh asserting_absent
|
||||
|
|
@ -1791,6 +1791,34 @@
|
|||
|
||||
See also https://miller.readthedocs.io/reference-dsl for more context.
|
||||
|
||||
1mrank0m
|
||||
Usage: mlr rank [options]
|
||||
For each record's value in specified fields, computes the standard
|
||||
competition rank (1,2,2,4,...) of that value among all input records,
|
||||
optionally within groups.
|
||||
E.g. with input records x=10, x=20, x=20, and x=30, emits output records
|
||||
x=10,x_rank=1 x=20,x_rank=2 x=20,x_rank=2 and x=30,x_rank=4.
|
||||
|
||||
Note: by default this is a two-pass algorithm: on the first pass it retains
|
||||
input records and their values; on the second pass it computes ranks and
|
||||
emits output records, in original input order. This means it produces no
|
||||
output until all input is read, but gives correct ranks regardless of input
|
||||
order. Use --sorted for a single-pass streaming alternative.
|
||||
|
||||
Options:
|
||||
-f {a,b,c} Field name(s) to rank.
|
||||
-g {d,e,f} Optional group-by-field name(s).
|
||||
--sorted Promise that the input is already sorted by the field(s) being ranked
|
||||
(within each group, if -g is given). This computes rank in a single
|
||||
streaming pass and O(1) space, by comparing each record's value only
|
||||
to the immediately preceding one, rather than buffering all records
|
||||
to compute an order-independent rank. Produces wrong output if the
|
||||
input is not in fact sorted.
|
||||
-h|--help Show this message.
|
||||
Example: mlr rank -f x data/rank-example.csv
|
||||
Example: mlr rank -f x -g g data/rank-example.csv
|
||||
Example: mlr sort -f x then rank -f x --sorted data/rank-example.csv
|
||||
|
||||
1mregularize0m
|
||||
Usage: mlr regularize [options]
|
||||
Outputs records sorted lexically ascending by keys.
|
||||
|
|
|
|||
44
man/mlr.1
44
man/mlr.1
|
|
@ -249,11 +249,11 @@ altkv bar bootstrap case cat check clean-whitespace count-distinct count
|
|||
count-similar cut decimate describe fill-down fill-empty filter flatten
|
||||
format-values fraction gap grep group-by group-like gsub having-fields head
|
||||
histogram json-parse json-stringify join label latin1-to-utf8 least-frequent
|
||||
merge-fields most-frequent nest nothing put regularize remove-empty-columns
|
||||
rename reorder repeat reshape sample sec2gmtdate sec2gmt seqgen shuffle
|
||||
skip-trivial-records sort sort-within-records sparkline sparsify split ssub
|
||||
stats1 stats2 step sub summary surv tac tail tee template top utf8-to-latin1
|
||||
unflatten uniq unspace unsparsify
|
||||
merge-fields most-frequent nest nothing put rank regularize
|
||||
remove-empty-columns rename reorder repeat reshape sample sec2gmtdate sec2gmt
|
||||
seqgen shuffle skip-trivial-records sort sort-within-records sparkline
|
||||
sparsify split ssub stats1 stats2 step sub summary surv tac tail tee template
|
||||
top utf8-to-latin1 unflatten uniq unspace unsparsify
|
||||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
|
|
@ -2209,6 +2209,40 @@ See also https://miller.readthedocs.io/reference-dsl for more context.
|
|||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
.SS "rank"
|
||||
.if n \{\
|
||||
.RS 0
|
||||
.\}
|
||||
.nf
|
||||
Usage: mlr rank [options]
|
||||
For each record's value in specified fields, computes the standard
|
||||
competition rank (1,2,2,4,...) of that value among all input records,
|
||||
optionally within groups.
|
||||
E.g. with input records x=10, x=20, x=20, and x=30, emits output records
|
||||
x=10,x_rank=1 x=20,x_rank=2 x=20,x_rank=2 and x=30,x_rank=4.
|
||||
|
||||
Note: by default this is a two-pass algorithm: on the first pass it retains
|
||||
input records and their values; on the second pass it computes ranks and
|
||||
emits output records, in original input order. This means it produces no
|
||||
output until all input is read, but gives correct ranks regardless of input
|
||||
order. Use --sorted for a single-pass streaming alternative.
|
||||
|
||||
Options:
|
||||
-f {a,b,c} Field name(s) to rank.
|
||||
-g {d,e,f} Optional group-by-field name(s).
|
||||
--sorted Promise that the input is already sorted by the field(s) being ranked
|
||||
(within each group, if -g is given). This computes rank in a single
|
||||
streaming pass and O(1) space, by comparing each record's value only
|
||||
to the immediately preceding one, rather than buffering all records
|
||||
to compute an order-independent rank. Produces wrong output if the
|
||||
input is not in fact sorted.
|
||||
-h|--help Show this message.
|
||||
Example: mlr rank -f x data/rank-example.csv
|
||||
Example: mlr rank -f x -g g data/rank-example.csv
|
||||
Example: mlr sort -f x then rank -f x --sorted data/rank-example.csv
|
||||
.fi
|
||||
.if n \{\
|
||||
.RE
|
||||
.SS "regularize"
|
||||
.if n \{\
|
||||
.RS 0
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue