Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-18 00:45:47 +00:00
Code Issues Projects Releases Wiki Activity
miller/pkg/transformers
History
John Kerl 5976b43d04
Fix exponential memory use when chaining seqgen (#2072) (#2094) 
seqgen.Transform was generating its full sequence on every call,
including once per upstream record. When chaining two seqgens of
N records each, the second produced N+1 copies of N records in
memory (O(N²)) before writing only the first N. Fix: return
immediately for non-EOS input; generate the sequence only when
the upstream end-of-stream arrives, which is the correct
semantics for a verb that discards its input record stream.

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-06-19 18:09:54 -04:00
..
utils Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
aaa_chain_transformer.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
aaa_record_transformer.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
aaa_transformer_table.go Neaten some comments (#1973) 2026-02-16 13:38:49 -05:00
altkv.go yet more staticcheck 2026-03-03 10:17:01 -05:00
bar.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
bootstrap.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
case.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
cat.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
check.go yet more staticcheck 2026-03-03 10:17:01 -05:00
clean_whitespace.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
count.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
count_similar.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
cut.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
decimate.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
doc.go Export library code in pkg/ (#1391) 2023-09-10 17:15:13 -04:00
fill_down.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
fill_empty.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
flatten.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
format_values.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
fraction.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
gap.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
grep.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
group_by.go yet more staticcheck 2026-03-03 10:17:01 -05:00
group_like.go yet more staticcheck 2026-03-03 10:17:01 -05:00
having_fields.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
head.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
histogram.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
join.go Apply join prefixes/rename to unpaired records (#1821) (#2062) 2026-05-17 12:35:58 -04:00
json_parse.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
json_stringify.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
label.go yet more staticcheck 2026-03-03 10:17:01 -05:00
latin1_to_utf8.go yet more staticcheck 2026-03-03 10:17:01 -05:00
merge_fields.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
most_or_least_frequent.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
nest.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
nothing.go yet more staticcheck 2026-03-03 10:17:01 -05:00
put_or_filter.go Replace GOCC parser-generator with PGPG (#2015) 2026-03-15 22:28:57 -04:00
README.md Export library code in pkg/ (#1391) 2023-09-10 17:15:13 -04:00
regularize.go yet more staticcheck 2026-03-03 10:17:01 -05:00
remove_empty_columns.go yet more staticcheck 2026-03-03 10:17:01 -05:00
rename.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
reorder.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
repeat.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
reshape.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
sample.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
sec2gmt.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
sec2gmtdate.go yet more staticcheck 2026-03-03 10:17:01 -05:00
seqgen.go Fix exponential memory use when chaining seqgen (#2072) (#2094) 2026-06-19 18:09:54 -04:00
shuffle.go yet more staticcheck 2026-03-03 10:17:01 -05:00
skip_trivial_records.go yet more staticcheck 2026-03-03 10:17:01 -05:00
sort.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
sort_within_records.go run make fmt 2026-04-19 11:37:23 -04:00
sparsify.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
split.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
stats1.go fix: stats1 null_count with --fr regex gives wrong results (#1994) 2026-04-19 10:52:37 -04:00
stats2.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
step.go Fix race condition in mlr step (#2041) 2026-04-19 09:39:39 -04:00
subs.go Use "\n" in replace for gsub and sub (#2042) 2026-04-19 10:21:51 -04:00
summary.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
surv.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
tac.go yet more staticcheck 2026-03-03 10:17:01 -05:00
tail.go Performance and style fixes (#1981) 2026-02-18 09:19:31 -05:00
tee.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
template.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
top.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
unflatten.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
uniq.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
unspace.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
unsparsify.go Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
utf8_to_latin1.go yet more staticcheck 2026-03-03 10:17:01 -05:00

README.md

Logic for transforming input records into output records as requested by the user (sort, filter, etc.).:

  • The IRecordTransformer abstract record-transformer interface datatype, as well as the ChainTransformer Go-channel chaining mechanism for piping one transformer into the next.
  • The transformer lookup table, used for Miller command-line parsing, verb construction, and online help.
  • All the concrete record-transformers such as cat, tac, sort, put, and so on.