Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-01-23 02:14:13 +00:00
Code Issues Projects Releases Wiki Activity
9124 commits 34 branches 72 tags 221 MiB
Commit graph

96 commits

Author SHA1 Message Date
John Kerl
71d9388bff
Be smarter about auto-unflatten (#1584) 2024-06-08 20:58:26 -04:00
John Kerl
6520bf4758
Bash process substitution not working with put -f (#1583) 
* Bash process substitution not working with `put -f`

* run `make dev`
 2024-06-08 20:37:31 -04:00
John Kerl
202a79d0e2
On-line help for mlr summary --transpose (#1581) 
* On-line help for `mlr summary --transpose`

* run `make dev`
 2024-06-08 13:37:07 -04:00
John Kerl
8223903621
Support $NO_COLOR (#1580) 
* Support `$NO_COLOR`

* run `make dev`
 2024-06-08 13:08:15 -04:00
Andrew Onyshchuk
66abef6704
fraction bugfix (#1579) 2024-06-06 09:04:25 -04:00
John Kerl
16ab199194
Add mad accumulator for stats1 DSL function (#1561) 
* Add `mad` accumulator for `stats1` DSL function

* regression files

* make dev output
 2024-05-11 15:55:27 -04:00
John Kerl
5ac48516f7
Add a stat DSL function (#1560) 
* Add a `stat` DSL function [WIP]

* artifacts from `make dev`

* regression test
 2024-05-09 18:39:44 -04:00
forcedebug
0e30619966
Fix mismatched method names in comments (#1549) 
Signed-off-by: forcedebug <forcedebug@outlook.com>
 2024-04-22 12:16:30 -04:00
camcui
12480c4ab5
chore: fix function name in comment (#1543) 
Signed-off-by: camcui <cuishua@sina.cn>
 2024-04-12 09:38:31 -04:00
John Kerl
e714738a7d
Fix typo in online help for --no-jlistwrap (#1541) 
* Add --no-auto-unsparsify flag

* Fix typo in online help for `--no-jlistwrap`

* Artifacts from `make dev`
 2024-04-11 08:12:45 -04:00
John Kerl
b37c3a5e56 6.12.0 doc link 2024-03-16 17:51:17 -04:00
John Kerl
a0bead4093 miller 6.12.0 2024-03-16 17:19:05 -04:00
John Kerl
83c44e6d74
Add descriptions for put and filter verbs (#1529) 
* Add more info in online help about what put/filter do

* `make dev` artifacts
 2024-03-16 17:09:01 -04:00
John Kerl
f01bb92da7
Avoid spurious [] on JSON output in some cases (#1528) 
* JSON empty vs `[]` handling [WIP]

* unit-test mods
 2024-03-16 17:00:59 -04:00
John Kerl
aff4b9f32d
Improved file-not-found handling (#1508) 2024-02-26 00:12:31 -05:00
John Kerl
fb1f7f8421
Enable record-hashing by default (#1507) 
* Enable record-hashing by default

* comments
 2024-02-25 21:51:41 -05:00
John Kerl
3ff43fa818
Miller produces no output on TSV with > 64K characters per line (#1505) 
* Switch to bufio.Reader, first pass

* temp

* Simplify ILineReader by making it stateless

* Interface not necessary; ILineReader -> TLineReader

* neaten

* iterating
 2024-02-25 15:50:50 -05:00
John Kerl
57b32c3e9b
Separate out ILineReader abstraction (#1504) 
* Split up pkg/input/record_reader.go

* new ILineReader/TLineReader
 2024-02-24 22:07:56 -05:00
John Kerl
7bd460a3b8
Support thousands separator in fmtnum (#1499) 
* Support thousands separator in `fmtnum`

* doc bits
 2024-02-18 14:01:46 -05:00
John Kerl
f5eaf290cf
mlr sparsify (#1498) 
* mlr sparsify

* regression-test cases

* typofix

* Remove mods due to processor-architecture change
 2024-02-18 10:56:26 -05:00
John Kerl
c0e9be0e0c
6.11.0-dev (#1484) 
* 6.11.0-dev

* 6.11.0-dev
 2024-01-24 13:27:04 -05:00
John Kerl
1834a925b3
Miller 6.11.0 (#1481) 
* miller 6.11.0

* Artifacts from `make dev`
 2024-01-23 17:31:58 -05:00
John Kerl
e5ec9f67bd
Implement all/by-regex field selection (-a/-r) for mlr sub, gsub, and ssub (#1480) 
* Code-dedupe `sub`, `gsub`, and `ssub` verbs

* More dedupe

* Start with -a

* Implement -r

* unit-test cases

* Windows command-line parsing
 2024-01-23 17:18:13 -05:00
John Kerl
81d11365a0
mlr reorder with regex support [WIP] (#1473) 
* mlr reorder with regex support for field-name selection

* neaten

* -r -b/-a; unit-test cases
 2024-01-21 15:17:33 -05:00
John Kerl
ac65675ab1
Auto-unsparsify CSV and TSV on output (#1479) 
* Auto-unsparsify CSV

* Update unit-test cases

* More unit-test cases

* Key-change handling for CSV output

* Same for TSV, with unit-test and doc updates
 2024-01-20 18:43:49 -05:00
John Kerl
af021f28d7
Support markdown format on input (#1478) 
* Support markdown on input

* unit-test files

* doc mods

* Unit-test cases for I/O-format keystroke-savers

* -i/-o md as well as -i/-o markdown
 2024-01-20 16:51:15 -05:00
John Kerl
bfc829a381
Internal name-neatens (#1475) 2024-01-20 13:36:28 -05:00
John Kerl
794a754c36
Support PPRINT barred input (#1472) 
* Support PPRINT barred input

* regression-test files

* output from `make dev`

* doc updates
 2024-01-20 12:59:12 -05:00
John Kerl
d2559b8387
Have clean_whitespace re-run type inference (#1464) 
* Have `clean_whitespace` re-infer types

* make dev output

* unit-test files

* drive-by typofix

* make dev output
 2024-01-01 18:39:27 -05:00
John Kerl
2f42c6f508
Fix #1462: remove limit of 1000 on dedupe field names (#1463) 
* Fix #1462: remove limit of 1000 on dedupe field names

* make dev output
 2024-01-01 17:50:56 -05:00
John Kerl
e3b98cd621
On-line help info for mlr join --lk "" (#1458) 
* Doc info for `mlr join --lk ""`

* make dev output
 2023-12-24 12:43:26 -05:00
John Kerl
0e3a54ed68
Implement mlr uniq -x (#1457) 
* mlr uniq -x

* unit-test cases

* make dev
 2023-12-23 16:20:11 -05:00
Eng Zer Jun
f4cf166358
Replace deprecated io/ioutil functions (#1452) 
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2023-12-20 09:44:02 -05:00
John Kerl
c6b745537a
New strmatch/strmatchx DSL functions (#1448) 
* New `match`/`matchx` DSL functions

* unit-test cases

* match/matchx -> strmatch/strmatchx

* help strings for strmatch and strmatchx

* update regex doc page re strmatch/strmatchx

* unit-test update
 2023-12-19 14:34:54 -05:00
John Kerl
b13adbe6c0
mlr --norc cat was erroring (#1450) 2023-12-19 09:33:34 -05:00
John Kerl
4053d7684c
Preserve regex captures across stack frames (#1447) 
* privatize state.RegexCaptures

* stack frame for regex captures

* merge

* unit-test case

* docs re stack frames for regex captures

* more
 2023-12-18 10:21:09 -05:00
John Kerl
1ae670fd4a
Rename internal regex functions (#1446) 2023-12-17 12:46:28 -05:00
John Kerl
856131f7a2 6.10.0-dev 2023-12-13 19:31:59 -05:00
John Kerl
9caa24d7f1
miller 6.10.0 (#1442) 
* neaten

* miller 6.10.0
 2023-12-13 18:43:00 -05:00
John Kerl
bae1daf847
Absent variable on left side of boolean OR (||) expression makes it absent (#1434) 
* Absent-handling with short-circuiting operators `&&` and `||`

* add a missing file

* artifacts from make dev

* type-errors

* doc content

* artifacts from make dev
 2023-12-02 16:00:05 -05:00
John Kerl
18a9eaa377
Fix ragged-CSV auto-pad (#1428) 2023-11-19 23:53:53 -05:00
John Kerl
2bcf8813d3
Add a --files option (#1426) 
* mlr --files

* doc mods
 2023-11-11 19:09:02 -05:00
John Kerl
5b6a1d4713
JSONL output does not properly handle keys with quotes (#1425) 
* mlr --l2j, --j2l

* make dev for previous commit

* fix #1424

* unit-test cases

* iterate
 2023-11-11 18:58:49 -05:00
Eng Zer Jun
4b34f80f6a
transformers/grep: avoid allocations with (*regexp.Regexp).MatchString (#1416) 
We should use `(*regexp.Regexp).MatchString` instead of
`(*regexp.Regexp).Match([]byte(...))` when matching string to avoid
unnecessary `[]byte` conversions and reduce allocations.

Example benchmark:

var grepRegex = regexp.MustCompile("foo.*")

func BenchmarkMatch(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := grepRegex.Match([]byte("foo bar baz")); !match {
			b.Fail()
		}
	}
}

func BenchmarkMatchString(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := grepRegex.MatchString("foo bar baz"); !match {
			b.Fail()
		}
	}
}

goos: linux
goarch: amd64
pkg: github.com/johnkerl/miller/pkg/transformers
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkMatch-16          	 5700908	       210.3 ns/op	      16 B/op	       1 allocs/op
BenchmarkMatchString-16    	 8006731	       156.4 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/johnkerl/miller/pkg/transformers	2.857s

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2023-10-27 09:15:12 -04:00
John Kerl
087f4bb4c9
Include null in any typemask (#1395) 2023-09-11 17:15:37 -04:00
John Kerl
268a96d002
Export library code in pkg/ (#1391) 
* Export library code in `pkg/`

* new doc page
 2023-09-10 17:15:13 -04:00