Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-01-23 02:14:13 +00:00
Code Issues Projects Releases Wiki Activity
8931 commits 34 branches 72 tags 221 MiB
Commit graph

62 commits

Author SHA1 Message Date
John Kerl
cc8a3c4b4e
Static-check fixes from @lespea #1657, batch 2/n (#1704) 2024-10-27 11:50:15 -04:00
John Kerl
047cb4bc28
Static-check fixes from @lespea #1657, batch 1/n (#1703) 2024-10-27 11:42:43 -04:00
John Kerl
05aa16cfcf
Join docs wrong link (#1695) 
* Fix join-docs link in online help

* run `make dev` and commit the artifacts
 2024-10-17 09:11:03 -04:00
Stephen Kitt
7a0320fc27
Typo fix: programmatically (#1679) 
Signed-off-by: Stephen Kitt <steve@sk2.org>
 2024-10-06 17:30:12 -04:00
John Kerl
a0d65c3035 6.13.0-dev 2024-10-05 10:47:43 -04:00
John Kerl
c66094a184 miller 6.13.0 2024-10-05 09:32:15 -04:00
John Kerl
31d6164181
Fix 1668 error-source (#1672) 
* Fix 1668 error-source

* run `make dev`
 2024-10-05 09:25:47 -04:00
John Kerl
4a2f349289
Update source material for #1665 (#1666) 
* Fix source info for #1665

* run `make dev`
 2024-10-02 08:46:27 -04:00
Adam Lesperance
085e831668
The package version must match the major tag version (#1654) 
* Update package version

* Update makefile targets

* Update readme packages

* Remaining old packages via rg/sd
 2024-09-20 12:10:11 -04:00
Balki
d1767e7c18
Fix local time when TZ is not set (#1649) 
Do not override time.Local when TZ is empty or unset. It is already set correctly by go standard library.
 2024-09-17 10:58:09 -04:00
John Kerl
f33c0b2cd6
Error in splita/splitax when field contains a single non-string value (#1629) 2024-08-25 19:00:24 -04:00
John Kerl
73e2117b43
Misc. codespell findings (#1628) 2024-08-25 17:40:57 -04:00
John Kerl
1015f18e7b
Fix prepipe handling when filenames have whitespace (#1627) 
* Fix prepipe handling when filenames have whitespace

* unit-test data

* Windows-only unit-test item

* Fix Windows fails; neaten
 2024-08-25 17:40:07 -04:00
John Kerl
16a898cff4
Fix binary data in JSON output (#1626) 2024-08-25 15:00:51 -04:00
John Kerl
6bee4ebbf2
RS aliases for ASCII top-of-table control characters are misnamed (#1620) 
* Fix misnames of ASCII control-character aliases

* artifacts from `make dev`
 2024-08-16 10:25:25 -04:00
Eng Zer Jun
3966a6a0a1
lib/regex: use string version of regexp methods to reduce allocs (#1614) 
Both `(*Regexp).Match` and `(*Regexp).FindAllSubmatchIndex` have
string-based equivalents: `(*Regexp).MatchString` and
`(*Regexp).FindAllStringSubmatchIndex`. We should use the string version
to avoid unnecessary `[]byte` conversions.

Benchmark:

var regex = regexp.MustCompile("foo.*")

func BenchmarkMatch(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := regex.Match([]byte("foo bar baz")); !match {
			b.Fail()
		}
	}
}

func BenchmarkMatchString(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := regex.MatchString("foo bar baz"); !match {
			b.Fail()
		}
	}
}

func BenchmarkFindAllSubmatchIndex(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := regex.FindAllSubmatchIndex([]byte("foo bar baz"), -1); len(match) == 0 {
			b.Fail()
		}
	}
}

func BenchmarkFindAllStringSubmatchIndex(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := regex.FindAllStringSubmatchIndex("foo bar baz", -1); len(match) == 0 {
			b.Fail()
		}
	}
}

goos: linux
goarch: amd64
pkg: github.com/johnkerl/miller/pkg/lib
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkMatch-16                         	 2198350	       517.5 ns/op	      16 B/op	       1 allocs/op
BenchmarkMatchString-16                   	 3143605	       371.5 ns/op	       0 B/op	       0 allocs/op
BenchmarkFindAllSubmatchIndex-16          	  921711	      1199 ns/op	     273 B/op	       3 allocs/op
BenchmarkFindAllStringSubmatchIndex-16    	 1212321	       981.0 ns/op	     257 B/op	       2 allocs/op
PASS
coverage: 0.0% of statements
ok  	github.com/johnkerl/miller/pkg/lib	6.576s

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2024-08-09 13:09:53 -04:00
John Kerl
71d9388bff
Be smarter about auto-unflatten (#1584) 2024-06-08 20:58:26 -04:00
John Kerl
6520bf4758
Bash process substitution not working with put -f (#1583) 
* Bash process substitution not working with `put -f`

* run `make dev`
 2024-06-08 20:37:31 -04:00
John Kerl
202a79d0e2
On-line help for mlr summary --transpose (#1581) 
* On-line help for `mlr summary --transpose`

* run `make dev`
 2024-06-08 13:37:07 -04:00
John Kerl
8223903621
Support $NO_COLOR (#1580) 
* Support `$NO_COLOR`

* run `make dev`
 2024-06-08 13:08:15 -04:00
Andrew Onyshchuk
66abef6704
fraction bugfix (#1579) 2024-06-06 09:04:25 -04:00
John Kerl
16ab199194
Add mad accumulator for stats1 DSL function (#1561) 
* Add `mad` accumulator for `stats1` DSL function

* regression files

* make dev output
 2024-05-11 15:55:27 -04:00
John Kerl
5ac48516f7
Add a stat DSL function (#1560) 
* Add a `stat` DSL function [WIP]

* artifacts from `make dev`

* regression test
 2024-05-09 18:39:44 -04:00
forcedebug
0e30619966
Fix mismatched method names in comments (#1549) 
Signed-off-by: forcedebug <forcedebug@outlook.com>
 2024-04-22 12:16:30 -04:00
camcui
12480c4ab5
chore: fix function name in comment (#1543) 
Signed-off-by: camcui <cuishua@sina.cn>
 2024-04-12 09:38:31 -04:00
John Kerl
e714738a7d
Fix typo in online help for --no-jlistwrap (#1541) 
* Add --no-auto-unsparsify flag

* Fix typo in online help for `--no-jlistwrap`

* Artifacts from `make dev`
 2024-04-11 08:12:45 -04:00
John Kerl
b37c3a5e56 6.12.0 doc link 2024-03-16 17:51:17 -04:00
John Kerl
a0bead4093 miller 6.12.0 2024-03-16 17:19:05 -04:00
John Kerl
83c44e6d74
Add descriptions for put and filter verbs (#1529) 
* Add more info in online help about what put/filter do

* `make dev` artifacts
 2024-03-16 17:09:01 -04:00
John Kerl
f01bb92da7
Avoid spurious [] on JSON output in some cases (#1528) 
* JSON empty vs `[]` handling [WIP]

* unit-test mods
 2024-03-16 17:00:59 -04:00
John Kerl
aff4b9f32d
Improved file-not-found handling (#1508) 2024-02-26 00:12:31 -05:00
John Kerl
fb1f7f8421
Enable record-hashing by default (#1507) 
* Enable record-hashing by default

* comments
 2024-02-25 21:51:41 -05:00
John Kerl
3ff43fa818
Miller produces no output on TSV with > 64K characters per line (#1505) 
* Switch to bufio.Reader, first pass

* temp

* Simplify ILineReader by making it stateless

* Interface not necessary; ILineReader -> TLineReader

* neaten

* iterating
 2024-02-25 15:50:50 -05:00
John Kerl
57b32c3e9b
Separate out ILineReader abstraction (#1504) 
* Split up pkg/input/record_reader.go

* new ILineReader/TLineReader
 2024-02-24 22:07:56 -05:00
John Kerl
7bd460a3b8
Support thousands separator in fmtnum (#1499) 
* Support thousands separator in `fmtnum`

* doc bits
 2024-02-18 14:01:46 -05:00
John Kerl
f5eaf290cf
mlr sparsify (#1498) 
* mlr sparsify

* regression-test cases

* typofix

* Remove mods due to processor-architecture change
 2024-02-18 10:56:26 -05:00
John Kerl
c0e9be0e0c
6.11.0-dev (#1484) 
* 6.11.0-dev

* 6.11.0-dev
 2024-01-24 13:27:04 -05:00
John Kerl
1834a925b3
Miller 6.11.0 (#1481) 
* miller 6.11.0

* Artifacts from `make dev`
 2024-01-23 17:31:58 -05:00
John Kerl
e5ec9f67bd
Implement all/by-regex field selection (-a/-r) for mlr sub, gsub, and ssub (#1480) 
* Code-dedupe `sub`, `gsub`, and `ssub` verbs

* More dedupe

* Start with -a

* Implement -r

* unit-test cases

* Windows command-line parsing
 2024-01-23 17:18:13 -05:00
John Kerl
81d11365a0
mlr reorder with regex support [WIP] (#1473) 
* mlr reorder with regex support for field-name selection

* neaten

* -r -b/-a; unit-test cases
 2024-01-21 15:17:33 -05:00
John Kerl
ac65675ab1
Auto-unsparsify CSV and TSV on output (#1479) 
* Auto-unsparsify CSV

* Update unit-test cases

* More unit-test cases

* Key-change handling for CSV output

* Same for TSV, with unit-test and doc updates
 2024-01-20 18:43:49 -05:00
John Kerl
af021f28d7
Support markdown format on input (#1478) 
* Support markdown on input

* unit-test files

* doc mods

* Unit-test cases for I/O-format keystroke-savers

* -i/-o md as well as -i/-o markdown
 2024-01-20 16:51:15 -05:00
John Kerl
bfc829a381
Internal name-neatens (#1475) 2024-01-20 13:36:28 -05:00
John Kerl
794a754c36
Support PPRINT barred input (#1472) 
* Support PPRINT barred input

* regression-test files

* output from `make dev`

* doc updates
 2024-01-20 12:59:12 -05:00
John Kerl
d2559b8387
Have clean_whitespace re-run type inference (#1464) 
* Have `clean_whitespace` re-infer types

* make dev output

* unit-test files

* drive-by typofix

* make dev output
 2024-01-01 18:39:27 -05:00
John Kerl
2f42c6f508
Fix #1462: remove limit of 1000 on dedupe field names (#1463) 
* Fix #1462: remove limit of 1000 on dedupe field names

* make dev output
 2024-01-01 17:50:56 -05:00
John Kerl
e3b98cd621
On-line help info for mlr join --lk "" (#1458) 
* Doc info for `mlr join --lk ""`

* make dev output
 2023-12-24 12:43:26 -05:00
John Kerl
0e3a54ed68
Implement mlr uniq -x (#1457) 
* mlr uniq -x

* unit-test cases

* make dev
 2023-12-23 16:20:11 -05:00
Eng Zer Jun
f4cf166358
Replace deprecated io/ioutil functions (#1452) 
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2023-12-20 09:44:02 -05:00
John Kerl
c6b745537a
New strmatch/strmatchx DSL functions (#1448) 
* New `match`/`matchx` DSL functions

* unit-test cases

* match/matchx -> strmatch/strmatchx

* help strings for strmatch and strmatchx

* update regex doc page re strmatch/strmatchx

* unit-test update
 2023-12-19 14:34:54 -05:00