John Kerl
a0bead4093
miller 6.12.0
2024-03-16 17:19:05 -04:00
John Kerl
83c44e6d74
Add descriptions for put and filter verbs ( #1529 )
...
* Add more info in online help about what put/filter do
* `make dev` artifacts
2024-03-16 17:09:01 -04:00
John Kerl
f01bb92da7
Avoid spurious [] on JSON output in some cases ( #1528 )
...
* JSON empty vs `[]` handling [WIP]
* unit-test mods
2024-03-16 17:00:59 -04:00
John Kerl
aff4b9f32d
Improved file-not-found handling ( #1508 )
2024-02-26 00:12:31 -05:00
John Kerl
fb1f7f8421
Enable record-hashing by default ( #1507 )
...
* Enable record-hashing by default
* comments
2024-02-25 21:51:41 -05:00
John Kerl
3ff43fa818
Miller produces no output on TSV with > 64K characters per line ( #1505 )
...
* Switch to bufio.Reader, first pass
* temp
* Simplify ILineReader by making it stateless
* Interface not necessary; ILineReader -> TLineReader
* neaten
* iterating
2024-02-25 15:50:50 -05:00
John Kerl
57b32c3e9b
Separate out ILineReader abstraction ( #1504 )
...
* Split up pkg/input/record_reader.go
* new ILineReader/TLineReader
2024-02-24 22:07:56 -05:00
John Kerl
7bd460a3b8
Support thousands separator in fmtnum ( #1499 )
...
* Support thousands separator in `fmtnum`
* doc bits
2024-02-18 14:01:46 -05:00
John Kerl
f5eaf290cf
mlr sparsify ( #1498 )
...
* mlr sparsify
* regression-test cases
* typofix
* Remove mods due to processor-architecture change
2024-02-18 10:56:26 -05:00
John Kerl
c0e9be0e0c
6.11.0-dev ( #1484 )
...
* 6.11.0-dev
* 6.11.0-dev
2024-01-24 13:27:04 -05:00
John Kerl
1834a925b3
Miller 6.11.0 ( #1481 )
...
* miller 6.11.0
* Artifacts from `make dev`
2024-01-23 17:31:58 -05:00
John Kerl
e5ec9f67bd
Implement all/by-regex field selection (-a/-r) for mlr sub, gsub, and ssub ( #1480 )
...
* Code-dedupe `sub`, `gsub`, and `ssub` verbs
* More dedupe
* Start with -a
* Implement -r
* unit-test cases
* Windows command-line parsing
2024-01-23 17:18:13 -05:00
John Kerl
81d11365a0
mlr reorder with regex support [WIP] (#1473 )
...
* mlr reorder with regex support for field-name selection
* neaten
* -r -b/-a; unit-test cases
2024-01-21 15:17:33 -05:00
John Kerl
ac65675ab1
Auto-unsparsify CSV and TSV on output ( #1479 )
...
* Auto-unsparsify CSV
* Update unit-test cases
* More unit-test cases
* Key-change handling for CSV output
* Same for TSV, with unit-test and doc updates
2024-01-20 18:43:49 -05:00
John Kerl
af021f28d7
Support markdown format on input ( #1478 )
...
* Support markdown on input
* unit-test files
* doc mods
* Unit-test cases for I/O-format keystroke-savers
* -i/-o md as well as -i/-o markdown
2024-01-20 16:51:15 -05:00
John Kerl
bfc829a381
Internal name-neatens ( #1475 )
2024-01-20 13:36:28 -05:00
John Kerl
794a754c36
Support PPRINT barred input ( #1472 )
...
* Support PPRINT barred input
* regression-test files
* output from `make dev`
* doc updates
2024-01-20 12:59:12 -05:00
John Kerl
d2559b8387
Have clean_whitespace re-run type inference ( #1464 )
...
* Have `clean_whitespace` re-infer types
* make dev output
* unit-test files
* drive-by typofix
* make dev output
2024-01-01 18:39:27 -05:00
John Kerl
2f42c6f508
Fix #1462 : remove limit of 1000 on dedupe field names ( #1463 )
...
* Fix #1462 : remove limit of 1000 on dedupe field names
* make dev output
2024-01-01 17:50:56 -05:00
John Kerl
e3b98cd621
On-line help info for mlr join --lk "" ( #1458 )
...
* Doc info for `mlr join --lk ""`
* make dev output
2023-12-24 12:43:26 -05:00
John Kerl
0e3a54ed68
Implement mlr uniq -x ( #1457 )
...
* mlr uniq -x
* unit-test cases
* make dev
2023-12-23 16:20:11 -05:00
Eng Zer Jun
f4cf166358
Replace deprecated io/ioutil functions ( #1452 )
...
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-12-20 09:44:02 -05:00
John Kerl
c6b745537a
New strmatch/strmatchx DSL functions ( #1448 )
...
* New `match`/`matchx` DSL functions
* unit-test cases
* match/matchx -> strmatch/strmatchx
* help strings for strmatch and strmatchx
* update regex doc page re strmatch/strmatchx
* unit-test update
2023-12-19 14:34:54 -05:00
John Kerl
b13adbe6c0
mlr --norc cat was erroring ( #1450 )
2023-12-19 09:33:34 -05:00
John Kerl
4053d7684c
Preserve regex captures across stack frames ( #1447 )
...
* privatize state.RegexCaptures
* stack frame for regex captures
* merge
* unit-test case
* docs re stack frames for regex captures
* more
2023-12-18 10:21:09 -05:00
John Kerl
1ae670fd4a
Rename internal regex functions ( #1446 )
2023-12-17 12:46:28 -05:00
John Kerl
856131f7a2
6.10.0-dev
2023-12-13 19:31:59 -05:00
John Kerl
9caa24d7f1
miller 6.10.0 ( #1442 )
...
* neaten
* miller 6.10.0
2023-12-13 18:43:00 -05:00
John Kerl
bae1daf847
Absent variable on left side of boolean OR (||) expression makes it absent ( #1434 )
...
* Absent-handling with short-circuiting operators `&&` and `||`
* add a missing file
* artifacts from make dev
* type-errors
* doc content
* artifacts from make dev
2023-12-02 16:00:05 -05:00
John Kerl
18a9eaa377
Fix ragged-CSV auto-pad ( #1428 )
2023-11-19 23:53:53 -05:00
John Kerl
2bcf8813d3
Add a --files option ( #1426 )
...
* mlr --files
* doc mods
2023-11-11 19:09:02 -05:00
John Kerl
5b6a1d4713
JSONL output does not properly handle keys with quotes ( #1425 )
...
* mlr --l2j, --j2l
* make dev for previous commit
* fix #1424
* unit-test cases
* iterate
2023-11-11 18:58:49 -05:00
Eng Zer Jun
4b34f80f6a
transformers/grep: avoid allocations with (*regexp.Regexp).MatchString ( #1416 )
...
We should use `(*regexp.Regexp).MatchString` instead of
`(*regexp.Regexp).Match([]byte(...))` when matching string to avoid
unnecessary `[]byte` conversions and reduce allocations.
Example benchmark:
var grepRegex = regexp.MustCompile("foo.*")
func BenchmarkMatch(b *testing.B) {
for i := 0; i < b.N; i++ {
if match := grepRegex.Match([]byte("foo bar baz")); !match {
b.Fail()
}
}
}
func BenchmarkMatchString(b *testing.B) {
for i := 0; i < b.N; i++ {
if match := grepRegex.MatchString("foo bar baz"); !match {
b.Fail()
}
}
}
goos: linux
goarch: amd64
pkg: github.com/johnkerl/miller/pkg/transformers
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkMatch-16 5700908 210.3 ns/op 16 B/op 1 allocs/op
BenchmarkMatchString-16 8006731 156.4 ns/op 0 B/op 0 allocs/op
PASS
ok github.com/johnkerl/miller/pkg/transformers 2.857s
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-10-27 09:15:12 -04:00
John Kerl
087f4bb4c9
Include null in any typemask ( #1395 )
2023-09-11 17:15:37 -04:00
John Kerl
268a96d002
Export library code in pkg/ ( #1391 )
...
* Export library code in `pkg/`
* new doc page
2023-09-10 17:15:13 -04:00
