Commit graph

69 commits

Author SHA1 Message Date
John Kerl
ebef818a00
More Miller 6 doc info (#849)
* Info on operator precedence and the REPL

* Information on indexed emit/emit1 in Miller 6
2022-01-09 19:25:44 -05:00
John Kerl
da91878939
Distinguish between JSON and JSON Lines formats (#844)
* Draw a distinction between JSON and JSON Lines formats

* Add JSON Lines to on-line help example

* Have JSON format default to --jlistwrap and --jvstack

* Update test cases for --jlistwrap output for JSON output format

* Have JSON format default to --jlistwrap and --jvstack for --{X}2j as well

* Make --jlistwrap / --jvstack as legacy flags, since now --json and --jsonl

* Add --c2l, --l2c, etc. command-line flags

* docmods for JSON Lines

* Update regression-test cases for JSON / JSON Lines distinction
2022-01-09 11:11:54 -05:00
John Kerl
7143288689 Merge branch 'main' of https://github.com/johnkerl/miller 2022-01-08 21:58:14 -05:00
John Kerl
bcd98f1183 typofix 2022-01-08 21:52:29 -05:00
John Kerl
8267b2220d 6.0.0.rc1 2022-01-08 15:47:19 -05:00
John Kerl
ed98b50d0f
ngrams update (#841) 2022-01-08 15:34:07 -05:00
John Kerl
0c75713b94
Fix codespell config to skip multiple files (#837)
* trying to get codespell to trigger before merge to main

* iterating

* https://github.com/codespell-project/actions-codespell

* iterating

* iterating

* iterating

* iterating

* iterating

* more

* more

* more
2022-01-03 23:16:43 -05:00
John Kerl
1b9526e585
More codespell fixes (#834)
* Fix mlr tail -n4

* More codespell fixes
2022-01-03 21:40:53 -05:00
John Kerl
a343741b73 Merge branch 'main' of https://github.com/johnkerl/miller 2022-01-01 12:55:48 -05:00
John Kerl
42ffeb04bd doc-neaten 2022-01-01 12:51:51 -05:00
John Kerl
8d5bb3742e doc neatens 2021-12-31 11:18:55 -05:00
John Kerl
fd81a3218a typofix in release-docs page 2021-12-30 11:20:45 -05:00
John Kerl
08b582cf46 issues link for new-in-miller6.md 2021-12-30 00:19:04 -05:00
John Kerl
c0e8fcb811
Simplify and unit-test cmp disposition matrices (#819)
* change version from 6.0.0-dev to 6.0.0-rc

* Simplify and unit-test cmp-matrices
2021-12-29 21:53:56 -05:00
John Kerl
db866f5e02 change version from 6.0.0-dev to 6.0.0-rc 2021-12-29 21:40:51 -05:00
John Kerl
8b9e1226cf Add mlr-put example to coverart.png 2021-12-28 19:59:35 -05:00
John Kerl
769a97907f
Update regex doc (#811) 2021-12-28 19:45:58 -05:00
John Kerl
d271d59100 neaten to-do 2021-12-28 19:17:19 -05:00
John Kerl
2e431d4234 Update README-profiling.md (#810) 2021-12-27 23:49:11 -05:00
John Kerl
a296476ecc neaten 2021-12-27 02:08:30 -05:00
John Kerl
73c715d95b todo 2021-12-27 02:06:56 -05:00
John Kerl
a736f651b2 Update perf docs to include #809, as well as Linux/Mac 2021-12-27 02:03:38 -05:00
John Kerl
e10fee0724
Improve type-inference performance (#809)
* To-do items for broader platform/go-version benchmarking

* neaten inferrer API

* extend type-inference unit-test cases

* Add benchmark scripts for comparing compiler versions

* mlr version in addition to mlr --version

* some go-benchmark files for Mac/Linux perf comparisons

* neaten perf-scripts

* merge

* type-scan optimization tests

* type-scan optimization infra

* test new inferrer

* mlr --time option

* include --cpuprofile and --traceprofile in on-line help

* sharpen inferred/deferred-type API distinction

* replace old inferrer with newer/faster

* update docs for new type-inferrer
2021-12-27 00:54:21 -05:00
John Kerl
f2f528ee41 more doc-neatens re windows and mlr -s 2021-12-25 14:30:58 -05:00
John Kerl
a9c9b6dfba Add new files for previous commit 2021-12-25 14:14:30 -05:00
John Kerl
25fb7f1d78
Document --norc flag (#806) 2021-12-25 13:11:58 -05:00
John Kerl
f56ab87c2d
Docs for funct keyword (#805) 2021-12-25 12:58:02 -05:00
John Kerl
8aa62db860
Update perf-docs to include XTAB I/O improvement on #799 (#804) 2021-12-25 12:25:34 -05:00
John Kerl
984df274bb Minor neatens 2021-12-25 12:16:10 -05:00
John Kerl
096bb9bc12
Make --ifs-regex and --ips-regex explicit command-line flags (#799)
* Function-pointerize IXS/IXSRegex to reduce runtime iffelsing

* remove IsRegexString and SuppressIXSRegex

* regression tests passing

* doc updates
2021-12-25 00:00:18 -05:00
John Kerl
c43a7109d1 Merge branch 'perf-doc' 2021-12-23 00:20:24 -05:00
John Kerl
cf8dd134e6
Add performance stats to "What's new in Miller 6" webdoc page (#795)
* Perf notes in README-go-port.md

* Update CPU webdoc

* Add perf-stats to new-in-miller-6.md
2021-12-23 00:16:20 -05:00
John Kerl
ed1147cf14 Add perf-stats to new-in-miller-6.md 2021-12-23 00:15:34 -05:00
John Kerl
a21143f692 Update CPU webdoc 2021-12-22 22:20:20 -05:00
John Kerl
9c8f8680d6 Fix minor doc-formatting issue on PR #794 2021-12-22 21:29:39 -05:00
John Kerl
157e567909
Dedupe field names by default (#794) 2021-12-22 21:07:29 -05:00
John Kerl
3c1ee965ca Document exit codes 2021-12-22 18:37:24 -05:00
John Kerl
3e71ddde21
Add instructions for makeless install (#793)
* Build-from-source info with/without make

* todo

* go install instructions
2021-12-22 18:21:12 -05:00
John Kerl
772ee0d00d
Build-from-source info with/without make (#792) 2021-12-22 17:54:41 -05:00
John Kerl
765df9a995 typofix 2021-12-22 14:29:52 -05:00
John Kerl
bcb8c0ca3b perf notes in new-in-miller-6.md 2021-12-22 13:57:14 -05:00
John Kerl
1d3bc11b29
urandelement DSL function (#789) 2021-12-21 23:15:14 -05:00
John Kerl
93862f16f9
update mlr -O behavior for #756 (#788) 2021-12-21 22:40:34 -05:00
John Kerl
58d9ad19bc
Record hashing perf (#781)
* todo

* Rename inputChannel,outputChannel to readerChannel,writerChannel

* Rename inputChannel,outputChannel to readerChannel,writerChannel (#772)

* Start batched-reader API mods

* Singleton-list step for reader-batching at input

* CLI options for records-per-batch and hash-records

* Push channelized-reader logic into DKVP reader

* Push batching logic into chain-transformer, transformers, and channel-writer

* foo

* cmd/mprof and cmd/mprof2

* cmd/mprof3 and cmd/mprof4

* narrowed in on regexp-splitting on IFS/IPS as perf-hit

* neaten

* channelize nidx

* cmd/mprof5

* channelize CSV reader

* channelize NIDX reader

* Dedupe DKVP-reader and NIDX-reader source files

* channelize CSV-lite reader

* channelize XTAB reader

* batchify JSON reader

* channelize GEN pseudo-reader

* scripts for perf-testing on larger files

* merge with main for #776

* Fix record-batching for join and repl

* Fix comment-handling in channelized XTAB reader

* Fix bug found in positional-rename

* Use --no-hash-records by default
2021-12-14 22:41:40 -05:00
John Kerl
4c3e141df8
Split separators from context (#771)
* Fix mlr -O to return strings for all octal numbers along with 0* numbers like 07 and 08

* Factor separators out of Context struct
2021-12-06 22:47:58 -05:00
John Kerl
cef753c232
Performance improvement: buffered output (#765)
* Experimental #!/usr/bin/env mlr -s feature

* Allow flags after verbs, for shebang support

* Performance optmization: less frequent syscall.write on output
2021-12-01 20:12:31 -05:00
John Kerl
d9dbb1e92e
Improvements for shebang-script handling (#758)
* Experimental #!/usr/bin/env mlr -s feature

* Allow flags after verbs, for shebang support
2021-11-24 01:15:28 -05:00
John Kerl
f097a8dcd2 merge 2021-11-22 22:39:28 -05:00
John Kerl
6a41f73328 Update webdocs for latest mlr compile 2021-11-19 13:55:57 -05:00
John Kerl
4394819cf8
Approximate-match feature for online help (#754)
* Explicitly support approximate-match help

* Unit tests for exact and approximate help
2021-11-17 23:37:48 -05:00