Commit graph

20 commits

Author SHA1 Message Date
John Kerl
39286923dc
Don't double-quote a CSV field only for having a leading space (#1101)
* Don't double-quote a CSV field only for having a leading space

* unit-test case
2022-10-03 22:40:16 -04:00
John Kerl
abf58f4bb5
Restore --quote-all for CSV output (#1084)
* Restore --quote-all flag for CSV output

* make dev: for doc-build artifacts
2022-08-22 09:38:53 -04:00
John Kerl
a8cc3032dc
Restore the --jvquoteall flag (#1083)
* Restore --jvquoteall flag

* make dev: for doc-build artifacts

* unit-test file
2022-08-22 09:36:58 -04:00
John Kerl
0d6f330eda forgot to make a branch/PR for --quote-all 2022-08-22 09:25:09 -04:00
John Kerl
b27dcddfe3 Restore --quote-all flag for CSV output 2022-08-22 09:17:37 -04:00
John Kerl
66c4a077fd
Make TSV finally true TSV (#923)
* Spec-TSV

* doc mods; more test cases
2022-02-06 00:13:55 -05:00
Stephen Kitt
d536318ed6
Use int64 wherever "64-bit integer" is assumed (#902)
Miller assumes 64-bit integers, but in Go, the int type varies in size
depending on the architecture: 32-bit architectures have int
equivalent to int32. As a result, the supported range of integer
values is greatly reduced on 32-bit architectures compared to what is
suggested by the documentation.

This patch explicitly uses int64 wherever 64-bit integers are
assumed.

Test cases affected by the behaviour of the random generator are
updated to reflect the new values (the existing seed doesn't produce
the same behaviour since the way random values are generated has
changed).

Signed-off-by: Stephen Kitt <steve@sk2.org>
2022-01-27 12:06:25 -05:00
John Kerl
f7ff63124b
mlr split verb (#898)
* mlr split

* regression-test cases

* doc-build artifacts
2022-01-26 23:16:25 -05:00
John Kerl
dedc7ff06f replace - with _ in *.go filenames 2022-01-23 23:22:55 -05:00
Stephen Kitt
bc8ad1b579
Clean up file output handler error handling (#886)
The "= nil" assignment isn't necessary. Merge the error handling, and
drop the empty "if outputHandler != nil" branch.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2022-01-18 20:49:58 -05:00
John Kerl
3cdf7c74f6
Fix #853 (#860) 2022-01-11 07:59:54 -05:00
John Kerl
0c75713b94
Fix codespell config to skip multiple files (#837)
* trying to get codespell to trigger before merge to main

* iterating

* https://github.com/codespell-project/actions-codespell

* iterating

* iterating

* iterating

* iterating

* iterating

* more

* more

* more
2022-01-03 23:16:43 -05:00
John Kerl
a977617797
Address some staticcheck issues (#823)
* address some staticcheck issues

* address some staticcheck issues

* address some staticcheck issues

* address some staticcheck issues
2022-01-01 14:28:19 -05:00
John Kerl
7a97c9b868
Performance improvement by JIT type inference (#786)
* JIT mlrval type-interfence: mlrval package

* mlrmap refactor

* complete merge from #779

* iterating

* mlrval/format.go

* mlrval/copy.go

* bifs/arithmetic_test.go

* iterate on bifs/collections_test.go

* mlrval_cmp.go

* mlrval JSON iterate

* iterate applying mlrval refactors to dependent packages

* first clean compile in a long while on this branch

* results of first post-compile profiling

* testing

* bugfix in ofmt formatting

* bugfix in octal-supporess

* go fmt

* neaten

* regression tests all passing
2021-12-20 23:56:04 -05:00
John Kerl
f233923351
Performance improvement: record-batching (#779)
* Rename inputChannel,outputChannel to readerChannel,writerChannel

* Rename inputChannel,outputChannel to readerChannel,writerChannel (#772)

* Start batched-reader API mods

* Singleton-list step for reader-batching at input

* CLI options for records-per-batch and hash-records

* Push channelized-reader logic into DKVP reader

* Push batching logic into chain-transformer, transformers, and channel-writer

* foo

* cmd/mprof and cmd/mprof2

* cmd/mprof3 and cmd/mprof4

* narrowed in on regexp-splitting on IFS/IPS as perf-hit

* neaten

* channelize nidx

* cmd/mprof5

* channelize CSV reader

* channelize NIDX reader

* Dedupe DKVP-reader and NIDX-reader source files

* channelize CSV-lite reader

* channelize XTAB reader

* batchify JSON reader

* channelize GEN pseudo-reader

* scripts for perf-testing on larger files

* merge with main for #776

* Fix record-batching for join and repl

* Fix comment-handling in channelized XTAB reader

* Fix bug found in positional-rename
2021-12-13 00:57:52 -05:00
John Kerl
4ddb8ff25c Rename inputChannel,outputChannel to readerChannel,writerChannel (#772) 2021-12-07 08:02:30 -05:00
John Kerl
cef753c232
Performance improvement: buffered output (#765)
* Experimental #!/usr/bin/env mlr -s feature

* Allow flags after verbs, for shebang support

* Performance optmization: less frequent syscall.write on output
2021-12-01 20:12:31 -05:00
John Kerl
a73089be20 mlr --csv --irs/--ors crlf is OK, sine autodetected per platform 2021-11-17 23:39:16 -05:00
John Kerl
bc72cd1857
More Go-package restructuring (#748) 2021-11-12 12:49:55 -05:00
John Kerl
e2b6ec2391
Standardize Go-package structure (#746) 2021-11-11 14:15:13 -05:00