Commit graph

53 commits

Author SHA1 Message Date
John Kerl
95be06b752 make fmt 2022-11-26 00:23:12 -05:00
Matt Forrester
e55efbbfad
doc: fix docs for system/exec() (#1071)
Co-authored-by: Matthew Forrester <matthew.forrester@speechmarks.com>
2022-08-13 23:55:05 -04:00
John Kerl
5ffb8a59b3 make dev after #1067 2022-08-07 00:40:44 -04:00
Matt Forrester
7a30f3552e
feat: system/exec() function call (#1067)
Co-authored-by: Matthew Forrester <matthew.forrester@speechmarks.com>
2022-08-07 00:36:53 -04:00
John Kerl
c1a1589174
Support simplified sort-map-by-value in the DSL (#1069)
* add shorthands for sorting maps by value

* unit-test files

* on-line help content
2022-08-07 00:31:08 -04:00
John Kerl
6b32e1f419
Reduce number of os.Exit callsites, part 1 of n (#1055)
* neaten some uses of os.Exit

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating
2022-07-09 18:53:24 -04:00
John Kerl
e91d19de25 typofix 2022-05-14 14:47:14 -04:00
John Kerl
d50f22924a
Allow x ** - y and x ** + y in the DSL grammar (#1021)
* Allow x ** - y in the DSL grammar

* Parser-generator artifacts

* AST-builder support

* regression-test cases

* comments
2022-05-02 01:03:24 -04:00
John Kerl
d03ef16cfc
Add line/column info for DSL runtime non-parse failures (#998)
* Add line/column info for DSL runtime non-parse failures

* Other related callsites

* test cases

* Update already-existing test cases
2022-03-20 21:57:11 -04:00
John Kerl
2408915160
DSL functions and verbs for UTF-8 <-> Latin-1 (#997)
* latin1_to_utf8 and utf8_to_latin1 DSL functions

* doc-build artifacts for previous commit

* Test cases for latin1_to_utf8 and utf8_to_latin1

* extend on-line help

* latin1_to_utf8 and utf8_to_latin1 verbs

* unit-test cases for verbs

* Keep with kebab-case naming convention for verbs

* webdocs
2022-03-20 17:29:40 -04:00
John Kerl
4191c35b7c
Re-use whitespace regexp in clean_whitespace (#994)
* DSL strict mode

* Re-use whitespace regexp in clean_whitespace
2022-03-20 12:17:51 -04:00
John Kerl
f9e3256367
gssub DSL function (#989)
* gssub DSL function

* build artifacts for gssub source-code change

* unit-test files

* gssub / Latin 1 example in docs

* code-dedupe ssub and gssub
2022-03-18 23:48:27 -04:00
John Kerl
42628b83d3 Typofix in unformat/unformatx on-line help 2022-02-24 09:56:08 -05:00
John Kerl
d75dfa956a
Fix typo in on-line help for splitax DSL function (#964)
* Fix typo in splitax on-line help

* doc-build artifacts
2022-02-23 21:14:45 -05:00
John Kerl
3961ee1695 Apply correct array-slicing logic to formerly incorrect substr0 2022-02-21 23:36:55 -05:00
John Kerl
16d873427a Code-dedupe logic for array slices and string slices 2022-02-21 23:17:37 -05:00
John Kerl
47a427b00c
Fix "%%" in strptime; more test cases for strptime (#951)
* strptime debug mode

* Fix strptime %% format code

* internal/pkg/pbnjay-strptime/strptime_test.go; reg-test files

* Web-doc improvements for strftime and strptime

* doc-build artifacts
2022-02-21 12:21:04 -05:00
John Kerl
fb70f7a9d5 extend fmtifnum on-line-help examples 2022-02-19 23:26:24 -05:00
Jordi
8aecba305f
Fix: multiple documentation tweaks (#949)
* Fix: multiple documentation tweaks

* Fix: undo two lines removal

* Fix: recover the lost Oxford commas
2022-02-18 18:38:30 -05:00
John Kerl
bb95f37066
New fmtifnum DSL function; make fmtnum/fmtifnum recursive over maps and arrays (#946)
* New fmtifnum DSL function

* doc-build artifacts for previous commit

* Make fmtnum/fmtifnum recursive over maps and arrays

* Online-help examples for fmtnum/fmtifnum

* doc-build artifacts for previous commit

* unit-test cases

* code-dedupe
2022-02-14 23:57:14 -05:00
John Kerl
d67868bd33
Add Inf and NaN literals to the DSL (#933)
* Add Inf and NaN literals to the DSL

* Parser-build artifacts

* is_nan DSL function

* doc-build artifacts

* webdoc material
2022-02-08 09:58:50 -05:00
John Kerl
ca9505dfaf
Natural sort (#932)
* Add natural sort order as an option for the sort verb

* Add natural sort order as an option for the sort DSL function

* doc-build artifacts for on-line help

* webdocs

* codespell fix

* unit-test files for sort verb

* unit-test files for sort DSL function
2022-02-08 00:35:28 -05:00
John Kerl
b983f825c0 Clarify time on-line help 2022-02-06 13:25:35 -05:00
John Kerl
9f3b1c8711
Fix is_non_empty for absent case (#928) 2022-02-06 13:10:36 -05:00
John Kerl
4ac5c80b11
Expand on-line help for split* DSL functions (#927)
* Expand on-line for split* DSL functions

* doc-build artifacts
2022-02-06 12:34:23 -05:00
John Kerl
4f16054a62
Let :resetblocks/:rb in the REPL take optional begin/main/end (#924) 2022-02-06 00:31:45 -05:00
John Kerl
ac47c7052a typofix 2022-02-05 00:27:25 -05:00
John Kerl
b7f76cd622
Multiple on-line-help issues from #908 (#921)
* More on issue 908

* doc-build artifacts
2022-02-05 00:18:55 -05:00
John Kerl
9c29f0cef9
Multiple on-line-help issues from #908 (#913)
* Populate help strings for systime, systimeint, and uptime DSL functions

* Extend help strings for joink, joinv, and joinkv

* Expand on-line help for ssub/sub/gsub

* doc artifacts
2022-02-04 22:43:33 -05:00
John Kerl
aa960040d8
Add :resetblocks / :rb to REPL (#920) 2022-02-04 09:53:18 -05:00
John Kerl
c327055db7
Fix multiple on-line-help issues from #907 (#910)
* Fix multiple on-line-help issues from #907

* build-artifacts for previous commit
2022-01-31 23:41:32 -05:00
Stephen Kitt
1beacbd381
More dead code removal (#905)
Signed-off-by: Stephen Kitt <steve@sk2.org>
2022-01-30 12:03:41 -05:00
Stephen Kitt
d536318ed6
Use int64 wherever "64-bit integer" is assumed (#902)
Miller assumes 64-bit integers, but in Go, the int type varies in size
depending on the architecture: 32-bit architectures have int
equivalent to int32. As a result, the supported range of integer
values is greatly reduced on 32-bit architectures compared to what is
suggested by the documentation.

This patch explicitly uses int64 wherever 64-bit integers are
assumed.

Test cases affected by the behaviour of the random generator are
updated to reflect the new values (the existing seed doesn't produce
the same behaviour since the way random values are generated has
changed).

Signed-off-by: Stephen Kitt <steve@sk2.org>
2022-01-27 12:06:25 -05:00
John Kerl
dad6456022
Clarify source for printf-style formatting (#895) 2022-01-24 22:52:38 -05:00
John Kerl
dedc7ff06f replace - with _ in *.go filenames 2022-01-23 23:22:55 -05:00
Stephen Kitt
521215992d
Specify constant types except with iota (#884)
Constant types need to be repeated, unless iota is used or the
constants are declared in the same expression
(const A, B, C Type = 1, 2, 3). See
https://go.dev/ref/spec#Constant_declarations for details.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2022-01-18 20:51:53 -05:00
John Kerl
4fb0b716ba
Make is_null/is_not_null DSL functions include new JSON-null type (#883)
* Make is_null/is_not_null DSL functions include new JSON-null type

* regression-test files
2022-01-18 08:25:03 -05:00
John Kerl
471a110e85
Mlrval arrayval from []Mlrval to []*Mlrval (#880)
* to do
* Mlrval arrayval from []Mlrval to []*Mlrval
2022-01-17 15:13:57 -05:00
John Kerl
1785f8d111 doc-build artifacts for previous commit 2022-01-13 09:46:39 -05:00
John Kerl
478966a535 on-line help for unformat 2022-01-13 09:34:59 -05:00
John Kerl
49e551c794 New unformat DSL function 2022-01-13 01:32:27 -05:00
John Kerl
a0048f0393
New format DSL function (#869)
* New format DSL function

* Updated affected test cases involving on-line help on "for" prefix

* doc-build artifacts for previous commit

* regression-test cases
2022-01-12 22:40:59 -05:00
John Kerl
1e1e03e4b3
New concat DSL function for arrays (#868)
* New concat DSL function for arrays

* Documentation artifacts for previous commit

* Regression-test cases
2022-01-12 21:14:27 -05:00
John Kerl
1b9526e585
More codespell fixes (#834)
* Fix mlr tail -n4

* More codespell fixes
2022-01-03 21:40:53 -05:00
John Kerl
a977617797
Address some staticcheck issues (#823)
* address some staticcheck issues

* address some staticcheck issues

* address some staticcheck issues

* address some staticcheck issues
2022-01-01 14:28:19 -05:00
John Kerl
f56ab87c2d
Docs for funct keyword (#805) 2021-12-25 12:58:02 -05:00
John Kerl
1d3bc11b29
urandelement DSL function (#789) 2021-12-21 23:15:14 -05:00
John Kerl
7a97c9b868
Performance improvement by JIT type inference (#786)
* JIT mlrval type-interfence: mlrval package

* mlrmap refactor

* complete merge from #779

* iterating

* mlrval/format.go

* mlrval/copy.go

* bifs/arithmetic_test.go

* iterate on bifs/collections_test.go

* mlrval_cmp.go

* mlrval JSON iterate

* iterate applying mlrval refactors to dependent packages

* first clean compile in a long while on this branch

* results of first post-compile profiling

* testing

* bugfix in ofmt formatting

* bugfix in octal-supporess

* go fmt

* neaten

* regression tests all passing
2021-12-20 23:56:04 -05:00
John Kerl
f233923351
Performance improvement: record-batching (#779)
* Rename inputChannel,outputChannel to readerChannel,writerChannel

* Rename inputChannel,outputChannel to readerChannel,writerChannel (#772)

* Start batched-reader API mods

* Singleton-list step for reader-batching at input

* CLI options for records-per-batch and hash-records

* Push channelized-reader logic into DKVP reader

* Push batching logic into chain-transformer, transformers, and channel-writer

* foo

* cmd/mprof and cmd/mprof2

* cmd/mprof3 and cmd/mprof4

* narrowed in on regexp-splitting on IFS/IPS as perf-hit

* neaten

* channelize nidx

* cmd/mprof5

* channelize CSV reader

* channelize NIDX reader

* Dedupe DKVP-reader and NIDX-reader source files

* channelize CSV-lite reader

* channelize XTAB reader

* batchify JSON reader

* channelize GEN pseudo-reader

* scripts for perf-testing on larger files

* merge with main for #776

* Fix record-batching for join and repl

* Fix comment-handling in channelized XTAB reader

* Fix bug found in positional-rename
2021-12-13 00:57:52 -05:00
John Kerl
4c3e141df8
Split separators from context (#771)
* Fix mlr -O to return strings for all octal numbers along with 0* numbers like 07 and 08

* Factor separators out of Context struct
2021-12-06 22:47:58 -05:00