Commit graph

124 commits

Author SHA1 Message Date
John Kerl
879f272f79
Typofix in uif/uof percentiles (#1375)
* typofix in uif/uof percentiles

* fix regression-test data
2023-08-30 11:13:35 -04:00
John Kerl
5146dd7f90
New contains DSL function (#1374)
* New `contains` DSL function

* unit-test files, and docs
2023-08-27 21:46:24 -04:00
John Kerl
069c068298
Summing up empty data (#1370)
* empty plus value is value

* unit-test cases

* make-docs output

* docs files

* on-line table for null-handling arithmetic rules

* doc mods
2023-08-26 21:24:34 -04:00
John Kerl
d341cc6dd3
DSL functions for summary stats over arrays / maps (#1364)
* DSL stats functions [WIP]

* refactor

* move percentile computation to bifs module; iterate

* mode and antimode

* percentile iterate

* percentile sketching

* neaten

* unit-test iterate

* unify old & new min & max functions

* unit-test cases

* code-dedupe between mode and antimode

* make mode/antimode ties deterministic via first-found-wins rule

* online help strings for new stats DSL functions

* artifacts from `make dev`

* help info on how min/max now recurse into collections

* artifacts from `make dev`

* typofix
2023-08-26 16:02:30 -04:00
John Kerl
4405f732a1 make-dev artifacts from previous commit 2023-08-23 16:19:37 -04:00
Mr. Lance E Sloan
e2338195ba
filename options for split (iss. #1365) (#1366)
* #1365 - filename options for `split`

* Don't use joiner string when prefix is empty.
* Add option to specify joiner string.
* Add option to not URL-escape file names.

* #1365 - update documentation

* #1365 - don't URL-escape file name prefix

I **_thought_** it'd be cool to apply URL-escaping to the file name prefix as well, just in case it included spaces or other characters.  I forgot that a common use for the prefix is to specify a directory path that will contain the file.  When the slashes ("`/`") of the path are URL-escaped, they become "`%2F`" and the directories will not be created.  So, I moved the prefix handling code to come after the URL-escaping.

* #1365 - new `split` options for CLI help output

* #1365 - fix escape/suffix logic error

Trying to make the `return` statement cleaner, I thought it'd be good to add the file name suffix immediately after the file name is URL-escaped.  I'd forgotten that the suffix will not be added if the new `-e` option is used to skip URL-escaping.  So, I put the suffix back where I had it.

* #1365 - add `split` to the "10 minutes" document

Not strictly part of this issue, but as I was checking for docs that I should update as a result of my changes, I noticed this document showed how to split data using the `put` and `tee` combination, but not about the `split` verb.

* #1365 - updated manpage

When I ran `make dev`, generating `data-diving-examples.md` failed.  The two `manpage.txt` files ended up empty, but `mlr.1` seems to be correct.

---------

Co-authored-by: Mr. Lance E Sloan (sloanlance) <sloanlance@users.noreply.github.com>
2023-08-23 16:08:48 -04:00
John Kerl
2107d520fa
Can't use ${field_name} if it contains UTF-8 characters also encodeable as Latin-1 (#1363)
* unit-test data

* docgen

* windows unit-test accommodations
2023-08-20 12:20:15 -04:00
John Kerl
9d1d2e07ca
Do wildcard globbing on Windows (#1362)
* Glob wildcards on Windows

* test/cases/globbing/0001
2023-08-19 17:40:35 -04:00
John Kerl
793f52c470
sub, gsub, and ssub verbs (#1361)
* sub, gsub, and ssub verbs

* doc mods

* content for verbs reference page

* test/cases/verb-sub-gsub-ssub/
2023-08-19 17:23:01 -04:00
John Kerl
d4a3bf99b2
Support ZSTD compression in-process (#1360)
* Support ZSTD compression in-process

* doc mods

* unit-test cases

* doc-gen artifacts
2023-08-19 15:22:59 -04:00
John Kerl
52db2bf422
Small typos in documentation of mlr nest (#1352)
* Typofix in `nest` documentation

* update test/cases/cli-help

* artifacts from `make dev`
2023-08-09 10:50:26 -04:00
John Kerl
b30aceae36
Add %s format specifier for strftime (#1335) 2023-07-04 17:00:02 -04:00
John Kerl
3baebea7a3
Add %N and %O for strfntime (#1334)
* Add `%N` and `%O` for strfntime

* Unit-test mods

* artifacts from `make dev`
2023-07-02 15:49:41 -04:00
John Kerl
3e5c3e2398
Add empty-key check to mlr check (#1330)
* Add empty-key check to `mlr check`

* Update `mlr check --help`

* Update to on-line help
2023-06-25 19:12:26 -04:00
John Kerl
d72ef826fb
Add DSL functions for integer nanoseconds since the epoch (#1326)
* DSL functions for 64-bit nano-epoch timestamps

* strfntime

* nsec2gmt; move sec/nsec pairs adjacent to one another

* update on-line help

* artifacts from `make dev`

* unit-test files
2023-06-24 17:05:15 -04:00
John Kerl
c5ceb20a4e
Fix mlr grep docs re OFS/OPS (#1309)
* Fix `mlr grep` doc re OFS/OPS

* make-dev artifacts
2023-06-06 00:18:51 -04:00
John Kerl
59e63272ae
Unify the case verb, and add options (#1306)
* temp

* Unified `case` verb

* unit-test cases

* doc-page update

* make-dev artifacts
2023-06-04 14:38:22 -04:00
John Kerl
394681c4c1
Values-only -a option for mlr grep (#1305)
* Values-only option for `mlr grep`

* Artifacts from `make dev`
2023-06-03 17:19:40 -04:00
John Kerl
60b1f1b451
Let mlr help take pre-flags, such as --always-color (#1292)
* Let `mlr help` take pre-flags, such as `--always-color`

* Better on-line help for auxents and terminals

* Support `mlr help flag --foo`
2023-05-15 01:14:59 -04:00
John Kerl
d16310ead8
New mlr json-parse -k flag (#1291)
* New `mlr json-parse -k` flag

* docs / make-dev artifacts

* Add unit-test files for the feature

* codespell
2023-05-13 11:22:01 -04:00
John Kerl
5801f5c0a2
Implement --csv-trim-leading-space flag (#1272)
* Implement --csv-trim-leading-space flag

* Artifacts from `make dev`
2023-04-20 13:13:55 -03:00
John Kerl
ae61dc5a4f
Fix precedence of coalesce operators ?? and ??? (#1270)
* Fix precedence of coalesce operators

* GOCC codegen

* unit-test updates

* operator-precedence documentation

* doc-build artifacts
2023-04-15 23:33:49 -04:00
John Kerl
b8f3902e6e
Treaet data-file numbers with leading + as numeric (#1269)
* accept data-file numbers with leading + as numeric

* augment unit-test cases

* artifacts from `make dev`
2023-04-15 15:57:22 -04:00
John Kerl
2d00b44ead
Better error message on unparseable TZ environment variable (#1249) 2023-03-30 08:37:33 -04:00
John Kerl
fa9b00ee64
index DSL function [WIP] (#1247) 2023-03-27 00:37:33 -04:00
John Kerl
ff659b8a7e
Add optional second base argument to int DSL function (#1244)
* Add optional second base argument to `int` DSL function

* artifacts from `make dev`

* unit-test cases

* typofix
2023-03-24 00:28:39 -04:00
John Kerl
bfc8ab5ce2
Add mlr step -a rprod for running products (#1228)
* mlr step -a rprod

* artifacts from `make dev`

* codespell
2023-03-12 13:32:23 -04:00
John Kerl
6ea94a002b
Add new upcase and downcase verbs (#1217)
* upcase/downcase verbs

* unit-test cases

* doc files

* remaining `make dev` artifacts: manpage
2023-03-04 01:08:19 -05:00
John Kerl
38d7de545d
Docs re tail -f and --records-per-batch 1 (#1218) 2023-03-04 00:15:01 -05:00
John Kerl
1c5e97b93c
Fix bug on DSL comment with no final newline (#1216) 2023-03-03 21:42:10 -05:00
John Kerl
5965d46a1f
Support more backslashed special characters in DSL strings (#1212)
* BNF mod for `\^` `\$` et al.

* go.mod go.sum

* Update DSL-build instructions

* tools/build-dsl artifacts

* Unit-test updates

* fix build error

* unit-test files
2023-03-01 01:00:25 -05:00
John Kerl
6fc33c682a
--ofmte, --ofmtf, --ofmtg (#1206) 2023-02-27 22:50:46 -05:00
John Kerl
ee0ac3d7bb
New leftpad and rightpad DSL functions (#1205)
* Initial code for `leftpad` and `rightpad` functions

* typofix, and test cases

* doc-gen

* windows UT
2023-02-26 17:14:35 -05:00
John Kerl
f3fd2c2416
Complete #1181 (#1184)
* Source impl of #1181

* Full flow for #1181
2023-01-27 22:59:14 -05:00
John Kerl
31fdc1c3ed
Fix #1164: regression on CSV blank-line handling (#1168)
* Fix #1164: regression on CSV blank lines

* unit-test case
2023-01-01 16:49:14 -05:00
John Kerl
b518bf0fe5
mlr unspace verb (#1167)
* mlr unspace verb

* unit tests

* unit tests

* lint
2023-01-01 16:44:06 -05:00
John Kerl
7bf358e497
Fix #1102 (#1163) 2022-12-29 14:30:47 -05:00
John Kerl
a55e78c45a
Fix #1146 (#1157)
* Fix #1146

* added a doc link

* update unit-test case
2022-12-29 13:58:50 -05:00
John Kerl
472223c609
Fix typo in mlr put documentation (#1140)
* Fix typo

* Build artifacts from previous commit
2022-12-05 14:28:51 -05:00
Jakub Wilk
636ca8873d
Fix typos (#1135)
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
2022-11-30 09:27:42 -05:00
John Kerl
6f22401d53
Exclude median from summary default (#1131) 2022-11-26 10:34:28 -05:00
John Kerl
39286923dc
Don't double-quote a CSV field only for having a leading space (#1101)
* Don't double-quote a CSV field only for having a leading space

* unit-test case
2022-10-03 22:40:16 -04:00
John Kerl
f131087713
Check -- terminator on --mfrom (#1098) 2022-09-28 00:30:05 -04:00
John Kerl
63db1a9059
mlr reshape multiple regex via multiple -r (#1091) 2022-09-05 22:05:54 -04:00
John Kerl
4a53a6b278
Fix labels for mlr histogram --auto (#1089) 2022-09-03 09:55:44 -04:00
John Kerl
87faddd661
Continue #1086 (#1087) 2022-08-24 22:36:23 -04:00
Stephen Kitt
cdbe8b82ef
Account for varying mlr locations (#1086)
The expected error output for the "not-a-valid-timezone" test assumes
that mlr is invoked as mlr. Use "${MLR}" instead to cope with any
invocation path.

This also adds a regression test to catch future instances before they
get merged.

Signed-off-by: Stephen Kitt <steve@sk2.org>

Signed-off-by: Stephen Kitt <steve@sk2.org>
2022-08-23 14:56:47 -04:00
John Kerl
a8cc3032dc
Restore the --jvquoteall flag (#1083)
* Restore --jvquoteall flag

* make dev: for doc-build artifacts

* unit-test file
2022-08-22 09:36:58 -04:00
John Kerl
2090d0ff5a unit-test case 2022-08-22 09:18:16 -04:00
John Kerl
dafec6dc09
mlr cat --filename / --filenum (#1080)
* mlr cat --filename / --filenum

* codespell
2022-08-20 08:42:15 -04:00