Commit graph

151 commits

Author SHA1 Message Date
John Kerl
5b6a1d4713
JSONL output does not properly handle keys with quotes (#1425)
* mlr --l2j, --j2l

* make dev for previous commit

* fix #1424

* unit-test cases

* iterate
2023-11-11 18:58:49 -05:00
John Kerl
268a96d002
Export library code in pkg/ (#1391)
* Export library code in `pkg/`

* new doc page
2023-09-10 17:15:13 -04:00
John Kerl
acc10cdc37 miller 6.9.0 2023-08-31 09:00:29 -04:00
John Kerl
0493a0debd
Fatal-on-data-error mlr -x option (#1373)
* Fatal-on-data-error `mlr -x` option [WIP]

* arithmetic.go error-reason propagation

* more

* more

* more

* renames

* doc page

* namefix

* fix broken test

* make dev
2023-08-30 19:39:22 -04:00
John Kerl
2fd353c6be docmods for typofix 2023-08-30 09:00:24 -04:00
John Kerl
5146dd7f90
New contains DSL function (#1374)
* New `contains` DSL function

* unit-test files, and docs
2023-08-27 21:46:24 -04:00
John Kerl
67d16c89c1 typofix 2023-08-26 22:49:28 -04:00
John Kerl
fb3e3d15cd make dev 2023-08-26 16:47:19 -04:00
John Kerl
4cfb0ba112 neaten online help for the percentiles function 2023-08-26 16:30:21 -04:00
John Kerl
d341cc6dd3
DSL functions for summary stats over arrays / maps (#1364)
* DSL stats functions [WIP]

* refactor

* move percentile computation to bifs module; iterate

* mode and antimode

* percentile iterate

* percentile sketching

* neaten

* unit-test iterate

* unify old & new min & max functions

* unit-test cases

* code-dedupe between mode and antimode

* make mode/antimode ties deterministic via first-found-wins rule

* online help strings for new stats DSL functions

* artifacts from `make dev`

* help info on how min/max now recurse into collections

* artifacts from `make dev`

* typofix
2023-08-26 16:02:30 -04:00
John Kerl
4405f732a1 make-dev artifacts from previous commit 2023-08-23 16:19:37 -04:00
Mr. Lance E Sloan
e2338195ba
filename options for split (iss. #1365) (#1366)
* #1365 - filename options for `split`

* Don't use joiner string when prefix is empty.
* Add option to specify joiner string.
* Add option to not URL-escape file names.

* #1365 - update documentation

* #1365 - don't URL-escape file name prefix

I **_thought_** it'd be cool to apply URL-escaping to the file name prefix as well, just in case it included spaces or other characters.  I forgot that a common use for the prefix is to specify a directory path that will contain the file.  When the slashes ("`/`") of the path are URL-escaped, they become "`%2F`" and the directories will not be created.  So, I moved the prefix handling code to come after the URL-escaping.

* #1365 - new `split` options for CLI help output

* #1365 - fix escape/suffix logic error

Trying to make the `return` statement cleaner, I thought it'd be good to add the file name suffix immediately after the file name is URL-escaped.  I'd forgotten that the suffix will not be added if the new `-e` option is used to skip URL-escaping.  So, I put the suffix back where I had it.

* #1365 - add `split` to the "10 minutes" document

Not strictly part of this issue, but as I was checking for docs that I should update as a result of my changes, I noticed this document showed how to split data using the `put` and `tee` combination, but not about the `split` verb.

* #1365 - updated manpage

When I ran `make dev`, generating `data-diving-examples.md` failed.  The two `manpage.txt` files ended up empty, but `mlr.1` seems to be correct.

---------

Co-authored-by: Mr. Lance E Sloan (sloanlance) <sloanlance@users.noreply.github.com>
2023-08-23 16:08:48 -04:00
John Kerl
2107d520fa
Can't use ${field_name} if it contains UTF-8 characters also encodeable as Latin-1 (#1363)
* unit-test data

* docgen

* windows unit-test accommodations
2023-08-20 12:20:15 -04:00
John Kerl
793f52c470
sub, gsub, and ssub verbs (#1361)
* sub, gsub, and ssub verbs

* doc mods

* content for verbs reference page

* test/cases/verb-sub-gsub-ssub/
2023-08-19 17:23:01 -04:00
John Kerl
d4a3bf99b2
Support ZSTD compression in-process (#1360)
* Support ZSTD compression in-process

* doc mods

* unit-test cases

* doc-gen artifacts
2023-08-19 15:22:59 -04:00
John Kerl
8b22708c27
Support comments in mlr -s files (#1359)
* Support comments in `mlr -s` files

* doc mods

* artifacts from `make dev`

* neaten
2023-08-19 13:32:09 -04:00
John Kerl
52db2bf422
Small typos in documentation of mlr nest (#1352)
* Typofix in `nest` documentation

* update test/cases/cli-help

* artifacts from `make dev`
2023-08-09 10:50:26 -04:00
John Kerl
b30aceae36
Add %s format specifier for strftime (#1335) 2023-07-04 17:00:02 -04:00
John Kerl
3baebea7a3
Add %N and %O for strfntime (#1334)
* Add `%N` and `%O` for strfntime

* Unit-test mods

* artifacts from `make dev`
2023-07-02 15:49:41 -04:00
John Kerl
3e5c3e2398
Add empty-key check to mlr check (#1330)
* Add empty-key check to `mlr check`

* Update `mlr check --help`

* Update to on-line help
2023-06-25 19:12:26 -04:00
John Kerl
d72ef826fb
Add DSL functions for integer nanoseconds since the epoch (#1326)
* DSL functions for 64-bit nano-epoch timestamps

* strfntime

* nsec2gmt; move sec/nsec pairs adjacent to one another

* update on-line help

* artifacts from `make dev`

* unit-test files
2023-06-24 17:05:15 -04:00
John Kerl
c5ceb20a4e
Fix mlr grep docs re OFS/OPS (#1309)
* Fix `mlr grep` doc re OFS/OPS

* make-dev artifacts
2023-06-06 00:18:51 -04:00
John Kerl
e158e1c616 post-6.8.0 2023-06-04 16:38:18 -04:00
John Kerl
7ec36162ba miller 6.8.0 2023-06-04 15:07:38 -04:00
John Kerl
59e63272ae
Unify the case verb, and add options (#1306)
* temp

* Unified `case` verb

* unit-test cases

* doc-page update

* make-dev artifacts
2023-06-04 14:38:22 -04:00
John Kerl
394681c4c1
Values-only -a option for mlr grep (#1305)
* Values-only option for `mlr grep`

* Artifacts from `make dev`
2023-06-03 17:19:40 -04:00
John Kerl
d16310ead8
New mlr json-parse -k flag (#1291)
* New `mlr json-parse -k` flag

* docs / make-dev artifacts

* Add unit-test files for the feature

* codespell
2023-05-13 11:22:01 -04:00
John Kerl
5801f5c0a2
Implement --csv-trim-leading-space flag (#1272)
* Implement --csv-trim-leading-space flag

* Artifacts from `make dev`
2023-04-20 13:13:55 -03:00
John Kerl
ae61dc5a4f
Fix precedence of coalesce operators ?? and ??? (#1270)
* Fix precedence of coalesce operators

* GOCC codegen

* unit-test updates

* operator-precedence documentation

* doc-build artifacts
2023-04-15 23:33:49 -04:00
John Kerl
b8f3902e6e
Treaet data-file numbers with leading + as numeric (#1269)
* accept data-file numbers with leading + as numeric

* augment unit-test cases

* artifacts from `make dev`
2023-04-15 15:57:22 -04:00
John Kerl
fa9b00ee64
index DSL function [WIP] (#1247) 2023-03-27 00:37:33 -04:00
John Kerl
ff659b8a7e
Add optional second base argument to int DSL function (#1244)
* Add optional second base argument to `int` DSL function

* artifacts from `make dev`

* unit-test cases

* typofix
2023-03-24 00:28:39 -04:00
John Kerl
bfc8ab5ce2
Add mlr step -a rprod for running products (#1228)
* mlr step -a rprod

* artifacts from `make dev`

* codespell
2023-03-12 13:32:23 -04:00
John Kerl
f528e65802 Merge branch 'main' of github.com:johnkerl/miller 2023-03-06 00:11:59 -05:00
John Kerl
e9846313a5 doc note for #1218 2023-03-06 00:11:54 -05:00
John Kerl
6ea94a002b
Add new upcase and downcase verbs (#1217)
* upcase/downcase verbs

* unit-test cases

* doc files

* remaining `make dev` artifacts: manpage
2023-03-04 01:08:19 -05:00
John Kerl
38d7de545d
Docs re tail -f and --records-per-batch 1 (#1218) 2023-03-04 00:15:01 -05:00
John Kerl
b741b6ea78 build artifacts 2023-03-01 23:03:22 -05:00
John Kerl
79265e0fce miller 6.7.0 2023-03-01 22:37:58 -05:00
John Kerl
6fc33c682a
--ofmte, --ofmtf, --ofmtg (#1206) 2023-02-27 22:50:46 -05:00
John Kerl
ee0ac3d7bb
New leftpad and rightpad DSL functions (#1205)
* Initial code for `leftpad` and `rightpad` functions

* typofix, and test cases

* doc-gen

* windows UT
2023-02-26 17:14:35 -05:00
John Kerl
f3fd2c2416
Complete #1181 (#1184)
* Source impl of #1181

* Full flow for #1181
2023-01-27 22:59:14 -05:00
John Kerl
a6055ef58e post-6.6.0 2023-01-01 17:17:28 -05:00
John Kerl
b1913c296d
6.6.0 (#1169) 2023-01-01 17:12:00 -05:00
John Kerl
472223c609
Fix typo in mlr put documentation (#1140)
* Fix typo

* Build artifacts from previous commit
2022-12-05 14:28:51 -05:00
Jakub Wilk
636ca8873d
Fix typos (#1135)
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
2022-11-30 09:27:42 -05:00
John Kerl
d1b79c5231 Post-6.5.0 2022-11-27 13:19:29 -05:00
John Kerl
2ac27714d8 Miller 6.5.0 2022-11-27 12:48:33 -05:00
John Kerl
63db1a9059
mlr reshape multiple regex via multiple -r (#1091) 2022-09-05 22:05:54 -04:00
John Kerl
abf58f4bb5
Restore --quote-all for CSV output (#1084)
* Restore --quote-all flag for CSV output

* make dev: for doc-build artifacts
2022-08-22 09:38:53 -04:00