Commit graph

516 commits

Author SHA1 Message Date
John Kerl
3e2eabfd55
Apply join prefixes/rename to unpaired records (#1821) (#2062)
The --lp/--rp prefixes and -j output-field rename were applied only to
paired records, leaving unpaired records emitted via --ul/--ur with their
original (left/right) field names. This broke downstream operations like
unsparsify that expected consistent column names across paired and
unpaired output.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 12:35:58 -04:00
John Kerl
86c8097dc2
Default to "cat" verb when none is supplied (#2060)
* Default to "cat" verb when none is supplied (#2029)

Invocations like 'mlr --j2y' or 'mlr --c2p' previously failed with
"no verb supplied", forcing users to type the trailing 'cat'
explicitly for pure format conversions. Default the verb to 'cat'
in that case. Bare 'mlr' with no flags, no verb, and no files
still prints the main usage banner.

This handles flag-only invocations (e.g. 'mlr --c2j < input.csv'
or 'mlr --c2j --from input.csv'). File names without a preceding
verb are still parsed as verb candidates and continue to error if
not found; that broader change is out of scope here.

* Use ${MLR} substitution for bare-mlr regression case (#2029)

The regtester only substitutes the mlr executable when the cmd
starts with "mlr " (with a trailing space). The bare-mlr usage-banner
test had a cmd of just "mlr", so on CI -- which invokes regtest with
a relative path like 'test/../mlr' -- the test shelled out to a
literal 'mlr' that isn't on PATH and failed with exit 127.

Switch the cmd to ${MLR} (the regtester's explicit substitution
token) so the case runs the right binary in any invocation context.
2026-05-17 12:13:21 -04:00
John Kerl
1bb9ba12c2
Fix mlr -s shebang doc and reject arrays/maps in contains/index (#1658) (#2058)
The shebang example `#!/usr/bin/env mlr -s` does not work on Linux
because env(1) does not accept arguments to the interpreter; use
`#!/usr/bin/env -S mlr -s` instead.

`contains` and `index` previously stringified their inputs silently,
so `contains([1,2], $foo)` would match against the literal string
`[1, 2]` and produce surprising results. Both now return a type-error
when either argument is an array or map; help text points users at
`any(arr, func(e){return e == x})` for membership tests.
2026-05-17 11:40:39 -04:00
John Kerl
e0cf596853
Add --omd-aligned flag for column-padded markdown output (#2057)
Adds a new Markdown-only flag --omd-aligned (alias --omarkdown-aligned)
that left-justifies cells and pads each column to a uniform width.
The rendered table is unaffected; the goal is readability of the raw
markdown source. The flag implies --omd, so users do not need to pass
both.

Implementation batches records by schema (like pprint), computes max
column widths, then emits header / separator / data rows with bars
vertically aligned. Default markdown writer behavior is unchanged.
2026-05-16 11:41:01 -04:00
John Kerl
9a25ba5dfa Post-6.18.1 release: back to 6.18.1-dev 2026-04-19 12:03:05 -04:00
John Kerl
22bed96346 Prepare 6.18.1 release 2026-04-19 11:41:48 -04:00
John Kerl
655193732b
Add regexed field-selection to sort-within-records (#1964)
* BROKEN

* add a test case

* fix

* fix natural-sorting issue

* fix silent-drop issue

* test case

* doc update
2026-04-19 11:35:58 -04:00
John Kerl
1a50215d3d
Add a ./tools/release.sh script (#2043)
* tools/release.sh

* Updates doc page re release script

* testing

* Prepare 6.18.0 release
2026-04-19 10:43:01 -04:00
John Kerl
3e429f5b42
Use "\n" in replace for gsub and sub (#2042)
* Fix issue 1805

* Run `make dev`
2026-04-19 10:21:51 -04:00
John Kerl
0395cbddc6
Run make dev after PR #2026 (#2027) 2026-04-08 08:50:39 -04:00
John Kerl
af1adf80ad
Replace GOCC parser-generator with PGPG (#2015)
* Porting

* Update some tests that depend on AST-print output

* iterating on GOCC -> PGPG

* iterating

* iterating

* iterating

* iterating

* iterating

* Modify expout files that need only AST-print-syntax updates

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* iterating

* sync test cases from mlr-6.17.0, except AST-prints ...

* sync test cases from mlr-6.17.0, except AST-prints ...

* Fix lashed emit for $* and @*

* Fix --1 and ++1 chained unary ops

* Emit with function callsite

* test cases

* dot operator

* M_PI and M_E

* Iterating on lashed emit cases

* error-wording differences

* rm some should-fail files

* Fix issue with leading semicolon

* trailing comma in func params; most AST-print deltas

* error-wording delta

* fix unset all

* AST-print deltas

* go mod tidy: forced `go 1.25` to `go 1.25.0`

* Depend on PGPG v1.0.0

* Fix Windows CI failure

* Remove cmd/experiments/dsl_parser

* GOCC -> PGPG

* neaten

* Fix regex issue found in doc gen
2026-03-15 22:28:57 -04:00
John Kerl
f26b6be555 sieve2.mlr 2026-03-05 20:50:12 -05:00
John Kerl
f8a8f29584
Experiment with mlr script (#2009) 2026-03-05 20:14:32 -05:00
John Kerl
102c624cee
Add DKVPX file format (#2002)
* Initial DKVPX parser

* pkg/input/record_reader_dkvpx*go

* pkg/output/record_writer_dkvpx.go

* docs update

* make fmt

* perf note
2026-03-02 22:35:08 -05:00
John Kerl
763b4d8f71 typofix 2026-02-22 17:08:15 -05:00
John Kerl
f60314daaf
Update performance docs (#1991)
* Organize perf-stats runs

* reorg

* scripts/perf/timings-2026-02-22.txt

* iterating

* iterating

* more

* .gitignore

* more

* link checks
2026-02-22 17:04:35 -05:00
John Kerl
3103b11323 typofix 2026-02-22 13:06:10 -05:00
John Kerl
2e3f0b3a2f typofix 2026-02-22 13:05:08 -05:00
John Kerl
b10e6e3d34
Update performance numbers (#1990)
* wip

* re-run
2026-02-22 12:58:52 -05:00
John Kerl
8a84388b11
Add docs example on custom record-output formatting (#1989) 2026-02-22 12:45:45 -05:00
John Kerl
5815ab1cea
Add record-templating example to docs (#1988) 2026-02-22 12:01:03 -05:00
John Kerl
cd39c4e1bd
make dev ater PR 1984 (#1985) 2026-02-21 23:24:20 -05:00
John Kerl
4535922db0 Update release docs 2026-02-21 09:44:23 -05:00
John Kerl
47ca5eb88a Post-6.17.0 2026-02-21 09:40:15 -05:00
John Kerl
dc00aa3e16 Miller 6.17.0 2026-02-21 09:16:55 -05:00
John Kerl
5eb40c9e7b
Multiple style updates (#1974)
* Comment style

* IRecordTransformer -> RecordTransfomer

* make fmt

* else-return style mod

* snake-case -> camel-case

* Remove redundant err = nil and similar zero-value initializations.

* redundant break;

* bugfix

* neaten

* typofix

* simplify/standardize init of zero-length slices

* Standardize fmt.Fprintf w/ errors

* fix double print of "mlr:"

* neatening

* Uniformize error messages

* make docs

* avoid shadowing package names

* shorten some receiver names
2026-02-16 15:49:21 -05:00
John Kerl
daae7ff7f0
Add DCF (Debian Control Format) as supported file type (#1970)
* reader

* record writer

* pkg/output/record_writer_dcf.go

* test/input/test.dcf

* test/cases/io-dcf/

* pkg/cli/option_parse.go

* make fmt

* make dev

* docs and `make dev`
2026-02-16 11:10:39 -05:00
John Kerl
f375440bd4
Support YAML I/O (#1963)
* docs/src/proposal-yaml-io.md

* initial step

* Add `--y2c` etc

* test cases

* docs

* more testing

* more

* more test cases

* git rm docs/src/proposal-yaml-io.md

* make dev

* ylistwrap -> yarray
2026-02-15 09:24:38 -05:00
John Kerl
0003b83654
mlr split with --folder (#1962) 2026-02-14 20:22:24 -05:00
John Kerl
26600f77aa
Add -r (for regex field-name matching) to mlr nest (#1961)
* initial attempt

* fix up cli flag

* make dev
2026-02-14 19:59:51 -05:00
John Kerl
5e0114e0cf
mlr reorder -r (#1960)
* `mlr reorder -r`

* fix

* update test/cases/cli-help/

* make dev
2026-02-14 19:40:20 -05:00
John Kerl
18e5204513
Require that filter expressions be boolean (or absent) (#1935)
* Insist that filter expressions be boolean

* docs
2026-02-01 12:36:38 -05:00
John Kerl
e08e3ca80c
Add snapcraft.io link to install instructions 2026-01-07 13:45:36 -05:00
John Kerl
cee04c0747 Fix Snap link 2026-01-02 14:50:20 -05:00
John Kerl
b8db798a2f
Miller 6.16.0 (#1930)
* Miller 6.16.0

* make dev
2026-01-02 13:57:59 -05:00
John Kerl
d0f824aefe
Run make dev after merge of PR 1868 (#1869) 2025-08-20 10:21:51 -07:00
John Kerl
3c2d4b22d2
Miller 6.15.0-dev (#1862)
* 6.15.0-dev

* make dev
2025-08-15 19:55:46 -04:00
John Kerl
8b524b3ada make dev 2025-08-15 19:48:48 -04:00
John Kerl
4ebef873d2
Miller 6.15.0 (#1860)
* miller 6.15.0

* make dev
2025-08-14 18:00:22 -04:00
John Kerl
52b7a47ae9
Use Go 1.24.5 (#1843) 2025-07-22 20:15:48 -04:00
John Kerl
fccdf215e6
DKVP --incr-key option (#1839)
* Code support for --incr-key

* Add source code for online help for new flag

* Run `make dev`
2025-07-20 17:05:24 -04:00
John Kerl
d264f562dc
Fix doc typo re empty and multiplication (#1838)
* Fix docs typo re empty and multiplication

* Run `make dev`
2025-07-20 16:36:50 -04:00
John Kerl
e7fe363d9a
mlr sort -b feature (#1833)
* `mlr sort -b` feature

* mlr regtest -p test/cases/cli-help && make dev
2025-07-11 12:41:04 -04:00
John Kerl
3137313867 Release-specific docs for 6.14.0 2025-07-04 15:18:08 -04:00
John Kerl
0ba6710a79 Update main version to 6.14.0-dev 2025-07-04 15:10:26 -04:00
John Kerl
127c4925a2 Merge branch 'main' of https://github.com/johnkerl/miller 2025-07-04 14:16:14 -04:00
John Kerl
fefb304650 Update release docs on xattr trick for MacOS 2025-07-04 14:15:59 -04:00
John Kerl
7a6958926d
Miller 6.14.0 (#1828) 2025-07-04 13:55:56 -04:00
John Kerl
b7248bae98
Doc copy edits (#1827)
* Update index.md.in

* more copy-editing

* swipes.sh

* swipes.sh

* run `make docs` to generate `*.md` from `*.md.in`
2025-07-04 13:43:22 -04:00
John Kerl
99a98b0dc7
Add -c, -t, -j to doc matrix in PR 1824 (#1826)
* Add `-c`, `-t`, `-j` to doc matrix in PR 1824

* Run `make dev`
2025-07-03 19:23:38 -04:00