Commit graph

36 commits

Author SHA1 Message Date
John Kerl
7e1c9b4119
Next batch of lint fixes (#2108) 2026-06-28 17:36:36 -04:00
John Kerl
6f1b3958ed
Switch to --as-json (#2106) 2026-06-28 17:04:10 -04:00
John Kerl
2dd94cb181
Add machine-readable help catalog: mlr help --json (#2098) (#2099)
Emit Miller's existing help catalog (verbs, functions, flags, keywords)
as structured JSON so AI agents and tooling can model Miller's surface
without scraping prose. The --json token may appear anywhere on a
`mlr help ...` command line; plain text help is unchanged.

  mlr help --json                  # full catalog
  mlr help verb cat --json         # one or more verbs
  mlr help function splitax --json # one or more functions
  mlr help flag --ifs --json       # one or more flags
  mlr help keyword ENV --json      # one or more keywords

Functions and flags serialize fully (name/class/arity/help/examples;
section/name/alt_names/arg/help). Verbs carry a summary, ignores_input,
and captured raw usage_text as a Tier-1 fallback, since per-verb options
are prose-only today (each verb hand-writes its UsageFunc). Structured
verb options are a planned follow-on (see #2098).

This is a serialization layer over the existing registries -- no
refactor of the text-help path.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 16:55:54 -04:00
John Kerl
809d312bd4
Add long-over --md flag (#2100) 2026-06-24 13:58:12 -04:00
John Kerl
299636038c
Shell tab-completion for bash and zsh (#2096)
* initial attempt

* fix bash

* fix zsh

* Add shell-completion docs page

Documents the new 'mlr completion {bash,zsh}' feature: the then-chain
context model, install instructions for bash and zsh (including the macOS
bash-3.2 'eval' caveat and zsh compinit self-init), and examples of
context-aware completion. Added to the nav under "Miller in more detail".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Add enum value completion for format and separator flags

Completes the argument value for arg-taking main flags whose values are a
known set: file-format names for -i/-o/--io, separator aliases for
--ifs/--ofs/--ips/etc., and regex-separator aliases for --ifs-regex/--ips-regex.
Other arg-taking flags continue to fall back to filename completion.

Candidate sets come from new cli getters (GetFileFormatNames,
GetSeparatorAliasNames, GetSeparatorRegexAliasNames) that read the same maps
Miller uses at runtime, so there is no separate list to keep in sync. The
command-line walk now records which flag a value position belongs to.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Include format-conversion keystroke-savers in bare-dash completion

Reverts the suppression of --c2j/--x2y-style flags from 'mlr -<TAB>'. The full
set of main flags (297) is now offered, matching what is valid on the command
line. GetFlagNames no longer takes an includeSuppressed argument.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Complete terminal subcommands and top-level help/version flags

'mlr <TAB>' now offers subcommand names (help, version, repl, regtest, script,
completion, terminal-list) alongside verb names, and 'mlr -<TAB>' offers the
top-level terminal flags (-h, --help, --version, --bare-version, and the help
shorthands -g/-l/-L/-f/-F/-k/-K). Subcommand names are offered only as the
first non-flag token, where they are valid.

To let the completion engine know these names without an import cycle
(pkg/terminals imports pkg/terminals/completion), the canonical terminal names
and version-flag spellings are factored into a new leaf package
pkg/terminals/registry, imported by pkg/terminals, pkg/climain, and completion.
The help-flag spellings come from a new help.GetTerminalFlagNames derived from
the existing shorthand table, so nothing drifts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Complete 'mlr help' topics and topic arguments

'mlr help <TAB>' now completes help topics (flags, verb, function, keyword,
list-verbs, ...), and topics that take a name argument complete it too:
'mlr help verb <TAB>' -> verb names, 'mlr help function <TAB>' -> function
names, 'mlr help keyword <TAB>' -> keyword names, 'mlr help flag <TAB>' ->
flag names. 'mlr completion <TAB>' completes bash/zsh.

A terminal subcommand consumes the rest of the command line, so the walk now
returns a ctxTerminalArgs context carrying the terminal name and the words
typed after it. New getters supply the candidate names without drift:
help.GetTopicNames, help.GetFunctionNames/GetKeywordNames (wrapping new
cst.BuiltinFunctionManager.GetBuiltinFunctionNames and cst.GetKeywordNames).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs-neaten

* Move flag-value-candidate logic into pkg/cli; fix verb-flag collision

The mapping of which flags take a format/separator/regex-separator argument is
flag metadata, so it now lives with the flags in pkg/cli as
cli.FlagValueCandidates, alongside the existing GetFileFormatNames /
GetSeparatorAliasNames getters, replacing the maps that were in
pkg/terminals/completion/value_completion.go (now removed).

This also fixes a bug: value completion now applies only to main flags, not to
identically-spelled verb flags. Previously 'mlr uniq -o <TAB>' offered file
formats because uniq's -o (an output field name) collided with the main -o
format flag; it now correctly falls back to filename completion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 12:47:54 -04:00
John Kerl
e0cf596853
Add --omd-aligned flag for column-padded markdown output (#2057)
Adds a new Markdown-only flag --omd-aligned (alias --omarkdown-aligned)
that left-justifies cells and pads each column to a uniform width.
The rendered table is unaffected; the goal is readability of the raw
markdown source. The flag implies --omd, so users do not need to pass
both.

Implementation batches records by schema (like pprint), computes max
column widths, then emits header / separator / data rows with bars
vertically aligned. Default markdown writer behavior is unchanged.
2026-05-16 11:41:01 -04:00
John Kerl
102c624cee
Add DKVPX file format (#2002)
* Initial DKVPX parser

* pkg/input/record_reader_dkvpx*go

* pkg/output/record_writer_dkvpx.go

* docs update

* make fmt

* perf note
2026-03-02 22:35:08 -05:00
Balki
de31cb1fc8
Add fixed width support for pprint reader (#1999)
* Add fixed width support for pprint reader

* fix windows fail

* fix windows fail 2
2026-03-01 11:25:17 -05:00
Balki
d9ee24460c
Add unicode printing support for barred output (#1984) 2026-02-21 23:15:03 -05:00
Stephen Kitt
f0a7c5832f
Performance and style fixes (#1981)
* Switch to integer ranges in for loops

Signed-off-by: Stephen Kitt <steve@sk2.org>

* Switch to slices functions where appropriate

A number of utility functions can be replaced outright; since Miller
can technically be used as a library, these are deprecated rather than
removed. go:fix directives ensure that they can be replaced
automatically.

Signed-off-by: Stephen Kitt <steve@sk2.org>

* Switch to reflect.TypeFor

This is slightly more efficient than TypeOf when the type is known at
compile time.

Signed-off-by: Stephen Kitt <steve@sk2.org>

* Switch to strings.SplitSeq instead of strings.Split

SplitSeq results in fewer allocations.

Signed-off-by: Stephen Kitt <steve@sk2.org>

* Drop obsolete build directives

Signed-off-by: Stephen Kitt <steve@sk2.org>

* Use min/max instead of explicit comparisons

Signed-off-by: Stephen Kitt <steve@sk2.org>

* Append slices instead of looping

Signed-off-by: Stephen Kitt <steve@sk2.org>

---------

Signed-off-by: Stephen Kitt <steve@sk2.org>
2026-02-18 09:19:31 -05:00
John Kerl
e8c4187e67
Fix: mlr -i nonesuch cat says mlr: empty IRS (#1980)
* Fix: `mlr -i nonesuch cat` says `mlr: empty IRS`

* fix experr
2026-02-17 15:30:28 -05:00
John Kerl
e1346691da
Improve error propagation (#1975) 2026-02-16 16:48:41 -05:00
John Kerl
5eb40c9e7b
Multiple style updates (#1974)
* Comment style

* IRecordTransformer -> RecordTransfomer

* make fmt

* else-return style mod

* snake-case -> camel-case

* Remove redundant err = nil and similar zero-value initializations.

* redundant break;

* bugfix

* neaten

* typofix

* simplify/standardize init of zero-length slices

* Standardize fmt.Fprintf w/ errors

* fix double print of "mlr:"

* neatening

* Uniformize error messages

* make docs

* avoid shadowing package names

* shorten some receiver names
2026-02-16 15:49:21 -05:00
John Kerl
a3b5d25933
Neaten some comments (#1973)
* comment-neaten

* comment-neaten

* more
2026-02-16 13:38:49 -05:00
John Kerl
daae7ff7f0
Add DCF (Debian Control Format) as supported file type (#1970)
* reader

* record writer

* pkg/output/record_writer_dcf.go

* test/input/test.dcf

* test/cases/io-dcf/

* pkg/cli/option_parse.go

* make fmt

* make dev

* docs and `make dev`
2026-02-16 11:10:39 -05:00
John Kerl
f375440bd4
Support YAML I/O (#1963)
* docs/src/proposal-yaml-io.md

* initial step

* Add `--y2c` etc

* test cases

* docs

* more testing

* more

* more test cases

* git rm docs/src/proposal-yaml-io.md

* make dev

* ylistwrap -> yarray
2026-02-15 09:24:38 -05:00
John Kerl
3b9f169162
Support -o jsonl as well as --ojsonl (#1879)
* `mlr sort -b` feature

* mlr regtest -p test/cases/cli-help && make dev

* Support `-o jsonl` as well as `--ojsonl`
2025-09-02 16:47:19 -04:00
John Kerl
4d83e88ff6 Note that comment prefix for CSV must be single-character 2025-08-15 19:45:18 -04:00
John Kerl
fccdf215e6
DKVP --incr-key option (#1839)
* Code support for --incr-key

* Add source code for online help for new flag

* Run `make dev`
2025-07-20 17:05:24 -04:00
John Kerl
99a98b0dc7
Add -c, -t, -j to doc matrix in PR 1824 (#1826)
* Add `-c`, `-t`, `-j` to doc matrix in PR 1824

* Run `make dev`
2025-07-03 19:23:38 -04:00
Balki
d6cd981c87
Add Keystroke savers for same format (#1824) 2025-07-03 19:01:17 -04:00
John Kerl
cc1cd954ea
Fix unflatten with field names like . .x or x..y (#1735)
* Fix unflatten with field name like `.` `.x` or `x..y`

* docs & test data
2024-12-23 12:27:08 -05:00
Adam Lesperance
085e831668
The package version must match the major tag version (#1654)
* Update package version

* Update makefile targets

* Update readme packages

* Remaining old packages via rg/sd
2024-09-20 12:10:11 -04:00
John Kerl
73e2117b43
Misc. codespell findings (#1628) 2024-08-25 17:40:57 -04:00
John Kerl
6bee4ebbf2
RS aliases for ASCII top-of-table control characters are misnamed (#1620)
* Fix misnames of ASCII control-character aliases

* artifacts from `make dev`
2024-08-16 10:25:25 -04:00
John Kerl
71d9388bff
Be smarter about auto-unflatten (#1584) 2024-06-08 20:58:26 -04:00
John Kerl
8223903621
Support $NO_COLOR (#1580)
* Support `$NO_COLOR`

* run `make dev`
2024-06-08 13:08:15 -04:00
John Kerl
e714738a7d
Fix typo in online help for --no-jlistwrap (#1541)
* Add --no-auto-unsparsify flag

* Fix typo in online help for `--no-jlistwrap`

* Artifacts from `make dev`
2024-04-11 08:12:45 -04:00
John Kerl
3ff43fa818
Miller produces no output on TSV with > 64K characters per line (#1505)
* Switch to bufio.Reader, first pass

* temp

* Simplify ILineReader by making it stateless

* Interface not necessary; ILineReader -> TLineReader

* neaten

* iterating
2024-02-25 15:50:50 -05:00
John Kerl
af021f28d7
Support markdown format on input (#1478)
* Support markdown on input

* unit-test files

* doc mods

* Unit-test cases for I/O-format keystroke-savers

* -i/-o md as well as -i/-o markdown
2024-01-20 16:51:15 -05:00
John Kerl
bfc829a381
Internal name-neatens (#1475) 2024-01-20 13:36:28 -05:00
John Kerl
794a754c36
Support PPRINT barred input (#1472)
* Support PPRINT barred input

* regression-test files

* output from `make dev`

* doc updates
2024-01-20 12:59:12 -05:00
John Kerl
b13adbe6c0
mlr --norc cat was erroring (#1450) 2023-12-19 09:33:34 -05:00
John Kerl
2bcf8813d3
Add a --files option (#1426)
* mlr --files

* doc mods
2023-11-11 19:09:02 -05:00
John Kerl
5b6a1d4713
JSONL output does not properly handle keys with quotes (#1425)
* mlr --l2j, --j2l

* make dev for previous commit

* fix #1424

* unit-test cases

* iterate
2023-11-11 18:58:49 -05:00
John Kerl
268a96d002
Export library code in pkg/ (#1391)
* Export library code in `pkg/`

* new doc page
2023-09-10 17:15:13 -04:00