Commit graph

9 commits

Author SHA1 Message Date
John Kerl
1004ba5097
DSL format function: support 1-based positional placeholders like {1} (#1650) (#2160)
Plain {} placeholders consume arguments sequentially as before; {N}
refers to the Nth argument (1-based) and may be used to repeat and/or
reorder arguments, e.g. format("{1}/{2}/{1}_{3}.ext", $p1, $p2, $p3).
Mixing is allowed: the {} sequence counter is independent of positional
placeholders (as in Rust format!). Out-of-range indices interpolate the
empty string, consistent with existing too-few-arguments behavior; {0}
is an error value since indices are 1-based.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 15:15:33 -04:00
John Kerl
f637633420
Tier-2 structured verb options: OptionSpec, initial migration (#2098) (#2111)
* Tier-2 structured verb options: OptionSpec, initial migration (#2098)

PR 3 of the AI-friendly roadmap (plans/plan-2098-llm.md).

Infrastructure:
- Add OptionSpec{Flag,Arg,Type,Desc,Repeatable,Values} to
  pkg/transformers/aaa_record_transformer.go alongside TransformerSetup.
  Type is one of: bool, string, int, float, csv-list, regex, filename,
  format, enum. For type=="enum", Values lists the valid choices.
- Add Options []OptionSpec to TransformerSetup (nil = not yet migrated).
- Emit Options in VerbInfoForJSON (omitempty so unmigrated verbs stay
  backward-compatible; agents check key presence for Tier-2 availability).
  UsageText is always present as the Tier-1 prose fallback.
- Add VerbOptionsNilCheck() in aaa_verb_options_check.go: progress report
  of migrated vs. unmigrated verbs, analogous to FLAG_TABLE.NilCheck().
- Wire verb-options-nil-check into mlr help (internal/docgen section).

Initial migration (5/70 verbs):
- nothing: empty Options (no verb-specific options, explicitly migrated)
- cat: -n (bool), -N (string), -g (csv-list), --filename, --filenum (bool)
- head: -g (csv-list), -n (int)
- tail: -g (csv-list), -n (int)
- tee: -a, -p (bool)

Tests:
- 5 new unit tests in aaa_transformer_json_test.go covering migrated/
  unmigrated paths, field population, JSON round-trip, and key-presence.
- Regression test case 0003: mlr help verb-options-nil-check golden output.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Migrate all 70 verbs to structured OptionSpec; bump catalog schema to v2

Completes the Tier-2 migration started in the previous commit. Every verb
in TRANSFORMER_LOOKUP_TABLE now has a non-nil Options field.

- Workflow-migrated all 65 remaining verbs. Each Setup var now carries
  Options: []OptionSpec{...} with Flag/Arg/Type/Desc fields. Verbs with
  no verb-specific options (altkv, check, group-like, nothing, etc.) use
  an empty slice to signal "migrated but no options."
- Drop `omitempty` from VerbInfoForJSON.Options: empty slices were silently
  dropped, making migrated-no-option verbs indistinguishable from unmigrated
  ones in JSON. Without omitempty: null=unmigrated, []=migrated-no-options,
  [...]= migrated-with-options. Bump catalogSchemaVersion 1→2 for this shape
  change.
- Replace the two "unmigrated-verb" unit tests (which used stats1 as an
  example) with TestAllVerbsFullyMigrated (asserts every verb has non-nil
  Options) and TestAllVerbsHaveOptionsKeyInJSON (asserts every migrated
  verb emits the "options" key in JSON).
- Regenerate test/cases/cli-help/0003/expout: now reads
  "Verb options migration: 70/70 migrated."

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* remove a transitional helper

* git rms

* Render verb usage Options blocks from structured OptionSpec

Each verb's usage message and its Tier-2 OptionSpec list previously
duplicated the option text. New WriteVerbOptions (aaa_verb_usage.go)
renders the "Options:" block from the specs: aligned flag column,
descriptions word-wrapped at 80, uniform trailing -h|--help line.

- OptionSpec gains Aliases (JSON "aliases") so long-form spellings
  like join's --lk|--left-keep-field-names survive in both outputs
- All 70 verbs migrated; options literals hoisted to package-level
  vars (usage funcs can't reference their Setup var without a Go
  init cycle)
- Hand-written per-option details the specs had condensed away are
  merged into Desc, enriching the JSON catalog
- Non-option prose (examples, cross-references, dynamic accumulator
  listings) kept verbatim
- Regenerated the six usage-embedding regression expectations and
  the two affected doc pages

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix pre-existing usage-text bugs surfaced by the OptionSpec migration

- gap: usage said "One of -f or -g is required" but the parser takes
  -n or -g
- seqgen: drop description line copy-pasted from cat ("Passes input
  records directly to output...") which contradicted "Discards the
  input record stream"
- utf8-to-latin1: description read inverted ("from Latin-1 to UTF-8")
- sec2gmtdate: usage said "../c/mlr" instead of "mlr"
- top: document the accepted-but-undocumented --max flag
- stats2: add linreg-pca to the -a enum values, matching the runtime
  accumulator table

Regression expectations and docs regenerated accordingly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix check usage sentence order; stats1 usage blank line to usage stream

- check: the description's second and third lines were swapped,
  reading "Consumes records without printing any output, / Useful for
  doing a well-formatted check on input data. / with the exception
  that warnings are printed to stderr."
- stats1: a bare fmt.Println() in the usage func wrote its blank line
  to process stdout instead of the usage output stream

Regression expectation and docs regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-07-03 14:27:23 -04:00
John Kerl
d72ef826fb
Add DSL functions for integer nanoseconds since the epoch (#1326)
* DSL functions for 64-bit nano-epoch timestamps

* strfntime

* nsec2gmt; move sec/nsec pairs adjacent to one another

* update on-line help

* artifacts from `make dev`

* unit-test files
2023-06-24 17:05:15 -04:00
John Kerl
5801f5c0a2
Implement --csv-trim-leading-space flag (#1272)
* Implement --csv-trim-leading-space flag

* Artifacts from `make dev`
2023-04-20 13:13:55 -03:00
John Kerl
3aebb68ad6 test cases and doc-build artifacts 2022-02-24 09:57:34 -05:00
John Kerl
b6bad96caf
Fix #891 (#892)
* Fix #891

* update tests; artifacts for previous commit
2022-01-20 13:34:58 -05:00
John Kerl
1785f8d111 doc-build artifacts for previous commit 2022-01-13 09:46:39 -05:00
John Kerl
a0048f0393
New format DSL function (#869)
* New format DSL function

* Updated affected test cases involving on-line help on "for" prefix

* doc-build artifacts for previous commit

* regression-test cases
2022-01-12 22:40:59 -05:00
John Kerl
4394819cf8
Approximate-match feature for online help (#754)
* Explicitly support approximate-match help

* Unit tests for exact and approximate help
2021-11-17 23:37:48 -05:00