miller/pkg/transformers/aaa_transformer_json_test.go
John Kerl f637633420
Tier-2 structured verb options: OptionSpec, initial migration (#2098) (#2111)
* Tier-2 structured verb options: OptionSpec, initial migration (#2098)

PR 3 of the AI-friendly roadmap (plans/plan-2098-llm.md).

Infrastructure:
- Add OptionSpec{Flag,Arg,Type,Desc,Repeatable,Values} to
  pkg/transformers/aaa_record_transformer.go alongside TransformerSetup.
  Type is one of: bool, string, int, float, csv-list, regex, filename,
  format, enum. For type=="enum", Values lists the valid choices.
- Add Options []OptionSpec to TransformerSetup (nil = not yet migrated).
- Emit Options in VerbInfoForJSON (omitempty so unmigrated verbs stay
  backward-compatible; agents check key presence for Tier-2 availability).
  UsageText is always present as the Tier-1 prose fallback.
- Add VerbOptionsNilCheck() in aaa_verb_options_check.go: progress report
  of migrated vs. unmigrated verbs, analogous to FLAG_TABLE.NilCheck().
- Wire verb-options-nil-check into mlr help (internal/docgen section).

Initial migration (5/70 verbs):
- nothing: empty Options (no verb-specific options, explicitly migrated)
- cat: -n (bool), -N (string), -g (csv-list), --filename, --filenum (bool)
- head: -g (csv-list), -n (int)
- tail: -g (csv-list), -n (int)
- tee: -a, -p (bool)

Tests:
- 5 new unit tests in aaa_transformer_json_test.go covering migrated/
  unmigrated paths, field population, JSON round-trip, and key-presence.
- Regression test case 0003: mlr help verb-options-nil-check golden output.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Migrate all 70 verbs to structured OptionSpec; bump catalog schema to v2

Completes the Tier-2 migration started in the previous commit. Every verb
in TRANSFORMER_LOOKUP_TABLE now has a non-nil Options field.

- Workflow-migrated all 65 remaining verbs. Each Setup var now carries
  Options: []OptionSpec{...} with Flag/Arg/Type/Desc fields. Verbs with
  no verb-specific options (altkv, check, group-like, nothing, etc.) use
  an empty slice to signal "migrated but no options."
- Drop `omitempty` from VerbInfoForJSON.Options: empty slices were silently
  dropped, making migrated-no-option verbs indistinguishable from unmigrated
  ones in JSON. Without omitempty: null=unmigrated, []=migrated-no-options,
  [...]= migrated-with-options. Bump catalogSchemaVersion 1→2 for this shape
  change.
- Replace the two "unmigrated-verb" unit tests (which used stats1 as an
  example) with TestAllVerbsFullyMigrated (asserts every verb has non-nil
  Options) and TestAllVerbsHaveOptionsKeyInJSON (asserts every migrated
  verb emits the "options" key in JSON).
- Regenerate test/cases/cli-help/0003/expout: now reads
  "Verb options migration: 70/70 migrated."

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* remove a transitional helper

* git rms

* Render verb usage Options blocks from structured OptionSpec

Each verb's usage message and its Tier-2 OptionSpec list previously
duplicated the option text. New WriteVerbOptions (aaa_verb_usage.go)
renders the "Options:" block from the specs: aligned flag column,
descriptions word-wrapped at 80, uniform trailing -h|--help line.

- OptionSpec gains Aliases (JSON "aliases") so long-form spellings
  like join's --lk|--left-keep-field-names survive in both outputs
- All 70 verbs migrated; options literals hoisted to package-level
  vars (usage funcs can't reference their Setup var without a Go
  init cycle)
- Hand-written per-option details the specs had condensed away are
  merged into Desc, enriching the JSON catalog
- Non-option prose (examples, cross-references, dynamic accumulator
  listings) kept verbatim
- Regenerated the six usage-embedding regression expectations and
  the two affected doc pages

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix pre-existing usage-text bugs surfaced by the OptionSpec migration

- gap: usage said "One of -f or -g is required" but the parser takes
  -n or -g
- seqgen: drop description line copy-pasted from cat ("Passes input
  records directly to output...") which contradicted "Discards the
  input record stream"
- utf8-to-latin1: description read inverted ("from Latin-1 to UTF-8")
- sec2gmtdate: usage said "../c/mlr" instead of "mlr"
- top: document the accepted-but-undocumented --max flag
- stats2: add linreg-pca to the -a enum values, matching the runtime
  accumulator table

Regression expectations and docs regenerated accordingly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix check usage sentence order; stats1 usage blank line to usage stream

- check: the description's second and third lines were swapped,
  reading "Consumes records without printing any output, / Useful for
  doing a well-formatted check on input data. / with the exception
  that warnings are printed to stderr."
- stats1: a bare fmt.Println() in the usage func wrote its blank line
  to process stdout instead of the usage output stream

Regression expectation and docs regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-07-03 14:27:23 -04:00

138 lines
4 KiB
Go

package transformers
import (
"encoding/json"
"testing"
)
// TestMigratedVerbsHaveOptions checks that verbs which have been given
// structured Options emit them in their JSON catalog entry (non-nil, non-empty
// for verbs that actually have options).
func TestMigratedVerbsHaveOptions(t *testing.T) {
cases := []struct {
verb string
wantNonNilOpts bool
wantMinOptCount int
}{
{"nothing", true, 0}, // explicitly migrated, no verb-specific options
{"cat", true, 5},
{"head", true, 2},
{"tail", true, 2},
{"tee", true, 2},
}
for _, tc := range cases {
info := GetVerbInfoForJSON(tc.verb)
if info == nil {
t.Errorf("verb %q not found in catalog", tc.verb)
continue
}
if tc.wantNonNilOpts && info.Options == nil {
t.Errorf("verb %q: Options is nil, want non-nil (migrated)", tc.verb)
continue
}
if len(info.Options) < tc.wantMinOptCount {
t.Errorf("verb %q: got %d options, want >= %d", tc.verb, len(info.Options), tc.wantMinOptCount)
}
}
}
// TestAllVerbsFullyMigrated asserts the full-migration invariant: every verb
// in the catalog has a non-nil Options slice and a non-empty UsageText.
// This is the Tier-2 completion check; it fails if a new verb is added without
// populating Options.
func TestAllVerbsFullyMigrated(t *testing.T) {
for i := range TRANSFORMER_LOOKUP_TABLE {
setup := &TRANSFORMER_LOOKUP_TABLE[i]
if setup.Options == nil {
t.Errorf("verb %q has nil Options (not yet migrated to Tier-2)", setup.Verb)
}
info := GetVerbInfoForJSON(setup.Verb)
if info == nil {
t.Errorf("verb %q: GetVerbInfoForJSON returned nil", setup.Verb)
continue
}
if info.UsageText == "" {
t.Errorf("verb %q: UsageText is empty", setup.Verb)
}
}
}
// TestOptionSpecFieldsPopulated verifies that migrated OptionSpec entries have
// the required fields filled in.
func TestOptionSpecFieldsPopulated(t *testing.T) {
info := GetVerbInfoForJSON("cat")
if info == nil {
t.Fatal("cat not found")
}
for _, opt := range info.Options {
if opt.Flag == "" {
t.Errorf("cat: OptionSpec with empty Flag: %+v", opt)
}
if opt.Type == "" {
t.Errorf("cat: OptionSpec %q has empty Type", opt.Flag)
}
if opt.Desc == "" {
t.Errorf("cat: OptionSpec %q has empty Desc", opt.Flag)
}
}
}
// TestOptionsRoundTripJSON verifies that OptionSpec survives JSON
// marshal/unmarshal with all fields intact.
func TestOptionsRoundTripJSON(t *testing.T) {
info := GetVerbInfoForJSON("cat")
if info == nil {
t.Fatal("cat not found")
}
b, err := json.Marshal(info)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var roundTrip VerbInfoForJSON
if err := json.Unmarshal(b, &roundTrip); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if len(roundTrip.Options) != len(info.Options) {
t.Errorf("options count: got %d, want %d", len(roundTrip.Options), len(info.Options))
}
for i, orig := range info.Options {
got := roundTrip.Options[i]
if got.Flag != orig.Flag || got.Type != orig.Type || got.Desc != orig.Desc {
t.Errorf("option[%d] mismatch: got %+v, want %+v", i, got, orig)
}
}
}
// TestAllVerbsHaveOptionsKeyInJSON verifies that every fully-migrated verb
// emits an "options" key in its JSON output (even when the list is empty),
// so agents can rely on key presence as the Tier-2 signal.
func TestAllVerbsHaveOptionsKeyInJSON(t *testing.T) {
for i := range TRANSFORMER_LOOKUP_TABLE {
setup := &TRANSFORMER_LOOKUP_TABLE[i]
if setup.Options == nil {
continue // not yet migrated; skip rather than double-fail
}
info := GetVerbInfoForJSON(setup.Verb)
if info == nil {
t.Errorf("verb %q: GetVerbInfoForJSON returned nil", setup.Verb)
continue
}
b, err := json.Marshal(info)
if err != nil {
t.Errorf("verb %q: marshal error: %v", setup.Verb, err)
continue
}
var raw map[string]any
if err := json.Unmarshal(b, &raw); err != nil {
t.Errorf("verb %q: unmarshal error: %v", setup.Verb, err)
continue
}
if _, ok := raw["options"]; !ok {
t.Errorf("verb %q: missing \"options\" key in JSON", setup.Verb)
}
}
}