mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
* Tier-2 structured verb options: OptionSpec, initial migration (#2098) PR 3 of the AI-friendly roadmap (plans/plan-2098-llm.md). Infrastructure: - Add OptionSpec{Flag,Arg,Type,Desc,Repeatable,Values} to pkg/transformers/aaa_record_transformer.go alongside TransformerSetup. Type is one of: bool, string, int, float, csv-list, regex, filename, format, enum. For type=="enum", Values lists the valid choices. - Add Options []OptionSpec to TransformerSetup (nil = not yet migrated). - Emit Options in VerbInfoForJSON (omitempty so unmigrated verbs stay backward-compatible; agents check key presence for Tier-2 availability). UsageText is always present as the Tier-1 prose fallback. - Add VerbOptionsNilCheck() in aaa_verb_options_check.go: progress report of migrated vs. unmigrated verbs, analogous to FLAG_TABLE.NilCheck(). - Wire verb-options-nil-check into mlr help (internal/docgen section). Initial migration (5/70 verbs): - nothing: empty Options (no verb-specific options, explicitly migrated) - cat: -n (bool), -N (string), -g (csv-list), --filename, --filenum (bool) - head: -g (csv-list), -n (int) - tail: -g (csv-list), -n (int) - tee: -a, -p (bool) Tests: - 5 new unit tests in aaa_transformer_json_test.go covering migrated/ unmigrated paths, field population, JSON round-trip, and key-presence. - Regression test case 0003: mlr help verb-options-nil-check golden output. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Migrate all 70 verbs to structured OptionSpec; bump catalog schema to v2 Completes the Tier-2 migration started in the previous commit. Every verb in TRANSFORMER_LOOKUP_TABLE now has a non-nil Options field. - Workflow-migrated all 65 remaining verbs. Each Setup var now carries Options: []OptionSpec{...} with Flag/Arg/Type/Desc fields. Verbs with no verb-specific options (altkv, check, group-like, nothing, etc.) use an empty slice to signal "migrated but no options." - Drop `omitempty` from VerbInfoForJSON.Options: empty slices were silently dropped, making migrated-no-option verbs indistinguishable from unmigrated ones in JSON. Without omitempty: null=unmigrated, []=migrated-no-options, [...]= migrated-with-options. Bump catalogSchemaVersion 1→2 for this shape change. - Replace the two "unmigrated-verb" unit tests (which used stats1 as an example) with TestAllVerbsFullyMigrated (asserts every verb has non-nil Options) and TestAllVerbsHaveOptionsKeyInJSON (asserts every migrated verb emits the "options" key in JSON). - Regenerate test/cases/cli-help/0003/expout: now reads "Verb options migration: 70/70 migrated." Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * remove a transitional helper * git rms * Render verb usage Options blocks from structured OptionSpec Each verb's usage message and its Tier-2 OptionSpec list previously duplicated the option text. New WriteVerbOptions (aaa_verb_usage.go) renders the "Options:" block from the specs: aligned flag column, descriptions word-wrapped at 80, uniform trailing -h|--help line. - OptionSpec gains Aliases (JSON "aliases") so long-form spellings like join's --lk|--left-keep-field-names survive in both outputs - All 70 verbs migrated; options literals hoisted to package-level vars (usage funcs can't reference their Setup var without a Go init cycle) - Hand-written per-option details the specs had condensed away are merged into Desc, enriching the JSON catalog - Non-option prose (examples, cross-references, dynamic accumulator listings) kept verbatim - Regenerated the six usage-embedding regression expectations and the two affected doc pages Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix pre-existing usage-text bugs surfaced by the OptionSpec migration - gap: usage said "One of -f or -g is required" but the parser takes -n or -g - seqgen: drop description line copy-pasted from cat ("Passes input records directly to output...") which contradicted "Discards the input record stream" - utf8-to-latin1: description read inverted ("from Latin-1 to UTF-8") - sec2gmtdate: usage said "../c/mlr" instead of "mlr" - top: document the accepted-but-undocumented --max flag - stats2: add linreg-pca to the -a enum values, matching the runtime accumulator table Regression expectations and docs regenerated accordingly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix check usage sentence order; stats1 usage blank line to usage stream - check: the description's second and third lines were swapped, reading "Consumes records without printing any output, / Useful for doing a well-formatted check on input data. / with the exception that warnings are printed to stderr." - stats1: a bare fmt.Println() in the usage func wrote its blank line to process stdout instead of the usage output stream Regression expectation and docs regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
95 lines
2.9 KiB
Go
95 lines
2.9 KiB
Go
// Machine-readable (JSON) accessors over the verb (transformer) catalog, for
|
|
// `mlr help --as-json` and similar tooling.
|
|
//
|
|
// Tier-1: every verb exposes Summary and UsageText (prose fallback).
|
|
// Tier-2: verbs that have been migrated populate Options on their
|
|
// TransformerSetup; those entries appear as a structured option list in the
|
|
// JSON and agents no longer need to scrape the prose.
|
|
|
|
package transformers
|
|
|
|
import (
|
|
"bytes"
|
|
"io"
|
|
"os"
|
|
"strings"
|
|
)
|
|
|
|
// VerbInfoForJSON is the structured view of a single verb.
|
|
// - Options is non-nil for Tier-2 verbs that have been migrated; agents
|
|
// should prefer it when available.
|
|
// - UsageText is always present as the Tier-1 prose fallback.
|
|
type VerbInfoForJSON struct {
|
|
Name string `json:"name"`
|
|
Summary string `json:"summary"`
|
|
IgnoresInput bool `json:"ignores_input"`
|
|
Options []OptionSpec `json:"options"`
|
|
UsageText string `json:"usage_text"`
|
|
}
|
|
|
|
// captureUsageFunc runs a verb's UsageFunc against a pipe and returns what it
|
|
// printed. The UsageFunc signature takes an *os.File, so we hand it the
|
|
// write-end of a pipe directly -- no global-stdout swap needed. A goroutine
|
|
// drains the read-end so we never block on the OS pipe buffer.
|
|
func captureUsageFunc(usageFunc TransformerUsageFunc) string {
|
|
r, w, err := os.Pipe()
|
|
if err != nil {
|
|
return ""
|
|
}
|
|
|
|
done := make(chan string)
|
|
go func() {
|
|
var buf bytes.Buffer
|
|
_, _ = io.Copy(&buf, r)
|
|
done <- buf.String()
|
|
}()
|
|
|
|
usageFunc(w)
|
|
_ = w.Close()
|
|
s := <-done
|
|
_ = r.Close()
|
|
return s
|
|
}
|
|
|
|
// summarizeUsageText returns the first line that isn't blank or a "Usage:"
|
|
// banner -- the closest thing each verb has to a one-line description.
|
|
func summarizeUsageText(usageText string) string {
|
|
for _, line := range strings.Split(usageText, "\n") {
|
|
trimmed := strings.TrimSpace(line)
|
|
if trimmed == "" || strings.HasPrefix(trimmed, "Usage:") {
|
|
continue
|
|
}
|
|
return trimmed
|
|
}
|
|
return ""
|
|
}
|
|
|
|
func makeVerbInfoForJSON(setup *TransformerSetup) *VerbInfoForJSON {
|
|
usageText := captureUsageFunc(setup.UsageFunc)
|
|
return &VerbInfoForJSON{
|
|
Name: setup.Verb,
|
|
Summary: summarizeUsageText(usageText),
|
|
IgnoresInput: setup.IgnoresInput,
|
|
Options: setup.Options, // nil for unmigrated verbs; omitted from JSON via omitempty
|
|
UsageText: strings.TrimRight(usageText, "\n"),
|
|
}
|
|
}
|
|
|
|
// GetVerbInfosForJSON returns the full verb catalog in table order.
|
|
func GetVerbInfosForJSON() []*VerbInfoForJSON {
|
|
infos := make([]*VerbInfoForJSON, 0, len(TRANSFORMER_LOOKUP_TABLE))
|
|
for i := range TRANSFORMER_LOOKUP_TABLE {
|
|
infos = append(infos, makeVerbInfoForJSON(&TRANSFORMER_LOOKUP_TABLE[i]))
|
|
}
|
|
return infos
|
|
}
|
|
|
|
// GetVerbInfoForJSON returns the structured view of a single verb, or nil if
|
|
// there is no such verb.
|
|
func GetVerbInfoForJSON(verb string) *VerbInfoForJSON {
|
|
setup := LookUp(verb)
|
|
if setup == nil {
|
|
return nil
|
|
}
|
|
return makeVerbInfoForJSON(setup)
|
|
}
|