The join verb's help text listed only '-i {one of csv,dkvp,nidx,pprint,xtab}'
for overriding the left-file input format, but the verb also accepts the
--icsv/--ijson-style main-flag shorthands (and formats beyond the five
listed, e.g. json and tsv), since unrecognized verb flags fall through to
the main flag table. Update the usage text to say so, and regenerate the
man page and docs content that embed this help output.
Fixes#444.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Tier-2 structured verb options: OptionSpec, initial migration (#2098)
PR 3 of the AI-friendly roadmap (plans/plan-2098-llm.md).
Infrastructure:
- Add OptionSpec{Flag,Arg,Type,Desc,Repeatable,Values} to
pkg/transformers/aaa_record_transformer.go alongside TransformerSetup.
Type is one of: bool, string, int, float, csv-list, regex, filename,
format, enum. For type=="enum", Values lists the valid choices.
- Add Options []OptionSpec to TransformerSetup (nil = not yet migrated).
- Emit Options in VerbInfoForJSON (omitempty so unmigrated verbs stay
backward-compatible; agents check key presence for Tier-2 availability).
UsageText is always present as the Tier-1 prose fallback.
- Add VerbOptionsNilCheck() in aaa_verb_options_check.go: progress report
of migrated vs. unmigrated verbs, analogous to FLAG_TABLE.NilCheck().
- Wire verb-options-nil-check into mlr help (internal/docgen section).
Initial migration (5/70 verbs):
- nothing: empty Options (no verb-specific options, explicitly migrated)
- cat: -n (bool), -N (string), -g (csv-list), --filename, --filenum (bool)
- head: -g (csv-list), -n (int)
- tail: -g (csv-list), -n (int)
- tee: -a, -p (bool)
Tests:
- 5 new unit tests in aaa_transformer_json_test.go covering migrated/
unmigrated paths, field population, JSON round-trip, and key-presence.
- Regression test case 0003: mlr help verb-options-nil-check golden output.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* Migrate all 70 verbs to structured OptionSpec; bump catalog schema to v2
Completes the Tier-2 migration started in the previous commit. Every verb
in TRANSFORMER_LOOKUP_TABLE now has a non-nil Options field.
- Workflow-migrated all 65 remaining verbs. Each Setup var now carries
Options: []OptionSpec{...} with Flag/Arg/Type/Desc fields. Verbs with
no verb-specific options (altkv, check, group-like, nothing, etc.) use
an empty slice to signal "migrated but no options."
- Drop `omitempty` from VerbInfoForJSON.Options: empty slices were silently
dropped, making migrated-no-option verbs indistinguishable from unmigrated
ones in JSON. Without omitempty: null=unmigrated, []=migrated-no-options,
[...]= migrated-with-options. Bump catalogSchemaVersion 1→2 for this shape
change.
- Replace the two "unmigrated-verb" unit tests (which used stats1 as an
example) with TestAllVerbsFullyMigrated (asserts every verb has non-nil
Options) and TestAllVerbsHaveOptionsKeyInJSON (asserts every migrated
verb emits the "options" key in JSON).
- Regenerate test/cases/cli-help/0003/expout: now reads
"Verb options migration: 70/70 migrated."
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* remove a transitional helper
* git rms
* Render verb usage Options blocks from structured OptionSpec
Each verb's usage message and its Tier-2 OptionSpec list previously
duplicated the option text. New WriteVerbOptions (aaa_verb_usage.go)
renders the "Options:" block from the specs: aligned flag column,
descriptions word-wrapped at 80, uniform trailing -h|--help line.
- OptionSpec gains Aliases (JSON "aliases") so long-form spellings
like join's --lk|--left-keep-field-names survive in both outputs
- All 70 verbs migrated; options literals hoisted to package-level
vars (usage funcs can't reference their Setup var without a Go
init cycle)
- Hand-written per-option details the specs had condensed away are
merged into Desc, enriching the JSON catalog
- Non-option prose (examples, cross-references, dynamic accumulator
listings) kept verbatim
- Regenerated the six usage-embedding regression expectations and
the two affected doc pages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix pre-existing usage-text bugs surfaced by the OptionSpec migration
- gap: usage said "One of -f or -g is required" but the parser takes
-n or -g
- seqgen: drop description line copy-pasted from cat ("Passes input
records directly to output...") which contradicted "Discards the
input record stream"
- utf8-to-latin1: description read inverted ("from Latin-1 to UTF-8")
- sec2gmtdate: usage said "../c/mlr" instead of "mlr"
- top: document the accepted-but-undocumented --max flag
- stats2: add linreg-pca to the -a enum values, matching the runtime
accumulator table
Regression expectations and docs regenerated accordingly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix check usage sentence order; stats1 usage blank line to usage stream
- check: the description's second and third lines were swapped,
reading "Consumes records without printing any output, / Useful for
doing a well-formatted check on input data. / with the exception
that warnings are printed to stderr."
- stats1: a bare fmt.Println() in the usage func wrote its blank line
to process stdout instead of the usage output stream
Regression expectation and docs regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* refine the plan
* Fix all staticcheck lint findings (uncapped)
golangci-lint's default max-same-issues=3 was hiding most of the backlog:
the true pre-fix count was 69 staticcheck findings, not 34. This fixes all
of them, driving staticcheck to zero:
- ST1023/QF1011 (37): omit explicit types inferred from the RHS
- S1009/S1031 (15): drop redundant nil checks before len()/range
- SA9003 (9): remove comment-only empty branches, keeping the comments
- QF1007 (3): merge conditional assignment into declaration
- QF1006 (3): lift break conditions into loop conditions
- QF1001 (3): apply De Morgan's law / name the negated predicate
Also updates plans/lintfixes.md with the cap discovery and the corrected
errcheck picture (1202 uncapped, ~949 of them fmt.Fprint*).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drive errcheck to zero: config for bulk categories, propagate real errors
Adds .golangci.yml with errcheck exclude-functions for fmt.Fprint* (usage
printers), (*bufio.Writer).Write/WriteString (sticky errors, surfaced at the
now-checked final Flush), and (*strings.Builder).WriteString; pins
max-issues-per-linter/max-same-issues to 0 so CI reports true counts.
Real error paths now propagate instead of being dropped:
- Finalize{Reader,Writer}Options in join/put/filter/split/tee and the
repl/script entry points: 'mlr join -i badformat' now errors instead of
silently using wrong separators
- final output-stream Flush in pkg/stream: write failure no longer exits 0
- DSL emit/print/dump redirect writes, matching their sibling branches
- CSV writer WriteCSVRecordMaybeColorized, close-time Flush in file output
handlers, ENV[...] Setenv, REPL record-write and redirect-close errors
- termcvt write-side Close before rename (had "TODO: check return status")
The rest are deliberate ignores, marked with _ = and a comment where the
reason isn't obvious: unset-of-missing-path no-ops, read-side closes,
mid-stream FlushOnEveryRecord, init-time strftime registrations, in-memory
usage-capture pipes, and regtest-harness env/temp-file teardown.
golangci-lint now reports 0 issues on ./cmd/mlr ./pkg/... with all caps off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Replaces 100+ if/else-if chains on a single variable with tagged switch
statements across 72 files. The bulk are transformer option-parsing loops
(switch on opt string), plus a handful of value-dispatch sites in mlrval,
dsl/cst, repl, lib, auxents, and bifs. One case (surv.go) required a
labeled break to preserve the loop-exit behavior of the original else branch.
Fixes staticcheck QF1003 findings.
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
The --lp/--rp prefixes and -j output-field rename were applied only to
paired records, leaving unpaired records emitted via --ul/--ur with their
original (left/right) field names. This broke downstream operations like
unsparsify that expected consistent column names across paired and
unpaired output.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Switch to integer ranges in for loops
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Switch to slices functions where appropriate
A number of utility functions can be replaced outright; since Miller
can technically be used as a library, these are deprecated rather than
removed. go:fix directives ensure that they can be replaced
automatically.
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Switch to reflect.TypeFor
This is slightly more efficient than TypeOf when the type is known at
compile time.
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Switch to strings.SplitSeq instead of strings.Split
SplitSeq results in fewer allocations.
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Drop obsolete build directives
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Use min/max instead of explicit comparisons
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Append slices instead of looping
Signed-off-by: Stephen Kitt <steve@sk2.org>
---------
Signed-off-by: Stephen Kitt <steve@sk2.org>
