* Rebuild docs: help-catalog index count drifted on main (666 -> 667)
Pre-existing drift from a recent catalog addition; surfaced by make dev.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove os.Exit callsites below the entrypoint: phase 1 (plans/exit.md)
Phase 1 of plans/exit.md: the mechanical swaps, plus the single-exit-point
scaffolding.
- New sentinel lib.ExitRequest{Code} (io.EOF-style control flow): returned by
code paths that have already printed what they wanted (--version, -h,
'put --explain' on a valid expression, 'put -x') instead of exiting mid-stack.
- pkg/entrypoint: new exitOnError() maps errors to exit codes in one place:
ErrHelpRequested -> 0, ErrUsagePrinted -> 1, ExitRequest -> its code,
anything else printed (JSON if --errors-json) -> 1.
- pkg/climain: version/help/usage/validation exits become returned errors;
the ErrHelpRequested/ErrUsagePrinted -> exit translation moves up to the
entrypoint; loadMlrrcOrDie becomes error-returning loadMlrrcFiles.
- Transformer ParseCLI/constructor exits -> returned errors (cut,
having_fields, nest, reorder, merge_fields, reshape, rename, split, tee,
subs, put_or_filter). merge_fields' bad-regex message formerly mis-reported
itself as coming from the cut verb; tee's unrecognized-option exit was
formerly silent; split/tee constructor failures formerly exited without
printing the constructor's error.
- YAML/JSON record-writer marshal errors -> returned through Write, matching
the CSV writer's error path.
- lib.WriteTempFileOrDie -> WriteTempFile (string, error); its sole caller
(regtest diff helper) degrades gracefully.
Stderr messages and exit codes are byte-identical for all regression-covered
paths (4779 cases pass); runtime Transform-path exits are phase 3.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add lib.NewExitZeroRequest() constructor for exit-0 sentinel returns
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Plan: flesh out PR7 (MCP server + Agent Skill) design
stdio transport (no HTTP port), mlr mcp terminal in the main binary,
SDK-vs-handroll decision, tool list, in-process vs subprocess split,
run-tool safety (--no-shell prerequisite), single-sourced skill, tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add mlr mcp: MCP server + agent playbook; --no-shell gate (#2098 PR7)
New terminal `mlr mcp` runs a Model Context Protocol server over stdio
(spawned by MCP clients; no network port), exposing five tools --
list_capabilities, which, validate_dsl, describe_data, run -- plus an
agent playbook as MCP prompt/resource. Catalog tools are served
in-process from the help registries; the rest subprocess this same
binary with MLR_ERRORS_JSON=1, a timeout, and an output cap.
Prerequisite: a new --no-shell flag / MLR_NO_SHELL env var (one-way
gate) disables the DSL system/exec functions, piped redirects, and
--prepipe/--prepipex; the MCP server sets it on the commands it runs
unless started with --allow-shell.
Adds the github.com/modelcontextprotocol/go-sdk dependency.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Force LF checkout for the embedded SKILL.md (Windows CI fix)
go:embed embeds checkout bytes, so a CRLF checkout on Windows made the
embedded playbook differ per platform and failed
TestPlaybookHasFrontmatter. Pin the file to eol=lf in .gitattributes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Move no-shell test DSL into per-case mlr files (Windows CI fix)
Inline single-quoted DSL in cmd files is mangled by the Windows shell
(single quotes are not quote characters there); the harness's
put -f ${CASEDIR}/mlr pattern avoids shell quoting entirely.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refine the plan
* Fix all staticcheck lint findings (uncapped)
golangci-lint's default max-same-issues=3 was hiding most of the backlog:
the true pre-fix count was 69 staticcheck findings, not 34. This fixes all
of them, driving staticcheck to zero:
- ST1023/QF1011 (37): omit explicit types inferred from the RHS
- S1009/S1031 (15): drop redundant nil checks before len()/range
- SA9003 (9): remove comment-only empty branches, keeping the comments
- QF1007 (3): merge conditional assignment into declaration
- QF1006 (3): lift break conditions into loop conditions
- QF1001 (3): apply De Morgan's law / name the negated predicate
Also updates plans/lintfixes.md with the cap discovery and the corrected
errcheck picture (1202 uncapped, ~949 of them fmt.Fprint*).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drive errcheck to zero: config for bulk categories, propagate real errors
Adds .golangci.yml with errcheck exclude-functions for fmt.Fprint* (usage
printers), (*bufio.Writer).Write/WriteString (sticky errors, surfaced at the
now-checked final Flush), and (*strings.Builder).WriteString; pins
max-issues-per-linter/max-same-issues to 0 so CI reports true counts.
Real error paths now propagate instead of being dropped:
- Finalize{Reader,Writer}Options in join/put/filter/split/tee and the
repl/script entry points: 'mlr join -i badformat' now errors instead of
silently using wrong separators
- final output-stream Flush in pkg/stream: write failure no longer exits 0
- DSL emit/print/dump redirect writes, matching their sibling branches
- CSV writer WriteCSVRecordMaybeColorized, close-time Flush in file output
handlers, ENV[...] Setenv, REPL record-write and redirect-close errors
- termcvt write-side Close before rename (had "TODO: check return status")
The rest are deliberate ignores, marked with _ = and a comment where the
reason isn't obvious: unset-of-missing-path no-ops, read-side closes,
mid-stream FlushOnEveryRecord, init-time strftime registrations, in-memory
usage-capture pipes, and regtest-harness env/temp-file teardown.
golangci-lint now reports 0 issues on ./cmd/mlr ./pkg/... with all caps off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Found with golang.org/x/tools/cmd/deadcode (rooted at cmd/mlr + tests)
and staticcheck U1000; each finding verified by hand before deletion.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Replaces 100+ if/else-if chains on a single variable with tagged switch
statements across 72 files. The bulk are transformer option-parsing loops
(switch on opt string), plus a handful of value-dispatch sites in mlrval,
dsl/cst, repl, lib, auxents, and bifs. One case (surv.go) required a
labeled break to preserve the loop-exit behavior of the original else branch.
Fixes staticcheck QF1003 findings.
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Inspired by GNU head & tail, they match their behavior while supporting
the usual grouping operations.
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
PPRINT, markdown-aligned, and XTAB writers measured column widths with
utf8.RuneCountInString, which counts codepoints rather than terminal
display columns. East-Asian fullwidth characters (counted as 1 but
displayed as 2) and zero-width combining marks (counted as 1 but
displayed as 0) both caused misalignment.
Add lib.DisplayWidth wrapping uniseg.StringWidth, and use it from the
three writers. The XTAB right-aligned path also drops fmt.Sprintf with
%*s since Go's %*s pads by rune count too.
UTF8Strlen is unchanged so DSL strlen semantics are preserved.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Switch to integer ranges in for loops
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Switch to slices functions where appropriate
A number of utility functions can be replaced outright; since Miller
can technically be used as a library, these are deprecated rather than
removed. go:fix directives ensure that they can be replaced
automatically.
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Switch to reflect.TypeFor
This is slightly more efficient than TypeOf when the type is known at
compile time.
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Switch to strings.SplitSeq instead of strings.Split
SplitSeq results in fewer allocations.
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Drop obsolete build directives
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Use min/max instead of explicit comparisons
Signed-off-by: Stephen Kitt <steve@sk2.org>
* Append slices instead of looping
Signed-off-by: Stephen Kitt <steve@sk2.org>
---------
Signed-off-by: Stephen Kitt <steve@sk2.org>
* reader
* record writer
* pkg/output/record_writer_dcf.go
* test/input/test.dcf
* test/cases/io-dcf/
* pkg/cli/option_parse.go
* make fmt
* make dev
* docs and `make dev`
Both `(*Regexp).Match` and `(*Regexp).FindAllSubmatchIndex` have
string-based equivalents: `(*Regexp).MatchString` and
`(*Regexp).FindAllStringSubmatchIndex`. We should use the string version
to avoid unnecessary `[]byte` conversions.
Benchmark:
var regex = regexp.MustCompile("foo.*")
func BenchmarkMatch(b *testing.B) {
for i := 0; i < b.N; i++ {
if match := regex.Match([]byte("foo bar baz")); !match {
b.Fail()
}
}
}
func BenchmarkMatchString(b *testing.B) {
for i := 0; i < b.N; i++ {
if match := regex.MatchString("foo bar baz"); !match {
b.Fail()
}
}
}
func BenchmarkFindAllSubmatchIndex(b *testing.B) {
for i := 0; i < b.N; i++ {
if match := regex.FindAllSubmatchIndex([]byte("foo bar baz"), -1); len(match) == 0 {
b.Fail()
}
}
}
func BenchmarkFindAllStringSubmatchIndex(b *testing.B) {
for i := 0; i < b.N; i++ {
if match := regex.FindAllStringSubmatchIndex("foo bar baz", -1); len(match) == 0 {
b.Fail()
}
}
}
goos: linux
goarch: amd64
pkg: github.com/johnkerl/miller/pkg/lib
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkMatch-16 2198350 517.5 ns/op 16 B/op 1 allocs/op
BenchmarkMatchString-16 3143605 371.5 ns/op 0 B/op 0 allocs/op
BenchmarkFindAllSubmatchIndex-16 921711 1199 ns/op 273 B/op 3 allocs/op
BenchmarkFindAllStringSubmatchIndex-16 1212321 981.0 ns/op 257 B/op 2 allocs/op
PASS
coverage: 0.0% of statements
ok github.com/johnkerl/miller/pkg/lib 6.576s
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
