miller/regression_test.go
John Kerl 2f3e54d1f6
Remove os.Exit callsites below the entrypoint: phase 4 (plans/exit.md) (#2205)
Phase 4 of plans/exit.md: the aux/terminal sub-entrypoints. pkg/auxents and
pkg/terminals are now os.Exit-free; even the dispatchers return exit codes
instead of exiting.

- auxents.Dispatch returns (handled bool, exitCode int); entrypoint.Main --
  the one sanctioned exit point -- exits with the code.
- terminals.Dispatch returns an exit code; the climain caller wraps it in a
  lib.ExitRequest, which also retires the 'terminal did not exit the process'
  panic.
- Usage functions (hex, unhex, lecat, termcvt, repl, script, regtest) no
  longer take an exitCode parameter and exit; call sites print usage and
  return the code explicitly.
- Inner I/O helpers (hexDumpFile, unhexFile, lecatFile, termcvtFile) return
  errors; each sub-main prints 'mlr {verb}: ...' as before and returns 1.
  lecatFile formerly looped forever on a non-EOF read error; it now returns
  the error.
- The regtest directory walk (Execute / executeSinglePath /
  executeSingleDirectory / hasCaseSubdirectories) plumbs errors up to
  RegTestMain; regression_test.go updated for the new Execute signature.

Stderr messages and exit codes are unchanged across mlr aux-list, hex, unhex,
lecat, termcvt, help, version, repl, script, and regtest surfaces (checked by
hand); all 4779 regression cases pass; make lint is clean; docs rebuild with
zero churn.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 15:27:26 -04:00

75 lines
2.4 KiB
Go

package main
import (
"fmt"
"os"
"testing"
"github.com/johnkerl/miller/v6/pkg/terminals/regtest"
)
// TestRegression is a familiar entry point for regression testing. Miller
// regression tests are more flexibly invoked via 'mlr regtest'. However here
// is a standard location so people can get at them via 'go test'. Please see
// (as of this writing) pkg/terminals/regtest for the Miller regtest package.
func TestRegression(t *testing.T) {
// How much detail to show? There are thousands of cases, organized into a
// few hundred top-level directories under ./test/cases.
//
// Default behavior is to show PASS/FAIL for those top-level directories.
// If (for whatever reason) lots of tests are systematically failing then
// verbosityLevel = 3 for all cases is probably too much output to be
// useful.
//
// Also note our regtest framework supports four verbosity levels, 'mlr
// regtest' (0) through 'mlr regtest -vvv' (3), while 'go test' has only
// 'go test' and 'go test -v'. Our regtest framework also has 'mlr regtest
// -s 20' which means *re-run* up to 20 failing tests (after having failed
// once with verbosityLevel = 0) as if those had been invoked with
// verbosityLevel = 3.
//
// What we do is:
// * go test: like 'mlr regtest'
// * go test -v: like 'mlr regtest -s 20'
//
// This is (I hope) sufficient flexibility for use in GitHub Actions
// continuous-integration jobs. If more detail is needed then one may:
//
// * For CI debugging: simply edit the below parameters verbosityLevel
// and firstNFailsToShow and re-push to GitHub.
// * For interactive debug: run 'mlr regtest -v', 'mlr regtest -vv', 'mlr
// regtest -vvv' instead of going through 'go test'.
firstNFailsToShow := 0
if testing.Verbose() {
firstNFailsToShow = 20
}
// Let the tests find ./mlr
cwd, err := os.Getwd()
if err != nil {
fmt.Fprintln(os.Stderr, "mlr: could not find current working directory.")
os.Exit(1)
}
path := os.Getenv("PATH")
os.Setenv("PATH", cwd+":"+path)
// With 'go test' invoked from the repo base directory, the cwd for this code will
// be the tests/ subdirectory.
casePaths := []string{"./test/cases"} // use default
regtester := regtest.NewRegTester(
"mlr", // exeName
false, // doPopulate
0, // verbosityLevel
false, // plainMode
firstNFailsToShow,
)
ok, err := regtester.Execute(casePaths)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal()
}
}