Remove os.Exit callsites below the entrypoint: phase 3 (plans/exit.md) (#2204)

Phase 3 of plans/exit.md: the streaming-interface change, the load-bearing
piece for #341 (DSL exit statement) and #440 (strict mode).

- RecordTransformer.Transform and RecordTransformerFunc now return error.
  All 69 Transform implementations and their dispatch helpers updated
  (mechanical rewrite, compiler- and errcheck-verified).
- runSingleTransformerBatch, on a Transform error, forwards any output
  produced before the failure plus an end-of-stream marker downstream, so
  the rest of the chain and the record-writer drain and finish cleanly;
  runSingleTransformer then surfaces the error to stream.Stream's select
  loop (non-blocking send; first error wins) and signals upstream-done so
  the record-reader stops. This is exactly the flush-then-exit sequencing a
  future DSL 'exit N' needs.
- dataProcessingErrorChannel and FileOutputHandler.recordErroredChannel are
  now chan error instead of chan bool; ChannelWriter still prints write-
  error details at the site and sends the 'exiting due to data error'
  sentinel, preserving the exact stderr shape pinned by regression cases.
- Mid-stream os.Exit sites converted to returned errors: put/filter DSL
  begin/main/end-block errors and the non-boolean filter-expression case,
  tee write/close failures, split write/open/close failures, join left-file
  ingest failures (both half-streaming and sorted paths, with full error
  plumbing through JoinBucketKeeper), histogram/stats2 ingest errors, surv
  fit errors, and step stepper allocation (tStepperAllocator now returns
  (tStepper, error); bad EWMA coefficients propagate; negative slwin
  parameters are reported by the CLI parser via the existing
  bad-stepper-name pattern).
- The two genuinely internal join-bucket-keeper states now use
  lib.InternalCodingErrorWithMessageIf instead of hand-rolled print+exit.
- pkg/transformers is now os.Exit-free.

Behavior notes: the non-boolean filter message gains the standard 'mlr: '
prefix and a newline (it previously printed with neither); tee errors now
include the underlying cause. All 4779 regression cases pass unchanged;
mlr head early-out latency is unaffected (0.02s over 50M records).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
John Kerl 2026-07-15 15:07:09 -04:00 committed by GitHub
parent f17276e0c4
commit 570fcf0de4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
78 changed files with 604 additions and 405 deletions

View file

@ -2,7 +2,6 @@ package stream
import (
"bufio"
"errors"
"io"
"github.com/johnkerl/miller/v6/pkg/cli"
@ -66,10 +65,14 @@ func Stream(
// We're done when a fatal error is registered on input (file not found,
// etc) or when the record-writer has written all its output. We use
// channels to communicate both of these conditions.
// channels to communicate both of these conditions. The
// dataProcessingErrorChannel carries mid-stream errors from the
// transformer chain (e.g. DSL runtime errors, tee/split write failures)
// and from the record-writer; senders use a non-blocking send, so the
// first error wins and is returned once the writer finishes draining.
inputErrorChannel := make(chan error, 1)
doneWritingChannel := make(chan bool, 1)
dataProcessingErrorChannel := make(chan bool, 1)
dataProcessingErrorChannel := make(chan error, 1)
// For mlr head, so a transformer can communicate it will disregard all
// further input. It writes this back upstream, and that is passed back to
@ -84,7 +87,7 @@ func Stream(
go recordReader.Read(fileNames, *initialContext, readerChannel, inputErrorChannel, readerDownstreamDoneChannel)
go transformers.ChainTransformer(readerChannel, readerDownstreamDoneChannel, recordTransformers,
writerChannel, options)
writerChannel, dataProcessingErrorChannel, options)
go output.ChannelWriter(writerChannel, recordWriter, &options.WriterOptions, doneWritingChannel,
dataProcessingErrorChannel, bufferedOutputStream, outputIsStdout)
@ -94,8 +97,8 @@ func Stream(
select {
case ierr := <-inputErrorChannel:
retval = ierr
case <-dataProcessingErrorChannel:
retval = errors.New("exiting due to data error") // details already printed
case derr := <-dataProcessingErrorChannel:
retval = derr
case <-doneWritingChannel:
done = true
}