mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
Remove os.Exit callsites below the entrypoint: phase 3 (plans/exit.md) (#2204)
Phase 3 of plans/exit.md: the streaming-interface change, the load-bearing piece for #341 (DSL exit statement) and #440 (strict mode). - RecordTransformer.Transform and RecordTransformerFunc now return error. All 69 Transform implementations and their dispatch helpers updated (mechanical rewrite, compiler- and errcheck-verified). - runSingleTransformerBatch, on a Transform error, forwards any output produced before the failure plus an end-of-stream marker downstream, so the rest of the chain and the record-writer drain and finish cleanly; runSingleTransformer then surfaces the error to stream.Stream's select loop (non-blocking send; first error wins) and signals upstream-done so the record-reader stops. This is exactly the flush-then-exit sequencing a future DSL 'exit N' needs. - dataProcessingErrorChannel and FileOutputHandler.recordErroredChannel are now chan error instead of chan bool; ChannelWriter still prints write- error details at the site and sends the 'exiting due to data error' sentinel, preserving the exact stderr shape pinned by regression cases. - Mid-stream os.Exit sites converted to returned errors: put/filter DSL begin/main/end-block errors and the non-boolean filter-expression case, tee write/close failures, split write/open/close failures, join left-file ingest failures (both half-streaming and sorted paths, with full error plumbing through JoinBucketKeeper), histogram/stats2 ingest errors, surv fit errors, and step stepper allocation (tStepperAllocator now returns (tStepper, error); bad EWMA coefficients propagate; negative slwin parameters are reported by the CLI parser via the existing bad-stepper-name pattern). - The two genuinely internal join-bucket-keeper states now use lib.InternalCodingErrorWithMessageIf instead of hand-rolled print+exit. - pkg/transformers is now os.Exit-free. Behavior notes: the non-boolean filter message gains the standard 'mlr: ' prefix and a newline (it previously printed with neither); tee errors now include the underlying cause. All 4779 regression cases pass unchanged; mlr head early-out latency is unaffected (0.02s over 50M records). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
f17276e0c4
commit
570fcf0de4
78 changed files with 604 additions and 405 deletions
|
|
@ -2,7 +2,6 @@ package stream
|
|||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"io"
|
||||
|
||||
"github.com/johnkerl/miller/v6/pkg/cli"
|
||||
|
|
@ -66,10 +65,14 @@ func Stream(
|
|||
|
||||
// We're done when a fatal error is registered on input (file not found,
|
||||
// etc) or when the record-writer has written all its output. We use
|
||||
// channels to communicate both of these conditions.
|
||||
// channels to communicate both of these conditions. The
|
||||
// dataProcessingErrorChannel carries mid-stream errors from the
|
||||
// transformer chain (e.g. DSL runtime errors, tee/split write failures)
|
||||
// and from the record-writer; senders use a non-blocking send, so the
|
||||
// first error wins and is returned once the writer finishes draining.
|
||||
inputErrorChannel := make(chan error, 1)
|
||||
doneWritingChannel := make(chan bool, 1)
|
||||
dataProcessingErrorChannel := make(chan bool, 1)
|
||||
dataProcessingErrorChannel := make(chan error, 1)
|
||||
|
||||
// For mlr head, so a transformer can communicate it will disregard all
|
||||
// further input. It writes this back upstream, and that is passed back to
|
||||
|
|
@ -84,7 +87,7 @@ func Stream(
|
|||
|
||||
go recordReader.Read(fileNames, *initialContext, readerChannel, inputErrorChannel, readerDownstreamDoneChannel)
|
||||
go transformers.ChainTransformer(readerChannel, readerDownstreamDoneChannel, recordTransformers,
|
||||
writerChannel, options)
|
||||
writerChannel, dataProcessingErrorChannel, options)
|
||||
go output.ChannelWriter(writerChannel, recordWriter, &options.WriterOptions, doneWritingChannel,
|
||||
dataProcessingErrorChannel, bufferedOutputStream, outputIsStdout)
|
||||
|
||||
|
|
@ -94,8 +97,8 @@ func Stream(
|
|||
select {
|
||||
case ierr := <-inputErrorChannel:
|
||||
retval = ierr
|
||||
case <-dataProcessingErrorChannel:
|
||||
retval = errors.New("exiting due to data error") // details already printed
|
||||
case derr := <-dataProcessingErrorChannel:
|
||||
retval = derr
|
||||
case <-doneWritingChannel:
|
||||
done = true
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue