mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
5ede886106
When a transformer fails mid-stream (e.g. join -s with a malformed left file), the error could be lost, yielding exit 0 with no stderr message: - runSingleTransformerBatch forwarded the end-of-stream marker downstream before runSingleTransformer sent the error to dataProcessingErrorChannel, so the record-writer could finish and signal done-writing while the error was still unsent. - Even with the error buffered, stream.Stream's select loop chooses among simultaneously-ready channels at random, and exiting on the done-writing signal dropped the buffered error. Send the error before forwarding the end-of-stream marker (so it is always buffered before the writer can finish), and drain the error channels after the select loop exits. Observed as a one-off Windows CI failure of test/cases/verb-join/left-file-malformed-sorted, where the same case passed on automatic rerun within the same job. Reproduced locally by widening the deschedule window with a sleep between the end-of-stream forward and the error send: 5/5 runs exited 0 with empty stderr; with this fix, 200/200 runs exit 1 with the expected message even with adversarial delays injected on both sides of the end-of-stream forward. Follow-up to the os.Exit-removal refactor (plans/exit.md, #2204/#2205). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| doc.go | ||
| README.md | ||
| stream.go | ||
The streamer uses Go channels to pipe together file-reads, to record-reading/parsing, to a chain of record-transformers, to record-writing/formatting, to terminal standard output.
This is the high-level sketch of Miller.