mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
Lint round 5+6: staticcheck and errcheck to zero (#2130)
* refine the plan
* Fix all staticcheck lint findings (uncapped)
golangci-lint's default max-same-issues=3 was hiding most of the backlog:
the true pre-fix count was 69 staticcheck findings, not 34. This fixes all
of them, driving staticcheck to zero:
- ST1023/QF1011 (37): omit explicit types inferred from the RHS
- S1009/S1031 (15): drop redundant nil checks before len()/range
- SA9003 (9): remove comment-only empty branches, keeping the comments
- QF1007 (3): merge conditional assignment into declaration
- QF1006 (3): lift break conditions into loop conditions
- QF1001 (3): apply De Morgan's law / name the negated predicate
Also updates plans/lintfixes.md with the cap discovery and the corrected
errcheck picture (1202 uncapped, ~949 of them fmt.Fprint*).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drive errcheck to zero: config for bulk categories, propagate real errors
Adds .golangci.yml with errcheck exclude-functions for fmt.Fprint* (usage
printers), (*bufio.Writer).Write/WriteString (sticky errors, surfaced at the
now-checked final Flush), and (*strings.Builder).WriteString; pins
max-issues-per-linter/max-same-issues to 0 so CI reports true counts.
Real error paths now propagate instead of being dropped:
- Finalize{Reader,Writer}Options in join/put/filter/split/tee and the
repl/script entry points: 'mlr join -i badformat' now errors instead of
silently using wrong separators
- final output-stream Flush in pkg/stream: write failure no longer exits 0
- DSL emit/print/dump redirect writes, matching their sibling branches
- CSV writer WriteCSVRecordMaybeColorized, close-time Flush in file output
handlers, ENV[...] Setenv, REPL record-write and redirect-close errors
- termcvt write-side Close before rename (had "TODO: check return status")
The rest are deliberate ignores, marked with _ = and a comment where the
reason isn't obvious: unset-of-missing-path no-ops, read-side closes,
mid-stream FlushOnEveryRecord, init-time strftime registrations, in-memory
usage-capture pipes, and regtest-harness env/temp-file teardown.
golangci-lint now reports 0 issues on ./cmd/mlr ./pkg/... with all caps off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
2bec10fc54
commit
91eaff1341
63 changed files with 435 additions and 236 deletions
28
.golangci.yml
Normal file
28
.golangci.yml
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
# Configuration for golangci-lint, used by .github/workflows/golangci-lint.yml
|
||||||
|
# and available locally via `golangci-lint run ./cmd/mlr ./pkg/...`.
|
||||||
|
# Tracking issue for lint cleanup: https://github.com/johnkerl/miller/issues/2109
|
||||||
|
|
||||||
|
version: "2"
|
||||||
|
|
||||||
|
linters:
|
||||||
|
settings:
|
||||||
|
errcheck:
|
||||||
|
exclude-functions:
|
||||||
|
# Writes to stdout/stderr for usage and error messages; there is
|
||||||
|
# nothing useful to do if these fail.
|
||||||
|
- fmt.Fprint
|
||||||
|
- fmt.Fprintf
|
||||||
|
- fmt.Fprintln
|
||||||
|
# bufio.Writer errors are sticky: once a write fails, all subsequent
|
||||||
|
# writes and the next Flush report the same error. Record-writer
|
||||||
|
# output goes through a final checked Flush in pkg/stream.
|
||||||
|
- (*bufio.Writer).Write
|
||||||
|
- (*bufio.Writer).WriteString
|
||||||
|
# Documented to always return a nil error.
|
||||||
|
- (*strings.Builder).WriteString
|
||||||
|
|
||||||
|
issues:
|
||||||
|
# Report all findings: the defaults (50/3) hide large categories and make
|
||||||
|
# fix progress impossible to track. See plans/lintfixes.md.
|
||||||
|
max-issues-per-linter: 0
|
||||||
|
max-same-issues: 0
|
||||||
|
|
@ -55,7 +55,7 @@ func main() {
|
||||||
fmt.Fprintln(os.Stderr, os.Args[0], ": ", "Could not start CPU profile: ", err)
|
fmt.Fprintln(os.Stderr, os.Args[0], ": ", "Could not start CPU profile: ", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
defer handle.Close()
|
defer func() { _ = handle.Close() }()
|
||||||
|
|
||||||
if err := pprof.StartCPUProfile(handle); err != nil {
|
if err := pprof.StartCPUProfile(handle); err != nil {
|
||||||
fmt.Fprintln(os.Stderr, os.Args[0], ": ", "Could not start CPU profile: ", err)
|
fmt.Fprintln(os.Stderr, os.Args[0], ": ", "Could not start CPU profile: ", err)
|
||||||
|
|
|
||||||
|
|
@ -68,7 +68,7 @@ func hexMain(args []string) int {
|
||||||
|
|
||||||
hexDumpFile(istream, doRaw)
|
hexDumpFile(istream, doRaw)
|
||||||
|
|
||||||
istream.Close()
|
_ = istream.Close()
|
||||||
if !doRaw && len(args) > 1 {
|
if !doRaw && len(args) > 1 {
|
||||||
fmt.Println()
|
fmt.Println()
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -54,7 +54,7 @@ func lecatMain(args []string) int {
|
||||||
|
|
||||||
lecatFile(istream, doColor)
|
lecatFile(istream, doColor)
|
||||||
|
|
||||||
istream.Close()
|
_ = istream.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return 0
|
return 0
|
||||||
|
|
|
||||||
|
|
@ -96,9 +96,11 @@ func termcvtMain(args []string) int {
|
||||||
|
|
||||||
termcvtFile(istream, ostream, inputTerminator, outputTerminator)
|
termcvtFile(istream, ostream, inputTerminator, outputTerminator)
|
||||||
|
|
||||||
istream.Close()
|
_ = istream.Close()
|
||||||
// TODO: check return status
|
if err := ostream.Close(); err != nil {
|
||||||
ostream.Close()
|
fmt.Fprintf(os.Stderr, "mlr termcvt: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
err = os.Rename(tempname, filename)
|
err = os.Rename(tempname, filename)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -120,7 +122,7 @@ func termcvtMain(args []string) int {
|
||||||
|
|
||||||
termcvtFile(istream, os.Stdout, inputTerminator, outputTerminator)
|
termcvtFile(istream, os.Stdout, inputTerminator, outputTerminator)
|
||||||
|
|
||||||
istream.Close()
|
_ = istream.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return 0
|
return 0
|
||||||
|
|
@ -144,6 +146,9 @@ func termcvtFile(istream *os.File, ostream *os.File, inputTerminator string, out
|
||||||
|
|
||||||
// This is how to do a chomp:
|
// This is how to do a chomp:
|
||||||
line = strings.TrimRight(line, inputTerminator)
|
line = strings.TrimRight(line, inputTerminator)
|
||||||
ostream.Write([]byte(line + outputTerminator))
|
if _, err := ostream.Write([]byte(line + outputTerminator)); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "mlr termcvt: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,7 @@ func unhexMain(args []string) int {
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
unhexFile(istream, os.Stdout)
|
unhexFile(istream, os.Stdout)
|
||||||
istream.Close()
|
_ = istream.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -83,7 +83,10 @@ func unhexFile(istream *os.File, ostream *os.File) {
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
byteArray[0] = byte(scanValue)
|
byteArray[0] = byte(scanValue)
|
||||||
ostream.Write(byteArray)
|
if _, err := ostream.Write(byteArray); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, "mlr unhex:", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -718,26 +718,26 @@ func BIF_mod_exp(input1, input2, input3 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
// * empty-null always loses against numbers
|
// * empty-null always loses against numbers
|
||||||
|
|
||||||
func min_f_ff(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func min_f_ff(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a float64 = input1.AcquireFloatValue()
|
a := input1.AcquireFloatValue()
|
||||||
var b float64 = input2.AcquireFloatValue()
|
b := input2.AcquireFloatValue()
|
||||||
return mlrval.FromFloat(math.Min(a, b))
|
return mlrval.FromFloat(math.Min(a, b))
|
||||||
}
|
}
|
||||||
|
|
||||||
func min_f_fi(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func min_f_fi(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a float64 = input1.AcquireFloatValue()
|
a := input1.AcquireFloatValue()
|
||||||
var b float64 = float64(input2.AcquireIntValue())
|
b := float64(input2.AcquireIntValue())
|
||||||
return mlrval.FromFloat(math.Min(a, b))
|
return mlrval.FromFloat(math.Min(a, b))
|
||||||
}
|
}
|
||||||
|
|
||||||
func min_f_if(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func min_f_if(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a float64 = float64(input1.AcquireIntValue())
|
a := float64(input1.AcquireIntValue())
|
||||||
var b float64 = input2.AcquireFloatValue()
|
b := input2.AcquireFloatValue()
|
||||||
return mlrval.FromFloat(math.Min(a, b))
|
return mlrval.FromFloat(math.Min(a, b))
|
||||||
}
|
}
|
||||||
|
|
||||||
func min_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func min_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a int64 = input1.AcquireIntValue()
|
a := input1.AcquireIntValue()
|
||||||
var b int64 = input2.AcquireIntValue()
|
b := input2.AcquireIntValue()
|
||||||
if a < b {
|
if a < b {
|
||||||
return input1
|
return input1
|
||||||
}
|
}
|
||||||
|
|
@ -756,8 +756,8 @@ func min_b_bb(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
}
|
}
|
||||||
|
|
||||||
func min_s_ss(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func min_s_ss(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a string = input1.AcquireStringValue()
|
a := input1.AcquireStringValue()
|
||||||
var b string = input2.AcquireStringValue()
|
b := input2.AcquireStringValue()
|
||||||
if a < b {
|
if a < b {
|
||||||
return input1
|
return input1
|
||||||
}
|
}
|
||||||
|
|
@ -883,26 +883,26 @@ func BIF_minlen_within_map_values(m *mlrval.Mlrmap) *mlrval.Mlrval {
|
||||||
}
|
}
|
||||||
|
|
||||||
func max_f_ff(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func max_f_ff(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a float64 = input1.AcquireFloatValue()
|
a := input1.AcquireFloatValue()
|
||||||
var b float64 = input2.AcquireFloatValue()
|
b := input2.AcquireFloatValue()
|
||||||
return mlrval.FromFloat(math.Max(a, b))
|
return mlrval.FromFloat(math.Max(a, b))
|
||||||
}
|
}
|
||||||
|
|
||||||
func max_f_fi(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func max_f_fi(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a float64 = input1.AcquireFloatValue()
|
a := input1.AcquireFloatValue()
|
||||||
var b float64 = float64(input2.AcquireIntValue())
|
b := float64(input2.AcquireIntValue())
|
||||||
return mlrval.FromFloat(math.Max(a, b))
|
return mlrval.FromFloat(math.Max(a, b))
|
||||||
}
|
}
|
||||||
|
|
||||||
func max_f_if(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func max_f_if(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a float64 = float64(input1.AcquireIntValue())
|
a := float64(input1.AcquireIntValue())
|
||||||
var b float64 = input2.AcquireFloatValue()
|
b := input2.AcquireFloatValue()
|
||||||
return mlrval.FromFloat(math.Max(a, b))
|
return mlrval.FromFloat(math.Max(a, b))
|
||||||
}
|
}
|
||||||
|
|
||||||
func max_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func max_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a int64 = input1.AcquireIntValue()
|
a := input1.AcquireIntValue()
|
||||||
var b int64 = input2.AcquireIntValue()
|
b := input2.AcquireIntValue()
|
||||||
if a > b {
|
if a > b {
|
||||||
return input1
|
return input1
|
||||||
}
|
}
|
||||||
|
|
@ -921,8 +921,8 @@ func max_b_bb(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
}
|
}
|
||||||
|
|
||||||
func max_s_ss(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func max_s_ss(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var a string = input1.AcquireStringValue()
|
a := input1.AcquireStringValue()
|
||||||
var b string = input2.AcquireStringValue()
|
b := input2.AcquireStringValue()
|
||||||
if a > b {
|
if a > b {
|
||||||
return input1
|
return input1
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -223,8 +223,8 @@ func BIF_signed_right_shift(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
// Unsigned right shift
|
// Unsigned right shift
|
||||||
|
|
||||||
func ursh_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
func ursh_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
var ua uint64 = uint64(input1.AcquireIntValue())
|
ua := uint64(input1.AcquireIntValue())
|
||||||
var ub uint64 = uint64(input2.AcquireIntValue())
|
ub := uint64(input2.AcquireIntValue())
|
||||||
var uc = ua >> ub
|
var uc = ua >> ub
|
||||||
return mlrval.FromInt(int64(uc))
|
return mlrval.FromInt(int64(uc))
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -448,18 +448,18 @@ func init() {
|
||||||
})
|
})
|
||||||
|
|
||||||
ss := strftime.NewSpecificationSet()
|
ss := strftime.NewSpecificationSet()
|
||||||
ss.Set('1', appender1)
|
_ = ss.Set('1', appender1)
|
||||||
ss.Set('2', appender2)
|
_ = ss.Set('2', appender2)
|
||||||
ss.Set('3', appender3)
|
_ = ss.Set('3', appender3)
|
||||||
ss.Set('4', appender4)
|
_ = ss.Set('4', appender4)
|
||||||
ss.Set('5', appender5)
|
_ = ss.Set('5', appender5)
|
||||||
ss.Set('6', appender6)
|
_ = ss.Set('6', appender6)
|
||||||
ss.Set('7', appender7)
|
_ = ss.Set('7', appender7)
|
||||||
ss.Set('8', appender8)
|
_ = ss.Set('8', appender8)
|
||||||
ss.Set('9', appender9)
|
_ = ss.Set('9', appender9)
|
||||||
ss.Set('N', appenderN)
|
_ = ss.Set('N', appenderN)
|
||||||
ss.Set('O', appenderO)
|
_ = ss.Set('O', appenderO)
|
||||||
ss.Set('s', appenderS)
|
_ = ss.Set('s', appenderS)
|
||||||
|
|
||||||
strftimeExtensions = strftime.WithSpecificationSet(ss)
|
strftimeExtensions = strftime.WithSpecificationSet(ss)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -27,10 +27,7 @@ func BIF_dhms2sec(input1 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
remainingInput := input
|
remainingInput := input
|
||||||
var seconds int64
|
var seconds int64
|
||||||
|
|
||||||
for {
|
for remainingInput != "" {
|
||||||
if remainingInput == "" {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
var n int64
|
var n int64
|
||||||
var rest string
|
var rest string
|
||||||
|
|
||||||
|
|
@ -89,10 +86,7 @@ func BIF_dhms2fsec(input1 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||||
remainingInput := input
|
remainingInput := input
|
||||||
var seconds float64
|
var seconds float64
|
||||||
|
|
||||||
for {
|
for remainingInput != "" {
|
||||||
if remainingInput == "" {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
var f float64
|
var f float64
|
||||||
var rest string
|
var rest string
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3458,7 +3458,7 @@ var MiscFlagSection = FlagSection{
|
||||||
fmt.Fprintf(os.Stderr, "mlr: %v\n", err)
|
fmt.Fprintf(os.Stderr, "mlr: %v\n", err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
defer handle.Close()
|
defer func() { _ = handle.Close() }()
|
||||||
|
|
||||||
lineReader := bufio.NewReader(handle)
|
lineReader := bufio.NewReader(handle)
|
||||||
|
|
||||||
|
|
@ -3565,7 +3565,7 @@ var MiscFlagSection = FlagSection{
|
||||||
help: "Specify timezone, overriding `$TZ` environment variable (if any).",
|
help: "Specify timezone, overriding `$TZ` environment variable (if any).",
|
||||||
parser: func(args []string, argc int, pargi *int, options *TOptions) {
|
parser: func(args []string, argc int, pargi *int, options *TOptions) {
|
||||||
CheckArgCount(args, *pargi, argc, 2)
|
CheckArgCount(args, *pargi, argc, 2)
|
||||||
os.Setenv("TZ", args[*pargi+1])
|
_ = os.Setenv("TZ", args[*pargi+1])
|
||||||
*pargi += 2
|
*pargi += 2
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
|
|
||||||
|
|
@ -47,7 +47,7 @@ func tryLoadMlrrc(
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
defer handle.Close()
|
defer func() { _ = handle.Close() }()
|
||||||
|
|
||||||
lineReader := bufio.NewReader(handle)
|
lineReader := bufio.NewReader(handle)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -63,7 +63,7 @@ func (root *RootNode) BuildStatementBlockNodeFromBeginOrEnd(
|
||||||
// With "parent":1,"children":[1], StatementBlockInBraces.Children[0] is the StatementBlock.
|
// With "parent":1,"children":[1], StatementBlockInBraces.Children[0] is the StatementBlock.
|
||||||
// Unwrap so we pass StatementBlock to BuildStatementBlockNode.
|
// Unwrap so we pass StatementBlock to BuildStatementBlockNode.
|
||||||
if astStatementBlockNode.Type == asts.NodeType(NodeTypeStatementBlockInBraces) {
|
if astStatementBlockNode.Type == asts.NodeType(NodeTypeStatementBlockInBraces) {
|
||||||
lib.InternalCodingErrorIf(astStatementBlockNode.Children == nil || len(astStatementBlockNode.Children) < 1)
|
lib.InternalCodingErrorIf(len(astStatementBlockNode.Children) < 1)
|
||||||
astStatementBlockNode = astStatementBlockNode.Children[0]
|
astStatementBlockNode = astStatementBlockNode.Children[0]
|
||||||
}
|
}
|
||||||
statementBlockNode, err := root.BuildStatementBlockNode(astStatementBlockNode)
|
statementBlockNode, err := root.BuildStatementBlockNode(astStatementBlockNode)
|
||||||
|
|
|
||||||
|
|
@ -875,7 +875,7 @@ func (node *LogicalANDOperatorNode) Evaluate(
|
||||||
|
|
||||||
bout := node.b.Evaluate(state)
|
bout := node.b.Evaluate(state)
|
||||||
btype := bout.Type()
|
btype := bout.Type()
|
||||||
if !(btype == mlrval.MT_ABSENT || btype == mlrval.MT_BOOL) {
|
if btype != mlrval.MT_ABSENT && btype != mlrval.MT_BOOL {
|
||||||
return mlrval.FromNotNamedTypeError("&&", bout, "absent or boolean")
|
return mlrval.FromNotNamedTypeError("&&", bout, "absent or boolean")
|
||||||
}
|
}
|
||||||
if btype == mlrval.MT_ABSENT {
|
if btype == mlrval.MT_ABSENT {
|
||||||
|
|
@ -957,7 +957,7 @@ func (node *LogicalOROperatorNode) Evaluate(
|
||||||
|
|
||||||
bout := node.b.Evaluate(state)
|
bout := node.b.Evaluate(state)
|
||||||
btype := bout.Type()
|
btype := bout.Type()
|
||||||
if !(btype == mlrval.MT_ABSENT || btype == mlrval.MT_BOOL) {
|
if btype != mlrval.MT_ABSENT && btype != mlrval.MT_BOOL {
|
||||||
return mlrval.FromNotNamedTypeError("||", bout, "absent or boolean")
|
return mlrval.FromNotNamedTypeError("||", bout, "absent or boolean")
|
||||||
}
|
}
|
||||||
if btype == mlrval.MT_ABSENT {
|
if btype == mlrval.MT_ABSENT {
|
||||||
|
|
|
||||||
|
|
@ -202,7 +202,9 @@ func (node *DumpStatementNode) Execute(state *runtime.State) (*BlockExitPayload,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
outputString := buffer.String()
|
outputString := buffer.String()
|
||||||
node.dumpToRedirectFunc(outputString, state)
|
if err := node.dumpToRedirectFunc(outputString, state); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return nil, nil
|
return nil, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -244,6 +246,5 @@ func (node *DumpStatementNode) dumpToFileOrPipe(
|
||||||
}
|
}
|
||||||
outputFileName := redirectorTarget.String()
|
outputFileName := redirectorTarget.String()
|
||||||
|
|
||||||
node.outputHandlerManager.WriteString(outputString, outputFileName)
|
return node.outputHandlerManager.WriteString(outputString, outputFileName)
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -537,7 +537,10 @@ func (node *EmitXStatementNode) executeNonIndexedNonLashedEmit(
|
||||||
nextLevelNames = append(nextLevelNames, pe.Key)
|
nextLevelNames = append(nextLevelNames, pe.Key)
|
||||||
nextLevelValues = append(nextLevelValues, pe.Value.Copy())
|
nextLevelValues = append(nextLevelValues, pe.Value.Copy())
|
||||||
}
|
}
|
||||||
node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
err := node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -625,7 +628,10 @@ func (node *EmitXStatementNode) executeNonIndexedLashedEmit(
|
||||||
nextLevelNames = append(nextLevelNames, pe.Key)
|
nextLevelNames = append(nextLevelNames, pe.Key)
|
||||||
nextLevelValues = append(nextLevelValues, pe.Value.Copy())
|
nextLevelValues = append(nextLevelValues, pe.Value.Copy())
|
||||||
}
|
}
|
||||||
node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
err := node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -821,13 +827,16 @@ func (node *EmitXStatementNode) executeIndexedNonLashedEmitAux(
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
node.executeIndexedNonLashedEmitPAux(
|
err := node.executeIndexedNonLashedEmitPAux(
|
||||||
newrec,
|
newrec,
|
||||||
[]string{names[i]},
|
[]string{names[i]},
|
||||||
[]*mlrval.Mlrmap{valueAsMap},
|
[]*mlrval.Mlrmap{valueAsMap},
|
||||||
indices[1:],
|
indices[1:],
|
||||||
state,
|
state,
|
||||||
)
|
)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -966,13 +975,16 @@ func (node *EmitXStatementNode) executeIndexedLashedEmitAux(
|
||||||
if len(indices) > 1 && nextLevelMaps[0] != nil {
|
if len(indices) > 1 && nextLevelMaps[0] != nil {
|
||||||
// Recurse. The leading map drives the iteration; we don't
|
// Recurse. The leading map drives the iteration; we don't
|
||||||
// continue even if other maps aren't empty
|
// continue even if other maps aren't empty
|
||||||
node.executeIndexedLashedEmitAux(
|
err := node.executeIndexedLashedEmitAux(
|
||||||
newrec,
|
newrec,
|
||||||
names,
|
names,
|
||||||
nextLevelMaps,
|
nextLevelMaps,
|
||||||
indices[1:],
|
indices[1:],
|
||||||
state,
|
state,
|
||||||
)
|
)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
} else { // end of recursion
|
} else { // end of recursion
|
||||||
for i, nextLevelValue := range nextLevelValues {
|
for i, nextLevelValue := range nextLevelValues {
|
||||||
if nextLevelValue != nil {
|
if nextLevelValue != nil {
|
||||||
|
|
@ -1027,13 +1039,16 @@ func (node *EmitXStatementNode) executeIndexedNonLashedEmitPAux(
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
node.executeIndexedNonLashedEmitPAux(
|
err := node.executeIndexedNonLashedEmitPAux(
|
||||||
newrec,
|
newrec,
|
||||||
[]string{names[i]},
|
[]string{names[i]},
|
||||||
[]*mlrval.Mlrmap{valueAsMap},
|
[]*mlrval.Mlrmap{valueAsMap},
|
||||||
indices[1:],
|
indices[1:],
|
||||||
state,
|
state,
|
||||||
)
|
)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1083,13 +1098,16 @@ func (node *EmitXStatementNode) executeIndexedLashedEmitPAux(
|
||||||
|
|
||||||
if nextLevelMaps[0] != nil && len(indices) >= 2 {
|
if nextLevelMaps[0] != nil && len(indices) >= 2 {
|
||||||
// recurse
|
// recurse
|
||||||
node.executeIndexedLashedEmitPAux(
|
err := node.executeIndexedLashedEmitPAux(
|
||||||
newrec,
|
newrec,
|
||||||
names,
|
names,
|
||||||
nextLevelMaps,
|
nextLevelMaps,
|
||||||
indices[1:],
|
indices[1:],
|
||||||
state,
|
state,
|
||||||
)
|
)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
// end of recursion
|
// end of recursion
|
||||||
for i, nextLevel := range nextLevels {
|
for i, nextLevel := range nextLevels {
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,7 @@ import (
|
||||||
|
|
||||||
func (root *RootNode) BuildEvaluableNode(astNode *asts.ASTNode) (IEvaluable, error) {
|
func (root *RootNode) BuildEvaluableNode(astNode *asts.ASTNode) (IEvaluable, error) {
|
||||||
// Try BuildLeafNode first for terminals
|
// Try BuildLeafNode first for terminals
|
||||||
if astNode.Children == nil || len(astNode.Children) == 0 {
|
if len(astNode.Children) == 0 {
|
||||||
if leaf, err := root.BuildLeafNode(astNode); err == nil {
|
if leaf, err := root.BuildLeafNode(astNode); err == nil {
|
||||||
return leaf, nil
|
return leaf, nil
|
||||||
}
|
}
|
||||||
|
|
@ -97,7 +97,7 @@ func (root *RootNode) BuildEvaluableNode(astNode *asts.ASTNode) (IEvaluable, err
|
||||||
|
|
||||||
// Fallback: try BuildLeafNode for unhandled types (e.g. DirectFieldValue, IntLiteral).
|
// Fallback: try BuildLeafNode for unhandled types (e.g. DirectFieldValue, IntLiteral).
|
||||||
// Only for leaf-like nodes (0 or 1 child); nodes with 2+ children are not leaves.
|
// Only for leaf-like nodes (0 or 1 child); nodes with 2+ children are not leaves.
|
||||||
if astNode.Children == nil || len(astNode.Children) <= 1 {
|
if len(astNode.Children) <= 1 {
|
||||||
if leaf, err := root.BuildLeafNode(astNode); err == nil {
|
if leaf, err := root.BuildLeafNode(astNode); err == nil {
|
||||||
return leaf, nil
|
return leaf, nil
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -182,9 +182,8 @@ func (node *ForLoopOneVariableNode) Execute(state *runtime.State) (*BlockExitPay
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
} else if indexMlrval.IsAbsent() {
|
|
||||||
// Data-heterogeneity no-op
|
|
||||||
}
|
}
|
||||||
|
// else if indexMlrval.IsAbsent(): data-heterogeneity no-op
|
||||||
|
|
||||||
// TODO: backwards compatibility with the C port means we treat this as
|
// TODO: backwards compatibility with the C port means we treat this as
|
||||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||||
|
|
@ -372,9 +371,8 @@ func (node *ForLoopTwoVariableNode) Execute(state *runtime.State) (*BlockExitPay
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
} else if indexMlrval.IsAbsent() {
|
|
||||||
// Data-heterogeneity no-op
|
|
||||||
}
|
}
|
||||||
|
// else if indexMlrval.IsAbsent(): data-heterogeneity no-op
|
||||||
|
|
||||||
// TODO: backwards compatibility with the C port means we treat this as
|
// TODO: backwards compatibility with the C port means we treat this as
|
||||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||||
|
|
@ -581,9 +579,8 @@ func (node *ForLoopMultivariableNode) executeOuter(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
} else if mv.IsAbsent() {
|
|
||||||
// Data-heterogeneity no-op
|
|
||||||
}
|
}
|
||||||
|
// else if mv.IsAbsent(): data-heterogeneity no-op
|
||||||
|
|
||||||
// TODO: backwards compatibility with the C port means we treat this as
|
// TODO: backwards compatibility with the C port means we treat this as
|
||||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||||
|
|
@ -675,9 +672,8 @@ func (node *ForLoopMultivariableNode) executeInner(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
} else if mv.IsAbsent() {
|
|
||||||
// Data-heterogeneity no-op
|
|
||||||
}
|
}
|
||||||
|
// else if mv.IsAbsent(): data-heterogeneity no-op
|
||||||
|
|
||||||
// TODO: backwards compatibility with the C port means we treat this as
|
// TODO: backwards compatibility with the C port means we treat this as
|
||||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||||
|
|
|
||||||
|
|
@ -33,16 +33,16 @@ func captureStdout(f func()) string {
|
||||||
done := make(chan string)
|
done := make(chan string)
|
||||||
go func() {
|
go func() {
|
||||||
var buf bytes.Buffer
|
var buf bytes.Buffer
|
||||||
io.Copy(&buf, r)
|
_, _ = io.Copy(&buf, r)
|
||||||
done <- buf.String()
|
done <- buf.String()
|
||||||
}()
|
}()
|
||||||
|
|
||||||
f()
|
f()
|
||||||
|
|
||||||
w.Close()
|
_ = w.Close()
|
||||||
os.Stdout = old
|
os.Stdout = old
|
||||||
s := <-done
|
s := <-done
|
||||||
r.Close()
|
_ = r.Close()
|
||||||
return s
|
return s
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,9 +18,9 @@ func (root *RootNode) BuildLeafNode(
|
||||||
) (IEvaluable, error) {
|
) (IEvaluable, error) {
|
||||||
// The BNF uses empty slice for terminals. It may also produce reduced literal nodes
|
// The BNF uses empty slice for terminals. It may also produce reduced literal nodes
|
||||||
// (float_literal, int_literal) with one child (the terminal).
|
// (float_literal, int_literal) with one child (the terminal).
|
||||||
lib.InternalCodingErrorIf(astNode.Children != nil && len(astNode.Children) > 1)
|
lib.InternalCodingErrorIf(len(astNode.Children) > 1)
|
||||||
sval := tokenLit(astNode)
|
sval := tokenLit(astNode)
|
||||||
if sval == "" && astNode.Children != nil && len(astNode.Children) == 1 {
|
if sval == "" && len(astNode.Children) == 1 {
|
||||||
sval = tokenLit(astNode.Children[0])
|
sval = tokenLit(astNode.Children[0])
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -143,7 +143,8 @@ func (node *DirectFieldValueLvalueNode) UnassignIndexed(
|
||||||
name := node.lhsFieldName.String()
|
name := node.lhsFieldName.String()
|
||||||
state.Inrec.Remove(name)
|
state.Inrec.Remove(name)
|
||||||
} else {
|
} else {
|
||||||
state.Inrec.RemoveIndexed(
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Inrec.RemoveIndexed(
|
||||||
append([]*mlrval.Mlrval{node.lhsFieldName}, indices...),
|
append([]*mlrval.Mlrval{node.lhsFieldName}, indices...),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -254,7 +255,8 @@ func (node *IndirectFieldValueLvalueNode) UnassignIndexed(
|
||||||
name := lhsFieldName.String()
|
name := lhsFieldName.String()
|
||||||
state.Inrec.Remove(name)
|
state.Inrec.Remove(name)
|
||||||
} else {
|
} else {
|
||||||
state.Inrec.RemoveIndexed(
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Inrec.RemoveIndexed(
|
||||||
append([]*mlrval.Mlrval{lhsFieldName.Copy()}, indices...),
|
append([]*mlrval.Mlrval{lhsFieldName.Copy()}, indices...),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -357,12 +359,12 @@ func (node *PositionalFieldNameLvalueNode) UnassignIndexed(
|
||||||
index, ok := lhsFieldIndex.GetIntValue()
|
index, ok := lhsFieldIndex.GetIntValue()
|
||||||
if ok {
|
if ok {
|
||||||
state.Inrec.RemoveWithPositionalIndex(index)
|
state.Inrec.RemoveWithPositionalIndex(index)
|
||||||
} else {
|
|
||||||
// TODO: incorporate error-return into this API
|
|
||||||
}
|
}
|
||||||
|
// TODO: incorporate error-return into this API for the non-int case
|
||||||
} else {
|
} else {
|
||||||
// xxx positional
|
// xxx positional
|
||||||
state.Inrec.RemoveIndexed(
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Inrec.RemoveIndexed(
|
||||||
append([]*mlrval.Mlrval{lhsFieldIndex}, indices...),
|
append([]*mlrval.Mlrval{lhsFieldIndex}, indices...),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -475,12 +477,12 @@ func (node *PositionalFieldValueLvalueNode) UnassignIndexed(
|
||||||
index, ok := lhsFieldIndex.GetIntValue()
|
index, ok := lhsFieldIndex.GetIntValue()
|
||||||
if ok {
|
if ok {
|
||||||
state.Inrec.RemoveWithPositionalIndex(index)
|
state.Inrec.RemoveWithPositionalIndex(index)
|
||||||
} else {
|
|
||||||
// TODO: incorporate error-return into this API
|
|
||||||
}
|
}
|
||||||
|
// TODO: incorporate error-return into this API for the non-int case
|
||||||
} else {
|
} else {
|
||||||
// xxx positional
|
// xxx positional
|
||||||
state.Inrec.RemoveIndexed(
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Inrec.RemoveIndexed(
|
||||||
append([]*mlrval.Mlrval{lhsFieldIndex}, indices...),
|
append([]*mlrval.Mlrval{lhsFieldIndex}, indices...),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -493,7 +495,7 @@ func (root *RootNode) BuildFullSrecLvalueNode(astNode *asts.ASTNode) (IAssignabl
|
||||||
lib.InternalCodingErrorIf(astNode.Type != asts.NodeType(NodeTypeFullSrec))
|
lib.InternalCodingErrorIf(astNode.Type != asts.NodeType(NodeTypeFullSrec))
|
||||||
lib.InternalCodingErrorIf(astNode == nil)
|
lib.InternalCodingErrorIf(astNode == nil)
|
||||||
// PGPG FullSrec has empty list as children
|
// PGPG FullSrec has empty list as children
|
||||||
lib.InternalCodingErrorIf(astNode.Children != nil && len(astNode.Children) > 0)
|
lib.InternalCodingErrorIf(len(astNode.Children) > 0)
|
||||||
return NewFullSrecLvalueNode(), nil
|
return NewFullSrecLvalueNode(), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -556,7 +558,8 @@ func (node *FullSrecLvalueNode) UnassignIndexed(
|
||||||
if indices == nil {
|
if indices == nil {
|
||||||
state.Inrec.Clear()
|
state.Inrec.Clear()
|
||||||
} else {
|
} else {
|
||||||
state.Inrec.RemoveIndexed(indices)
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Inrec.RemoveIndexed(indices)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -625,7 +628,8 @@ func (node *DirectOosvarValueLvalueNode) UnassignIndexed(
|
||||||
name := node.lhsOosvarName.String()
|
name := node.lhsOosvarName.String()
|
||||||
state.Oosvars.Remove(name)
|
state.Oosvars.Remove(name)
|
||||||
} else {
|
} else {
|
||||||
state.Oosvars.RemoveIndexed(
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Oosvars.RemoveIndexed(
|
||||||
append([]*mlrval.Mlrval{node.lhsOosvarName}, indices...),
|
append([]*mlrval.Mlrval{node.lhsOosvarName}, indices...),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -705,7 +709,8 @@ func (node *IndirectOosvarValueLvalueNode) UnassignIndexed(
|
||||||
sname := lhsOosvarName.String()
|
sname := lhsOosvarName.String()
|
||||||
state.Oosvars.Remove(sname)
|
state.Oosvars.Remove(sname)
|
||||||
} else {
|
} else {
|
||||||
state.Oosvars.RemoveIndexed(
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Oosvars.RemoveIndexed(
|
||||||
append([]*mlrval.Mlrval{lhsOosvarName}, indices...),
|
append([]*mlrval.Mlrval{lhsOosvarName}, indices...),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -717,7 +722,7 @@ type FullOosvarLvalueNode struct {
|
||||||
func (root *RootNode) BuildFullOosvarLvalueNode(astNode *asts.ASTNode) (IAssignable, error) {
|
func (root *RootNode) BuildFullOosvarLvalueNode(astNode *asts.ASTNode) (IAssignable, error) {
|
||||||
lib.InternalCodingErrorIf(astNode.Type != asts.NodeType(NodeTypeFullOosvar))
|
lib.InternalCodingErrorIf(astNode.Type != asts.NodeType(NodeTypeFullOosvar))
|
||||||
lib.InternalCodingErrorIf(astNode == nil)
|
lib.InternalCodingErrorIf(astNode == nil)
|
||||||
lib.InternalCodingErrorIf(astNode.Children != nil && len(astNode.Children) > 0)
|
lib.InternalCodingErrorIf(len(astNode.Children) > 0)
|
||||||
return NewFullOosvarLvalueNode(), nil
|
return NewFullOosvarLvalueNode(), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -762,7 +767,8 @@ func (node *FullOosvarLvalueNode) UnassignIndexed(
|
||||||
if indices == nil {
|
if indices == nil {
|
||||||
state.Oosvars.Clear()
|
state.Oosvars.Clear()
|
||||||
} else {
|
} else {
|
||||||
state.Oosvars.RemoveIndexed(indices)
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = state.Oosvars.RemoveIndexed(indices)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -780,7 +786,7 @@ type LocalVariableLvalueNode struct {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (root *RootNode) BuildTypedeclLocalVariableLvalueNode(astNode *asts.ASTNode) (IAssignable, error) {
|
func (root *RootNode) BuildTypedeclLocalVariableLvalueNode(astNode *asts.ASTNode) (IAssignable, error) {
|
||||||
lib.InternalCodingErrorIf(astNode.Children == nil || len(astNode.Children) < 2)
|
lib.InternalCodingErrorIf(len(astNode.Children) < 2)
|
||||||
typeNode := astNode.Children[0]
|
typeNode := astNode.Children[0]
|
||||||
varNode := astNode.Children[1]
|
varNode := astNode.Children[1]
|
||||||
// PGPG Typedecl produces kw_int, kw_bool, etc. (no Typedecl wrapper node)
|
// PGPG Typedecl produces kw_int, kw_bool, etc. (no Typedecl wrapper node)
|
||||||
|
|
@ -803,7 +809,7 @@ func (root *RootNode) BuildLocalVariableLvalueNode(astNode *asts.ASTNode) (IAssi
|
||||||
typeName := "any"
|
typeName := "any"
|
||||||
defineTypedAtScope := false
|
defineTypedAtScope := false
|
||||||
// PGPG: LocalVariable is terminal (children nil or empty). Miller had typed params with Children[0]=Typedecl.
|
// PGPG: LocalVariable is terminal (children nil or empty). Miller had typed params with Children[0]=Typedecl.
|
||||||
if astNode.Children != nil && len(astNode.Children) > 0 { // typed, like 'num x = 3'
|
if len(astNode.Children) > 0 { // typed, like 'num x = 3'
|
||||||
typeNode := astNode.Children[0]
|
typeNode := astNode.Children[0]
|
||||||
lib.InternalCodingErrorIf(typeNode.Type != asts.NodeType(NodeTypeTypedecl))
|
lib.InternalCodingErrorIf(typeNode.Type != asts.NodeType(NodeTypeTypedecl))
|
||||||
typeName = tokenLit(typeNode)
|
typeName = tokenLit(typeNode)
|
||||||
|
|
@ -1062,7 +1068,9 @@ func (node *EnvironmentVariableLvalueNode) Assign(
|
||||||
|
|
||||||
sname := name.String()
|
sname := name.String()
|
||||||
svalue := rvalue.String()
|
svalue := rvalue.String()
|
||||||
os.Setenv(sname, svalue)
|
if err := os.Setenv(sname, svalue); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if sname == "TZ" {
|
if sname == "TZ" {
|
||||||
err := lib.SetTZFromEnv() // affects the time library; notify it
|
err := lib.SetTZFromEnv() // affects the time library; notify it
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -1093,7 +1101,7 @@ func (node *EnvironmentVariableLvalueNode) Unassign(
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
os.Unsetenv(name.String())
|
_ = os.Unsetenv(name.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
func (node *EnvironmentVariableLvalueNode) UnassignIndexed(
|
func (node *EnvironmentVariableLvalueNode) UnassignIndexed(
|
||||||
|
|
|
||||||
|
|
@ -327,7 +327,9 @@ func (root *RootNode) buildPrintxStatementNode(
|
||||||
|
|
||||||
func (node *PrintStatementNode) Execute(state *runtime.State) (*BlockExitPayload, error) {
|
func (node *PrintStatementNode) Execute(state *runtime.State) (*BlockExitPayload, error) {
|
||||||
if len(node.expressionEvaluables) == 0 {
|
if len(node.expressionEvaluables) == 0 {
|
||||||
node.printToRedirectFunc(node.terminator, state)
|
if err := node.printToRedirectFunc(node.terminator, state); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
// 5x faster than fmt.Print() separately: note that os.Stdout is
|
// 5x faster than fmt.Print() separately: note that os.Stdout is
|
||||||
// non-buffered in Go whereas stdout is buffered in C.
|
// non-buffered in Go whereas stdout is buffered in C.
|
||||||
|
|
@ -347,7 +349,9 @@ func (node *PrintStatementNode) Execute(state *runtime.State) (*BlockExitPayload
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
buffer.WriteString(node.terminator)
|
buffer.WriteString(node.terminator)
|
||||||
node.printToRedirectFunc(buffer.String(), state)
|
if err := node.printToRedirectFunc(buffer.String(), state); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return nil, nil
|
return nil, nil
|
||||||
}
|
}
|
||||||
|
|
@ -387,6 +391,5 @@ func (node *PrintStatementNode) printToFileOrPipe(
|
||||||
}
|
}
|
||||||
outputFileName := redirectorTarget.String()
|
outputFileName := redirectorTarget.String()
|
||||||
|
|
||||||
node.outputHandlerManager.WriteString(outputString, outputFileName)
|
return node.outputHandlerManager.WriteString(outputString, outputFileName)
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -368,12 +368,9 @@ func (root *RootNode) resolveFunctionCallsites() error {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if udf == nil {
|
// A nil udf here is unresolvable at CST-build time but perhaps a local
|
||||||
// Unresolvable at CST-build time but perhaps a local variable. For example,
|
// variable. For example, the UDF callsite '$z = f($x, $y)', and supposing
|
||||||
// the UDF callsite '$z = f($x, $y)', and supposing
|
// there will be 'f = func(a, b) { return a*b }' in scope at runtime.
|
||||||
// there will be 'f = func(a, b) { return a*b }' in scope at runtime.
|
|
||||||
}
|
|
||||||
|
|
||||||
unresolvedFunctionCallsite.udf = udf
|
unresolvedFunctionCallsite.udf = udf
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
|
|
|
||||||
|
|
@ -563,13 +563,13 @@ func (root *RootNode) BuildUDF(
|
||||||
for i, parameterASTNode := range flatParams {
|
for i, parameterASTNode := range flatParams {
|
||||||
// PGPG: Parameter has one child (LocalVariable), or two (Typedecl, LocalVariable) for typed params.
|
// PGPG: Parameter has one child (LocalVariable), or two (Typedecl, LocalVariable) for typed params.
|
||||||
var variableName string
|
var variableName string
|
||||||
var typeName string = "any"
|
typeName := "any"
|
||||||
if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 2 {
|
if len(parameterASTNode.Children) == 2 {
|
||||||
// Typedecl LocalVariable -> [Typedecl, LocalVariable]
|
// Typedecl LocalVariable -> [Typedecl, LocalVariable]
|
||||||
typeNode := parameterASTNode.Children[0]
|
typeNode := parameterASTNode.Children[0]
|
||||||
nameNode := parameterASTNode.Children[1]
|
nameNode := parameterASTNode.Children[1]
|
||||||
typeName = tokenLit(typeNode)
|
typeName = tokenLit(typeNode)
|
||||||
if typeName == "" && typeNode.Children != nil && len(typeNode.Children) > 0 {
|
if typeName == "" && len(typeNode.Children) > 0 {
|
||||||
typeName = tokenLit(typeNode.Children[0])
|
typeName = tokenLit(typeNode.Children[0])
|
||||||
}
|
}
|
||||||
if typeName == "" {
|
if typeName == "" {
|
||||||
|
|
@ -579,14 +579,14 @@ func (root *RootNode) BuildUDF(
|
||||||
if string(nameNode.Type) == NodeTypeLocalVariable || nameNode.Type == asts.NodeType(NodeTypeLocalVariable) {
|
if string(nameNode.Type) == NodeTypeLocalVariable || nameNode.Type == asts.NodeType(NodeTypeLocalVariable) {
|
||||||
variableName = tokenLit(nameNode)
|
variableName = tokenLit(nameNode)
|
||||||
}
|
}
|
||||||
} else if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 1 {
|
} else if len(parameterASTNode.Children) == 1 {
|
||||||
typeGatedParameterNameASTNode := parameterASTNode.Children[0]
|
typeGatedParameterNameASTNode := parameterASTNode.Children[0]
|
||||||
if string(typeGatedParameterNameASTNode.Type) != NodeTypeLocalVariable &&
|
if string(typeGatedParameterNameASTNode.Type) != NodeTypeLocalVariable &&
|
||||||
typeGatedParameterNameASTNode.Type != asts.NodeType(NodeTypeLocalVariable) {
|
typeGatedParameterNameASTNode.Type != asts.NodeType(NodeTypeLocalVariable) {
|
||||||
lib.InternalCodingErrorWithMessageIf(true, "expected LocalVariable as parameter name")
|
lib.InternalCodingErrorWithMessageIf(true, "expected LocalVariable as parameter name")
|
||||||
}
|
}
|
||||||
variableName = tokenLit(typeGatedParameterNameASTNode)
|
variableName = tokenLit(typeGatedParameterNameASTNode)
|
||||||
} else if parameterASTNode.Children == nil || len(parameterASTNode.Children) == 0 {
|
} else if len(parameterASTNode.Children) == 0 {
|
||||||
// Direct LocalVariable (e.g. from with_prepended_children flattening)
|
// Direct LocalVariable (e.g. from with_prepended_children flattening)
|
||||||
if string(parameterASTNode.Type) != NodeTypeLocalVariable &&
|
if string(parameterASTNode.Type) != NodeTypeLocalVariable &&
|
||||||
parameterASTNode.Type != asts.NodeType(NodeTypeLocalVariable) {
|
parameterASTNode.Type != asts.NodeType(NodeTypeLocalVariable) {
|
||||||
|
|
|
||||||
|
|
@ -254,13 +254,13 @@ func (root *RootNode) BuildAndInstallUDS(astNode *asts.ASTNode) error {
|
||||||
typeGatedParameterNames := make([]*types.TypeGatedMlrvalName, arity)
|
typeGatedParameterNames := make([]*types.TypeGatedMlrvalName, arity)
|
||||||
for i, parameterASTNode := range flatParams {
|
for i, parameterASTNode := range flatParams {
|
||||||
var variableName string
|
var variableName string
|
||||||
var typeName string = "any"
|
typeName := "any"
|
||||||
if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 2 {
|
if len(parameterASTNode.Children) == 2 {
|
||||||
// Typedecl LocalVariable -> [Typedecl, LocalVariable]
|
// Typedecl LocalVariable -> [Typedecl, LocalVariable]
|
||||||
typeNode := parameterASTNode.Children[0]
|
typeNode := parameterASTNode.Children[0]
|
||||||
nameNode := parameterASTNode.Children[1]
|
nameNode := parameterASTNode.Children[1]
|
||||||
typeName = tokenLit(typeNode)
|
typeName = tokenLit(typeNode)
|
||||||
if typeName == "" && typeNode.Children != nil && len(typeNode.Children) > 0 {
|
if typeName == "" && len(typeNode.Children) > 0 {
|
||||||
typeName = tokenLit(typeNode.Children[0])
|
typeName = tokenLit(typeNode.Children[0])
|
||||||
}
|
}
|
||||||
if typeName == "" {
|
if typeName == "" {
|
||||||
|
|
@ -269,7 +269,7 @@ func (root *RootNode) BuildAndInstallUDS(astNode *asts.ASTNode) error {
|
||||||
if string(nameNode.Type) == NodeTypeLocalVariable || nameNode.Type == asts.NodeType(NodeTypeLocalVariable) {
|
if string(nameNode.Type) == NodeTypeLocalVariable || nameNode.Type == asts.NodeType(NodeTypeLocalVariable) {
|
||||||
variableName = tokenLit(nameNode)
|
variableName = tokenLit(nameNode)
|
||||||
}
|
}
|
||||||
} else if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 1 {
|
} else if len(parameterASTNode.Children) == 1 {
|
||||||
typeGatedParameterNameASTNode := parameterASTNode.Children[0]
|
typeGatedParameterNameASTNode := parameterASTNode.Children[0]
|
||||||
lib.InternalCodingErrorIf(typeGatedParameterNameASTNode.Type != asts.NodeType(NodeTypeLocalVariable))
|
lib.InternalCodingErrorIf(typeGatedParameterNameASTNode.Type != asts.NodeType(NodeTypeLocalVariable))
|
||||||
variableName = tokenLit(typeGatedParameterNameASTNode)
|
variableName = tokenLit(typeGatedParameterNameASTNode)
|
||||||
|
|
|
||||||
|
|
@ -163,14 +163,14 @@ func processFileInPlace(
|
||||||
// Get a handle with, perhaps, a recompression wrapper around it.
|
// Get a handle with, perhaps, a recompression wrapper around it.
|
||||||
wrappedHandle, isNew, err := lib.WrapOutputHandle(handle, inputFileEncoding)
|
wrappedHandle, isNew, err := lib.WrapOutputHandle(handle, inputFileEncoding)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
os.Remove(tempFileName)
|
_ = os.Remove(tempFileName)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Run the Miller processing stream from the input file to the temp-output file.
|
// Run the Miller processing stream from the input file to the temp-output file.
|
||||||
err = stream.Stream([]string{fileName}, options, recordTransformers, wrappedHandle, false)
|
err = stream.Stream([]string{fileName}, options, recordTransformers, wrappedHandle, false)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
os.Remove(tempFileName)
|
_ = os.Remove(tempFileName)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -178,7 +178,7 @@ func processFileInPlace(
|
||||||
if isNew {
|
if isNew {
|
||||||
err = wrappedHandle.Close()
|
err = wrappedHandle.Close()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
os.Remove(tempFileName)
|
_ = os.Remove(tempFileName)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -187,14 +187,14 @@ func processFileInPlace(
|
||||||
// it must be error-checked.
|
// it must be error-checked.
|
||||||
err = handle.Close()
|
err = handle.Close()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
os.Remove(tempFileName)
|
_ = os.Remove(tempFileName)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Rename the temp-output file on top of the input file.
|
// Rename the temp-output file on top of the input file.
|
||||||
err = os.Rename(tempFileName, fileName)
|
err = os.Rename(tempFileName, fileName)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
os.Remove(tempFileName)
|
_ = os.Remove(tempFileName)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -82,7 +82,7 @@ func (reader *RecordReaderCSV) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -335,9 +335,8 @@ func (reader *RecordReaderCSV) maybeConsumeComment(
|
||||||
lib.InternalCodingErrorIf(len(csvRecord) != 1)
|
lib.InternalCodingErrorIf(len(csvRecord) != 1)
|
||||||
*recordsAndContexts = append(*recordsAndContexts, types.NewOutputString(csvRecord[0], context))
|
*recordsAndContexts = append(*recordsAndContexts, types.NewOutputString(csvRecord[0], context))
|
||||||
|
|
||||||
} else /* reader.readerOptions.CommentHandling == cli.SkipComments */ {
|
|
||||||
// discard entirely
|
|
||||||
}
|
}
|
||||||
|
// else, reader.readerOptions.CommentHandling == cli.SkipComments: discard entirely
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -122,7 +122,7 @@ func (reader *RecordReaderCSVLite) Read(
|
||||||
errorChannel,
|
errorChannel,
|
||||||
downstreamDoneChannel,
|
downstreamDoneChannel,
|
||||||
)
|
)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -76,7 +76,7 @@ func (reader *RecordReaderDCF) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -86,7 +86,7 @@ func (reader *RecordReaderDKVPNIDX) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -216,7 +216,7 @@ func recordFromNIDXLine(reader *RecordReaderDKVPNIDX, line string) (*mlrval.Mlrm
|
||||||
|
|
||||||
values := reader.fieldSplitter.Split(line)
|
values := reader.fieldSplitter.Split(line)
|
||||||
|
|
||||||
var i int = 0
|
i := 0
|
||||||
for _, value := range values {
|
for _, value := range values {
|
||||||
i++
|
i++
|
||||||
str_key := strconv.Itoa(i)
|
str_key := strconv.Itoa(i)
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,8 @@ import (
|
||||||
|
|
||||||
func TestRecordFromDKVPLine(t *testing.T) {
|
func TestRecordFromDKVPLine(t *testing.T) {
|
||||||
readerOptions := cli.DefaultReaderOptions()
|
readerOptions := cli.DefaultReaderOptions()
|
||||||
cli.FinalizeReaderOptions(&readerOptions) // compute IPS, IFS -> IPSRegex, IFSRegex
|
err := cli.FinalizeReaderOptions(&readerOptions) // compute IPS, IFS -> IPSRegex, IFSRegex
|
||||||
|
assert.Nil(t, err)
|
||||||
reader, err := NewRecordReaderDKVP(&readerOptions, 1)
|
reader, err := NewRecordReaderDKVP(&readerOptions, 1)
|
||||||
assert.NotNil(t, reader)
|
assert.NotNil(t, reader)
|
||||||
assert.Nil(t, err)
|
assert.Nil(t, err)
|
||||||
|
|
|
||||||
|
|
@ -62,7 +62,7 @@ func (reader *RecordReaderDKVPX) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -60,7 +60,7 @@ func (reader *RecordReaderJSON) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -122,7 +122,7 @@ func (reader *RecordReaderPprintFixedSplit) Read(
|
||||||
errorChannel,
|
errorChannel,
|
||||||
downstreamDoneChannel,
|
downstreamDoneChannel,
|
||||||
)
|
)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -351,7 +351,7 @@ func (reader *RecordReaderPprintBarredOrMarkdown) Read(
|
||||||
errorChannel,
|
errorChannel,
|
||||||
downstreamDoneChannel,
|
downstreamDoneChannel,
|
||||||
)
|
)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -104,7 +104,7 @@ func (reader *RecordReaderTSV) Read(
|
||||||
errorChannel,
|
errorChannel,
|
||||||
downstreamDoneChannel,
|
downstreamDoneChannel,
|
||||||
)
|
)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -88,7 +88,7 @@ func (reader *RecordReaderXTAB) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -58,7 +58,7 @@ func (reader *RecordReaderYAML) Read(
|
||||||
errorChannel <- err
|
errorChannel <- err
|
||||||
} else {
|
} else {
|
||||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||||
handle.Close()
|
_ = handle.Close()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -40,7 +40,7 @@ func OpenOutboundHalfPipe(commandString string) (*os.File, error) {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
go process.Wait()
|
go func() { _, _ = process.Wait() }()
|
||||||
|
|
||||||
return writePipe, nil
|
return writePipe, nil
|
||||||
}
|
}
|
||||||
|
|
@ -87,7 +87,7 @@ func OpenInboundHalfPipe(commandString string) (*os.File, error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "mlr: %v\n", err)
|
fmt.Fprintf(os.Stderr, "mlr: %v\n", err)
|
||||||
}
|
}
|
||||||
readPipe.Close()
|
_ = readPipe.Close()
|
||||||
}(process, readPipe)
|
}(process, readPipe)
|
||||||
|
|
||||||
return readPipe, nil
|
return readPipe, nil
|
||||||
|
|
|
||||||
|
|
@ -51,7 +51,7 @@ func LoadStringsFromDir(dirname string, extension string) ([]string, error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
defer f.Close()
|
defer func() { _ = f.Close() }()
|
||||||
|
|
||||||
names, err := f.Readdirnames(-1)
|
names, err := f.Readdirnames(-1)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -80,7 +80,7 @@ func ReadCSVHeader(filename string) ([]string, error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
defer handle.Close()
|
defer func() { _ = handle.Close() }()
|
||||||
csvReader := csv.NewReader(handle)
|
csvReader := csv.NewReader(handle)
|
||||||
header, err := csvReader.Read()
|
header, err := csvReader.Read()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
|
||||||
|
|
@ -755,10 +755,7 @@ func (mlrmap *Mlrmap) Label(newNames []string) {
|
||||||
|
|
||||||
i := 0
|
i := 0
|
||||||
numNewNames := len(newNames)
|
numNewNames := len(newNames)
|
||||||
for {
|
for i < numNewNames {
|
||||||
if i >= numNewNames {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
pe := mlrmap.pop()
|
pe := mlrmap.pop()
|
||||||
if pe == nil {
|
if pe == nil {
|
||||||
break
|
break
|
||||||
|
|
|
||||||
|
|
@ -179,7 +179,8 @@ func (mlrmap *Mlrmap) CopyUnflattened(
|
||||||
baseIndex := arrayval[0].String()
|
baseIndex := arrayval[0].String()
|
||||||
affectedBaseIndices[baseIndex] = true
|
affectedBaseIndices[baseIndex] = true
|
||||||
// Use PutIndexed to assign $x["a"] = 7, or $x["b"] = 8, etc.
|
// Use PutIndexed to assign $x["a"] = 7, or $x["b"] = 8, etc.
|
||||||
other.PutIndexed(
|
// Unflattening is best-effort: this API has no error return.
|
||||||
|
_ = other.PutIndexed(
|
||||||
CopyMlrvalArray(arrayval),
|
CopyMlrvalArray(arrayval),
|
||||||
unflattenTerminal(pe.Value).Copy(),
|
unflattenTerminal(pe.Value).Copy(),
|
||||||
)
|
)
|
||||||
|
|
@ -226,7 +227,8 @@ func (mlrmap *Mlrmap) CopyUnflattenFields(
|
||||||
baseIndex := arrayval[0].String()
|
baseIndex := arrayval[0].String()
|
||||||
if fieldNameSet[baseIndex] {
|
if fieldNameSet[baseIndex] {
|
||||||
// Use PutIndexed to assign $x["a"] = 7, or $x["b"] = 8, etc.
|
// Use PutIndexed to assign $x["a"] = 7, or $x["b"] = 8, etc.
|
||||||
other.PutIndexed(
|
// Unflattening is best-effort: this API has no error return.
|
||||||
|
_ = other.PutIndexed(
|
||||||
CopyMlrvalArray(arrayval),
|
CopyMlrvalArray(arrayval),
|
||||||
unflattenTerminal(pe.Value).Copy(),
|
unflattenTerminal(pe.Value).Copy(),
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -197,14 +197,12 @@ func mlrvalToYAMLNode(mv *Mlrval) (*yaml.Node, error) {
|
||||||
case MT_ARRAY:
|
case MT_ARRAY:
|
||||||
arr := mv.GetArray()
|
arr := mv.GetArray()
|
||||||
seqNode := &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
seqNode := &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||||
if arr != nil {
|
for _, elem := range arr {
|
||||||
for _, elem := range arr {
|
v, err := mlrvalToYAMLNode(elem)
|
||||||
v, err := mlrvalToYAMLNode(elem)
|
if err != nil {
|
||||||
if err != nil {
|
return nil, err
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
seqNode.Content = append(seqNode.Content, v)
|
|
||||||
}
|
}
|
||||||
|
seqNode.Content = append(seqNode.Content, v)
|
||||||
}
|
}
|
||||||
return seqNode, nil
|
return seqNode, nil
|
||||||
case MT_MAP:
|
case MT_MAP:
|
||||||
|
|
|
||||||
|
|
@ -105,7 +105,8 @@ func channelWriterHandleBatch(
|
||||||
}
|
}
|
||||||
|
|
||||||
if writerOptions.FlushOnEveryRecord {
|
if writerOptions.FlushOnEveryRecord {
|
||||||
bufferedOutputStream.Flush()
|
// bufio.Writer errors are sticky; the final Flush in pkg/stream is checked
|
||||||
|
_ = bufferedOutputStream.Flush()
|
||||||
}
|
}
|
||||||
|
|
||||||
} else {
|
} else {
|
||||||
|
|
|
||||||
|
|
@ -500,7 +500,9 @@ func (handler *FileOutputHandler) Close() (retval error) {
|
||||||
return retval
|
return retval
|
||||||
}
|
}
|
||||||
|
|
||||||
handler.bufferedOutputStream.Flush()
|
if err := handler.bufferedOutputStream.Flush(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if handler.closeable {
|
if handler.closeable {
|
||||||
return handler.handle.Close()
|
return handler.handle.Close()
|
||||||
} // e.g. stdout
|
} // e.g. stdout
|
||||||
|
|
|
||||||
|
|
@ -73,11 +73,14 @@ func (writer *RecordWriterCSV) Write(
|
||||||
fields[i] = pe.Key
|
fields[i] = pe.Key
|
||||||
i++
|
i++
|
||||||
}
|
}
|
||||||
writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, true, writer.quoteAll)
|
err := writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, true, writer.quoteAll)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
writer.needToPrintHeader = false
|
writer.needToPrintHeader = false
|
||||||
}
|
}
|
||||||
|
|
||||||
var outputNF int64 = outrec.FieldCount
|
outputNF := outrec.FieldCount
|
||||||
if outputNF < writer.firstRecordNF {
|
if outputNF < writer.firstRecordNF {
|
||||||
outputNF = writer.firstRecordNF
|
outputNF = writer.firstRecordNF
|
||||||
}
|
}
|
||||||
|
|
@ -103,7 +106,5 @@ func (writer *RecordWriterCSV) Write(
|
||||||
fields[i] = ""
|
fields[i] = ""
|
||||||
}
|
}
|
||||||
|
|
||||||
writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, false, writer.quoteAll)
|
return writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, false, writer.quoteAll)
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,6 @@ package output
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
"fmt"
|
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/johnkerl/miller/v6/pkg/cli"
|
"github.com/johnkerl/miller/v6/pkg/cli"
|
||||||
|
|
@ -211,7 +210,8 @@ func (writer *RecordWriterPPRINT) writeHeterogenousListNonBarred(
|
||||||
}
|
}
|
||||||
|
|
||||||
if writer.writerOptions.FlushOnEveryRecord {
|
if writer.writerOptions.FlushOnEveryRecord {
|
||||||
bufferedOutputStream.Flush()
|
// bufio.Writer errors are sticky; the final Flush in pkg/stream is checked
|
||||||
|
_ = bufferedOutputStream.Flush()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -351,7 +351,7 @@ func (writer *RecordWriterPPRINT) writeHeterogenousListBarred(
|
||||||
bufferedOutputStream.WriteString(colorizer.MaybeColorizeValue(s, outputIsStdout))
|
bufferedOutputStream.WriteString(colorizer.MaybeColorizeValue(s, outputIsStdout))
|
||||||
}
|
}
|
||||||
if pe.Next != nil {
|
if pe.Next != nil {
|
||||||
bufferedOutputStream.WriteString(fmt.Sprint(bc.verticalMiddle))
|
bufferedOutputStream.WriteString(bc.verticalMiddle)
|
||||||
} else {
|
} else {
|
||||||
bufferedOutputStream.WriteString(bc.verticalEnd)
|
bufferedOutputStream.WriteString(bc.verticalEnd)
|
||||||
bufferedOutputStream.WriteString(writer.writerOptions.ORS)
|
bufferedOutputStream.WriteString(writer.writerOptions.ORS)
|
||||||
|
|
@ -373,7 +373,8 @@ func (writer *RecordWriterPPRINT) writeHeterogenousListBarred(
|
||||||
}
|
}
|
||||||
|
|
||||||
if writer.writerOptions.FlushOnEveryRecord {
|
if writer.writerOptions.FlushOnEveryRecord {
|
||||||
bufferedOutputStream.Flush()
|
// bufio.Writer errors are sticky; the final Flush in pkg/stream is checked
|
||||||
|
_ = bufferedOutputStream.Flush()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -75,7 +75,7 @@ func (writer *RecordWriterTSV) Write(
|
||||||
writer.needToPrintHeader = false
|
writer.needToPrintHeader = false
|
||||||
}
|
}
|
||||||
|
|
||||||
var outputNF int64 = outrec.FieldCount
|
outputNF := outrec.FieldCount
|
||||||
if outputNF < writer.firstRecordNF {
|
if outputNF < writer.firstRecordNF {
|
||||||
outputNF = writer.firstRecordNF
|
outputNF = writer.firstRecordNF
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -454,7 +454,9 @@ func (frame *StackFrame) setIndexed(
|
||||||
leadingIndex := indices[0]
|
leadingIndex := indices[0]
|
||||||
if leadingIndex.IsString() || leadingIndex.IsInt() {
|
if leadingIndex.IsString() || leadingIndex.IsInt() {
|
||||||
newval := mlrval.FromMap(mlrval.NewMlrmap())
|
newval := mlrval.FromMap(mlrval.NewMlrmap())
|
||||||
newval.PutIndexed(indices, mv)
|
if err := newval.PutIndexed(indices, mv); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
return frame.set(stackVariable, newval)
|
return frame.set(stackVariable, newval)
|
||||||
}
|
}
|
||||||
return fmt.Errorf(
|
return fmt.Errorf(
|
||||||
|
|
@ -485,5 +487,6 @@ func (frame *StackFrame) unsetIndexed(
|
||||||
if value == nil {
|
if value == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
value.RemoveIndexed(indices)
|
// unset of a non-existent path is a no-op
|
||||||
|
_ = value.RemoveIndexed(indices)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -101,7 +101,9 @@ func Stream(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
bufferedOutputStream.Flush()
|
if err := bufferedOutputStream.Flush(); err != nil && retval == nil {
|
||||||
|
retval = err
|
||||||
|
}
|
||||||
|
|
||||||
return retval
|
return retval
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -821,12 +821,8 @@ func helpByExactSearch(things []string) bool {
|
||||||
// We need to look various places, e.g. "sec2gmt" is the name of a verb as well
|
// We need to look various places, e.g. "sec2gmt" is the name of a verb as well
|
||||||
// as a DSL function.
|
// as a DSL function.
|
||||||
func helpByExactSearchOne(thing string) bool {
|
func helpByExactSearchOne(thing string) bool {
|
||||||
found := false
|
|
||||||
|
|
||||||
// flag
|
// flag
|
||||||
if cli.FLAG_TABLE.ShowHelpForFlagWithName(thing) {
|
found := cli.FLAG_TABLE.ShowHelpForFlagWithName(thing)
|
||||||
found = true
|
|
||||||
}
|
|
||||||
|
|
||||||
// verb
|
// verb
|
||||||
if transformers.ShowHelpForTransformer(thing) {
|
if transformers.ShowHelpForTransformer(thing) {
|
||||||
|
|
@ -857,12 +853,8 @@ func helpByApproximateSearch(things []string) bool {
|
||||||
}
|
}
|
||||||
|
|
||||||
func helpByApproximateSearchOne(thing string) bool {
|
func helpByApproximateSearchOne(thing string) bool {
|
||||||
found := false
|
|
||||||
|
|
||||||
// flag
|
// flag
|
||||||
if cli.FLAG_TABLE.ShowHelpForFlagApproximateWithName(thing) {
|
found := cli.FLAG_TABLE.ShowHelpForFlagApproximateWithName(thing)
|
||||||
found = true
|
|
||||||
}
|
|
||||||
|
|
||||||
// verb
|
// verb
|
||||||
if transformers.ShowHelpForTransformerApproximate(thing) {
|
if transformers.ShowHelpForTransformerApproximate(thing) {
|
||||||
|
|
|
||||||
|
|
@ -164,7 +164,8 @@ var whichStopwords = map[string]bool{
|
||||||
// single-character tokens and stopwords.
|
// single-character tokens and stopwords.
|
||||||
func whichTokenize(query string) []string {
|
func whichTokenize(query string) []string {
|
||||||
words := strings.FieldsFunc(strings.ToLower(query), func(r rune) bool {
|
words := strings.FieldsFunc(strings.ToLower(query), func(r rune) bool {
|
||||||
return !('a' <= r && r <= 'z') && !('0' <= r && r <= '9') && r != '-' && r != '_'
|
isWordRune := ('a' <= r && r <= 'z') || ('0' <= r && r <= '9') || r == '-' || r == '_'
|
||||||
|
return !isWordRune
|
||||||
})
|
})
|
||||||
var tokens []string
|
var tokens []string
|
||||||
seen := map[string]bool{}
|
seen := map[string]bool{}
|
||||||
|
|
|
||||||
|
|
@ -72,8 +72,8 @@ func RunDiffCommandOnStrings(
|
||||||
) {
|
) {
|
||||||
actualOutputFileName := lib.WriteTempFileOrDie(actualOutput)
|
actualOutputFileName := lib.WriteTempFileOrDie(actualOutput)
|
||||||
expectedOutputFileName := lib.WriteTempFileOrDie(expectedOutput)
|
expectedOutputFileName := lib.WriteTempFileOrDie(expectedOutput)
|
||||||
defer os.Remove(actualOutputFileName)
|
defer func() { _ = os.Remove(actualOutputFileName) }()
|
||||||
defer os.Remove(expectedOutputFileName)
|
defer func() { _ = os.Remove(expectedOutputFileName) }()
|
||||||
|
|
||||||
// This is diff or fc
|
// This is diff or fc
|
||||||
diffRunArray := platform.GetDiffRunArray(actualOutputFileName, expectedOutputFileName)
|
diffRunArray := platform.GetDiffRunArray(actualOutputFileName, expectedOutputFileName)
|
||||||
|
|
|
||||||
|
|
@ -149,16 +149,16 @@ func (rt *RegTester) Execute(
|
||||||
) bool {
|
) bool {
|
||||||
// Don't let the current user's settings affect expected results
|
// Don't let the current user's settings affect expected results
|
||||||
for _, name := range envVarsToUnset {
|
for _, name := range envVarsToUnset {
|
||||||
os.Unsetenv(name)
|
_ = os.Unsetenv(name)
|
||||||
}
|
}
|
||||||
// If there is an accessible .mlrrc file, we don't want it to be read for the regression test.
|
// If there is an accessible .mlrrc file, we don't want it to be read for the regression test.
|
||||||
os.Setenv("MLRRC", "__none__")
|
_ = os.Setenv("MLRRC", "__none__")
|
||||||
|
|
||||||
// This is important for multi-platform regression testing, wherein default floating-point
|
// This is important for multi-platform regression testing, wherein default floating-point
|
||||||
// output format has varying numbers of decimal places between the platform where
|
// output format has varying numbers of decimal places between the platform where
|
||||||
// the expected results were generated, and the platform where the actual values are being
|
// the expected results were generated, and the platform where the actual values are being
|
||||||
// computed. For regression-test we OFMT from an environment variable.
|
// computed. For regression-test we OFMT from an environment variable.
|
||||||
os.Setenv("MLR_OFMT", "%.8f")
|
_ = os.Setenv("MLR_OFMT", "%.8f")
|
||||||
|
|
||||||
rt.resetCounts()
|
rt.resetCounts()
|
||||||
|
|
||||||
|
|
@ -328,7 +328,7 @@ func (rt *RegTester) hasCaseSubdirectories(
|
||||||
fmt.Printf("%s: %v\n", dirName, err)
|
fmt.Printf("%s: %v\n", dirName, err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
defer f.Close()
|
defer func() { _ = f.Close() }()
|
||||||
|
|
||||||
names, err := f.Readdirnames(-1)
|
names, err := f.Readdirnames(-1)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -461,12 +461,12 @@ func (rt *RegTester) executeSingleCmdFile(
|
||||||
if verbosityLevel >= 3 {
|
if verbosityLevel >= 3 {
|
||||||
fmt.Printf("SETENV %s=%s\n", key, value)
|
fmt.Printf("SETENV %s=%s\n", key, value)
|
||||||
}
|
}
|
||||||
os.Setenv(key, value)
|
_ = os.Setenv(key, value)
|
||||||
}
|
}
|
||||||
// This is so 'mlr' files can find the case-directory if they need it --
|
// This is so 'mlr' files can find the case-directory if they need it --
|
||||||
// typically, for redirected emit/dump/etc statements where we want
|
// typically, for redirected emit/dump/etc statements where we want
|
||||||
// the redirected-to files to be written into the case-directory.
|
// the redirected-to files to be written into the case-directory.
|
||||||
os.Setenv("CASEDIR", caseDir)
|
_ = os.Setenv("CASEDIR", caseDir)
|
||||||
|
|
||||||
// Copy any files requested by the test. (Most don't; some do, e.g. those
|
// Copy any files requested by the test. (Most don't; some do, e.g. those
|
||||||
// which test the write-in-place logic of mlr -I.)
|
// which test the write-in-place logic of mlr -I.)
|
||||||
|
|
@ -496,9 +496,9 @@ func (rt *RegTester) executeSingleCmdFile(
|
||||||
if verbosityLevel >= 3 {
|
if verbosityLevel >= 3 {
|
||||||
fmt.Printf("UNSETENV %s\n", key)
|
fmt.Printf("UNSETENV %s\n", key)
|
||||||
}
|
}
|
||||||
os.Setenv(key, "")
|
_ = os.Setenv(key, "")
|
||||||
}
|
}
|
||||||
os.Setenv("CASEDIR", "")
|
_ = os.Setenv("CASEDIR", "")
|
||||||
|
|
||||||
// The .postcmp needn't exist (most test cases don't have one) in which case
|
// The .postcmp needn't exist (most test cases don't have one) in which case
|
||||||
// the returned map will be empty.
|
// the returned map will be empty.
|
||||||
|
|
@ -542,7 +542,7 @@ func (rt *RegTester) executeSingleCmdFile(
|
||||||
// Write the .should-fail file
|
// Write the .should-fail file
|
||||||
if actualExitCode == 0 {
|
if actualExitCode == 0 {
|
||||||
// Remove it, if it exists.
|
// Remove it, if it exists.
|
||||||
os.Remove(expectFailFileName)
|
_ = os.Remove(expectFailFileName)
|
||||||
} else {
|
} else {
|
||||||
err = rt.storeFile(expectFailFileName, "")
|
err = rt.storeFile(expectFailFileName, "")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -716,7 +716,7 @@ func (rt *RegTester) executeSingleCmdFile(
|
||||||
// Clean up any requested file-copies so that we're git-clean after the regression-test run.
|
// Clean up any requested file-copies so that we're git-clean after the regression-test run.
|
||||||
for _, pair := range preCopySrcDestPairs {
|
for _, pair := range preCopySrcDestPairs {
|
||||||
dst := pair.second
|
dst := pair.second
|
||||||
os.Remove(dst)
|
_ = os.Remove(dst)
|
||||||
if verbosityLevel >= 3 {
|
if verbosityLevel >= 3 {
|
||||||
fmt.Printf("%s: clean up %s\n", cmdFilePath, dst)
|
fmt.Printf("%s: clean up %s\n", cmdFilePath, dst)
|
||||||
}
|
}
|
||||||
|
|
@ -725,7 +725,7 @@ func (rt *RegTester) executeSingleCmdFile(
|
||||||
// Clean up any extra output files so that we're git-clean after the regression-test run.
|
// Clean up any extra output files so that we're git-clean after the regression-test run.
|
||||||
for _, pair := range postCompareExpectedActualPairs {
|
for _, pair := range postCompareExpectedActualPairs {
|
||||||
actualFileName := pair.second
|
actualFileName := pair.second
|
||||||
os.Remove(actualFileName)
|
_ = os.Remove(actualFileName)
|
||||||
if verbosityLevel >= 3 {
|
if verbosityLevel >= 3 {
|
||||||
fmt.Printf("%s: clean up %s\n", cmdFilePath, actualFileName)
|
fmt.Printf("%s: clean up %s\n", cmdFilePath, actualFileName)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -147,8 +147,14 @@ func ReplMain(args []string) int {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
cli.FinalizeReaderOptions(&options.ReaderOptions)
|
if err := cli.FinalizeReaderOptions(&options.ReaderOptions); err != nil {
|
||||||
cli.FinalizeWriterOptions(&options.WriterOptions)
|
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", replName, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if err := cli.FinalizeWriterOptions(&options.WriterOptions); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", replName, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
// --auto-flatten is on by default. But if input and output formats are both JSON,
|
// --auto-flatten is on by default. But if input and output formats are both JSON,
|
||||||
// then we don't need to actually do anything. See also mlrcli_parse.go.
|
// then we don't need to actually do anything. See also mlrcli_parse.go.
|
||||||
|
|
@ -186,7 +192,11 @@ func ReplMain(args []string) int {
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
|
||||||
repl.bufferedRecordOutputStream.Flush()
|
err = repl.bufferedRecordOutputStream.Flush()
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", repl.replName, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
err = repl.closeBufferedOutputStream()
|
err = repl.closeBufferedOutputStream()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", repl.replName, err)
|
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", repl.replName, err)
|
||||||
|
|
|
||||||
|
|
@ -200,9 +200,7 @@ func handleOpen(repl *Repl, args []string) bool {
|
||||||
// something instead of waiting to show them an error only when they type
|
// something instead of waiting to show them an error only when they type
|
||||||
// ':read'.
|
// ':read'.
|
||||||
func openFilesPreCheck(repl *Repl, args []string) bool {
|
func openFilesPreCheck(repl *Repl, args []string) bool {
|
||||||
if len(args) == 0 {
|
// Zero file names is stdin, which is readable.
|
||||||
// Zero file names is stdin, which is readable
|
|
||||||
}
|
|
||||||
for _, arg := range args {
|
for _, arg := range args {
|
||||||
fileInfo, err := os.Stat(arg)
|
fileInfo, err := os.Stat(arg)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -554,8 +552,9 @@ func skipOrProcessRecord(
|
||||||
// Strings to be printed from put/filter DSL print/dump/etc statements.
|
// Strings to be printed from put/filter DSL print/dump/etc statements.
|
||||||
if recordAndContext.Record == nil {
|
if recordAndContext.Record == nil {
|
||||||
if processingNotSkipping {
|
if processingNotSkipping {
|
||||||
repl.bufferedRecordOutputStream.WriteString(recordAndContext.OutputString)
|
// Interactive terminal output: nothing useful to do on write failure
|
||||||
repl.bufferedRecordOutputStream.Flush()
|
_, _ = repl.bufferedRecordOutputStream.WriteString(recordAndContext.OutputString)
|
||||||
|
_ = repl.bufferedRecordOutputStream.Flush()
|
||||||
}
|
}
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
@ -617,8 +616,12 @@ func writeRecord(repl *Repl, outrec *mlrval.Mlrmap) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// Write and flush immediately for REPL output.
|
// Write and flush immediately for REPL output.
|
||||||
repl.recordWriter.Write(outrec, nil, repl.bufferedRecordOutputStream, true /*outputIsStdout*/)
|
err := repl.recordWriter.Write(outrec, nil, repl.bufferedRecordOutputStream, true /*outputIsStdout*/)
|
||||||
repl.bufferedRecordOutputStream.Flush()
|
if err != nil {
|
||||||
|
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_ = repl.bufferedRecordOutputStream.Flush()
|
||||||
}
|
}
|
||||||
|
|
||||||
func usageReadWrite(repl *Repl) {
|
func usageReadWrite(repl *Repl) {
|
||||||
|
|
@ -642,7 +645,9 @@ func usageRedirectWrite(repl *Repl) {
|
||||||
func handleRedirectWrite(repl *Repl, args []string) bool {
|
func handleRedirectWrite(repl *Repl, args []string) bool {
|
||||||
args = args[1:] // strip off verb
|
args = args[1:] // strip off verb
|
||||||
if len(args) == 0 {
|
if len(args) == 0 {
|
||||||
repl.closeBufferedOutputStream()
|
if err := repl.closeBufferedOutputStream(); err != nil {
|
||||||
|
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||||
|
}
|
||||||
repl.setBufferedOutputStream("(stdout)", os.Stdout)
|
repl.setBufferedOutputStream("(stdout)", os.Stdout)
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
@ -665,7 +670,9 @@ func handleRedirectWrite(repl *Repl, args []string) bool {
|
||||||
}
|
}
|
||||||
fmt.Printf("Redirecting record output to \"%s\"\n", filename)
|
fmt.Printf("Redirecting record output to \"%s\"\n", filename)
|
||||||
|
|
||||||
repl.closeBufferedOutputStream()
|
if err := repl.closeBufferedOutputStream(); err != nil {
|
||||||
|
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||||
|
}
|
||||||
repl.setBufferedOutputStream(filename, handle)
|
repl.setBufferedOutputStream(filename, handle)
|
||||||
|
|
||||||
return true
|
return true
|
||||||
|
|
@ -695,7 +702,9 @@ func handleRedirectAppend(repl *Repl, args []string) bool {
|
||||||
}
|
}
|
||||||
fmt.Printf("Redirecting record output to \"%s\"\n", filename)
|
fmt.Printf("Redirecting record output to \"%s\"\n", filename)
|
||||||
|
|
||||||
repl.closeBufferedOutputStream()
|
if err := repl.closeBufferedOutputStream(); err != nil {
|
||||||
|
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||||
|
}
|
||||||
repl.setBufferedOutputStream(filename, handle)
|
repl.setBufferedOutputStream(filename, handle)
|
||||||
|
|
||||||
return true
|
return true
|
||||||
|
|
@ -874,11 +883,7 @@ func handleHelp(repl *Repl, args []string) bool {
|
||||||
}
|
}
|
||||||
|
|
||||||
func handleHelpFindSingle(repl *Repl, arg string) {
|
func handleHelpFindSingle(repl *Repl, arg string) {
|
||||||
foundAny := false
|
foundAny := cst.TryUsageForKeywordApproximate(arg)
|
||||||
|
|
||||||
if cst.TryUsageForKeywordApproximate(arg) {
|
|
||||||
foundAny = true
|
|
||||||
}
|
|
||||||
|
|
||||||
if cst.BuiltinFunctionManagerInstance.TryListBuiltinFunctionUsageApproximate(arg) {
|
if cst.BuiltinFunctionManagerInstance.TryListBuiltinFunctionUsageApproximate(arg) {
|
||||||
foundAny = true
|
foundAny = true
|
||||||
|
|
|
||||||
|
|
@ -123,8 +123,14 @@ func ScriptMain(args []string) int {
|
||||||
scriptUsage(scriptName, os.Stderr, 1)
|
scriptUsage(scriptName, os.Stderr, 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
cli.FinalizeReaderOptions(&options.ReaderOptions)
|
if err := cli.FinalizeReaderOptions(&options.ReaderOptions); err != nil {
|
||||||
cli.FinalizeWriterOptions(&options.WriterOptions)
|
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", scriptName, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if err := cli.FinalizeWriterOptions(&options.WriterOptions); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", scriptName, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
options.WriterOptions.AutoFlatten = cli.DecideFinalFlatten(&options.WriterOptions)
|
options.WriterOptions.AutoFlatten = cli.DecideFinalFlatten(&options.WriterOptions)
|
||||||
options.WriterOptions.AutoUnflatten = cli.DecideFinalUnflatten(options, [][]string{})
|
options.WriterOptions.AutoUnflatten = cli.DecideFinalUnflatten(options, [][]string{})
|
||||||
|
|
||||||
|
|
@ -143,7 +149,11 @@ func ScriptMain(args []string) int {
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
|
||||||
scr.bufferedRecordOutputStream.Flush()
|
err = scr.bufferedRecordOutputStream.Flush()
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "mlr script: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
err = scr.closeBufferedOutputStream()
|
err = scr.closeBufferedOutputStream()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "mlr script: %v\n", err)
|
fmt.Fprintf(os.Stderr, "mlr script: %v\n", err)
|
||||||
|
|
|
||||||
|
|
@ -126,9 +126,10 @@ func (scr *Script) run() error {
|
||||||
}
|
}
|
||||||
|
|
||||||
if rac.Record == nil {
|
if rac.Record == nil {
|
||||||
// Output string from print/dump etc
|
// Output string from print/dump etc; terminal output, nothing
|
||||||
bufferedOutput.WriteString(rac.OutputString)
|
// useful to do on write failure
|
||||||
bufferedOutput.Flush()
|
_, _ = bufferedOutput.WriteString(rac.OutputString)
|
||||||
|
_ = bufferedOutput.Flush()
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -40,14 +40,14 @@ func captureUsageFunc(usageFunc TransformerUsageFunc) string {
|
||||||
done := make(chan string)
|
done := make(chan string)
|
||||||
go func() {
|
go func() {
|
||||||
var buf bytes.Buffer
|
var buf bytes.Buffer
|
||||||
io.Copy(&buf, r)
|
_, _ = io.Copy(&buf, r)
|
||||||
done <- buf.String()
|
done <- buf.String()
|
||||||
}()
|
}()
|
||||||
|
|
||||||
usageFunc(w)
|
usageFunc(w)
|
||||||
w.Close()
|
_ = w.Close()
|
||||||
s := <-done
|
s := <-done
|
||||||
r.Close()
|
_ = r.Close()
|
||||||
return s
|
return s
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -253,7 +253,9 @@ func transformerJoinParseCLI(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
cli.FinalizeReaderOptions(&opts.joinFlagOptions.ReaderOptions)
|
if err := cli.FinalizeReaderOptions(&opts.joinFlagOptions.ReaderOptions); err != nil {
|
||||||
|
return nil, cli.VerbErrorf(verb, "%v", err)
|
||||||
|
}
|
||||||
|
|
||||||
if opts.leftFileName == "" {
|
if opts.leftFileName == "" {
|
||||||
return nil, cli.VerbErrorf(verb, "need left file name")
|
return nil, cli.VerbErrorf(verb, "need left file name")
|
||||||
|
|
|
||||||
|
|
@ -192,7 +192,7 @@ func transformerPutOrFilterParseCLI(
|
||||||
verb := args[argi]
|
verb := args[argi]
|
||||||
argi++
|
argi++
|
||||||
|
|
||||||
var dslStrings []string = []string{}
|
dslStrings := []string{}
|
||||||
haveDSLStringsHere := false
|
haveDSLStringsHere := false
|
||||||
echoDSLString := false
|
echoDSLString := false
|
||||||
printASTAsTree := false
|
printASTAsTree := false
|
||||||
|
|
@ -340,7 +340,9 @@ func transformerPutOrFilterParseCLI(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
cli.FinalizeWriterOptions(&options.WriterOptions)
|
if err := cli.FinalizeWriterOptions(&options.WriterOptions); err != nil {
|
||||||
|
return nil, cli.VerbErrorf(verb, "%v", err)
|
||||||
|
}
|
||||||
|
|
||||||
// If they've used either of 'mlr put -f {filename}' or 'mlr put -e
|
// If they've used either of 'mlr put -f {filename}' or 'mlr put -e
|
||||||
// {expression}' then that specifies their DSL expression. But if they've
|
// {expression}' then that specifies their DSL expression. But if they've
|
||||||
|
|
|
||||||
|
|
@ -88,17 +88,17 @@ func transformerSplitParseCLI(
|
||||||
argi++
|
argi++
|
||||||
|
|
||||||
var n int64 = 0
|
var n int64 = 0
|
||||||
var doMod bool = false
|
doMod := false
|
||||||
var doSize bool = false
|
doSize := false
|
||||||
var groupByFieldNames []string = nil
|
var groupByFieldNames []string = nil
|
||||||
var emitDownstream bool = false
|
emitDownstream := false
|
||||||
var escapeFileNameCharacters bool = true
|
escapeFileNameCharacters := true
|
||||||
var fileNamePartJoiner string = splitDefaultFileNamePartJoiner
|
fileNamePartJoiner := splitDefaultFileNamePartJoiner
|
||||||
var doAppend bool = false
|
doAppend := false
|
||||||
var outputFileNamePrefix string = splitDefaultOutputFileNamePrefix
|
outputFileNamePrefix := splitDefaultOutputFileNamePrefix
|
||||||
var outputFileNameSuffix string = "uninit"
|
outputFileNameSuffix := "uninit"
|
||||||
haveOutputFileNameSuffix := false
|
haveOutputFileNameSuffix := false
|
||||||
var outputFolder string = ""
|
outputFolder := ""
|
||||||
|
|
||||||
var localOptions *cli.TOptions = nil
|
var localOptions *cli.TOptions = nil
|
||||||
if mainOptions != nil {
|
if mainOptions != nil {
|
||||||
|
|
@ -200,7 +200,9 @@ func transformerSplitParseCLI(
|
||||||
return nil, cli.VerbErrorf(verb, "-n, -g, and -s are mutually exclusive")
|
return nil, cli.VerbErrorf(verb, "-n, -g, and -s are mutually exclusive")
|
||||||
}
|
}
|
||||||
|
|
||||||
cli.FinalizeWriterOptions(&localOptions.WriterOptions)
|
if err := cli.FinalizeWriterOptions(&localOptions.WriterOptions); err != nil {
|
||||||
|
return nil, cli.VerbErrorf(verb, "%v", err)
|
||||||
|
}
|
||||||
if !haveOutputFileNameSuffix {
|
if !haveOutputFileNameSuffix {
|
||||||
outputFileNameSuffix = localOptions.WriterOptions.OutputFileFormat
|
outputFileNameSuffix = localOptions.WriterOptions.OutputFileFormat
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -98,7 +98,9 @@ func transformerTeeParseCLI(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
cli.FinalizeWriterOptions(&localOptions.WriterOptions)
|
if err := cli.FinalizeWriterOptions(&localOptions.WriterOptions); err != nil {
|
||||||
|
return nil, cli.VerbErrorf(verbNameTee, "%v", err)
|
||||||
|
}
|
||||||
|
|
||||||
// Get the filename/command from the command line, after the flags
|
// Get the filename/command from the command line, after the flags
|
||||||
if argi >= argc {
|
if argi >= argc {
|
||||||
|
|
|
||||||
|
|
@ -87,6 +87,118 @@ nil check before len), 3 QF1007 (merge conditional assignment), 3 QF1006 (lift i
|
||||||
- `os.Setenv`/`os.Unsetenv` — low-risk but cheap to fix
|
- `os.Setenv`/`os.Unsetenv` — low-risk but cheap to fix
|
||||||
2. `staticcheck` style — 34 findings, mostly mechanical (type inference, nil-check cleanup, etc.)
|
2. `staticcheck` style — 34 findings, mostly mechanical (type inference, nil-check cleanup, etc.)
|
||||||
|
|
||||||
|
## IMPORTANT: issue counts before round 5 were capped
|
||||||
|
|
||||||
|
All counts above (the "134 baseline", the "84 remaining") were taken with golangci-lint's
|
||||||
|
**default `max-same-issues=3`**, which reports at most 3 findings per identical message text.
|
||||||
|
Running with `--max-same-issues=0 --max-issues-per-linter=0` on the same tree shows the true
|
||||||
|
backlog before round 5 was **1271 issues: 1202 errcheck, 69 staticcheck** (not 84/50/34).
|
||||||
|
CI shares this cap, so fixing the visible 3 of a kind just surfaces the next 3 — whack-a-mole.
|
||||||
|
All counts from here on are uncapped.
|
||||||
|
|
||||||
|
## Round 5 (branch `johnkerl/lint5`, 2026-07-03): staticcheck batch — DONE
|
||||||
|
|
||||||
|
Fixed all 69 staticcheck findings; **staticcheck is now at zero** (uncapped). `make check`
|
||||||
|
passes (4676 regression cases). Defers errcheck to rounds 6+; this inverts the priority order
|
||||||
|
above, deliberately: the staticcheck fixes are low-risk and retire an entire linter, while the
|
||||||
|
errcheck sites need per-site judgment and a few of them (see rounds 6+ below) may be real
|
||||||
|
user-facing bugs deserving focused PRs.
|
||||||
|
|
||||||
|
What was fixed (uncapped counts):
|
||||||
|
|
||||||
|
1. **ST1023 + QF1011 — omit explicit type (37):** `pkg/bifs/arithmetic.go` (20: all min/max
|
||||||
|
helpers), `pkg/bifs/bits.go` (2), `pkg/dsl/cst/udf.go`, `pkg/dsl/cst/uds.go`,
|
||||||
|
`pkg/input/record_reader_dkvp_nidx.go`, `pkg/output/record_writer_csv.go`,
|
||||||
|
`pkg/output/record_writer_tsv.go`, `pkg/transformers/put_or_filter.go`,
|
||||||
|
`pkg/transformers/split.go` (9, whole option-declaration block for consistency).
|
||||||
|
2. **S1009 + S1031 — redundant nil checks (14+1):** `pkg/dsl/cst/{blocks,evaluable,leaves,lvalues,udf,uds}.go`,
|
||||||
|
`pkg/mlrval/mlrval_yaml.go`. Also fixed two unflagged-but-identical adjacent patterns
|
||||||
|
(`udf.go`/`uds.go` inner `typeNode.Children` check, `leaves.go:23`) for consistency.
|
||||||
|
3. **QF1007 — merge conditional assignment into declaration (3):**
|
||||||
|
`found := false; if cond { found = true }` → `found := cond` in
|
||||||
|
`pkg/terminals/help/entry.go` (×2) and `pkg/terminals/repl/verbs.go`. Only the first
|
||||||
|
conditional merges; the subsequent `if ... { found = true }` blocks must stay as-is since
|
||||||
|
each callee is invoked for its help-printing side effect (no `||` short-circuiting).
|
||||||
|
4. **QF1006 — lift break into loop condition (3):** `pkg/bifs/relative_time.go` (×2) →
|
||||||
|
`for remainingInput != "" {`; `pkg/mlrval/mlrmap_accessors.go` (`Mlrmap.Label`) →
|
||||||
|
`for i < numNewNames {` (interior `pe == nil` break stays).
|
||||||
|
5. **QF1001 — De Morgan (3):** `pkg/dsl/cst/builtin_functions.go:878,960` →
|
||||||
|
`btype != mlrval.MT_ABSENT && btype != mlrval.MT_BOOL`.
|
||||||
|
`pkg/terminals/help/entry_which.go:167`: mechanical De Morgan reads worse, and staticcheck
|
||||||
|
also flags a merely-outermost negation `!((...) || (...))`; settled on naming the predicate
|
||||||
|
(`isWordRune := ...; return !isWordRune`), which staticcheck accepts and reads best.
|
||||||
|
6. **SA9003 — empty branches (9):** all were no-op branches existing only to hold a comment;
|
||||||
|
deleted the branch, kept the comment adjacent. `pkg/dsl/cst/for.go` (×4 data-heterogeneity
|
||||||
|
no-ops), `pkg/dsl/cst/lvalues.go` (×2 TODO-comment else branches), `pkg/dsl/cst/root.go`
|
||||||
|
(nil-udf explainer), `pkg/input/record_reader_csv.go` (SkipComments else),
|
||||||
|
`pkg/terminals/repl/verbs.go` (zero-filenames-is-stdin).
|
||||||
|
|
||||||
|
After this round: **1202 issues remain, all errcheck** (uncapped).
|
||||||
|
|
||||||
|
## Round 6 (branch `johnkerl/lint5`, 2026-07-03): errcheck — DONE
|
||||||
|
|
||||||
|
**golangci-lint now reports 0 issues** (uncapped) on `./cmd/mlr ./pkg/...`. `make check` passes.
|
||||||
|
Done in the same PR as round 5. True pre-round breakdown by callee: 893 `Fprintf` + 31
|
||||||
|
`Fprintln` + 25 `Fprint` (= 949 `fmt.Fprint*`, of which ~894 were `pkg/transformers` usage
|
||||||
|
printers), 140 `WriteString`, 28 `Close`, 12 `Set`, 10 `Remove`, 10 `Flush`, 9 `RemoveIndexed`,
|
||||||
|
8 `Setenv`, 5 `Write`, 9 `Finalize{Reader,Writer}Options`, 3 `PutIndexed`, plus a small tail.
|
||||||
|
|
||||||
|
### 6a — `.golangci.yml` with errcheck exclusions (~1090 findings)
|
||||||
|
|
||||||
|
Added `.golangci.yml` (picked up automatically by the CI action) with
|
||||||
|
`errcheck.exclude-functions` for `fmt.Fprint/Fprintf/Fprintln` (usage/error printers),
|
||||||
|
`(*bufio.Writer).Write/WriteString` (sticky errors, surface at the now-checked final Flush),
|
||||||
|
and `(*strings.Builder).WriteString` (documented never to fail). Also pinned
|
||||||
|
`max-issues-per-linter: 0` and `max-same-issues: 0` so CI reports true counts from now on.
|
||||||
|
Caveat found: the bufio exclusion doesn't match calls through struct fields
|
||||||
|
(e.g. `repl.bufferedRecordOutputStream.WriteString`) — those got explicit `_ =` instead.
|
||||||
|
|
||||||
|
### 6b — propagated: real error paths (~30 findings)
|
||||||
|
|
||||||
|
- `cli.Finalize{Reader,Writer}Options` (9): join/put-or-filter/split/tee verb constructors now
|
||||||
|
return the error (`mlr join -i badformat` now exits 1 with "unrecognized input format"
|
||||||
|
instead of silently proceeding with wrong separators — verified end-to-end); repl/script
|
||||||
|
entry points print to stderr and exit; the unit-test site asserts nil.
|
||||||
|
- `pkg/stream/stream.go` final `Flush` → propagated into `retval` (full disk / closed pipe no
|
||||||
|
longer exits 0 silently).
|
||||||
|
- DSL `emit`/`print`/`dump` redirect writes (11): recursive emit calls, `printToRedirectFunc`,
|
||||||
|
`dumpToRedirectFunc`, and both `outputHandlerManager.WriteString` sites now propagate through
|
||||||
|
`Execute`, matching their sibling branches which already did.
|
||||||
|
- `pkg/output/record_writer_csv.go` `WriteCSVRecordMaybeColorized` (2) → propagated through
|
||||||
|
`IRecordWriter.Write`, whose error the channel writer already reports.
|
||||||
|
- `pkg/output/file_output_handlers.go` close-time `Flush` → propagated.
|
||||||
|
- `pkg/runtime/stack.go` `PutIndexed` on fresh map → propagated (setIndexed returns error).
|
||||||
|
- `ENV[...]` assignment `os.Setenv` → propagated through `Assign`.
|
||||||
|
- REPL `writeRecord`: `recordWriter.Write` error now printed to the terminal (e.g. CSV
|
||||||
|
schema-change errors were silently swallowed in the REPL); `closeBufferedOutputStream` on
|
||||||
|
`:>` / `:>>` redirect switches now prints on error. Flush-before-close at repl/script exit
|
||||||
|
now checked like the close next to it.
|
||||||
|
- `pkg/auxents/termcvt.go`: write-side `ostream.Close()` before the rename-over-original now
|
||||||
|
checked (had a `TODO: check return status`); `ostream.Write` in termcvt/unhex now exits on
|
||||||
|
error like the surrounding error handling.
|
||||||
|
|
||||||
|
### 6c — explicit `_ =` ignores (~65 findings)
|
||||||
|
|
||||||
|
- Unset-style DSL/runtime paths (9 `RemoveIndexed`, 1 `Unsetenv`, in `lvalues.go`/`stack.go`):
|
||||||
|
unset of a non-existent path is a no-op by design; the enclosing `Unassign` API has no error
|
||||||
|
return. Commented at each site.
|
||||||
|
- `Mlrmap` unflatten `PutIndexed` (2): best-effort API with no error return; commented.
|
||||||
|
- Mid-stream `FlushOnEveryRecord` flushes (pprint, channel writer): bufio errors are sticky and
|
||||||
|
the final Flush in `pkg/stream` is checked; commented.
|
||||||
|
- Read-side `Close` (input readers ×10, auxents ×4, readfiles ×2, halfpipe, mlrrc, option_parse
|
||||||
|
`--load`, CPU-profile handle), `go process.Wait()` in halfpipe.
|
||||||
|
- Init-time strftime `ss.Set` registrations (12) in `pkg/bifs/datetime.go` (constant specs).
|
||||||
|
- In-memory usage-capture pipes (`io.Copy`/`Close`) in `keyword_usage_json.go` and
|
||||||
|
`aaa_transformer_json.go`.
|
||||||
|
- regtest harness `os.Setenv`/`Unsetenv`/`Remove` (14) and `entrypoint.go` temp-file `os.Remove`
|
||||||
|
on error paths (5).
|
||||||
|
- REPL/script terminal `WriteString`/`Flush` where the config exclusion can't reach (field
|
||||||
|
access); commented.
|
||||||
|
|
||||||
|
Also fixed in passing: staticcheck QF1012 in `record_writer_pprint.go` (surfaced once the
|
||||||
|
errcheck finding on the same line was excluded): `WriteString(fmt.Sprint(x))` where `x` is
|
||||||
|
already a string → `WriteString(x)`; dropped the then-unused `fmt` import.
|
||||||
|
|
||||||
## Bug noticed in passing (fixed on `johnkerl/lint4`)
|
## Bug noticed in passing (fixed on `johnkerl/lint4`)
|
||||||
|
|
||||||
In `pkg/mlrval/mlrval_collections.go`, `removeIndexedOnArray` with a single in-bounds index removed
|
In `pkg/mlrval/mlrval_collections.go`, `removeIndexedOnArray` with a single in-bounds index removed
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue