mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
Lint round 5+6: staticcheck and errcheck to zero (#2130)
* refine the plan
* Fix all staticcheck lint findings (uncapped)
golangci-lint's default max-same-issues=3 was hiding most of the backlog:
the true pre-fix count was 69 staticcheck findings, not 34. This fixes all
of them, driving staticcheck to zero:
- ST1023/QF1011 (37): omit explicit types inferred from the RHS
- S1009/S1031 (15): drop redundant nil checks before len()/range
- SA9003 (9): remove comment-only empty branches, keeping the comments
- QF1007 (3): merge conditional assignment into declaration
- QF1006 (3): lift break conditions into loop conditions
- QF1001 (3): apply De Morgan's law / name the negated predicate
Also updates plans/lintfixes.md with the cap discovery and the corrected
errcheck picture (1202 uncapped, ~949 of them fmt.Fprint*).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drive errcheck to zero: config for bulk categories, propagate real errors
Adds .golangci.yml with errcheck exclude-functions for fmt.Fprint* (usage
printers), (*bufio.Writer).Write/WriteString (sticky errors, surfaced at the
now-checked final Flush), and (*strings.Builder).WriteString; pins
max-issues-per-linter/max-same-issues to 0 so CI reports true counts.
Real error paths now propagate instead of being dropped:
- Finalize{Reader,Writer}Options in join/put/filter/split/tee and the
repl/script entry points: 'mlr join -i badformat' now errors instead of
silently using wrong separators
- final output-stream Flush in pkg/stream: write failure no longer exits 0
- DSL emit/print/dump redirect writes, matching their sibling branches
- CSV writer WriteCSVRecordMaybeColorized, close-time Flush in file output
handlers, ENV[...] Setenv, REPL record-write and redirect-close errors
- termcvt write-side Close before rename (had "TODO: check return status")
The rest are deliberate ignores, marked with _ = and a comment where the
reason isn't obvious: unset-of-missing-path no-ops, read-side closes,
mid-stream FlushOnEveryRecord, init-time strftime registrations, in-memory
usage-capture pipes, and regtest-harness env/temp-file teardown.
golangci-lint now reports 0 issues on ./cmd/mlr ./pkg/... with all caps off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
2bec10fc54
commit
91eaff1341
63 changed files with 435 additions and 236 deletions
28
.golangci.yml
Normal file
28
.golangci.yml
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# Configuration for golangci-lint, used by .github/workflows/golangci-lint.yml
|
||||
# and available locally via `golangci-lint run ./cmd/mlr ./pkg/...`.
|
||||
# Tracking issue for lint cleanup: https://github.com/johnkerl/miller/issues/2109
|
||||
|
||||
version: "2"
|
||||
|
||||
linters:
|
||||
settings:
|
||||
errcheck:
|
||||
exclude-functions:
|
||||
# Writes to stdout/stderr for usage and error messages; there is
|
||||
# nothing useful to do if these fail.
|
||||
- fmt.Fprint
|
||||
- fmt.Fprintf
|
||||
- fmt.Fprintln
|
||||
# bufio.Writer errors are sticky: once a write fails, all subsequent
|
||||
# writes and the next Flush report the same error. Record-writer
|
||||
# output goes through a final checked Flush in pkg/stream.
|
||||
- (*bufio.Writer).Write
|
||||
- (*bufio.Writer).WriteString
|
||||
# Documented to always return a nil error.
|
||||
- (*strings.Builder).WriteString
|
||||
|
||||
issues:
|
||||
# Report all findings: the defaults (50/3) hide large categories and make
|
||||
# fix progress impossible to track. See plans/lintfixes.md.
|
||||
max-issues-per-linter: 0
|
||||
max-same-issues: 0
|
||||
|
|
@ -55,7 +55,7 @@ func main() {
|
|||
fmt.Fprintln(os.Stderr, os.Args[0], ": ", "Could not start CPU profile: ", err)
|
||||
return
|
||||
}
|
||||
defer handle.Close()
|
||||
defer func() { _ = handle.Close() }()
|
||||
|
||||
if err := pprof.StartCPUProfile(handle); err != nil {
|
||||
fmt.Fprintln(os.Stderr, os.Args[0], ": ", "Could not start CPU profile: ", err)
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ func hexMain(args []string) int {
|
|||
|
||||
hexDumpFile(istream, doRaw)
|
||||
|
||||
istream.Close()
|
||||
_ = istream.Close()
|
||||
if !doRaw && len(args) > 1 {
|
||||
fmt.Println()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ func lecatMain(args []string) int {
|
|||
|
||||
lecatFile(istream, doColor)
|
||||
|
||||
istream.Close()
|
||||
_ = istream.Close()
|
||||
}
|
||||
}
|
||||
return 0
|
||||
|
|
|
|||
|
|
@ -96,9 +96,11 @@ func termcvtMain(args []string) int {
|
|||
|
||||
termcvtFile(istream, ostream, inputTerminator, outputTerminator)
|
||||
|
||||
istream.Close()
|
||||
// TODO: check return status
|
||||
ostream.Close()
|
||||
_ = istream.Close()
|
||||
if err := ostream.Close(); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr termcvt: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
err = os.Rename(tempname, filename)
|
||||
if err != nil {
|
||||
|
|
@ -120,7 +122,7 @@ func termcvtMain(args []string) int {
|
|||
|
||||
termcvtFile(istream, os.Stdout, inputTerminator, outputTerminator)
|
||||
|
||||
istream.Close()
|
||||
_ = istream.Close()
|
||||
}
|
||||
}
|
||||
return 0
|
||||
|
|
@ -144,6 +146,9 @@ func termcvtFile(istream *os.File, ostream *os.File, inputTerminator string, out
|
|||
|
||||
// This is how to do a chomp:
|
||||
line = strings.TrimRight(line, inputTerminator)
|
||||
ostream.Write([]byte(line + outputTerminator))
|
||||
if _, err := ostream.Write([]byte(line + outputTerminator)); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr termcvt: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ func unhexMain(args []string) int {
|
|||
os.Exit(1)
|
||||
}
|
||||
unhexFile(istream, os.Stdout)
|
||||
istream.Close()
|
||||
_ = istream.Close()
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -83,7 +83,10 @@ func unhexFile(istream *os.File, ostream *os.File) {
|
|||
os.Exit(1)
|
||||
}
|
||||
byteArray[0] = byte(scanValue)
|
||||
ostream.Write(byteArray)
|
||||
if _, err := ostream.Write(byteArray); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "mlr unhex:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -718,26 +718,26 @@ func BIF_mod_exp(input1, input2, input3 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
// * empty-null always loses against numbers
|
||||
|
||||
func min_f_ff(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a float64 = input1.AcquireFloatValue()
|
||||
var b float64 = input2.AcquireFloatValue()
|
||||
a := input1.AcquireFloatValue()
|
||||
b := input2.AcquireFloatValue()
|
||||
return mlrval.FromFloat(math.Min(a, b))
|
||||
}
|
||||
|
||||
func min_f_fi(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a float64 = input1.AcquireFloatValue()
|
||||
var b float64 = float64(input2.AcquireIntValue())
|
||||
a := input1.AcquireFloatValue()
|
||||
b := float64(input2.AcquireIntValue())
|
||||
return mlrval.FromFloat(math.Min(a, b))
|
||||
}
|
||||
|
||||
func min_f_if(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a float64 = float64(input1.AcquireIntValue())
|
||||
var b float64 = input2.AcquireFloatValue()
|
||||
a := float64(input1.AcquireIntValue())
|
||||
b := input2.AcquireFloatValue()
|
||||
return mlrval.FromFloat(math.Min(a, b))
|
||||
}
|
||||
|
||||
func min_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a int64 = input1.AcquireIntValue()
|
||||
var b int64 = input2.AcquireIntValue()
|
||||
a := input1.AcquireIntValue()
|
||||
b := input2.AcquireIntValue()
|
||||
if a < b {
|
||||
return input1
|
||||
}
|
||||
|
|
@ -756,8 +756,8 @@ func min_b_bb(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
}
|
||||
|
||||
func min_s_ss(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a string = input1.AcquireStringValue()
|
||||
var b string = input2.AcquireStringValue()
|
||||
a := input1.AcquireStringValue()
|
||||
b := input2.AcquireStringValue()
|
||||
if a < b {
|
||||
return input1
|
||||
}
|
||||
|
|
@ -883,26 +883,26 @@ func BIF_minlen_within_map_values(m *mlrval.Mlrmap) *mlrval.Mlrval {
|
|||
}
|
||||
|
||||
func max_f_ff(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a float64 = input1.AcquireFloatValue()
|
||||
var b float64 = input2.AcquireFloatValue()
|
||||
a := input1.AcquireFloatValue()
|
||||
b := input2.AcquireFloatValue()
|
||||
return mlrval.FromFloat(math.Max(a, b))
|
||||
}
|
||||
|
||||
func max_f_fi(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a float64 = input1.AcquireFloatValue()
|
||||
var b float64 = float64(input2.AcquireIntValue())
|
||||
a := input1.AcquireFloatValue()
|
||||
b := float64(input2.AcquireIntValue())
|
||||
return mlrval.FromFloat(math.Max(a, b))
|
||||
}
|
||||
|
||||
func max_f_if(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a float64 = float64(input1.AcquireIntValue())
|
||||
var b float64 = input2.AcquireFloatValue()
|
||||
a := float64(input1.AcquireIntValue())
|
||||
b := input2.AcquireFloatValue()
|
||||
return mlrval.FromFloat(math.Max(a, b))
|
||||
}
|
||||
|
||||
func max_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a int64 = input1.AcquireIntValue()
|
||||
var b int64 = input2.AcquireIntValue()
|
||||
a := input1.AcquireIntValue()
|
||||
b := input2.AcquireIntValue()
|
||||
if a > b {
|
||||
return input1
|
||||
}
|
||||
|
|
@ -921,8 +921,8 @@ func max_b_bb(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
}
|
||||
|
||||
func max_s_ss(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var a string = input1.AcquireStringValue()
|
||||
var b string = input2.AcquireStringValue()
|
||||
a := input1.AcquireStringValue()
|
||||
b := input2.AcquireStringValue()
|
||||
if a > b {
|
||||
return input1
|
||||
}
|
||||
|
|
|
|||
|
|
@ -223,8 +223,8 @@ func BIF_signed_right_shift(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
// Unsigned right shift
|
||||
|
||||
func ursh_i_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
var ua uint64 = uint64(input1.AcquireIntValue())
|
||||
var ub uint64 = uint64(input2.AcquireIntValue())
|
||||
ua := uint64(input1.AcquireIntValue())
|
||||
ub := uint64(input2.AcquireIntValue())
|
||||
var uc = ua >> ub
|
||||
return mlrval.FromInt(int64(uc))
|
||||
}
|
||||
|
|
|
|||
|
|
@ -448,18 +448,18 @@ func init() {
|
|||
})
|
||||
|
||||
ss := strftime.NewSpecificationSet()
|
||||
ss.Set('1', appender1)
|
||||
ss.Set('2', appender2)
|
||||
ss.Set('3', appender3)
|
||||
ss.Set('4', appender4)
|
||||
ss.Set('5', appender5)
|
||||
ss.Set('6', appender6)
|
||||
ss.Set('7', appender7)
|
||||
ss.Set('8', appender8)
|
||||
ss.Set('9', appender9)
|
||||
ss.Set('N', appenderN)
|
||||
ss.Set('O', appenderO)
|
||||
ss.Set('s', appenderS)
|
||||
_ = ss.Set('1', appender1)
|
||||
_ = ss.Set('2', appender2)
|
||||
_ = ss.Set('3', appender3)
|
||||
_ = ss.Set('4', appender4)
|
||||
_ = ss.Set('5', appender5)
|
||||
_ = ss.Set('6', appender6)
|
||||
_ = ss.Set('7', appender7)
|
||||
_ = ss.Set('8', appender8)
|
||||
_ = ss.Set('9', appender9)
|
||||
_ = ss.Set('N', appenderN)
|
||||
_ = ss.Set('O', appenderO)
|
||||
_ = ss.Set('s', appenderS)
|
||||
|
||||
strftimeExtensions = strftime.WithSpecificationSet(ss)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -27,10 +27,7 @@ func BIF_dhms2sec(input1 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
remainingInput := input
|
||||
var seconds int64
|
||||
|
||||
for {
|
||||
if remainingInput == "" {
|
||||
break
|
||||
}
|
||||
for remainingInput != "" {
|
||||
var n int64
|
||||
var rest string
|
||||
|
||||
|
|
@ -89,10 +86,7 @@ func BIF_dhms2fsec(input1 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
remainingInput := input
|
||||
var seconds float64
|
||||
|
||||
for {
|
||||
if remainingInput == "" {
|
||||
break
|
||||
}
|
||||
for remainingInput != "" {
|
||||
var f float64
|
||||
var rest string
|
||||
|
||||
|
|
|
|||
|
|
@ -3458,7 +3458,7 @@ var MiscFlagSection = FlagSection{
|
|||
fmt.Fprintf(os.Stderr, "mlr: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer handle.Close()
|
||||
defer func() { _ = handle.Close() }()
|
||||
|
||||
lineReader := bufio.NewReader(handle)
|
||||
|
||||
|
|
@ -3565,7 +3565,7 @@ var MiscFlagSection = FlagSection{
|
|||
help: "Specify timezone, overriding `$TZ` environment variable (if any).",
|
||||
parser: func(args []string, argc int, pargi *int, options *TOptions) {
|
||||
CheckArgCount(args, *pargi, argc, 2)
|
||||
os.Setenv("TZ", args[*pargi+1])
|
||||
_ = os.Setenv("TZ", args[*pargi+1])
|
||||
*pargi += 2
|
||||
},
|
||||
},
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ func tryLoadMlrrc(
|
|||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer handle.Close()
|
||||
defer func() { _ = handle.Close() }()
|
||||
|
||||
lineReader := bufio.NewReader(handle)
|
||||
|
||||
|
|
|
|||
|
|
@ -63,7 +63,7 @@ func (root *RootNode) BuildStatementBlockNodeFromBeginOrEnd(
|
|||
// With "parent":1,"children":[1], StatementBlockInBraces.Children[0] is the StatementBlock.
|
||||
// Unwrap so we pass StatementBlock to BuildStatementBlockNode.
|
||||
if astStatementBlockNode.Type == asts.NodeType(NodeTypeStatementBlockInBraces) {
|
||||
lib.InternalCodingErrorIf(astStatementBlockNode.Children == nil || len(astStatementBlockNode.Children) < 1)
|
||||
lib.InternalCodingErrorIf(len(astStatementBlockNode.Children) < 1)
|
||||
astStatementBlockNode = astStatementBlockNode.Children[0]
|
||||
}
|
||||
statementBlockNode, err := root.BuildStatementBlockNode(astStatementBlockNode)
|
||||
|
|
|
|||
|
|
@ -875,7 +875,7 @@ func (node *LogicalANDOperatorNode) Evaluate(
|
|||
|
||||
bout := node.b.Evaluate(state)
|
||||
btype := bout.Type()
|
||||
if !(btype == mlrval.MT_ABSENT || btype == mlrval.MT_BOOL) {
|
||||
if btype != mlrval.MT_ABSENT && btype != mlrval.MT_BOOL {
|
||||
return mlrval.FromNotNamedTypeError("&&", bout, "absent or boolean")
|
||||
}
|
||||
if btype == mlrval.MT_ABSENT {
|
||||
|
|
@ -957,7 +957,7 @@ func (node *LogicalOROperatorNode) Evaluate(
|
|||
|
||||
bout := node.b.Evaluate(state)
|
||||
btype := bout.Type()
|
||||
if !(btype == mlrval.MT_ABSENT || btype == mlrval.MT_BOOL) {
|
||||
if btype != mlrval.MT_ABSENT && btype != mlrval.MT_BOOL {
|
||||
return mlrval.FromNotNamedTypeError("||", bout, "absent or boolean")
|
||||
}
|
||||
if btype == mlrval.MT_ABSENT {
|
||||
|
|
|
|||
|
|
@ -202,7 +202,9 @@ func (node *DumpStatementNode) Execute(state *runtime.State) (*BlockExitPayload,
|
|||
}
|
||||
}
|
||||
outputString := buffer.String()
|
||||
node.dumpToRedirectFunc(outputString, state)
|
||||
if err := node.dumpToRedirectFunc(outputString, state); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
|
|
@ -244,6 +246,5 @@ func (node *DumpStatementNode) dumpToFileOrPipe(
|
|||
}
|
||||
outputFileName := redirectorTarget.String()
|
||||
|
||||
node.outputHandlerManager.WriteString(outputString, outputFileName)
|
||||
return nil
|
||||
return node.outputHandlerManager.WriteString(outputString, outputFileName)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -537,7 +537,10 @@ func (node *EmitXStatementNode) executeNonIndexedNonLashedEmit(
|
|||
nextLevelNames = append(nextLevelNames, pe.Key)
|
||||
nextLevelValues = append(nextLevelValues, pe.Value.Copy())
|
||||
}
|
||||
node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
||||
err := node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -625,7 +628,10 @@ func (node *EmitXStatementNode) executeNonIndexedLashedEmit(
|
|||
nextLevelNames = append(nextLevelNames, pe.Key)
|
||||
nextLevelValues = append(nextLevelValues, pe.Value.Copy())
|
||||
}
|
||||
node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
||||
err := node.executeNonIndexedNonLashedEmit(nextLevelNames, nextLevelValues, state)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -821,13 +827,16 @@ func (node *EmitXStatementNode) executeIndexedNonLashedEmitAux(
|
|||
return err
|
||||
}
|
||||
} else {
|
||||
node.executeIndexedNonLashedEmitPAux(
|
||||
err := node.executeIndexedNonLashedEmitPAux(
|
||||
newrec,
|
||||
[]string{names[i]},
|
||||
[]*mlrval.Mlrmap{valueAsMap},
|
||||
indices[1:],
|
||||
state,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -966,13 +975,16 @@ func (node *EmitXStatementNode) executeIndexedLashedEmitAux(
|
|||
if len(indices) > 1 && nextLevelMaps[0] != nil {
|
||||
// Recurse. The leading map drives the iteration; we don't
|
||||
// continue even if other maps aren't empty
|
||||
node.executeIndexedLashedEmitAux(
|
||||
err := node.executeIndexedLashedEmitAux(
|
||||
newrec,
|
||||
names,
|
||||
nextLevelMaps,
|
||||
indices[1:],
|
||||
state,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else { // end of recursion
|
||||
for i, nextLevelValue := range nextLevelValues {
|
||||
if nextLevelValue != nil {
|
||||
|
|
@ -1027,13 +1039,16 @@ func (node *EmitXStatementNode) executeIndexedNonLashedEmitPAux(
|
|||
return err
|
||||
}
|
||||
} else {
|
||||
node.executeIndexedNonLashedEmitPAux(
|
||||
err := node.executeIndexedNonLashedEmitPAux(
|
||||
newrec,
|
||||
[]string{names[i]},
|
||||
[]*mlrval.Mlrmap{valueAsMap},
|
||||
indices[1:],
|
||||
state,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1083,13 +1098,16 @@ func (node *EmitXStatementNode) executeIndexedLashedEmitPAux(
|
|||
|
||||
if nextLevelMaps[0] != nil && len(indices) >= 2 {
|
||||
// recurse
|
||||
node.executeIndexedLashedEmitPAux(
|
||||
err := node.executeIndexedLashedEmitPAux(
|
||||
newrec,
|
||||
names,
|
||||
nextLevelMaps,
|
||||
indices[1:],
|
||||
state,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
// end of recursion
|
||||
for i, nextLevel := range nextLevels {
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ import (
|
|||
|
||||
func (root *RootNode) BuildEvaluableNode(astNode *asts.ASTNode) (IEvaluable, error) {
|
||||
// Try BuildLeafNode first for terminals
|
||||
if astNode.Children == nil || len(astNode.Children) == 0 {
|
||||
if len(astNode.Children) == 0 {
|
||||
if leaf, err := root.BuildLeafNode(astNode); err == nil {
|
||||
return leaf, nil
|
||||
}
|
||||
|
|
@ -97,7 +97,7 @@ func (root *RootNode) BuildEvaluableNode(astNode *asts.ASTNode) (IEvaluable, err
|
|||
|
||||
// Fallback: try BuildLeafNode for unhandled types (e.g. DirectFieldValue, IntLiteral).
|
||||
// Only for leaf-like nodes (0 or 1 child); nodes with 2+ children are not leaves.
|
||||
if astNode.Children == nil || len(astNode.Children) <= 1 {
|
||||
if len(astNode.Children) <= 1 {
|
||||
if leaf, err := root.BuildLeafNode(astNode); err == nil {
|
||||
return leaf, nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -182,9 +182,8 @@ func (node *ForLoopOneVariableNode) Execute(state *runtime.State) (*BlockExitPay
|
|||
}
|
||||
}
|
||||
|
||||
} else if indexMlrval.IsAbsent() {
|
||||
// Data-heterogeneity no-op
|
||||
}
|
||||
// else if indexMlrval.IsAbsent(): data-heterogeneity no-op
|
||||
|
||||
// TODO: backwards compatibility with the C port means we treat this as
|
||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||
|
|
@ -372,9 +371,8 @@ func (node *ForLoopTwoVariableNode) Execute(state *runtime.State) (*BlockExitPay
|
|||
}
|
||||
}
|
||||
|
||||
} else if indexMlrval.IsAbsent() {
|
||||
// Data-heterogeneity no-op
|
||||
}
|
||||
// else if indexMlrval.IsAbsent(): data-heterogeneity no-op
|
||||
|
||||
// TODO: backwards compatibility with the C port means we treat this as
|
||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||
|
|
@ -581,9 +579,8 @@ func (node *ForLoopMultivariableNode) executeOuter(
|
|||
}
|
||||
}
|
||||
|
||||
} else if mv.IsAbsent() {
|
||||
// Data-heterogeneity no-op
|
||||
}
|
||||
// else if mv.IsAbsent(): data-heterogeneity no-op
|
||||
|
||||
// TODO: backwards compatibility with the C port means we treat this as
|
||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||
|
|
@ -675,9 +672,8 @@ func (node *ForLoopMultivariableNode) executeInner(
|
|||
}
|
||||
}
|
||||
|
||||
} else if mv.IsAbsent() {
|
||||
// Data-heterogeneity no-op
|
||||
}
|
||||
// else if mv.IsAbsent(): data-heterogeneity no-op
|
||||
|
||||
// TODO: backwards compatibility with the C port means we treat this as
|
||||
// silent zero-pass. But maybe we should surface it as an error. Maybe
|
||||
|
|
|
|||
|
|
@ -33,16 +33,16 @@ func captureStdout(f func()) string {
|
|||
done := make(chan string)
|
||||
go func() {
|
||||
var buf bytes.Buffer
|
||||
io.Copy(&buf, r)
|
||||
_, _ = io.Copy(&buf, r)
|
||||
done <- buf.String()
|
||||
}()
|
||||
|
||||
f()
|
||||
|
||||
w.Close()
|
||||
_ = w.Close()
|
||||
os.Stdout = old
|
||||
s := <-done
|
||||
r.Close()
|
||||
_ = r.Close()
|
||||
return s
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -18,9 +18,9 @@ func (root *RootNode) BuildLeafNode(
|
|||
) (IEvaluable, error) {
|
||||
// The BNF uses empty slice for terminals. It may also produce reduced literal nodes
|
||||
// (float_literal, int_literal) with one child (the terminal).
|
||||
lib.InternalCodingErrorIf(astNode.Children != nil && len(astNode.Children) > 1)
|
||||
lib.InternalCodingErrorIf(len(astNode.Children) > 1)
|
||||
sval := tokenLit(astNode)
|
||||
if sval == "" && astNode.Children != nil && len(astNode.Children) == 1 {
|
||||
if sval == "" && len(astNode.Children) == 1 {
|
||||
sval = tokenLit(astNode.Children[0])
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -143,7 +143,8 @@ func (node *DirectFieldValueLvalueNode) UnassignIndexed(
|
|||
name := node.lhsFieldName.String()
|
||||
state.Inrec.Remove(name)
|
||||
} else {
|
||||
state.Inrec.RemoveIndexed(
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Inrec.RemoveIndexed(
|
||||
append([]*mlrval.Mlrval{node.lhsFieldName}, indices...),
|
||||
)
|
||||
}
|
||||
|
|
@ -254,7 +255,8 @@ func (node *IndirectFieldValueLvalueNode) UnassignIndexed(
|
|||
name := lhsFieldName.String()
|
||||
state.Inrec.Remove(name)
|
||||
} else {
|
||||
state.Inrec.RemoveIndexed(
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Inrec.RemoveIndexed(
|
||||
append([]*mlrval.Mlrval{lhsFieldName.Copy()}, indices...),
|
||||
)
|
||||
}
|
||||
|
|
@ -357,12 +359,12 @@ func (node *PositionalFieldNameLvalueNode) UnassignIndexed(
|
|||
index, ok := lhsFieldIndex.GetIntValue()
|
||||
if ok {
|
||||
state.Inrec.RemoveWithPositionalIndex(index)
|
||||
} else {
|
||||
// TODO: incorporate error-return into this API
|
||||
}
|
||||
// TODO: incorporate error-return into this API for the non-int case
|
||||
} else {
|
||||
// xxx positional
|
||||
state.Inrec.RemoveIndexed(
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Inrec.RemoveIndexed(
|
||||
append([]*mlrval.Mlrval{lhsFieldIndex}, indices...),
|
||||
)
|
||||
}
|
||||
|
|
@ -475,12 +477,12 @@ func (node *PositionalFieldValueLvalueNode) UnassignIndexed(
|
|||
index, ok := lhsFieldIndex.GetIntValue()
|
||||
if ok {
|
||||
state.Inrec.RemoveWithPositionalIndex(index)
|
||||
} else {
|
||||
// TODO: incorporate error-return into this API
|
||||
}
|
||||
// TODO: incorporate error-return into this API for the non-int case
|
||||
} else {
|
||||
// xxx positional
|
||||
state.Inrec.RemoveIndexed(
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Inrec.RemoveIndexed(
|
||||
append([]*mlrval.Mlrval{lhsFieldIndex}, indices...),
|
||||
)
|
||||
}
|
||||
|
|
@ -493,7 +495,7 @@ func (root *RootNode) BuildFullSrecLvalueNode(astNode *asts.ASTNode) (IAssignabl
|
|||
lib.InternalCodingErrorIf(astNode.Type != asts.NodeType(NodeTypeFullSrec))
|
||||
lib.InternalCodingErrorIf(astNode == nil)
|
||||
// PGPG FullSrec has empty list as children
|
||||
lib.InternalCodingErrorIf(astNode.Children != nil && len(astNode.Children) > 0)
|
||||
lib.InternalCodingErrorIf(len(astNode.Children) > 0)
|
||||
return NewFullSrecLvalueNode(), nil
|
||||
}
|
||||
|
||||
|
|
@ -556,7 +558,8 @@ func (node *FullSrecLvalueNode) UnassignIndexed(
|
|||
if indices == nil {
|
||||
state.Inrec.Clear()
|
||||
} else {
|
||||
state.Inrec.RemoveIndexed(indices)
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Inrec.RemoveIndexed(indices)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -625,7 +628,8 @@ func (node *DirectOosvarValueLvalueNode) UnassignIndexed(
|
|||
name := node.lhsOosvarName.String()
|
||||
state.Oosvars.Remove(name)
|
||||
} else {
|
||||
state.Oosvars.RemoveIndexed(
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Oosvars.RemoveIndexed(
|
||||
append([]*mlrval.Mlrval{node.lhsOosvarName}, indices...),
|
||||
)
|
||||
}
|
||||
|
|
@ -705,7 +709,8 @@ func (node *IndirectOosvarValueLvalueNode) UnassignIndexed(
|
|||
sname := lhsOosvarName.String()
|
||||
state.Oosvars.Remove(sname)
|
||||
} else {
|
||||
state.Oosvars.RemoveIndexed(
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Oosvars.RemoveIndexed(
|
||||
append([]*mlrval.Mlrval{lhsOosvarName}, indices...),
|
||||
)
|
||||
}
|
||||
|
|
@ -717,7 +722,7 @@ type FullOosvarLvalueNode struct {
|
|||
func (root *RootNode) BuildFullOosvarLvalueNode(astNode *asts.ASTNode) (IAssignable, error) {
|
||||
lib.InternalCodingErrorIf(astNode.Type != asts.NodeType(NodeTypeFullOosvar))
|
||||
lib.InternalCodingErrorIf(astNode == nil)
|
||||
lib.InternalCodingErrorIf(astNode.Children != nil && len(astNode.Children) > 0)
|
||||
lib.InternalCodingErrorIf(len(astNode.Children) > 0)
|
||||
return NewFullOosvarLvalueNode(), nil
|
||||
}
|
||||
|
||||
|
|
@ -762,7 +767,8 @@ func (node *FullOosvarLvalueNode) UnassignIndexed(
|
|||
if indices == nil {
|
||||
state.Oosvars.Clear()
|
||||
} else {
|
||||
state.Oosvars.RemoveIndexed(indices)
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = state.Oosvars.RemoveIndexed(indices)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -780,7 +786,7 @@ type LocalVariableLvalueNode struct {
|
|||
}
|
||||
|
||||
func (root *RootNode) BuildTypedeclLocalVariableLvalueNode(astNode *asts.ASTNode) (IAssignable, error) {
|
||||
lib.InternalCodingErrorIf(astNode.Children == nil || len(astNode.Children) < 2)
|
||||
lib.InternalCodingErrorIf(len(astNode.Children) < 2)
|
||||
typeNode := astNode.Children[0]
|
||||
varNode := astNode.Children[1]
|
||||
// PGPG Typedecl produces kw_int, kw_bool, etc. (no Typedecl wrapper node)
|
||||
|
|
@ -803,7 +809,7 @@ func (root *RootNode) BuildLocalVariableLvalueNode(astNode *asts.ASTNode) (IAssi
|
|||
typeName := "any"
|
||||
defineTypedAtScope := false
|
||||
// PGPG: LocalVariable is terminal (children nil or empty). Miller had typed params with Children[0]=Typedecl.
|
||||
if astNode.Children != nil && len(astNode.Children) > 0 { // typed, like 'num x = 3'
|
||||
if len(astNode.Children) > 0 { // typed, like 'num x = 3'
|
||||
typeNode := astNode.Children[0]
|
||||
lib.InternalCodingErrorIf(typeNode.Type != asts.NodeType(NodeTypeTypedecl))
|
||||
typeName = tokenLit(typeNode)
|
||||
|
|
@ -1062,7 +1068,9 @@ func (node *EnvironmentVariableLvalueNode) Assign(
|
|||
|
||||
sname := name.String()
|
||||
svalue := rvalue.String()
|
||||
os.Setenv(sname, svalue)
|
||||
if err := os.Setenv(sname, svalue); err != nil {
|
||||
return err
|
||||
}
|
||||
if sname == "TZ" {
|
||||
err := lib.SetTZFromEnv() // affects the time library; notify it
|
||||
if err != nil {
|
||||
|
|
@ -1093,7 +1101,7 @@ func (node *EnvironmentVariableLvalueNode) Unassign(
|
|||
return
|
||||
}
|
||||
|
||||
os.Unsetenv(name.String())
|
||||
_ = os.Unsetenv(name.String())
|
||||
}
|
||||
|
||||
func (node *EnvironmentVariableLvalueNode) UnassignIndexed(
|
||||
|
|
|
|||
|
|
@ -327,7 +327,9 @@ func (root *RootNode) buildPrintxStatementNode(
|
|||
|
||||
func (node *PrintStatementNode) Execute(state *runtime.State) (*BlockExitPayload, error) {
|
||||
if len(node.expressionEvaluables) == 0 {
|
||||
node.printToRedirectFunc(node.terminator, state)
|
||||
if err := node.printToRedirectFunc(node.terminator, state); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
// 5x faster than fmt.Print() separately: note that os.Stdout is
|
||||
// non-buffered in Go whereas stdout is buffered in C.
|
||||
|
|
@ -347,7 +349,9 @@ func (node *PrintStatementNode) Execute(state *runtime.State) (*BlockExitPayload
|
|||
}
|
||||
}
|
||||
buffer.WriteString(node.terminator)
|
||||
node.printToRedirectFunc(buffer.String(), state)
|
||||
if err := node.printToRedirectFunc(buffer.String(), state); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
|
@ -387,6 +391,5 @@ func (node *PrintStatementNode) printToFileOrPipe(
|
|||
}
|
||||
outputFileName := redirectorTarget.String()
|
||||
|
||||
node.outputHandlerManager.WriteString(outputString, outputFileName)
|
||||
return nil
|
||||
return node.outputHandlerManager.WriteString(outputString, outputFileName)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -368,12 +368,9 @@ func (root *RootNode) resolveFunctionCallsites() error {
|
|||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if udf == nil {
|
||||
// Unresolvable at CST-build time but perhaps a local variable. For example,
|
||||
// the UDF callsite '$z = f($x, $y)', and supposing
|
||||
// A nil udf here is unresolvable at CST-build time but perhaps a local
|
||||
// variable. For example, the UDF callsite '$z = f($x, $y)', and supposing
|
||||
// there will be 'f = func(a, b) { return a*b }' in scope at runtime.
|
||||
}
|
||||
|
||||
unresolvedFunctionCallsite.udf = udf
|
||||
}
|
||||
return nil
|
||||
|
|
|
|||
|
|
@ -563,13 +563,13 @@ func (root *RootNode) BuildUDF(
|
|||
for i, parameterASTNode := range flatParams {
|
||||
// PGPG: Parameter has one child (LocalVariable), or two (Typedecl, LocalVariable) for typed params.
|
||||
var variableName string
|
||||
var typeName string = "any"
|
||||
if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 2 {
|
||||
typeName := "any"
|
||||
if len(parameterASTNode.Children) == 2 {
|
||||
// Typedecl LocalVariable -> [Typedecl, LocalVariable]
|
||||
typeNode := parameterASTNode.Children[0]
|
||||
nameNode := parameterASTNode.Children[1]
|
||||
typeName = tokenLit(typeNode)
|
||||
if typeName == "" && typeNode.Children != nil && len(typeNode.Children) > 0 {
|
||||
if typeName == "" && len(typeNode.Children) > 0 {
|
||||
typeName = tokenLit(typeNode.Children[0])
|
||||
}
|
||||
if typeName == "" {
|
||||
|
|
@ -579,14 +579,14 @@ func (root *RootNode) BuildUDF(
|
|||
if string(nameNode.Type) == NodeTypeLocalVariable || nameNode.Type == asts.NodeType(NodeTypeLocalVariable) {
|
||||
variableName = tokenLit(nameNode)
|
||||
}
|
||||
} else if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 1 {
|
||||
} else if len(parameterASTNode.Children) == 1 {
|
||||
typeGatedParameterNameASTNode := parameterASTNode.Children[0]
|
||||
if string(typeGatedParameterNameASTNode.Type) != NodeTypeLocalVariable &&
|
||||
typeGatedParameterNameASTNode.Type != asts.NodeType(NodeTypeLocalVariable) {
|
||||
lib.InternalCodingErrorWithMessageIf(true, "expected LocalVariable as parameter name")
|
||||
}
|
||||
variableName = tokenLit(typeGatedParameterNameASTNode)
|
||||
} else if parameterASTNode.Children == nil || len(parameterASTNode.Children) == 0 {
|
||||
} else if len(parameterASTNode.Children) == 0 {
|
||||
// Direct LocalVariable (e.g. from with_prepended_children flattening)
|
||||
if string(parameterASTNode.Type) != NodeTypeLocalVariable &&
|
||||
parameterASTNode.Type != asts.NodeType(NodeTypeLocalVariable) {
|
||||
|
|
|
|||
|
|
@ -254,13 +254,13 @@ func (root *RootNode) BuildAndInstallUDS(astNode *asts.ASTNode) error {
|
|||
typeGatedParameterNames := make([]*types.TypeGatedMlrvalName, arity)
|
||||
for i, parameterASTNode := range flatParams {
|
||||
var variableName string
|
||||
var typeName string = "any"
|
||||
if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 2 {
|
||||
typeName := "any"
|
||||
if len(parameterASTNode.Children) == 2 {
|
||||
// Typedecl LocalVariable -> [Typedecl, LocalVariable]
|
||||
typeNode := parameterASTNode.Children[0]
|
||||
nameNode := parameterASTNode.Children[1]
|
||||
typeName = tokenLit(typeNode)
|
||||
if typeName == "" && typeNode.Children != nil && len(typeNode.Children) > 0 {
|
||||
if typeName == "" && len(typeNode.Children) > 0 {
|
||||
typeName = tokenLit(typeNode.Children[0])
|
||||
}
|
||||
if typeName == "" {
|
||||
|
|
@ -269,7 +269,7 @@ func (root *RootNode) BuildAndInstallUDS(astNode *asts.ASTNode) error {
|
|||
if string(nameNode.Type) == NodeTypeLocalVariable || nameNode.Type == asts.NodeType(NodeTypeLocalVariable) {
|
||||
variableName = tokenLit(nameNode)
|
||||
}
|
||||
} else if parameterASTNode.Children != nil && len(parameterASTNode.Children) == 1 {
|
||||
} else if len(parameterASTNode.Children) == 1 {
|
||||
typeGatedParameterNameASTNode := parameterASTNode.Children[0]
|
||||
lib.InternalCodingErrorIf(typeGatedParameterNameASTNode.Type != asts.NodeType(NodeTypeLocalVariable))
|
||||
variableName = tokenLit(typeGatedParameterNameASTNode)
|
||||
|
|
|
|||
|
|
@ -163,14 +163,14 @@ func processFileInPlace(
|
|||
// Get a handle with, perhaps, a recompression wrapper around it.
|
||||
wrappedHandle, isNew, err := lib.WrapOutputHandle(handle, inputFileEncoding)
|
||||
if err != nil {
|
||||
os.Remove(tempFileName)
|
||||
_ = os.Remove(tempFileName)
|
||||
return err
|
||||
}
|
||||
|
||||
// Run the Miller processing stream from the input file to the temp-output file.
|
||||
err = stream.Stream([]string{fileName}, options, recordTransformers, wrappedHandle, false)
|
||||
if err != nil {
|
||||
os.Remove(tempFileName)
|
||||
_ = os.Remove(tempFileName)
|
||||
return err
|
||||
}
|
||||
|
||||
|
|
@ -178,7 +178,7 @@ func processFileInPlace(
|
|||
if isNew {
|
||||
err = wrappedHandle.Close()
|
||||
if err != nil {
|
||||
os.Remove(tempFileName)
|
||||
_ = os.Remove(tempFileName)
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
|
@ -187,14 +187,14 @@ func processFileInPlace(
|
|||
// it must be error-checked.
|
||||
err = handle.Close()
|
||||
if err != nil {
|
||||
os.Remove(tempFileName)
|
||||
_ = os.Remove(tempFileName)
|
||||
return err
|
||||
}
|
||||
|
||||
// Rename the temp-output file on top of the input file.
|
||||
err = os.Rename(tempFileName, fileName)
|
||||
if err != nil {
|
||||
os.Remove(tempFileName)
|
||||
_ = os.Remove(tempFileName)
|
||||
return err
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -82,7 +82,7 @@ func (reader *RecordReaderCSV) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -335,9 +335,8 @@ func (reader *RecordReaderCSV) maybeConsumeComment(
|
|||
lib.InternalCodingErrorIf(len(csvRecord) != 1)
|
||||
*recordsAndContexts = append(*recordsAndContexts, types.NewOutputString(csvRecord[0], context))
|
||||
|
||||
} else /* reader.readerOptions.CommentHandling == cli.SkipComments */ {
|
||||
// discard entirely
|
||||
}
|
||||
// else, reader.readerOptions.CommentHandling == cli.SkipComments: discard entirely
|
||||
return false
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -122,7 +122,7 @@ func (reader *RecordReaderCSVLite) Read(
|
|||
errorChannel,
|
||||
downstreamDoneChannel,
|
||||
)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ func (reader *RecordReaderDCF) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -86,7 +86,7 @@ func (reader *RecordReaderDKVPNIDX) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -216,7 +216,7 @@ func recordFromNIDXLine(reader *RecordReaderDKVPNIDX, line string) (*mlrval.Mlrm
|
|||
|
||||
values := reader.fieldSplitter.Split(line)
|
||||
|
||||
var i int = 0
|
||||
i := 0
|
||||
for _, value := range values {
|
||||
i++
|
||||
str_key := strconv.Itoa(i)
|
||||
|
|
|
|||
|
|
@ -10,7 +10,8 @@ import (
|
|||
|
||||
func TestRecordFromDKVPLine(t *testing.T) {
|
||||
readerOptions := cli.DefaultReaderOptions()
|
||||
cli.FinalizeReaderOptions(&readerOptions) // compute IPS, IFS -> IPSRegex, IFSRegex
|
||||
err := cli.FinalizeReaderOptions(&readerOptions) // compute IPS, IFS -> IPSRegex, IFSRegex
|
||||
assert.Nil(t, err)
|
||||
reader, err := NewRecordReaderDKVP(&readerOptions, 1)
|
||||
assert.NotNil(t, reader)
|
||||
assert.Nil(t, err)
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ func (reader *RecordReaderDKVPX) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ func (reader *RecordReaderJSON) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -122,7 +122,7 @@ func (reader *RecordReaderPprintFixedSplit) Read(
|
|||
errorChannel,
|
||||
downstreamDoneChannel,
|
||||
)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -351,7 +351,7 @@ func (reader *RecordReaderPprintBarredOrMarkdown) Read(
|
|||
errorChannel,
|
||||
downstreamDoneChannel,
|
||||
)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -104,7 +104,7 @@ func (reader *RecordReaderTSV) Read(
|
|||
errorChannel,
|
||||
downstreamDoneChannel,
|
||||
)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -88,7 +88,7 @@ func (reader *RecordReaderXTAB) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ func (reader *RecordReaderYAML) Read(
|
|||
errorChannel <- err
|
||||
} else {
|
||||
reader.processHandle(handle, filename, &context, readerChannel, errorChannel, downstreamDoneChannel)
|
||||
handle.Close()
|
||||
_ = handle.Close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ func OpenOutboundHalfPipe(commandString string) (*os.File, error) {
|
|||
return nil, err
|
||||
}
|
||||
|
||||
go process.Wait()
|
||||
go func() { _, _ = process.Wait() }()
|
||||
|
||||
return writePipe, nil
|
||||
}
|
||||
|
|
@ -87,7 +87,7 @@ func OpenInboundHalfPipe(commandString string) (*os.File, error) {
|
|||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr: %v\n", err)
|
||||
}
|
||||
readPipe.Close()
|
||||
_ = readPipe.Close()
|
||||
}(process, readPipe)
|
||||
|
||||
return readPipe, nil
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ func LoadStringsFromDir(dirname string, extension string) ([]string, error) {
|
|||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
defer func() { _ = f.Close() }()
|
||||
|
||||
names, err := f.Readdirnames(-1)
|
||||
if err != nil {
|
||||
|
|
@ -80,7 +80,7 @@ func ReadCSVHeader(filename string) ([]string, error) {
|
|||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer handle.Close()
|
||||
defer func() { _ = handle.Close() }()
|
||||
csvReader := csv.NewReader(handle)
|
||||
header, err := csvReader.Read()
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -755,10 +755,7 @@ func (mlrmap *Mlrmap) Label(newNames []string) {
|
|||
|
||||
i := 0
|
||||
numNewNames := len(newNames)
|
||||
for {
|
||||
if i >= numNewNames {
|
||||
break
|
||||
}
|
||||
for i < numNewNames {
|
||||
pe := mlrmap.pop()
|
||||
if pe == nil {
|
||||
break
|
||||
|
|
|
|||
|
|
@ -179,7 +179,8 @@ func (mlrmap *Mlrmap) CopyUnflattened(
|
|||
baseIndex := arrayval[0].String()
|
||||
affectedBaseIndices[baseIndex] = true
|
||||
// Use PutIndexed to assign $x["a"] = 7, or $x["b"] = 8, etc.
|
||||
other.PutIndexed(
|
||||
// Unflattening is best-effort: this API has no error return.
|
||||
_ = other.PutIndexed(
|
||||
CopyMlrvalArray(arrayval),
|
||||
unflattenTerminal(pe.Value).Copy(),
|
||||
)
|
||||
|
|
@ -226,7 +227,8 @@ func (mlrmap *Mlrmap) CopyUnflattenFields(
|
|||
baseIndex := arrayval[0].String()
|
||||
if fieldNameSet[baseIndex] {
|
||||
// Use PutIndexed to assign $x["a"] = 7, or $x["b"] = 8, etc.
|
||||
other.PutIndexed(
|
||||
// Unflattening is best-effort: this API has no error return.
|
||||
_ = other.PutIndexed(
|
||||
CopyMlrvalArray(arrayval),
|
||||
unflattenTerminal(pe.Value).Copy(),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -197,7 +197,6 @@ func mlrvalToYAMLNode(mv *Mlrval) (*yaml.Node, error) {
|
|||
case MT_ARRAY:
|
||||
arr := mv.GetArray()
|
||||
seqNode := &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||
if arr != nil {
|
||||
for _, elem := range arr {
|
||||
v, err := mlrvalToYAMLNode(elem)
|
||||
if err != nil {
|
||||
|
|
@ -205,7 +204,6 @@ func mlrvalToYAMLNode(mv *Mlrval) (*yaml.Node, error) {
|
|||
}
|
||||
seqNode.Content = append(seqNode.Content, v)
|
||||
}
|
||||
}
|
||||
return seqNode, nil
|
||||
case MT_MAP:
|
||||
m := mv.GetMap()
|
||||
|
|
|
|||
|
|
@ -105,7 +105,8 @@ func channelWriterHandleBatch(
|
|||
}
|
||||
|
||||
if writerOptions.FlushOnEveryRecord {
|
||||
bufferedOutputStream.Flush()
|
||||
// bufio.Writer errors are sticky; the final Flush in pkg/stream is checked
|
||||
_ = bufferedOutputStream.Flush()
|
||||
}
|
||||
|
||||
} else {
|
||||
|
|
|
|||
|
|
@ -500,7 +500,9 @@ func (handler *FileOutputHandler) Close() (retval error) {
|
|||
return retval
|
||||
}
|
||||
|
||||
handler.bufferedOutputStream.Flush()
|
||||
if err := handler.bufferedOutputStream.Flush(); err != nil {
|
||||
return err
|
||||
}
|
||||
if handler.closeable {
|
||||
return handler.handle.Close()
|
||||
} // e.g. stdout
|
||||
|
|
|
|||
|
|
@ -73,11 +73,14 @@ func (writer *RecordWriterCSV) Write(
|
|||
fields[i] = pe.Key
|
||||
i++
|
||||
}
|
||||
writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, true, writer.quoteAll)
|
||||
err := writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, true, writer.quoteAll)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writer.needToPrintHeader = false
|
||||
}
|
||||
|
||||
var outputNF int64 = outrec.FieldCount
|
||||
outputNF := outrec.FieldCount
|
||||
if outputNF < writer.firstRecordNF {
|
||||
outputNF = writer.firstRecordNF
|
||||
}
|
||||
|
|
@ -103,7 +106,5 @@ func (writer *RecordWriterCSV) Write(
|
|||
fields[i] = ""
|
||||
}
|
||||
|
||||
writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, false, writer.quoteAll)
|
||||
|
||||
return nil
|
||||
return writer.WriteCSVRecordMaybeColorized(fields, bufferedOutputStream, outputIsStdout, false, writer.quoteAll)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@ package output
|
|||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/johnkerl/miller/v6/pkg/cli"
|
||||
|
|
@ -211,7 +210,8 @@ func (writer *RecordWriterPPRINT) writeHeterogenousListNonBarred(
|
|||
}
|
||||
|
||||
if writer.writerOptions.FlushOnEveryRecord {
|
||||
bufferedOutputStream.Flush()
|
||||
// bufio.Writer errors are sticky; the final Flush in pkg/stream is checked
|
||||
_ = bufferedOutputStream.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -351,7 +351,7 @@ func (writer *RecordWriterPPRINT) writeHeterogenousListBarred(
|
|||
bufferedOutputStream.WriteString(colorizer.MaybeColorizeValue(s, outputIsStdout))
|
||||
}
|
||||
if pe.Next != nil {
|
||||
bufferedOutputStream.WriteString(fmt.Sprint(bc.verticalMiddle))
|
||||
bufferedOutputStream.WriteString(bc.verticalMiddle)
|
||||
} else {
|
||||
bufferedOutputStream.WriteString(bc.verticalEnd)
|
||||
bufferedOutputStream.WriteString(writer.writerOptions.ORS)
|
||||
|
|
@ -373,7 +373,8 @@ func (writer *RecordWriterPPRINT) writeHeterogenousListBarred(
|
|||
}
|
||||
|
||||
if writer.writerOptions.FlushOnEveryRecord {
|
||||
bufferedOutputStream.Flush()
|
||||
// bufio.Writer errors are sticky; the final Flush in pkg/stream is checked
|
||||
_ = bufferedOutputStream.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -75,7 +75,7 @@ func (writer *RecordWriterTSV) Write(
|
|||
writer.needToPrintHeader = false
|
||||
}
|
||||
|
||||
var outputNF int64 = outrec.FieldCount
|
||||
outputNF := outrec.FieldCount
|
||||
if outputNF < writer.firstRecordNF {
|
||||
outputNF = writer.firstRecordNF
|
||||
}
|
||||
|
|
|
|||
|
|
@ -454,7 +454,9 @@ func (frame *StackFrame) setIndexed(
|
|||
leadingIndex := indices[0]
|
||||
if leadingIndex.IsString() || leadingIndex.IsInt() {
|
||||
newval := mlrval.FromMap(mlrval.NewMlrmap())
|
||||
newval.PutIndexed(indices, mv)
|
||||
if err := newval.PutIndexed(indices, mv); err != nil {
|
||||
return err
|
||||
}
|
||||
return frame.set(stackVariable, newval)
|
||||
}
|
||||
return fmt.Errorf(
|
||||
|
|
@ -485,5 +487,6 @@ func (frame *StackFrame) unsetIndexed(
|
|||
if value == nil {
|
||||
return
|
||||
}
|
||||
value.RemoveIndexed(indices)
|
||||
// unset of a non-existent path is a no-op
|
||||
_ = value.RemoveIndexed(indices)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -101,7 +101,9 @@ func Stream(
|
|||
}
|
||||
}
|
||||
|
||||
bufferedOutputStream.Flush()
|
||||
if err := bufferedOutputStream.Flush(); err != nil && retval == nil {
|
||||
retval = err
|
||||
}
|
||||
|
||||
return retval
|
||||
}
|
||||
|
|
|
|||
|
|
@ -821,12 +821,8 @@ func helpByExactSearch(things []string) bool {
|
|||
// We need to look various places, e.g. "sec2gmt" is the name of a verb as well
|
||||
// as a DSL function.
|
||||
func helpByExactSearchOne(thing string) bool {
|
||||
found := false
|
||||
|
||||
// flag
|
||||
if cli.FLAG_TABLE.ShowHelpForFlagWithName(thing) {
|
||||
found = true
|
||||
}
|
||||
found := cli.FLAG_TABLE.ShowHelpForFlagWithName(thing)
|
||||
|
||||
// verb
|
||||
if transformers.ShowHelpForTransformer(thing) {
|
||||
|
|
@ -857,12 +853,8 @@ func helpByApproximateSearch(things []string) bool {
|
|||
}
|
||||
|
||||
func helpByApproximateSearchOne(thing string) bool {
|
||||
found := false
|
||||
|
||||
// flag
|
||||
if cli.FLAG_TABLE.ShowHelpForFlagApproximateWithName(thing) {
|
||||
found = true
|
||||
}
|
||||
found := cli.FLAG_TABLE.ShowHelpForFlagApproximateWithName(thing)
|
||||
|
||||
// verb
|
||||
if transformers.ShowHelpForTransformerApproximate(thing) {
|
||||
|
|
|
|||
|
|
@ -164,7 +164,8 @@ var whichStopwords = map[string]bool{
|
|||
// single-character tokens and stopwords.
|
||||
func whichTokenize(query string) []string {
|
||||
words := strings.FieldsFunc(strings.ToLower(query), func(r rune) bool {
|
||||
return !('a' <= r && r <= 'z') && !('0' <= r && r <= '9') && r != '-' && r != '_'
|
||||
isWordRune := ('a' <= r && r <= 'z') || ('0' <= r && r <= '9') || r == '-' || r == '_'
|
||||
return !isWordRune
|
||||
})
|
||||
var tokens []string
|
||||
seen := map[string]bool{}
|
||||
|
|
|
|||
|
|
@ -72,8 +72,8 @@ func RunDiffCommandOnStrings(
|
|||
) {
|
||||
actualOutputFileName := lib.WriteTempFileOrDie(actualOutput)
|
||||
expectedOutputFileName := lib.WriteTempFileOrDie(expectedOutput)
|
||||
defer os.Remove(actualOutputFileName)
|
||||
defer os.Remove(expectedOutputFileName)
|
||||
defer func() { _ = os.Remove(actualOutputFileName) }()
|
||||
defer func() { _ = os.Remove(expectedOutputFileName) }()
|
||||
|
||||
// This is diff or fc
|
||||
diffRunArray := platform.GetDiffRunArray(actualOutputFileName, expectedOutputFileName)
|
||||
|
|
|
|||
|
|
@ -149,16 +149,16 @@ func (rt *RegTester) Execute(
|
|||
) bool {
|
||||
// Don't let the current user's settings affect expected results
|
||||
for _, name := range envVarsToUnset {
|
||||
os.Unsetenv(name)
|
||||
_ = os.Unsetenv(name)
|
||||
}
|
||||
// If there is an accessible .mlrrc file, we don't want it to be read for the regression test.
|
||||
os.Setenv("MLRRC", "__none__")
|
||||
_ = os.Setenv("MLRRC", "__none__")
|
||||
|
||||
// This is important for multi-platform regression testing, wherein default floating-point
|
||||
// output format has varying numbers of decimal places between the platform where
|
||||
// the expected results were generated, and the platform where the actual values are being
|
||||
// computed. For regression-test we OFMT from an environment variable.
|
||||
os.Setenv("MLR_OFMT", "%.8f")
|
||||
_ = os.Setenv("MLR_OFMT", "%.8f")
|
||||
|
||||
rt.resetCounts()
|
||||
|
||||
|
|
@ -328,7 +328,7 @@ func (rt *RegTester) hasCaseSubdirectories(
|
|||
fmt.Printf("%s: %v\n", dirName, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer f.Close()
|
||||
defer func() { _ = f.Close() }()
|
||||
|
||||
names, err := f.Readdirnames(-1)
|
||||
if err != nil {
|
||||
|
|
@ -461,12 +461,12 @@ func (rt *RegTester) executeSingleCmdFile(
|
|||
if verbosityLevel >= 3 {
|
||||
fmt.Printf("SETENV %s=%s\n", key, value)
|
||||
}
|
||||
os.Setenv(key, value)
|
||||
_ = os.Setenv(key, value)
|
||||
}
|
||||
// This is so 'mlr' files can find the case-directory if they need it --
|
||||
// typically, for redirected emit/dump/etc statements where we want
|
||||
// the redirected-to files to be written into the case-directory.
|
||||
os.Setenv("CASEDIR", caseDir)
|
||||
_ = os.Setenv("CASEDIR", caseDir)
|
||||
|
||||
// Copy any files requested by the test. (Most don't; some do, e.g. those
|
||||
// which test the write-in-place logic of mlr -I.)
|
||||
|
|
@ -496,9 +496,9 @@ func (rt *RegTester) executeSingleCmdFile(
|
|||
if verbosityLevel >= 3 {
|
||||
fmt.Printf("UNSETENV %s\n", key)
|
||||
}
|
||||
os.Setenv(key, "")
|
||||
_ = os.Setenv(key, "")
|
||||
}
|
||||
os.Setenv("CASEDIR", "")
|
||||
_ = os.Setenv("CASEDIR", "")
|
||||
|
||||
// The .postcmp needn't exist (most test cases don't have one) in which case
|
||||
// the returned map will be empty.
|
||||
|
|
@ -542,7 +542,7 @@ func (rt *RegTester) executeSingleCmdFile(
|
|||
// Write the .should-fail file
|
||||
if actualExitCode == 0 {
|
||||
// Remove it, if it exists.
|
||||
os.Remove(expectFailFileName)
|
||||
_ = os.Remove(expectFailFileName)
|
||||
} else {
|
||||
err = rt.storeFile(expectFailFileName, "")
|
||||
if err != nil {
|
||||
|
|
@ -716,7 +716,7 @@ func (rt *RegTester) executeSingleCmdFile(
|
|||
// Clean up any requested file-copies so that we're git-clean after the regression-test run.
|
||||
for _, pair := range preCopySrcDestPairs {
|
||||
dst := pair.second
|
||||
os.Remove(dst)
|
||||
_ = os.Remove(dst)
|
||||
if verbosityLevel >= 3 {
|
||||
fmt.Printf("%s: clean up %s\n", cmdFilePath, dst)
|
||||
}
|
||||
|
|
@ -725,7 +725,7 @@ func (rt *RegTester) executeSingleCmdFile(
|
|||
// Clean up any extra output files so that we're git-clean after the regression-test run.
|
||||
for _, pair := range postCompareExpectedActualPairs {
|
||||
actualFileName := pair.second
|
||||
os.Remove(actualFileName)
|
||||
_ = os.Remove(actualFileName)
|
||||
if verbosityLevel >= 3 {
|
||||
fmt.Printf("%s: clean up %s\n", cmdFilePath, actualFileName)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -147,8 +147,14 @@ func ReplMain(args []string) int {
|
|||
}
|
||||
}
|
||||
|
||||
cli.FinalizeReaderOptions(&options.ReaderOptions)
|
||||
cli.FinalizeWriterOptions(&options.WriterOptions)
|
||||
if err := cli.FinalizeReaderOptions(&options.ReaderOptions); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", replName, err)
|
||||
return 1
|
||||
}
|
||||
if err := cli.FinalizeWriterOptions(&options.WriterOptions); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", replName, err)
|
||||
return 1
|
||||
}
|
||||
|
||||
// --auto-flatten is on by default. But if input and output formats are both JSON,
|
||||
// then we don't need to actually do anything. See also mlrcli_parse.go.
|
||||
|
|
@ -186,7 +192,11 @@ func ReplMain(args []string) int {
|
|||
os.Exit(1)
|
||||
}
|
||||
|
||||
repl.bufferedRecordOutputStream.Flush()
|
||||
err = repl.bufferedRecordOutputStream.Flush()
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", repl.replName, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
err = repl.closeBufferedOutputStream()
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", repl.replName, err)
|
||||
|
|
|
|||
|
|
@ -200,9 +200,7 @@ func handleOpen(repl *Repl, args []string) bool {
|
|||
// something instead of waiting to show them an error only when they type
|
||||
// ':read'.
|
||||
func openFilesPreCheck(repl *Repl, args []string) bool {
|
||||
if len(args) == 0 {
|
||||
// Zero file names is stdin, which is readable
|
||||
}
|
||||
// Zero file names is stdin, which is readable.
|
||||
for _, arg := range args {
|
||||
fileInfo, err := os.Stat(arg)
|
||||
if err != nil {
|
||||
|
|
@ -554,8 +552,9 @@ func skipOrProcessRecord(
|
|||
// Strings to be printed from put/filter DSL print/dump/etc statements.
|
||||
if recordAndContext.Record == nil {
|
||||
if processingNotSkipping {
|
||||
repl.bufferedRecordOutputStream.WriteString(recordAndContext.OutputString)
|
||||
repl.bufferedRecordOutputStream.Flush()
|
||||
// Interactive terminal output: nothing useful to do on write failure
|
||||
_, _ = repl.bufferedRecordOutputStream.WriteString(recordAndContext.OutputString)
|
||||
_ = repl.bufferedRecordOutputStream.Flush()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
|
@ -617,8 +616,12 @@ func writeRecord(repl *Repl, outrec *mlrval.Mlrmap) {
|
|||
}
|
||||
}
|
||||
// Write and flush immediately for REPL output.
|
||||
repl.recordWriter.Write(outrec, nil, repl.bufferedRecordOutputStream, true /*outputIsStdout*/)
|
||||
repl.bufferedRecordOutputStream.Flush()
|
||||
err := repl.recordWriter.Write(outrec, nil, repl.bufferedRecordOutputStream, true /*outputIsStdout*/)
|
||||
if err != nil {
|
||||
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||
return
|
||||
}
|
||||
_ = repl.bufferedRecordOutputStream.Flush()
|
||||
}
|
||||
|
||||
func usageReadWrite(repl *Repl) {
|
||||
|
|
@ -642,7 +645,9 @@ func usageRedirectWrite(repl *Repl) {
|
|||
func handleRedirectWrite(repl *Repl, args []string) bool {
|
||||
args = args[1:] // strip off verb
|
||||
if len(args) == 0 {
|
||||
repl.closeBufferedOutputStream()
|
||||
if err := repl.closeBufferedOutputStream(); err != nil {
|
||||
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||
}
|
||||
repl.setBufferedOutputStream("(stdout)", os.Stdout)
|
||||
return true
|
||||
}
|
||||
|
|
@ -665,7 +670,9 @@ func handleRedirectWrite(repl *Repl, args []string) bool {
|
|||
}
|
||||
fmt.Printf("Redirecting record output to \"%s\"\n", filename)
|
||||
|
||||
repl.closeBufferedOutputStream()
|
||||
if err := repl.closeBufferedOutputStream(); err != nil {
|
||||
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||
}
|
||||
repl.setBufferedOutputStream(filename, handle)
|
||||
|
||||
return true
|
||||
|
|
@ -695,7 +702,9 @@ func handleRedirectAppend(repl *Repl, args []string) bool {
|
|||
}
|
||||
fmt.Printf("Redirecting record output to \"%s\"\n", filename)
|
||||
|
||||
repl.closeBufferedOutputStream()
|
||||
if err := repl.closeBufferedOutputStream(); err != nil {
|
||||
fmt.Printf("mlr %s: %v\n", repl.replName, err)
|
||||
}
|
||||
repl.setBufferedOutputStream(filename, handle)
|
||||
|
||||
return true
|
||||
|
|
@ -874,11 +883,7 @@ func handleHelp(repl *Repl, args []string) bool {
|
|||
}
|
||||
|
||||
func handleHelpFindSingle(repl *Repl, arg string) {
|
||||
foundAny := false
|
||||
|
||||
if cst.TryUsageForKeywordApproximate(arg) {
|
||||
foundAny = true
|
||||
}
|
||||
foundAny := cst.TryUsageForKeywordApproximate(arg)
|
||||
|
||||
if cst.BuiltinFunctionManagerInstance.TryListBuiltinFunctionUsageApproximate(arg) {
|
||||
foundAny = true
|
||||
|
|
|
|||
|
|
@ -123,8 +123,14 @@ func ScriptMain(args []string) int {
|
|||
scriptUsage(scriptName, os.Stderr, 1)
|
||||
}
|
||||
|
||||
cli.FinalizeReaderOptions(&options.ReaderOptions)
|
||||
cli.FinalizeWriterOptions(&options.WriterOptions)
|
||||
if err := cli.FinalizeReaderOptions(&options.ReaderOptions); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", scriptName, err)
|
||||
return 1
|
||||
}
|
||||
if err := cli.FinalizeWriterOptions(&options.WriterOptions); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr %s: %v\n", scriptName, err)
|
||||
return 1
|
||||
}
|
||||
options.WriterOptions.AutoFlatten = cli.DecideFinalFlatten(&options.WriterOptions)
|
||||
options.WriterOptions.AutoUnflatten = cli.DecideFinalUnflatten(options, [][]string{})
|
||||
|
||||
|
|
@ -143,7 +149,11 @@ func ScriptMain(args []string) int {
|
|||
os.Exit(1)
|
||||
}
|
||||
|
||||
scr.bufferedRecordOutputStream.Flush()
|
||||
err = scr.bufferedRecordOutputStream.Flush()
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr script: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
err = scr.closeBufferedOutputStream()
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "mlr script: %v\n", err)
|
||||
|
|
|
|||
|
|
@ -126,9 +126,10 @@ func (scr *Script) run() error {
|
|||
}
|
||||
|
||||
if rac.Record == nil {
|
||||
// Output string from print/dump etc
|
||||
bufferedOutput.WriteString(rac.OutputString)
|
||||
bufferedOutput.Flush()
|
||||
// Output string from print/dump etc; terminal output, nothing
|
||||
// useful to do on write failure
|
||||
_, _ = bufferedOutput.WriteString(rac.OutputString)
|
||||
_ = bufferedOutput.Flush()
|
||||
continue
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -40,14 +40,14 @@ func captureUsageFunc(usageFunc TransformerUsageFunc) string {
|
|||
done := make(chan string)
|
||||
go func() {
|
||||
var buf bytes.Buffer
|
||||
io.Copy(&buf, r)
|
||||
_, _ = io.Copy(&buf, r)
|
||||
done <- buf.String()
|
||||
}()
|
||||
|
||||
usageFunc(w)
|
||||
w.Close()
|
||||
_ = w.Close()
|
||||
s := <-done
|
||||
r.Close()
|
||||
_ = r.Close()
|
||||
return s
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -253,7 +253,9 @@ func transformerJoinParseCLI(
|
|||
}
|
||||
}
|
||||
|
||||
cli.FinalizeReaderOptions(&opts.joinFlagOptions.ReaderOptions)
|
||||
if err := cli.FinalizeReaderOptions(&opts.joinFlagOptions.ReaderOptions); err != nil {
|
||||
return nil, cli.VerbErrorf(verb, "%v", err)
|
||||
}
|
||||
|
||||
if opts.leftFileName == "" {
|
||||
return nil, cli.VerbErrorf(verb, "need left file name")
|
||||
|
|
|
|||
|
|
@ -192,7 +192,7 @@ func transformerPutOrFilterParseCLI(
|
|||
verb := args[argi]
|
||||
argi++
|
||||
|
||||
var dslStrings []string = []string{}
|
||||
dslStrings := []string{}
|
||||
haveDSLStringsHere := false
|
||||
echoDSLString := false
|
||||
printASTAsTree := false
|
||||
|
|
@ -340,7 +340,9 @@ func transformerPutOrFilterParseCLI(
|
|||
}
|
||||
}
|
||||
|
||||
cli.FinalizeWriterOptions(&options.WriterOptions)
|
||||
if err := cli.FinalizeWriterOptions(&options.WriterOptions); err != nil {
|
||||
return nil, cli.VerbErrorf(verb, "%v", err)
|
||||
}
|
||||
|
||||
// If they've used either of 'mlr put -f {filename}' or 'mlr put -e
|
||||
// {expression}' then that specifies their DSL expression. But if they've
|
||||
|
|
|
|||
|
|
@ -88,17 +88,17 @@ func transformerSplitParseCLI(
|
|||
argi++
|
||||
|
||||
var n int64 = 0
|
||||
var doMod bool = false
|
||||
var doSize bool = false
|
||||
doMod := false
|
||||
doSize := false
|
||||
var groupByFieldNames []string = nil
|
||||
var emitDownstream bool = false
|
||||
var escapeFileNameCharacters bool = true
|
||||
var fileNamePartJoiner string = splitDefaultFileNamePartJoiner
|
||||
var doAppend bool = false
|
||||
var outputFileNamePrefix string = splitDefaultOutputFileNamePrefix
|
||||
var outputFileNameSuffix string = "uninit"
|
||||
emitDownstream := false
|
||||
escapeFileNameCharacters := true
|
||||
fileNamePartJoiner := splitDefaultFileNamePartJoiner
|
||||
doAppend := false
|
||||
outputFileNamePrefix := splitDefaultOutputFileNamePrefix
|
||||
outputFileNameSuffix := "uninit"
|
||||
haveOutputFileNameSuffix := false
|
||||
var outputFolder string = ""
|
||||
outputFolder := ""
|
||||
|
||||
var localOptions *cli.TOptions = nil
|
||||
if mainOptions != nil {
|
||||
|
|
@ -200,7 +200,9 @@ func transformerSplitParseCLI(
|
|||
return nil, cli.VerbErrorf(verb, "-n, -g, and -s are mutually exclusive")
|
||||
}
|
||||
|
||||
cli.FinalizeWriterOptions(&localOptions.WriterOptions)
|
||||
if err := cli.FinalizeWriterOptions(&localOptions.WriterOptions); err != nil {
|
||||
return nil, cli.VerbErrorf(verb, "%v", err)
|
||||
}
|
||||
if !haveOutputFileNameSuffix {
|
||||
outputFileNameSuffix = localOptions.WriterOptions.OutputFileFormat
|
||||
}
|
||||
|
|
|
|||
|
|
@ -98,7 +98,9 @@ func transformerTeeParseCLI(
|
|||
}
|
||||
}
|
||||
|
||||
cli.FinalizeWriterOptions(&localOptions.WriterOptions)
|
||||
if err := cli.FinalizeWriterOptions(&localOptions.WriterOptions); err != nil {
|
||||
return nil, cli.VerbErrorf(verbNameTee, "%v", err)
|
||||
}
|
||||
|
||||
// Get the filename/command from the command line, after the flags
|
||||
if argi >= argc {
|
||||
|
|
|
|||
|
|
@ -87,6 +87,118 @@ nil check before len), 3 QF1007 (merge conditional assignment), 3 QF1006 (lift i
|
|||
- `os.Setenv`/`os.Unsetenv` — low-risk but cheap to fix
|
||||
2. `staticcheck` style — 34 findings, mostly mechanical (type inference, nil-check cleanup, etc.)
|
||||
|
||||
## IMPORTANT: issue counts before round 5 were capped
|
||||
|
||||
All counts above (the "134 baseline", the "84 remaining") were taken with golangci-lint's
|
||||
**default `max-same-issues=3`**, which reports at most 3 findings per identical message text.
|
||||
Running with `--max-same-issues=0 --max-issues-per-linter=0` on the same tree shows the true
|
||||
backlog before round 5 was **1271 issues: 1202 errcheck, 69 staticcheck** (not 84/50/34).
|
||||
CI shares this cap, so fixing the visible 3 of a kind just surfaces the next 3 — whack-a-mole.
|
||||
All counts from here on are uncapped.
|
||||
|
||||
## Round 5 (branch `johnkerl/lint5`, 2026-07-03): staticcheck batch — DONE
|
||||
|
||||
Fixed all 69 staticcheck findings; **staticcheck is now at zero** (uncapped). `make check`
|
||||
passes (4676 regression cases). Defers errcheck to rounds 6+; this inverts the priority order
|
||||
above, deliberately: the staticcheck fixes are low-risk and retire an entire linter, while the
|
||||
errcheck sites need per-site judgment and a few of them (see rounds 6+ below) may be real
|
||||
user-facing bugs deserving focused PRs.
|
||||
|
||||
What was fixed (uncapped counts):
|
||||
|
||||
1. **ST1023 + QF1011 — omit explicit type (37):** `pkg/bifs/arithmetic.go` (20: all min/max
|
||||
helpers), `pkg/bifs/bits.go` (2), `pkg/dsl/cst/udf.go`, `pkg/dsl/cst/uds.go`,
|
||||
`pkg/input/record_reader_dkvp_nidx.go`, `pkg/output/record_writer_csv.go`,
|
||||
`pkg/output/record_writer_tsv.go`, `pkg/transformers/put_or_filter.go`,
|
||||
`pkg/transformers/split.go` (9, whole option-declaration block for consistency).
|
||||
2. **S1009 + S1031 — redundant nil checks (14+1):** `pkg/dsl/cst/{blocks,evaluable,leaves,lvalues,udf,uds}.go`,
|
||||
`pkg/mlrval/mlrval_yaml.go`. Also fixed two unflagged-but-identical adjacent patterns
|
||||
(`udf.go`/`uds.go` inner `typeNode.Children` check, `leaves.go:23`) for consistency.
|
||||
3. **QF1007 — merge conditional assignment into declaration (3):**
|
||||
`found := false; if cond { found = true }` → `found := cond` in
|
||||
`pkg/terminals/help/entry.go` (×2) and `pkg/terminals/repl/verbs.go`. Only the first
|
||||
conditional merges; the subsequent `if ... { found = true }` blocks must stay as-is since
|
||||
each callee is invoked for its help-printing side effect (no `||` short-circuiting).
|
||||
4. **QF1006 — lift break into loop condition (3):** `pkg/bifs/relative_time.go` (×2) →
|
||||
`for remainingInput != "" {`; `pkg/mlrval/mlrmap_accessors.go` (`Mlrmap.Label`) →
|
||||
`for i < numNewNames {` (interior `pe == nil` break stays).
|
||||
5. **QF1001 — De Morgan (3):** `pkg/dsl/cst/builtin_functions.go:878,960` →
|
||||
`btype != mlrval.MT_ABSENT && btype != mlrval.MT_BOOL`.
|
||||
`pkg/terminals/help/entry_which.go:167`: mechanical De Morgan reads worse, and staticcheck
|
||||
also flags a merely-outermost negation `!((...) || (...))`; settled on naming the predicate
|
||||
(`isWordRune := ...; return !isWordRune`), which staticcheck accepts and reads best.
|
||||
6. **SA9003 — empty branches (9):** all were no-op branches existing only to hold a comment;
|
||||
deleted the branch, kept the comment adjacent. `pkg/dsl/cst/for.go` (×4 data-heterogeneity
|
||||
no-ops), `pkg/dsl/cst/lvalues.go` (×2 TODO-comment else branches), `pkg/dsl/cst/root.go`
|
||||
(nil-udf explainer), `pkg/input/record_reader_csv.go` (SkipComments else),
|
||||
`pkg/terminals/repl/verbs.go` (zero-filenames-is-stdin).
|
||||
|
||||
After this round: **1202 issues remain, all errcheck** (uncapped).
|
||||
|
||||
## Round 6 (branch `johnkerl/lint5`, 2026-07-03): errcheck — DONE
|
||||
|
||||
**golangci-lint now reports 0 issues** (uncapped) on `./cmd/mlr ./pkg/...`. `make check` passes.
|
||||
Done in the same PR as round 5. True pre-round breakdown by callee: 893 `Fprintf` + 31
|
||||
`Fprintln` + 25 `Fprint` (= 949 `fmt.Fprint*`, of which ~894 were `pkg/transformers` usage
|
||||
printers), 140 `WriteString`, 28 `Close`, 12 `Set`, 10 `Remove`, 10 `Flush`, 9 `RemoveIndexed`,
|
||||
8 `Setenv`, 5 `Write`, 9 `Finalize{Reader,Writer}Options`, 3 `PutIndexed`, plus a small tail.
|
||||
|
||||
### 6a — `.golangci.yml` with errcheck exclusions (~1090 findings)
|
||||
|
||||
Added `.golangci.yml` (picked up automatically by the CI action) with
|
||||
`errcheck.exclude-functions` for `fmt.Fprint/Fprintf/Fprintln` (usage/error printers),
|
||||
`(*bufio.Writer).Write/WriteString` (sticky errors, surface at the now-checked final Flush),
|
||||
and `(*strings.Builder).WriteString` (documented never to fail). Also pinned
|
||||
`max-issues-per-linter: 0` and `max-same-issues: 0` so CI reports true counts from now on.
|
||||
Caveat found: the bufio exclusion doesn't match calls through struct fields
|
||||
(e.g. `repl.bufferedRecordOutputStream.WriteString`) — those got explicit `_ =` instead.
|
||||
|
||||
### 6b — propagated: real error paths (~30 findings)
|
||||
|
||||
- `cli.Finalize{Reader,Writer}Options` (9): join/put-or-filter/split/tee verb constructors now
|
||||
return the error (`mlr join -i badformat` now exits 1 with "unrecognized input format"
|
||||
instead of silently proceeding with wrong separators — verified end-to-end); repl/script
|
||||
entry points print to stderr and exit; the unit-test site asserts nil.
|
||||
- `pkg/stream/stream.go` final `Flush` → propagated into `retval` (full disk / closed pipe no
|
||||
longer exits 0 silently).
|
||||
- DSL `emit`/`print`/`dump` redirect writes (11): recursive emit calls, `printToRedirectFunc`,
|
||||
`dumpToRedirectFunc`, and both `outputHandlerManager.WriteString` sites now propagate through
|
||||
`Execute`, matching their sibling branches which already did.
|
||||
- `pkg/output/record_writer_csv.go` `WriteCSVRecordMaybeColorized` (2) → propagated through
|
||||
`IRecordWriter.Write`, whose error the channel writer already reports.
|
||||
- `pkg/output/file_output_handlers.go` close-time `Flush` → propagated.
|
||||
- `pkg/runtime/stack.go` `PutIndexed` on fresh map → propagated (setIndexed returns error).
|
||||
- `ENV[...]` assignment `os.Setenv` → propagated through `Assign`.
|
||||
- REPL `writeRecord`: `recordWriter.Write` error now printed to the terminal (e.g. CSV
|
||||
schema-change errors were silently swallowed in the REPL); `closeBufferedOutputStream` on
|
||||
`:>` / `:>>` redirect switches now prints on error. Flush-before-close at repl/script exit
|
||||
now checked like the close next to it.
|
||||
- `pkg/auxents/termcvt.go`: write-side `ostream.Close()` before the rename-over-original now
|
||||
checked (had a `TODO: check return status`); `ostream.Write` in termcvt/unhex now exits on
|
||||
error like the surrounding error handling.
|
||||
|
||||
### 6c — explicit `_ =` ignores (~65 findings)
|
||||
|
||||
- Unset-style DSL/runtime paths (9 `RemoveIndexed`, 1 `Unsetenv`, in `lvalues.go`/`stack.go`):
|
||||
unset of a non-existent path is a no-op by design; the enclosing `Unassign` API has no error
|
||||
return. Commented at each site.
|
||||
- `Mlrmap` unflatten `PutIndexed` (2): best-effort API with no error return; commented.
|
||||
- Mid-stream `FlushOnEveryRecord` flushes (pprint, channel writer): bufio errors are sticky and
|
||||
the final Flush in `pkg/stream` is checked; commented.
|
||||
- Read-side `Close` (input readers ×10, auxents ×4, readfiles ×2, halfpipe, mlrrc, option_parse
|
||||
`--load`, CPU-profile handle), `go process.Wait()` in halfpipe.
|
||||
- Init-time strftime `ss.Set` registrations (12) in `pkg/bifs/datetime.go` (constant specs).
|
||||
- In-memory usage-capture pipes (`io.Copy`/`Close`) in `keyword_usage_json.go` and
|
||||
`aaa_transformer_json.go`.
|
||||
- regtest harness `os.Setenv`/`Unsetenv`/`Remove` (14) and `entrypoint.go` temp-file `os.Remove`
|
||||
on error paths (5).
|
||||
- REPL/script terminal `WriteString`/`Flush` where the config exclusion can't reach (field
|
||||
access); commented.
|
||||
|
||||
Also fixed in passing: staticcheck QF1012 in `record_writer_pprint.go` (surfaced once the
|
||||
errcheck finding on the same line was excluded): `WriteString(fmt.Sprint(x))` where `x` is
|
||||
already a string → `WriteString(x)`; dropped the then-unused `fmt` import.
|
||||
|
||||
## Bug noticed in passing (fixed on `johnkerl/lint4`)
|
||||
|
||||
In `pkg/mlrval/mlrval_collections.go`, `removeIndexedOnArray` with a single in-bounds index removed
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue