mirror of
https://github.com/johnkerl/miller.git
synced 2026-08-03 04:52:24 +00:00
more
This commit is contained in:
parent
07f5ea5c33
commit
3cf9c2f7bd
23 changed files with 30 additions and 36 deletions
|
|
@ -6,3 +6,4 @@ nd
|
|||
nin
|
||||
numer
|
||||
Wit
|
||||
te
|
||||
|
|
|
|||
9
.github/workflows/codespell.yml
vendored
9
.github/workflows/codespell.yml
vendored
|
|
@ -32,12 +32,5 @@ jobs:
|
|||
with:
|
||||
check_filenames: true
|
||||
ignore_words_file: .codespellignore
|
||||
# This simply does not work despite multiple attempts, despite
|
||||
# https://github.com/codespell-project/codespell
|
||||
#
|
||||
# skip: "*.csv","*.txt"
|
||||
skip: "*.csv,*.txt,./test/cases"
|
||||
#skip:
|
||||
# - docs/src/data/english-words.txt
|
||||
# - docs/src/ngrams/gsl-2000.txt
|
||||
# - "test/cases/**/*"
|
||||
skip: "*.csv,*.dkvp,*.txt,./test/cases"
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ respectively.
|
|||
|
||||
The [`select`](reference-dsl-builtin-functions.md#select) function takes a map
|
||||
or array as its first argument and a function as second argument. It includes
|
||||
each input element in the ouptut if the function returns true.
|
||||
each input element in the output if the function returns true.
|
||||
|
||||
For arrays, that function should take one argument, for array element; for
|
||||
maps, it should take two, for map-element key and value. In either case it
|
||||
|
|
@ -776,7 +776,7 @@ Sorted, then cubed, then summed:
|
|||
|
||||
### Remember return
|
||||
|
||||
From other languages it's easy to accidentially write
|
||||
From other languages it's easy to accidentally write
|
||||
|
||||
<pre class="pre-highlight-in-pair">
|
||||
<b>mlr -n put 'end { print select([1,2,3,4,5], func (e) { e >= 3 })}'</b>
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ respectively.
|
|||
|
||||
The [`select`](reference-dsl-builtin-functions.md#select) function takes a map
|
||||
or array as its first argument and a function as second argument. It includes
|
||||
each input element in the ouptut if the function returns true.
|
||||
each input element in the output if the function returns true.
|
||||
|
||||
For arrays, that function should take one argument, for array element; for
|
||||
maps, it should take two, for map-element key and value. In either case it
|
||||
|
|
@ -426,7 +426,7 @@ GENMD-EOF
|
|||
|
||||
### Remember return
|
||||
|
||||
From other languages it's easy to accidentially write
|
||||
From other languages it's easy to accidentally write
|
||||
|
||||
GENMD-RUN-COMMAND-TOLERATING-ERROR
|
||||
mlr -n put 'end { print select([1,2,3,4,5], func (e) { e >= 3 })}'
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Quick links:
|
|||
Dates/times are not a separate data type; Miller uses ints for
|
||||
[seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time) and strings for formatted
|
||||
date/times. In this page we take a look at what some of the various options are
|
||||
for processing datetimes andd timezones in your data.
|
||||
for processing datetimes and timezones in your data.
|
||||
|
||||
See also the [section on time-related
|
||||
functions](reference-dsl-builtin-functions.md#time-functions) for
|
||||
|
|
@ -43,7 +43,7 @@ they're independent of timezone or daylight-savings time.
|
|||
One minus is that, being just numbers, they're not particularly human-readable
|
||||
-- hence the to-string and from-string functions described below. Another
|
||||
caveat (not really a minus) is that _epoch milliseconds_, rather than epoch
|
||||
seconds, are common in some contexts, particulary JavaScript. If you ever
|
||||
seconds, are common in some contexts, particularly JavaScript. If you ever
|
||||
(anywhere) see a timestamp for the year 49,000-something -- probably someone is
|
||||
treating epoch-milliseconds as epoch-seconds.
|
||||
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
Dates/times are not a separate data type; Miller uses ints for
|
||||
[seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time) and strings for formatted
|
||||
date/times. In this page we take a look at what some of the various options are
|
||||
for processing datetimes andd timezones in your data.
|
||||
for processing datetimes and timezones in your data.
|
||||
|
||||
See also the [section on time-related
|
||||
functions](reference-dsl-builtin-functions.md#time-functions) for
|
||||
|
|
@ -27,7 +27,7 @@ they're independent of timezone or daylight-savings time.
|
|||
One minus is that, being just numbers, they're not particularly human-readable
|
||||
-- hence the to-string and from-string functions described below. Another
|
||||
caveat (not really a minus) is that _epoch milliseconds_, rather than epoch
|
||||
seconds, are common in some contexts, particulary JavaScript. If you ever
|
||||
seconds, are common in some contexts, particularly JavaScript. If you ever
|
||||
(anywhere) see a timestamp for the year 49,000-something -- probably someone is
|
||||
treating epoch-milliseconds as epoch-seconds.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
// ================================================================
|
||||
// TOOO
|
||||
// TODO
|
||||
// ================================================================
|
||||
|
||||
package regtest
|
||||
|
|
|
|||
|
|
@ -98,7 +98,7 @@ FILENAME="foo.dat",FILENUM=1,NR=1,FNR=1
|
|||
|
||||
This is a small modification around the [CST](../../dsl/cst/) and the [`put` verb](../../transformers/put_or_filter.go).. Most of the keystroking here is for online help and command-line parsing.
|
||||
|
||||
One subtlety is that non-assignment expressions like `NR < 10` are filter statements within `put` -- they can be used to control whether or not a given record is included in the outut stream. Here, in the REPL, these expressions are simply printed to the terminal. And for `:skip until ...` or `:process until ...`, they're used as the exit condition to break out of reading input records.
|
||||
One subtlety is that non-assignment expressions like `NR < 10` are filter statements within `put` -- they can be used to control whether or not a given record is included in the output stream. Here, in the REPL, these expressions are simply printed to the terminal. And for `:skip until ...` or `:process until ...`, they're used as the exit condition to break out of reading input records.
|
||||
|
||||
# File structure
|
||||
|
||||
|
|
|
|||
|
|
@ -81,7 +81,7 @@ func (repl *Repl) handleDSLStringAux(
|
|||
repl.runtimeState.Inrec = outrec
|
||||
|
||||
// The filter expression for the main Miller DSL is any non-assignment
|
||||
// statment like 'true' or '$x > 0.5' etc. For the REPL, we re-use this for
|
||||
// statement like 'true' or '$x > 0.5' etc. For the REPL, we re-use this for
|
||||
// interactive expressions to be printed to the terminal. For the main DSL,
|
||||
// the default is mlrval.FromTrue(); for the REPL, the default is
|
||||
// mlrval.VOID.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// ================================================================
|
||||
// Top-level handler for a REPL session, including setup/construction, and
|
||||
// ingesting command-lines. Command-line strings are triaged and send off to
|
||||
// the appropriate handlers: DSL parse/execute if the comand is a DSL statement
|
||||
// the appropriate handlers: DSL parse/execute if the command is a DSL statement
|
||||
// (like '$z = $x + $y'); REPL-command-line parse/execute otherwise (like
|
||||
// ':open foo.dat' or ':help').
|
||||
//
|
||||
|
|
@ -66,7 +66,7 @@ func NewRepl(
|
|||
runtimeState := runtime.NewEmptyState(options)
|
||||
runtimeState.Update(inrec, context)
|
||||
// The filter expression for the main Miller DSL is any non-assignment
|
||||
// statment like 'true' or '$x > 0.5' etc. For the REPL, we re-use this for
|
||||
// statement like 'true' or '$x > 0.5' etc. For the REPL, we re-use this for
|
||||
// interactive expressions to be printed to the terminal. For the main DSL,
|
||||
// the default is mlrval.FromTrue(); for the REPL, the default is
|
||||
// mlrval.VOID.
|
||||
|
|
|
|||
|
|
@ -165,7 +165,7 @@ func handleLoad(repl *Repl, args []string) bool {
|
|||
// ----------------------------------------------------------------
|
||||
func usageOpen(repl *Repl) {
|
||||
fmt.Printf(
|
||||
":open {one or more data-file names in the format specifed by %s %s}.\n",
|
||||
":open {one or more data-file names in the format specified by %s %s}.\n",
|
||||
repl.exeName, repl.replName,
|
||||
)
|
||||
fmt.Print(
|
||||
|
|
@ -515,7 +515,7 @@ func handleSkipOrProcessUntil(repl *Repl, dslString string, processingNotSkippin
|
|||
repl,
|
||||
recordAndContext,
|
||||
processingNotSkipping,
|
||||
true, // testingByFilterExpression -- since we're continuing until the filter expresssion is true
|
||||
true, // testingByFilterExpression -- since we're continuing until the filter expression is true
|
||||
)
|
||||
if shouldBreak {
|
||||
break
|
||||
|
|
@ -597,7 +597,7 @@ func skipOrProcessRecord(
|
|||
// ----------------------------------------------------------------
|
||||
func usageWrite(repl *Repl) {
|
||||
fmt.Println(":write with no arguments.")
|
||||
fmt.Println("Sends the current record (maybe modifed by statements you enter)")
|
||||
fmt.Println("Sends the current record (maybe modified by statements you enter)")
|
||||
fmt.Printf("to standard output, with format as specified by %s %s.\n",
|
||||
repl.exeName, repl.replName)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -202,7 +202,7 @@ func BIF_minus_binary(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
// 2**63-1 (or is less than -2**63) using integer arithmetic (it may have
|
||||
// already overflowed) *or* using double-precision (granularity). Instead we
|
||||
// check if the absolute value of the product exceeds the largest representable
|
||||
// double less than 2**63. (An alterative would be to do all integer multiplies
|
||||
// double less than 2**63. (An alternative would be to do all integer multiplies
|
||||
// using handcrafted multi-word 128-bit arithmetic).
|
||||
|
||||
func times_n_ii(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
||||
|
|
|
|||
|
|
@ -659,7 +659,7 @@ func BIF_append(input1, input2 *mlrval.Mlrval) *mlrval.Mlrval {
|
|||
}
|
||||
|
||||
// ----------------------------------------------------------------
|
||||
// First argumemnt is prefix.
|
||||
// First argument is prefix.
|
||||
// Second argument is delimiter.
|
||||
// Third argument is map or array.
|
||||
// flatten("a", ".", {"b": { "c": 4 }}) is {"a.b.c" : 4}.
|
||||
|
|
|
|||
|
|
@ -185,7 +185,7 @@ func (reader *RecordReaderJSON) processHandle(
|
|||
//
|
||||
// Miller lets users (on an opt-in basis) have comments in their data files,
|
||||
// for all formats including JSON. Comments are only honored at start of line.
|
||||
// Users can have them be printed to stdout straightaway, or simply discarded.
|
||||
// Users can have them be printed to stdout straight away, or simply discarded.
|
||||
//
|
||||
// For most file formats Miller is doing line-based I/O and can deal with
|
||||
// comment lines easily and simply. But for JSON, the Go library needs an
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ var unbackslashReplacements = map[byte]string{
|
|||
'?': "?",
|
||||
}
|
||||
|
||||
// UnbackslashStringLiteral replaces "\t" with TAB, etc. for DSL expresions
|
||||
// UnbackslashStringLiteral replaces "\t" with TAB, etc. for DSL expressions
|
||||
// like '$foo = "a\tb"'. See also
|
||||
// https://en.wikipedia.org/wiki/Escape_sequences_in_C
|
||||
// (predates the port of Miller from C to Go).
|
||||
|
|
|
|||
|
|
@ -209,7 +209,7 @@ func MlrvalDecodeFromJSON(decoder *json.Decoder) (
|
|||
if !key.IsString() {
|
||||
return nil, false, fmt.Errorf(
|
||||
// TODO: print out what was gotten
|
||||
"mlr JSON reader: obejct keys must be string-valued.",
|
||||
"mlr JSON reader: object keys must be string-valued.",
|
||||
)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ var TYPE_NAMES = [MT_DIM]string{
|
|||
"int",
|
||||
"float",
|
||||
"bool",
|
||||
"empty", // For backward compatiblity with the C impl: this is user-visible
|
||||
"empty", // For backward compatibility with the C impl: this is user-visible
|
||||
"string",
|
||||
"array",
|
||||
"map",
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ type MultiOutputHandlerManager struct {
|
|||
// For stdout or stderr
|
||||
singleHandler *FileOutputHandler
|
||||
|
||||
// TOOD: make an enum
|
||||
// TODO: make an enum
|
||||
append bool // True for ">>", false for ">" and "|"
|
||||
pipe bool // True for "|", false for ">" and ">>"
|
||||
recordWriterOptions *cli.TWriterOptions
|
||||
|
|
|
|||
|
|
@ -338,7 +338,7 @@ Root
|
|||
;
|
||||
|
||||
// ----------------------------------------------------------------
|
||||
// A StatementBlock is a sequence of statemnts: either the stuff in between
|
||||
// A StatementBlock is a sequence of statements: either the stuff in between
|
||||
// (but not including) the curly braces in things like 'if (NR > 2) { $x = 1;
|
||||
// $y = 2 }', or, top-level Miller DSL statements like '$x = 1; $y = 2'.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
// Package runtime contains state for the concrete-syntax-tree executor which
|
||||
// runs user-specifed statements in the Miller domain-specific langauge.
|
||||
// runs user-specifed statements in the Miller domain-specific language.
|
||||
package runtime
|
||||
|
|
|
|||
|
|
@ -138,7 +138,7 @@ func (tr *TransformerBootstrap) Transform(
|
|||
//
|
||||
// However, in Go, there is concurrent processing. It would be
|
||||
// possible for us to emit a pointer to a particular record without
|
||||
// copying, then when emitting that saem record a second time, copy it.
|
||||
// copying, then when emitting that same record a second time, copy it.
|
||||
// But due to concurrency, the pointed-to record could have already
|
||||
// been mutated downstream. We wouldn't be copying our input as we
|
||||
// received it -- we'd be copying something potentially modified.
|
||||
|
|
|
|||
|
|
@ -129,7 +129,7 @@ func (tr *TransformerShuffle) Transform(
|
|||
images[u] = images[i]
|
||||
images[i] = temp
|
||||
// Decrease the size of the pool by 1. (Yes, unusedStart and k always have the same value.
|
||||
// Using two variables wastes neglible memory and makes the code easier to understand.)
|
||||
// Using two variables wastes negligible memory and makes the code easier to understand.)
|
||||
unusedStart++
|
||||
numUnused--
|
||||
}
|
||||
|
|
|
|||
|
|
@ -326,7 +326,7 @@ func (keeper *JoinBucketKeeper) FindJoinBucket(
|
|||
}
|
||||
|
||||
// ----------------------------------------------------------------
|
||||
// This finds the first peek record which posseses all the necessary join-field
|
||||
// This finds the first peek record which possesses all the necessary join-field
|
||||
// keys. Any other records found along the way, lacking the necessary
|
||||
// join-field keys, are moved to the left-unpaired list.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue