Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-20 18:10:07 +00:00
Code Issues Projects Releases Wiki Activity

type-check doc iterate

Browse source
This commit is contained in:
John Kerl 2016-12-13 09:23:31 -05:00
parent f73159a2a1
commit 383da943d7
9 changed files with 478 additions and 220 deletions
Download patch file Download diff file Expand all files Collapse all files

1
c/todo.txt
View file

@ -35,7 +35,6 @@ MAPVAR CHECKLIST:
! UT cases detailed below
! assert_map/assert_scalar/etc.
- impl
- UTs
- mlh
- mld

53
doc/content-for-reference-dsl.html
View file

@ -307,15 +307,51 @@ POKI_RUN_COMMAND{{cat data/type-decl-example.mlr}}HERE
<h2>Type-checking</h2>
<p/> Miller&rsquo;s <tt>put</tt>/<tt>filter</tt> DSLs support two optional
kinds of type-checking: one is <b>type declaration</b> for assignments to local
variables, binding of arguments to user-defined functions, and return values
from user-defined functions, and the other is inline <b>assertions</b> within
expressions. These are discussed in the following subsections.
kinds of type-checking. One is inline <b>type-tests</b> and
<b>type-assertions</b> within expressions. The other is <b>type
declarations</b> for assignments to local variables, binding of arguments to
user-defined functions, and return values from user-defined functions, These
are discussed in the following subsections.
<p/> Use of type-checking is entirely up to you: omit it if you want
flexibility with heterogeneous data; use it if you want to help catch
misspellings in your DSL code or unexpected irregularities in your input data.
<!-- ================================================================ -->
<h3>Type-test expressions and type-assertion expressions</h3>
<p/> The following <tt>is...</tt> functions take a value and return a boolean
indicating whether the argument is of the indicated type. The
<tt>assert_...</tt> functions return their argument if it is of the specified
type, and cause a fatal error otherwise:
<table>
<tr>
<td>
POKI_RUN_COMMAND{{mlr -F | grep ^is}}HERE
</td>
<td>
POKI_RUN_COMMAND{{mlr -F | grep ^assert}}HERE
</td>
</tr>
</table>
<p/> Here are some ways to use them. Suppose you have the following data file,
with inconsistent typing for boolean. (Also imagine that, for the sake of
discussion, we have a million-line file rather than a four-line file, so we
can&rsquo;t see it all at once and some automation is called for.)
POKI_RUN_COMMAND{{cat data/het-bool.csv}}HERE
<p/> One option is to coerce everything to boolean, or integer:
POKI_RUN_COMMAND{{mlr --icsvlite --opprint put '$reachable = boolean($reachable)' data/het-bool.csv}}HERE
POKI_RUN_COMMAND{{mlr --icsvlite --opprint put '$reachable = int(boolean($reachable))' data/het-bool.csv}}HERE
<p/> A second option is to flag badly formatted data:
POKI_RUN_COMMAND{{mlr --icsvlite --opprint put '$format_ok = isboolean($reachable)' data/het-bool.csv}}HERE
<!-- ================================================================ -->
<h3>Type-declarations for local variables, function parameter, and function return values</h3>
@ -414,15 +450,6 @@ when the function is called:
</pre>
</div>
<!-- ================================================================ -->
<h3>Type-assertion expressions</h3>
<p/> Similarly to type declarations, there are pass-through functions named with
<tt>assert...</tt> which can be used to for optional type-checking within
expressions: <tt>$o = assert_num($x)**2</tt> means that a numeric value will be
squared and stored, while a non-numeric value will result in a fatal error.
xxx need full list, and examples here
<!-- ================================================================ -->
<h2>Out-of-stream variables</h2>

102
doc/cookbook.html
View file

@ -607,33 +607,33 @@ $ mlr --ofmt '%.9lf' --opprint seqgen --start 1 --stop 28 then put '
' then put '$seconds=systime()' then step -a delta -f seconds then cut -x -f seconds
i o fcount seconds_delta
1 1 1 0
2 2 3 0.000025988
3 3 5 0.000012875
4 5 9 0.000017166
5 8 15 0.000022888
6 13 25 0.000034094
7 21 41 0.000048876
8 34 67 0.000075102
9 55 109 0.000118017
10 89 177 0.000188828
11 144 287 0.000300169
12 233 465 0.000481844
13 377 753 0.000771999
14 610 1219 0.001245975
15 987 1973 0.002014160
16 1597 3193 0.003319979
17 2584 5167 0.005117893
18 4181 8361 0.011898994
19 6765 13529 0.013988018
20 10946 21891 0.021378994
21 17711 35421 0.036515951
22 28657 57313 0.063183069
23 46368 92735 0.096009016
24 75025 150049 0.153168917
25 121393 242785 0.255737066
26 196418 392835 0.409193993
27 317811 635621 0.651291132
28 514229 1028457 1.080612898
2 2 3 0.000053883
3 3 5 0.000025034
4 5 9 0.000031948
5 8 15 0.000043154
6 13 25 0.000077963
7 21 41 0.000100851
8 34 67 0.000143051
9 55 109 0.000231981
10 89 177 0.000347137
11 144 287 0.000603914
12 233 465 0.000911951
13 377 753 0.000730991
14 610 1219 0.001175165
15 987 1973 0.002352953
16 1597 3193 0.005199909
17 2584 5167 0.008666039
18 4181 8361 0.013283014
19 6765 13529 0.020358086
20 10946 21891 0.035039902
21 17711 35421 0.049962997
22 28657 57313 0.086282015
23 46368 92735 0.132637978
24 75025 150049 0.213582993
25 121393 242785 0.358917952
26 196418 392835 0.503535032
27 317811 635621 0.830383062
28 514229 1028457 1.674153090
</pre>
</div>
<p/>
@ -666,32 +666,32 @@ $ mlr --ofmt '%.9lf' --opprint seqgen --start 1 --stop 28 then put '
i o fcount seconds_delta
1 1 1 0
2 2 3 0.000030041
3 3 3 0.000012875
4 5 3 0.000011206
5 8 3 0.000011921
6 13 3 0.000010967
7 21 3 0.000010014
8 34 3 0.000010967
9 55 3 0.000010967
10 89 3 0.000010014
11 144 3 0.000010967
12 233 3 0.000015020
13 377 3 0.000010967
14 610 3 0.000010967
15 987 3 0.000010014
16 1597 3 0.000011206
17 2584 3 0.000010967
18 4181 3 0.000010014
19 6765 3 0.000010014
3 3 3 0.000015020
4 5 3 0.000012875
5 8 3 0.000014067
6 13 3 0.000012875
7 21 3 0.000012159
8 34 3 0.000011921
9 55 3 0.000011921
10 89 3 0.000012159
11 144 3 0.000011921
12 233 3 0.000016928
13 377 3 0.000013113
14 610 3 0.000011921
15 987 3 0.000013113
16 1597 3 0.000011921
17 2584 3 0.000011921
18 4181 3 0.000012159
19 6765 3 0.000011921
20 10946 3 0.000010967
21 17711 3 0.000010014
21 17711 3 0.000014067
22 28657 3 0.000012875
23 46368 3 0.000012159
24 75025 3 0.000010967
25 121393 3 0.000010014
26 196418 3 0.000010967
27 317811 3 0.000010014
28 514229 3 0.000010014
23 46368 3 0.000014067
24 75025 3 0.000012159
25 121393 3 0.000012875
26 196418 3 0.000011921
27 317811 3 0.000012159
28 514229 3 0.000010967
</pre>
</div>
<p/>

5
doc/data/het-bool.csv Normal file
View file

@ -0,0 +1,5 @@
name,reachable
barney,false
betty,true
fred,true
wilma,1
1 name reachable
2 barney false
3 betty true
4 fred true
5 wilma 1

73
doc/manpage.html
View file

@ -284,10 +284,11 @@ OPTIONS
hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime
isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap
isnotempty isnotmap isnotnull isnull isnumeric ispresent isstring
assert_notnull assert_present assert_empty assert_notempty assert_numeric
assert_int assert_float assert_bool assert_boolean assert_string boolean float
fmtnum hexfmt int string typeof depth haskey joink joinkv joinv leafcount
length mapdiff mapsum splitkv splitkvx splitnv splitnvx
assert_absent assert_bool assert_boolean assert_empty assert_emptymap
assert_float assert_int assert_map assert_nonemptymap assert_notempty
assert_notmap assert_notnull assert_null assert_numeric assert_present
assert_string boolean float fmtnum hexfmt int string typeof depth haskey joink
joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv splitnvx
Please use "mlr --help-function {function name}" for function-specific help.
Please use "mlr --help-all-functions" or "mlr -f" for help on all functions.
Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords.
@ -1572,35 +1573,53 @@ FUNCTIONS FOR FILTER/PUT
isstring
(class=typing #args=1): True if field is present with string (including empty-string) value
assert_notnull
(class=typing #args=1): Returns argument if non-null (non-empty and non-absent), else throws an error.
assert_present
(class=typing #args=1): Returns argument if present in input, else throws an error.
assert_empty
(class=typing #args=1): Returns argument if present in input with empty value, else throws an error.
assert_notempty
(class=typing #args=1): Returns argument if present in input with non-empty value, else throws an error.
assert_numeric
(class=typing #args=1): Returns argument if present with int or float value, else throws an error.
assert_int
(class=typing #args=1): Returns argument if present with int value, else throws an error.
assert_float
(class=typing #args=1): Returns argument if present with float value, else throws an error.
assert_absent
(class=typing #args=1): Returns argument if it is absent in the input ata, else throws an error.
assert_bool
(class=typing #args=1): Returns argument if present with boolean value, else throws an error.
(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error.
assert_boolean
(class=typing #args=1): Returns argument if present with boolean value, else throws an error.
(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error.
assert_empty
(class=typing #args=1): Returns argument if it is present in input with empty value, else throws an error.
assert_emptymap
(class=typing #args=1): Returns argument if it is a map with empty value, else throws an error.
assert_float
(class=typing #args=1): Returns argument if it is present with float value, else throws an error.
assert_int
(class=typing #args=1): Returns argument if it is present with int value, else throws an error.
assert_map
(class=typing #args=1): Returns argument if it is a map, else throws an error.
assert_nonemptymap
(class=typing #args=1): Returns argument if it is a non-empty map, else throws an error.
assert_notempty
(class=typing #args=1): Returns argument if it is present in input with non-empty value, else throws an error.
assert_notmap
(class=typing #args=1): Returns argument if it is not a map, else throws an error.
assert_notnull
(class=typing #args=1): Returns argument if it is non-null (non-empty and non-absent), else throws an error.
assert_null
(class=typing #args=1): Returns argument if it is null (empty or absent), else throws an error.
assert_numeric
(class=typing #args=1): Returns argument if it is present with int or float value, else throws an error.
assert_present
(class=typing #args=1): Returns argument if it is present in input, else throws an error.
assert_string
(class=typing #args=1): Returns argument if present with string (including empty-string) value, else throws an error.
(class=typing #args=1): Returns argument if it is present with string (including empty-string) value, else throws an error.
boolean
(class=conversion #args=1): Convert int/float/bool/string to boolean.

73
doc/manpage.txt
View file

@ -135,10 +135,11 @@ OPTIONS
hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime
isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap
isnotempty isnotmap isnotnull isnull isnumeric ispresent isstring
assert_notnull assert_present assert_empty assert_notempty assert_numeric
assert_int assert_float assert_bool assert_boolean assert_string boolean float
fmtnum hexfmt int string typeof depth haskey joink joinkv joinv leafcount
length mapdiff mapsum splitkv splitkvx splitnv splitnvx
assert_absent assert_bool assert_boolean assert_empty assert_emptymap
assert_float assert_int assert_map assert_nonemptymap assert_notempty
assert_notmap assert_notnull assert_null assert_numeric assert_present
assert_string boolean float fmtnum hexfmt int string typeof depth haskey joink
joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv splitnvx
Please use "mlr --help-function {function name}" for function-specific help.
Please use "mlr --help-all-functions" or "mlr -f" for help on all functions.
Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords.
@ -1423,35 +1424,53 @@ FUNCTIONS FOR FILTER/PUT
isstring
(class=typing #args=1): True if field is present with string (including empty-string) value
assert_notnull
(class=typing #args=1): Returns argument if non-null (non-empty and non-absent), else throws an error.
assert_present
(class=typing #args=1): Returns argument if present in input, else throws an error.
assert_empty
(class=typing #args=1): Returns argument if present in input with empty value, else throws an error.
assert_notempty
(class=typing #args=1): Returns argument if present in input with non-empty value, else throws an error.
assert_numeric
(class=typing #args=1): Returns argument if present with int or float value, else throws an error.
assert_int
(class=typing #args=1): Returns argument if present with int value, else throws an error.
assert_float
(class=typing #args=1): Returns argument if present with float value, else throws an error.
assert_absent
(class=typing #args=1): Returns argument if it is absent in the input ata, else throws an error.
assert_bool
(class=typing #args=1): Returns argument if present with boolean value, else throws an error.
(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error.
assert_boolean
(class=typing #args=1): Returns argument if present with boolean value, else throws an error.
(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error.
assert_empty
(class=typing #args=1): Returns argument if it is present in input with empty value, else throws an error.
assert_emptymap
(class=typing #args=1): Returns argument if it is a map with empty value, else throws an error.
assert_float
(class=typing #args=1): Returns argument if it is present with float value, else throws an error.
assert_int
(class=typing #args=1): Returns argument if it is present with int value, else throws an error.
assert_map
(class=typing #args=1): Returns argument if it is a map, else throws an error.
assert_nonemptymap
(class=typing #args=1): Returns argument if it is a non-empty map, else throws an error.
assert_notempty
(class=typing #args=1): Returns argument if it is present in input with non-empty value, else throws an error.
assert_notmap
(class=typing #args=1): Returns argument if it is not a map, else throws an error.
assert_notnull
(class=typing #args=1): Returns argument if it is non-null (non-empty and non-absent), else throws an error.
assert_null
(class=typing #args=1): Returns argument if it is null (empty or absent), else throws an error.
assert_numeric
(class=typing #args=1): Returns argument if it is present with int or float value, else throws an error.
assert_present
(class=typing #args=1): Returns argument if it is present in input, else throws an error.
assert_string
(class=typing #args=1): Returns argument if present with string (including empty-string) value, else throws an error.
(class=typing #args=1): Returns argument if it is present with string (including empty-string) value, else throws an error.
boolean
(class=conversion #args=1): Convert int/float/bool/string to boolean.

181
doc/mlr.1
View file

@ -184,10 +184,11 @@ output separator to the given value.
hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime
isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap
isnotempty isnotmap isnotnull isnull isnumeric ispresent isstring
assert_notnull assert_present assert_empty assert_notempty assert_numeric
assert_int assert_float assert_bool assert_boolean assert_string boolean float
fmtnum hexfmt int string typeof depth haskey joink joinkv joinv leafcount
length mapdiff mapsum splitkv splitkvx splitnv splitnvx
assert_absent assert_bool assert_boolean assert_empty assert_emptymap
assert_float assert_int assert_map assert_nonemptymap assert_notempty
assert_notmap assert_notnull assert_null assert_numeric assert_present
assert_string boolean float fmtnum hexfmt int string typeof depth haskey joink
joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv splitnvx
Please use "mlr --help-function {function name}" for function-specific help.
Please use "mlr --help-all-functions" or "mlr -f" for help on all functions.
Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords.
@ -2402,66 +2403,12 @@ e.g. 1440768801.748936.
.fi
.if n \{\
.RE
.SS "assert_notnull"
.SS "assert_absent"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if non-null (non-empty and non-absent), else throws an error.
.fi
.if n \{\
.RE
.SS "assert_present"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present in input, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_empty"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present in input with empty value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_notempty"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present in input with non-empty value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_numeric"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present with int or float value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_int"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present with int value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_float"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present with float value, else throws an error.
(class=typing #args=1): Returns argument if it is absent in the input ata, else throws an error.
.fi
.if n \{\
.RE
@ -2470,7 +2417,7 @@ e.g. 1440768801.748936.
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present with boolean value, else throws an error.
(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error.
.fi
.if n \{\
.RE
@ -2479,7 +2426,115 @@ e.g. 1440768801.748936.
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present with boolean value, else throws an error.
(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_empty"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is present in input with empty value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_emptymap"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is a map with empty value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_float"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is present with float value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_int"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is present with int value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_map"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is a map, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_nonemptymap"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is a non-empty map, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_notempty"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is present in input with non-empty value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_notmap"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is not a map, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_notnull"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is non-null (non-empty and non-absent), else throws an error.
.fi
.if n \{\
.RE
.SS "assert_null"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is null (empty or absent), else throws an error.
.fi
.if n \{\
.RE
.SS "assert_numeric"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is present with int or float value, else throws an error.
.fi
.if n \{\
.RE
.SS "assert_present"
.if n \{\
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if it is present in input, else throws an error.
.fi
.if n \{\
.RE
@ -2488,7 +2543,7 @@ e.g. 1440768801.748936.
.RS 0
.\}
.nf
(class=typing #args=1): Returns argument if present with string (including empty-string) value, else throws an error.
(class=typing #args=1): Returns argument if it is present with string (including empty-string) value, else throws an error.
.fi
.if n \{\
.RE

199
doc/reference-dsl.html
View file

@ -153,8 +153,8 @@ Miller commands were run with pretty-print-tabular output format.
&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Field_names">Field names</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Local_variables">Local variables</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Type-checking">Type-checking</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Type-test_expressions_and_type-assertion_expressions">Type-test expressions and type-assertion expressions</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Type-declarations_for_local_variables,_function_parameter,_and_function_return_values">Type-declarations for local variables, function parameter, and function return values</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Type-assertion_expressions">Type-assertion expressions</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Out-of-stream_variables">Out-of-stream variables</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Indexed_out-of-stream_variables">Indexed out-of-stream variables</a><br/>
&nbsp;&nbsp;&nbsp;&nbsp;&bull;&nbsp;<a href="#Aggregate_variable_assignments">Aggregate variable assignments</a><br/>
@ -890,15 +890,139 @@ print "outer j =" . j; # j is undefined in this scope
<a id="Type-checking"/><h2>Type-checking</h2>
<p/> Miller&rsquo;s <tt>put</tt>/<tt>filter</tt> DSLs support two optional
kinds of type-checking: one is <b>type declaration</b> for assignments to local
variables, binding of arguments to user-defined functions, and return values
from user-defined functions, and the other is inline <b>assertions</b> within
expressions. These are discussed in the following subsections.
kinds of type-checking. One is inline <b>type-tests</b> and
<b>type-assertions</b> within expressions. The other is <b>type
declarations</b> for assignments to local variables, binding of arguments to
user-defined functions, and return values from user-defined functions, These
are discussed in the following subsections.
<p/> Use of type-checking is entirely up to you: omit it if you want
flexibility with heterogeneous data; use it if you want to help catch
misspellings in your DSL code or unexpected irregularities in your input data.
<!-- ================================================================ -->
<a id="Type-test_expressions_and_type-assertion_expressions"/><h3>Type-test expressions and type-assertion expressions</h3>
<p/> The following <tt>is...</tt> functions take a value and return a boolean
indicating whether the argument is of the indicated type. The
<tt>assert_...</tt> functions return their argument if it is of the specified
type, and cause a fatal error otherwise:
<table>
<tr>
<td>
<p/>
<div class="pokipanel">
<pre>
$ mlr -F | grep ^is
isabsent
isbool
isboolean
isempty
isemptymap
isfloat
isint
ismap
isnonemptymap
isnotempty
isnotmap
isnotnull
isnull
isnumeric
ispresent
isstring
</pre>
</div>
<p/>
</td>
<td>
<p/>
<div class="pokipanel">
<pre>
$ mlr -F | grep ^assert
assert_absent
assert_bool
assert_boolean
assert_empty
assert_emptymap
assert_float
assert_int
assert_map
assert_nonemptymap
assert_notempty
assert_notmap
assert_notnull
assert_null
assert_numeric
assert_present
assert_string
</pre>
</div>
<p/>
</td>
</tr>
</table>
<p/> Here are some ways to use them. Suppose you have the following data file,
with inconsistent typing for boolean. (Also imagine that, for the sake of
discussion, we have a million-line file rather than a four-line file, so we
can&rsquo;t see it all at once and some automation is called for.)
<p/>
<div class="pokipanel">
<pre>
$ cat data/het-bool.csv
name,reachable
barney,false
betty,true
fred,true
wilma,1
</pre>
</div>
<p/>
<p/> One option is to coerce everything to boolean, or integer:
<p/>
<div class="pokipanel">
<pre>
$ mlr --icsvlite --opprint put '$reachable = boolean($reachable)' data/het-bool.csv
name reachable
barney false
betty true
fred true
wilma true
</pre>
</div>
<p/>
<p/>
<div class="pokipanel">
<pre>
$ mlr --icsvlite --opprint put '$reachable = int(boolean($reachable))' data/het-bool.csv
name reachable
barney 0
betty 1
fred 1
wilma 1
</pre>
</div>
<p/>
<p/> A second option is to flag badly formatted data:
<p/>
<div class="pokipanel">
<pre>
$ mlr --icsvlite --opprint put '$format_ok = isboolean($reachable)' data/het-bool.csv
name reachable format_ok
barney false false
betty true false
fred true false
wilma 1 false
</pre>
</div>
<p/>
<!-- ================================================================ -->
<a id="Type-declarations_for_local_variables,_function_parameter,_and_function_return_values"/><h3>Type-declarations for local variables, function parameter, and function return values</h3>
@ -997,15 +1121,6 @@ when the function is called:
</pre>
</div>
<!-- ================================================================ -->
<a id="Type-assertion_expressions"/><h3>Type-assertion expressions</h3>
<p/> Similarly to type declarations, there are pass-through functions named with
<tt>assert...</tt> which can be used to for optional type-checking within
expressions: <tt>$o = assert_num($x)**2</tt> means that a numeric value will be
squared and stored, while a non-numeric value will result in a fatal error.
xxx need full list, and examples here
<!-- ================================================================ -->
<a id="Out-of-stream_variables"/><h2>Out-of-stream variables</h2>
@ -3265,35 +3380,53 @@ otherwise.
isstring (class=typing #args=1): True if field is present with string (including
empty-string) value
assert_notnull (class=typing #args=1): Returns argument if non-null (non-empty
and non-absent), else throws an error.
assert_absent (class=typing #args=1): Returns argument if it is absent in the
input ata, else throws an error.
assert_present (class=typing #args=1): Returns argument if present in input,
else throws an error.
assert_bool (class=typing #args=1): Returns argument if it is present with
boolean value, else throws an error.
assert_empty (class=typing #args=1): Returns argument if present in input with
assert_boolean (class=typing #args=1): Returns argument if it is present with
boolean value, else throws an error.
assert_empty (class=typing #args=1): Returns argument if it is present in input
with empty value, else throws an error.
assert_emptymap (class=typing #args=1): Returns argument if it is a map with
empty value, else throws an error.
assert_notempty (class=typing #args=1): Returns argument if present in input
with non-empty value, else throws an error.
assert_numeric (class=typing #args=1): Returns argument if present with int or
assert_float (class=typing #args=1): Returns argument if it is present with
float value, else throws an error.
assert_int (class=typing #args=1): Returns argument if present with int value,
else throws an error.
assert_float (class=typing #args=1): Returns argument if present with float
assert_int (class=typing #args=1): Returns argument if it is present with int
value, else throws an error.
assert_bool (class=typing #args=1): Returns argument if present with boolean
value, else throws an error.
assert_map (class=typing #args=1): Returns argument if it is a map, else throws
an error.
assert_boolean (class=typing #args=1): Returns argument if present with boolean
value, else throws an error.
assert_nonemptymap (class=typing #args=1): Returns argument if it is a non-empty
map, else throws an error.
assert_string (class=typing #args=1): Returns argument if present with string
(including empty-string) value, else throws an error.
assert_notempty (class=typing #args=1): Returns argument if it is present in
input with non-empty value, else throws an error.
assert_notmap (class=typing #args=1): Returns argument if it is not a map, else
throws an error.
assert_notnull (class=typing #args=1): Returns argument if it is non-null
(non-empty and non-absent), else throws an error.
assert_null (class=typing #args=1): Returns argument if it is null (empty or
absent), else throws an error.
assert_numeric (class=typing #args=1): Returns argument if it is present with
int or float value, else throws an error.
assert_present (class=typing #args=1): Returns argument if it is present in
input, else throws an error.
assert_string (class=typing #args=1): Returns argument if it is present with
string (including empty-string) value, else throws an error.
boolean (class=conversion #args=1): Convert int/float/bool/string to boolean.

11
doc/reference.html
View file

@ -384,11 +384,12 @@ Functions for the filter and put verbs:
gmt2sec hms2fsec hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime
strptime systime isabsent isbool isboolean isempty isemptymap isfloat isint
ismap isnonemptymap isnotempty isnotmap isnotnull isnull isnumeric ispresent
isstring assert_notnull assert_present assert_empty assert_notempty
assert_numeric assert_int assert_float assert_bool assert_boolean
assert_string boolean float fmtnum hexfmt int string typeof depth haskey
joink joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv
splitnvx
isstring assert_absent assert_bool assert_boolean assert_empty
assert_emptymap assert_float assert_int assert_map assert_nonemptymap
assert_notempty assert_notmap assert_notnull assert_null assert_numeric
assert_present assert_string boolean float fmtnum hexfmt int string typeof
depth haskey joink joinkv joinv leafcount length mapdiff mapsum splitkv
splitkvx splitnv splitnvx
Please use "mlr --help-function {function name}" for function-specific help.
Please use "mlr --help-all-functions" or "mlr -f" for help on all functions.
Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords.