From 383da943d76016d40a80248001d7a204e73fa8e7 Mon Sep 17 00:00:00 2001 From: John Kerl Date: Tue, 13 Dec 2016 09:23:31 -0500 Subject: [PATCH] type-check doc iterate --- c/todo.txt | 1 - doc/content-for-reference-dsl.html | 53 ++++++-- doc/cookbook.html | 102 +++++++-------- doc/data/het-bool.csv | 5 + doc/manpage.html | 73 +++++++---- doc/manpage.txt | 73 +++++++---- doc/mlr.1 | 181 +++++++++++++++++--------- doc/reference-dsl.html | 199 ++++++++++++++++++++++++----- doc/reference.html | 11 +- 9 files changed, 478 insertions(+), 220 deletions(-) create mode 100644 doc/data/het-bool.csv diff --git a/c/todo.txt b/c/todo.txt index 7cfad80c9..0e1f9a95f 100644 --- a/c/todo.txt +++ b/c/todo.txt @@ -35,7 +35,6 @@ MAPVAR CHECKLIST: ! UT cases detailed below ! assert_map/assert_scalar/etc. - - impl - UTs - mlh - mld diff --git a/doc/content-for-reference-dsl.html b/doc/content-for-reference-dsl.html index f72f7a1dc..cfe7811da 100644 --- a/doc/content-for-reference-dsl.html +++ b/doc/content-for-reference-dsl.html @@ -307,15 +307,51 @@ POKI_RUN_COMMAND{{cat data/type-decl-example.mlr}}HERE

Type-checking

Miller’s put/filter DSLs support two optional -kinds of type-checking: one is type declaration for assignments to local -variables, binding of arguments to user-defined functions, and return values -from user-defined functions, and the other is inline assertions within -expressions. These are discussed in the following subsections. +kinds of type-checking. One is inline type-tests and +type-assertions within expressions. The other is type +declarations for assignments to local variables, binding of arguments to +user-defined functions, and return values from user-defined functions, These +are discussed in the following subsections.

Use of type-checking is entirely up to you: omit it if you want flexibility with heterogeneous data; use it if you want to help catch misspellings in your DSL code or unexpected irregularities in your input data. + +

Type-test expressions and type-assertion expressions

+ +

The following is... functions take a value and return a boolean +indicating whether the argument is of the indicated type. The +assert_... functions return their argument if it is of the specified +type, and cause a fatal error otherwise: + + + + + + +
+POKI_RUN_COMMAND{{mlr -F | grep ^is}}HERE + +POKI_RUN_COMMAND{{mlr -F | grep ^assert}}HERE +
+ +

Here are some ways to use them. Suppose you have the following data file, +with inconsistent typing for boolean. (Also imagine that, for the sake of +discussion, we have a million-line file rather than a four-line file, so we +can’t see it all at once and some automation is called for.) + +POKI_RUN_COMMAND{{cat data/het-bool.csv}}HERE + +

One option is to coerce everything to boolean, or integer: + +POKI_RUN_COMMAND{{mlr --icsvlite --opprint put '$reachable = boolean($reachable)' data/het-bool.csv}}HERE +POKI_RUN_COMMAND{{mlr --icsvlite --opprint put '$reachable = int(boolean($reachable))' data/het-bool.csv}}HERE + +

A second option is to flag badly formatted data: + +POKI_RUN_COMMAND{{mlr --icsvlite --opprint put '$format_ok = isboolean($reachable)' data/het-bool.csv}}HERE +

Type-declarations for local variables, function parameter, and function return values

@@ -414,15 +450,6 @@ when the function is called: - -

Type-assertion expressions

- -

Similarly to type declarations, there are pass-through functions named with -assert... which can be used to for optional type-checking within -expressions: $o = assert_num($x)**2 means that a numeric value will be -squared and stored, while a non-numeric value will result in a fatal error. -xxx need full list, and examples here -

Out-of-stream variables

diff --git a/doc/cookbook.html b/doc/cookbook.html index d8d99e58f..79e853502 100644 --- a/doc/cookbook.html +++ b/doc/cookbook.html @@ -607,33 +607,33 @@ $ mlr --ofmt '%.9lf' --opprint seqgen --start 1 --stop 28 then put ' ' then put '$seconds=systime()' then step -a delta -f seconds then cut -x -f seconds i o fcount seconds_delta 1 1 1 0 -2 2 3 0.000025988 -3 3 5 0.000012875 -4 5 9 0.000017166 -5 8 15 0.000022888 -6 13 25 0.000034094 -7 21 41 0.000048876 -8 34 67 0.000075102 -9 55 109 0.000118017 -10 89 177 0.000188828 -11 144 287 0.000300169 -12 233 465 0.000481844 -13 377 753 0.000771999 -14 610 1219 0.001245975 -15 987 1973 0.002014160 -16 1597 3193 0.003319979 -17 2584 5167 0.005117893 -18 4181 8361 0.011898994 -19 6765 13529 0.013988018 -20 10946 21891 0.021378994 -21 17711 35421 0.036515951 -22 28657 57313 0.063183069 -23 46368 92735 0.096009016 -24 75025 150049 0.153168917 -25 121393 242785 0.255737066 -26 196418 392835 0.409193993 -27 317811 635621 0.651291132 -28 514229 1028457 1.080612898 +2 2 3 0.000053883 +3 3 5 0.000025034 +4 5 9 0.000031948 +5 8 15 0.000043154 +6 13 25 0.000077963 +7 21 41 0.000100851 +8 34 67 0.000143051 +9 55 109 0.000231981 +10 89 177 0.000347137 +11 144 287 0.000603914 +12 233 465 0.000911951 +13 377 753 0.000730991 +14 610 1219 0.001175165 +15 987 1973 0.002352953 +16 1597 3193 0.005199909 +17 2584 5167 0.008666039 +18 4181 8361 0.013283014 +19 6765 13529 0.020358086 +20 10946 21891 0.035039902 +21 17711 35421 0.049962997 +22 28657 57313 0.086282015 +23 46368 92735 0.132637978 +24 75025 150049 0.213582993 +25 121393 242785 0.358917952 +26 196418 392835 0.503535032 +27 317811 635621 0.830383062 +28 514229 1028457 1.674153090

@@ -666,32 +666,32 @@ $ mlr --ofmt '%.9lf' --opprint seqgen --start 1 --stop 28 then put ' i o fcount seconds_delta 1 1 1 0 2 2 3 0.000030041 -3 3 3 0.000012875 -4 5 3 0.000011206 -5 8 3 0.000011921 -6 13 3 0.000010967 -7 21 3 0.000010014 -8 34 3 0.000010967 -9 55 3 0.000010967 -10 89 3 0.000010014 -11 144 3 0.000010967 -12 233 3 0.000015020 -13 377 3 0.000010967 -14 610 3 0.000010967 -15 987 3 0.000010014 -16 1597 3 0.000011206 -17 2584 3 0.000010967 -18 4181 3 0.000010014 -19 6765 3 0.000010014 +3 3 3 0.000015020 +4 5 3 0.000012875 +5 8 3 0.000014067 +6 13 3 0.000012875 +7 21 3 0.000012159 +8 34 3 0.000011921 +9 55 3 0.000011921 +10 89 3 0.000012159 +11 144 3 0.000011921 +12 233 3 0.000016928 +13 377 3 0.000013113 +14 610 3 0.000011921 +15 987 3 0.000013113 +16 1597 3 0.000011921 +17 2584 3 0.000011921 +18 4181 3 0.000012159 +19 6765 3 0.000011921 20 10946 3 0.000010967 -21 17711 3 0.000010014 +21 17711 3 0.000014067 22 28657 3 0.000012875 -23 46368 3 0.000012159 -24 75025 3 0.000010967 -25 121393 3 0.000010014 -26 196418 3 0.000010967 -27 317811 3 0.000010014 -28 514229 3 0.000010014 +23 46368 3 0.000014067 +24 75025 3 0.000012159 +25 121393 3 0.000012875 +26 196418 3 0.000011921 +27 317811 3 0.000012159 +28 514229 3 0.000010967

diff --git a/doc/data/het-bool.csv b/doc/data/het-bool.csv new file mode 100644 index 000000000..cc094b90f --- /dev/null +++ b/doc/data/het-bool.csv @@ -0,0 +1,5 @@ +name,reachable +barney,false +betty,true +fred,true +wilma,1 diff --git a/doc/manpage.html b/doc/manpage.html index 92df14a83..da68b14ab 100644 --- a/doc/manpage.html +++ b/doc/manpage.html @@ -284,10 +284,11 @@ OPTIONS hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap isnotempty isnotmap isnotnull isnull isnumeric ispresent isstring - assert_notnull assert_present assert_empty assert_notempty assert_numeric - assert_int assert_float assert_bool assert_boolean assert_string boolean float - fmtnum hexfmt int string typeof depth haskey joink joinkv joinv leafcount - length mapdiff mapsum splitkv splitkvx splitnv splitnvx + assert_absent assert_bool assert_boolean assert_empty assert_emptymap + assert_float assert_int assert_map assert_nonemptymap assert_notempty + assert_notmap assert_notnull assert_null assert_numeric assert_present + assert_string boolean float fmtnum hexfmt int string typeof depth haskey joink + joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv splitnvx Please use "mlr --help-function {function name}" for function-specific help. Please use "mlr --help-all-functions" or "mlr -f" for help on all functions. Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords. @@ -1572,35 +1573,53 @@ FUNCTIONS FOR FILTER/PUT isstring (class=typing #args=1): True if field is present with string (including empty-string) value - assert_notnull - (class=typing #args=1): Returns argument if non-null (non-empty and non-absent), else throws an error. - - assert_present - (class=typing #args=1): Returns argument if present in input, else throws an error. - - assert_empty - (class=typing #args=1): Returns argument if present in input with empty value, else throws an error. - - assert_notempty - (class=typing #args=1): Returns argument if present in input with non-empty value, else throws an error. - - assert_numeric - (class=typing #args=1): Returns argument if present with int or float value, else throws an error. - - assert_int - (class=typing #args=1): Returns argument if present with int value, else throws an error. - - assert_float - (class=typing #args=1): Returns argument if present with float value, else throws an error. + assert_absent + (class=typing #args=1): Returns argument if it is absent in the input ata, else throws an error. assert_bool - (class=typing #args=1): Returns argument if present with boolean value, else throws an error. + (class=typing #args=1): Returns argument if it is present with boolean value, else throws an error. assert_boolean - (class=typing #args=1): Returns argument if present with boolean value, else throws an error. + (class=typing #args=1): Returns argument if it is present with boolean value, else throws an error. + + assert_empty + (class=typing #args=1): Returns argument if it is present in input with empty value, else throws an error. + + assert_emptymap + (class=typing #args=1): Returns argument if it is a map with empty value, else throws an error. + + assert_float + (class=typing #args=1): Returns argument if it is present with float value, else throws an error. + + assert_int + (class=typing #args=1): Returns argument if it is present with int value, else throws an error. + + assert_map + (class=typing #args=1): Returns argument if it is a map, else throws an error. + + assert_nonemptymap + (class=typing #args=1): Returns argument if it is a non-empty map, else throws an error. + + assert_notempty + (class=typing #args=1): Returns argument if it is present in input with non-empty value, else throws an error. + + assert_notmap + (class=typing #args=1): Returns argument if it is not a map, else throws an error. + + assert_notnull + (class=typing #args=1): Returns argument if it is non-null (non-empty and non-absent), else throws an error. + + assert_null + (class=typing #args=1): Returns argument if it is null (empty or absent), else throws an error. + + assert_numeric + (class=typing #args=1): Returns argument if it is present with int or float value, else throws an error. + + assert_present + (class=typing #args=1): Returns argument if it is present in input, else throws an error. assert_string - (class=typing #args=1): Returns argument if present with string (including empty-string) value, else throws an error. + (class=typing #args=1): Returns argument if it is present with string (including empty-string) value, else throws an error. boolean (class=conversion #args=1): Convert int/float/bool/string to boolean. diff --git a/doc/manpage.txt b/doc/manpage.txt index 8317ab1a1..0964b420c 100644 --- a/doc/manpage.txt +++ b/doc/manpage.txt @@ -135,10 +135,11 @@ OPTIONS hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap isnotempty isnotmap isnotnull isnull isnumeric ispresent isstring - assert_notnull assert_present assert_empty assert_notempty assert_numeric - assert_int assert_float assert_bool assert_boolean assert_string boolean float - fmtnum hexfmt int string typeof depth haskey joink joinkv joinv leafcount - length mapdiff mapsum splitkv splitkvx splitnv splitnvx + assert_absent assert_bool assert_boolean assert_empty assert_emptymap + assert_float assert_int assert_map assert_nonemptymap assert_notempty + assert_notmap assert_notnull assert_null assert_numeric assert_present + assert_string boolean float fmtnum hexfmt int string typeof depth haskey joink + joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv splitnvx Please use "mlr --help-function {function name}" for function-specific help. Please use "mlr --help-all-functions" or "mlr -f" for help on all functions. Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords. @@ -1423,35 +1424,53 @@ FUNCTIONS FOR FILTER/PUT isstring (class=typing #args=1): True if field is present with string (including empty-string) value - assert_notnull - (class=typing #args=1): Returns argument if non-null (non-empty and non-absent), else throws an error. - - assert_present - (class=typing #args=1): Returns argument if present in input, else throws an error. - - assert_empty - (class=typing #args=1): Returns argument if present in input with empty value, else throws an error. - - assert_notempty - (class=typing #args=1): Returns argument if present in input with non-empty value, else throws an error. - - assert_numeric - (class=typing #args=1): Returns argument if present with int or float value, else throws an error. - - assert_int - (class=typing #args=1): Returns argument if present with int value, else throws an error. - - assert_float - (class=typing #args=1): Returns argument if present with float value, else throws an error. + assert_absent + (class=typing #args=1): Returns argument if it is absent in the input ata, else throws an error. assert_bool - (class=typing #args=1): Returns argument if present with boolean value, else throws an error. + (class=typing #args=1): Returns argument if it is present with boolean value, else throws an error. assert_boolean - (class=typing #args=1): Returns argument if present with boolean value, else throws an error. + (class=typing #args=1): Returns argument if it is present with boolean value, else throws an error. + + assert_empty + (class=typing #args=1): Returns argument if it is present in input with empty value, else throws an error. + + assert_emptymap + (class=typing #args=1): Returns argument if it is a map with empty value, else throws an error. + + assert_float + (class=typing #args=1): Returns argument if it is present with float value, else throws an error. + + assert_int + (class=typing #args=1): Returns argument if it is present with int value, else throws an error. + + assert_map + (class=typing #args=1): Returns argument if it is a map, else throws an error. + + assert_nonemptymap + (class=typing #args=1): Returns argument if it is a non-empty map, else throws an error. + + assert_notempty + (class=typing #args=1): Returns argument if it is present in input with non-empty value, else throws an error. + + assert_notmap + (class=typing #args=1): Returns argument if it is not a map, else throws an error. + + assert_notnull + (class=typing #args=1): Returns argument if it is non-null (non-empty and non-absent), else throws an error. + + assert_null + (class=typing #args=1): Returns argument if it is null (empty or absent), else throws an error. + + assert_numeric + (class=typing #args=1): Returns argument if it is present with int or float value, else throws an error. + + assert_present + (class=typing #args=1): Returns argument if it is present in input, else throws an error. assert_string - (class=typing #args=1): Returns argument if present with string (including empty-string) value, else throws an error. + (class=typing #args=1): Returns argument if it is present with string (including empty-string) value, else throws an error. boolean (class=conversion #args=1): Convert int/float/bool/string to boolean. diff --git a/doc/mlr.1 b/doc/mlr.1 index 19b5cc16c..5154bef53 100644 --- a/doc/mlr.1 +++ b/doc/mlr.1 @@ -184,10 +184,11 @@ output separator to the given value. hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap isnotempty isnotmap isnotnull isnull isnumeric ispresent isstring - assert_notnull assert_present assert_empty assert_notempty assert_numeric - assert_int assert_float assert_bool assert_boolean assert_string boolean float - fmtnum hexfmt int string typeof depth haskey joink joinkv joinv leafcount - length mapdiff mapsum splitkv splitkvx splitnv splitnvx + assert_absent assert_bool assert_boolean assert_empty assert_emptymap + assert_float assert_int assert_map assert_nonemptymap assert_notempty + assert_notmap assert_notnull assert_null assert_numeric assert_present + assert_string boolean float fmtnum hexfmt int string typeof depth haskey joink + joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv splitnvx Please use "mlr --help-function {function name}" for function-specific help. Please use "mlr --help-all-functions" or "mlr -f" for help on all functions. Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords. @@ -2402,66 +2403,12 @@ e.g. 1440768801.748936. .fi .if n \{\ .RE -.SS "assert_notnull" +.SS "assert_absent" .if n \{\ .RS 0 .\} .nf -(class=typing #args=1): Returns argument if non-null (non-empty and non-absent), else throws an error. -.fi -.if n \{\ -.RE -.SS "assert_present" -.if n \{\ -.RS 0 -.\} -.nf -(class=typing #args=1): Returns argument if present in input, else throws an error. -.fi -.if n \{\ -.RE -.SS "assert_empty" -.if n \{\ -.RS 0 -.\} -.nf -(class=typing #args=1): Returns argument if present in input with empty value, else throws an error. -.fi -.if n \{\ -.RE -.SS "assert_notempty" -.if n \{\ -.RS 0 -.\} -.nf -(class=typing #args=1): Returns argument if present in input with non-empty value, else throws an error. -.fi -.if n \{\ -.RE -.SS "assert_numeric" -.if n \{\ -.RS 0 -.\} -.nf -(class=typing #args=1): Returns argument if present with int or float value, else throws an error. -.fi -.if n \{\ -.RE -.SS "assert_int" -.if n \{\ -.RS 0 -.\} -.nf -(class=typing #args=1): Returns argument if present with int value, else throws an error. -.fi -.if n \{\ -.RE -.SS "assert_float" -.if n \{\ -.RS 0 -.\} -.nf -(class=typing #args=1): Returns argument if present with float value, else throws an error. +(class=typing #args=1): Returns argument if it is absent in the input ata, else throws an error. .fi .if n \{\ .RE @@ -2470,7 +2417,7 @@ e.g. 1440768801.748936. .RS 0 .\} .nf -(class=typing #args=1): Returns argument if present with boolean value, else throws an error. +(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error. .fi .if n \{\ .RE @@ -2479,7 +2426,115 @@ e.g. 1440768801.748936. .RS 0 .\} .nf -(class=typing #args=1): Returns argument if present with boolean value, else throws an error. +(class=typing #args=1): Returns argument if it is present with boolean value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_empty" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is present in input with empty value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_emptymap" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is a map with empty value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_float" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is present with float value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_int" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is present with int value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_map" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is a map, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_nonemptymap" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is a non-empty map, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_notempty" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is present in input with non-empty value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_notmap" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is not a map, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_notnull" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is non-null (non-empty and non-absent), else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_null" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is null (empty or absent), else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_numeric" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is present with int or float value, else throws an error. +.fi +.if n \{\ +.RE +.SS "assert_present" +.if n \{\ +.RS 0 +.\} +.nf +(class=typing #args=1): Returns argument if it is present in input, else throws an error. .fi .if n \{\ .RE @@ -2488,7 +2543,7 @@ e.g. 1440768801.748936. .RS 0 .\} .nf -(class=typing #args=1): Returns argument if present with string (including empty-string) value, else throws an error. +(class=typing #args=1): Returns argument if it is present with string (including empty-string) value, else throws an error. .fi .if n \{\ .RE diff --git a/doc/reference-dsl.html b/doc/reference-dsl.html index 7c0226c9c..b38ca8e2c 100644 --- a/doc/reference-dsl.html +++ b/doc/reference-dsl.html @@ -153,8 +153,8 @@ Miller commands were run with pretty-print-tabular output format.     • Field names
    • Local variables
    • Type-checking
+        • Type-test expressions and type-assertion expressions
        • Type-declarations for local variables, function parameter, and function return values
-        • Type-assertion expressions
    • Out-of-stream variables
    • Indexed out-of-stream variables
    • Aggregate variable assignments
@@ -890,15 +890,139 @@ print "outer j =" . j; # j is undefined in this scope

Type-checking

Miller’s put/filter DSLs support two optional -kinds of type-checking: one is type declaration for assignments to local -variables, binding of arguments to user-defined functions, and return values -from user-defined functions, and the other is inline assertions within -expressions. These are discussed in the following subsections. +kinds of type-checking. One is inline type-tests and +type-assertions within expressions. The other is type +declarations for assignments to local variables, binding of arguments to +user-defined functions, and return values from user-defined functions, These +are discussed in the following subsections.

Use of type-checking is entirely up to you: omit it if you want flexibility with heterogeneous data; use it if you want to help catch misspellings in your DSL code or unexpected irregularities in your input data. + +

Type-test expressions and type-assertion expressions

+ +

The following is... functions take a value and return a boolean +indicating whether the argument is of the indicated type. The +assert_... functions return their argument if it is of the specified +type, and cause a fatal error otherwise: + + + + + + +
+

+

+
+$ mlr -F | grep ^is
+isabsent
+isbool
+isboolean
+isempty
+isemptymap
+isfloat
+isint
+ismap
+isnonemptymap
+isnotempty
+isnotmap
+isnotnull
+isnull
+isnumeric
+ispresent
+isstring
+
+
+

+

 +

+

+
+$ mlr -F | grep ^assert
+assert_absent
+assert_bool
+assert_boolean
+assert_empty
+assert_emptymap
+assert_float
+assert_int
+assert_map
+assert_nonemptymap
+assert_notempty
+assert_notmap
+assert_notnull
+assert_null
+assert_numeric
+assert_present
+assert_string
+
+
+

+

+ +

Here are some ways to use them. Suppose you have the following data file, +with inconsistent typing for boolean. (Also imagine that, for the sake of +discussion, we have a million-line file rather than a four-line file, so we +can’t see it all at once and some automation is called for.) + +

+

+
+$ cat data/het-bool.csv
+name,reachable
+barney,false
+betty,true
+fred,true
+wilma,1
+
+
+

+ +

One option is to coerce everything to boolean, or integer: + +

+

+
+$ mlr --icsvlite --opprint put '$reachable = boolean($reachable)' data/het-bool.csv
+name   reachable
+barney false
+betty  true
+fred   true
+wilma  true
+
+
+

+

+

+
+$ mlr --icsvlite --opprint put '$reachable = int(boolean($reachable))' data/het-bool.csv
+name   reachable
+barney 0
+betty  1
+fred   1
+wilma  1
+
+
+

+ +

A second option is to flag badly formatted data: + +

+

+
+$ mlr --icsvlite --opprint put '$format_ok = isboolean($reachable)' data/het-bool.csv
+name   reachable format_ok
+barney false     false
+betty  true      false
+fred   true      false
+wilma  1         false
+
+
+

+

Type-declarations for local variables, function parameter, and function return values

@@ -997,15 +1121,6 @@ when the function is called: - -

Type-assertion expressions

- -

Similarly to type declarations, there are pass-through functions named with -assert... which can be used to for optional type-checking within -expressions: $o = assert_num($x)**2 means that a numeric value will be -squared and stored, while a non-numeric value will result in a fatal error. -xxx need full list, and examples here -

Out-of-stream variables

@@ -3265,35 +3380,53 @@ otherwise. isstring (class=typing #args=1): True if field is present with string (including empty-string) value -assert_notnull (class=typing #args=1): Returns argument if non-null (non-empty -and non-absent), else throws an error. +assert_absent (class=typing #args=1): Returns argument if it is absent in the +input ata, else throws an error. -assert_present (class=typing #args=1): Returns argument if present in input, -else throws an error. +assert_bool (class=typing #args=1): Returns argument if it is present with +boolean value, else throws an error. -assert_empty (class=typing #args=1): Returns argument if present in input with +assert_boolean (class=typing #args=1): Returns argument if it is present with +boolean value, else throws an error. + +assert_empty (class=typing #args=1): Returns argument if it is present in input +with empty value, else throws an error. + +assert_emptymap (class=typing #args=1): Returns argument if it is a map with empty value, else throws an error. -assert_notempty (class=typing #args=1): Returns argument if present in input -with non-empty value, else throws an error. - -assert_numeric (class=typing #args=1): Returns argument if present with int or +assert_float (class=typing #args=1): Returns argument if it is present with float value, else throws an error. -assert_int (class=typing #args=1): Returns argument if present with int value, -else throws an error. - -assert_float (class=typing #args=1): Returns argument if present with float +assert_int (class=typing #args=1): Returns argument if it is present with int value, else throws an error. -assert_bool (class=typing #args=1): Returns argument if present with boolean -value, else throws an error. +assert_map (class=typing #args=1): Returns argument if it is a map, else throws +an error. -assert_boolean (class=typing #args=1): Returns argument if present with boolean -value, else throws an error. +assert_nonemptymap (class=typing #args=1): Returns argument if it is a non-empty +map, else throws an error. -assert_string (class=typing #args=1): Returns argument if present with string -(including empty-string) value, else throws an error. +assert_notempty (class=typing #args=1): Returns argument if it is present in +input with non-empty value, else throws an error. + +assert_notmap (class=typing #args=1): Returns argument if it is not a map, else +throws an error. + +assert_notnull (class=typing #args=1): Returns argument if it is non-null +(non-empty and non-absent), else throws an error. + +assert_null (class=typing #args=1): Returns argument if it is null (empty or +absent), else throws an error. + +assert_numeric (class=typing #args=1): Returns argument if it is present with +int or float value, else throws an error. + +assert_present (class=typing #args=1): Returns argument if it is present in +input, else throws an error. + +assert_string (class=typing #args=1): Returns argument if it is present with +string (including empty-string) value, else throws an error. boolean (class=conversion #args=1): Convert int/float/bool/string to boolean. diff --git a/doc/reference.html b/doc/reference.html index 2df3f39cf..fa70aff17 100644 --- a/doc/reference.html +++ b/doc/reference.html @@ -384,11 +384,12 @@ Functions for the filter and put verbs: gmt2sec hms2fsec hms2sec sec2dhms sec2gmt sec2gmtdate sec2hms strftime strptime systime isabsent isbool isboolean isempty isemptymap isfloat isint ismap isnonemptymap isnotempty isnotmap isnotnull isnull isnumeric ispresent - isstring assert_notnull assert_present assert_empty assert_notempty - assert_numeric assert_int assert_float assert_bool assert_boolean - assert_string boolean float fmtnum hexfmt int string typeof depth haskey - joink joinkv joinv leafcount length mapdiff mapsum splitkv splitkvx splitnv - splitnvx + isstring assert_absent assert_bool assert_boolean assert_empty + assert_emptymap assert_float assert_int assert_map assert_nonemptymap + assert_notempty assert_notmap assert_notnull assert_null assert_numeric + assert_present assert_string boolean float fmtnum hexfmt int string typeof + depth haskey joink joinkv joinv leafcount length mapdiff mapsum splitkv + splitkvx splitnv splitnvx Please use "mlr --help-function {function name}" for function-specific help. Please use "mlr --help-all-functions" or "mlr -f" for help on all functions. Please use "mlr --help-all-keywords" or "mlr -k" for help on all keywords.