Do not type-infer Inf, Infinity, NaN, true, false, etc. on file-read

This commit is contained in:
John Kerl 2021-09-04 22:46:58 -04:00
parent cfc986b5f4
commit a4d39a1b4b
77 changed files with 204 additions and 104 deletions

View file

@ -86,8 +86,5 @@ A third way is to abort the process on first instance of bad data:
<b>mlr --csv put '$reachable = asserting_string($reachable)' data/het-bool.csv</b>
</pre>
<pre class="pre-non-highlight-in-pair">
name,reachable
barney,false
Miller: is_string type-assertion failed at NR=4 FNR=4 FILENAME=data/het-bool.csv
betty,true
</pre>

View file

@ -1,12 +1,5 @@
TOP:
c data-types:
! don't auto-infer bools; then, data-types page
! don't auto-infer Infinity, etc -- ?!? then, data-types page
RT-check
https://github.com/johnkerl/miller/issues/241
https://github.com/johnkerl/miller/issues/357
MlrvalPointerFromInferredType -> mk variant
c! seps \001 etc !
mlrc --iusv --oxtab cat regtest/input/example.usv
mlr --iusv --oxtab cat regtest/input/example.usv
@ -112,24 +105,12 @@ reference-verbs:
E data/colored-shapes.dkvp (this page & throughout) a CSV file ...
* ... I ONLY READ UP TO CUT & PAUSED ...
reference-main-data-types:
! don't auto-infer bool from input data :(
E field values are usually strings -> update
E Field values are treated as numeric for the following: -> update
E true/false -> add info about NaN and Inf et al.
!! very much missing stuff!! where's the listing of mlrval types?!?
- reference-dsl-variables.md.in should link to it
- ditto programming-language.md.in
c UTs for r"a" . r"b" and so on and so on
reference-dsl-variables.md:
? IFLATSEP & OFLATSEP -- ?
reference-dsl-variables.md: FLATSEP
reference-main-null-data:
? more variants of is_... ?
? Records with one or more empty sort-field values sort after records with all sort-field values present -> apparently not true for sort -nr
e split out h2's
! spell out JSON-null
reference-main-arithmetic:
? test stats1/step -F flag

View file

@ -126,9 +126,6 @@ If you `mlr csv cat` this, you'll get an error message:
</pre>
<pre class="pre-non-highlight-in-pair">
mlr : Miller: CSV header/data length mismatch 3 != 2 at filename data/het/ragged.csv row 3.
a,b,c
1,2,3
</pre>
There are two kinds of raggedness here. Since CSVs form records by zipping the

View file

@ -144,6 +144,28 @@ avoid this, use the dot operator for string-concatenation instead.
Similarly, a final newline is printed for you; use [`printn`](reference-dsl-output-statements.md#print-statements) to avoid this.
## String literals with double quotes only
In some languages, like Ruby and Bash, string literals can be in single quotes or double quotes,
where single quotes suppress `\n` converting to a newline character and double quotes allowing it:
`'a\nb'` prints as the four characters `a`, `\`, `n`, and `b` on one line; `"a\nb"` prints as an
`a` on one line and a `b` on another.
In others, like Python and JavaScript, string literals can be in single quotes or double quotes,
interchangeably -- so you can have `"don't"` or `'the "right" thing'` as you wish.
In yet others, such as C/C++ and Java, string literals are in double auotes, like `"abc"`,
while single quotes are for character literals like `'a'` or `'\n'`. In these, if `s` is a non-empty string,
then `s[0]` is its first character.
In the [Miller programming language](programming-language.md):
* String literals are always in double quotes, like `"abc"`.
* String-indexing/slicing always results in strings (even of length 1): `"abc"[1:1]` is the string `"a"`, and there is no notion in the Miller programming language of a character type.
* The single-quote character plays no role whatsoever in the grammar of the Miller programming language.
* Single quotes are reserved for wrapping expressions at the system command line. For example, in `mlr put '$message = "hello"' ...`, the [`put` verb](reference-dsl.md) gets the string `$message = "hello"`; the shell has consumed the outer single quotes by the time the Miller parser receives it.
* Things are a little different on Windows, where `"""` sequences are sometimes necessary: see the [Miller on Windows page](miller-on-windows.md).
## Absent-null
Miller has a somewhat novel flavor of null data called _absent_: if a record
@ -195,7 +217,7 @@ Miller has a [key-value loop flavor](reference-dsl-control-structures.md#key-val
Miller also has a [single-variable loop flavor](reference-dsl-control-structures.md#single-variable-for-loops). If `x` is a map then `for (e in x) { ... }` binds `e` to successive map _keys_ (not values as in PHP). But if `x` is an array then `for e in x) { ... }` binds `e` to successive array _values_ (not indices).
## JSON parse, stringify, decode, encode
## JSON parse, stringify, decode, and encode
Miller has the verbs
[`json-parse`](reference-verbs.md#json-parse) and

View file

@ -116,6 +116,28 @@ GENMD_EOF
Similarly, a final newline is printed for you; use [`printn`](reference-dsl-output-statements.md#print-statements) to avoid this.
## String literals with double quotes only
In some languages, like Ruby and Bash, string literals can be in single quotes or double quotes,
where single quotes suppress `\n` converting to a newline character and double quotes allowing it:
`'a\nb'` prints as the four characters `a`, `\`, `n`, and `b` on one line; `"a\nb"` prints as an
`a` on one line and a `b` on another.
In others, like Python and JavaScript, string literals can be in single quotes or double quotes,
interchangeably -- so you can have `"don't"` or `'the "right" thing'` as you wish.
In yet others, such as C/C++ and Java, string literals are in double auotes, like `"abc"`,
while single quotes are for character literals like `'a'` or `'\n'`. In these, if `s` is a non-empty string,
then `s[0]` is its first character.
In the [Miller programming language](programming-language.md):
* String literals are always in double quotes, like `"abc"`.
* String-indexing/slicing always results in strings (even of length 1): `"abc"[1:1]` is the string `"a"`, and there is no notion in the Miller programming language of a character type.
* The single-quote character plays no role whatsoever in the grammar of the Miller programming language.
* Single quotes are reserved for wrapping expressions at the system command line. For example, in `mlr put '$message = "hello"' ...`, the [`put` verb](reference-dsl.md) gets the string `$message = "hello"`; the shell has consumed the outer single quotes by the time the Miller parser receives it.
* Things are a little different on Windows, where `"""` sequences are sometimes necessary: see the [Miller on Windows page](miller-on-windows.md).
## Absent-null
Miller has a somewhat novel flavor of null data called _absent_: if a record
@ -156,7 +178,7 @@ Miller has a [key-value loop flavor](reference-dsl-control-structures.md#key-val
Miller also has a [single-variable loop flavor](reference-dsl-control-structures.md#single-variable-for-loops). If `x` is a map then `for (e in x) { ... }` binds `e` to successive map _keys_ (not values as in PHP). But if `x` is an array then `for e in x) { ... }` binds `e` to successive array _values_ (not indices).
## JSON parse, stringify, decode, encode
## JSON parse, stringify, decode, and encode
Miller has the verbs
[`json-parse`](reference-verbs.md#json-parse) and

View file

@ -162,9 +162,9 @@ If you want to auto-extend an [array](reference-main-arrays.md), initialize it e
Once an array is initialized, it can be extended by assigning to indices beyond
its length. If each write is one past the end of the array, the array will
grow by one. (Memory management, handled for you, is a little more careful: not
to worry, capacity is doubled so performance doesn't suffer a rellocate on
every single extend.)
grow by one. (Memory management, handled for you, is careful handled here in
Miller: not to worry, capacity is doubled so performance doesn't suffer a
rellocate on every single extend.)
This is important in Miller so you can do things like `@records[NR] = $*` with
a minimum of keystrokes without worrying about explicitly resizing arrays. In

View file

@ -108,9 +108,9 @@ GENMD_EOF
Once an array is initialized, it can be extended by assigning to indices beyond
its length. If each write is one past the end of the array, the array will
grow by one. (Memory management, handled for you, is a little more careful: not
to worry, capacity is doubled so performance doesn't suffer a rellocate on
every single extend.)
grow by one. (Memory management, handled for you, is careful handled here in
Miller: not to worry, capacity is doubled so performance doesn't suffer a
rellocate on every single extend.)
This is important in Miller so you can do things like `@records[NR] = $*` with
a minimum of keystrokes without worrying about explicitly resizing arrays. In

View file

@ -125,6 +125,12 @@ of words would result in some strings mixed with some booleans. Use the
`boolean` function to coerce: e.g. giving the record `x=1,y=2,w=false` to `mlr
filter '$z=($x<$y) || boolean($w)'`.
The same is true for `inf`, `+inf`, `-inf`, `infinity`, `+infinity`,
`-infinity`, `NaN`, and all upper-cased/lower-cased/mixed-case variants of
those. These are valid IEEE floating-point numbers, but Miller treats these as
strings. You can explicit force conversion: if `x=infinity` in a data file,
then `typeof($x)` is `string` but `typeof(float($x))` is `float`.
## JSON parse and stringify
If you have, say, a CSV file whose columns contain strings which are well-formatted JSON,

View file

@ -75,6 +75,12 @@ of words would result in some strings mixed with some booleans. Use the
`boolean` function to coerce: e.g. giving the record `x=1,y=2,w=false` to `mlr
filter '$z=($x<$y) || boolean($w)'`.
The same is true for `inf`, `+inf`, `-inf`, `infinity`, `+infinity`,
`-infinity`, `NaN`, and all upper-cased/lower-cased/mixed-case variants of
those. These are valid IEEE floating-point numbers, but Miller treats these as
strings. You can explicit force conversion: if `x=infinity` in a data file,
then `typeof($x)` is `string` but `typeof(float($x))` is `float`.
## JSON parse and stringify
If you have, say, a CSV file whose columns contain strings which are well-formatted JSON,

View file

@ -22,7 +22,7 @@ Miller has three kinds of null data:
* **Absent (key not present)**: a field name is not present, e.g. input record is `x=1,y=2` and a `put` or `filter` expression refers to `$z`. Or, reading an out-of-stream variable which hasn't been assigned a value yet, e.g. `mlr put -q '@sum += $x; end{emit @sum}'` or `mlr put -q '@sum[$a][$b] += $x; end{emit @sum, "a", "b"}'`.
* **JSON null**: (TODO: describe)
* **JSON null**: The main purpose of this is to support reading the `null` type in JSON files. The [Miller programming language](programming-language.md) has a `null` keyword as well, so you can also write the null type using `$x = null`. Addtionally, though, when you write past the end of an array, leaving gaps -- e.g. writing `a[12]` when the array `a` has length 10 -- JSON-null is used to fill the gaps. See also the [arrays page](reference-main-arrays.md#auto-extend-and-null-gaps).
You can test these programatically using the functions `is_empty`/`is_not_empty`, `is_absent`/`is_present`, and `is_null`/`is_not_null`. For the last pair, note that null means either empty or absent.

View file

@ -8,7 +8,7 @@ Miller has three kinds of null data:
* **Absent (key not present)**: a field name is not present, e.g. input record is `x=1,y=2` and a `put` or `filter` expression refers to `$z`. Or, reading an out-of-stream variable which hasn't been assigned a value yet, e.g. `mlr put -q '@sum += $x; end{emit @sum}'` or `mlr put -q '@sum[$a][$b] += $x; end{emit @sum, "a", "b"}'`.
* **JSON null**: (TODO: describe)
* **JSON null**: The main purpose of this is to support reading the `null` type in JSON files. The [Miller programming language](programming-language.md) has a `null` keyword as well, so you can also write the null type using `$x = null`. Addtionally, though, when you write past the end of an array, leaving gaps -- e.g. writing `a[12]` when the array `a` has length 10 -- JSON-null is used to fill the gaps. See also the [arrays page](reference-main-arrays.md#auto-extend-and-null-gaps).
You can test these programatically using the functions `is_empty`/`is_not_empty`, `is_absent`/`is_present`, and `is_null`/`is_not_null`. For the last pair, note that null means either empty or absent.

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab filter -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab filter -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --oxtab put -f ${CASEDIR}/mlr ./${CASEDIR}/input
mlr --oxtab put -f ${CASEDIR}/mlr ./${CASEDIR}/input

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -F -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -1 +1 @@
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab
mlr --xtab put -f ${CASEDIR}/mlr regtest/input/mixed-types.xtab

View file

@ -0,0 +1 @@
mlr --idkvp --opprint put -f ${CASEDIR}/mlr ./${CASEDIR}/input

View file

@ -0,0 +1,33 @@
a ta
12 int
1.2 float
inf string
+inf string
-inf string
Inf string
+Inf string
-Inf string
INF string
+INF string
-INF string
infinity string
+infinity string
-infinity string
Infinity string
+Infinity string
-Infinity string
INFINITY string
+INFINITY string
-INFINITY string
InFiNiTy string
+InFiNiTy string
-InFiNiTy string
infiniti string
+infiniti string
-infiniti string
true string
false string
True string
False string
TRUE string
FALSE string

View file

@ -0,0 +1,32 @@
a=12
a=1.2
a=inf
a=+inf
a=-inf
a=Inf
a=+Inf
a=-Inf
a=INF
a=+INF
a=-INF
a=infinity
a=+infinity
a=-infinity
a=Infinity
a=+Infinity
a=-Infinity
a=INFINITY
a=+INFINITY
a=-INFINITY
a=InFiNiTy
a=+InFiNiTy
a=-InFiNiTy
a=infiniti
a=+infiniti
a=-infiniti
a=true
a=false
a=True
a=False
a=TRUE
a=FALSE

View file

@ -0,0 +1 @@
$ta = typeof($a)

View file

@ -6,6 +6,7 @@ package types
import (
"errors"
"strings"
"mlr/src/lib"
)
@ -101,15 +102,14 @@ func MlrvalPointerFromBoolString(input string) *Mlrval {
}
}
var floatNamesToNotInfer = map[string]bool{
"Inf": true,
"+Inf": true,
"-Inf": true,
"Infinity": true,
"+Infinity": true,
"-Infinity": true,
var downcasedFloatNamesToNotInfer = map[string]bool{
"inf": true,
"+inf": true,
"-inf": true,
"infinity": true,
"+infinity": true,
"-infinity": true,
"nan": true,
"NaN": true,
}
// MlrvalPointerFromInferredTypeForDataFiles is for parsing field values from
@ -126,7 +126,7 @@ func MlrvalPointerFromInferredTypeForDataFiles(input string) *Mlrval {
return MlrvalPointerFromIntString(input)
}
if floatNamesToNotInfer[input] == false {
if downcasedFloatNamesToNotInfer[strings.ToLower(input)] == false {
_, fok := lib.TryFloat64FromString(input)
if fok {
return MlrvalPointerFromFloat64String(input)

View file

@ -113,6 +113,8 @@ w survey
----------------------------------------------------------------
BLOCKERS
* UTs for r"a" . r"b" and so on and so on
* document precedence issue:
! probably remove this feature !