+
+Rules for null-handling:
+
+- Records with one or more null sort-field values sort after records with all sort-field values present:
+
+
- 64-bit integer 64-bit integer Casted to double Back to 64-bit
- in hex in decimal integer
-0x7ffffffffffff9ff 9223372036854774271 9223372036854773760.000000 0x7ffffffffffff800
-0x7ffffffffffffa00 9223372036854774272 9223372036854773760.000000 0x7ffffffffffff800
-0x7ffffffffffffbff 9223372036854774783 9223372036854774784.000000 0x7ffffffffffffc00
-0x7ffffffffffffc00 9223372036854774784 9223372036854774784.000000 0x7ffffffffffffc00
-0x7ffffffffffffdff 9223372036854775295 9223372036854774784.000000 0x7ffffffffffffc00
-0x7ffffffffffffe00 9223372036854775296 9223372036854775808.000000 0x8000000000000000
-0x7ffffffffffffffe 9223372036854775806 9223372036854775808.000000 0x8000000000000000
-0x7fffffffffffffff 9223372036854775807 9223372036854775808.000000 0x8000000000000000
+$ mlr cat data/sort-null.dat
+a=3,b=2
+a=1,b=8
+a=,b=4
+x=9,b=10
+a=5,b=7
+
+
+
+
+$ mlr sort -n a data/sort-null.dat
+a=1,b=8
+a=3,b=2
+a=5,b=7
+a=,b=4
+x=9,b=10
+
+
+
+$ mlr sort -nr a data/sort-null.dat
+a=,b=4
+a=5,b=7
+a=3,b=2
+a=1,b=8
+x=9,b=10
+
+
- Functions which have one or more null arguments produce null output: e.g.
+
+
+
+$ echo 'x=2,y=3' | mlr put '$a=$x+$y'
+x=2,y=3,a=5
+
+
+
+$ echo 'x=,y=3' | mlr put '$a=$x+$y'
+x=,y=3,a=
+
+
+
+$ echo 'x=,y=3' | mlr put '$a=log($x);$b=log($y)'
+x=,y=3,a=,b=1.098612
+
+
- The min and max functions are special: if one argument is non-null, it wins:
+
+
+
+$ echo 'x=,y=3' | mlr put '$a=min($x,$y);$b=max($x,$y)'
+x=,y=3,a=3,b=3
+
+
-
That is, one cannot check an integer product to see if it is precisely
-greater than 2
63-1 or less than -2
63 using either integer
-arithmetic (it may have already overflowed) or using double-precision (due to
-granularity). Instead Miller checks for overflow in 64-bit integer
-multiplication by seeing whether the absolute value of the double-precision
-product exceeds the largest representable IEEE double less than 2
63,
-which we see from the listing above is 9223372036854774784. (An alternative
-would be to do all integer multiplies using handcrafted multi-word 128-bit
-arithmetic. This approach is not taken.)
+
String literals
@@ -5142,6 +5031,148 @@ mlr filter '$a =~ "(..)_(....)'
\0 is the entire match string;
\15 is treated as
\1
followed by an unrelated
5.
+
+
+
Operator precedence
+
+
Operators are listed in order of decreasing precedence, highest first.
+
+
+
+
+Operators Associativity
+--------- -------------
+() left to right
+** right to left
+! ~ unary+ unary- & right to left
+binary* / // % left to right
+binary+ binary- . left to right
+<< >> left to right
+& left to right
+^ left to right
+| left to right
+< <= > >= left to right
+== != =~ !=~ left to right
+&& left to right
+^^ left to right
+|| left to right
+? : right to left
+= N/A for Miller (there is no $a=$b=$c)
+
+
+
+
+
+
Operator and function semantics
+
+
+
+ Functions are in general pass-throughs straight to the system-standard C
+library.
+
+ The min and max functions are different from other
+multi-argument functions which return null if any of their inputs are null: for
+min and max, by contrast, if one argument is null, the other
+is returned.
+
+ Symmetrically with respect to the bitwise OR, XOR, and AND operators
+|, ^, &, Miller has logical operators
+||, ^^, &&: the logical XOR not existing in
+C.
+
+ The exponentiation operator ** is familiar from many languages.
+
+ The regex-match and regex-not-match operators =~ and
+!=~ are similar to those in Ruby and Perl.
+
+
+
+
+
Arithmetic
+
+
Input scanning
+
+
Numbers in Miller are double-precision float or 64-bit signed integers.
+Anything scannable as int, e.g
123 or
0xabcd, is treated as
+an integer; otherwise, input scannable as float (
4.56 or
8e9)
+is treated as float; everything else is a string.
+
+
If you want all numbers to be treated as floats, then you may use
+
float() in your filter/put expressions (e.g. replacing
$c = $a *
+$b with
$c = float($a) * float($b)) — or, more simply, use
+
mlr filter -F and
mlr put -F which forces all numeric input,
+whether from expression literals or field values, to float. Likewise
mlr
+stats1 -F and
mlr step -F force integerable accumulators (such as
+
count) to be done in floating-point.
+
+
Conversion by math routines
+
+
For most math functions, integers are cast to float on input, and produce
+float output: e.g.
exp(0) = 1.0 rather than
1. The
+following, however, produce integer output if their inputs are integers:
+
+ - * / // % abs
+
ceil floor max min round
+
roundm sgn. As well,
stats1 -a min,
stats1 -a
+max,
stats1 -a sum,
step -a delta, and
step -a
+rsum produce integer output if their inputs are integers.
+
+
Conversion by arithmetic operators
+
+
The sum, difference, and product of integers is again integer, except for
+when that would overflow a 64-bit integer at which point Miller converts the
+result to float.
+
+
The short of it is that Miller does this transparently for you so you
+needn’t think about it.
+
+
Implementation details of this, for the interested: integer adds and
+subtracts overflow by at most one bit so it suffices to check sign-changes.
+Thus, Miller allows you to add and subtract arbitrary 64-bit signed integers,
+converting only to float precisely when the result is less than -2
63
+or greater than 2
63-1. Multiplies, on the other hand, can overflow
+by a word size and a sign-change technique does not suffice to detect overflow.
+Instead Miller tests whether the floating-point product exceeds the
+representable integer range. Now, 64-bit integers have 64-bit precision while
+IEEE-doubles have only 52-bit mantissas — so, there are 53 bits including
+implicit leading one. The following experiment explicitly demonstrates the
+resolution at this range:
+
+
+
+ 64-bit integer 64-bit integer Casted to double Back to 64-bit
+ in hex in decimal integer
+0x7ffffffffffff9ff 9223372036854774271 9223372036854773760.000000 0x7ffffffffffff800
+0x7ffffffffffffa00 9223372036854774272 9223372036854773760.000000 0x7ffffffffffff800
+0x7ffffffffffffbff 9223372036854774783 9223372036854774784.000000 0x7ffffffffffffc00
+0x7ffffffffffffc00 9223372036854774784 9223372036854774784.000000 0x7ffffffffffffc00
+0x7ffffffffffffdff 9223372036854775295 9223372036854774784.000000 0x7ffffffffffffc00
+0x7ffffffffffffe00 9223372036854775296 9223372036854775808.000000 0x8000000000000000
+0x7ffffffffffffffe 9223372036854775806 9223372036854775808.000000 0x8000000000000000
+0x7fffffffffffffff 9223372036854775807 9223372036854775808.000000 0x8000000000000000
+
+
+
+
That is, one cannot check an integer product to see if it is precisely
+greater than 2
63-1 or less than -2
63 using either integer
+arithmetic (it may have already overflowed) or using double-precision (due to
+granularity). Instead Miller checks for overflow in 64-bit integer
+multiplication by seeing whether the absolute value of the double-precision
+product exceeds the largest representable IEEE double less than 2
63,
+which we see from the listing above is 9223372036854774784. (An alternative
+would be to do all integer multiplies using handcrafted multi-word 128-bit
+arithmetic. This approach is not taken.)
+
+
Pythonic division
+
+
Division and remainder are
+
+pythonic:
+
+ Quotient of integers is floating-point: 7/2 is 3.5.
+ Integer division is done with //: 7/2 is 3.
+ This rounds toward the negative.
+ Remainders are non-negative.
+