mirror of
https://github.com/johnkerl/miller.git
synced 2026-01-24 02:36:15 +00:00
102 lines
No EOL
11 KiB
HTML
102 lines
No EOL
11 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<title>Reference: arithmetic — Miller 6.0.0-alpha documentation</title>
|
||
|
||
<link rel="stylesheet" href="_static/scrolls.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/print.css" type="text/css" />
|
||
|
||
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
|
||
<script src="_static/jquery.js"></script>
|
||
<script src="_static/underscore.js"></script>
|
||
<script src="_static/doctools.js"></script>
|
||
<script src="_static/language_data.js"></script>
|
||
<script src="_static/theme_extras.js"></script>
|
||
<link rel="index" title="Index" href="genindex.html" />
|
||
<link rel="search" title="Search" href="search.html" />
|
||
<link rel="next" title="Reference: regular expressions" href="reference-main-regular-expressions.html" />
|
||
<link rel="prev" title="Reference: null data" href="reference-main-null-data.html" />
|
||
</head><body>
|
||
<div id="content">
|
||
<div class="header">
|
||
<h1 class="heading"><a href="index.html"
|
||
title="back to the documentation overview"><span>Reference: arithmetic</span></a></h1>
|
||
</div>
|
||
<div class="relnav" role="navigation" aria-label="related navigation">
|
||
<a href="reference-main-null-data.html">« Reference: null data</a> |
|
||
<a href="#">Reference: arithmetic</a>
|
||
| <a href="reference-main-regular-expressions.html">Reference: regular expressions »</a>
|
||
</div>
|
||
<div id="contentwrapper">
|
||
<div id="toc" role="navigation" aria-label="table of contents navigation">
|
||
<h3>Table of Contents</h3>
|
||
<ul>
|
||
<li><a class="reference internal" href="#">Reference: arithmetic</a><ul>
|
||
<li><a class="reference internal" href="#input-scanning">Input scanning</a></li>
|
||
<li><a class="reference internal" href="#conversion-by-math-routines">Conversion by math routines</a></li>
|
||
<li><a class="reference internal" href="#conversion-by-arithmetic-operators">Conversion by arithmetic operators</a></li>
|
||
<li><a class="reference internal" href="#pythonic-division">Pythonic division</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
</div>
|
||
<div role="main">
|
||
|
||
<div class="section" id="reference-arithmetic">
|
||
<h1>Reference: arithmetic<a class="headerlink" href="#reference-arithmetic" title="Permalink to this headline">¶</a></h1>
|
||
<div class="section" id="input-scanning">
|
||
<h2>Input scanning<a class="headerlink" href="#input-scanning" title="Permalink to this headline">¶</a></h2>
|
||
<p>Numbers in Miller are double-precision float or 64-bit signed integers. Anything scannable as int, e.g <code class="docutils literal notranslate"><span class="pre">123</span></code> or <code class="docutils literal notranslate"><span class="pre">0xabcd</span></code>, is treated as an integer; otherwise, input scannable as float (<code class="docutils literal notranslate"><span class="pre">4.56</span></code> or <code class="docutils literal notranslate"><span class="pre">8e9</span></code>) is treated as float; everything else is a string.</p>
|
||
<p>If you want all numbers to be treated as floats, then you may use <code class="docutils literal notranslate"><span class="pre">float()</span></code> in your filter/put expressions (e.g. replacing <code class="docutils literal notranslate"><span class="pre">$c</span> <span class="pre">=</span> <span class="pre">$a</span> <span class="pre">*</span> <span class="pre">$b</span></code> with <code class="docutils literal notranslate"><span class="pre">$c</span> <span class="pre">=</span> <span class="pre">float($a)</span> <span class="pre">*</span> <span class="pre">float($b)</span></code>) – or, more simply, use <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">filter</span> <span class="pre">-F</span></code> and <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">put</span> <span class="pre">-F</span></code> which forces all numeric input, whether from expression literals or field values, to float. Likewise <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">stats1</span> <span class="pre">-F</span></code> and <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">step</span> <span class="pre">-F</span></code> force integerable accumulators (such as <code class="docutils literal notranslate"><span class="pre">count</span></code>) to be done in floating-point.</p>
|
||
</div>
|
||
<div class="section" id="conversion-by-math-routines">
|
||
<h2>Conversion by math routines<a class="headerlink" href="#conversion-by-math-routines" title="Permalink to this headline">¶</a></h2>
|
||
<p>For most math functions, integers are cast to float on input, and produce float output: e.g. <code class="docutils literal notranslate"><span class="pre">exp(0)</span> <span class="pre">=</span> <span class="pre">1.0</span></code> rather than <code class="docutils literal notranslate"><span class="pre">1</span></code>. The following, however, produce integer output if their inputs are integers: <code class="docutils literal notranslate"><span class="pre">+</span></code> <code class="docutils literal notranslate"><span class="pre">-</span></code> <code class="docutils literal notranslate"><span class="pre">*</span></code> <code class="docutils literal notranslate"><span class="pre">/</span></code> <code class="docutils literal notranslate"><span class="pre">//</span></code> <code class="docutils literal notranslate"><span class="pre">%</span></code> <code class="docutils literal notranslate"><span class="pre">abs</span></code> <code class="docutils literal notranslate"><span class="pre">ceil</span></code> <code class="docutils literal notranslate"><span class="pre">floor</span></code> <code class="docutils literal notranslate"><span class="pre">max</span></code> <code class="docutils literal notranslate"><span class="pre">min</span></code> <code class="docutils literal notranslate"><span class="pre">round</span></code> <code class="docutils literal notranslate"><span class="pre">roundm</span></code> <code class="docutils literal notranslate"><span class="pre">sgn</span></code>. As well, <code class="docutils literal notranslate"><span class="pre">stats1</span> <span class="pre">-a</span> <span class="pre">min</span></code>, <code class="docutils literal notranslate"><span class="pre">stats1</span> <span class="pre">-a</span> <span class="pre">max</span></code>, <code class="docutils literal notranslate"><span class="pre">stats1</span> <span class="pre">-a</span> <span class="pre">sum</span></code>, <code class="docutils literal notranslate"><span class="pre">step</span> <span class="pre">-a</span> <span class="pre">delta</span></code>, and <code class="docutils literal notranslate"><span class="pre">step</span> <span class="pre">-a</span> <span class="pre">rsum</span></code> produce integer output if their inputs are integers.</p>
|
||
</div>
|
||
<div class="section" id="conversion-by-arithmetic-operators">
|
||
<h2>Conversion by arithmetic operators<a class="headerlink" href="#conversion-by-arithmetic-operators" title="Permalink to this headline">¶</a></h2>
|
||
<p>The sum, difference, and product of integers is again integer, except for when that would overflow a 64-bit integer at which point Miller converts the result to float.</p>
|
||
<p>The short of it is that Miller does this transparently for you so you needn’t think about it.</p>
|
||
<p>Implementation details of this, for the interested: integer adds and subtracts overflow by at most one bit so it suffices to check sign-changes. Thus, Miller allows you to add and subtract arbitrary 64-bit signed integers, converting only to float precisely when the result is less than -2<sup>63</sup> or greater than 2<sup>63</sup>-1. Multiplies, on the other hand, can overflow by a word size and a sign-change technique does not suffice to detect overflow. Instead Miller tests whether the floating-point product exceeds the representable integer range. Now, 64-bit integers have 64-bit precision while IEEE-doubles have only 52-bit mantissas – so, there are 53 bits including implicit leading one. The following experiment explicitly demonstrates the resolution at this range:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>64-bit integer 64-bit integer Casted to double Back to 64-bit
|
||
in hex in decimal integer
|
||
0x7ffffffffffff9ff 9223372036854774271 9223372036854773760.000000 0x7ffffffffffff800
|
||
0x7ffffffffffffa00 9223372036854774272 9223372036854773760.000000 0x7ffffffffffff800
|
||
0x7ffffffffffffbff 9223372036854774783 9223372036854774784.000000 0x7ffffffffffffc00
|
||
0x7ffffffffffffc00 9223372036854774784 9223372036854774784.000000 0x7ffffffffffffc00
|
||
0x7ffffffffffffdff 9223372036854775295 9223372036854774784.000000 0x7ffffffffffffc00
|
||
0x7ffffffffffffe00 9223372036854775296 9223372036854775808.000000 0x8000000000000000
|
||
0x7ffffffffffffffe 9223372036854775806 9223372036854775808.000000 0x8000000000000000
|
||
0x7fffffffffffffff 9223372036854775807 9223372036854775808.000000 0x8000000000000000
|
||
</pre></div>
|
||
</div>
|
||
<p>That is, one cannot check an integer product to see if it is precisely greater than 2<sup>63</sup>-1 or less than -2<sup>63</sup> using either integer arithmetic (it may have already overflowed) or using double-precision (due to granularity). Instead Miller checks for overflow in 64-bit integer multiplication by seeing whether the absolute value of the double-precision product exceeds the largest representable IEEE double less than 2<sup>63</sup>, which we see from the listing above is 9223372036854774784. (An alternative would be to do all integer multiplies using handcrafted multi-word 128-bit arithmetic. This approach is not taken.)</p>
|
||
</div>
|
||
<div class="section" id="pythonic-division">
|
||
<h2>Pythonic division<a class="headerlink" href="#pythonic-division" title="Permalink to this headline">¶</a></h2>
|
||
<p>Division and remainder are <a class="reference external" href="http://python-history.blogspot.com/2010/08/why-pythons-integer-division-floors.html">pythonic</a>:</p>
|
||
<ul class="simple">
|
||
<li><p>Quotient of integers is floating-point: <code class="docutils literal notranslate"><span class="pre">7/2</span></code> is <code class="docutils literal notranslate"><span class="pre">3.5</span></code>.</p></li>
|
||
<li><p>Integer division is done with <code class="docutils literal notranslate"><span class="pre">//</span></code>: <code class="docutils literal notranslate"><span class="pre">7//2</span></code> is <code class="docutils literal notranslate"><span class="pre">3</span></code>. This rounds toward the negative.</p></li>
|
||
<li><p>Remainders are non-negative.</p></li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="footer" role="contentinfo">
|
||
© Copyright 2021, John Kerl.
|
||
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
|
||
</div>
|
||
</body>
|
||
</html> |