mirror of
https://github.com/johnkerl/miller.git
synced 2026-01-24 02:36:15 +00:00
374 lines
No EOL
26 KiB
HTML
374 lines
No EOL
26 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<title>Intro to Miller’s programming language — Miller 6.0.0-alpha documentation</title>
|
||
|
||
<link rel="stylesheet" href="_static/scrolls.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/print.css" type="text/css" />
|
||
|
||
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
|
||
<script src="_static/jquery.js"></script>
|
||
<script src="_static/underscore.js"></script>
|
||
<script src="_static/doctools.js"></script>
|
||
<script src="_static/language_data.js"></script>
|
||
<script src="_static/theme_extras.js"></script>
|
||
<link rel="index" title="Index" href="genindex.html" />
|
||
<link rel="search" title="Search" href="search.html" />
|
||
<link rel="next" title="Miller on Windows" href="miller-on-windows.html" />
|
||
<link rel="prev" title="Keystroke-savers" href="keystroke-savers.html" />
|
||
</head><body>
|
||
<div id="content">
|
||
<div class="header">
|
||
<h1 class="heading"><a href="index.html"
|
||
title="back to the documentation overview"><span>Intro to Miller’s programming language</span></a></h1>
|
||
</div>
|
||
<div class="relnav" role="navigation" aria-label="related navigation">
|
||
<a href="keystroke-savers.html">« Keystroke-savers</a> |
|
||
<a href="#">Intro to Miller’s programming language</a>
|
||
| <a href="miller-on-windows.html">Miller on Windows »</a>
|
||
</div>
|
||
<div id="contentwrapper">
|
||
<div id="toc" role="navigation" aria-label="table of contents navigation">
|
||
<h3>Table of Contents</h3>
|
||
<ul>
|
||
<li><a class="reference internal" href="#">Intro to Miller’s programming language</a><ul>
|
||
<li><a class="reference internal" href="#records-and-fields">Records and fields</a></li>
|
||
<li><a class="reference internal" href="#multi-line-statements-and-statements-from-file">Multi-line statements, and statements-from-file</a></li>
|
||
<li><a class="reference internal" href="#out-of-stream-variables-begin-and-end">Out-of-stream variables, begin, and end</a></li>
|
||
<li><a class="reference internal" href="#context-variables">Context variables</a></li>
|
||
<li><a class="reference internal" href="#functions-and-local-variables">Functions and local variables</a></li>
|
||
<li><a class="reference internal" href="#if-statements-loops-and-local-variables">If-statements, loops, and local variables</a></li>
|
||
<li><a class="reference internal" href="#arithmetic">Arithmetic</a></li>
|
||
<li><a class="reference internal" href="#absent-data">Absent data</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
</div>
|
||
<div role="main">
|
||
|
||
<div class="section" id="intro-to-miller-s-programming-language">
|
||
<h1>Intro to Miller’s programming language<a class="headerlink" href="#intro-to-miller-s-programming-language" title="Permalink to this headline">¶</a></h1>
|
||
<p>In the <a class="reference internal" href="10min.html"><span class="doc">Miller in 10 minutes</span></a> page we took a tour of some of Miller’s most-used verbs including <code class="docutils literal notranslate"><span class="pre">cat</span></code>, <code class="docutils literal notranslate"><span class="pre">head</span></code>, <code class="docutils literal notranslate"><span class="pre">tail</span></code>, <code class="docutils literal notranslate"><span class="pre">cut</span></code>, and <code class="docutils literal notranslate"><span class="pre">sort</span></code>. These are analogs of familiar system commands, but empowered by field-name indexing and file-format awareness: the system <code class="docutils literal notranslate"><span class="pre">sort</span></code> command only knows about lines and column names like <code class="docutils literal notranslate"><span class="pre">1,2,3,4</span></code>, while <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">sort</span></code> knows about CSV/TSV/JSON/etc records, and field names like <code class="docutils literal notranslate"><span class="pre">color,shape,flag,index</span></code>.</p>
|
||
<p>We also caught a glimpse of Miller’s <code class="docutils literal notranslate"><span class="pre">put</span></code> and <code class="docutils literal notranslate"><span class="pre">filter</span></code> verbs. These two are special since they let you express statements using Miller’s programming language. It’s a <em>embedded domain-specific language</em> since it’s inside Miller: often referred to simply as the <em>Miller DSL</em>.</p>
|
||
<p>In the <a class="reference internal" href="reference-dsl.html"><span class="doc">DSL reference: overview</span></a> page we have a complete reference to Miller’s programming language. For now, let’s take a quick look at key features – you can use as few or as many features as you like.</p>
|
||
<div class="section" id="records-and-fields">
|
||
<h2>Records and fields<a class="headerlink" href="#records-and-fields" title="Permalink to this headline">¶</a></h2>
|
||
<p>Let’s keep using the sample <a class="reference external" href="./example.csv">example.csv</a>. When we type</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p put '$cost = $quantity * $rate' example.csv
|
||
</span> color shape flag index quantity rate cost
|
||
yellow triangle true 11 43.6498 9.8870 431.5655726
|
||
red square true 15 79.2778 0.0130 1.0306114
|
||
red circle true 16 13.8103 2.9010 40.063680299999994
|
||
red square false 48 77.5542 7.4670 579.0972113999999
|
||
purple triangle false 51 81.2290 8.5910 697.8383389999999
|
||
red square false 64 77.1991 9.5310 735.7846221000001
|
||
purple triangle false 65 80.1405 5.8240 466.738272
|
||
yellow circle true 73 63.9785 4.2370 271.0769045
|
||
yellow circle true 87 63.5058 8.3350 529.3208430000001
|
||
purple square false 91 72.3735 8.2430 596.5747605000001
|
||
</pre></div>
|
||
</div>
|
||
<p>a few things are happening:</p>
|
||
<ul class="simple">
|
||
<li><p>We refer to fields in the input data using a dollar sign and then the field name, e.g. <code class="docutils literal notranslate"><span class="pre">$quantity</span></code>. (If a field name has special characters like a dot or slash, just use curly braces: <code class="docutils literal notranslate"><span class="pre">${field.name}</span></code>.)</p></li>
|
||
<li><p>The expression <code class="docutils literal notranslate"><span class="pre">$cost</span> <span class="pre">=</span> <span class="pre">$quantity</span> <span class="pre">*</span> <span class="pre">$rate</span></code> is executed once per record of the data file. Our <a class="reference external" href="./example.csv">example.csv</a> has 10 records so this expression was executed 10 times, with the field names <code class="docutils literal notranslate"><span class="pre">$quantity</span></code> and <code class="docutils literal notranslate"><span class="pre">$rate</span></code> bound to the current record’s values for those fields.</p></li>
|
||
<li><p>On the left-hand side we have the new field name <code class="docutils literal notranslate"><span class="pre">$cost</span></code> which didn’t come from the input data. Assignments to new variables result in a new field being placed after all the other ones. If we’d assigned to an existing field name, it would have been updated in-place.</p></li>
|
||
<li><p>The entire expression is surrounded by single quotes, to get it past the system shell. Inside those, only double quotes have meaning in Miller’s programming language.</p></li>
|
||
</ul>
|
||
</div>
|
||
<div class="section" id="multi-line-statements-and-statements-from-file">
|
||
<h2>Multi-line statements, and statements-from-file<a class="headerlink" href="#multi-line-statements-and-statements-from-file" title="Permalink to this headline">¶</a></h2>
|
||
<p>You can use more than one statement, separating them with semicolons, and optionally putting them on lines of their own:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p put '$cost = $quantity * $rate; $index = $index * 100' example.csv
|
||
</span> color shape flag index quantity rate cost
|
||
yellow triangle true 1100 43.6498 9.8870 431.5655726
|
||
red square true 1500 79.2778 0.0130 1.0306114
|
||
red circle true 1600 13.8103 2.9010 40.063680299999994
|
||
red square false 4800 77.5542 7.4670 579.0972113999999
|
||
purple triangle false 5100 81.2290 8.5910 697.8383389999999
|
||
red square false 6400 77.1991 9.5310 735.7846221000001
|
||
purple triangle false 6500 80.1405 5.8240 466.738272
|
||
yellow circle true 7300 63.9785 4.2370 271.0769045
|
||
yellow circle true 8700 63.5058 8.3350 529.3208430000001
|
||
purple square false 9100 72.3735 8.2430 596.5747605000001
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p put '
|
||
</span><span class="hll"> $cost = $quantity * $rate;
|
||
</span><span class="hll"> $index *= 100
|
||
</span><span class="hll"> ' example.csv
|
||
</span> color shape flag index quantity rate cost
|
||
yellow triangle true 1100 43.6498 9.8870 431.5655726
|
||
red square true 1500 79.2778 0.0130 1.0306114
|
||
red circle true 1600 13.8103 2.9010 40.063680299999994
|
||
red square false 4800 77.5542 7.4670 579.0972113999999
|
||
purple triangle false 5100 81.2290 8.5910 697.8383389999999
|
||
red square false 6400 77.1991 9.5310 735.7846221000001
|
||
purple triangle false 6500 80.1405 5.8240 466.738272
|
||
yellow circle true 7300 63.9785 4.2370 271.0769045
|
||
yellow circle true 8700 63.5058 8.3350 529.3208430000001
|
||
purple square false 9100 72.3735 8.2430 596.5747605000001
|
||
</pre></div>
|
||
</div>
|
||
<p>One of Miller’s key features is the ability to express data-transformation right there at the keyboard, interactively. But if you find yourself using expressions repeatedly, you can put everything between the single quotes into a file and refer to that using <code class="docutils literal notranslate"><span class="pre">put</span> <span class="pre">-f</span></code>:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> cat dsl-example.mlr
|
||
</span> $cost = $quantity * $rate;
|
||
$index *= 100
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p put -f dsl-example.mlr example.csv
|
||
</span> color shape flag index quantity rate cost
|
||
yellow triangle true 1100 43.6498 9.8870 431.5655726
|
||
red square true 1500 79.2778 0.0130 1.0306114
|
||
red circle true 1600 13.8103 2.9010 40.063680299999994
|
||
red square false 4800 77.5542 7.4670 579.0972113999999
|
||
purple triangle false 5100 81.2290 8.5910 697.8383389999999
|
||
red square false 6400 77.1991 9.5310 735.7846221000001
|
||
purple triangle false 6500 80.1405 5.8240 466.738272
|
||
yellow circle true 7300 63.9785 4.2370 271.0769045
|
||
yellow circle true 8700 63.5058 8.3350 529.3208430000001
|
||
purple square false 9100 72.3735 8.2430 596.5747605000001
|
||
</pre></div>
|
||
</div>
|
||
<p>This becomes particularly important on Windows. Quite a bit of effort was put into making Miller on Windows be able to handle the kinds of single-quoted expressions we’re showing here, but if you get syntax-error messages on Windows using examples in this documentation, you can put the parts between single quotes into a file and refer to that using <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">put</span> <span class="pre">-f</span></code>.</p>
|
||
</div>
|
||
<div class="section" id="out-of-stream-variables-begin-and-end">
|
||
<h2>Out-of-stream variables, begin, and end<a class="headerlink" href="#out-of-stream-variables-begin-and-end" title="Permalink to this headline">¶</a></h2>
|
||
<p>Above we saw that your expression is executed once per record – if a file has a million records, your expression will be executed a million times, once for each record. But you can mark statements to only be executed once, either before the record stream begins, or after the record stream is ended. If you know about <a class="reference external" href="https://en.wikipedia.org/wiki/AWK">AWK</a>, you might have noticed that Miller’s programming language is loosely inspired by it, including the <code class="docutils literal notranslate"><span class="pre">begin</span></code> and <code class="docutils literal notranslate"><span class="pre">end</span></code> statements.</p>
|
||
<p>Above we also saw that names like <code class="docutils literal notranslate"><span class="pre">$quantity</span></code> are bound to each record in turn.</p>
|
||
<p>To make <code class="docutils literal notranslate"><span class="pre">begin</span></code> and <code class="docutils literal notranslate"><span class="pre">end</span></code> statements useful, we need somewhere to put things that persist across the duration of the record stream, and a way to emit them. Miller uses <strong>out-of-stream variables</strong> (or <strong>oosvars</strong> for short) whose names start with an <code class="docutils literal notranslate"><span class="pre">@</span></code> sigil, and the <strong>emit</strong> keyword to write them into the output record stream:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p --from example.csv put 'begin { @sum = 0 } @sum += $quantity; end {emit @sum}'
|
||
</span> color shape flag index quantity rate
|
||
yellow triangle true 11 43.6498 9.8870
|
||
red square true 15 79.2778 0.0130
|
||
red circle true 16 13.8103 2.9010
|
||
red square false 48 77.5542 7.4670
|
||
purple triangle false 51 81.2290 8.5910
|
||
red square false 64 77.1991 9.5310
|
||
purple triangle false 65 80.1405 5.8240
|
||
yellow circle true 73 63.9785 4.2370
|
||
yellow circle true 87 63.5058 8.3350
|
||
purple square false 91 72.3735 8.2430
|
||
|
||
sum
|
||
652.7185
|
||
</pre></div>
|
||
</div>
|
||
<p>If you want the end-block output to be the only output, and not include the input data, you can use <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">put</span> <span class="pre">-q</span></code>:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p --from example.csv put -q 'begin { @sum = 0 } @sum += $quantity; end {emit @sum}'
|
||
</span> sum
|
||
652.7185
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2j --from example.csv put -q 'begin { @sum = 0 } @sum += $quantity; end {emit @sum}'
|
||
</span> {
|
||
"sum": 652.7185
|
||
}
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2j --from example.csv put -q '
|
||
</span><span class="hll"> begin { @count = 0; @sum = 0 }
|
||
</span><span class="hll"> @count += 1;
|
||
</span><span class="hll"> @sum += $quantity;
|
||
</span><span class="hll"> end {emit (@count, @sum)}
|
||
</span><span class="hll"> '
|
||
</span> {
|
||
"count": 10,
|
||
"sum": 652.7185
|
||
}
|
||
</pre></div>
|
||
</div>
|
||
<p>We’ll see in the documentation for <a class="reference internal" href="reference-verbs.html#reference-verbs-stats1"><span class="std std-ref">stats1</span></a> that there’s a lower-keystroking way to get counts and sums of things – so, take this sum/count example as an indication of the kinds of things you can do using Miller’s programming language.</p>
|
||
</div>
|
||
<div class="section" id="context-variables">
|
||
<h2>Context variables<a class="headerlink" href="#context-variables" title="Permalink to this headline">¶</a></h2>
|
||
<p>Also inspired by <a class="reference external" href="https://en.wikipedia.org/wiki/AWK">AWK</a>, the Miller DSL has the following special <strong>context variables</strong>:</p>
|
||
<ul class="simple">
|
||
<li><p><code class="docutils literal notranslate"><span class="pre">FILENAME</span></code> – the filename the current record came from. Especially useful in things like <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">...</span> <span class="pre">*.csv</span></code>.</p></li>
|
||
<li><p><code class="docutils literal notranslate"><span class="pre">FILENUM</span></code> – similarly, but integer 1,2,3,… rather than filenam.e</p></li>
|
||
<li><p><code class="docutils literal notranslate"><span class="pre">NF</span></code> – the number of fields in the current record. Note that if you assign <code class="docutils literal notranslate"><span class="pre">$newcolumn</span> <span class="pre">=</span> <span class="pre">some</span> <span class="pre">value</span></code> then <code class="docutils literal notranslate"><span class="pre">NF</span></code> will increment.</p></li>
|
||
<li><p><code class="docutils literal notranslate"><span class="pre">NR</span></code> – starting from 1, counter of how many records processed so far.</p></li>
|
||
<li><p><code class="docutils literal notranslate"><span class="pre">FNR</span></code> – similar, but resets to 1 at the start of each file.</p></li>
|
||
</ul>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> cat context-example.mlr
|
||
</span> $nf = NF;
|
||
$nr = NR;
|
||
$fnr = FNR;
|
||
$filename = FILENAME;
|
||
$filenum = FILENUM;
|
||
$newnf = NF;
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p put -f context-example.mlr data/a.csv data/b.csv
|
||
</span> a b c nf nr fnr filename filenum newnf
|
||
1 2 3 3 1 1 data/a.csv 1 8
|
||
4 5 6 3 2 2 data/a.csv 1 8
|
||
7 8 9 3 3 1 data/b.csv 2 8
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="functions-and-local-variables">
|
||
<h2>Functions and local variables<a class="headerlink" href="#functions-and-local-variables" title="Permalink to this headline">¶</a></h2>
|
||
<p>You can define your own functions:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> cat factorial-example.mlr
|
||
</span> func factorial(n) {
|
||
if (n <= 1) {
|
||
return n
|
||
} else {
|
||
return n * factorial(n-1)
|
||
}
|
||
}
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p --from example.csv put -f factorial-example.mlr -e '$fact = factorial(NR)'
|
||
</span> color shape flag index quantity rate fact
|
||
yellow triangle true 11 43.6498 9.8870 1
|
||
red square true 15 79.2778 0.0130 2
|
||
red circle true 16 13.8103 2.9010 6
|
||
red square false 48 77.5542 7.4670 24
|
||
purple triangle false 51 81.2290 8.5910 120
|
||
red square false 64 77.1991 9.5310 720
|
||
purple triangle false 65 80.1405 5.8240 5040
|
||
yellow circle true 73 63.9785 4.2370 40320
|
||
yellow circle true 87 63.5058 8.3350 362880
|
||
purple square false 91 72.3735 8.2430 3628800
|
||
</pre></div>
|
||
</div>
|
||
<p>Note that here we used the <code class="docutils literal notranslate"><span class="pre">-f</span></code> flag to <code class="docutils literal notranslate"><span class="pre">put</span></code> to load our function
|
||
definition, and also the <code class="docutils literal notranslate"><span class="pre">-e</span></code> flag to add another statement on the command
|
||
line. (We could have also put <code class="docutils literal notranslate"><span class="pre">$fact</span> <span class="pre">=</span> <span class="pre">factorial(NR)</span></code> inside
|
||
<code class="docutils literal notranslate"><span class="pre">factorial-example.mlr</span></code> but that would have made that file less flexible for our
|
||
future use.)</p>
|
||
</div>
|
||
<div class="section" id="if-statements-loops-and-local-variables">
|
||
<h2>If-statements, loops, and local variables<a class="headerlink" href="#if-statements-loops-and-local-variables" title="Permalink to this headline">¶</a></h2>
|
||
<p>Suppose you want to only compute sums conditionally – you can use an <code class="docutils literal notranslate"><span class="pre">if</span></code> statement:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> cat if-example.mlr
|
||
</span> begin {
|
||
@count_of_red = 0;
|
||
@sum_of_red = 0
|
||
}
|
||
|
||
if ($color == "red") {
|
||
@count_of_red += 1;
|
||
@sum_of_red += $quantity;
|
||
}
|
||
|
||
end {
|
||
emit (@count_of_red, @sum_of_red)
|
||
}
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --c2p --from example.csv put -q -f if-example.mlr
|
||
</span> count_of_red sum_of_red
|
||
4 247.84139999999996
|
||
</pre></div>
|
||
</div>
|
||
<p>Miller’s else-if is spelled <code class="docutils literal notranslate"><span class="pre">elif</span></code>.</p>
|
||
<p>As we’ll see more of in section (TODO:linkify), Miller has a few kinds of
|
||
for-loops. In addition to the usual 3-part <code class="docutils literal notranslate"><span class="pre">for</span> <span class="pre">(i</span> <span class="pre">=</span> <span class="pre">0;</span> <span class="pre">i</span> <span class="pre"><</span> <span class="pre">10;</span> <span class="pre">i</span> <span class="pre">+=</span> <span class="pre">1)</span></code> kind
|
||
that many programming languages have, Miller also lets you loop over arrays and
|
||
hashmaps. We haven’t encountered arrays and hashmaps yet in this introduction,
|
||
but for now it suffices to know that <code class="docutils literal notranslate"><span class="pre">$*</span></code> is a special variable holding the
|
||
current record as a hashmap:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> cat for-example.mlr
|
||
</span> for (k, v in $*) {
|
||
print "KEY IS ". k . " VALUE IS ". v;
|
||
}
|
||
print
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --csv cat data/a.csv
|
||
</span> a,b,c
|
||
1,2,3
|
||
4,5,6
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --csv --from data/a.csv put -qf for-example.mlr
|
||
</span> KEY IS a VALUE IS 1
|
||
KEY IS b VALUE IS 2
|
||
KEY IS c VALUE IS 3
|
||
|
||
KEY IS a VALUE IS 4
|
||
KEY IS b VALUE IS 5
|
||
KEY IS c VALUE IS 6
|
||
</pre></div>
|
||
</div>
|
||
<p>Here we used the local variables <code class="docutils literal notranslate"><span class="pre">k</span></code> and <code class="docutils literal notranslate"><span class="pre">v</span></code>. Now we’ve seen four kinds of variables:</p>
|
||
<ul class="simple">
|
||
<li><p>Record fields like <code class="docutils literal notranslate"><span class="pre">$shape</span></code></p></li>
|
||
<li><p>Out-of-stream variables like <code class="docutils literal notranslate"><span class="pre">@sum</span></code></p></li>
|
||
<li><p>Local variables like <code class="docutils literal notranslate"><span class="pre">k</span></code></p></li>
|
||
<li><p>Built-in context variables like <code class="docutils literal notranslate"><span class="pre">NF</span></code> and <code class="docutils literal notranslate"><span class="pre">NR</span></code></p></li>
|
||
</ul>
|
||
<p>If you’re curious about scope and extent of local variables, you can read more in (TODO:linkify) the section on variables.</p>
|
||
</div>
|
||
<div class="section" id="arithmetic">
|
||
<h2>Arithmetic<a class="headerlink" href="#arithmetic" title="Permalink to this headline">¶</a></h2>
|
||
<p>Numbers in Miller’s programming language are intended to operate with the principle of least surprise:</p>
|
||
<ul class="simple">
|
||
<li><p>Internally, numbers are either 64-bit signed integers or double-precision floating-point.</p></li>
|
||
<li><p>Sums, differences, and products of integers are also integers (so <code class="docutils literal notranslate"><span class="pre">2*3=6</span></code> not <code class="docutils literal notranslate"><span class="pre">6.0</span></code>) – unless the result of the operation would overflow a 64-bit signed integer in which case the result is automatically converted to float. (If you ever want integer-to-integer arithmetic, use <code class="docutils literal notranslate"><span class="pre">x</span> <span class="pre">.+</span> <span class="pre">y</span></code>, <code class="docutils literal notranslate"><span class="pre">x</span> <span class="pre">.*</span> <span class="pre">y</span></code>, etc.)</p></li>
|
||
<li><p>Quotients of integers are integers if the division is exact, else floating-point: so <code class="docutils literal notranslate"><span class="pre">6/2=3</span></code> but <code class="docutils literal notranslate"><span class="pre">7/2=3.5</span></code>.</p></li>
|
||
</ul>
|
||
<p>You can read more about this at (TODO:linkify).</p>
|
||
</div>
|
||
<div class="section" id="absent-data">
|
||
<h2>Absent data<a class="headerlink" href="#absent-data" title="Permalink to this headline">¶</a></h2>
|
||
<p>In addition to types including string, number (int/float), arrays, and hashmaps, Miller varibles can also be <strong>absent</strong>. This is when a variable never had a value assigned to it. Miller’s treatment of absent data is intended to make it easy for you to handle non-heterogeneous data. We’ll see more in section (TODO:linkify) but the basic idea is:</p>
|
||
<ul class="simple">
|
||
<li><p>Adding a number to absent gives the number back. This means you don’t have to put <code class="docutils literal notranslate"><span class="pre">@sum</span> <span class="pre">=</span> <span class="pre">0</span></code> in your <code class="docutils literal notranslate"><span class="pre">begin</span></code> blocks.</p></li>
|
||
<li><p>Any variable which has the absent value is not assigned. This means you don’t have to check presence of things from one record to the next.</p></li>
|
||
</ul>
|
||
<p>For example, you can sum up all the <code class="docutils literal notranslate"><span class="pre">$a</span></code> values across records without having to check whether they’re present or not:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --json cat absent-example.json
|
||
</span> {
|
||
"a": 1,
|
||
"b": 2
|
||
}
|
||
{
|
||
"c": 3
|
||
}
|
||
{
|
||
"a": 4,
|
||
"b": 5
|
||
}
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr --json put '@sum_of_a += $a; end {emit @sum_of_a}' absent-example.json
|
||
</span> {
|
||
"a": 1,
|
||
"b": 2
|
||
}
|
||
{
|
||
"c": 3
|
||
}
|
||
{
|
||
"a": 4,
|
||
"b": 5
|
||
}
|
||
{
|
||
"sum_of_a": 5
|
||
}
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="footer" role="contentinfo">
|
||
© Copyright 2021, John Kerl.
|
||
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
|
||
</div>
|
||
</body>
|
||
</html> |