Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-08-02 04:22:59 +00:00
Code Issues Projects Releases Wiki Activity
miller/docs6/_build/html/quick-examples.html
John Kerl 1aaec752af Start port of docs to Miller 6
2021-05-24 00:11:53 -04:00

172 lines
No EOL
12 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Quick examples &#8212; Miller 5.10.2 documentation</title>
<link rel="stylesheet" href="_static/classic.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/language_data.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Features" href="features.html" />
<link rel="prev" title="Miller Docs v2" href="index.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="features.html" title="Features"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="index.html" title="Miller Docs v2"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Miller 5.10.2 documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Quick examples</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="quick-examples">
<h1>Quick examples<a class="headerlink" href="#quick-examples" title="Permalink to this headline"></a></h1>
<img alt="_images/cover-combined.png" src="_images/cover-combined.png" />
<p>Column select:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="n">csv</span> <span class="n">cut</span> <span class="o">-</span><span class="n">f</span> <span class="n">hostname</span><span class="p">,</span><span class="n">uptime</span> <span class="n">mydata</span><span class="o">.</span><span class="n">csv</span>
</pre></div>
</div>
<p>Add new columns as function of other columns:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="n">nidx</span> <span class="n">put</span> <span class="s1">&#39;$sum = $7 &lt; 0.0 ? 3.5 : $7 + 2.1*$8&#39;</span> <span class="o">*.</span><span class="n">dat</span>
</pre></div>
</div>
<p>Row filter:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="n">csv</span> <span class="nb">filter</span> <span class="s1">&#39;$status != &quot;down&quot; &amp;&amp; $upsec &gt;= 10000&#39;</span> <span class="o">*.</span><span class="n">csv</span>
</pre></div>
</div>
<p>Apply column labels and pretty-print:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">grep</span> <span class="o">-</span><span class="n">v</span> <span class="s1">&#39;^#&#39;</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">group</span> <span class="o">|</span> <span class="n">mlr</span> <span class="o">--</span><span class="n">ifs</span> <span class="p">:</span> <span class="o">--</span><span class="n">nidx</span> <span class="o">--</span><span class="n">opprint</span> <span class="n">label</span> <span class="n">group</span><span class="p">,</span><span class="k">pass</span><span class="p">,</span><span class="n">gid</span><span class="p">,</span><span class="n">member</span> <span class="n">then</span> <span class="n">sort</span> <span class="o">-</span><span class="n">f</span> <span class="n">group</span>
</pre></div>
</div>
<p>Join multiple data sources on key columns:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="n">join</span> <span class="o">-</span><span class="n">j</span> <span class="n">account_id</span> <span class="o">-</span><span class="n">f</span> <span class="n">accounts</span><span class="o">.</span><span class="n">dat</span> <span class="n">then</span> <span class="n">group</span><span class="o">-</span><span class="n">by</span> <span class="n">account_name</span> <span class="n">balances</span><span class="o">.</span><span class="n">dat</span>
</pre></div>
</div>
<p>Multiple formats including JSON:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="n">json</span> <span class="n">put</span> <span class="s1">&#39;$attr = sub($attr, &quot;([0-9]+)_([0-9]+)_.*&quot;, &quot;</span><span class="se">\1</span><span class="s1">:</span><span class="se">\2</span><span class="s1">&quot;)&#39;</span> <span class="n">data</span><span class="o">/*.</span><span class="n">json</span>
</pre></div>
</div>
<p>Aggregate per-column statistics:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="n">stats1</span> <span class="o">-</span><span class="n">a</span> <span class="nb">min</span><span class="p">,</span><span class="n">mean</span><span class="p">,</span><span class="nb">max</span><span class="p">,</span><span class="n">p10</span><span class="p">,</span><span class="n">p50</span><span class="p">,</span><span class="n">p90</span> <span class="o">-</span><span class="n">f</span> <span class="n">flag</span><span class="p">,</span><span class="n">u</span><span class="p">,</span><span class="n">v</span> <span class="n">data</span><span class="o">/*</span>
</pre></div>
</div>
<p>Linear regression:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="n">stats2</span> <span class="o">-</span><span class="n">a</span> <span class="n">linreg</span><span class="o">-</span><span class="n">pca</span> <span class="o">-</span><span class="n">f</span> <span class="n">u</span><span class="p">,</span><span class="n">v</span> <span class="o">-</span><span class="n">g</span> <span class="n">shape</span> <span class="n">data</span><span class="o">/*</span>
</pre></div>
</div>
<p>Aggregate custom per-column statistics:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="n">put</span> <span class="o">-</span><span class="n">q</span> <span class="s1">&#39;@sum[$a][$b] += $x; end {emit @sum, &quot;a&quot;, &quot;b&quot;}&#39;</span> <span class="n">data</span><span class="o">/*</span>
</pre></div>
</div>
<p>Iterate over data using DSL expressions:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>% mlr --from estimates.tbl put &#39;
for (k,v in $*) {
if (is_numeric(v) &amp;&amp; k =~ &quot;^[t-z].*$&quot;) {
$sum += v; $count += 1
}
}
$mean = $sum / $count # no assignment if count unset
&#39;
</pre></div>
</div>
<p>Run DSL expressions from a script file:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="kn">from</span> <span class="nn">infile.dat</span> <span class="n">put</span> <span class="o">-</span><span class="n">f</span> <span class="n">analyze</span><span class="o">.</span><span class="n">mlr</span>
</pre></div>
</div>
<p>Split/reduce output to multiple filenames:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="kn">from</span> <span class="nn">infile.dat</span> <span class="n">put</span> <span class="s1">&#39;tee &gt; &quot;./taps/data-&quot;.$a.&quot;-&quot;.$b, $*&#39;</span>
</pre></div>
</div>
<p>Compressed I/O:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="kn">from</span> <span class="nn">infile.dat</span> <span class="n">put</span> <span class="s1">&#39;tee | &quot;gzip &gt; ./taps/data-&quot;.$a.&quot;-&quot;.$b.&quot;.gz&quot;, $*&#39;</span>
</pre></div>
</div>
<p>Interoperate with other data-processing tools using standard pipes:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="kn">from</span> <span class="nn">infile.dat</span> <span class="n">put</span> <span class="o">-</span><span class="n">q</span> <span class="s1">&#39;@v=$*; dump | &quot;jq .[]&quot;&#39;</span>
</pre></div>
</div>
<p>Tap/trace:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">%</span> <span class="n">mlr</span> <span class="o">--</span><span class="kn">from</span> <span class="nn">infile.dat</span> <span class="n">put</span> <span class="s1">&#39;(NR % 1000 == 0) { print &gt; stderr, &quot;Checkpoint &quot;.NR}&#39;</span>
</pre></div>
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h4>Previous topic</h4>
<p class="topless"><a href="index.html"
title="previous chapter">Miller Docs v2</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="features.html"
title="next chapter">Features</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/quick-examples.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="features.html" title="Features"
>next</a> |</li>
<li class="right" >
<a href="index.html" title="Miller Docs v2"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Miller 5.10.2 documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Quick examples</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2020, John Kerl.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink