mirror of
https://github.com/johnkerl/miller.git
synced 2026-01-24 02:36:15 +00:00
82 lines
No EOL
4.3 KiB
HTML
82 lines
No EOL
4.3 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<title>Reference: then-chaining — Miller 6.0.0-alpha documentation</title>
|
||
|
||
<link rel="stylesheet" href="_static/scrolls.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/print.css" type="text/css" />
|
||
|
||
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
|
||
<script src="_static/jquery.js"></script>
|
||
<script src="_static/underscore.js"></script>
|
||
<script src="_static/doctools.js"></script>
|
||
<script src="_static/language_data.js"></script>
|
||
<script src="_static/theme_extras.js"></script>
|
||
<link rel="index" title="Index" href="genindex.html" />
|
||
<link rel="search" title="Search" href="search.html" />
|
||
<link rel="next" title="Reference: auxiliary commands" href="reference-main-auxiliary-commands.html" />
|
||
<link rel="prev" title="Reference: I/O options" href="reference-main-io-options.html" />
|
||
</head><body>
|
||
<div id="content">
|
||
<div class="header">
|
||
<h1 class="heading"><a href="index.html"
|
||
title="back to the documentation overview"><span>Reference: then-chaining</span></a></h1>
|
||
</div>
|
||
<div class="relnav" role="navigation" aria-label="related navigation">
|
||
<a href="reference-main-io-options.html">« Reference: I/O options</a> |
|
||
<a href="#">Reference: then-chaining</a>
|
||
| <a href="reference-main-auxiliary-commands.html">Reference: auxiliary commands »</a>
|
||
</div>
|
||
<div id="contentwrapper">
|
||
<div role="main">
|
||
|
||
<div class="section" id="reference-then-chaining">
|
||
<h1>Reference: then-chaining<a class="headerlink" href="#reference-then-chaining" title="Permalink to this headline">¶</a></h1>
|
||
<p>In accord with the <a class="reference external" href="http://en.wikipedia.org/wiki/Unix_philosophy">Unix philosophy</a>, you can pipe data into or out of Miller. For example:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr cut --complement -f os_version *.dat | mlr sort -f hostname,uptime
|
||
</span></pre></div>
|
||
</div>
|
||
<p>You can, if you like, instead simply chain commands together using the <code class="docutils literal notranslate"><span class="pre">then</span></code> keyword:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr cut --complement -f os_version then sort -f hostname,uptime *.dat
|
||
</span></pre></div>
|
||
</div>
|
||
<p>(You can precede the very first verb with <code class="docutils literal notranslate"><span class="pre">then</span></code>, if you like, for symmetry.)</p>
|
||
<p>Here’s a performance comparison:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>% cat piped.sh
|
||
mlr cut -x -f i,y data/big | mlr sort -n y > /dev/null
|
||
|
||
% time sh piped.sh
|
||
real 0m2.828s
|
||
user 0m3.183s
|
||
sys 0m0.137s
|
||
|
||
|
||
% cat chained.sh
|
||
mlr cut -x -f i,y then sort -n y data/big > /dev/null
|
||
|
||
% time sh chained.sh
|
||
real 0m2.082s
|
||
user 0m1.933s
|
||
sys 0m0.137s
|
||
</pre></div>
|
||
</div>
|
||
<p>There are two reasons to use then-chaining: one is for performance, although I don’t expect this to be a win in all cases. Using then-chaining avoids redundant string-parsing and string-formatting at each pipeline step: instead input records are parsed once, they are fed through each pipeline stage in memory, and then output records are formatted once. On the other hand, Miller is single-threaded, while modern systems are usually multi-processor, and when streaming-data programs operate through pipes, each one can use a CPU. Rest assured you get the same results either way.</p>
|
||
<p>The other reason to use then-chaining is for simplicity: you don’t have re-type formatting flags (e.g. <code class="docutils literal notranslate"><span class="pre">--csv</span> <span class="pre">--fs</span> <span class="pre">tab</span></code>) at every pipeline stage.</p>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="footer" role="contentinfo">
|
||
© Copyright 2021, John Kerl.
|
||
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
|
||
</div>
|
||
</body>
|
||
</html> |