Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-20 18:10:07 +00:00
Code Issues Projects Releases Wiki Activity
miller/docs6/docs/_build/html/reference-main-regular-expressions.html

122 lines
No EOL
11 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Reference: regular expressions &#8212; Miller 6.0.0-alpha documentation</title>
<link rel="stylesheet" href="_static/scrolls.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/print.css" type="text/css" />
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/language_data.js"></script>
<script src="_static/theme_extras.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Reference: Miller environment variables" href="reference-main-env-vars.html" />
<link rel="prev" title="Reference: arithmetic" href="reference-main-arithmetic.html" />
</head><body>
<div id="content">
<div class="header">
<h1 class="heading"><a href="index.html"
title="back to the documentation overview"><span>Reference: regular expressions</span></a></h1>
</div>
<div class="relnav" role="navigation" aria-label="related navigation">
<a href="reference-main-arithmetic.html">&laquo; Reference: arithmetic</a> |
<a href="#">Reference: regular expressions</a>
| <a href="reference-main-env-vars.html">Reference: Miller environment variables &raquo;</a>
</div>
<div id="contentwrapper">
<div id="toc" role="navigation" aria-label="table of contents navigation">
<h3>Table of Contents</h3>
<ul>
<li><a class="reference internal" href="#">Reference: regular expressions</a><ul>
<li><a class="reference internal" href="#regex-captures">Regex captures</a></li>
</ul>
</li>
</ul>
</div>
<div role="main">
<div class="section" id="reference-regular-expressions">
<h1>Reference: regular expressions<a class="headerlink" href="#reference-regular-expressions" title="Permalink to this headline"></a></h1>
<p>Miller lets you use regular expressions (of type POSIX.2) in the following contexts:</p>
<ul class="simple">
<li><p>In <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">filter</span></code> with <code class="docutils literal notranslate"><span class="pre">=~</span></code> or <code class="docutils literal notranslate"><span class="pre">!=~</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">filter</span> <span class="pre">'$url</span> <span class="pre">=~</span> <span class="pre">&quot;http.*com&quot;'</span></code></p></li>
<li><p>In <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">put</span></code> with <code class="docutils literal notranslate"><span class="pre">sub</span></code> or <code class="docutils literal notranslate"><span class="pre">gsub</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">put</span> <span class="pre">'$url</span> <span class="pre">=</span> <span class="pre">sub($url,</span> <span class="pre">&quot;http.*com&quot;,</span> <span class="pre">&quot;&quot;)'</span></code></p></li>
<li><p>In <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">having-fields</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">having-fields</span> <span class="pre">--any-matching</span> <span class="pre">'^sda[0-9]'</span></code></p></li>
<li><p>In <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">cut</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">cut</span> <span class="pre">-r</span> <span class="pre">-f</span> <span class="pre">'^status$,^sda[0-9]'</span></code></p></li>
<li><p>In <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">rename</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">rename</span> <span class="pre">-r</span> <span class="pre">'^(sda[0-9]).*$,dev/\1'</span></code></p></li>
<li><p>In <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">grep</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">mlr</span> <span class="pre">--csv</span> <span class="pre">grep</span> <span class="pre">00188555487</span> <span class="pre">myfiles*.csv</span></code></p></li>
</ul>
<p>Points demonstrated by the above examples:</p>
<ul class="simple">
<li><p>There are no implicit start-of-string or end-of-string anchors; please use <code class="docutils literal notranslate"><span class="pre">^</span></code> and/or <code class="docutils literal notranslate"><span class="pre">$</span></code> explicitly.</p></li>
<li><p>Miller regexes are wrapped with double quotes rather than slashes.</p></li>
<li><p>The <code class="docutils literal notranslate"><span class="pre">i</span></code> after the ending double quote indicates a case-insensitive regex.</p></li>
<li><p>Capture groups are wrapped with <code class="docutils literal notranslate"><span class="pre">(...)</span></code> rather than <code class="docutils literal notranslate"><span class="pre">\(...\)</span></code>; use <code class="docutils literal notranslate"><span class="pre">\(</span></code> and <code class="docutils literal notranslate"><span class="pre">\)</span></code> to match against parentheses.</p></li>
</ul>
<p>For <code class="docutils literal notranslate"><span class="pre">filter</span></code> and <code class="docutils literal notranslate"><span class="pre">put</span></code>, if the regular expression is a string literal (the normal case), it is precompiled at process start and reused thereafter, which is efficient. If the regular expression is a more complex expression, including string concatenation using <code class="docutils literal notranslate"><span class="pre">.</span></code>, or a column name (in which case you can take regular expressions from input data!), then regexes are compiled on each record which works but is less efficient. As well, in this case there is no way to specify case-insensitive matching.</p>
<p>Example:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> cat data/regex-in-data.dat
</span> name=jane,regex=^j.*e$
name=bill,regex=^b[ou]ll$
name=bull,regex=^b[ou]ll$
</pre></div>
</div>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr filter &#39;$name =~ $regex&#39; data/regex-in-data.dat
</span> name=jane,regex=^j.*e$
name=bull,regex=^b[ou]ll$
</pre></div>
</div>
<div class="section" id="regex-captures">
<h2>Regex captures<a class="headerlink" href="#regex-captures" title="Permalink to this headline"></a></h2>
<p>Regex captures of the form <code class="docutils literal notranslate"><span class="pre">\0</span></code> through <code class="docutils literal notranslate"><span class="pre">\9</span></code> are supported as</p>
<ul class="simple">
<li><p>Captures have in-function context for <code class="docutils literal notranslate"><span class="pre">sub</span></code> and <code class="docutils literal notranslate"><span class="pre">gsub</span></code>. For example, the first <code class="docutils literal notranslate"><span class="pre">\1,\2</span></code> pair belong to the first <code class="docutils literal notranslate"><span class="pre">sub</span></code> and the second <code class="docutils literal notranslate"><span class="pre">\1,\2</span></code> pair belong to the second <code class="docutils literal notranslate"><span class="pre">sub</span></code>:</p></li>
</ul>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr put &#39;$b = sub($a, &quot;(..)_(...)&quot;, &quot;\2-\1&quot;); $c = sub($a, &quot;(..)_(.)(..)&quot;, &quot;:\1:\2:\3&quot;)&#39;
</span></pre></div>
</div>
<ul class="simple">
<li><p>Captures endure for the entirety of a <code class="docutils literal notranslate"><span class="pre">put</span></code> for the <code class="docutils literal notranslate"><span class="pre">=~</span></code> and <code class="docutils literal notranslate"><span class="pre">!=~</span></code> operators. For example, here the <code class="docutils literal notranslate"><span class="pre">\1,\2</span></code> are set by the <code class="docutils literal notranslate"><span class="pre">=~</span></code> operator and are used by both subsequent assignment statements:</p></li>
</ul>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr put &#39;$a =~ &quot;(..)_(....); $b = &quot;left_\1&quot;; $c = &quot;right_\2&quot;&#39;
</span></pre></div>
</div>
<ul class="simple">
<li><p>The captures are not retained across multiple puts. For example, here the <code class="docutils literal notranslate"><span class="pre">\1,\2</span></code> won’t be expanded from the regex capture:</p></li>
</ul>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span><span class="hll"> mlr put &#39;$a =~ &quot;(..)_(....)&#39; then {... something else ...} then put &#39;$b = &quot;left_\1&quot;; $c = &quot;right_\2&quot;&#39;
</span></pre></div>
</div>
<ul class="simple">
<li><p>Captures are ignored in <code class="docutils literal notranslate"><span class="pre">filter</span></code> for the <code class="docutils literal notranslate"><span class="pre">=~</span></code> and <code class="docutils literal notranslate"><span class="pre">!=~</span></code> operators. For example, there is no mechanism provided to refer to the first <code class="docutils literal notranslate"><span class="pre">(..)</span></code> as <code class="docutils literal notranslate"><span class="pre">\1</span></code> or to the second <code class="docutils literal notranslate"><span class="pre">(....)</span></code> as <code class="docutils literal notranslate"><span class="pre">\2</span></code> in the following filter statement:</p></li>
</ul>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>mlr filter &#39;$a =~ &quot;(..)_(....)&#39;
</pre></div>
</div>
<ul class="simple">
<li><p>Up to nine matches are supported: <code class="docutils literal notranslate"><span class="pre">\1</span></code> through <code class="docutils literal notranslate"><span class="pre">\9</span></code>, while <code class="docutils literal notranslate"><span class="pre">\0</span></code> is the entire match string; <code class="docutils literal notranslate"><span class="pre">\15</span></code> is treated as <code class="docutils literal notranslate"><span class="pre">\1</span></code> followed by an unrelated <code class="docutils literal notranslate"><span class="pre">5</span></code>.</p></li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2021, John Kerl.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink