Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-18 17:04:50 +00:00
Code Issues Projects Releases Wiki Activity
miller/doc/feature-comparison.html
johnkerl e5c43cb4da fix 
missing data-sharing.html link
2019-09-17 22:39:56 -04:00

325 lines
11 KiB
HTML
Raw Blame History

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en">
<!-- PAGE GENERATED FROM template.html and content-for-feature-comparison.html BY poki. -->
<!-- PLEASE MAKE CHANGES THERE AND THEN RE-RUN poki. -->
<head>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8"/>
<meta name="description" content="Miller documentation"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <!-- mobile-friendly -->
<meta name="keywords"
content="John Kerl, Kerl, Miller, miller, mlr, OLAP, data analysis software, regression, correlation, variance, data tools, " />
<title> Unix-toolkit context </title>
<link rel="stylesheet" type="text/css" href="css2/miller.css"/>
<link rel="stylesheet" type="text/css" href="css2/poki-callbacks.css"/>
</head>
<!-- ================================================================ -->
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-15651652-1");
pageTracker._trackPageview();
} catch(err) {}
</script>
<!-- ================================================================ -->
<script type="text/javascript">
function vis_expand_all_navbar_sections() {
//console.log('vis_expand_all_navbar_sections ENTER');
vis_expand_all_with_prefix('navbar_section_toggle_');
//console.log('vis_expand_all_navbar_sections EXIT');
}
function vis_collapse_all_navbar_sections() {
//console.log('vis_collapse_all_navbar_sections ENTER');
vis_collapse_all_with_prefix('navbar_section_toggle_');
//console.log('vis_collapse_all_navbar_sections EXIT');
}
function vis_expand_all_body_sections() {
//console.log('vis_expand_all_body_sections ENTER');
vis_expand_all_with_prefix('body_section_toggle_');
//console.log('vis_expand_all_body_sections EXIT');
}
function vis_collapse_all_body_sections() {
//console.log('vis_collapse_all_body_sections ENTER');
vis_collapse_all_with_prefix('body_section_toggle_');
//console.log('vis_collapse_all_body_sections EXIT');
}
function vis_expand_all_with_prefix(prefix) {
//console.log('vis_expand_all_with_prefix ENTER', prefix);
var divs = document.getElementsByTagName('div');
console.log("DIVS", divs);
for(var i = 0; i < divs.length; i++) {
const div = divs[i];
if (div.id.startsWith(prefix)) {
vis_expand_div(div);
}
}
//console.log('vis_expand_all_with_prefix EXIT', prefix);
}
function vis_collapse_all_with_prefix(prefix) {
var divs = document.getElementsByTagName('div');
for(var i = 0; i < divs.length; i++){
const div = divs[i];
if (div.id.startsWith(prefix)) {
vis_collapse_div(div);
}
}
}
function vis_toggle_by_name(divName) {
vis_toggle_div(document.getElementById(divName));
}
function vis_expand_by_name(divName) {
vis_expand_div(document.getElementById(divName));
}
function vis_collapse_by_name(divName) {
vis_collapse_div(document.getElementById(divName));
}
function vis_toggle_div(div) {
if (div != null) {
var state = div.style.display;
if (state == 'block') {
div.style.display = 'none';
} else {
div.style.display = 'block';
}
}
}
function vis_expand_div(div) {
if (div != null) {
div.style.display = 'block';
}
}
function vis_collapse_div(div) {
if (div != null) {
div.style.display = 'none';
}
}
</script>
<body bgcolor="#ffffff">
<!-- ================================================================ -->
<!-- navbar -->
<div class="pokinav">
<center><titleinbody>Miller</titleinbody></center>
<!-- NAVBAR GENERATED FROM template.html BY poki -->
<button
style="font-weight:bold;font-size:100%;color:maroon;border:0"
padding=0
onclick="vis_collapse_all_navbar_sections(); vis_toggle_by_name('navbar_section_toggle_overview');"
href="javascript:;"
>
Overview
</button>
<button
style="font-weight:normal;font-size:100%;color:maroon;border:0"
padding=0
onclick="vis_collapse_all_navbar_sections(); vis_toggle_by_name('navbar_section_toggle_using');"
href="javascript:;"
>
Using
</button>
<button
style="font-weight:normal;font-size:100%;color:maroon;border:0"
padding=0
onclick="vis_collapse_all_navbar_sections(); vis_toggle_by_name('navbar_section_toggle_reference');"
href="javascript:;"
>
Reference
</button>
<button
style="font-weight:normal;font-size:100%;color:maroon;border:0"
padding=0
onclick="vis_collapse_all_navbar_sections(); vis_toggle_by_name('navbar_section_toggle_background');"
href="javascript:;"
>
Background
</button>
<button
style="font-weight:normal;font-size:100%;color:maroon;border:0"
padding=0
onclick="vis_collapse_all_navbar_sections(); vis_toggle_by_name('navbar_section_toggle_repository');"
href="javascript:;"
>
Repository
</button>
<div id="navbar_section_toggle_overview" style="display: block;text-align: left">
<br/><a href="index.html">About</a>
<br/><a href="10-min.html">10-minute intro</a>
<br/><a href="file-formats.html">File formats</a>
<br/><a href="feature-comparison.html"><b>Unix-toolkit context</b></a>
<br/><a href="record-heterogeneity.html">Record-hetrogeneity</a>
<br/><a href="internationalization.html">Internationalization</a>
</div>
<div id="navbar_section_toggle_using" style="display: none;text-align: left">
<br/><a href="faq.html">FAQ</a>
<br/><a href="10-min.html">10-minute intro</a>
<br/><a href="data-sharing.html">Mixing with other languages</a>
<br/><a href="cookbook.html">Cookbook part 1</a>
<br/><a href="cookbook2.html">Cookbook part 2</a>
<br/><a href="cookbook3.html">Cookbook part 3</a>
<br/><a href="data-examples.html">Data-diving examples</a>
</div>
<div id="navbar_section_toggle_reference" style="display: none;text-align: left">
<br/><a href="reference.html">Main reference</a>
<br/><a href="reference-verbs.html">Verbs reference</a>
<br/><a href="reference-dsl.html">DSL reference</a>
<br/><a href="manpage.html">Manpage</a>
<br/><a href="release-docs.html">Documents by release</a>
<br/><a href="build.html">Installation</a>
</div>
<div id="navbar_section_toggle_background" style="display: none;text-align: left">
<br/><a href="why.html">Why?</a>
<br/><a href="whyc.html">Why C?</a>
<br/><a href="etymology.html">Why call it Miller?</a>
<br/><a href="originality.html">How original is Miller?</a>
<br/><a href="performance.html">Performance</a>
</div>
<div id="navbar_section_toggle_repository" style="display: none;text-align: left">
<br/><a href="contact.html">Contact information</a>
<br/><a href="https://github.com/johnkerl/miller">https://github.com/johnkerl/miller</a>
</div>
</div>
<!-- page body -->
<p/>
<!-- BODY COPIED FROM content-for-feature-comparison.html BY poki -->
<div class="pokitoc">
<center><titleinbody>Unix-toolkit context</titleinbody></center>
&bull;&nbsp;<a href="#File-format_awareness">File-format awareness</a><br/>
&bull;&nbsp;<a href="#awk-like_features:_mlr_filter_and_mlr_put">awk-like features: mlr filter and mlr put</a><br/>
&bull;&nbsp;<a href="#See_also">See also</a><br/>
</div>
<p/>
<a id="File-format_awareness"/><h1>File-format awareness</h1>
Miller respects CSV headers. If you do <code>mlr --csv cat *.csv</code> then the header line is written once:
<table><tr>
<td>
<p/>
<div class="pokipanel">
<pre>
$ cat data/a.csv
a,b,c
1,2,3
4,5,6
</pre>
</div>
<p/>
</td>
<td>
<p/>
<div class="pokipanel">
<pre>
$ cat data/b.csv
a,b,c
7,8,9
</pre>
</div>
<p/>
</td>
<td>
<p/>
<div class="pokipanel">
<pre>
$ mlr --csv cat data/a.csv data/b.csv
a,b,c
1,2,3
4,5,6
7,8,9
</pre>
</div>
<p/>
</td>
<td>
<p/>
<div class="pokipanel">
<pre>
$ mlr --csv sort -nr b data/a.csv data/b.csv
a,b,c
7,8,9
4,5,6
1,2,3
</pre>
</div>
<p/>
</td>
</tr></table>
Likewise with <code>mlr sort</code>, <code>mlr tac</code>, and so on.
<a id="awk-like_features:_mlr_filter_and_mlr_put"/><h1>awk-like features: mlr filter and mlr put</h1>
<ul>
<li/> <code>mlr filter</code> includes/excludes records based on a filter
expression, e.g. <code>mlr filter '$count &gt; 10'</code>.
<li/> <code>mlr put</code> adds a new field as a function of others, e.g. <code>mlr
put '$xy = $x * $y'</code> or <code>mlr put '$counter = NR'</code>.
<li/> The <code>$name</code> syntax is straight from <code>awk</code>&rsquo;s <code>$1 $2
$3</code> (adapted to name-based indexing), as are the variables <code>FS</code>,
<code>OFS</code>, <code>RS</code>, <code>ORS</code>, <code>NF</code>, <code>NR</code>, and
<code>FILENAME</code>. The <code>ENV[...]</code> syntax is from Ruby.
<li/> While <code>awk</code> functions are record-based, Miller subcommands (or
<i>verbs</i>) are stream-based: each of them maps a stream of records into
another stream of records.
<li/> Like <code>awk</code>, Miller (as of v5.0.0) allows you to define new
functions within its <code>put</code> and <code>filter</code> expression language.
Further programmability comes from chaining with <code>then</code>.
<li/> As with <code>awk</code>, <code>$</code>-variables are stream variables and all
verbs (such as <code>cut</code>, <code>stats1</code>, <code>put</code>, etc.) as well as
<code>put</code>/<code>filter</code> statements operate on streams. This means that
you define actions to be done on each record and then stream your data through
those actions. The built-in variables <code>NF</code>, <code>NR</code>, etc. change
from one line to another, <code>$x</code> is a label for field <code>x</code> in the
current record, and the input to <code>sqrt($x)</code> changes from one record to
the next. The expression language for the <code>put</code> and <code>filter</code>
verbs additionally allows you to define <code>begin {...}</code> and
<code>end {...}</code> blocks for actions to be taken before and after records are
processed, respectively.
<li/> As with <code>awk</code>, Miller&rsquo;s <code>put</code>/<code>filter</code>
language lets you set <code>@sum=0</code> before records are read, then update that
sum on each record, then print its value at the end. Unlike <code>awk</code>,
Miller makes syntactically explicit the difference between variables with
extent across all records (names starting with <code>@</code>, such as
<code>@sum</code>) and variables which are local to the current expression (names
starting without <code>@</code>, such as <code>sum</code>).
<li/> Miller can be faster than <code>awk</code>, <code>cut</code>, and so on,
depending on platform; see also <a href="performance.html">Performance</a>).
In particular, Miller&rsquo;s DSL syntax is parsed into C control structures at
startup time, with the bulk data-stream processing all done in C.
</ul>
<a id="See_also"/><h1>See also</h1>
<p/>See <a href="reference.html">Main reference</a> for more on Miller&rsquo;s
subcommands <code>cat</code>, <code>cut</code>, <code>head</code>, <code>sort</code>,
<code>tac</code>, <code>tail</code>, <code>top</code>, and <code>uniq</code>, as well as awk-like
<code>mlr filter</code> and <code>mlr put</code>.
</body>
</html>
Reference in a new issue View git blame Copy permalink