mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-18 00:45:47 +00:00
169 lines
5.9 KiB
HTML
169 lines
5.9 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
|
|
<html lang="en">
|
|
|
|
<!-- PAGE GENERATED FROM template.html and content-for-index.html BY poki. -->
|
|
<!-- PLEASE MAKE CHANGES THERE AND THEN RE-RUN poki. -->
|
|
<head>
|
|
<meta http-equiv="Content-type" content="text/html;charset=UTF-8"/>
|
|
<meta name="description" content="Miller documentation"/>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0"/> <!-- mobile-friendly -->
|
|
<title> About </title>
|
|
<link rel="stylesheet" type="text/css" href="css/miller.css"/>
|
|
<link rel="stylesheet" type="text/css" href="css/poki-callbacks.css"/>
|
|
</head>
|
|
|
|
<!-- ================================================================ -->
|
|
<script type="text/javascript">
|
|
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
|
|
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
|
|
</script>
|
|
<script type="text/javascript">
|
|
try {
|
|
var pageTracker = _gat._getTracker("UA-15651652-1");
|
|
pageTracker._trackPageview();
|
|
} catch(err) {}
|
|
</script>
|
|
|
|
<script type="text/javascript">
|
|
function toggle(divName) {
|
|
var eleDiv = document.getElementById(divName);
|
|
if (eleDiv != null) {
|
|
if (eleDiv.style.display == "block") {
|
|
eleDiv.style.display = "none";
|
|
} else {
|
|
eleDiv.style.display = "block";
|
|
}
|
|
}
|
|
}
|
|
</script>
|
|
|
|
<!--
|
|
The background image is from a screenshot of a Google search for "data analysis
|
|
tools", lightened and sepia-toned. Over this was placed a Mac Terminal app with
|
|
very light-grey font and translucent background, in which a few statistical
|
|
Miller commands were run with pretty-print-tabular output format.
|
|
-->
|
|
<body background="pix/sepia-overlay.jpg">
|
|
|
|
<!-- ================================================================ -->
|
|
<table width="100%">
|
|
<tr>
|
|
|
|
<!-- navbar -->
|
|
<td width="15%">
|
|
<!--
|
|
<img src="pix/mlr.jpg" />
|
|
<img style="border-width:1px; color:black;" src="pix/mlr.jpg" />
|
|
-->
|
|
|
|
<div class="pokinav">
|
|
<center><titleinbody>Miller</titleinbody></center>
|
|
|
|
<!-- PAGE LIST GENERATED FROM template.html BY poki -->
|
|
<br/>User info:
|
|
<br/>• <a href="index.html"><b>About</b></a>
|
|
<br/>• <a href="file-formats.html">File formats</a>
|
|
<br/>• <a href="feature-comparison.html">Miller features in the context of the Unix toolkit</a>
|
|
<br/>• <a href="record-heterogeneity.html">Record-heterogeneity</a>
|
|
<br/>• <a href="performance.html">Performance</a>
|
|
<br/>• <a href="etymology.html">Why call it Miller?</a>
|
|
<br/>• <a href="originality.html">How original is Miller?</a>
|
|
<br/>• <a href="reference.html">Reference</a>
|
|
<br/>• <a href="data-examples.html">Data examples</a>
|
|
<br/>• <a href="to-do.html">Things to do</a>
|
|
<br/>Developer info:
|
|
<br/>• <a href="build.html">Compiling, portability, dependencies, and testing</a>
|
|
<br/>• <a href="whyc.html">Why C?</a>
|
|
<br/>• <a href="contact.html">Contact information</a>
|
|
<br/>• <a href="https://github.com/johnkerl/miller">GitHub repo</a>
|
|
<br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/>
|
|
<br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/>
|
|
<br/> <br/> <br/> <br/> <br/> <br/>
|
|
</div>
|
|
</td>
|
|
|
|
<!-- page body -->
|
|
<td>
|
|
<center> <titleinbody> About </titleinbody> </center>
|
|
<p>
|
|
|
|
<!-- BODY COPIED FROM content-for-index.html BY poki -->
|
|
<center><table border=1 width="60%"><tr><td><center>
|
|
<b>
|
|
Miller is like sed, awk, cut, and sort for name-indexed data such as CSV.
|
|
</b>
|
|
</center> </td></tr></table></center>
|
|
|
|
<p/> For example, compare
|
|
|
|
<table><tr><td>
|
|
<p/>
|
|
<div class="pokipanel">
|
|
<pre>
|
|
% cut -d, -f 11,17 mydata.csv
|
|
% sort -t, -k 11,17 mydata.csv
|
|
% awk -F, '{print $7+2.7*$8}' mydata.csv
|
|
% awk -F, '$15 != "down" {print}' mydata.csv
|
|
</pre>
|
|
</div>
|
|
<p/>
|
|
</td><td>
|
|
|
|
|
|
|
|
to
|
|
|
|
|
|
|
|
</td><td>
|
|
<p/>
|
|
<div class="pokipanel">
|
|
<pre>
|
|
% mlr --csv cut -f hostname,uptime mydata.csv
|
|
% mlr --csv sort hostname,uptime mydata.csv
|
|
% mlr --csv put '$z = $x + 2.7*$y' mydata.csv
|
|
% mlr --csv filter '$status != "down"' mydata.csv
|
|
</pre>
|
|
</div>
|
|
<p/>
|
|
</td></tr></table>
|
|
|
|
<p/> With Miller you get to use named fields without needing to count
|
|
positional indices. This is something the Unix toolkit always could have done,
|
|
and arguably always should have done. It operates on key-value-pair data while
|
|
the familiar Unix tools operate on integer-indexed fields: if the natural
|
|
data structure for the latter is the array, then Miller’s natural data
|
|
structure is the insertion-ordered hash map. This encompasses a <b>variety of
|
|
data formats</b>, including but not limited to the familiar CSV. (Miller can
|
|
handle positionally-indexed data as a special case.)
|
|
|
|
<p/> Features:
|
|
<ul>
|
|
|
|
<li> I/O formats including <b>tabular pretty-printing</b>
|
|
<li> <b>Conversion</b> between formats
|
|
<li> Format-aware processing: e.g. CSV sort and tac keep header lines first
|
|
<li> High-throughput <b>performance</b> on par with the Unix toolkit
|
|
|
|
<li/> Miller is <b>pipe-friendly</b> and interoperates with Unix toolkit.
|
|
|
|
<li/> It complements SQL <b>databases</b>: you can slice, dice, and reformat
|
|
data on the client side on its way into or out of a database. You can also reap
|
|
some of the benefits of databases for quick, setup-free one-off tasks when just
|
|
need to query some data in disk files in a hurry.
|
|
|
|
<li/>Miller also goes beyond classic Unix tools by stepping into our modern,
|
|
<b>no-SQL</b> world: its essential record-heterogeneity property allows it to
|
|
operate on data where records with different schema (field names) are
|
|
interleaved.
|
|
|
|
<li/> Like <a href="http://stedolan.github.io/jq/">jq</a> (for JSON), Miller is
|
|
written in portable C, and it has zero runtime dependencies. You can download
|
|
or compile a single binary, <tt>scp</tt> it to a faraway machine, and expect it
|
|
to work.
|
|
</ul>
|
|
</td>
|
|
|
|
</table>
|
|
</body>
|
|
</html>
|