mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-18 00:45:47 +00:00
51 lines
2.3 KiB
HTML
51 lines
2.3 KiB
HTML
POKI_PUT_TOC_HERE
|
|
|
|
<p/>
|
|
<button style="font-weight:bold;color:maroon;border:0" onclick="expand_all();" href="javascript:;">Expand all sections</button>
|
|
<button style="font-weight:bold;color:maroon;border:0" onclick="collapse_all();" href="javascript:;">Collapse all sections</button>
|
|
|
|
<h1>Disclaimer</h1>
|
|
|
|
In a previous version of this page (see <a
|
|
href="http://johnkerl.org/miller-releases/miller-5.1.0/doc/performance.html">here</a>)
|
|
I compared Miller to some items in the Unix toolkit in terms of run time. But
|
|
such comparisons are very much not apples-to-apples:
|
|
|
|
<ul>
|
|
|
|
<li/> Miller’s principal strength is that it handles <b>key-value data in
|
|
various formats</b> while the system tools <b>do not</b>. So if you time
|
|
<tt>mlr sort</tt> on a CSV file against system <tt>sort</tt>, it's not relevant
|
|
to say which is faster by how many percent — Miller will respect the
|
|
header line, leaving it in place, while the system sort will move it, sorting
|
|
it along with all the other header lines. This would be comparing the run times
|
|
of two programs produce different outputs. Likewise, <tt>awk</tt>
|
|
doesn’t respect header lines, although you can code up some CSV-handling
|
|
using <tt>if (NR==1) { ... } else { ... }</tt>. And that’s just CSV: I
|
|
don’t know any simple way to get <tt>sort</tt>, <tt>awk</tt>, etc. to
|
|
handle DKVP, JSON, etc. — which is the main rreason I wrote Miller.
|
|
|
|
<li/> <b>Implementations differ by platform</b>: one <tt>awk</tt> may be
|
|
fundamentally faster than another, and <tt>mawk</tt> has a very efficient
|
|
bytecode implementation — which handles positionally indexed data
|
|
far faster than Miller does.
|
|
|
|
<li/> The system <tt>sort</tt> command will, on some systems, handle
|
|
too-large-for-RAM datasets by spilling to disk; Miller (as of version 5.2.0,
|
|
mid-2017) does not. Miller sorts are always stable; GNU supports stable and
|
|
unstable variants.
|
|
|
|
<li/> Etc.
|
|
|
|
</ul>
|
|
|
|
<h1>Summary</h1>
|
|
|
|
<p/> Miller can do many kinds of processing on key-value-pair data using
|
|
elapsed time roughly of the same order of magnitude as items in the Unix
|
|
toolkit can handle positionally indexed data. Specific results vary widely
|
|
by platform, implementation details, multi-core use (or not). Lastly,
|
|
specific special-purpose non-record-aware processing will run far faster
|
|
in <tt>grep</tt>, <tt>sed</tt>, etc.
|
|
|
|
</div>
|