Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-20 18:10:07 +00:00
Code Issues Projects Releases Wiki Activity

todo

Browse source
This commit is contained in:
John Kerl 2016-05-06 13:12:38 -07:00
parent ff12d20361
commit 64c63f28ad
3 changed files with 41 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

6
c/todo.txt
View file

@ -10,15 +10,11 @@ TOP-OF-LIST SUMMARY
! rh/fedora/centos mlr-3.5.0 or mlr-4.0.0
* doc section on programmability spectrum (perl to asic) & how much effort is/isn't worth putting into miller. & maybe
shouldabeen pickalanguage w/ I/O support and API.
* in why-miller: list the intersection points. perf; multifmt; math+strings; expressive/programmable; compact notation; pipe-friendly.
? call for use-cases? (go first.)
* dsls deps still not quite right?
* lrec get followed by put/remove: getext variant returning node for unlink, valpoke, or null == append to avoid
double-searching.
* reduce parameter-marshaling by pevalbag; measure perf delta (suspect negligible)
* typed overlay -> mlhmmv rather than lhmsv
* 64-bit lengths for containers. test with 5-billion-integer-seq data.

20
doc/content-for-originality.html
View file

@ -35,7 +35,9 @@ Miller’s added values include:
<ul>
<li> Name-indexing, compared to the Unix toolkit&rsquo;s positional indexing.
<li> Raw speed, compared to <tt>awk</tt>, RecordStream, <tt>stats_m</tt>, or various other kinds of Python/Ruby/etc. scripts one can easily create.
<li> Compact keystroking for many common tasks, with a decent amount of flexibility.
<li> Ability to handle text files on the Unix pipe, without need for creating database tables, compared to SQL databases.
<li> Various file formats, and on-the-fly format conversion.
</ul>
<p/><boldmaroon>jq</boldmaroon>: Miller does for name-indexed text what
@ -59,6 +61,24 @@ be a tool which collects together format-aware record-stream processing into
one place, with good reuse of Miller-internal library code for its various
features.
<p/><boldmaroon>Why not use Perl/Python/Ruby etc.?</boldmaroon> Maybe you
should. With those tools you&rsquo;ll get far more expressive power, and
sufficiently quick turnaround time for small-to-medium-sized data. Using
Miller you&rsquo;ll get something less than a complete programming language,
but which is fast, with moderate amounts of flexibility and much less
keystroking.
<p/>When I was first developing Miller I made a survey of several languages.
Using low-level implementation languages like C, Go, Rust, and Nim, I&rsquo;d
need to create my own domain-specific language (DSL) which would always be less
featured than a full programming language, but I&rsquo;d get better
performance. Using high-level interpreted languages such as Perl/Python/Ruby
I&rsquo;d get the language&rsquo;s <tt>eval</tt> for free and I wouldn&rsquo;t
need a DSL; Miller would have mainly been a set of format-specific I/O hooks.
If I&rsquo;d gotten good enough performance from the latter I&rsquo;d have done
it without question and Miller would be far more flexible. But C won the
performance criteria by a landslide so we have Miller in C with a custom DSL.
<p/> <boldmaroon>No, really, why one more command-line data-manipulation
tool?</boldmaroon> I wrote Miller because I was frustrated with tools like
<tt>grep</tt>, <tt>sed</tt>, and so on being <i>line-aware</i> without being

20
doc/originality.html
View file

@ -174,7 +174,9 @@ Miller&rsquo;s added values include:
<ul>
<li> Name-indexing, compared to the Unix toolkit&rsquo;s positional indexing.
<li> Raw speed, compared to <tt>awk</tt>, RecordStream, <tt>stats_m</tt>, or various other kinds of Python/Ruby/etc. scripts one can easily create.
<li> Compact keystroking for many common tasks, with a decent amount of flexibility.
<li> Ability to handle text files on the Unix pipe, without need for creating database tables, compared to SQL databases.
<li> Various file formats, and on-the-fly format conversion.
</ul>
<p/><boldmaroon>jq</boldmaroon>: Miller does for name-indexed text what
@ -198,6 +200,24 @@ be a tool which collects together format-aware record-stream processing into
one place, with good reuse of Miller-internal library code for its various
features.
<p/><boldmaroon>Why not use Perl/Python/Ruby etc.?</boldmaroon> Maybe you
should. With those tools you&rsquo;ll get far more expressive power, and
sufficiently quick turnaround time for small-to-medium-sized data. Using
Miller you&rsquo;ll get something less than a complete programming language,
but which is fast, with moderate amounts of flexibility and much less
keystroking.
<p/>When I was first developing Miller I made a survey of several languages.
Using low-level implementation languages like C, Go, Rust, and Nim, I&rsquo;d
need to create my own domain-specific language (DSL) which would always be less
featured than a full programming language, but I&rsquo;d get better
performance. Using high-level interpreted languages such as Perl/Python/Ruby
I&rsquo;d get the language&rsquo;s <tt>eval</tt> for free and I wouldn&rsquo;t
need a DSL; Miller would have mainly been a set of format-specific I/O hooks.
If I&rsquo;d gotten good enough performance from the latter I&rsquo;d have done
it without question and Miller would be far more flexible. But C won the
performance criteria by a landslide so we have Miller in C with a custom DSL.
<p/> <boldmaroon>No, really, why one more command-line data-manipulation
tool?</boldmaroon> I wrote Miller because I was frustrated with tools like
<tt>grep</tt>, <tt>sed</tt>, and so on being <i>line-aware</i> without being