mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-17 16:38:54 +00:00
134 lines
4.9 KiB
Text
134 lines
4.9 KiB
Text
================================================================
|
|
!! BUGFIX !!
|
|
|
|
$ echo 'x=0' | mlr put '$y=-2*$x+3'
|
|
x=0,y=-3.000000
|
|
|
|
$ echo 'x=0' | mlr put '$y=(-2*$x)+3'
|
|
x=0,y=3.000000
|
|
|
|
$ echo 'x=0' | mlr put '$y=-(2*$x)+3'
|
|
x=0,y=-3.000000
|
|
|
|
----------------------------------------------------------------
|
|
!! BUGFIX !!
|
|
|
|
mlr put '$x/$y*$z' not right
|
|
|
|
================================================================
|
|
NEATEN
|
|
|
|
* prune pix
|
|
* makefile go/d/rs stuff -> language-comparisons
|
|
* catc.c/catc0.c -> language-comparisons
|
|
* remove/coalesce/gzip the large data files
|
|
* mk perfcomp dir
|
|
* "index-numbered" -> "implicitly index-numbered" in mlrwik
|
|
* source hygiene: xxx's, top-of-header comments, readme re memory management, etc.
|
|
* "level_1", "level_2", etc. are just confusing. come up with more descriptive names.
|
|
|
|
================================================================
|
|
FEATURES
|
|
!! sort -nr !!
|
|
!! quantiles !!
|
|
http://en.wikipedia.org/wiki/Order_statistic_tree
|
|
* stats1 mode
|
|
! mod op!! (c-like, or sane) and put into wikidoc if so.
|
|
* needs sub function. e.g. "300ms" -> "300"
|
|
* online help: have a categorized function lister -- by string/math or arity, or some such ...
|
|
* nidx feature: bulk rename cols 1,2,3,4,5 this,that,something,foo,bar
|
|
* feature: ordered cut (a la reorder)
|
|
* impl mlr check: no output
|
|
|
|
================================================================
|
|
ONLINE HELP
|
|
* then-chaining note into mlr online help
|
|
* jko mlrdoc & gh/jk/mlr urls into mlr online help
|
|
|
|
================================================================
|
|
IMPROVEMENTS
|
|
* pprint reader: read using field widths?!? with strip ... that would solve the embedded-whitespace problem.
|
|
* xtab -> vtab everywhere? (code+doc)
|
|
* null-handling everywhere!! :/
|
|
* 80-column-wrap slls printer (for mlr --help)
|
|
* char -> char* for RS/FS/PS. then, NEWLINENEWLINE + repifs -> xtab is a more or less special case of dkvp.
|
|
BUT this makes mlr_get_line harder :/
|
|
* ofmts for stepper/stats1/stats2 *defaulting* to global ofmt
|
|
|
|
================================================================
|
|
TESTING
|
|
* xtab input & UTs for it
|
|
* CSV uts
|
|
* reorder UTs per se in test_lrec
|
|
* reorder reg-test cases
|
|
* ut cases: multifile & multimapper often. e.g. wrapping everything in cat then X then cat ...
|
|
|
|
================================================================
|
|
DOC
|
|
* doc: to-do page e.g. multi-char separators (harder for getline); quantiles; what else.
|
|
* doc: reference: be sure to include no-sql examples throughout
|
|
* doc: mem-mgmt: streaming & as near stateless as possible. e.g. csv cat retains only the header line.
|
|
dkvp cat completely stateless. stats1/2 retain only agg state. tac/sort/etc. obvious retain all lines.
|
|
you can snarf a 20GB file with 4GB RAM no problem.
|
|
* doc: performance: Go/D/Rust ...
|
|
* doc: intro note about more structuring: grep/sed/ruby/perl/sed etc. for some structuring, then pipe to miller?
|
|
* doc: maybe restore mem-mgmt page for dev-info? or a readme?!?
|
|
* doc: somewhere in mlrwik put perf-stat analysis (my tn.rb)
|
|
* doc: somewhere in mlrwik put some realistic log-dive data -- not abixy & not just resource/ok.
|
|
* doc: at the top link to data/small/etc. make small.txt hardlinks for browser download.
|
|
* doc: rename data/big to data/large or data/1m everywhere (code+doc)
|
|
* doc: section on to-do things, e.g. multichar rs/fs/ps; & regexes; & quantiles
|
|
* put colored-shapes.dkvp in more places in wikset
|
|
* ref to http://datapipes.okfnlabs.org/
|
|
|
|
================================================================
|
|
DATA
|
|
* move out of c dir ...
|
|
|
|
================================================================
|
|
MEM MGMT:
|
|
|
|
* full void-star-payload frees
|
|
* multi-level frees in stats1/stats2/step subcmds (control-plane structures)
|
|
* multi-level frees in stats1/stats2/step hashmaps (data-plane structures)
|
|
* _free funcptr/funcs for mappers
|
|
* free last rec in streamer?
|
|
|
|
================================================================
|
|
FCNS INCL. STRxTIME
|
|
? length i_s_
|
|
! fmtnum s_f_, s_i_
|
|
? strptime f_s_
|
|
? strftime s_f_, s_i_
|
|
? match b_ss
|
|
? sub, gsub s_sss (w/ regexes!!)
|
|
? index i_ss
|
|
? bit ops & | ^ << >> ??? i_ii
|
|
|
|
================================================================
|
|
UT/REG
|
|
* general coverage
|
|
- include -D__XXX_MAIN__ content
|
|
* cat/X/cat for all X
|
|
* tac/X/cat for all X
|
|
* cat/X/tac for all X
|
|
* tac/X/tac for all X
|
|
* put & filter
|
|
* multi-csv I/O: include --icsv --odkvp and --idkvp --ocsv, as well as --csv cases
|
|
* het-xtab out
|
|
|
|
================================================================
|
|
I/O
|
|
? right-align only some in --opprint?
|
|
|
|
================================================================
|
|
INTERNAL DOCS (e.g. README)
|
|
* sllv==NULL vs. sllv_single(NULL); and mem-mgmt cmts thruout
|
|
|
|
================================================================
|
|
HARDER HYGIENE
|
|
* eliminate compiler warnings for *.l/*.y/etc.
|
|
|
|
================================================================
|
|
PYTHON
|
|
* pgr + stats_m same I/O modules??
|