Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-07-25 17:04:01 +00:00
Code Issues Projects Releases Wiki Activity

shuffle UT

Browse source
This commit is contained in:
John Kerl 2016-02-17 22:09:07 -05:00
parent 70d6dd65d8
commit d191d38297
3 changed files with 40 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

36
c/reg_test/expected/out
View file

@ -702,6 +702,42 @@ a=eks,b=zee,iii=7,x=0.6117840605678454,y=0.1878849191181694
a=zee,b=wye,i=8,x=0.5985540091064224,yyy=0.976181385699006
a=pan,b=wye,i=10,x=0.5026260055412137,y=0.9526183602969864
mlr --seed 12345 shuffle ./reg_test/input/abixy-het
a=pan,b=wye,i=10,x=0.5026260055412137,y=0.9526183602969864
a=eks,bbb=wye,i=4,x=0.38139939387114097,y=0.13418874328430463
a=eks,b=pan,i=2,x=0.7586799647899636,y=0.5221511083334797
a=wye,b=pan,i=5,xxx=0.5732889198020006,y=0.8636244699032729
a=zee,b=wye,i=8,x=0.5985540091064224,yyy=0.976181385699006
aaa=wye,b=wye,i=3,x=0.20460330576630303,y=0.33831852551664776
a=pan,b=pan,i=1,x=0.3467901443380824,y=0.7268028627434533
aaa=hat,bbb=wye,i=9,x=0.03144187646093577,y=0.7495507603507059
a=eks,b=zee,iii=7,x=0.6117840605678454,y=0.1878849191181694
a=zee,b=pan,i=6,x=0.5271261600918548,y=0.49322128674835697
mlr --seed 23456 shuffle ./reg_test/input/abixy-het
a=eks,bbb=wye,i=4,x=0.38139939387114097,y=0.13418874328430463
a=pan,b=pan,i=1,x=0.3467901443380824,y=0.7268028627434533
a=pan,b=wye,i=10,x=0.5026260055412137,y=0.9526183602969864
a=zee,b=pan,i=6,x=0.5271261600918548,y=0.49322128674835697
a=wye,b=pan,i=5,xxx=0.5732889198020006,y=0.8636244699032729
a=eks,b=zee,iii=7,x=0.6117840605678454,y=0.1878849191181694
aaa=hat,bbb=wye,i=9,x=0.03144187646093577,y=0.7495507603507059
aaa=wye,b=wye,i=3,x=0.20460330576630303,y=0.33831852551664776
a=zee,b=wye,i=8,x=0.5985540091064224,yyy=0.976181385699006
a=eks,b=pan,i=2,x=0.7586799647899636,y=0.5221511083334797
mlr --seed 34567 shuffle ./reg_test/input/abixy-het
a=zee,b=pan,i=6,x=0.5271261600918548,y=0.49322128674835697
aaa=hat,bbb=wye,i=9,x=0.03144187646093577,y=0.7495507603507059
a=eks,b=zee,iii=7,x=0.6117840605678454,y=0.1878849191181694
aaa=wye,b=wye,i=3,x=0.20460330576630303,y=0.33831852551664776
a=eks,bbb=wye,i=4,x=0.38139939387114097,y=0.13418874328430463
a=wye,b=pan,i=5,xxx=0.5732889198020006,y=0.8636244699032729
a=zee,b=wye,i=8,x=0.5985540091064224,yyy=0.976181385699006
a=pan,b=pan,i=1,x=0.3467901443380824,y=0.7268028627434533
a=pan,b=wye,i=10,x=0.5026260055412137,y=0.9526183602969864
a=eks,b=pan,i=2,x=0.7586799647899636,y=0.5221511083334797
mlr uniq -g a ./reg_test/input/abixy-het
a=pan
a=eks

4
c/reg_test/run
View file

@ -187,6 +187,10 @@ run_mlr --seed 12345 sample -k 2 $indir/abixy-het
run_mlr --seed 12345 sample -k 2 -g a $indir/abixy-het
run_mlr --seed 12345 sample -k 2 -g a,b $indir/abixy-het
run_mlr --seed 12345 shuffle $indir/abixy-het
run_mlr --seed 23456 shuffle $indir/abixy-het
run_mlr --seed 34567 shuffle $indir/abixy-het
run_mlr uniq -g a $indir/abixy-het
run_mlr uniq -g a,b $indir/abixy-het

36
c/todo.txt
View file

@ -8,7 +8,6 @@ TOP-OF-LIST SUMMARY
* rh/fedora/centos mlr-3.4.0
! nest x 8
? json arrays -> nested w/ some delimiter? only if all array elements are terminals.
* shuffle: needs UT & mld:ref:shuffle
! 64-bit lengths for containers. test with 5-billion-integer-seq data.
@ -37,41 +36,6 @@ TOP-OF-LIST SUMMARY
- nest
- shuffle
================================================================
NEST:
! UT cases varying fs/ps
wolen:
Being able to deal with replicates in both directions would be really useful.
Here's a real-world example I was working on today, which contains two different measures of association between pairs of
genetic markers (rs*):
rs10 rs2237570,0.3,-0.72;rs117465896,0.75,-0.88
rs10041592 rs1628466,0.21,0.92;rs1729036,0.24,1;rs79177551,0.2,0.85
rs10033329 rs4835294,0.2,0.46
There are two fields, tab-delimited. Records in the second field provide the association results for all markers
relevant to the marker in the first field. Converting this format into a "long" format requires two operators. First,
the unnesting step I described above, which spreads the semicolon-separated replicates across new records (explode-values):
rs10 rs2237570,0.3,-0.72
rs10 rs117465896,0.75,-0.88
rs10041592 rs1628466,0.21,0.92
rs10041592 rs1729036,0.24,1
rs10041592 rs79177551,0.2,0.85
rs10033329 rs4835294,0.2,0.46
followed by a separating step (as described by @komosa) to spread the comma-separated replicates across new fields:
rs10 rs2237570 0.3 -0.72
rs10 rs117465896 0.75 -0.88
rs10041592 rs1628466 0.21 0.92
rs10041592 rs1729036 0.24 1
rs10041592 rs79177551 0.2 0.85
rs10033329 rs4835294 0.2 0.46
================================================================
TRI-SPLIT NULL