mirror of
https://github.com/johnkerl/miller.git
synced 2026-01-23 02:14:13 +00:00
a977617797
* address some staticcheck issues * address some staticcheck issues * address some staticcheck issues * address some staticcheck issues
99 lines
3.9 KiB
Text
99 lines
3.9 KiB
Text
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
* stack optimization:
|
|
k StackVariable opaque struct
|
|
- name string
|
|
- frameSetIndex int
|
|
- frameIndex int
|
|
k convert StackFrame from map of name to *TypeGatedMlrvalVariable to array of *TypeGatedMlrvalVariable
|
|
k convert StackFrameSet from list of StackFrame to array of StackFrame
|
|
- comments about S/SF indef large b/c recursion; SF/SFS known statically
|
|
o Define/Set/Get need to modify the indices:
|
|
? add comments for retain in CST node?
|
|
- within Stack: map from name to idxpair
|
|
- within Set/Get (re-use): fastcall to indexer if non-trivial
|
|
o mods to use the indices when set
|
|
|
|
o profile mand.mlr
|
|
- c 3.2s
|
|
- go pre-alloccate 45s
|
|
|
|
Why 5MB goal with GOGC=100????
|
|
|
|
GOGC=100000 GODEBUG=gctrace=1 mlr -n put -q -f u/mand.mlr 1> /dev/null
|
|
|
|
* u/mand.mlr silent option
|
|
|
|
https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-26c2462549a2/
|
|
ballast := make([]byte, 10<<30)
|
|
|
|
https://dave.cheney.net/2014/07/11/visualising-the-go-garbage-collector
|
|
go get -u -v github.com/davecheney/gcvis
|
|
gcvis mlr -n put -q -s iheight=500 -s iwidth=500 -f u/mand.mlr > /dev/null
|
|
|
|
* duffcopy/madvise (w/ GOGC=off), & heavy GC (w/ GOGC != off), are both indicators of the same thing: lots of allocation and lots of copying
|
|
|
|
https://github.com/golang/go/issues/23044
|
|
"It has often been noted that programs which make a lot of allocations while
|
|
maintaining a small live heap end up doing excessive garbage collections."
|
|
yyyuuuuuuuuup
|
|
--> maybe plan out the CST w/ mlrvals for all slots pre-allocated, and point to them -- ?
|
|
--> and/or: Evaluate return *types.Mlrval ?
|
|
o or instead of Evaluate(state*State) -> Mlrval, do Evaluate(state *State, poutput *Mlrval)
|
|
o and not type BinaryFunc func(*Mlrval, *Mlrval) Mlrval but type BinaryFunc func(out *Mlrval, in1 *Mlrval, in2 *Mlrval)
|
|
o idea: don't generate ANY garbage w/ this:
|
|
zt = zr*zr - zi*zi + cr;
|
|
zi = 2*zr*zi + ci;
|
|
zr = zt;
|
|
o leaf evaluables:
|
|
- literals & local vars etc just point to storage
|
|
|
|
i https://hub.packtpub.com/implementing-memory-management-with-golang-garbage-collector/
|
|
|
|
i mlr --cpuprofile cpu.pprof -n put -q -s iheight=100 -s iwidth=100 -f u/mand.mlr > /dev/null
|
|
|
|
? SetGCPercent
|
|
? SetMaxHeap
|
|
|
|
mlr --cpuprofile cpu.pprof -n put -q -s iheight=500 -s iwidth=500 -f u/mand.mlr > /dev/null
|
|
go tool pprof mlr cpu.pprof
|
|
top10
|
|
|
|
go tool pprof --pdf mlr cpu.pprof > mlr-call-graph.pdf
|
|
mv mlr-call-graph.pdf ~/Desktop
|
|
|
|
runtime.duffcopy
|
|
https://stackoverflow.com/questions/45786687/runtime-duffcopy-is-called-a-lot
|
|
runtime.madvise
|
|
|
|
GOGC=off
|
|
GODEBUG=gctrace=1
|
|
|
|
export PATH=${PATH}:~/git/brendangregg/FlameGraph/
|
|
go-torch cpu.pprof
|
|
mv torch.svg ~/Desktop/
|
|
|
|
i mlr --cpuprofile cpu.pprof -n put -q -s iheight=500 -s iwidth=500 -f u/mand.mlr > /dev/null
|
|
|
|
i https://hub.packtpub.com/implementing-memory-management-with-golang-garbage-collector/
|
|
|
|
i https://golang.org/pkg/runtime/
|
|
|
|
! flame-graph readme; & profile-readme out of mlr.go & into separate .md file
|
|
i mlr --cpuprofile cpu.pprof -n put -q -s iheight=100 -s iwidth=100 -f u/mand.mlr > /dev/null
|
|
i GODEBUG=gctrace=1 mlr -n put -q -s iheight=500 -s iwidth=500 -f u/mand.mlr > /dev/null| head -n 100
|
|
gc 1 @0.129s 1%: 0.012+3.9+0.002 ms clock, 0.048+3.7/3.7/7.5+0.010 ms cpu, 10240->10240->0 MB, 10241 MB goal, 4 P
|
|
gc 2 @0.140s 2%: 0.009+2.1+0.002 ms clock, 0.038+2.0/2.0/4.0+0.011 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
|
|
gc 3 @0.149s 2%: 0.032+2.1+0.021 ms clock, 0.12+2.0/2.0/4.0+0.087 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
|
|
gc 4 @0.157s 3%: 0.035+2.1+0.014 ms clock, 0.14+2.0/1.9/4.1+0.059 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
|
|
gc 5 @0.166s 3%: 0.034+2.2+0.024 ms clock, 0.13+2.0/1.9/4.0+0.098 ms cpu, 4->4->0 MB, 5 MB goal, 4 P
|
|
|
|
mem.Alloc: 176104
|
|
mem.TotalAlloc: 176104
|
|
mem.HeapAlloc: 176104
|
|
mem.NumGC: 0
|
|
|
|
mem.Alloc: 1179440
|
|
mem.TotalAlloc: 16,529,643,664
|
|
mem.HeapAlloc: 1179440
|
|
mem.NumGC: 4254
|
|
|