mirror of
https://github.com/johnkerl/miller.git
synced 2026-07-29 02:30:12 +00:00
88 lines
2.9 KiB
ReStructuredText
88 lines
2.9 KiB
ReStructuredText
Randomizing examples
|
|
================================================================
|
|
|
|
Generating random numbers from various distributions
|
|
----------------------------------------------------------------
|
|
|
|
Here we can chain together a few simple building blocks:
|
|
|
|
GENRST_RUN_COMMAND
|
|
cat expo-sample.sh
|
|
GENRST_EOF
|
|
|
|
Namely:
|
|
|
|
* Set the Miller random-number seed so this webdoc looks the same every time I regenerate it.
|
|
* Use pretty-printed tabular output.
|
|
* Use pretty-printed tabular output.
|
|
* Use ``seqgen`` to produce 100,000 records ``i=0``, ``i=1``, etc.
|
|
* Send those to a ``put`` step which defines an inverse-transform-sampling function and calls it twice, then computes the sum and product of samples.
|
|
* Send those to a histogram, and from there to a bar-plotter. This is just for visualization; you could just as well output CSV and send that off to your own plotting tool, etc.
|
|
|
|
The output is as follows:
|
|
|
|
GENRST_RUN_COMMAND
|
|
sh expo-sample.sh
|
|
GENRST_EOF
|
|
|
|
Randomly selecting words from a list
|
|
----------------------------------------------------------------
|
|
|
|
Given this `word list <./data/english-words.txt>`_, first take a look to see what the first few lines look like:
|
|
|
|
GENRST_CARDIFY_HIGHLIGHT_ONE
|
|
head data/english-words.txt
|
|
a
|
|
aa
|
|
aal
|
|
aalii
|
|
aam
|
|
aardvark
|
|
aardwolf
|
|
aba
|
|
abac
|
|
abaca
|
|
GENRST_EOF
|
|
|
|
Then the following will randomly sample ten words with four to eight characters in them:
|
|
|
|
GENRST_CARDIFY_HIGHLIGHT_ONE
|
|
mlr --from data/english-words.txt --nidx filter -S 'n=strlen($1);4<=n&&n<=8' then sample -k 10
|
|
thionine
|
|
birchman
|
|
mildewy
|
|
avigate
|
|
addedly
|
|
abaze
|
|
askant
|
|
aiming
|
|
insulant
|
|
coinmate
|
|
GENRST_EOF
|
|
|
|
Randomly generating jabberwocky words
|
|
----------------------------------------------------------------
|
|
|
|
These are simple *n*-grams as `described here <http://johnkerl.org/randspell/randspell-slides-ts.pdf>`_. Some common functions are `located here <https://github.com/johnkerl/miller/blob/master/docs/ngrams/ngfuncs.mlr.txt>`_. Then here are scripts for `1-grams <https://github.com/johnkerl/miller/blob/master/docs/ngrams/ng1.mlr.txt>`_ `2-grams <https://github.com/johnkerl/miller/blob/master/docs/ngrams/ng2.mlr.txt>`_ `3-grams <https://github.com/johnkerl/miller/blob/master/docs/ngrams/ng3.mlr.txt>`_ `4-grams <https://github.com/johnkerl/miller/blob/master/docs/ngrams/ng4.mlr.txt>`_, and `5-grams <https://github.com/johnkerl/miller/blob/master/docs/ngrams/ng5.mlr.txt>`_.
|
|
|
|
The idea is that words from the input file are consumed, then taken apart and pasted back together in ways which imitate the letter-to-letter transitions found in the word list -- giving us automatically generated words in the same vein as *bromance* and *spork*:
|
|
|
|
GENRST_CARDIFY_HIGHLIGHT_ONE
|
|
mlr --nidx --from ./ngrams/gsl-2000.txt put -q -f ./ngrams/ngfuncs.mlr -f ./ngrams/ng5.mlr
|
|
beard
|
|
plastinguish
|
|
politicially
|
|
noise
|
|
loan
|
|
country
|
|
controductionary
|
|
suppery
|
|
lose
|
|
lessors
|
|
dollar
|
|
judge
|
|
rottendence
|
|
lessenger
|
|
diffendant
|
|
suggestional
|
|
GENRST_EOF
|