miller/doc/content-for-build.html

POKI_PUT_TOC_HERE

<p/>
<button style="font-weight:bold;color:maroon;border:0" onclick="expand_all();" href="javascript:;">Expand all sections</button>
<button style="font-weight:bold;color:maroon;border:0" onclick="collapse_all();" href="javascript:;">Collapse all sections</button>

<h1>Miller license</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_miller_license');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_miller_license" style="display: block">

<p/>Two-clause BSD license <a href="https://github.com/johnkerl/miller/blob/master/LICENSE.txt">
https://github.com/johnkerl/miller/blob/master/LICENSE.txt</a>.

</div>
<h1>Prebuilt executables</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_prebuilt_executables');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_prebuilt_executables" style="display: block">

<p/> Please see <a href="https://github.com/johnkerl/miller/releases">
https://github.com/johnkerl/miller/releases</a> where there are builds for OSX
Yosemite, Linux i686 (statically linked), and Linux x86-64 (dynamically
linked).

<p/> Homebrew installation support for OSX is available via

POKI_CARDIFY(brew update && brew install miller)HERE

<p/> You may already have <tt>mlr</tt> available in your platform&rsquo;s
package manager on NetBSD, Debian Linux, Ubuntu Xenial and upward, Arch Linux, or perhaps other distributions.
For example, on Debian/Ubuntu, you can try

POKI_CARDIFY(sudo apt-get install miller)HERE

</div>
<h1>Building from source</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_building_from_source');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_building_from_source" style="display: block">

<h2>From release tarball using autoconfig</h2>

<p/>Miller allows you the option of using GNU autoconfigure to build portably.

<p/>Grateful acknowledgement: Miller&rsquo;s GNU autoconfig work was done by
the generous and expert efforts of <a href="https://github.com/0-wiz-0/">Thomas
Klausner</a>.


<ul>
<li/> Obtain <tt>mlr-i.j.k.tar.gz</tt> from <a
  href="https://github.com/johnkerl/miller/tags">https://github.com/johnkerl/miller/tags</a>,
  replacing <tt>i.j.k</tt> with the desired release, e.g. <tt>2.2.1</tt>.
<li/> <tt>tar zxvf mlr-i.j.k.tar.gz</tt>
<li/> <tt>cd mlr-i.j.k</tt>
<li/> Install the following packages using your system&rsquo;s package manager
  (<tt>apt-get</tt>, <tt>yum install</tt>, etc.):
  <b>automake autoconf libtool flex</b> (if you need to run <tt>autoreconf -fiv</tt>) else just
  <b>libtool flex</b>
<li> Various configuration options of your choice, e.g.
  <ul>
  <li/> <tt>autoreconf -fiv</tt> (no longer necessary as of 5.2.0)
  <li/> <tt>./configure</tt>
  <li/> <tt>./configure --prefix=/usr/local</tt>
  <li/> <tt>./configure --prefix=$HOME/pkgs</tt>
  <li/> <tt>./configure CC=clang</tt>
  <li/> <tt>./configure --disable-shared</tt> (to make a statically linked executable)
  <li/> <tt>./configure 'CFLAGS=-Wall -std=gnu99 -O3'</tt>
  <li/> etc.
  </ul>
<li/> <tt>make</tt> creates the <tt>c/mlr</tt> executable
<li/> <tt>make check</tt>
<li/> <tt>make install</tt> copies the <tt>c/mlr</tt> executable to your prefix&rsquo;s <tt>bin</tt> subdirectory.
</ul>

<h2>From git clone using autoconfig</h2>

<ul>
<li/> <tt>git clone https://github.com/johnkerl/miller</tt>
<li/> <tt>cd miller</tt>
<li/> Then continue from &ldquo;Install the following ... &rdquo; as above.
</ul>

<h2>Without using autoconfig</h2>

<p/> GNU autoconfig is familiar to many users, and indeed plenty of folks won&rsquo;t bother to use an open-source
software package which doesn&rsquo;t have autoconfig support. And this is for good reason: GNU autoconfig allows
us to build software on a wide diversity of platforms. For this reason I&rsquo;m happy that Miller supports
autoconfig.

<p/>But, many others (myself included!) find autoconfig confusing: if it works
without errors, great, but if not, the <tt>./configure &amp;&amp; make</tt>
output can be exceedingly difficult to decipher. And this also can be a
turn-off for using open-source software: if you can&rsquo;t figure out the
build errors, you may just keep walking. For this reason I&rsquo;m happy that
Miller allows you to build without autoconfig. (Of course, if you have any
build errors, feel free to contact me at
<a href="mailto:kerl.john.r+miller@gmail.com">kerl.john.r+miller@gmail.com</a>,
&mdash; or, better, open an issue with &ldquo;New Issue&rdquo; at
<a href="https://github.com/johnkerl/miller/issues">
https://github.com/johnkerl/miller/issues</a>.)

<p/>Steps:
<ul>
<li/> Obtain a release tarball or git clone.
<li/> <tt>cd</tt> into the <tt>c</tt> subdirectory.
<li/> Edit the <tt>INSTALLDIR</tt> in <tt>Makefile.no-autoconfig</tt>.
<li/> To change the C compiler, edit the <tt>CC=</tt> lines in
  <tt>Makefile.no-autoconfig</tt> and <tt>dsls/Makefile.no-autoconfig</tt>.
<li/> <tt>make -f Makefile.no-autoconfig</tt> creates the <tt>mlr</tt>
  executable and runs unit/regression tests (i.e. the equivalent of both
  <tt>make</tt> and <tt>make check</tt> using autoconfig).
<li/> <tt>make install</tt> copies the <tt>mlr</tt> executable to your install directory.
</ul>

<p/>The <tt>Makefile.no-autoconfig</tt> is simple: little more than <tt>gcc
*.c</tt>.  Customzing is less automatic than autoconfig, but more transparent.
I expect this makefile to work with few modifications on a large fraction of
modern Linux/BSD-like systems: I&rsquo;m aware of successful use with
<tt>gcc</tt> and <tt>clang</tt>, on Ubuntu 12.04 LTS, SELinux, Darwin (MacOS
Yosemite), and FreeBSD.

</div>
<h1>Windows</h1>

<p/>Miller is built on Windows using MSYS2: <a
href="http://www.msys2.org/">http://www.msys2.org/</a>.  You can install MSYS2
and build Miller from its source code within MSYS2, and then you can use the
binary from outside MSYS2.  You can also use a precompiled binary.

<h2>Precompiled binaries</h2>

<p/>Miller is autobuilt for <b>Linux</b> using <b>Travis</b> on every commit (<a
href="https://travis-ci.org/johnkerl/miller/builds">https://travis-ci.org/johnkerl/miller/builds</a>).
This was set up by the generous assistance of <a
href="https://github.com/SikhNerd">SikhNerd</a> on Github, tracked in <a
href="https://github.com/johnkerl/miller/issues/15">https://github.com/johnkerl/miller/issues/15</a>.
Analogously, Miller is autobuilt for <b>Windows</b> using the <b>Appveyor</b> continuous-build system:
<a href="https://ci.appveyor.com/project/johnkerl/miller">https://ci.appveyor.com/project/johnkerl/miller</a>.

<p/>Miller releases from <a
href="https://github.com/johnkerl/miller/releases/tag/v5.1.0w">5.1.0w</a>
onward will have a precompiled Windows binary, in addition to the MacOSX, Linux
32-bit static, and Linux 64-bit precompiled binaries as on previous releases.
Specifically, at <a href="https://ci.appveyor.com/project/johnkerl/miller">
https://ci.appveyor.com/project/johnkerl/miller</a> you can select <i>Latest
Build</i> and then <i>Artifacts</i> to always get the current head build.
Miller releases from 5.3.0 onward will simply point to a particular Appveyor
artifact associated with the release.

<h2>Building from source</h2>

<p/> You will first need to install MSYS2: <a href="http://www.msys2.org/">http://www.msys2.org/</a>.

<p/> Then, start an MSYS2 shell, e.g. (supposing you installed MSYS2 to
<tt>C:\msys2\</tt>) run <tt>C:\msys2\mingw64.exe</tt>.  Within the MSYS2 shell,
you can run the following to install dependent packages:

<div class=pokipanel>
<pre>
pacman -Syu
pacman -Su
pacman -S base-devel
pacman -S msys2-devel
pacman -S mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-pcre
pacman -S msys2-runtime
</pre>
</div>

<p/>The list of dependent packages may be also found in <b>appveyor.yml</b>
in the Miller base directory.

<p/>Then, simply run <b>msys2-build.sh</b> which is a thin wrapper around
<tt>./configure &amp;&amp; make</tt> which accommodates certain Windows/MSYS2
idiosyncracies.

<p/> There is a unit-test false-negative issue involving the semantics of the
<tt>mkstemp</tt> library routine but a <tt>make -k</tt> in the <tt>c</tt>
subdirectory has been producing a <tt>mlr.exe</tt> for me.

<p/> Within MSYS2 you can run <tt>mlr</tt>: simply copy it from the <tt>c</tt>
subdirectory to your desired location somewhere within your MSYS2
<tt>$PATH</tt>.  To run <tt>mlr</tt> outside of MSYS2, just as with precompiled
binaries as described above, you&rsquo;ll need <tt>msys-2.0.dll</tt>.  One way
to do this is to augment your path:

<div class=pokipanel>
<pre>
C:\&gt; set PATH=%PATH%;\msys64\mingw64\bin
</pre>
</div>

<p/> Another way to do it is to copy the Miller executable and the DLL to the same directory:

<div class=pokipanel>
<pre>
C:\&gt; mkdir \mbin
C:\&gt; copy \msys64\mingw64\bin\msys-2.0.dll \mbin
C:\&gt; copy \msys64\wherever\you\installed\miller\c\mlr.exe \mbin
C:\&gt; set PATH=%PATH%;\mbin
</pre>
</div>

<p/> The Windows port is still recent: please open an issue at
<a href="https://github.com/johnkerl/miller/issues">https://github.com/johnkerl/miller/issues</a>
if you encounter any problems.

</div>
<h1>In case of problems</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_in_case_of_problems');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_in_case_of_problems" style="display: block">

<p/> If you have any build errors, feel free to contact me at <a
href="mailto:kerl.john.r+miller@gmail.com">kerl.john.r+miller@gmail.com</a>,
&mdash; or, better, open an issue with &ldquo;New Issue&rdquo; at
<a href="https://github.com/johnkerl/miller/issues">
https://github.com/johnkerl/miller/issues</a>.

</div>
<h1>Dependencies</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_dependencies');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_dependencies" style="display: block">

<h2>Required external dependencies</h2>

<p/> These are necessary to produce the <tt>mlr</tt>
executable.
<ul>
<li/> <tt>gcc</tt>, <tt>clang</tt>, etc. (or presumably other compilers; please
open an issue or send me a pull request if you have information for me about
other 21st-century compilers)
<li/> The standard C library
<li/> <tt>flex</tt>
<li/> <tt>automake</tt>, <tt>autoconf</tt>, and <tt>libtool</tt>, if you build with autoconfig
</ul>

<h2>Optional external dependencies</h2>

This documentation pageset is built using <boldmaroon>Poki</boldmaroon>:
<a href="http://johnkerl.org/poki/doc">docs here</a>,
<a href="https://github.com/johnkerl/poki">source code here</a>.
Note that <a href="http://johnkerl.org/miller/doc/index.html">http://johnkerl.org/miller/doc/index.html</a>
is nothing more than Miller&rsquo;s <tt>doc</tt> directory served up by a web server.
You&rsquo;ll need <tt>poki</tt> if you modify documents, or if you modify the code in a way that affects the documents
(there are auto-run snippets inserted into the doc).
The best way to discover this is to run <tt>make install</tt> as above, then run <tt>poki</tt> in Miller&rsquo;s
<tt>doc</tt> subdirectory, then run <tt>git diff</tt> to see if docs were affected
by the code mod.  (If so, commit and push them.)

<h2>Internal dependencies</h2>
These are included within the <a href="https://github.com/johnkerl/miller">Miller source tree</a> and do not
need to be separately installed (and in fact any separate installation will not be picked up in the Miller build):
<ul>
<li/><a href="http://en.wikipedia.org/wiki/Mersenne_Twister">Mersenne Twister</a> for pseudorandom-number
  generation: <a href="https://github.com/johnkerl/miller/blob/master/c/lib/mtrand.c">C implementation by Nishimura and Matsumoto</a> with license terms respected.
<li/> <a href="http://www.jera.com/techinfo/jtns/jtn002.html">MinUnit</a>  for unit-testing,
with as-is-no-warranty license <a href="http://www.jera.com/techinfo/jtns/jtn002.html#License">http://www.jera.com/techinfo/jtns/jtn002.html#License</a>,
<a href="https://github.com/johnkerl/miller/blob/master/c/lib/minunit.h">
https://github.com/johnkerl/miller/blob/master/c/lib/minunit.h</a>.
<li/> The <a href="http://www.hwaci.com/sw/lemon/">Lemon parser-generator</a>, the author of which
<a href="https://github.com/johnkerl/miller/blob/master/c/dsls/lemon.c">explicitly disclaims
copyright</a>.
<li/> The <a href="https://github.com/udp/json-parser">udp JSON parser</a>, with BSD2 license.
<li/> The NetBSD <tt>strptime</tt> (needed for the Windows/MSYS2 port since MSYS2 lacks this), with BSD license.
</ul>

</div>
<h1>Creating a new release: for developers</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_creating_a_new_release');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_creating_a_new_release" style="display: block">

<p/>At present I&rsquo;m the primary developer so this is just my checklist for making new releases.

<p/>In this example I am using version 3.4.0; of course that will change for subsequent revisions.
<ul>

<li>Update version found in <tt>mlr --version</tt> and <tt>man mlr</tt>:
  <ul>
  <li/> Edit <tt>configure.ac</tt>, <tt>c/mlrvers.h</tt>, and <tt>mlr.spec</tt>
    from <tt>3.3.2-dev</tt> to <tt>3.4.0</tt>.
  <li/> <tt>make -C c -f Makefile.no-autoconfig installhome &amp;&amp;
    make -C doc -f Makefile.no-autoconfig all installhome</tt>
  <li/> Commit and push.
  <li/> ISP upload so <a href="http://johnkerl.org/miller/doc/index.html">http://johnkerl.org/miller/doc/index.html</a>
    will have updated head-docs.
  </ul>

<li>Create the release tarball and SRPM:
  <ul>
  <li/> On buildbox: <tt>./configure &amp;&amp; make distcheck</tt>
  <li/> On buildbox: make SRPM as in <a href="https://github.com/johnkerl/miller/blob/master/README-RPM.md">
    https://github.com/johnkerl/miller/blob/master/README-RPM.md</a>
  <li/> On all buildboxes: <tt>cd c</tt> and <tt>make -f Makefile.no-autoconfig mlr.static</tt>. Then copy
    <tt>mlr.static</tt> to <tt>../mlr.{arch}</tt>.
  <li/> Pull back release tarball <tt>mlr-3.4.0.tar.gz</tt> and SRPM <tt>mlr-3.4.0-1.el6.src.rpm</tt> from buildbox,
    and <tt>mlr.{arch}</tt> binaries from whatever buildboxes.
  </ul>

<li/>Create the Github release tag:
  <ul>
  <li/> Write the release notes
  <li/> Attach the release tarball, SRPM, and binaries
  <li/> Publish the release
  </ul>

<li/>Create the release-specific docs:
  <ul>
  <li/> Pull the Github-generated source <tt>.tgz</tt> from Github to ISP
    miller-releases directory and unpack and <tt>cd</tt> into it.
  <li/> <tt>cd</tt> into <tt>doc</tt> subdir. Edit <tt>template.html</tt>&rsquo;s <tt>titleinbody</tt> tag from
    <tt>Miller</tt> to <tt>Miller 3.4.0</tt> and poki-build.
  <li/> ISP upload
  <li/> At the ISP site, update the latest-revision symlink
    <tt>ln -s pub_http_internet/miller-releases/miller-3.4.0 pub_http_internet/miller</tt>.
    (This is the main doc URL linked to from the top of the Github repo so it
    must point to this newly published revision.)
  <li/> Back in the Github repo, edit <tt>doc/content-for-release-docs.html</tt> to include new release, poki-build,
    push/commit, and ISP upload.
  </ul>

<li/>Notify:
  <ul>
  <li/> Submit <tt>brew</tt> pull request; notify any other distros which don&rsquo;t appear to have autoupdated
    since the previous release.
<pre>
git remote add upstream https://github.com/Homebrew/homebrew-core # one-time setup only
git fetch upstream
git rebase upstream/master
git checkout -b miller-3.4.0
shasum -a 256 /path/to/mlr-3.4.0.tar.gz
edit Formula/miller.rb
git add Formula/miller.rb
git commit -m 'miller 3.4.0'
git push -u origin miller-3.4.0
(submit the pull request)
</pre>
  <li/> Social-media updates.
  </ul>

<li/>Afterwork:
  <ul>
  <li> Edit <tt>configure.ac</tt> and <tt>c/mlrvers.h</tt> to change version from <tt>3.4.0</tt> to
    <tt>3.4.0-dev</tt>.
  <li/> <tt>make -C c -f Makefile.no-autoconfig installhome &amp;&amp;
    make -C doc -f Makefile.no-autoconfig all installhome</tt>
  <li> Commit and push.
  </ul>

</ul>

</div>
<h1>Misc. development notes</h1>
<button style="font-weight:bold;color:maroon;border:0" padding=0 onclick="toggle_by_name('section_toggle_misc_devel_notes');" href="javascript:;">Toggle section visibility</button>
<div id="section_toggle_misc_devel_notes" style="display: block">

<p/>I use terminal width 120 and tabwidth 4.

</div>