miller/CLAUDE.md

Claude Development Guide for Miller

Project Overview

Miller is a command-line data processing tool for working with CSV, TSV, JSON, and other data formats. It's written in Go (v1.18+) and provides SQL-like operations on data.

Initial Setup

Setting Up staticcheck

The make staticcheck target requires the staticcheck tool. To set it up:

go install honnef.co/go/tools/cmd/staticcheck@latest

This installs staticcheck to ~/go/bin/ (the default Go binaries directory). For make staticcheck to work, you need ~/go/bin in your PATH.

Add to your shell profile (.bashrc, .zshrc, or equivalent):

export PATH="$PATH:$HOME/go/bin"

Verify the installation:

staticcheck -version

If this works, you can use make staticcheck without any further setup.

Setting Up golangci-lint

CI runs golangci-lint (config in .golangci.yml) on every push and PR via .github/workflows/golangci-lint.yml. To run it locally, install the version matching CI (currently v2.12.2) per the official instructions, e.g.:

curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.12.2

This also installs to ~/go/bin/, so the same PATH addition above covers it.

Verify the installation:

golangci-lint --version

Build & Test

Building

make build          # Build the mlr executable
make quiet          # Build silently (no output messages)

Testing

make check          # Run all tests (unit + regression)
make unit-test      # Run unit tests only
make regression-test # Run regression tests only
make bench          # Run benchmarks

Code Quality

make fmt            # Format code with go fmt
make staticcheck    # Run static analysis (see Initial Setup section above)
make lint           # Run golangci-lint, same invocation as CI (see Initial Setup section above)

make lint runs golangci-lint run ./cmd/mlr ./pkg/..., matching .github/workflows/golangci-lint.yml exactly, so a clean local run usually means CI's lint job will pass too. It is not part of make check or make dev, so run it explicitly before pushing.

Note: CI's lint job restores a persistent golangci-lint results cache shared across commits and PRs (via golangci-lint-action), which can occasionally serve stale staticcheck results (e.g. spurious SA5011 nil-check warnings) unrelated to your diff. The job is continue-on-error: true for this reason, so it won't block merging. If CI lint fails but make lint is clean locally, it's likely a stale cache — rerun the job, or a maintainer can clear it: gh cache list --key golangci-lint then gh cache delete <id>.

Full Developer Workflow

make dev            # Format, build, test, generate docs (comprehensive check before pushing)

Project Structure

• cmd/mlr - Main Miller executable entry point
• pkg/ - Core library code organized by functionality
  • pkg/lib/ - Utility libraries
  • pkg/scan/ - Input scanning
  • pkg/mlrval/ - Miller value types
  • pkg/bifs/ - Built-in functions
  • pkg/input/ - Input format handlers
• regression_test.go - Regression test suite
• docs/ - Documentation (Markdown with live code samples)
• man/ - Man page generation

Development Conventions

Code Style

• Follow Go conventions and go fmt output
• Use meaningful variable and function names
• Keep functions focused and testable

Testing

• Add unit tests for new functionality in pkg/*/ directories
• Use go test for unit tests
• Run make regression-test for integration testing
• The mlr regtest command provides more control for interactive debugging

Documentation

• Update relevant .md.in files in docs/src/ when adding features
• These are processed into live documentation with actual code examples
• Run make dev to rebuild documentation

Git Workflow

• Create descriptive commit messages
• Reference issue numbers when fixing bugs or implementing features
• Test locally with make check before committing
• For major changes, run make dev to ensure docs and tests pass

Key Dependencies

Miller uses the Go standard library. Check go.mod for specific versions.

Common Tasks

Adding a New Built-in Function

1. Implement in appropriate package (likely pkg/bifs/)
2. Add unit tests alongside
3. Update documentation in docs/src/
4. Run make check to verify

Fixing a Bug

1. Create a minimal test case (unit test or regression test)
2. Fix the code
3. Verify with make check
4. Commit with reference to issue number

Performance Optimization

1. Run make bench to establish baseline
2. Make changes
3. Run benchmarks again to measure improvement
4. Consider adding permanent benchmark in test files

Documentation Building

Documentation is built from .md.in template files that contain live code samples executed via Miller itself. When you make changes that affect command output or behavior, you may need to update these templates and rebuild docs with make -C docs/src forcebuild.

Before Pushing

Always run:

make dev
make lint

make dev ensures code formatting, builds successfully, passes all tests, and documentation is up to date. make lint is separate (not part of make dev/make check) and mirrors the CI lint job.

Additional Resources

• Full documentation
• Contributing guidelines
• Issue labeling notes