diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index e00364dbb..264ab13a1 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -41,7 +41,9 @@ nav: - "Shell completion": "shell-completion.md" - "Syntax highlighting: vimrc": "vimrc.md" - "The REPL": "repl.md" - - "The MCP server": "mcp-server.md" + - "The Miller Agent Skill": "agent-skill.md" + - "The Miller MCP server": "mcp-server.md" + - "Miller AI internals": "ai-support.md" - "Online help": "online-help.md" - "How to contribute": "contributing.md" - 'FAQs and examples': diff --git a/docs/src/agent-skill.md b/docs/src/agent-skill.md new file mode 100644 index 000000000..793e34b4d --- /dev/null +++ b/docs/src/agent-skill.md @@ -0,0 +1,244 @@ + +
+mlr skill print | head -n 15 ++
+--- +name: miller +description: > + Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing + mlr command lines: discover capabilities from the catalog rather than + guessing, learn the data's shape before writing expressions, validate DSL + before running, and recover from failures via structured errors. +--- + +# Miller agent playbook + +Miller (`mlr`) is a command-line data processor for CSV, TSV, JSON, JSON +Lines, and other tabular/record formats, with SQL-like verbs (`cut`, `sort`, +`join`, `stats1`, ...) and an awk-like DSL (`put`, `filter`). + ++ +For more background on the `mlr` commands the agent runs on your behalf, please see +[Miller AI internals](ai-support.md). + +## Setup + +Write the skill file to Claude Code's personal skills directory (do this before starting your +`claude` session): + +
+mlr skill install ~/.claude/skills/miller ++
+Wrote /Users/kerl/.claude/skills/miller/SKILL.md ++ +For Codex and Gemini: + +
+mlr skill install ~/.agents/skills/miller ++ +With no argument, `install` writes to `.claude/skills/miller/SKILL.md` under the current directory +instead. This is handy for a project-scoped skill checked into that project's repo rather than one +installed for every project on your machine: + +
+mlr skill install ++
+Wrote .claude/skills/miller/SKILL.md ++ +There's no "uninstall" subcommand, since `install` only ever writes one plain file. Removing it is +an ordinary file operation: + +
+rm -rf ~/.claude/skills/miller ++ +Then -- just interact with your agent as always! When you say something like `describe the data file example.csv`, +the agent will already know how to use Miller to help answer that question. + +## What the Miller skill maps to + +You don't have to type `skill` or anything else special in your agent session: rather you've +empowered the agent to discover things about Miller for itself. But if you're curious what's +actually placed in front of it: + +
+mlr skill --help ++
+Usage: mlr skill {print|install} [options]
+Puts the Miller Agent Skill (SKILL.md) where a coding agent can find it.
+This is the same playbook mlr mcp serves as its "miller-playbook"
+prompt/resource, packaged for agents that read Agent Skills from disk.
+
+Subcommands:
+ print Write the skill content to stdout.
+ install [DIR] Write DIR/SKILL.md, creating DIR if needed.
+ Default DIR is .claude/skills/miller
+
+ -h or --help Show this message.
+
+
+And here's the file itself -- the whole thing, not an excerpt, since this and nothing else is what
+the agent has to go on:
+
++mlr skill print ++
+---
+name: miller
+description: >
+ Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing
+ mlr command lines: discover capabilities from the catalog rather than
+ guessing, learn the data's shape before writing expressions, validate DSL
+ before running, and recover from failures via structured errors.
+---
+
+# Miller agent playbook
+
+Miller (`mlr`) is a command-line data processor for CSV, TSV, JSON, JSON
+Lines, and other tabular/record formats, with SQL-like verbs (`cut`, `sort`,
+`join`, `stats1`, ...) and an awk-like DSL (`put`, `filter`).
+
+Work this loop. Each step exists to prevent a specific, common failure.
+
+## 1. Discover — never invent names
+
+Everything valid is in the catalog; anything not in the catalog does not
+exist. Hallucinated flag/function names are the top failure mode.
+
+- Route an intent: `which` with e.g. `"join two files on a key"` → ranked
+ candidates. `confident: true` means a name matched; trust the top hit.
+- Browse cheaply: `list_capabilities` with `index: true` → every
+ verb/function/flag/keyword with one-line summaries.
+- Drill in: `list_capabilities` with `kind: "verb", names: ["join"]` → the
+ full entry. Prefer the structured `options` list (flag, arg, type, enum
+ `values`) when present; `usage_text` is the prose fallback.
+- The whole catalog is cacheable against `(mlr_version,
+ catalog_schema_version)` — re-fetch only when either changes.
+
+## 2. Constrain — learn the data before touching it
+
+Call `describe_data` on the input first. It returns, per field: name, types
+seen with counts, occurrence count, null count, cardinality, min/max, and —
+for low-cardinality fields — every distinct value.
+
+- Copy field names exactly from `describe_data`; never guess casing or
+ spelling.
+- For flags like `-g` (group-by) and DSL comparisons, use values from the
+ `values` array, not values you expect to exist.
+- Fields whose `count` is less than other fields' are absent in some records:
+ guard DSL with `is_present($field)`.
+
+## 3. Validate — check DSL before spending a run
+
+Before any `run` that includes `put` or `filter`, call `validate_dsl` with the
+expression. Cost: parse-only, no data read. On `valid: false`, the `error`
+document has `kind`, `hint`, and `did_you_mean` — apply the hint, don't
+re-guess syntax.
+
+## 4. Run — and read errors structurally
+
+Call `run` with argv as a list, one element per shell word (no shell quoting):
+
+ {"args": ["--icsv", "--ojson", "cat", "data.csv"]}
+
+Command-line shape rules that prevent most argv errors:
+
+- Main flags (I/O formats etc.) come **before** the verb: `mlr --icsv sort -f name f.csv`.
+- Format shorthands: `--icsv --ojson` (separate in/out), `--csv`/`--c2j` etc. (combined).
+- Chain verbs with `then`: `["--icsv", "sort", "-f", "k", "then", "head", "-n", "3", "f.csv"]`.
+- If a field value being compared in `filter` might collide with a verb flag,
+ end verb flags with `--` before filenames.
+- Inline data goes in `stdin_text`; files go at the end of `args`.
+
+On failure, `exit_code` is nonzero and `error` (when present) carries `kind`,
+`hint`, and `did_you_mean` — `hint` is often a corrected command line; prefer
+executing it over reasoning from the message. `stdout_truncated: true` means
+the output exceeded the server's cap: narrow the query (e.g. `head`, `cut`)
+rather than re-running the same command.
+
+## Notes
+
+- `run` cannot execute external commands (DSL `system`/`exec`, piped
+ redirects, `--prepipe`) unless the server was started with `--allow-shell`;
+ such calls fail cleanly. It **can** write files via `tee`, `split`, and DSL
+ output redirects — treat it as a write-capable tool.
+- Long inputs: prefer `describe_data` + targeted verbs over dumping whole
+ files through `run`.
+- One record format in, another out: Miller is format-to-format; there is no
+ separate conversion step.
+
+
+That playbook is prose, not named tools, but it rests on the Miller features documented
+in the [Miller AI internals](ai-support.md) page.
+
+## What using the Miller skill looks like in practice
+
+There's no server status to check and no tool list to browse -- the skill is just text the agent
+already has -- so "in practice" mostly looks like an ordinary conversation. Say you're looking at
+[example.csv](example.csv) for the first time:
+
+> **You:** In example.csv, show me the red rows.
+
+Without the skill, a plausible guess for the DSL is `$color == "Red"` -- and Miller silently
+returns nothing for it, since the real values are lowercase. With the skill installed, the agent
+runs `mlr --icsv --ojson describe example.csv` on your behalf first, sees the real value set for
+`color` (`yellow`, `red`, `purple`), and only then answers:
+
+> **Agent:** Four rows have color = red: rows 2, 3, 4, and 6.
+
+The full worked version of this example, including the exact commands run at each step, is in
+[Miller and AI](ai.md#before-and-after-a-first-session-with-the-skill-installed).
+
+## A note on sandboxing
+
+The [MCP server](mcp-server.md) enforces a sandbox by construction: subprocesses it spawns run with
+`MLR_NO_SHELL=1` unless you start it with `--allow-shell`, so an agent-constructed command line
+can't execute external commands even if the agent wanted it to.
+
+The skill file has no equivalent enforcement. It's advisory text, not a wrapper around subprocess
+execution -- nothing stops an agent from running `mlr put 'end{print system("whatever")}'` with
+your full shell permissions if it decides to. If you want that guarantee with the skill alone,
+set the `MLR_NO_SHELL` [environment variable](reference-main-env-vars.md) yourself (or pass
+`--no-shell` explicitly), rather than relying on the playbook text for isolation. If you want the
+enforced version, register the [MCP server](mcp-server.md) instead of, or alongside, the skill.
diff --git a/docs/src/agent-skill.md.in b/docs/src/agent-skill.md.in
new file mode 100644
index 000000000..ba5719d2d
--- /dev/null
+++ b/docs/src/agent-skill.md.in
@@ -0,0 +1,109 @@
+# The Miller Agent Skill
+
+As of Miller version 6.20, released in July 2026, there are two main ways to get your AI to know
+about a software tool (Miller, or others): **agent skills**, and [**MCP**](mcp-server.md). (See
+[Miller and AI](ai.md) for an introduction.)
+
+Miller ships a built-in [Agent Skill](https://www.anthropic.com/news/skills) -- a single `SKILL.md`
+file -- inside the `mlr` executable, so agents that read skills directly from disk (Claude Code,
+and other tools that support the Agent Skills format) can discover and drive Miller without
+scraping help text or guessing at flags.
+
+The skill is plain markdown with a YAML frontmatter header, placed where your agent already looks
+for skills. The agent reads it into context once, the same way it reads any other instructions, and
+from then on it runs `mlr` commands via whatever shell-executing tool it already has.
+
+Here's what the skill file looks like:
+
+GENMD-RUN-COMMAND
+mlr skill print | head -n 15
+GENMD-EOF
+
+For more background on the `mlr` commands the agent runs on your behalf, please see
+[Miller AI internals](ai-support.md).
+
+## Setup
+
+Write the skill file to Claude Code's personal skills directory (do this before starting your
+`claude` session):
+
+GENMD-CARDIFY-HIGHLIGHT-ONE
+mlr skill install ~/.claude/skills/miller
+Wrote /Users/kerl/.claude/skills/miller/SKILL.md
+GENMD-EOF
+
+For Codex and Gemini:
+
+GENMD-CARDIFY-HIGHLIGHT-ONE
+mlr skill install ~/.agents/skills/miller
+GENMD-EOF
+
+With no argument, `install` writes to `.claude/skills/miller/SKILL.md` under the current directory
+instead. This is handy for a project-scoped skill checked into that project's repo rather than one
+installed for every project on your machine:
+
+GENMD-CARDIFY-HIGHLIGHT-ONE
+mlr skill install
+Wrote .claude/skills/miller/SKILL.md
+GENMD-EOF
+
+There's no "uninstall" subcommand, since `install` only ever writes one plain file. Removing it is
+an ordinary file operation:
+
+GENMD-CARDIFY-HIGHLIGHT-ONE
+rm -rf ~/.claude/skills/miller
+GENMD-EOF
+
+Then -- just interact with your agent as always! When you say something like `describe the data file example.csv`,
+the agent will already know how to use Miller to help answer that question.
+
+## What the Miller skill maps to
+
+You don't have to type `skill` or anything else special in your agent session: rather you've
+empowered the agent to discover things about Miller for itself. But if you're curious what's
+actually placed in front of it:
+
+GENMD-RUN-COMMAND
+mlr skill --help
+GENMD-EOF
+
+And here's the file itself -- the whole thing, not an excerpt, since this and nothing else is what
+the agent has to go on:
+
+GENMD-RUN-COMMAND
+mlr skill print
+GENMD-EOF
+
+That playbook is prose, not named tools, but it rests on the Miller features documented
+in the [Miller AI internals](ai-support.md) page.
+
+## What using the Miller skill looks like in practice
+
+There's no server status to check and no tool list to browse -- the skill is just text the agent
+already has -- so "in practice" mostly looks like an ordinary conversation. Say you're looking at
+[example.csv](example.csv) for the first time:
+
+> **You:** In example.csv, show me the red rows.
+
+Without the skill, a plausible guess for the DSL is `$color == "Red"` -- and Miller silently
+returns nothing for it, since the real values are lowercase. With the skill installed, the agent
+runs `mlr --icsv --ojson describe example.csv` on your behalf first, sees the real value set for
+`color` (`yellow`, `red`, `purple`), and only then answers:
+
+> **Agent:** Four rows have color = red: rows 2, 3, 4, and 6.
+
+The full worked version of this example, including the exact commands run at each step, is in
+[Miller and AI](ai.md#before-and-after-a-first-session-with-the-skill-installed).
+
+## A note on sandboxing
+
+The [MCP server](mcp-server.md) enforces a sandbox by construction: subprocesses it spawns run with
+`MLR_NO_SHELL=1` unless you start it with `--allow-shell`, so an agent-constructed command line
+can't execute external commands even if the agent wanted it to.
+
+The skill file has no equivalent enforcement. It's advisory text, not a wrapper around subprocess
+execution -- nothing stops an agent from running `mlr put 'end{print system("whatever")}'` with
+your full shell permissions if it decides to. If you want that guarantee with the skill alone,
+set the `MLR_NO_SHELL` [environment variable](reference-main-env-vars.md) yourself (or pass
+`--no-shell` explicitly), rather than relying on the playbook text for isolation. If you want the
+enforced version, register the [MCP server](mcp-server.md) instead of, or alongside, the skill.
diff --git a/docs/src/ai-support.md b/docs/src/ai-support.md
new file mode 100644
index 000000000..4c01bc5f0
--- /dev/null
+++ b/docs/src/ai-support.md
@@ -0,0 +1,298 @@
+
++mlr help --as-json --index | mlr --json head -n 2 ++
+[
+{
+ "kind": "verb",
+ "name": "altkv",
+ "summary": "Given fields with values of the form a,b,c,d,e,f emits a=b,c=d,e=f pairs."
+},
+{
+ "kind": "verb",
+ "name": "bar",
+ "summary": "Replaces a numeric field with a number of asterisks, allowing for cheesy"
+}
+]
+
+
++mlr help --as-json --index | mlr --json count ++
+[
+{
+ "count": 661
+}
+]
+
+
+From the index, an agent drills into full entries one at a time: `mlr help verb sort --as-json`,
+`mlr help function splitax --as-json`, `mlr help flag --ifs --as-json`, `mlr help keyword ENV
+--as-json` -- each accepting one or more names. A verb entry carries a structured option list --
+flag, argument placeholder, type -- alongside the familiar usage text:
+
++mlr help verb decimate --as-json ++
+[
+ {
+ "name": "decimate",
+ "summary": "Passes through one of every n records, optionally by category.",
+ "ignores_input": false,
+ "options": [
+ {
+ "flag": "-b",
+ "type": "bool",
+ "desc": "Decimate by printing first of every n."
+ },
+ {
+ "flag": "-e",
+ "type": "bool",
+ "desc": "Decimate by printing last of every n (default)."
+ },
+ {
+ "flag": "-g",
+ "arg": "{a,b,c}",
+ "type": "csv-list",
+ "desc": "Optional group-by-field names for decimate counts, e.g. a,b,c."
+ },
+ {
+ "flag": "-n",
+ "arg": "{n}",
+ "type": "int",
+ "desc": "Decimation factor (default 10)."
+ }
+ ],
+ "usage_text": "Usage: mlr decimate [options]\nPasses through one of every n records, optionally by category.\nOptions:\n-b Decimate by printing first of every n.\n-e Decimate by printing last of every n (default).\n-g {a,b,c} Optional group-by-field names for decimate counts, e.g. a,b,c.\n-n {n} Decimation factor (default 10).\n-h|--help Show this message."
+ }
+]
+
+
+Note that `usage_text` -- what `mlr decimate --help` prints -- is rendered *from* the same
+structured options, so the human help and the machine help cannot drift apart. Function entries
+carry name, class, arity, help, and examples; the examples across the whole catalog are exercised by
+Miller's test suite, so they never rot.
+
+Three properties make the catalog cheap to use:
+
+* _It's a perfect cache key._ Every document carries `mlr_version` and
+ `catalog_schema_version`. Miller is a static binary, so the catalog changes
+ only when the binary does: fetch once, cache forever, re-fetch on a version
+ bump. No TTLs.
+* _It's deterministic._ One document per invocation, sorted entries, no
+ colorization -- stable for diffing and for prompt caches.
+* _It's opt-in twice over._ Per-call via `--as-json`, or set-once via a
+ truthy `MLR_HELP_JSON` environment variable.
+
+For routing an *intent* to a capability -- the reverse of browsing -- `mlr
+which` returns ranked candidates:
+
++mlr which "join two files on a key" | mlr --json head -n 2 ++
+[
+{
+ "kind": "verb",
+ "name": "join",
+ "score": 25,
+ "summary": "Joins records from specified left file name with records from all file names"
+},
+{
+ "kind": "function",
+ "name": "joink",
+ "score": 25,
+ "summary": "Makes string from map/array keys. First argument is map/array; second is separator string."
+}
+]
+
+
+Its exit code signals confidence -- 0 when a query word matched a
+capability's name, 2 when it didn't -- so a harness can branch on status
+without parsing anything.
+
+## Constrain: the tool's shape, and the data's shape
+
+This shows field names, types, cardinality, and value domains for your actual input data.
+
+It's implemented by `mlr describe`.
+
+Agents don't just hallucinate flags; they hallucinate *values*. Miller attacks that from both sides.
+
+Where an option's domain is fixed by the binary, the catalog says so:
+`type` is `enum` and `values` is the complete list. Here's one option of the
+[summary](reference-verbs.md#summary) verb, extracted from the catalog --
+using Miller to query Miller:
+
++mlr help verb summary --as-json | mlr --json put -q 'emit $options[1]' ++
+[
+{
+ "flag": "-a",
+ "arg": "{mean,sum,etc.}",
+ "type": "enum",
+ "desc": "Use only the specified summarizers.",
+ "values": ["field_type", "count", "null_count", "distinct_count", "mode", "sum", "mean", "stddev", "var", "skewness", "minlen", "maxlen", "min", "p25", "median", "p75", "max", "iqr", "lof", "lif", "uif", "uof"]
+}
+]
+
+
+Where the domain depends on your *data* -- which fields exist, what values
+`filter` could compare against, what to pass to `-g` -- the
+[describe](reference-verbs.md#describe) verb profiles the input in one pass:
+per field, the types seen, counts, cardinality, min/max, and (for
+low-cardinality fields) every distinct value:
+
++mlr --icsv --ojson describe then head -n 2 example.csv ++
+[
+{
+ "field_name": "color",
+ "types": {
+ "string": 10
+ },
+ "count": 10,
+ "null_count": 0,
+ "distinct_count": 3,
+ "min": "purple",
+ "max": "yellow",
+ "values": ["yellow", "red", "purple"]
+},
+{
+ "field_name": "shape",
+ "types": {
+ "string": 10
+ },
+ "count": 10,
+ "null_count": 0,
+ "distinct_count": 3,
+ "min": "circle",
+ "max": "triangle",
+ "values": ["triangle", "square", "circle"]
+}
+]
+
+
+The catalog is the *tool's* shape; `describe` is the *data's* shape. An
+agent that consults both has nothing left to guess.
+
+## Validate: check DSL before spending a run
+
+This lets the agent parse and type-check a DSL expression before reading any input files.
+
+It's implemented by `mlr put --explain` and `mlr filter --explain`.
+
+`mlr put --explain` (likewise `mlr filter --explain`) parses and type-checks
+an expression, then exits -- without opening any input at all:
+
++mlr put --explain '$z = $x + $y' ++
+mlr put: DSL expression is valid. ++ +
+mlr put --explain '$z = $x +' ++
+mlr: cannot parse DSL expression.
+mlr: parse error: unexpected EOF ("")
+
+
+## Run and recover: errors as data
+
+Agents are instructed to run Miller commands using `mlr` with the `--errors-json` flag so that a
+failure comes back as a structured document instead of prose.
+
+With `--errors-json` (or `MLR_ERRORS_JSON=true` environment variable), errors arrive as a structured
+document. The `kind` field gives an agent something to branch on; `hint` is a runnable next step,
+not a sentence; and `did_you_mean` is computed against the same catalog the agent discovered from,
+closing the self-correction loop:
+
++mlr --errors-json --icsv sorted -f shape example.csv ++
+{
+ "error": "mlr: verb \"sorted\" not found. Please use \"mlr -l\" for a list.",
+ "kind": "unknown-verb",
+ "token": "sorted",
+ "hint": "Run 'mlr -l' for a list of verbs, or 'mlr help verb \u003cname\u003e' for details.",
+ "did_you_mean": [
+ "sort"
+ ]
+}
+
+
+And since Miller's DSL includes [system and exec](shell-commands.md), there's a sandbox:
+`--no-shell` (or a truthy `MLR_NO_SHELL` environment variable) disables all external-command
+execution -- the DSL `system` and `exec` functions, piped redirects, and `--prepipe` fail cleanly:
+
+
+mlr --no-shell -n put 'end{print system("hostname")}'
+
++(error) ++ +## Summary + +A typical agent profile sets all three environment variables once: + +
+export MLR_HELP_JSON=1 # help/catalog output as JSON +export MLR_ERRORS_JSON=1 # errors as structured JSON +export MLR_NO_SHELL=1 # no external-command execution ++ +Put together, the sections above are a loop -- discover, constrain, validate, run -- where each step +feeds the next, and failures route back with structure instead of prose. diff --git a/docs/src/ai-support.md.in b/docs/src/ai-support.md.in new file mode 100644 index 000000000..2132bd86a --- /dev/null +++ b/docs/src/ai-support.md.in @@ -0,0 +1,151 @@ +# Miller AI internals + +When you use the [Miller agent skill](agent-skill.md) or the [Miller MCP server](mcp-server.md), +here are the `mlr` subcommands your agent runs on your behalf to acquire support. (See also [Miller +and AI](ai.md) for an introduction.) + +The new Miller subcommands as of version 6.20 allow agents to **discover** information about how to +use Miller, **constrain** attempted solutions to match the data, **validate** Miller commands before +running them, **run** them, and robustly **recover** from errors. + +If you like, you can run these subcommands yourself, although you don't need to. These AI-support +subcommands are documented here for transparency. + +## Discover: the machine-readable catalog + +This is the machine-readable catalog of [verbs](reference-verbs.md), [DSL +functions](reference-dsl-builtin-functions.md), [flags](reference-main-flag-list.md), and +[keywords](reference-dsl-variables.md#keywords-for-filter-and-put), plus intent-to-capability +routing. + +These are implemented by `mlr help --as-json` and `mlr which`. + +`mlr help --as-json` emits Miller's entire help catalog as one JSON document. +The `--index` form is the cheap first call -- every capability with a +one-line summary (here trimmed, and then counted, using Miller itself): + +GENMD-RUN-COMMAND +mlr help --as-json --index | mlr --json head -n 2 +GENMD-EOF + +GENMD-RUN-COMMAND +mlr help --as-json --index | mlr --json count +GENMD-EOF + +From the index, an agent drills into full entries one at a time: `mlr help verb sort --as-json`, +`mlr help function splitax --as-json`, `mlr help flag --ifs --as-json`, `mlr help keyword ENV +--as-json` -- each accepting one or more names. A verb entry carries a structured option list -- +flag, argument placeholder, type -- alongside the familiar usage text: + +GENMD-RUN-COMMAND +mlr help verb decimate --as-json +GENMD-EOF + +Note that `usage_text` -- what `mlr decimate --help` prints -- is rendered *from* the same +structured options, so the human help and the machine help cannot drift apart. Function entries +carry name, class, arity, help, and examples; the examples across the whole catalog are exercised by +Miller's test suite, so they never rot. + +Three properties make the catalog cheap to use: + +* _It's a perfect cache key._ Every document carries `mlr_version` and + `catalog_schema_version`. Miller is a static binary, so the catalog changes + only when the binary does: fetch once, cache forever, re-fetch on a version + bump. No TTLs. +* _It's deterministic._ One document per invocation, sorted entries, no + colorization -- stable for diffing and for prompt caches. +* _It's opt-in twice over._ Per-call via `--as-json`, or set-once via a + truthy `MLR_HELP_JSON` environment variable. + +For routing an *intent* to a capability -- the reverse of browsing -- `mlr +which` returns ranked candidates: + +GENMD-RUN-COMMAND +mlr which "join two files on a key" | mlr --json head -n 2 +GENMD-EOF + +Its exit code signals confidence -- 0 when a query word matched a +capability's name, 2 when it didn't -- so a harness can branch on status +without parsing anything. + +## Constrain: the tool's shape, and the data's shape + +This shows field names, types, cardinality, and value domains for your actual input data. + +It's implemented by `mlr describe`. + +Agents don't just hallucinate flags; they hallucinate *values*. Miller attacks that from both sides. + +Where an option's domain is fixed by the binary, the catalog says so: +`type` is `enum` and `values` is the complete list. Here's one option of the +[summary](reference-verbs.md#summary) verb, extracted from the catalog -- +using Miller to query Miller: + +GENMD-RUN-COMMAND +mlr help verb summary --as-json | mlr --json put -q 'emit $options[1]' +GENMD-EOF + +Where the domain depends on your *data* -- which fields exist, what values +`filter` could compare against, what to pass to `-g` -- the +[describe](reference-verbs.md#describe) verb profiles the input in one pass: +per field, the types seen, counts, cardinality, min/max, and (for +low-cardinality fields) every distinct value: + +GENMD-RUN-COMMAND +mlr --icsv --ojson describe then head -n 2 example.csv +GENMD-EOF + +The catalog is the *tool's* shape; `describe` is the *data's* shape. An +agent that consults both has nothing left to guess. + +## Validate: check DSL before spending a run + +This lets the agent parse and type-check a DSL expression before reading any input files. + +It's implemented by `mlr put --explain` and `mlr filter --explain`. + +`mlr put --explain` (likewise `mlr filter --explain`) parses and type-checks +an expression, then exits -- without opening any input at all: + +GENMD-RUN-COMMAND +mlr put --explain '$z = $x + $y' +GENMD-EOF + +GENMD-RUN-COMMAND-TOLERATING-ERROR +mlr put --explain '$z = $x +' +GENMD-EOF + +## Run and recover: errors as data + +Agents are instructed to run Miller commands using `mlr` with the `--errors-json` flag so that a +failure comes back as a structured document instead of prose. + +With `--errors-json` (or `MLR_ERRORS_JSON=true` environment variable), errors arrive as a structured +document. The `kind` field gives an agent something to branch on; `hint` is a runnable next step, +not a sentence; and `did_you_mean` is computed against the same catalog the agent discovered from, +closing the self-correction loop: + +GENMD-RUN-COMMAND-TOLERATING-ERROR +mlr --errors-json --icsv sorted -f shape example.csv +GENMD-EOF + +And since Miller's DSL includes [system and exec](shell-commands.md), there's a sandbox: +`--no-shell` (or a truthy `MLR_NO_SHELL` environment variable) disables all external-command +execution -- the DSL `system` and `exec` functions, piped redirects, and `--prepipe` fail cleanly: + +GENMD-RUN-COMMAND +mlr --no-shell -n put 'end{print system("hostname")}' +GENMD-EOF + +## Summary + +A typical agent profile sets all three environment variables once: + +GENMD-CARDIFY +export MLR_HELP_JSON=1 # help/catalog output as JSON +export MLR_ERRORS_JSON=1 # errors as structured JSON +export MLR_NO_SHELL=1 # no external-command execution +GENMD-EOF + +Put together, the sections above are a loop -- discover, constrain, validate, run -- where each step +feeds the next, and failures route back with structure instead of prose. diff --git a/docs/src/ai.md b/docs/src/ai.md index cb1cad2bb..20f218b21 100644 --- a/docs/src/ai.md +++ b/docs/src/ai.md @@ -16,231 +16,97 @@ Quick links: # Miller and AI -Miller treats AI agents as first-class users. When an agent drives a -command-line tool, it fails in predictable ways: it invents flags that don't -exist, guesses values that aren't in the data, misreads error prose, and -burns whole runs discovering a typo. Miller closes off each of those failure -modes with structure: +As of version 6.20, released in July 2026, Miller supports two ways to let agents know about it: +an **agent skill** and **MCP**. Either one works -- not sure which? Start with the Miller agent skill. -* Miller's entire surface -- verbs, DSL functions, flags, keywords -- is - available as **machine-readable JSON**, so agents ground themselves in what - actually exists. -* Options with fixed domains carry their **complete value sets**, and input - data can be **profiled in one pass** -- so agents copy real values instead - of inventing them. -* DSL expressions can be **validated before running**, without reading any - input. -* **Errors are structured** -- kind, hint, did-you-mean -- so agents branch - on data rather than parsing English. -* A **sandbox flag** removes external-command execution, so an - agent-constructed command line is just data processing. +This page covers essential setup, and an example session. For more on agent skills, see [The Miller +Agent Skill](agent-skill.md); for more on MCP, see [The Miller MCP server](mcp-server.md). -Everything on this page is an ordinary command-line feature: it works from -any agent harness, system prompt, or script -- and it's equally useful for -plain shell tooling like `jq`. The [MCP server](#plug-it-in-the-mcp-server) -at the end packages it all up for MCP-speaking agents. +## Quick start -## The essentials - -**To get the AI features:** install Miller 6.20 or newer ([Installing -Miller](installing-miller.md)). That's all. Everything on this page ships -inside the ordinary `mlr` binary -- there are no plugins, no separate +First, you need to **install Miller 6.20 or newer** (see [Installing Miller](installing-miller.md)). +Everything on this page ships inside the ordinary `mlr` binary -- there are no plugins, no separate installs, no API keys, and nothing here makes network calls. -**To get your AI to use them,** pick whichever matches your setup: - -* **If your agent speaks MCP** (Claude Code, Claude Desktop, Cursor, ...): - register the server -- for Claude Code that's `claude mcp add miller -- mlr - mcp` -- and you're done. The tools describe themselves, and the server - ships its own instructions and playbook, so you usually don't need to say - anything special; if the agent doesn't reach for them, a nudge like "use - the Miller tools" suffices. Details in [The MCP server](mcp-server.md). - -* **If your agent just runs shell commands** (a system prompt, a - `CLAUDE.md`, Cursor rules, a script harness): paste this standing - instruction into its context: - -
-Miller (mlr) is installed for processing CSV/TSV/JSON/etc. data. When -constructing mlr commands: -1. Discover: `mlr help --as-json --index` lists every verb/function/flag; - `mlr which "<intent>"` routes a goal to the right one; `mlr help - verb <name> --as-json` gives full details. Never invent flag or - function names. -2. Constrain: `mlr --icsv --ojson describe <file>` (or --ijson etc.) - shows the data's fields, types, and values. Copy names and values from it - rather than guessing them. -3. Validate: check DSL expressions with `mlr put --explain '<expr>'` - before using them. -4. Run with `--errors-json`; on failure, correct using the error's kind, - hint, and did_you_mean rather than re-guessing. -- - A fuller, ready-made version of that lesson ships in the Miller source - tree at - [pkg/terminals/mcp/SKILL.md](https://github.com/johnkerl/miller/blob/main/pkg/terminals/mcp/SKILL.md), - in Agent Skill format -- suitable for dropping into e.g. a - `.claude/skills/miller/` directory as-is. - -The rest of this page is what those instructions rest on, feature by -feature. - -## Discover: the machine-readable catalog - -`mlr help --as-json` emits Miller's entire help catalog as one JSON document. -The `--index` form is the cheap first call -- every capability with a -one-line summary (here trimmed, and then counted, using Miller itself): +To install as **skill file** for Claude:
-mlr help --as-json --index | mlr --json head -n 2 +mlr skill install ~/.claude/skills/miller
-[
-{
- "kind": "verb",
- "name": "altkv",
- "summary": "Given fields with values of the form a,b,c,d,e,f emits a=b,c=d,e=f pairs."
-},
-{
- "kind": "verb",
- "name": "bar",
- "summary": "Replaces a numeric field with a number of asterisks, allowing for cheesy"
-}
-]
+Wrote /Users/kerl/.claude/skills/miller/SKILL.md
+For Codex or Gemini:
+
++mlr skill install ~/.agents/skills/miller ++ +If you prefer to use **MCP**: +
-mlr help --as-json --index | mlr --json count +claude mcp add miller -- mlr mcp
-[
-{
- "count": 661
-}
-]
+Added stdio MCP server miller with command: mlr mcp to local config
+File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller]
-From the index, an agent drills into full entries one at a time: `mlr help
-verb sort --as-json`, `mlr help function splitax --as-json`, `mlr help flag
---ifs --as-json`, `mlr help keyword ENV --as-json` -- each accepting one or
-more names. A verb entry carries a structured option list -- flag, argument
-placeholder, type -- alongside the familiar usage text:
++codex mcp add miller -- mlr mcp ++ +
+gemini mcp add miller mlr mcp ++ +## Before and after: a first session with the skill installed + +If you're new to Miller, or you've used Miller before but this is your first time on 6.20 or newer, +here's a worked example: install the skill, then watch what changes about talking to your AI +assistant. + +One thing to be clear on before the example: you never type `mlr` yourself in this section. You +type plain English to your agent, same as always. Every `mlr` command shown below is the agent's +*own* work -- what it runs on your behalf, in the background, to answer you. They're printed here +so you can see exactly what changes, not because you'd type them. + +### Before: an agent guessing at your data + +Say you're looking at [example.csv](example.csv) for the first time. You type this, and nothing +else, to your AI assistant: + +> **You:** In example.csv, show me the red rows. + +Without the skill, a reasonable-sounding guess for the DSL might be `$color == "Red"`. Here's that +guess, run exactly as the agent would run it, behind the scenes, on your machine: + +
+mlr --csv filter '$color == "Red"' example.csv ++ +Nothing comes back -- no error, no rows, no warning, exit code 0. And here's the trap: the agent +still owes you an answer, so it turns that silence into a sentence: + +> **Agent:** I checked example.csv and there aren't any rows where color is red. + +That's wrong -- there are four -- but it *reads* like a fact, because a wrong guess about *your +data*, unlike a wrong flag or function name, doesn't look like a failure on Miller's end. It looks +like an empty result, which could just as easily have been true. + +### After: an agent that checks first + +Same question, word for word, with the skill installed: + +> **You:** In example.csv, show me the red rows. + +The skill's playbook puts a step between your question and any guesswork: "constrain -- learn the +data before touching it." So before writing any comparison, the agent runs a `describe`, again +invisibly, on your behalf:
-mlr help verb decimate --as-json --
-[
- {
- "name": "decimate",
- "summary": "Passes through one of every n records, optionally by category.",
- "ignores_input": false,
- "options": [
- {
- "flag": "-b",
- "type": "bool",
- "desc": "Decimate by printing first of every n."
- },
- {
- "flag": "-e",
- "type": "bool",
- "desc": "Decimate by printing last of every n (default)."
- },
- {
- "flag": "-g",
- "arg": "{a,b,c}",
- "type": "csv-list",
- "desc": "Optional group-by-field names for decimate counts, e.g. a,b,c."
- },
- {
- "flag": "-n",
- "arg": "{n}",
- "type": "int",
- "desc": "Decimation factor (default 10)."
- }
- ],
- "usage_text": "Usage: mlr decimate [options]\nPasses through one of every n records, optionally by category.\nOptions:\n-b Decimate by printing first of every n.\n-e Decimate by printing last of every n (default).\n-g {a,b,c} Optional group-by-field names for decimate counts, e.g. a,b,c.\n-n {n} Decimation factor (default 10).\n-h|--help Show this message."
- }
-]
-
-
-Note that `usage_text` -- what `mlr decimate --help` prints -- is rendered
-*from* the same structured options, so the human help and the machine help
-cannot drift apart. Function entries carry name, class, arity, help, and
-examples; the examples across the whole catalog are exercised by Miller's
-test suite, so they never rot.
-
-Three properties make the catalog cheap to use:
-
-* **It's a perfect cache key.** Every document carries `mlr_version` and
- `catalog_schema_version`. Miller is a static binary, so the catalog changes
- only when the binary does: fetch once, cache forever, re-fetch on a version
- bump. No TTLs.
-* **It's deterministic.** One document per invocation, sorted entries, no
- colorization -- stable for diffing and for prompt caches.
-* **It's opt-in twice over.** Per-call via `--as-json`, or set-once via a
- truthy `MLR_HELP_JSON` environment variable.
-
-For routing an *intent* to a capability -- the reverse of browsing -- `mlr
-which` returns ranked candidates:
-
--mlr which "join two files on a key" | mlr --json head -n 2 --
-[
-{
- "kind": "verb",
- "name": "join",
- "score": 25,
- "summary": "Joins records from specified left file name with records from all file names"
-},
-{
- "kind": "function",
- "name": "joink",
- "score": 25,
- "summary": "Makes string from map/array keys. First argument is map/array; second is separator string."
-}
-]
-
-
-Its exit code signals confidence -- 0 when a query word matched a
-capability's name, 2 when it didn't -- so a harness can branch on status
-without parsing anything.
-
-## Constrain: the tool's shape, and the data's shape
-
-Agents don't just hallucinate flags; they hallucinate *values*. Miller
-attacks that from both sides.
-
-Where an option's domain is fixed by the binary, the catalog says so:
-`type` is `enum` and `values` is the complete list. Here's one option of the
-[summary](reference-verbs.md#summary) verb, extracted from the catalog --
-using Miller to query Miller:
-
--mlr help verb summary --as-json | mlr --json put -q 'emit $options[1]' --
-[
-{
- "flag": "-a",
- "arg": "{mean,sum,etc.}",
- "type": "enum",
- "desc": "Use only the specified summarizers.",
- "values": ["field_type", "count", "null_count", "distinct_count", "mode", "sum", "mean", "stddev", "var", "skewness", "minlen", "maxlen", "min", "p25", "median", "p75", "max", "iqr", "lof", "lif", "uif", "uof"]
-}
-]
-
-
-Where the domain depends on your *data* -- which fields exist, what values
-`filter` could compare against, what to pass to `-g` -- the
-[describe](reference-verbs.md#describe) verb profiles the input in one pass:
-per field, the types seen, counts, cardinality, min/max, and (for
-low-cardinality fields) every distinct value:
-
--mlr --icsv --ojson describe then head -n 2 example.csv +mlr --icsv --ojson describe example.csv | mlr --json filter '$field_name == "color"'
[
@@ -255,105 +121,76 @@ low-cardinality fields) every distinct value:
"min": "purple",
"max": "yellow",
"values": ["yellow", "red", "purple"]
-},
-{
- "field_name": "shape",
- "types": {
- "string": 10
- },
- "count": 10,
- "null_count": 0,
- "distinct_count": 3,
- "min": "circle",
- "max": "triangle",
- "values": ["triangle", "square", "circle"]
}
]
-The catalog is the *tool's* shape; `describe` is the *data's* shape. An
-agent that consults both has nothing left to guess.
-
-## Validate: check DSL before spending a run
-
-`mlr put --explain` (likewise `mlr filter --explain`) parses and type-checks
-an expression, then exits -- without opening any input at all:
+That's the real value set for `color`: lowercase, three values, no `"Red"` anywhere. The agent
+corrects its own comparison before running anything user-facing:
-mlr put --explain '$z = $x + $y' +mlr --icsv --opprint filter '$color == "red"' example.csv
-mlr put: DSL expression is valid. +color shape flag k index quantity rate +red square true 2 15 79.2778 0.0130 +red circle true 3 16 13.8103 2.9010 +red square false 4 48 77.5542 7.4670 +red square false 6 64 77.1991 9.5310-## Run and recover: errors as data +-- and only now replies to you: -With `--errors-json` (or a truthy `MLR_ERRORS_JSON` environment variable), -errors arrive as a structured document. The `kind` field gives an agent -something to branch on; `hint` is a runnable next step, not a sentence; and -`did_you_mean` is computed against the same catalog the agent discovered -from, closing the self-correction loop: +> **Agent:** Four rows have color = red: rows 2, 3, 4, and 6 (all square or circle, quantity +> ranging from about 13.8 to 79.3). -
-mlr --errors-json --icsv sortt -f shape example.csv --
-{
- "error": "mlr: verb \"sortt\" not found. Please use \"mlr -l\" for a list.",
- "kind": "unknown-verb",
- "token": "sortt",
- "hint": "Run 'mlr -l' for a list of verbs, or 'mlr help verb \u003cname\u003e' for details.",
- "did_you_mean": [
- "sort"
- ]
-}
-
+Same question, same data, same underlying `mlr` binary -- the only thing that changed is that the
+agent looked before it leapt, and you never saw the intermediate `describe` unless you asked to.
+That one habit, *check the data before writing a comparison*, is the skill in miniature; the rest
+of the playbook applies the same idea to verb and function names (discover), DSL syntax (validate),
+and error messages (recover) -- see [Miller AI internals](ai-support.md) for how each of those works.
-And since Miller's DSL includes [system and exec](shell-commands.md), there's
-a sandbox: `--no-shell` (or a truthy `MLR_NO_SHELL` environment variable)
-disables all external-command execution -- the DSL `system` and `exec`
-functions, piped redirects, and `--prepipe` fail cleanly:
+## Why AI support
-
-mlr --no-shell -n put 'end{print system("hostname")}'
-
--(error) -+Miller treats AI agents as first-class users. When an agent drives a command-line tool, the agent +can fail in predictable ways: it invents flags that don't exist, guesses values that aren't in the +data, misreads error prose, and burns whole runs discovering a typo. Miller closes off each of those +failure modes with the following structure: -A typical agent profile sets all three environment variables once: +* Miller's entire surface -- verbs, DSL functions, flags, keywords -- is + available as **machine-readable JSON**, so agents ground themselves in what + actually exists. +* Options with fixed domains carry their **complete value sets**, and input + data can be **profiled in one pass**, so that agents copy real values instead + of inventing them. +* DSL expressions can be **validated before running**, without reading any + input. +* **Errors are structured** -- kind, hint, did-you-mean -- so agents branch + on data rather than parsing English. +* A **sandbox flag** removes external-command execution, so an + agent-constructed command line is just data processing. -
-export MLR_HELP_JSON=1 # help/catalog output as JSON -export MLR_ERRORS_JSON=1 # errors as structured JSON -export MLR_NO_SHELL=1 # no external-command execution -+Every one of those is an ordinary command-line feature, documented in [Miller AI +internals](ai-support.md): each works from any agent harness, system prompt, or script. -Put together, the sections above are a loop -- discover, constrain, -validate, run -- where each step feeds the next and failures route back with -structure instead of prose. +## Skill file or MCP: which should you use? -## Plug it in: the MCP server +For day one, the short version: start with the skill; add MCP later if you want it. They aren't +exclusive; nothing stops you running both. -If your agent speaks the [Model Context -Protocol](https://modelcontextprotocol.io) -- Claude Code, Claude Desktop, -Cursor, and many others -- everything above is one line away. For Claude -Code: +**Miller agent skill file:** -
-claude mcp add miller -- mlr mcp -+- Plus: One command, one static file -- no process, no client registration, nothing to reconnect. +- Plus: Works with any agent that reads Agent Skills from disk, not just MCP clients. +- Minus: No enforcement: it's advisory text, so no automatic `--no-shell` sandbox, no output caps or timeouts. +- Minus: The agent parses plain `mlr` text output and exit codes itself -- no structured JSON per call. -That's the whole setup. The server's five tools are exactly the features on -this page -- `list_capabilities` and `which` for discovery, `describe_data` -to constrain, `validate_dsl` to validate, and `run` (sandboxed with -`--no-shell` by default) to execute -- plus a shipped playbook, as MCP prompt -and resource, teaching the agent the loop. Then just talk to your agent -about your data: +**Miller MCP server:** -* "Which fields in `data.csv` have missing values?" -* "Convert this CSV to JSON, keeping only rows where status is active." -* "Join `a.csv` and `b.csv` on id, and give me the mean rate per group." +- Plus: Structured typed calls in, structured JSON back -- no text-parsing on the agent's side. +- Plus: Sandboxed by default (`MLR_NO_SHELL=1`), output-capped, timeout-guarded. +- Minus: One more moving part: per-client registration, plus a subprocess to spawn and reconnect each session. +- Minus: Only helps agents that actually speak MCP. -See [The MCP server](mcp-server.md) for the full tool reference and server -options. +In one line: the skill is less setup and the most portable, with weaker guarantees; MCP is a bit +more setup, with stronger guarantees, for a narrower set of clients. diff --git a/docs/src/ai.md.in b/docs/src/ai.md.in index 747f95602..5b50d89f0 100644 --- a/docs/src/ai.md.in +++ b/docs/src/ai.md.in @@ -1,16 +1,124 @@ # Miller and AI -Miller treats AI agents as first-class users. When an agent drives a -command-line tool, it fails in predictable ways: it invents flags that don't -exist, guesses values that aren't in the data, misreads error prose, and -burns whole runs discovering a typo. Miller closes off each of those failure -modes with structure: +As of version 6.20, released in July 2026, Miller supports two ways to let agents know about it: +an **agent skill** and **MCP**. Either one works -- not sure which? Start with the Miller agent skill. + +This page covers essential setup, and an example session. For more on agent skills, see [The Miller +Agent Skill](agent-skill.md); for more on MCP, see [The Miller MCP server](mcp-server.md). + +## Quick start + +First, you need to **install Miller 6.20 or newer** (see [Installing Miller](installing-miller.md)). +Everything on this page ships inside the ordinary `mlr` binary -- there are no plugins, no separate +installs, no API keys, and nothing here makes network calls. + +To install as **skill file** for Claude: + +GENMD-CARDIFY-HIGHLIGHT-ONE +mlr skill install ~/.claude/skills/miller +Wrote /Users/kerl/.claude/skills/miller/SKILL.md +GENMD-EOF + +For Codex or Gemini: + +GENMD-CARDIFY-HIGHLIGHT-ONE +mlr skill install ~/.agents/skills/miller +GENMD-EOF + +If you prefer to use **MCP**: + +GENMD-CARDIFY-HIGHLIGHT-ONE +claude mcp add miller -- mlr mcp +Added stdio MCP server miller with command: mlr mcp to local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] +GENMD-EOF + +GENMD-CARDIFY-HIGHLIGHT-ONE +codex mcp add miller -- mlr mcp +GENMD-EOF + +GENMD-CARDIFY-HIGHLIGHT-ONE +gemini mcp add miller mlr mcp +GENMD-EOF + +## Before and after: a first session with the skill installed + +If you're new to Miller, or you've used Miller before but this is your first time on 6.20 or newer, +here's a worked example: install the skill, then watch what changes about talking to your AI +assistant. + +One thing to be clear on before the example: you never type `mlr` yourself in this section. You +type plain English to your agent, same as always. Every `mlr` command shown below is the agent's +*own* work -- what it runs on your behalf, in the background, to answer you. They're printed here +so you can see exactly what changes, not because you'd type them. + +### Before: an agent guessing at your data + +Say you're looking at [example.csv](example.csv) for the first time. You type this, and nothing +else, to your AI assistant: + +> **You:** In example.csv, show me the red rows. + +Without the skill, a reasonable-sounding guess for the DSL might be `$color == "Red"`. Here's that +guess, run exactly as the agent would run it, behind the scenes, on your machine: + +GENMD-RUN-COMMAND +mlr --csv filter '$color == "Red"' example.csv +GENMD-EOF + +Nothing comes back -- no error, no rows, no warning, exit code 0. And here's the trap: the agent +still owes you an answer, so it turns that silence into a sentence: + +> **Agent:** I checked example.csv and there aren't any rows where color is red. + +That's wrong -- there are four -- but it *reads* like a fact, because a wrong guess about *your +data*, unlike a wrong flag or function name, doesn't look like a failure on Miller's end. It looks +like an empty result, which could just as easily have been true. + +### After: an agent that checks first + +Same question, word for word, with the skill installed: + +> **You:** In example.csv, show me the red rows. + +The skill's playbook puts a step between your question and any guesswork: "constrain -- learn the +data before touching it." So before writing any comparison, the agent runs a `describe`, again +invisibly, on your behalf: + +GENMD-RUN-COMMAND +mlr --icsv --ojson describe example.csv | mlr --json filter '$field_name == "color"' +GENMD-EOF + +That's the real value set for `color`: lowercase, three values, no `"Red"` anywhere. The agent +corrects its own comparison before running anything user-facing: + +GENMD-RUN-COMMAND +mlr --icsv --opprint filter '$color == "red"' example.csv +GENMD-EOF + +-- and only now replies to you: + +> **Agent:** Four rows have color = red: rows 2, 3, 4, and 6 (all square or circle, quantity +> ranging from about 13.8 to 79.3). + +Same question, same data, same underlying `mlr` binary -- the only thing that changed is that the +agent looked before it leapt, and you never saw the intermediate `describe` unless you asked to. +That one habit, *check the data before writing a comparison*, is the skill in miniature; the rest +of the playbook applies the same idea to verb and function names (discover), DSL syntax (validate), +and error messages (recover) -- see [Miller AI internals](ai-support.md) for how each of those works. + +## Why AI support + +Miller treats AI agents as first-class users. When an agent drives a command-line tool, the agent +can fail in predictable ways: it invents flags that don't exist, guesses values that aren't in the +data, misreads error prose, and burns whole runs discovering a typo. Miller closes off each of those +failure modes with the following structure: * Miller's entire surface -- verbs, DSL functions, flags, keywords -- is available as **machine-readable JSON**, so agents ground themselves in what actually exists. * Options with fixed domains carry their **complete value sets**, and input - data can be **profiled in one pass** -- so agents copy real values instead + data can be **profiled in one pass**, so that agents copy real values instead of inventing them. * DSL expressions can be **validated before running**, without reading any input. @@ -19,198 +127,27 @@ modes with structure: * A **sandbox flag** removes external-command execution, so an agent-constructed command line is just data processing. -Everything on this page is an ordinary command-line feature: it works from -any agent harness, system prompt, or script -- and it's equally useful for -plain shell tooling like `jq`. The [MCP server](#plug-it-in-the-mcp-server) -at the end packages it all up for MCP-speaking agents. +Every one of those is an ordinary command-line feature, documented in [Miller AI +internals](ai-support.md): each works from any agent harness, system prompt, or script. -## The essentials +## Skill file or MCP: which should you use? -**To get the AI features:** install Miller 6.20 or newer ([Installing -Miller](installing-miller.md)). That's all. Everything on this page ships -inside the ordinary `mlr` binary -- there are no plugins, no separate -installs, no API keys, and nothing here makes network calls. +For day one, the short version: start with the skill; add MCP later if you want it. They aren't +exclusive; nothing stops you running both. -**To get your AI to use them,** pick whichever matches your setup: +**Miller agent skill file:** -* **If your agent speaks MCP** (Claude Code, Claude Desktop, Cursor, ...): - register the server -- for Claude Code that's `claude mcp add miller -- mlr - mcp` -- and you're done. The tools describe themselves, and the server - ships its own instructions and playbook, so you usually don't need to say - anything special; if the agent doesn't reach for them, a nudge like "use - the Miller tools" suffices. Details in [The MCP server](mcp-server.md). +- Plus: One command, one static file -- no process, no client registration, nothing to reconnect. +- Plus: Works with any agent that reads Agent Skills from disk, not just MCP clients. +- Minus: No enforcement: it's advisory text, so no automatic `--no-shell` sandbox, no output caps or timeouts. +- Minus: The agent parses plain `mlr` text output and exit codes itself -- no structured JSON per call. -* **If your agent just runs shell commands** (a system prompt, a - `CLAUDE.md`, Cursor rules, a script harness): paste this standing - instruction into its context: +**Miller MCP server:** -GENMD-CARDIFY -Miller (mlr) is installed for processing CSV/TSV/JSON/etc. data. When -constructing mlr commands: -1. Discover: `mlr help --as-json --index` lists every verb/function/flag; - `mlr which "<intent>"` routes a goal to the right one; `mlr help - verb <name> --as-json` gives full details. Never invent flag or - function names. -2. Constrain: `mlr --icsv --ojson describe <file>` (or --ijson etc.) - shows the data's fields, types, and values. Copy names and values from it - rather than guessing them. -3. Validate: check DSL expressions with `mlr put --explain '<expr>'` - before using them. -4. Run with `--errors-json`; on failure, correct using the error's kind, - hint, and did_you_mean rather than re-guessing. -GENMD-EOF +- Plus: Structured typed calls in, structured JSON back -- no text-parsing on the agent's side. +- Plus: Sandboxed by default (`MLR_NO_SHELL=1`), output-capped, timeout-guarded. +- Minus: One more moving part: per-client registration, plus a subprocess to spawn and reconnect each session. +- Minus: Only helps agents that actually speak MCP. - A fuller, ready-made version of that lesson ships in the Miller source - tree at - [pkg/terminals/mcp/SKILL.md](https://github.com/johnkerl/miller/blob/main/pkg/terminals/mcp/SKILL.md), - in Agent Skill format -- suitable for dropping into e.g. a - `.claude/skills/miller/` directory as-is. - -The rest of this page is what those instructions rest on, feature by -feature. - -## Discover: the machine-readable catalog - -`mlr help --as-json` emits Miller's entire help catalog as one JSON document. -The `--index` form is the cheap first call -- every capability with a -one-line summary (here trimmed, and then counted, using Miller itself): - -GENMD-RUN-COMMAND -mlr help --as-json --index | mlr --json head -n 2 -GENMD-EOF - -GENMD-RUN-COMMAND -mlr help --as-json --index | mlr --json count -GENMD-EOF - -From the index, an agent drills into full entries one at a time: `mlr help -verb sort --as-json`, `mlr help function splitax --as-json`, `mlr help flag ---ifs --as-json`, `mlr help keyword ENV --as-json` -- each accepting one or -more names. A verb entry carries a structured option list -- flag, argument -placeholder, type -- alongside the familiar usage text: - -GENMD-RUN-COMMAND -mlr help verb decimate --as-json -GENMD-EOF - -Note that `usage_text` -- what `mlr decimate --help` prints -- is rendered -*from* the same structured options, so the human help and the machine help -cannot drift apart. Function entries carry name, class, arity, help, and -examples; the examples across the whole catalog are exercised by Miller's -test suite, so they never rot. - -Three properties make the catalog cheap to use: - -* **It's a perfect cache key.** Every document carries `mlr_version` and - `catalog_schema_version`. Miller is a static binary, so the catalog changes - only when the binary does: fetch once, cache forever, re-fetch on a version - bump. No TTLs. -* **It's deterministic.** One document per invocation, sorted entries, no - colorization -- stable for diffing and for prompt caches. -* **It's opt-in twice over.** Per-call via `--as-json`, or set-once via a - truthy `MLR_HELP_JSON` environment variable. - -For routing an *intent* to a capability -- the reverse of browsing -- `mlr -which` returns ranked candidates: - -GENMD-RUN-COMMAND -mlr which "join two files on a key" | mlr --json head -n 2 -GENMD-EOF - -Its exit code signals confidence -- 0 when a query word matched a -capability's name, 2 when it didn't -- so a harness can branch on status -without parsing anything. - -## Constrain: the tool's shape, and the data's shape - -Agents don't just hallucinate flags; they hallucinate *values*. Miller -attacks that from both sides. - -Where an option's domain is fixed by the binary, the catalog says so: -`type` is `enum` and `values` is the complete list. Here's one option of the -[summary](reference-verbs.md#summary) verb, extracted from the catalog -- -using Miller to query Miller: - -GENMD-RUN-COMMAND -mlr help verb summary --as-json | mlr --json put -q 'emit $options[1]' -GENMD-EOF - -Where the domain depends on your *data* -- which fields exist, what values -`filter` could compare against, what to pass to `-g` -- the -[describe](reference-verbs.md#describe) verb profiles the input in one pass: -per field, the types seen, counts, cardinality, min/max, and (for -low-cardinality fields) every distinct value: - -GENMD-RUN-COMMAND -mlr --icsv --ojson describe then head -n 2 example.csv -GENMD-EOF - -The catalog is the *tool's* shape; `describe` is the *data's* shape. An -agent that consults both has nothing left to guess. - -## Validate: check DSL before spending a run - -`mlr put --explain` (likewise `mlr filter --explain`) parses and type-checks -an expression, then exits -- without opening any input at all: - -GENMD-RUN-COMMAND -mlr put --explain '$z = $x + $y' -GENMD-EOF - -## Run and recover: errors as data - -With `--errors-json` (or a truthy `MLR_ERRORS_JSON` environment variable), -errors arrive as a structured document. The `kind` field gives an agent -something to branch on; `hint` is a runnable next step, not a sentence; and -`did_you_mean` is computed against the same catalog the agent discovered -from, closing the self-correction loop: - -GENMD-RUN-COMMAND-TOLERATING-ERROR -mlr --errors-json --icsv sortt -f shape example.csv -GENMD-EOF - -And since Miller's DSL includes [system and exec](shell-commands.md), there's -a sandbox: `--no-shell` (or a truthy `MLR_NO_SHELL` environment variable) -disables all external-command execution -- the DSL `system` and `exec` -functions, piped redirects, and `--prepipe` fail cleanly: - -GENMD-RUN-COMMAND -mlr --no-shell -n put 'end{print system("hostname")}' -GENMD-EOF - -A typical agent profile sets all three environment variables once: - -GENMD-CARDIFY -export MLR_HELP_JSON=1 # help/catalog output as JSON -export MLR_ERRORS_JSON=1 # errors as structured JSON -export MLR_NO_SHELL=1 # no external-command execution -GENMD-EOF - -Put together, the sections above are a loop -- discover, constrain, -validate, run -- where each step feeds the next and failures route back with -structure instead of prose. - -## Plug it in: the MCP server - -If your agent speaks the [Model Context -Protocol](https://modelcontextprotocol.io) -- Claude Code, Claude Desktop, -Cursor, and many others -- everything above is one line away. For Claude -Code: - -
-claude mcp add miller -- mlr mcp -- -That's the whole setup. The server's five tools are exactly the features on -this page -- `list_capabilities` and `which` for discovery, `describe_data` -to constrain, `validate_dsl` to validate, and `run` (sandboxed with -`--no-shell` by default) to execute -- plus a shipped playbook, as MCP prompt -and resource, teaching the agent the loop. Then just talk to your agent -about your data: - -* "Which fields in `data.csv` have missing values?" -* "Convert this CSV to JSON, keeping only rows where status is active." -* "Join `a.csv` and `b.csv` on id, and give me the mean rate per group." - -See [The MCP server](mcp-server.md) for the full tool reference and server -options. +In one line: the skill is less setup and the most portable, with weaker guarantees; MCP is a bit +more setup, with stronger guarantees, for a narrower set of clients. diff --git a/docs/src/installing-miller.md b/docs/src/installing-miller.md index 9de4558ff..ebb6cf58b 100644 --- a/docs/src/installing-miller.md +++ b/docs/src/installing-miller.md @@ -46,7 +46,7 @@ As a first check, you should be able to run `mlr --version` at your system's com mlr --version
-mlr 6.0.0 +mlr 6.20.1A note on documentation: diff --git a/docs/src/installing-miller.md.in b/docs/src/installing-miller.md.in index 74e5c9f53..bf05d945d 100644 --- a/docs/src/installing-miller.md.in +++ b/docs/src/installing-miller.md.in @@ -26,9 +26,8 @@ Note that the [Miller releases page](https://github.com/johnkerl/miller/releases As a first check, you should be able to run `mlr --version` at your system's command prompt and see something like the following: -GENMD-CARDIFY-HIGHLIGHT-ONE +GENMD-RUN-COMMAND mlr --version -mlr 6.0.0 GENMD-EOF A note on documentation: diff --git a/docs/src/mcp-server.md b/docs/src/mcp-server.md index e37828a09..ce496d5c2 100644 --- a/docs/src/mcp-server.md +++ b/docs/src/mcp-server.md @@ -14,22 +14,69 @@ Quick links: Release docs -# The MCP server +# The Miller MCP server -Miller ships with a built-in [Model Context Protocol](https://modelcontextprotocol.io) -server, so AI agents (Claude Code, Claude Desktop, Cursor, and other MCP -clients) can discover and drive Miller without scraping help text or guessing -at flags. (For the overview of Miller's whole AI feature set -- with or -without MCP -- see [Miller and AI](ai.md).) +As of Miller version 6.20, released in July 2026, there are two main ways to get your AI to know +about a software tool (Miller, or others): [**agent skills**](agent-skill.md), and **MCP**. (See +[Miller and AI](ai.md) for an introduction.) -The server speaks JSON-RPC over stdin/stdout (MCP's "stdio" transport): the -MCP client spawns `mlr mcp` as a subprocess. No network port is opened, and -the server exits when the client disconnects. Example registration, for -Claude Code: +Miller ships with a built-in [Model Context Protocol](https://modelcontextprotocol.io) server +included within the `mlr` executable, so AI agents (Claude Code, Claude Desktop, Cursor, and other +MCP clients) can discover and drive Miller without scraping help text or guessing at flags. + +The server speaks JSON-RPC over stdin/stdout (MCP's **stdio** transport): the MCP client spawns `mlr +mcp` as a subprocess. No network port is opened, and the server exits when the client disconnects. + +## Setup + +Example registration for some common CLI agents (do this before starting your session):
claude mcp add miller -- mlr mcp+
+Added stdio MCP server miller with command: mlr mcp to local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] ++ +
+codex mcp add miller -- mlr mcp ++ +
+gemini mcp add miller mlr mcp ++ +You can undo that as follows: + +
+claude mcp remove miller ++
+Removed MCP server "miller" from local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] ++ +
+codex mcp remove miller ++ +
+gemini mcp remove miller ++ +Then -- just interact with your agent as always! When you say something like `describe the data file example.csv`, +the agent will already know how to use Miller to help answer that question. + + + +For more background on the `mlr` commands the agent runs on your behalf, please see +[Miller AI internals](ai-support.md). + +## What the Miller MCP tools map to + +As shown below, you don't have to type `mcp` in your agent sessions: rather you've empowered the +agent to discover things about Miller. But if you're curious what the AI agent will see:
mlr mcp --help @@ -68,25 +115,68 @@ Options: -h or --help Show this message.-## What the tools map to +Each MCP tool is a thin wrapper over a Miller feature you can also, if you like, use directly from +the command line: -Each MCP tool is a thin wrapper over a Miller feature you can also use -directly from the command line: - -* `list_capabilities` is [`mlr help --as-json`](online-help.md) -- the - machine-readable catalog of verbs, DSL functions, flags, and keywords. -* `which` is `mlr which` -- natural-language intent to ranked capabilities. -* `validate_dsl` is `mlr put --explain` / `mlr filter --explain` -- parse and - type-check a DSL expression without reading any input. -* `describe_data` is [`mlr describe`](reference-verbs.md#describe) -- field +- `list_capabilities` is [`mlr help --as-json`](online-help.md): the + machine-readable catalog of [verbs](reference-verbs.md), + [DSL functions](reference-dsl-builtin-functions.md), [flags](reference-main-flag-list.md), and + [keywords](reference-dsl-variables.md#keywords-for-filter-and-put). +- `which` is `mlr which`: turns natural-language intent into ranked capabilities. +- `validate_dsl` is `mlr put --explain` / `mlr filter --explain`: to parse and + type-check a DSL expression before reading any input files. +- `describe_data` is [`mlr describe`](reference-verbs.md#describe): this shows field names, types, cardinality, and value domains for input data. -* `run` executes an `mlr` command line and reports exit code, output, and -- +- `run` executes an `mlr` command line and reports exit code, output, and -- on failure -- the structured error document from `mlr --errors-json`. -The catalog tools are answered in-process; the others run this same `mlr` -binary as a subprocess, so agents see exactly what a terminal user sees. +See also the [Miller AI internals page](ai-support.md) for more information. -## Sandboxing: --no-shell +## What Miller MCP looks like in practice + +Here are some screenshots from a Claude Code session. + +At the shell, before starting `claude`, we've first run + +
+claude mcp add miller -- mlr mcp ++
+Added stdio MCP server miller with command: mlr mcp to local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] ++ +Then, inside Claude code, we type `/mcp`: + + + +Then we select Miller: + + + +The status shows it's installed. Note that there is no long-running Miller "server" process: this is +just Claude remembering to run things like `mlr mcp ...` in order to get how-to instructions from +the `mlr` executable you already have installed. + + + +The MCP tools are names for Claude to remember -- you don't have to. For transparency, though, here they are: + + + +Here are descriptions of a couple of them: + + + + + +When you're in your AI session, you don't have to type `mcp` or the specific names of Miller MCP tools. +Rather, you just interact as always, and the AI remembers to call Miller MCP tools on your behalf. +For example: + + + +## A note on sandboxing Miller's DSL includes [`system` and `exec`](shell-commands.md), and `--prepipe`/piped redirects also run external commands. So that an @@ -95,19 +185,7 @@ subprocesses started by the MCP server run with `MLR_NO_SHELL=1`: those features fail cleanly instead of executing. Start the server with `mlr mcp --allow-shell` to turn that off. -The same gate is available outside the MCP server: pass `--no-shell` to any -`mlr` invocation, or set the `MLR_NO_SHELL` environment variable to a truthy -value. Note that Miller can still write files when asked to (`tee`, `split`, -DSL output redirection) -- the gate is specifically about executing external -commands. - -## The agent playbook - -The server also exposes a playbook -- as MCP prompt `miller-playbook` and MCP -resource `miller://playbook` -- encoding the loop that makes an agent -effective with Miller: **discover** capabilities from the catalog rather than -inventing them, **constrain** to the data's actual fields and values via -`describe_data`, **validate** DSL before running it, and **run** with -structured-error recovery. The same text lives in the Miller source tree at -[pkg/terminals/mcp/SKILL.md](https://github.com/johnkerl/miller/blob/main/pkg/terminals/mcp/SKILL.md) -in Agent Skill format. +The same gate is available outside the MCP server: pass `--no-shell` to any `mlr` invocation, or set +the `MLR_NO_SHELL` [environment variable](reference-main-env-vars.md) to `true`. Note that Miller +can still write files when asked to (`tee`, `split`, DSL output redirection): the gate is +specifically about executing external commands. diff --git a/docs/src/mcp-server.md.in b/docs/src/mcp-server.md.in index 8fe3b81c8..8c04fb1e2 100644 --- a/docs/src/mcp-server.md.in +++ b/docs/src/mcp-server.md.in @@ -1,43 +1,127 @@ -# The MCP server +# The Miller MCP server -Miller ships with a built-in [Model Context Protocol](https://modelcontextprotocol.io) -server, so AI agents (Claude Code, Claude Desktop, Cursor, and other MCP -clients) can discover and drive Miller without scraping help text or guessing -at flags. (For the overview of Miller's whole AI feature set -- with or -without MCP -- see [Miller and AI](ai.md).) +As of Miller version 6.20, released in July 2026, there are two main ways to get your AI to know +about a software tool (Miller, or others): [**agent skills**](agent-skill.md), and **MCP**. (See +[Miller and AI](ai.md) for an introduction.) -The server speaks JSON-RPC over stdin/stdout (MCP's "stdio" transport): the -MCP client spawns `mlr mcp` as a subprocess. No network port is opened, and -the server exits when the client disconnects. Example registration, for -Claude Code: +Miller ships with a built-in [Model Context Protocol](https://modelcontextprotocol.io) server +included within the `mlr` executable, so AI agents (Claude Code, Claude Desktop, Cursor, and other +MCP clients) can discover and drive Miller without scraping help text or guessing at flags. -
-claude mcp add miller -- mlr mcp -+The server speaks JSON-RPC over stdin/stdout (MCP's **stdio** transport): the MCP client spawns `mlr +mcp` as a subprocess. No network port is opened, and the server exits when the client disconnects. + +## Setup + +Example registration for some common CLI agents (do this before starting your session): + +GENMD-CARDIFY-HIGHLIGHT-ONE +claude mcp add miller -- mlr mcp +Added stdio MCP server miller with command: mlr mcp to local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] +GENMD-EOF + +GENMD-CARDIFY-HIGHLIGHT-ONE +codex mcp add miller -- mlr mcp +GENMD-EOF + +GENMD-CARDIFY-HIGHLIGHT-ONE +gemini mcp add miller mlr mcp +GENMD-EOF + +You can undo that as follows: + +GENMD-CARDIFY-HIGHLIGHT-ONE +claude mcp remove miller +Removed MCP server "miller" from local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] +GENMD-EOF + +GENMD-CARDIFY-HIGHLIGHT-ONE +codex mcp remove miller +GENMD-EOF + +GENMD-CARDIFY-HIGHLIGHT-ONE +gemini mcp remove miller +GENMD-EOF + +Then -- just interact with your agent as always! When you say something like `describe the data file example.csv`, +the agent will already know how to use Miller to help answer that question. + + + +For more background on the `mlr` commands the agent runs on your behalf, please see +[Miller AI internals](ai-support.md). + +## What the Miller MCP tools map to + +As shown below, you don't have to type `mcp` in your agent sessions: rather you've empowered the +agent to discover things about Miller. But if you're curious what the AI agent will see: GENMD-RUN-COMMAND mlr mcp --help GENMD-EOF -## What the tools map to +Each MCP tool is a thin wrapper over a Miller feature you can also, if you like, use directly from +the command line: -Each MCP tool is a thin wrapper over a Miller feature you can also use -directly from the command line: - -* `list_capabilities` is [`mlr help --as-json`](online-help.md) -- the - machine-readable catalog of verbs, DSL functions, flags, and keywords. -* `which` is `mlr which` -- natural-language intent to ranked capabilities. -* `validate_dsl` is `mlr put --explain` / `mlr filter --explain` -- parse and - type-check a DSL expression without reading any input. -* `describe_data` is [`mlr describe`](reference-verbs.md#describe) -- field +- `list_capabilities` is [`mlr help --as-json`](online-help.md): the + machine-readable catalog of [verbs](reference-verbs.md), + [DSL functions](reference-dsl-builtin-functions.md), [flags](reference-main-flag-list.md), and + [keywords](reference-dsl-variables.md#keywords-for-filter-and-put). +- `which` is `mlr which`: turns natural-language intent into ranked capabilities. +- `validate_dsl` is `mlr put --explain` / `mlr filter --explain`: to parse and + type-check a DSL expression before reading any input files. +- `describe_data` is [`mlr describe`](reference-verbs.md#describe): this shows field names, types, cardinality, and value domains for input data. -* `run` executes an `mlr` command line and reports exit code, output, and -- +- `run` executes an `mlr` command line and reports exit code, output, and -- on failure -- the structured error document from `mlr --errors-json`. -The catalog tools are answered in-process; the others run this same `mlr` -binary as a subprocess, so agents see exactly what a terminal user sees. +See also the [Miller AI internals page](ai-support.md) for more information. -## Sandboxing: --no-shell +## What Miller MCP looks like in practice + +Here are some screenshots from a Claude Code session. + +At the shell, before starting `claude`, we've first run + +GENMD-CARDIFY-HIGHLIGHT-ONE +claude mcp add miller -- mlr mcp +Added stdio MCP server miller with command: mlr mcp to local config +File modified: /Users/kerl/.claude.json [project: /Users/kerl/git/johnkerl/miller] +GENMD-EOF + +Then, inside Claude code, we type `/mcp`: + + + +Then we select Miller: + + + +The status shows it's installed. Note that there is no long-running Miller "server" process: this is +just Claude remembering to run things like `mlr mcp ...` in order to get how-to instructions from +the `mlr` executable you already have installed. + + + +The MCP tools are names for Claude to remember -- you don't have to. For transparency, though, here they are: + + + +Here are descriptions of a couple of them: + + + + + +When you're in your AI session, you don't have to type `mcp` or the specific names of Miller MCP tools. +Rather, you just interact as always, and the AI remembers to call Miller MCP tools on your behalf. +For example: + + + +## A note on sandboxing Miller's DSL includes [`system` and `exec`](shell-commands.md), and `--prepipe`/piped redirects also run external commands. So that an @@ -46,19 +130,7 @@ subprocesses started by the MCP server run with `MLR_NO_SHELL=1`: those features fail cleanly instead of executing. Start the server with `mlr mcp --allow-shell` to turn that off. -The same gate is available outside the MCP server: pass `--no-shell` to any -`mlr` invocation, or set the `MLR_NO_SHELL` environment variable to a truthy -value. Note that Miller can still write files when asked to (`tee`, `split`, -DSL output redirection) -- the gate is specifically about executing external -commands. - -## The agent playbook - -The server also exposes a playbook -- as MCP prompt `miller-playbook` and MCP -resource `miller://playbook` -- encoding the loop that makes an agent -effective with Miller: **discover** capabilities from the catalog rather than -inventing them, **constrain** to the data's actual fields and values via -`describe_data`, **validate** DSL before running it, and **run** with -structured-error recovery. The same text lives in the Miller source tree at -[pkg/terminals/mcp/SKILL.md](https://github.com/johnkerl/miller/blob/main/pkg/terminals/mcp/SKILL.md) -in Agent Skill format. +The same gate is available outside the MCP server: pass `--no-shell` to any `mlr` invocation, or set +the `MLR_NO_SHELL` [environment variable](reference-main-env-vars.md) to `true`. Note that Miller +can still write files when asked to (`tee`, `split`, DSL output redirection): the gate is +specifically about executing external commands. diff --git a/docs/src/pix/mcp-describe-data.png b/docs/src/pix/mcp-describe-data.png new file mode 100644 index 000000000..08245e79f Binary files /dev/null and b/docs/src/pix/mcp-describe-data.png differ diff --git a/docs/src/pix/mcp-describe.png b/docs/src/pix/mcp-describe.png new file mode 100644 index 000000000..1f1b3d0fb Binary files /dev/null and b/docs/src/pix/mcp-describe.png differ diff --git a/docs/src/pix/mcp-list-capabilities.png b/docs/src/pix/mcp-list-capabilities.png new file mode 100644 index 000000000..1baa56610 Binary files /dev/null and b/docs/src/pix/mcp-list-capabilities.png differ diff --git a/docs/src/pix/mcp-manage.png b/docs/src/pix/mcp-manage.png new file mode 100644 index 000000000..1150ba0a1 Binary files /dev/null and b/docs/src/pix/mcp-manage.png differ diff --git a/docs/src/pix/mcp-slash.png b/docs/src/pix/mcp-slash.png new file mode 100644 index 000000000..392cdba97 Binary files /dev/null and b/docs/src/pix/mcp-slash.png differ diff --git a/docs/src/pix/mcp-status.png b/docs/src/pix/mcp-status.png new file mode 100644 index 000000000..515bf46d8 Binary files /dev/null and b/docs/src/pix/mcp-status.png differ diff --git a/docs/src/pix/mcp-tools.png b/docs/src/pix/mcp-tools.png new file mode 100644 index 000000000..618776212 Binary files /dev/null and b/docs/src/pix/mcp-tools.png differ diff --git a/pkg/terminals/mcp/server.go b/pkg/terminals/mcp/server.go index 926f2a697..e5bdf4c7e 100644 --- a/pkg/terminals/mcp/server.go +++ b/pkg/terminals/mcp/server.go @@ -11,8 +11,13 @@ import ( mcpsdk "github.com/modelcontextprotocol/go-sdk/mcp" ) +// PlaybookText is the Miller Agent Skill / playbook content: the same text +// served here as the "miller-playbook" MCP prompt/resource, and reused +// as-is by `mlr skill` (pkg/terminals/skill) for agents that read Agent +// Skills from disk rather than over MCP. +// //go:embed SKILL.md -var playbookText string +var PlaybookText string const playbookPromptName = "miller-playbook" const playbookResourceURI = "miller://playbook" @@ -117,7 +122,7 @@ func playbookPromptHandler(_ context.Context, _ *mcpsdk.GetPromptRequest) (*mcps Messages: []*mcpsdk.PromptMessage{ { Role: "user", - Content: &mcpsdk.TextContent{Text: playbookText}, + Content: &mcpsdk.TextContent{Text: PlaybookText}, }, }, }, nil @@ -129,7 +134,7 @@ func playbookResourceHandler(_ context.Context, _ *mcpsdk.ReadResourceRequest) ( { URI: playbookResourceURI, MIMEType: "text/markdown", - Text: playbookText, + Text: PlaybookText, }, }, }, nil diff --git a/pkg/terminals/mcp/server_test.go b/pkg/terminals/mcp/server_test.go index 555eef149..fb7760fdc 100644 --- a/pkg/terminals/mcp/server_test.go +++ b/pkg/terminals/mcp/server_test.go @@ -315,6 +315,6 @@ func TestParseStructuredError(t *testing.T) { } func TestPlaybookHasFrontmatter(t *testing.T) { - assert.True(t, strings.HasPrefix(playbookText, "---\n")) - assert.Contains(t, playbookText, "name: miller") + assert.True(t, strings.HasPrefix(PlaybookText, "---\n")) + assert.Contains(t, PlaybookText, "name: miller") } diff --git a/pkg/terminals/registry/registry.go b/pkg/terminals/registry/registry.go index 41bbbcd9a..2e536cdee 100644 --- a/pkg/terminals/registry/registry.go +++ b/pkg/terminals/registry/registry.go @@ -19,6 +19,7 @@ const ( Regtest = "regtest" Repl = "repl" Script = "script" + Skill = "skill" Version = "version" Which = "which" ) @@ -32,6 +33,7 @@ var Names = []string{ Regtest, Repl, Script, + Skill, Version, Which, } diff --git a/pkg/terminals/skill/skill_main.go b/pkg/terminals/skill/skill_main.go new file mode 100644 index 000000000..a08b6717d --- /dev/null +++ b/pkg/terminals/skill/skill_main.go @@ -0,0 +1,92 @@ +// Entrypoint for `mlr skill`: puts the Miller Agent Skill (SKILL.md) where a +// coding agent can find it on disk, for tools that read Agent Skills +// directly rather than over MCP. +// +// The content is identical to what `mlr mcp` serves as its "miller-playbook" +// prompt/resource (pkg/terminals/mcp/SKILL.md, exported as mcp.PlaybookText) +// -- this is a second delivery path for the same text, not a second source +// of truth. + +package skill + +import ( + "fmt" + "os" + "path/filepath" + + "github.com/johnkerl/miller/v6/pkg/terminals/mcp" +) + +const defaultInstallDir = ".claude/skills/miller" + +func skillUsage(o *os.File) { + fmt.Fprintf(o, "Usage: mlr skill {print|install} [options]\n") + fmt.Fprintf(o, "Puts the Miller Agent Skill (SKILL.md) where a coding agent can find it.\n") + fmt.Fprintf(o, "This is the same playbook mlr mcp serves as its \"miller-playbook\"\n") + fmt.Fprintf(o, "prompt/resource, packaged for agents that read Agent Skills from disk.\n") + fmt.Fprintf(o, "\n") + fmt.Fprintf(o, "Subcommands:\n") + fmt.Fprintf(o, " print Write the skill content to stdout.\n") + fmt.Fprintf(o, " install [DIR] Write DIR/SKILL.md, creating DIR if needed.\n") + fmt.Fprintf(o, " Default DIR is %s\n", defaultInstallDir) + fmt.Fprintf(o, "\n") + fmt.Fprintf(o, " -h or --help Show this message.\n") +} + +// SkillMain is the entrypoint called by the terminals dispatcher for `mlr skill`. +func SkillMain(args []string) int { + args = args[1:] // strip "skill" + + if len(args) == 0 { + skillUsage(os.Stderr) + return 1 + } + + switch args[0] { + case "-h", "--help": + skillUsage(os.Stdout) + return 0 + case "print": + return printMain(args[1:]) + case "install": + return installMain(args[1:]) + default: + fmt.Fprintf(os.Stderr, "mlr skill: subcommand \"%s\" not recognized.\n", args[0]) + return 1 + } +} + +func printMain(args []string) int { + if len(args) != 0 { + fmt.Fprintf(os.Stderr, "mlr skill print: takes no arguments.\n") + return 1 + } + fmt.Print(mcp.PlaybookText) + return 0 +} + +func installMain(args []string) int { + dir := defaultInstallDir + switch len(args) { + case 0: + case 1: + dir = args[0] + default: + fmt.Fprintf(os.Stderr, "mlr skill install: takes at most one argument (the target directory).\n") + return 1 + } + + if err := os.MkdirAll(dir, 0o755); err != nil { + fmt.Fprintf(os.Stderr, "mlr skill install: could not create %s: %v\n", dir, err) + return 1 + } + + path := filepath.Join(dir, "SKILL.md") + if err := os.WriteFile(path, []byte(mcp.PlaybookText), 0o644); err != nil { + fmt.Fprintf(os.Stderr, "mlr skill install: could not write %s: %v\n", path, err) + return 1 + } + + fmt.Printf("Wrote %s\n", path) + return 0 +} diff --git a/pkg/terminals/terminals.go b/pkg/terminals/terminals.go index 40ca00525..0df047acc 100644 --- a/pkg/terminals/terminals.go +++ b/pkg/terminals/terminals.go @@ -15,6 +15,7 @@ import ( "github.com/johnkerl/miller/v6/pkg/terminals/regtest" "github.com/johnkerl/miller/v6/pkg/terminals/repl" "github.com/johnkerl/miller/v6/pkg/terminals/script" + "github.com/johnkerl/miller/v6/pkg/terminals/skill" "github.com/johnkerl/miller/v6/pkg/version" ) @@ -41,6 +42,7 @@ func init() { {registry.Regtest, regtest.RegTestMain}, {registry.Repl, repl.ReplMain}, {registry.Script, script.ScriptMain}, + {registry.Skill, skill.SkillMain}, {registry.Version, showVersion}, {registry.Which, help.WhichMain}, }