# Plan: nested-field ("JSON") accessors for non-DSL verbs

Motivating issues:

- [issue #1815](https://github.com/johnkerl/miller/issues/1815) — `mlr -j rename
  Body.meta,Body.renamed_meta` silently matches nothing when `Body` is a nested map. The
  workarounds are DSL (`put '$Body.renamed_meta = $Body.meta; unset $Body.meta'`) or the
  flatten-sandwich (`flatten then rename Body.meta,Body.renamed_meta then unflatten`). Thread
  consensus: worth documenting; the deeper ask is that `rename` (and friends) understand paths.
- [issue #1534](https://github.com/johnkerl/miller/issues/1534) — nominally about CSV
  schema-change errors, but architecturally on point: `group-like` judged record schemas by
  *nested* key lists (`name,location_1,field_1`) while the CSV writer saw *flattened* keys
  (`name,location_1.1.lat,...`), so "same schema" groups came out heterogeneous. It's a live
  example of verbs seeing different field names than users see in non-JSON output — and of
  dotted flat names (`location_1.1.lat`) being real, user-visible identifiers in CSV-land.
- [Flatten/unflatten docs](https://miller.readthedocs.io/en/latest/flatten-unflatten/) — the
  existing contract this feature must not break.

The asymmetry, stated once:

- In the put/filter DSL, `$x.y.z` traverses nested structures: on `{"x":{"y":{"z":4}}}` it
  yields 4.
- In every other verb, a field name is a single flat string: `mlr cut -f x.y.z` looks for a
  field literally named `x.y.z` and finds nothing in that record.

Scope of this doc: plan and sharp edges only — no code. The recommendation (§ Proposed
design) is native path accessors at the Mlrmap level, adopted by a small verb set first,
behind an explicit per-verb opt-in.

## Current architecture (survey)

### Mlrmap accessors are flat; indexed accessors already exist

- Every verb-facing accessor takes a single `string` key and funnels through
  `findEntry(key)` (`pkg/mlrval/mlrmap_accessors.go:199`): `Has` :17, `Get` :21,
  `PutCopy` :95, `Remove` :505, `Rename` :724, `MoveToHead`/`MoveToTail` :514/:522, plus the
  bulk group-by helpers `GetSelectedValuesJoined` :583 / `GetSelectedValuesAndJoined` :610 /
  `HasSelectedKeys` :678, all looping flat lookups.
- Nesting-capable primitives already exist, keyed by `[]*Mlrval` index chains rather than
  dotted strings: `PutIndexed` (`mlrmap_accessors.go:538` → `putIndexedOnMap`,
  `pkg/mlrval/mlrval_collections.go:266`, with auto-deepening), `RemoveIndexed`
  (:542 → `removeIndexedOnMap`, `mlrval_collections.go:402` — removes only the leaf,
  leaving the parent map in place, possibly empty), and `getWithMlrvalArrayIndex`
  (`mlrmap_accessors.go:371`, the `$x[["a","b"]]` walker). These are battle-tested by the
  DSL and are the machinery a verb-side feature should reuse.

### How the DSL does `$x.y.z`

- The dot form is parsed as nested `DotOperator` nodes; `DotCallsiteNode.Evaluate`
  (`pkg/dsl/cst/builtin_functions.go:591-610`) does one-level `Get` per dot when the LHS is
  a map, else falls back to string concatenation. So `$x.y.z` is `(($x).y).z` — strictly
  leftmost, one key per segment, never "try `x.y` as a single key".
- Bracket-form reads (`$x["y"]["z"]`) evaluate level-by-level through
  `ArrayOrMapIndexAccessNode` (`pkg/dsl/cst/collections.go:53`); lvalue assignments and
  unsets collect `[]*Mlrval` index chains and call `PutIndexed`/`RemoveIndexed`
  (`pkg/dsl/cst/lvalues.go:115, 447, 532`). Note there is no bulk `GetIndexed` — the
  closest read-side primitive is `GetWithMlrvalIndex` with an array-valued index
  (the `$x[["a","b"]]` form), which dispatches to the unexported
  `getWithMlrvalArrayIndex` walker.
- Crucially, the DSL already has a disambiguation *syntax*: `$x.y.z` means traversal,
  `${x.y.z}` means the literal flat name. The verbs have no equivalent syntax slot today —
  that's the heart of the design problem.

### How verbs consume field names

All flat, all inlined per verb; there is no shared field-name abstraction to hook:

- Parse: `cli.VerbGetStringArrayArg` turns `-f a,b,c` into `[]string`; ~40 verbs use it
  (cut, sort, having-fields, reorder, rename, stats1/2, top, uniq, count, fill-down,
  merge-fields, join, template, subs, case, ...).
- Lookup: cut tests `tr.fieldNameSet[pe.Key]` (`pkg/transformers/cut.go:197`) or
  `inrec.Get(name)` :221; having-fields iterates `inrec.Head` against a set; rename calls
  `inrec.Rename(pe.Key, ...)`; sort uses `GetSelectedValuesAndJoined`; reorder uses
  `MoveToHead`/`MoveToTail`.

### When verbs see nested vs flat records

Auto-flatten is appended *after* the whole verb chain, at write time
(`pkg/climain/mlrcli_parse.go:445-461`; decision logic and design rationale in
`pkg/cli/flatten_unflatten.go`):

- JSON→JSON: no flatten/unflatten inserted — verbs always see nested records. This is the
  case the feature targets.
- JSON→CSV: flatten runs after the last verb — verbs still see nested records. (#1534's
  group-like surprise lives here.)
- CSV→CSV / CSV→JSON: no unflatten on input — verbs see flat records, including literal
  dotted keys like `req.method`. The header comment at `flatten_unflatten.go:43-48`
  explicitly promises `mlr sort -f req.method` works on such data "with no surprises."
  That promise is the hard backward-compatibility constraint on this feature.

There is no per-verb "needs flattening" capability flag; the only verb-aware chain decision
is the `lastVerbName == "flatten"` check in `DecideFinalUnflatten`
(`flatten_unflatten.go:93`). `TransformerSetup` would be the natural home for such a flag if
one were needed.

### flatsep and prior art

- `TWriterOptions.FLATSEP`, default `"."` (`pkg/cli/option_types.go:94, :261`;
  `pkg/cli/separators.go:45`), set via `--flatsep`/`--jflatsep`. It's a writer-side option
  but is used by both the final flatten and the final unflatten, and as the default `-s`
  for the flatten/unflatten verbs.
- Unflatten's string→path splitter is `SplitAXHelper`
  (`pkg/mlrval/mlrmap_flatten_unflatten.go:279`) feeding `PutIndexed`; keys with empty
  segments (`.x`, `x..y`, trailing dot) are treated as literal, with a one-time stderr
  warning (:120-176). Flatten stringifies empty collections: `{}` → `"{}"`, `[]` → `"[]"`
  (`pkg/mlrval/mlrval_accessors.go:33-62`) — i.e. the flatten/unflatten round trip is
  *not* lossless.
- In-verb nesting precedent: `sort-within-records -r` recurses into submaps
  (`pkg/transformers/sort_within_records.go`); `flatten -f` flattens selected fields only.

## The core ambiguity

Given spec `x.y.z` and a record, the name can mean: a field literally named `x.y.z`; field
`x` holding map `{y: {z: ...}}`; field `x.y` holding `{z: ...}`; or field `x` holding
`{"y.z": ...}`. In general a spec with n dots has 2^n candidate splits — the user's
"triple-cased" is the n=2 case. Trying all splits is unpredictable and unimplementable in
any explainable way, so the plan is to never enumerate splits. Two deterministic rules:

1. **Exact flat key first.** If the record has a field literally named `x.y.z`, that's the
   match, full stop. This single rule preserves the CSV-world promise above: flat records
   with dotted headers behave exactly as today.
2. **Else strict per-segment traversal**, mirroring both the DSL dot operator and what
   flatten itself produces: split the spec on the separator; each segment is exactly one
   map key (or array index) at each level. `{"x.y": {"z": 4}}` is *not* reachable via
   `x.y.z` — accepted and documented; the escape hatches are the DSL and the
   flatten-sandwich. (Flatten produces the flat key `x.y.z` from that record, so the
   flatten-sandwich does reach it.)

This makes lookup two-cased, not exponential, and rule 1 means the flat interpretation
always wins when both exist in one record (sharp edge S3 below).

## Design options

- **Option A — flatten-sandwich sugar.** Automatically wrap the chain (or individual
  verbs) in `flatten ... unflatten`, mechanizing the known idiom. Pros: trivial to build;
  semantics are "the names you see in CSV output," which matches many users' mental model;
  regex verbs get path matching for free. Cons: the round trip is lossy (`{}`/`[]` become
  strings; the unflatten arrayify heuristic can turn maps with keys `"1","2",...` into
  arrays that weren't arrays; type inference re-runs on stringified values); whole-record
  flatten cost even when one field is touched; collisions when a record has both literal
  `x.y` and nested `x:{y:...}` (flatten produces duplicate keys); and it changes record
  shape for *other* verbs in the same chain unless scoped per-verb, which the chain
  architecture doesn't support today. Fine as a documented manual idiom; not recommended as
  the feature.
- **Option B — native path accessors** at the Mlrmap level, adopted verb by verb, reusing
  `PutIndexed`/`RemoveIndexed` and adding the missing bulk read (export/generalize
  `getWithMlrvalArrayIndex`, or build `GetPath` directly on `[]*Mlrval` chains). Pros: lossless, structure-preserving, precise per-verb
  semantics, no shape changes for neighboring verbs. Cons: real API surface; each verb
  needs its own semantic decisions (inventory below); long tail of verbs.
- **Option C — document-only.** What #1815 settled for. Zero risk, leaves the asymmetry.
  Worth doing regardless (the flatten-sandwich and DSL idioms belong in
  reference-verbs / flatten-unflatten docs), but it's not the feature.

**Recommendation: Option B, scoped to a small verb set, opt-in (Q1), with Option C's doc
work done in the same effort.**

## Opt-in surface

Never change default interpretation silently. Candidate surfaces:

- (a) Per-verb boolean flag, e.g. `mlr rename -p Body.meta,Body.renamed_meta` ("-p" for
  path; letter TBD per verb's free letters). Explicit, discoverable in each verb's help,
  adoptable verb-by-verb. Recommended.
- (b) Global main flag (`--nested-fields`) flipping interpretation for all supporting
  verbs. One switch, but action-at-a-distance, and a chain mixing verbs that do and don't
  support it becomes confusing.
- (c) In-name syntax, e.g. `-f '$.x.y.z'` (JSONPath-ish) or `x["y"]["z"]`. No flag needed,
  but invents a mini-language, collides in principle with literal names, and is unpleasant
  in shells.

Note that with the exact-key-first precedence rule, even a default-on behavior would be
almost backward compatible — the fallback only fires when the flat lookup misses, which
today yields "no match." But "almost" hides real changes: `cut -x -f x.y` and
`having-fields --none-defined x.y` would start *matching* where they matched nothing, and
per-record heterogeneity makes behavior data-dependent. Hence opt-in for v1; a default flip
can be revisited later (same posture as the `--iauto` plan).

## Proposed design

### 1. Path type and split

A parse-once type, e.g. in `pkg/mlrval` or `pkg/lib`:

```go
type FieldPath struct {
    original string     // the literal spec, for exact-match-first and for output naming
    indices  []*Mlrval  // split segments, ready for Get/Put/RemoveIndexed
}
```

Built once at verb-construction time from each `-f`/`-g` token (never per record). Splitting
uses the same separator and the same empty-segment rules as unflatten (`SplitAXHelper`
semantics): a spec with leading/trailing/doubled separators is treated as wholly literal —
no warning needed here since literal is always tried first anyway. Numeric segments become
int Mlrvals (1-based array indices, matching flatten output `x.1` and DSL indexing),
non-numeric become strings.

### 2. Mlrmap API additions

All additive, all delegating to existing indexed machinery, all honoring
exact-flat-key-first:

- `GetPath(path) *Mlrval` — `findEntry(original)` first, else walk `indices`.
- `HasPath(path) bool`.
- `RemovePath(path) bool` — leaf removal only; parent maps remain (matches DSL
  `unset $x.y`, per `removeIndexedOnMap`). Returns whether anything was removed.
- `PutPathCopy(path, value)` — via `PutIndexed` with auto-deepen (only needed by verbs
  that create fields; not needed for v1's cut/having-fields).
- `RenamePathLeaf(path, newLeafName) bool` — rename the last segment *within its parent
  map*, preserving position (`Mlrmap.Rename` semantics one level down). Cross-parent moves
  (`rename Body.meta,Other.meta`) are out of scope for v1 — that's a move, not a rename
  (S14).

Deliberately *not* added: any API that tries multiple splits of the original string.

### 3. Separator

Use `FLATSEP` (default `"."`), threaded from options to verb constructors, for consistency
with flatten/unflatten and with what users see in flattened output. Sharp edges S4 apply
(multi-char separators, writer-option provenance). Verbs that grow the opt-in flag could
also accept a per-verb `-s` override, mirroring the flatten/unflatten verbs.

### 4. v1 verb set and per-verb semantics

Driven by the issues and by expected demand; each needs its semantics pinned before code:

- **rename** (#1815): `rename -p old.path,new_leaf_name` — leaf rename in place. Decide:
  is the second element a full path (error unless it differs only in the leaf) or just the
  new leaf name? Recommend full path + validation, so the CLI shape matches non-p rename
  and the flatten-sandwich idiom (`rename Body.meta,Body.renamed_meta`).
- **cut**: `-f a.b` extracts preserving structure — output record `{"a": {"b": ...}}`, not
  `{"a.b": ...}` (Q6). Requires a "copy path into fresh record, creating parents" helper.
  `-x` removes the leaf, keeping siblings and the (possibly emptied) parent. `-o` ordering
  applies at top level of the reconstructed record. Interaction: two specs sharing a prefix
  (`-f a.b,a.c`) must merge into one `a`, preserving sub-order.
- **having-fields**: `--at-least`/`--all-defined` etc. gain path membership via `HasPath`.
  The regex variants stay flat (S10).
- **sort**: `-f a.b` sorts by the path value. Missing-path records need a defined ordering
  (today missing flat fields group at the end — reuse that). Path values that are maps or
  arrays: define as error-or-last (S12).
- **reorder**: plausible but semantically muddy (move leaf within its parent? hoist to top
  level?) — defer past v1 unless a crisp semantic emerges.

Follow-on tiers, each with its own sharp edges, explicitly out of v1:

- Group-by (`-g`) family: count, uniq, count-distinct, stats1, top, decimate,
  count-similar, fraction, histogram... Group-by keys go through joined-string map keys
  (`GetSelectedValuesJoined`); path lookups slot in, but map/array-valued results need a
  rule (S12), and the *output* field naming question (S7) hits every one of these.
- Value-field (`-f`) stats verbs: stats1/stats2/merge-fields/step — output names like
  `x.y_sum` are new *flat* names that will themselves auto-unflatten to `x: {y_sum: ...}`
  in CSV→JSON runs. Decide whether that's a feature or a bug before touching these.
- Leaf mutators: fill-down, fill-empty, sub/gsub/ssub, case, format-values — mechanical
  once the path API exists.
- join `-j/-l/-r` on nested keys — its own plan; join has a second reader and half/full
  streaming variants.

## Sharp edges inventory

- **S1 — split ambiguity.** 2^n candidate splits; resolved by exact-key-first + strict
  per-segment traversal, never split enumeration. `{"x.y": {"z": 4}}` unreachable via
  `x.y.z` — document with the flatten-sandwich as escape hatch.
- **S2 — literal dots are load-bearing in CSV-land.** `flatten_unflatten.go:43-48`
  explicitly promises flat `req.method` addressing; #1534 shows dotted flat names in real
  data. Exact-key-first preserves this even with the flag on; without the flag nothing
  changes at all.
- **S3 — both-present and per-record heterogeneity.** One record may carry literal `x.y`
  *and* nested `x:{y:...}`; different records in one stream may differ. Precedence makes
  each record deterministic, but users can still be surprised mid-stream — needs a docs
  callout, and `cut -x`/removal semantics must be verified against both-present records in
  regression tests (remove the flat one only? both? — recommend: flat only, since exact
  match won; test pins it).
- **S4 — separator provenance.** `FLATSEP` lives in *writer* options; a verb-side reader
  of it is a small layering smell (the `--iauto` plan hit the same issue from the other
  side). Multi-char separators (`--flatsep ::`) must work; a separator that also appears
  inside genuine nested keys (JSON keys may contain dots at any level) re-creates S1 one
  level down — strict segment matching means such keys are simply unreachable by path spec
  (reachable via DSL bracket syntax only). Document.
- **S5 — arrays.** Numeric segments as 1-based indices matches flatten output (`x.1`) and
  DSL aliasing (negative indices count from the end — decide whether to allow; recommend
  yes, free via `UnaliasArrayIndex`). Out-of-bounds reads → absent. Writes via auto-deepen
  create *maps* keyed `"1"`, not arrays (`NewMlrvalForAutoDeepen`) — v1 verbs don't
  auto-deepen, but any later Put-capable verb must decide (the unflatten `Arrayify` pass is
  what turns those into arrays today).
- **S6 — degenerate specs.** Leading/trailing/doubled separators: treat the whole spec as
  literal (unflatten precedent, `mlrmap_flatten_unflatten.go:120-176`); no path fallback.
- **S7 — derived-output naming.** Any verb that *creates* fields named after inputs
  (stats1 `x.y_sum`, merge-fields, step `x.y_delta`, count-distinct's `field` column)
  produces flat dotted names that downstream auto-unflatten will restructure. Deferring
  those verbs defers the problem, but the rule must exist before tier 2.
- **S8 — removal leftovers.** Removing the last child leaves `{}` (DSL-consistent). Under
  JSON output that's visible; under CSV output, flatten turns `{}` into the string `"{}"`.
  Consistent with `unset` today, but worth a regression case so it's chosen, not
  accidental.
- **S9 — performance.** Paths parse once at verb construction. Per-record cost when the
  flag is off: zero (existing code paths untouched). When on: exact `findEntry` first
  (hash-map hit for wide records), traversal only on miss. cut's hot loop currently does
  set-membership per record key — path mode inverts to per-spec probes; fine for typical
  spec counts, note in benchmarks (`make bench`).
- **S10 — regex forms.** `cut -r`, `having-fields --any-matching`, `rename -r` match flat
  key strings; a regex over nested structure is ill-defined. v1: regex + path flag is an
  error. A later option: match regexes against *flattened* names (S13's mental model), but
  that drags in flatten cost and S1 collisions — separate decision.
- **S11 — structure-preserving extraction (cut).** Building the output record requires
  copying partial subtrees with shared-prefix merging and stable ordering — new helper,
  needs its own unit tests (deep siblings, prefix overlap `a.b` + `a`, spec ordering vs
  `-o`).
- **S12 — non-scalar path results.** Group-by joining and sort comparison assume
  scalar-ish values. A path can resolve to a map/array. Options: error, json-encode for
  keying, or sort-last. Recommend json-encode for group-by keys (deterministic) and
  collections-sort-last for sort; decide before tier 2.
- **S13 — two mental models forever.** Users will hold both "flattened names" (CSV view)
  and "paths" (JSON view); this feature makes the second one real in verbs. The docs page
  (flatten-unflatten) must gain a section explaining that path specs and flattened names
  usually coincide (same separator, same segments) and exactly when they don't
  (S1's unreachable case, `{}`/`[]` lossiness, arrayify).
- **S14 — rename is not move.** `rename -p a.b,c.d` where the parent differs is a move
  with different ordering/overwrite semantics — reject in v1 with a clear error pointing
  at the DSL.
- **S15 — chain-position interactions.** The feature operates on whatever shape reaches
  the verb: on CSV input, paths mostly no-op (records are flat; exact-match rule handles
  it); after an explicit `flatten` verb, likewise. No new chain-insertion logic needed —
  and specifically, the existing `flatten then ... then unflatten` idiom must keep working
  unchanged (regression case).
- **S16 — REPL.** REPL verbs share transformer code and its own flatten/unflatten decision
  (`pkg/terminals/repl/verbs.go:600-615`); no divergence expected, but include a REPL
  smoke test.

## Phased implementation

Each phase independently mergeable, `make check` green throughout.

1. **Docs-first (Option C, immediate).** Document the DSL and flatten-sandwich idioms for
   nested rename/cut in reference-verbs and flatten-unflatten pages — closes the actual
   ask in #1815 regardless of the rest.
2. **Path core.** `FieldPath` + `GetPath`/`HasPath`/`RemovePath`/`RenamePathLeaf` in
   `pkg/mlrval`, unit tests covering S1/S3/S5/S6/S8 cases. Pure additive; no verb changes.
3. **v1 verbs.** rename, cut, having-fields, sort behind the per-verb opt-in flag; the
   structure-preserving extraction helper (S11); regression cases per verb including
   both-present records, arrays, CSV-input no-op, flatten-sandwich equivalence
   (`cut -p -f a.b` ≡ `flatten then cut -f a.b then unflatten` on lossless inputs).
4. **Tier 2.** Group-by family + S12 rule; then leaf mutators; stats value-fields last
   (S7 rule required first).
5. **Docs & help.** Verb help strings (feeds generated reference-verbs), flatten-unflatten
   page section (S13), man page via `make dev`.

## Open questions for the maintainer

- **Q1 — opt-in flag vs default-on.** Exact-key-first makes default-on *nearly* safe, but
  `cut -x`/having-fields matching semantics do change on nested data. Recommend per-verb
  opt-in flag now; revisit default at a major release.
- **Q2 — flag spelling.** One consistent letter across verbs (is `-p` free everywhere in
  the v1 set?) vs a long option `--paths` only. Long-option-only is safest against
  per-verb letter collisions.
- **Q3 — separator.** Reuse `FLATSEP` (recommended, one knob) vs hard `"."` vs per-verb
  `-s` only. If `FLATSEP`, note it's writer-scoped today (S4).
- **Q4 — rename second argument.** Full path (validated same-parent) vs bare new leaf
  name. Recommend full path for symmetry with flat rename and the sandwich idiom.
- **Q5 — negative array indices in specs.** DSL-consistent aliasing (recommend) vs
  positive-only (flatten never emits negatives, so specs-as-flattened-names don't need
  them).
- **Q6 — cut output shape.** Structure-preserving (recommended; JSON-native) vs
  flattened-key output (matches Option A's model). If anyone wants the latter they can
  say `flatten then cut`.

## Appendix: verb inventory — field-name arguments

Survey of all `pkg/transformers/*.go`. "Field-name args" means flags/positionals whose
payload is record keys — not values, separators, formats, counts, filenames, or
accumulator/stepper names (`-a` in stats1/step/merge-fields is an accumulator list, not
fields). put/filter are excluded: their `-f` is a DSL *filename*, and their nested access
is the DSL's own.

### Verbs that take field names

| Verb | Field-name args | Regex mode? | Record ops used | Derives output names from inputs? |
|---|---|---|---|---|
| bar | `-f`: value fields | no | Get, PutReference in place | no |
| bootstrap-ci | `-f`: value fields; `-g`: group-by | no | Get, grouping | emits summary records |
| case | `-f`: restrict to fields (`-k`/`-v` keys/values only) | no | set membership; rewrites key casing | yes — transforms key names |
| cat | `-g`: group-by; `-N`: counter field to create | no | GetSelectedValuesJoined, PrependReference | yes — `-N` |
| count | `-g`: group-by; `-o`: count field name | no | grouping | yes — `-o` |
| count-distinct | `-f`: distinctness fields; `-o`: count field | no | GetSelectedValues | yes — `-o` |
| count-similar | `-g`: group-by; `-o`: counter field | no | grouping | yes — `-o` |
| cut | `-f`: keep (or drop with `-x`) | **`-r`** | Has, Remove; regex vs keys | no |
| decimate | `-g`: group-by | no | grouping | no |
| fill-down | `-f`: fields to fill (`--all`) | no | Get, Has, PutReference | no |
| flatten | `-f`: fields to flatten (else all) | no | per-field flatten | creates dotted keys |
| fraction | `-f`: value fields; `-g`: group-by | no | Get, grouping | **yes** — `x_fraction` etc. |
| gap | `-g`: group-by | no | grouping | no |
| group-by | positional: group-by keys | no | GetSelectedValuesJoined | no |
| having-fields | `--at-least`/`--which-are`/`--at-most`: name lists | **yes** — `--all/any/none-matching` | key-set membership / regex vs keys | no |
| head | `-g`: group-by | no | grouping | no |
| histogram | `-f`: fields to bin; `-o`: output prefix | no | Get | **yes** — `PREFIX…` |
| join | `-j`/`-l`/`-r`: join keys; `--lk`: left fields to emit (`-f` is a *file*) | no | join-key matching | prefixed names on collision (`--lp`/`--rp`) |
| json-parse | `-f`: fields to parse (else all) | no | per-field in place | no |
| json-stringify | `-f`: fields to stringify (else all) | no | per-field in place | no |
| label | positional: new names by position | no | Label (bulk rename) | yes — renames keys |
| merge-fields | `-f`: value fields; `-c`: name substrings to collapse; `-o`: output basename | **`-r`** | Get; key iteration | **yes** — `basename_sum` etc. |
| most/least-frequent | `-f`: fields to count; `-o`: count field | no | grouping | yes — `-o` |
| nest | `-f`: single field to explode/implode | **`-r`** | Get, Remove, PutReference | explode-across-fields makes `field_1,…` |
| rank | `-f`: fields to rank; `-g`: group-by | no | Get, PutCopy | **yes** — `x_rank` |
| rename | positional `old,new,…` pairs | **`-r`**, `-g` (gsub on key text) | Rename; regex key rewrite | yes — the new names |
| reorder | `-f`: fields to move (`-e` tail); `-b`/`-a {center}`: pivot field | no | MoveToHead/Tail; Get + rebuild | no |
| repeat | `-f`: field holding the repeat count | no | Get | no |
| reshape | `-i`: input fields; `-o`: key,value output names; `-s`: key,value fields | **`-r`** (on `-i`) | reads/creates keys; pivots | **yes** — from `-o`, or from field *values* with `-s` |
| sample | `-g`: group-by | no | grouping | no |
| sec2gmt / sec2gmtdate | positional: fields to convert | no | Get, PutReference in place | no |
| seqgen | `-f`: output field to generate (no input) | no | PutCopy | yes — sole output name |
| sort | `-f`/`-r`/`-nf`/`-nr`/`-tf`/`-tr`/`-c`/`-cr`: sort keys (`-r` = reverse here, **not** regex) | no | GetSelectedValues | no |
| sort-within-records | `-f`: restrict/order keys | **`-r`** (with `-f`; standalone `-r` = recurse) | reorders keys in record | no |
| sparkline | `-f`: value fields | no | Get | emits per-field |
| sparsify | `-f`: fields to consider (else all) | no | Remove empties | no |
| split | `-g`: group-by | no | grouping | no (writes files) |
| stats1 | `-f`: value fields; `-g`: group-by | **yes** — `--fr`/`--fx`, `--gr`/`--gx` | Get; GetSelectedValues(Joined) | **yes** — `x_sum`, … |
| stats2 | `-f`: value-field pairs; `-g`: group-by | no | Get; grouping | **yes** — `x_y_corr`, … |
| step | `-f`: value fields; `-g`: group-by | no | Get; grouping | **yes** — `x_delta`, `x_ewma_*`, … |
| sub/gsub/ssub | `-f`: fields whose values to edit (`-a` = all) | **`-r`** | set/regex vs keys; edit values | no |
| surv | `-d`: duration field; `-s`: status field | no | Get | emits new records |
| tail | `-g`: group-by | no | grouping | no |
| template | `-f`: exact ordered output key set | no | builds record with these keys | yes — output keys are `-f` |
| top | `-f`: value fields; `-g`: group-by; `-o`: index field | no | Get; grouping | **yes** — `x_top`, `-o` |
| uniq | `-g`/`-f`: uniqueness fields; `-x`: exclude fields; `-o`: count field | no | GetSelectedValuesJoined, GetKeysExcept | yes — `-o` |
| unflatten | `-f`: fields to unflatten (else all) | no | splits keys on flatsep | rebuilds nested keys |
| unsparsify | `-f`: names to guarantee present | no | Has, PutCopy | yes — `-f` become columns |

### Verbs with no field-name args

altkv, bootstrap, check, clean-whitespace, describe, fill-empty, format-values, grep
(regex over serialized record text, not keys), group-like, latin1-to-utf8 /
utf8-to-latin1, nothing, regularize, remove-empty-columns, shuffle,
skip-trivial-records, summary (`-a`/`-x` are summarizer names), tac, tee, unspace.

### Cross-cutting observations

- **`-r` is overloaded across verbs**: field-name-regex in cut, having-fields
  (`--*-matching`), rename, merge-fields, nest, reshape, sort-within-records (with `-f`),
  sub/gsub/ssub, stats1 (`--fr/--gr` family); but *reverse* in sort, *right-join keys* in
  join, *recurse* in sort-within-records (standalone). Any path-flag naming must dodge
  these (Q2).
- **Derived-name verbs** (S7) are the long pole: stats1/stats2/step/fraction/rank/top/
  merge-fields/histogram/nest all synthesize new keys by suffixing input names — a path
  spec `x.y` as input yields flat `x.y_sum`-style outputs whose downstream unflatten
  behavior must be defined before those verbs adopt paths.
- **Group-by extraction is centralized** on `GetSelectedValuesJoined`/`GetSelectedValues`
  — one choke point covers `-g` for cat, count, count-similar, decimate, gap, group-by,
  head, sample, split, stats1/2, step, tail, top, uniq, fraction, rank, bootstrap-ci.
- Selection/removal concentrates on `Has`/`Get`/`Remove`; reordering on
  `MoveToHead`/`MoveToTail`; bulk rename on `Rename`/`Label` — the Mlrmap path API in §2
  maps one-to-one onto these.