astrolabe/docs/chart-builder-enhancement-scope.md

# Chart Builder — Enhancement Scope

> **Status:** scope consolidated 2026-06-10. This is the **single forward-looking home**
> for chart-builder enhancement work — it merges the research backlog from
> [`chart-builder-research.md`](./chart-builder-research.md) §8 (the M4 decision and its
> deferred items) with the interaction ideas from [`lyra-review.md`](./lyra-review.md) §5,
> read against the current spec ([`spec/06-chart-builder.md`](./spec/06-chart-builder.md))
> and the shipped code (`src/core/chart-builder.ts`).
>
> **Goal (the brief):** a **rapid, intuitive GUI for building Vega-Lite specs**, with
> **guidance and recommendations on the fly**, **moderately capable** — not a full
> visual-design IDE.
>
> **Decision (2026-06-10):** push the builder from its shipped **Tier B** ("smart +
> guarded") up to **Tier C** ("intent-first aid"). "Moderately capable" is the ceiling:
> we add the controls that are _both common and awkward in JSON_ and stop there; the long
> tail of styling/scale/axis breadth stays in Monaco. The two source docs remain the
> research record (the _why_ and the citations); this doc is the _plan_ (the _what next_
> and the _order_).

---

## Status log

Newest first. The at-a-glance build-order tracker is §4; per-item detail is §3. This log is
the quick "where are we" — read it first.

- **2026-06-12 (polish batch)** — **the 2B visual-pass findings fixed; the parked council
  batch resolved and applied; two capability gaps closed.**
  - **Aggregation for any field type.** `validAggregateOps` replaces the quantitative-only
    gate: **Count distinct** on anything (a nominal Colour/Y now measures unique values —
    emitted as a quantitative `{ aggregate: 'distinct' }`), Min/Max on temporal/ordinal,
    arithmetic still quantitative-only. Retyping keeps a still-valid aggregate.
  - **Title/subtitle** (pulled forward from the placement discussion): top-level `title`
    emission; a user title becomes the snippet name; subtitle gated on a title.
  - **Council batch applied:** the type chip and every native `<select>` in the builder are
    now **`SelectControl`** — a reusable value-picker disclosure (the SortControl primitive
    generalized; replaces native selects app-wide where the control is part of a designed
    surface). Field chips open an explicit **channel chooser** (occupied channels say what
    they'd replace); an **armed** channel short-circuits it and is now visible (accent ring +
    status line + Esc disarms). "or constant" → **"Use a constant"** ghost button. Chart
    properties (Title · Subtitle · W · H) moved to a **strip under the preview**; Sort/Stack
    stay by the encodings. Resolutions recorded in `architecture/10` §5; `ux-second-pass.md`
    cleared (drag stays deferred).
  - **Visual-pass fixes:** pill ✕ flush right (pills hug content), Swap X/Y beside the Axes
    heading, field shelf scrolls (280px viewport, sticky group heads), per-channel transforms
    inline beside the pill, preview honors explicit width/height (fit-mode only while auto).
  - **Verified:** `typecheck` + `test` (774, +10) + `eslint` + `build`. **Owed:** a fresh
    visual look at the reworked surfaces (user-driven).

- **2026-06-12 (scope)** — **3D added to Phase 3: builder entry points & discoverability**
  (council-reviewed: NN/g #6 recognition + #7 expert accelerators, Carbon empty-states).
  Today the builder's only door is Datasets → row "Build Chart" while the primary "Create
  New Snippet" lands in blank Monaco — the data model's shape, not the user's intent.
  Decided: fork the library's creation surface (primary **Build Chart**, ghost **New JSON
  snippet**), give the builder an internal dataset picker + a no-datasets empty state, add
  an onboarding tile. 3D lands **with or just before 3A** — the Build-Chart button opens
  onto the intent front door, making 3A the app's guided creation flow. Detail in §3 · 3D.

- **2026-06-11 (scope)** — **3C added to Phase 3: open-in-builder (strict spec hydration).**
  A snippet-list button, enabled exactly when the spec round-trips losslessly through the
  builder dialect (checked by re-assemble + deep-compare, not feature enumeration). Strict
  only — no lossy/residue modes (§5's round-trip trap stays closed). Detail in §3 · 3C.

- **2026-06-11 (Phase 2)** — **2A + 2B shipped: the field-first interaction substrate.** Up next: **3A intent-first front door**.
  - **2A · Value-or-field channels (the Property model).** `ChannelMapping` gained a constant
    `value` arm; the assembler emits `{ value }` and the measure/stack/area-split/prune logic
    all treat a constant Colour as "no series." New pure helpers (`isValueMapping`,
    `channelAcceptsValue`, `defaultChannelValue`, `coerceChannelValue`,
    `isColumnAllowedOnChannel`). `type` stays required (a constant carries a preserved-but-
    ignored type, so a field↔constant toggle round-trips). Tested.
  - **2B · Field-first builder.** The channel-first dropdown rows are replaced by a **field
    shelf** (columns as type-glyphed chips, auto Dimensions/Measures split past a column
    threshold), **click-to-assign** (armed channel, else first empty that accepts it —
    `assignField`/`focusChannel`), Tableau-style **pills** (a type chip that **cycles** the
    field type in-place, transforms beneath), and the **on-chart Columns/Rows shelves** above
    the preview (X/Y as a property of the chart) with a **reserved faceting placeholder** in
    each. Colour/Size live in a left **Marks card**, each switchable to a **constant** (2A).
  - **Decisions taken (overridable):** drag deferred (click/keyboard-first, fully tested);
    field grouping auto; aggregate stays a control under the pill (no pill context-menu yet).
  - **Spec §06** rewritten for the field-first model (Layout, Encoding channels, faceting
    placeholder, constants, validation/output).
  - **Verified:** `typecheck` + `test` (762, +20 core/store/modal) + `eslint` + `build`.
    **Owed:** a manual/visual pass against the live builder; a **`/council`** look at the
    type-cycle chip (a cycling button gives no direct type pick — explicit menu vs cycle is a
    real a11y trade-off) and the constant affordance.

- **2026-06-11 (later)** — **Owed debts on the Data section closed. Up next: Phase 2 (2A value-or-field channels, then 2B field shelf).**
  - **Council pass** on the new error/disclosure copy (the previously-deferred auto-fire
    surface). Three a11y conformance gaps against `architecture/10` were fixed: the inline
    expression feedback now carries a **status glyph** (round error / triangle warning), not
    colour alone (§3, WCAG 1.4.1); the parse error is a **polite `role="status"`**, not a
    per-keystroke assertive alert (APG Alert / WCAG 2.2.4); and the message is linked to its
    input via **`aria-describedby`** (GOV.UK error-message). Unknown-field copy clarified to
    "…— not a column in this dataset." Resolution recorded in `architecture/10` §5.
  - **Manual/visual pass** run via a headless-Chrome (Playwright) walk-through of the live
    builder — filter shelf (predicate + `is between`), field↔expression toggle, expression
    error/unknown-field glyphs (filter _and_ calc inputs), calculated field, data-preview
    table with type chips, smart-default chart. All surfaces render as intended.
  - **Bug found + fixed (mid-edit preview resilience).** `filterTransformObject` /
    `calculateTransformObject` emitted **any non-empty** expression — including a half-typed,
    unparseable one — so the preview blanked with a raw render error while the user typed.
    They now **drop a syntactically-invalid expression** like an empty/incomplete entry
    (guarded by core `validateExpression`), matching `buildTransforms`' own "a config
    mid-edit still renders" contract; the inline feedback still flags the typo. Pure core,
    tested (+2 cases). Visually confirmed: an invalid filter/calc now keeps the last-good
    chart instead of breaking the preview.
  - **Verified:** `typecheck` + `test` (741 passing) + `eslint` clean.
  - **Still open (copy judgment, user's call):** the preview's catch-all
    `"Couldn't render this chart: {raw Vega message}"` puts a diagnostic in the headline
    (arch 10 says diagnostics go in a disclosure) — correct in the editor, debatable in the
    builder; and the builder's terse `"Dataset «X» not found."` drops the next-step the
    contract mandates (near-unreachable in the builder). Both noted, not changed.

- **2026-06-11** — **1C + 1D + 1E shipped (the Data section).**
  - A new **Data section** at the top of the builder's left pane — "here are your rows;
    shape them, then encode them" — emits the spec's top-level `transform` array.
  - **1C · Filters** — a guarded **field + operator + value** predicate shelf. Operators
    narrow by field type (`validFilterOps`): a measure/temporal field offers ordering
    (`< ≤ > ≥`) + `is between`; a category offers `is` / `is not` / `is one of`. Values
    coerce by type (quantitative → number; others → string, so ISO dates sort right).
    `notEqual` emits a `{ not: { …equal } }` wrapper. A reversible **expression** power-mode
    takes a raw `datum.…` predicate. Incomplete filters are skipped so the preview keeps
    rendering. Multiple filters AND together. Pure core (`buildTransforms`,
    `validFilterOps`, `filterOpArity`), tested.
  - **1C · Calculated fields** — `{ calculate, as }` derived columns. A named field appears
    in the channel dropdowns via `effectiveColumns` (defaults Quantitative); emitted
    **before** filters (a row-wise calculate is order-independent, so calc-first is
    equivalent and lets filters reference derived fields). Removing/renaming a referenced
    field clears the dangling channel (`pruneEncodings`, in the store on calc edit/remove).
  - **1E · Expression validation** — new pure core `expr-validate.ts` using Vega's own
    `parseExpression` (already in the `vega` chunk, so ~zero bundle cost): inline syntax
    errors on both expression inputs, plus a soft **unknown-field** warning when a
    `datum.<field>` reference doesn't match a column (`referencedFields` walks the AST).
    Field discoverability is served by **dataset-derived placeholder examples** (e.g.
    `datum.revenue * 2`); a full Monaco-style completion popup is **deferred** (a bare
    `<input>` doesn't warrant it — noted, not built).
  - **1D · Data preview** — a collapsible, read-only first-N-rows table with a per-column
    **type chip** in each header, to sanity-check inferred types before building (reuses
    core `tabularRows`). Default collapsed; the scroll region is keyboard-reachable
    (`tabIndex=0` + labelled group — avoids the Datasets-manager a11y gap).
  - **Spec §06** gained a "Data (filters, calculated fields, preview)" section; the Layout
    and Output blocks cross-reference it.
  - **Council not yet run** on the new error/disclosure copy (the soft auto-fire surface:
    expression-error + unknown-field copy, the preview disclosure). Conventions were matched
    to the existing warnings region (arch 10 §5) and `SettingsPopover` disclosure; flag for a
    council pass on review if desired.
  - **Expression reference:** a contextual link to the Vega expression-language docs is
    shown when an expression input is in play (a calculated field, or a filter in
    expression mode) — the place the user needs to know the available functions/operators.
  - **Verified:** `typecheck` + `test` (full suite green; +103 new core/store cases, +4 modal
    smoke tests) + `eslint` + `build` (PWA, 45 precache entries). **Owed:** a manual/visual
    pass against the live builder (filter shelf, calc → channel, expr errors, preview
    table) — tests don't cover what the surface looks/feels like.
  - **Surfaced direction (now parked):** the 1D preview shows **raw source** rows; a
    transform-aware **data inspector** (resolved rows, à la vega-editor, in the builder
    _and_ below the main Live Preview) is the wanted evolution — documented in
    **[`docs/data-inspector-exploration.md`](./data-inspector-exploration.md)**, deferred.

- **2026-06-10** — **Up next: 1C (filter + calculate transforms), paired with 1D (data preview).**
  - **1B · Per-chart export** shipped: an **Export** disclosure in the Live Preview header
    (distinct from the workspace Export) — **Copy spec** + **Download JSON** (`.vl.json`) of
    the shown text, and **Download PNG/SVG** of the live chart. Image output rides a new
    `RenderHandle.toImageURL(format, { scale, background })` (PNG via `view.toCanvas` →
    `blob:` URL; SVG via `view.toSVG` → `data:` URL), so no component touches the Vega
    `view`. Filenames derive from the snippet name — filesystem-safe, script-preserving
    (pure `core/chart-export.ts`, tested). Home: the **preview header**, not the plan's
    "library row / editor toolbar" — image export needs the live view; the
    disclosure-of-controls (not an ARIA menu) mirrors `SettingsPopover`. - **Export options (from first-round feedback):** PNG **Resolution** `1×/2×/3×` is a
    multiplier of `devicePixelRatio`, so the default `1×` is Retina-crisp — the soft-1×
    export was a dpr bug (raw `toImageURL` scaleFactor ignores dpr). **Background**
    `Theme`(default)/`White`/`None` fixes transparent PNGs (the chart config is
    transparent so the on-screen pane colour shows; export composites the chosen colour
    under the PNG / adds an SVG `<rect>`). **Referenced data** `Inline`(default)/`Keep
refs` (shown only when the spec references saved datasets) inlines dataset values so
    the exported spec renders standalone (`inlineReferencedDatasets`, tested). - Spec §08 gained a _Per-chart export_ section (with the options); §04 cross-references
    it; `architecture/05` §2 records the handle's dpr-aware scale + background compositing.
  - **1A · Actionable hints** shipped: one-click fixes on guidance warnings
    (`BuilderWarning.fixes` + `applyWarningFix`), council-reviewed, with focus/announce a11y.
  - **Builder UX/perf batch** (from dogfooding the Superstore dataset) shipped: near-fullscreen
    `xlarge` modal tier; panes scroll internally (preview in its own viewport); backdrop-dismiss
    guard (`dismissOnBackdrop: false`); render-timing diagnostics; **canvas** preview renderer
    (SVG stays default elsewhere) + **canvas max-dimension guard** (`ChartTooLargeError` via a
    headless probe); **data-aware default pre-population** (`smartDefaultEncodings`).
  - Scope consolidated and **Tier-C target** set (this doc created); the research/Lyra forward
    sequences are superseded by §4 here.

---

## 1. Where the builder is today (the floor — don't rebuild)

Tier B is shipped and tested (M4 done). The intelligence that decides _which chart and
why_ already exists in `src/core/chart-builder.ts`:

- **Smart default mark** from the (X, Y) field-type shape — not unconditionally Bar
  (`defaultMark`).
- **Valid-type-only** field-type menus per column + **Size discipline** (Nominal /
  Temporal / negative-extent columns are _blocked_ on Size, not merely warned)
  (`validFieldTypes`, `isChannelTypeAllowed`).
- **Non-blocking guidance** — `builderWarnings` (6 rules: line/area needs both axes,
  all-categorical, two-measures-want-scatter, area-split-many-series, crowded category
  axis, negative-size guard).
- **Per-channel transforms** — Aggregate / Bin / `timeUnit`, chart-level Sort / Stack,
  and a field-less "Count of records" measure.
- **Swap X/Y**, debounced live preview, validation gate (≥1 channel mapped).

Everything in §3 below is **backlog** — verified not yet built: `BuilderWarning` has no
`fix` field, there is no per-chart export, no top-level `transform`, no data-table
preview, no channels beyond X/Y/Color/Size, no styling/scale controls.

---

## 2. The target experience (Tier C, concretely)

Reading the brief's four words against the research:

| Brief word                                | What it means here                                                            | The levers (from §3)                                                                                          |
| ----------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Rapid**                                 | shortest path from "a dataset" to "a chart I'd keep"                          | smart defaults (have), intent front door, starter examples, field shelf                                       |
| **Intuitive**                             | matches how people think ("I have fields; what shows my point?")              | intent front door, field shelf, data preview, value-or-field channels                                         |
| **Guidance & recommendations on the fly** | the app proposes and corrects, not just validates                             | **actionable hints**, **intent front door** (the defining Tier-C feature)                                     |
| **Moderately capable**                    | covers the common data-shaping + a few high-value encodings; _not_ every knob | filter / calculate, a _small_ set of promoted controls — and a hard stop short of Lyra's everything-inspector |

The defining shift from B→C is the **intent-first front door**: a _"what do you want to
show?"_ entry (FT / Datawrapper intent categories) that recommends a mark + channel layout
from `intent × column-types`, instead of starting the user at a blank mark picker. It is
the feature the brief most directly asks for, and it is the one piece the M4 research
_deferred_. Choosing Tier C is choosing to pull it forward.

Tier C **extends** §06 — it does not replace the mark-first builder. The front door is an
on-ramp; the user can still ignore it and drive the channels directly, and can always drop
to Monaco. This preserves Astrolabe's core invariant: **the JSON spec is the source of
truth; the builder is a view that emits it** (the Lyra anti-lesson — never let the GUI
become the document).

---

## 3. The consolidated enhancement set

Organized into build phases by dependency and value. Each item carries its **source**,
**value/effort**, **code home**, and **spec impact**. Phases 1–3 are the committed Tier-C
scope; Phase 4 is explicitly _beyond_ "moderately capable" and gated on a later decision.

### Phase 1 — Guidance + I/O (no new interaction model) — _do first_

High value-to-effort, mostly pure-core + thin UI, no architectural change. These make the
_current_ builder dramatically better and de-risk the bigger phases.

**1A · Actionable hints** — _done (2026-06-10)_
Turn advisory warnings into one-click fixes. Source: Lyra §3.1 (`Hints` carries an
`action`). Several existing warnings have an obvious remedy:

- _"draws one mark per row → "_ **[Aggregate as Sum]** (set the measure's `aggregate`)
- _"long labels → "_ **[Swap X/Y]** (the action already exists — just wire it)
- _"two measures usually read as a scatter → "_ **[Switch to Point]**
- _"area split into many series → "_ **[Stack]** or **[Remove colour]**

Implementation: extended `BuilderWarning` with `fixes?: BuilderWarningFix[]` (`{ label, apply }`, pure,
unit-tested in `chart-builder.test.ts`); the modal renders each as a ghost button wired to a
new `applyWarningFix` store action. Wired fixes: **[Aggregate as Sum]** + **[Swap X/Y]**
(one-mark-per-row), **[Swap X/Y]** (high-cardinality axis), **[Switch to Point]** (two
measures), **[Stack]** + **[Remove colour]** (area split — and a stacked area is no longer
flagged, so [Stack] resolves it). Council run (Carbon Actionable notification + APG Alert):
ghost buttons, remedy-in-button, polite "Applied: …" announcement, focus moved off the
removed button — resolution recorded in `architecture/10` §5. §06 "Guidance" amended to
document the one-click fixes.

**1B · Per-chart export** — _done (2026-06-10)_
Today export was workspace-backup only (§08); there was **no way to get one chart out**.
Source: Lyra §3.8. Shipped as an **Export disclosure in the Live Preview header**:

- **Copy spec** (clipboard) + **Download `.vl.json`** of the currently-shown text.
- **Download PNG / SVG** of the live chart via a new `RenderHandle.toImageURL` wrapping
  `view.toImageURL` (PNG at 2× → `blob:` URL, revoked after download; SVG → `data:` URL),
  so the embedding boundary holds — no component touches the raw view.
- Filenames from the snippet name, filesystem-safe and script-preserving (pure
  `core/chart-export.ts`, tested). Standalone HTML left out (the deferred optional).

Home decision: the **preview header**, not the plan's original "library row / editor
toolbar" suggestion — the image formats need the live rendered view, and "export this
chart" reads best beside the chart. The widget is a disclosure-of-action-buttons (not an
ARIA menu), mirroring `SettingsPopover` and sharing its single-open registry. _Spec impact:
new §08 "Per-chart export" section; §04 cross-reference; `architecture/05` §2 handle note._

**1C · Filter (+ Calculate) dataset transforms** — _done (2026-06-11; see status log)_
The transform layer the builder doesn't touch: top-level `transform: []`. Source: Lyra
§3.12. The builder _already tells users to filter_ in three warnings
(`chart-builder.ts:475,476,511`) while offering no way to do it.

- **Filter** _(highest)_ — a guarded **field + operator + value** predicate shelf
  (Voyager-style, no expression needed for the common case); power form is a raw `datum.…`
  expression validated with Vega's `parseExpr` (see 1E). VL applies top-level transforms
  _before_ encoding aggregation, so "filter raw rows, then aggregate" is the natural
  default; filtering on an aggregated value (HAVING) is the advanced case — defer.
- **Calculate / derived field** _(second)_ — `transform: [{calculate, as}]`; the new field
  then appears in column dropdowns like any other.
- **Lookup / join a second dataset** — larger data-model change (one dataset per snippet
  today) → **defer to Phase 4**.

Home: a new **"Data" section in the builder's left pane, above the channels** (`here are
your rows [+ Filter] [+ Calculate] → now encode them`), paired with 1D. _Spec impact: new
§06 "Data / transforms" subsection — this is genuinely new behaviour, write it._

**1D · Data-table preview** — _done (2026-06-11; see status log)_
Neither the builder nor the Datasets manager ever shows the **actual rows**. Source: Lyra
§3.2. A compact, **read-only** first-N-rows grid with a **per-column type chip** in each
header lets users sanity-check inferred types _before_ building — exactly when inference is
most likely to surprise. Type + cardinality + extent already come from `profile.ts`; we
only need the row sample. Homes: a collapsible "Data" strip in the builder's left pane
(under the dataset name) and/or the Datasets manager. Keep it read-only (editing data is
out of scope). _Spec impact: §06 + §05 (Datasets) additions._

**1E · Inline expression validation + field autocomplete** — _done (2026-06-11; completion popup deferred)_
When 1C's expression mode lands, validate the Vega/VL expression string with the library's
own `parseExpr` (Lyra §3.6) and surface errors inline; autocomplete the dataset's own
column names (we have the schema from `profile.ts`) (Lyra §3.10). Record the `parseExpr`
technique in `architecture/08`. _Spec impact: folded into 1C._

### Phase 2 — Interaction substrate (enables Tier C)

Two changes to _how the user touches fields_. They have standalone value but their main job
is to be the substrate Phase 3 (and any future added channels) stands on — sequence them
here so Tier C lands cleanly.

**2A · Value-or-field channels (the Property model)** — _capability gain; medium_
Source: Lyra §3.3 (`Property.tsx` — one droppable control that is _either_ a literal value
_or_ a bound field). Today a channel is field-only. Let a channel also hold a **constant
`value`** (fixed colour / size) with one consistent control and a chip showing the binding
kind. VL encodes exactly this (`field` vs `value` vs `datum`). Generalizes cleanly to any
channel we add later. _Spec impact: §06 "Encoding channels" — a channel may carry a
constant._

**2B · Field shelf + in-place type cycling** — _the largest interaction shift; the Tier-C/facet substrate_
Source: Lyra §3.4 (drop-zones) + Voyager (field list with type chips) + Lyra §3.5
(`FieldType` — the type icon _is_ the control, click cycles N→O→Q→T within the valid set).
Flip from **channel-first** ("pick a channel, then its column") to **field-first** ("here
are your columns — drag/click onto channels"), matching how people think. This is the
natural way to assign many fields across many channels, so it is the **interaction
substrate for Tier C and faceting**, not a standalone task. _Spec impact: §06 layout
revision (field shelf alongside the channel rows)._

### Phase 3 — Tier C (the intent-first front door) — _the defining feature of this push_

The grand idea, stated once: **the builder is the app's rapid, intuitive on-ramp; Monaco
is the expert surface; the JSON spec stays the document.** Phase 3 is where that becomes
true — 3A gives the builder an intent-first opening screen, 3D makes that screen the
app's guided creation flow (not a feature hidden in a modal), and 3B/3C seed and re-enter
it.

**3A · Intent-first front door** — _the B→C step_
Source: research §5/§8 Tier C (FT Visual Vocabulary + Datawrapper intent taxonomy). A
_"what do you want to show?"_ entry mapping **intent × column types → recommended mark +
channel layout**:

| Intent (FT / Datawrapper)  | Our expression (within 5 marks / our channels)                      |
| -------------------------- | ------------------------------------------------------------------- |
| **Magnitude / Comparison** | Bar (x=N, y=Q; horizontal for long labels)                          |
| **Ranking**                | Bar, sorted by value                                                |
| **Change over time**       | Line (x=T, y=Q; color=N for series)                                 |
| **Correlation**            | Point (x=Q, y=Q); Circle + size=Q for a 3rd measure                 |
| **Distribution**           | Bar of binned counts (histogram) — uses Bin                         |
| **Part-to-whole**          | Stacked / 100% bar (uses Stack); _true pie needs `theta` → Phase 4_ |
| **Deviation**              | diverging signed Bar                                                |

Honest coverage gaps stay honest (Spatial / Flow excluded; Part-to-whole partial until
`theta`). The front door is an **on-ramp, not a gate** — it pre-populates the mark-first
builder, which the user can then adjust or ignore. Munzner's typology and Wilke's directory
become seatable council sources at this point (research §2). Built **on the 2B field
shelf**. **Run the front-door copy + flow through `/council`.** _Spec impact: substantial
§06 amendment — a new "Intent" front-door subsection; the M4 spec note explicitly flagged
this as the deferred tier, so this is the planned amendment, not drift._

**3B · Starter examples gallery** — _cheap; pairs with 3A_
Source: Lyra §3.7. A small set of **curated starter snippets**, one per covered FT intent
(Magnitude/Bar, Change-over-time/Line, Correlation/Point, Distribution/histogram,
Part-to-whole/stacked). Improves first-run, doubles as living documentation of what the app
does well. Natural home: the snippet library. **Author them in the builder dialect so they
hydrate via 3C.** _Spec impact: §02 (library seed content)._

**3C · Open in builder (strict spec hydration)** — _decided 2026-06-11; pairs with 3B_
Reverse the assembler: a pure `parseChartSpec(spec) → BuilderConfig | null` so an existing
snippet can re-enter the builder populated. **Strict policy only** — a snippet-list button
("Open in builder") enabled exactly when the spec is **losslessly** representable in the
builder's dialect; anything richer stays Monaco-only. Losslessness is checked not by
enumerating unsupported features but by **re-assembling the parsed config and
deep-comparing against the original** (ignoring key order, `$schema`, and the injected
`tooltip: true`) — exact, ~20 lines, and it stays correct automatically as the dialect
grows (every Phase-4 addition widens what hydrates for free). No lossy or
residue-preserving mode: the moment the builder can overwrite a richer spec, the GUI starts
competing with the JSON as the document (the Lyra round-trip trap — §5).

Value: converts the builder from **create-only to create-and-revise** for its own output,
and makes the 3B starters openable as builder seeds. Known fiddly bits (all mechanical):
inverse of `escapeVegaField`, filter-value un-coercion back to the `BuilderFilter` string
shape, `sort: "-y"` → `{sort: 'descending'}`. Round-trip property tests
(`parse(build(config))` ≈ `config`) are the natural core coverage. Per-row check in the
library is cheap but memoize it. **Decisions at build time:** save semantics on an opened
snippet (update-in-place via a builder "edit" mode vs. always-create-new — today the
builder only creates) and the disabled-button affordance (disabled-with-reason vs. hidden —
park in `ux-second-pass.md` if non-obvious). _Spec impact: §02 (library row action) + §06
(hydration + the strict gate)._

**3D · Entry points & discoverability** — _decided 2026-06-12, council-reviewed; lands with or just before 3A_
Today the builder has **one entry, two levels deep**: Datasets modal → per-dataset "Build
Chart" (`DatasetsModal.tsx`) — the data model's shape ("charts come from datasets"), not
the user's intent ("I want to make a chart"). Meanwhile the library's pinned primary
action, "Create New Snippet", drops straight into blank Monaco — the **expert** path is
the default, and the audience the builder serves may never find it. Council ruling
(recorded here; contract updates at build time):

- **NN/g #6 (recognition over recall):** the build-a-chart action must be visible where
  the intent forms — the library's creation surface, not recalled as a dataset-row action.
- **NN/g #7 (flexibility/efficiency):** the guided path is what novices see; raw JSON is
  the expert accelerator — visible, one click, never hidden. Builder-forward, Monaco
  intact.
- **Carbon empty-states:** one primary action per surface ("pick the most important"; a
  tertiary button for the secondary CTA); a no-data state explains what the space will
  hold, offers one primary next step, and never dead-ends.

The decided shape:

1. **Fork the library's creation surface** — primary **Build Chart** (opens the builder),
   ghost/tertiary **New JSON snippet** (today's instant create, unchanged). Two plain
   buttons with clear hierarchy — no split/menu-button widget (two static choices don't
   earn an ARIA menu).
2. **Dataset picker inside the builder** (header select, default = most recently used), so
   the builder opens without a preselected dataset. Build-time detail: switching datasets
   mid-build re-derives smart defaults vs. prunes incompatible encodings — decide then.
   Useful beyond 3D (switch data without leaving the builder).
3. **No-datasets empty state** in the builder, per the Carbon no-data pattern: what the
   builder does + one primary action ("Add a dataset" → Datasets modal). No dead end.
4. **Onboarding tile** — a data-first path ("import your data → build a chart") beside the
   existing examples gallery.
5. **Keep the dataset-row "Build Chart"** — contextual shortcut, pre-picks the dataset; it
   just stops being the only door.

**Composition with 3A (the point of the sequencing):** the Build-Chart button lands on the
intent front door — "what do you want to show, with which data?" _is_ the builder's
opening screen. 3D without 3A opens onto the mark-first builder (fine, interim); 3A
without 3D is a front door nobody finds. Discoverability is a prerequisite for 3A's value,
so: **3D with or just before 3A.** _Spec impact: §02 (library creation actions +
onboarding) + §06 (dataset picker, empty state)._

### Builder UX & perf — in-flight fixes (2026-06-10, from dogfooding the Superstore dataset)

Pre-existing builder rough edges surfaced while testing on a 10k-row / ~24-col dataset.
Fixed in this batch (not part of 1A–3B, but the same surface):

- **Modal is a near-fullscreen work surface** — new `xlarge` shell tier (`ModalShell`,
  `min(1800px, 96vw) × min(1100px, 92vh)`); the Chart Builder no longer wastes screen.
- **Panes scroll internally, modal keeps its shape** — the `.builder` grid fills the body
  (`grid-template-rows: minmax(0,1fr)`), the config pane and the **preview** each scroll in
  their own viewport, so a tall one-mark-per-row chart scrolls inside the preview instead of
  pushing Create/Cancel below the fold.
- **Backdrop click no longer discards in-progress work** — `dismissOnBackdrop: false` on the
  builder (registry flag); Escape and × still close.
- **Data-aware default pre-population** — the builder no longer blindly takes the first two
  columns (which opened Superstore on a 9994-bar degenerate chart). When the dataset is
  profiled, `defaultBuilderConfig` picks a "safest bet": a low-cardinality category vs a
  count of records (tidy bar), else a time series of the first measure, else a scatter — each
  guaranteed to render. Falls back to positional when there are no stats. Pure + tested. This
  composes with (isn't replaced by) the future intent-first front door — the builder always
  needs a sane opening state.
- **Render-timing diagnostics** — `BuilderPreview` logs `parse · prepare · destroy · embed ·
paint · total` (+ mark, row count) to the console (dev always; prod only when slow). The
  **paint** phase (a double-rAF after `embed()`) captures the real freeze.

**Perf finding — confirmed and fixed.** Diagnostics on the Superstore dataset:
`embed 237ms · paint 6458ms` for the default one-bar-per-row chart (9994 rows), vs
`paint 6ms` once grouped to a few categories. The freeze was entirely **SVG layout/paint**
(one DOM node per mark), not chart compilation. **Fix shipped:** the builder preview now
renders with **canvas** (`renderSpec(…, { renderer: 'canvas' })`); SVG stays the default for
the editor's LivePreview and for image export. Contract divergence recorded in
`architecture/05` §2.

**Canvas max-dimension guard (measured, not guessed).** Canvas (unlike SVG) has a hard
max side (~32k px), so a chart that resolves taller than that fails to allocate (the
broken-image icon). The cause is **physical render size, not cardinality** — a vertical bar
with thousands of _X_ bands renders fine (width is container-bounded); only an unbounded
band axis (e.g. a horizontal bar's _Y_) overflows. So `renderSpec` now runs a **headless
(`'none'`) layout probe** for canvas charts, reads the chart's resolved **height**, and
throws `ChartTooLargeError(heightPx, limitPx)` when it exceeds `MAX_CANVAS_PX ÷ dpr`. The
builder catches it and shows the real numbers ("would be ~200,000px tall — larger than the
browser can draw on a canvas (~16,383px max here); aggregate or filter"). The earlier
band-count proxy (`previewBandCount`/`MAX_PREVIEW_BANDS`) was removed — readability
(cardinality) stays a `builderWarnings` concern; the render-size limit is now measured at
its true cause.

### Phase 4 — Beyond "moderately capable" (gated — decide later)

These exceed the stated ceiling. List them so they have a home, but **do not commit them in
this push** — revisit once Phases 1–3 land and we see real usage. Each must clear the
guardrail: _promote a control only when it is **both common AND awkward in JSON**._

- **More channels** — `theta` (unlocks pie/donut → _true_ part-to-whole), `opacity`,
  `shape`. (`theta` is the most defensible — it closes a real coverage gap.) Ride on 2A/2B.
- **Faceting (Row / Column → small multiples)** — research §8 B8. The clean way to compare
  many categories. Design crux: VL facets default to **shared scales** (keep that default);
  expose an "independent axes" toggle (`resolve.scale`) only as advanced. **Verify against
  the preview's `"container"` fit modes** (per-cell sizing on facets is finicky).
- **Light styling / scale controls** — colour-scheme picker (categorical / sequential /
  diverging), measure-axis `zero` / `log` toggle, custom axis title, legend title / hide.
  Implement as **auto-derived override panels** that start empty (inheriting VL defaults)
  and emit JSON **only when touched**; clearing a channel **drops its overrides** (Lyra
  §3.9 `cleanupUnused` — no orphaned `scale`/`axis` in the output spec).
- **Builder undo/redo / "reset to smart defaults"** — Lyra §3.11. Low priority (the modal
  is short-lived; the smart default already gives a sane start).
- **Lookup / join a second dataset** — Lyra §3.12; data-model change (multi-dataset
  snippets). Larger, separate effort.
- **Transform-aware data inspector** — evolve 1D from raw-source rows to the _resolved,
  post-transform_ data (filtered rows + calculated columns), and generalize it to a
  togglable data panel below the **main** Live Preview (vega-editor's "Data Viewer", a
  debugging aid for any snippet, not only builder output). Reads runtime rows via
  `view.data(name)` through the `RenderHandle`. Cross-cutting (editor + builder), so it
  has its own home: **[`docs/data-inspector-exploration.md`](./data-inspector-exploration.md)**.

---

## 4. Recommended build order

```
Phase 1  1A actionable hints      ✓ done
         1B per-chart export      ✓ done
         1C filter (+ calculate)  ✓ done
         1D data preview          ✓ done
         1E expr-validate         ✓ done (syntax + unknown-field; completion popup deferred)
Phase 2  2A value-or-field channels (Property model)  ✓ done
         2B field shelf + in-place type cycling       ✓ done (field-first + on-chart shelves)
Phase 3  3D entry points & discoverability         ← next, with/just before 3A (its prerequisite)
         3A intent-first front door (Tier C)       ← built on 2B; the defining feature
         3B starter examples
         3C open in builder (strict hydration)     ← added 2026-06-11; pairs with 3B
Phase 4  (gated) theta/facets/styling-overrides/undo/lookup — decide after Phase 3

Also shipped (builder UX/perf, from dogfooding): near-fullscreen modal, internal-scroll
panes, backdrop-dismiss guard, render diagnostics, canvas preview + canvas-size guard,
data-aware default pre-population. See "Builder UX & perf" above.
```

Rationale for the order: Phase 1 is the cheapest large quality jump and needs no new
interaction model, so it ships value while the bigger design settles. Phase 2 is pure
substrate — low _user-visible_ payoff alone, but Phase 3 is much cleaner on top of it than
bolted onto the channel-first UI. Phase 3 delivers the brief's headline ("recommendations
on the fly") — 3D first, because a front door nobody finds delivers nothing: the entry
points make the builder the app's guided creation flow, then 3A gives that flow its
intent-first opening. Phase 4 is deliberately deferred to protect the "moderately capable"
ceiling.

---

## 5. Constraints & anti-scope (the ceiling)

What "moderately capable" rules **out** — load-bearing, from `lyra-review.md` §2.1/§4:

- **No everything-inspector.** Lyra surfaces ~50 direct controls per primitive because the
  GUI _was_ its document. We split the work on purpose: a small **guarded** builder + a
  first-class **Monaco** editor for the long tail. Styling/scale/axis breadth for its own
  sake belongs in Monaco, not the builder.
- **The promotion test:** a control enters the builder only when it is **both common AND
  awkward in JSON**. Otherwise it stays in Monaco.
- **JSON stays the source of truth.** The builder _emits_ spec; it is never the document
  (the Lyra one-way-export trap that forbids round-trips).
- **No interaction-by-demonstration, no direct-manipulation canvas, no general
  data-pipeline editor.** If we ever add interactivity, expose VL `params`/selections as a
  small guarded action — never port Lyra's signal generator.
- **Take ideas from the reference clones, read no code into the repo** (`AGENTS.md`: no
  shared lib; patterns adapted, not imported).

---

## 6. Cross-cutting notes

- **Spec deltas:** 1A (minor §06), 1B (§08 + §02/§03), 1C (new §06 transforms subsection),
  1D (§06 + §05), 2A/2B (§06 layout), 3A (substantial §06 intent front-door amendment), 3B
  (§02). Per _"spec follows code now"_: build the decision, then amend the spec to match —
  don't let code and §06 drift.
- **Council:** auto-fires on guidance copy and new interactive-widget keyboard/focus work —
  so 1A (hint affordance + copy) and 3A (front-door flow + copy) both go through `/council`
  before committing. It advises; architecture 09/10 decide.
- **Verification:** pure rules get `chart-builder.test.ts` cases; every UI/affordance change
  gets a manual pass against the _live_ builder — a green build proves nothing about what
  the user sees (`AGENTS.md`; `docs/manual-verification.md`). Items 1A and 1C are mostly
  `src/core/`, squarely the "core-first, tested hardest" rule.
- **Source of truth going forward:** this doc. `chart-builder-research.md` §8 and
  `lyra-review.md` §5 remain the _research record_; their forward sequences are superseded
  by §4 here.