astrolabe/docs/chart-builder-enhancement-scope.md

# Chart Builder — Enhancement Scope

> **Status:** scope consolidated 2026-06-10. This is the **single forward-looking home**
> for chart-builder enhancement work — it merges the research backlog from
> [`chart-builder-research.md`](./chart-builder-research.md) §8 (the M4 decision and its
> deferred items) with the interaction ideas from [`lyra-review.md`](./lyra-review.md) §5,
> read against the current spec ([`spec/06-chart-builder.md`](./spec/06-chart-builder.md))
> and the shipped code (`src/core/chart-builder.ts`).
>
> **Goal (the brief):** a **rapid, intuitive GUI for building Vega-Lite specs**, with
> **guidance and recommendations on the fly**, **moderately capable** — not a full
> visual-design IDE.
>
> **Decision (2026-06-10):** push the builder from its shipped **Tier B** ("smart +
> guarded") up to **Tier C** ("intent-first aid"). "Moderately capable" is the ceiling:
> we add the controls that are _both common and awkward in JSON_ and stop there; the long
> tail of styling/scale/axis breadth stays in Monaco. The two source docs remain the
> research record (the _why_ and the citations); this doc is the _plan_ (the _what next_
> and the _order_).

---

## Status log

Newest first. The at-a-glance build-order tracker is §4; per-item detail is §3. This log is
the quick "where are we" — read it first.

- **2026-06-10** — **Up next: 1B (per-chart export).**
  - **1A · Actionable hints** shipped: one-click fixes on guidance warnings
    (`BuilderWarning.fixes` + `applyWarningFix`), council-reviewed, with focus/announce a11y.
  - **Builder UX/perf batch** (from dogfooding the Superstore dataset) shipped: near-fullscreen
    `xlarge` modal tier; panes scroll internally (preview in its own viewport); backdrop-dismiss
    guard (`dismissOnBackdrop: false`); render-timing diagnostics; **canvas** preview renderer
    (SVG stays default elsewhere) + **canvas max-dimension guard** (`ChartTooLargeError` via a
    headless probe); **data-aware default pre-population** (`smartDefaultEncodings`).
  - Scope consolidated and **Tier-C target** set (this doc created); the research/Lyra forward
    sequences are superseded by §4 here.

---

## 1. Where the builder is today (the floor — don't rebuild)

Tier B is shipped and tested (M4 done). The intelligence that decides _which chart and
why_ already exists in `src/core/chart-builder.ts`:

- **Smart default mark** from the (X, Y) field-type shape — not unconditionally Bar
  (`defaultMark`).
- **Valid-type-only** field-type menus per column + **Size discipline** (Nominal /
  Temporal / negative-extent columns are _blocked_ on Size, not merely warned)
  (`validFieldTypes`, `isChannelTypeAllowed`).
- **Non-blocking guidance** — `builderWarnings` (6 rules: line/area needs both axes,
  all-categorical, two-measures-want-scatter, area-split-many-series, crowded category
  axis, negative-size guard).
- **Per-channel transforms** — Aggregate / Bin / `timeUnit`, chart-level Sort / Stack,
  and a field-less "Count of records" measure.
- **Swap X/Y**, debounced live preview, validation gate (≥1 channel mapped).

Everything in §3 below is **backlog** — verified not yet built: `BuilderWarning` has no
`fix` field, there is no per-chart export, no top-level `transform`, no data-table
preview, no channels beyond X/Y/Color/Size, no styling/scale controls.

---

## 2. The target experience (Tier C, concretely)

Reading the brief's four words against the research:

| Brief word                                | What it means here                                                            | The levers (from §3)                                                                                          |
| ----------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Rapid**                                 | shortest path from "a dataset" to "a chart I'd keep"                          | smart defaults (have), intent front door, starter examples, field shelf                                       |
| **Intuitive**                             | matches how people think ("I have fields; what shows my point?")              | intent front door, field shelf, data preview, value-or-field channels                                         |
| **Guidance & recommendations on the fly** | the app proposes and corrects, not just validates                             | **actionable hints**, **intent front door** (the defining Tier-C feature)                                     |
| **Moderately capable**                    | covers the common data-shaping + a few high-value encodings; _not_ every knob | filter / calculate, a _small_ set of promoted controls — and a hard stop short of Lyra's everything-inspector |

The defining shift from B→C is the **intent-first front door**: a _"what do you want to
show?"_ entry (FT / Datawrapper intent categories) that recommends a mark + channel layout
from `intent × column-types`, instead of starting the user at a blank mark picker. It is
the feature the brief most directly asks for, and it is the one piece the M4 research
_deferred_. Choosing Tier C is choosing to pull it forward.

Tier C **extends** §06 — it does not replace the mark-first builder. The front door is an
on-ramp; the user can still ignore it and drive the channels directly, and can always drop
to Monaco. This preserves Astrolabe's core invariant: **the JSON spec is the source of
truth; the builder is a view that emits it** (the Lyra anti-lesson — never let the GUI
become the document).

---

## 3. The consolidated enhancement set

Organized into build phases by dependency and value. Each item carries its **source**,
**value/effort**, **code home**, and **spec impact**. Phases 1–3 are the committed Tier-C
scope; Phase 4 is explicitly _beyond_ "moderately capable" and gated on a later decision.

### Phase 1 — Guidance + I/O (no new interaction model) — _do first_

High value-to-effort, mostly pure-core + thin UI, no architectural change. These make the
_current_ builder dramatically better and de-risk the bigger phases.

**1A · Actionable hints** — _done (2026-06-10)_
Turn advisory warnings into one-click fixes. Source: Lyra §3.1 (`Hints` carries an
`action`). Several existing warnings have an obvious remedy:

- _"draws one mark per row → "_ **[Aggregate as Sum]** (set the measure's `aggregate`)
- _"long labels → "_ **[Swap X/Y]** (the action already exists — just wire it)
- _"two measures usually read as a scatter → "_ **[Switch to Point]**
- _"area split into many series → "_ **[Stack]** or **[Remove colour]**

Implementation: extended `BuilderWarning` with `fixes?: BuilderWarningFix[]` (`{ label, apply }`, pure,
unit-tested in `chart-builder.test.ts`); the modal renders each as a ghost button wired to a
new `applyWarningFix` store action. Wired fixes: **[Aggregate as Sum]** + **[Swap X/Y]**
(one-mark-per-row), **[Swap X/Y]** (high-cardinality axis), **[Switch to Point]** (two
measures), **[Stack]** + **[Remove colour]** (area split — and a stacked area is no longer
flagged, so [Stack] resolves it). Council run (Carbon Actionable notification + APG Alert):
ghost buttons, remedy-in-button, polite "Applied: …" announcement, focus moved off the
removed button — resolution recorded in `architecture/10` §5. §06 "Guidance" amended to
document the one-click fixes.

**1B · Per-chart export** — _highest value-to-effort overall_
Today export is workspace-backup only (§08); there is **no way to get one chart out**.
Source: Lyra §3.8. The renderer already holds the live Vega `view`
(`services/chart-renderer.ts`), so this is small:

- **Copy spec** (clipboard) + **Download `.vl.json`** for the active snippet (trivial — the
  snippet _is_ the spec)
- **Download PNG / SVG** via `view.toImageURL('png' | 'svg')`
- _(optional)_ standalone HTML (ties to the BYO-cloud "private-move" direction)

Home: a **snippet-level** "Export / Share" affordance (library row action or editor
toolbar), distinct from the workspace Export. _Spec impact: new export surface in §08;
arguably §02/§03 (where the affordance lives)._ Note: not strictly a _builder_ feature, but
the biggest single miss adjacent to it — sequence it here.

**1C · Filter (+ Calculate) dataset transforms** — _closes the loop the builder's own warnings open_
The transform layer the builder doesn't touch: top-level `transform: []`. Source: Lyra
§3.12. The builder _already tells users to filter_ in three warnings
(`chart-builder.ts:475,476,511`) while offering no way to do it.

- **Filter** _(highest)_ — a guarded **field + operator + value** predicate shelf
  (Voyager-style, no expression needed for the common case); power form is a raw `datum.…`
  expression validated with Vega's `parseExpr` (see 1E). VL applies top-level transforms
  _before_ encoding aggregation, so "filter raw rows, then aggregate" is the natural
  default; filtering on an aggregated value (HAVING) is the advanced case — defer.
- **Calculate / derived field** _(second)_ — `transform: [{calculate, as}]`; the new field
  then appears in column dropdowns like any other.
- **Lookup / join a second dataset** — larger data-model change (one dataset per snippet
  today) → **defer to Phase 4**.

Home: a new **"Data" section in the builder's left pane, above the channels** (`here are
your rows [+ Filter] [+ Calculate] → now encode them`), paired with 1D. _Spec impact: new
§06 "Data / transforms" subsection — this is genuinely new behaviour, write it._

**1D · Data-table preview** — _closes a confirmed gap; pairs with 1C_
Neither the builder nor the Datasets manager ever shows the **actual rows**. Source: Lyra
§3.2. A compact, **read-only** first-N-rows grid with a **per-column type chip** in each
header lets users sanity-check inferred types _before_ building — exactly when inference is
most likely to surprise. Type + cardinality + extent already come from `profile.ts`; we
only need the row sample. Homes: a collapsible "Data" strip in the builder's left pane
(under the dataset name) and/or the Datasets manager. Keep it read-only (editing data is
out of scope). _Spec impact: §06 + §05 (Datasets) additions._

**1E · Inline expression validation + field autocomplete** — _build with 1C, not standalone_
When 1C's expression mode lands, validate the Vega/VL expression string with the library's
own `parseExpr` (Lyra §3.6) and surface errors inline; autocomplete the dataset's own
column names (we have the schema from `profile.ts`) (Lyra §3.10). Record the `parseExpr`
technique in `architecture/08`. _Spec impact: folded into 1C._

### Phase 2 — Interaction substrate (enables Tier C)

Two changes to _how the user touches fields_. They have standalone value but their main job
is to be the substrate Phase 3 (and any future added channels) stands on — sequence them
here so Tier C lands cleanly.

**2A · Value-or-field channels (the Property model)** — _capability gain; medium_
Source: Lyra §3.3 (`Property.tsx` — one droppable control that is _either_ a literal value
_or_ a bound field). Today a channel is field-only. Let a channel also hold a **constant
`value`** (fixed colour / size) with one consistent control and a chip showing the binding
kind. VL encodes exactly this (`field` vs `value` vs `datum`). Generalizes cleanly to any
channel we add later. _Spec impact: §06 "Encoding channels" — a channel may carry a
constant._

**2B · Field shelf + in-place type cycling** — _the largest interaction shift; the Tier-C/facet substrate_
Source: Lyra §3.4 (drop-zones) + Voyager (field list with type chips) + Lyra §3.5
(`FieldType` — the type icon _is_ the control, click cycles N→O→Q→T within the valid set).
Flip from **channel-first** ("pick a channel, then its column") to **field-first** ("here
are your columns — drag/click onto channels"), matching how people think. This is the
natural way to assign many fields across many channels, so it is the **interaction
substrate for Tier C and faceting**, not a standalone task. _Spec impact: §06 layout
revision (field shelf alongside the channel rows)._

### Phase 3 — Tier C (the intent-first front door) — _the defining feature of this push_

**3A · Intent-first front door** — _the B→C step_
Source: research §5/§8 Tier C (FT Visual Vocabulary + Datawrapper intent taxonomy). A
_"what do you want to show?"_ entry mapping **intent × column types → recommended mark +
channel layout**:

| Intent (FT / Datawrapper)  | Our expression (within 5 marks / our channels)                      |
| -------------------------- | ------------------------------------------------------------------- |
| **Magnitude / Comparison** | Bar (x=N, y=Q; horizontal for long labels)                          |
| **Ranking**                | Bar, sorted by value                                                |
| **Change over time**       | Line (x=T, y=Q; color=N for series)                                 |
| **Correlation**            | Point (x=Q, y=Q); Circle + size=Q for a 3rd measure                 |
| **Distribution**           | Bar of binned counts (histogram) — uses Bin                         |
| **Part-to-whole**          | Stacked / 100% bar (uses Stack); _true pie needs `theta` → Phase 4_ |
| **Deviation**              | diverging signed Bar                                                |

Honest coverage gaps stay honest (Spatial / Flow excluded; Part-to-whole partial until
`theta`). The front door is an **on-ramp, not a gate** — it pre-populates the mark-first
builder, which the user can then adjust or ignore. Munzner's typology and Wilke's directory
become seatable council sources at this point (research §2). Built **on the 2B field
shelf**. **Run the front-door copy + flow through `/council`.** _Spec impact: substantial
§06 amendment — a new "Intent" front-door subsection; the M4 spec note explicitly flagged
this as the deferred tier, so this is the planned amendment, not drift._

**3B · Starter examples gallery** — _cheap; pairs with 3A_
Source: Lyra §3.7. A small set of **curated starter snippets**, one per covered FT intent
(Magnitude/Bar, Change-over-time/Line, Correlation/Point, Distribution/histogram,
Part-to-whole/stacked). Improves first-run, doubles as living documentation of what the app
does well. Natural home: the snippet library. _Spec impact: §02 (library seed content)._

### Builder UX & perf — in-flight fixes (2026-06-10, from dogfooding the Superstore dataset)

Pre-existing builder rough edges surfaced while testing on a 10k-row / ~24-col dataset.
Fixed in this batch (not part of 1A–3B, but the same surface):

- **Modal is a near-fullscreen work surface** — new `xlarge` shell tier (`ModalShell`,
  `min(1800px, 96vw) × min(1100px, 92vh)`); the Chart Builder no longer wastes screen.
- **Panes scroll internally, modal keeps its shape** — the `.builder` grid fills the body
  (`grid-template-rows: minmax(0,1fr)`), the config pane and the **preview** each scroll in
  their own viewport, so a tall one-mark-per-row chart scrolls inside the preview instead of
  pushing Create/Cancel below the fold.
- **Backdrop click no longer discards in-progress work** — `dismissOnBackdrop: false` on the
  builder (registry flag); Escape and × still close.
- **Data-aware default pre-population** — the builder no longer blindly takes the first two
  columns (which opened Superstore on a 9994-bar degenerate chart). When the dataset is
  profiled, `defaultBuilderConfig` picks a "safest bet": a low-cardinality category vs a
  count of records (tidy bar), else a time series of the first measure, else a scatter — each
  guaranteed to render. Falls back to positional when there are no stats. Pure + tested. This
  composes with (isn't replaced by) the future intent-first front door — the builder always
  needs a sane opening state.
- **Render-timing diagnostics** — `BuilderPreview` logs `parse · prepare · destroy · embed ·
paint · total` (+ mark, row count) to the console (dev always; prod only when slow). The
  **paint** phase (a double-rAF after `embed()`) captures the real freeze.

**Perf finding — confirmed and fixed.** Diagnostics on the Superstore dataset:
`embed 237ms · paint 6458ms` for the default one-bar-per-row chart (9994 rows), vs
`paint 6ms` once grouped to a few categories. The freeze was entirely **SVG layout/paint**
(one DOM node per mark), not chart compilation. **Fix shipped:** the builder preview now
renders with **canvas** (`renderSpec(…, { renderer: 'canvas' })`); SVG stays the default for
the editor's LivePreview and for image export. Contract divergence recorded in
`architecture/05` §2.

**Canvas max-dimension guard (measured, not guessed).** Canvas (unlike SVG) has a hard
max side (~32k px), so a chart that resolves taller than that fails to allocate (the
broken-image icon). The cause is **physical render size, not cardinality** — a vertical bar
with thousands of _X_ bands renders fine (width is container-bounded); only an unbounded
band axis (e.g. a horizontal bar's _Y_) overflows. So `renderSpec` now runs a **headless
(`'none'`) layout probe** for canvas charts, reads the chart's resolved **height**, and
throws `ChartTooLargeError(heightPx, limitPx)` when it exceeds `MAX_CANVAS_PX ÷ dpr`. The
builder catches it and shows the real numbers ("would be ~200,000px tall — larger than the
browser can draw on a canvas (~16,383px max here); aggregate or filter"). The earlier
band-count proxy (`previewBandCount`/`MAX_PREVIEW_BANDS`) was removed — readability
(cardinality) stays a `builderWarnings` concern; the render-size limit is now measured at
its true cause.

### Phase 4 — Beyond "moderately capable" (gated — decide later)

These exceed the stated ceiling. List them so they have a home, but **do not commit them in
this push** — revisit once Phases 1–3 land and we see real usage. Each must clear the
guardrail: _promote a control only when it is **both common AND awkward in JSON**._

- **More channels** — `theta` (unlocks pie/donut → _true_ part-to-whole), `opacity`,
  `shape`. (`theta` is the most defensible — it closes a real coverage gap.) Ride on 2A/2B.
- **Faceting (Row / Column → small multiples)** — research §8 B8. The clean way to compare
  many categories. Design crux: VL facets default to **shared scales** (keep that default);
  expose an "independent axes" toggle (`resolve.scale`) only as advanced. **Verify against
  the preview's `"container"` fit modes** (per-cell sizing on facets is finicky).
- **Light styling / scale controls** — colour-scheme picker (categorical / sequential /
  diverging), measure-axis `zero` / `log` toggle, custom axis title, legend title / hide.
  Implement as **auto-derived override panels** that start empty (inheriting VL defaults)
  and emit JSON **only when touched**; clearing a channel **drops its overrides** (Lyra
  §3.9 `cleanupUnused` — no orphaned `scale`/`axis` in the output spec).
- **Builder undo/redo / "reset to smart defaults"** — Lyra §3.11. Low priority (the modal
  is short-lived; the smart default already gives a sane start).
- **Lookup / join a second dataset** — Lyra §3.12; data-model change (multi-dataset
  snippets). Larger, separate effort.

---

## 4. Recommended build order

```
Phase 1  1A actionable hints      ✓ done
         1B per-chart export      ← next: highest value-to-effort
         1C filter (+ calculate)  ← closes the loop on warnings the builder already emits
         1D data preview          ← pairs with 1C
         1E expr-validate + autocomplete (with 1C)
Phase 2  2A value-or-field channels (Property model)
         2B field shelf + in-place type cycling   ← Tier-C substrate
Phase 3  3A intent-first front door (Tier C)       ← built on 2B; the defining feature
         3B starter examples
Phase 4  (gated) theta/facets/styling-overrides/undo/lookup — decide after Phase 3

Also shipped (builder UX/perf, from dogfooding): near-fullscreen modal, internal-scroll
panes, backdrop-dismiss guard, render diagnostics, canvas preview + canvas-size guard,
data-aware default pre-population. See "Builder UX & perf" above.
```

Rationale for the order: Phase 1 is the cheapest large quality jump and needs no new
interaction model, so it ships value while the bigger design settles. Phase 2 is pure
substrate — low _user-visible_ payoff alone, but Phase 3 is much cleaner on top of it than
bolted onto the channel-first UI. Phase 3 delivers the brief's headline ("recommendations
on the fly"). Phase 4 is deliberately deferred to protect the "moderately capable" ceiling.

---

## 5. Constraints & anti-scope (the ceiling)

What "moderately capable" rules **out** — load-bearing, from `lyra-review.md` §2.1/§4:

- **No everything-inspector.** Lyra surfaces ~50 direct controls per primitive because the
  GUI _was_ its document. We split the work on purpose: a small **guarded** builder + a
  first-class **Monaco** editor for the long tail. Styling/scale/axis breadth for its own
  sake belongs in Monaco, not the builder.
- **The promotion test:** a control enters the builder only when it is **both common AND
  awkward in JSON**. Otherwise it stays in Monaco.
- **JSON stays the source of truth.** The builder _emits_ spec; it is never the document
  (the Lyra one-way-export trap that forbids round-trips).
- **No interaction-by-demonstration, no direct-manipulation canvas, no general
  data-pipeline editor.** If we ever add interactivity, expose VL `params`/selections as a
  small guarded action — never port Lyra's signal generator.
- **Take ideas from the reference clones, read no code into the repo** (`AGENTS.md`: no
  shared lib; patterns adapted, not imported).

---

## 6. Cross-cutting notes

- **Spec deltas:** 1A (minor §06), 1B (§08 + §02/§03), 1C (new §06 transforms subsection),
  1D (§06 + §05), 2A/2B (§06 layout), 3A (substantial §06 intent front-door amendment), 3B
  (§02). Per _"spec follows code now"_: build the decision, then amend the spec to match —
  don't let code and §06 drift.
- **Council:** auto-fires on guidance copy and new interactive-widget keyboard/focus work —
  so 1A (hint affordance + copy) and 3A (front-door flow + copy) both go through `/council`
  before committing. It advises; architecture 09/10 decide.
- **Verification:** pure rules get `chart-builder.test.ts` cases; every UI/affordance change
  gets a manual pass against the _live_ builder — a green build proves nothing about what
  the user sees (`AGENTS.md`; `docs/manual-verification.md`). Items 1A and 1C are mostly
  `src/core/`, squarely the "core-first, tested hardest" rule.
- **Source of truth going forward:** this doc. `chart-builder-research.md` §8 and
  `lyra-review.md` §5 remain the _research record_; their forward sequences are superseded
  by §4 here.