astrolabe/docs/chart-builder-enhancement-scope.md

Chart Builder — Enhancement Scope

Status: scope consolidated 2026-06-10. This is the single forward-looking home for chart-builder enhancement work — it merges the research backlog from chart-builder-research.md §8 (the M4 decision and its deferred items) with the interaction ideas from lyra-review.md §5, read against the current spec (spec/06-chart-builder.md) and the shipped code (src/core/chart-builder.ts).

Goal (the brief): a rapid, intuitive GUI for building Vega-Lite specs, with guidance and recommendations on the fly, moderately capable — not a full visual-design IDE.

Decision (2026-06-10): push the builder from its shipped Tier B ("smart + guarded") up to Tier C ("intent-first aid"). "Moderately capable" is the ceiling: we add the controls that are both common and awkward in JSON and stop there; the long tail of styling/scale/axis breadth stays in Monaco. The two source docs remain the research record (the why and the citations); this doc is the plan (the what next and the order).

---

Status log

Newest first. The at-a-glance build-order tracker is §4; per-item detail is §3. This log is the quick "where are we" — read it first.

• 2026-06-10 — Up next: 1C (filter + calculate transforms), paired with 1D (data preview).
  • 1B · Per-chart export shipped: an Export disclosure in the Live Preview header (distinct from the workspace Export) — Copy spec + Download JSON (.vl.json) of the shown text, and Download PNG/SVG of the live chart. Image output rides a new RenderHandle.toImageURL(format, { scale, background }) (PNG via view.toCanvas → blob: URL; SVG via view.toSVG → data: URL), so no component touches the Vega view. Filenames derive from the snippet name — filesystem-safe, script-preserving (pure core/chart-export.ts, tested). Home: the preview header, not the plan's "library row / editor toolbar" — image export needs the live view; the disclosure-of-controls (not an ARIA menu) mirrors SettingsPopover.
    • Export options (from first-round feedback): PNG Resolution 1×/2×/3× is a multiplier of devicePixelRatio, so the default 1× is Retina-crisp — the soft-1× export was a dpr bug (raw toImageURL scaleFactor ignores dpr). Background Theme(default)/White/None fixes transparent PNGs (the chart config is transparent so the on-screen pane colour shows; export composites the chosen colour under the PNG / adds an SVG <rect>). Referenced data Inline(default)/Keep refs (shown only when the spec references saved datasets) inlines dataset values so the exported spec renders standalone (inlineReferencedDatasets, tested).
    • Spec §08 gained a Per-chart export section (with the options); §04 cross-references it; architecture/05 §2 records the handle's dpr-aware scale + background compositing.
  • 1A · Actionable hints shipped: one-click fixes on guidance warnings (BuilderWarning.fixes + applyWarningFix), council-reviewed, with focus/announce a11y.
  • Builder UX/perf batch (from dogfooding the Superstore dataset) shipped: near-fullscreen xlarge modal tier; panes scroll internally (preview in its own viewport); backdrop-dismiss guard (dismissOnBackdrop: false); render-timing diagnostics; canvas preview renderer (SVG stays default elsewhere) + canvas max-dimension guard (ChartTooLargeError via a headless probe); data-aware default pre-population (smartDefaultEncodings).
  • Scope consolidated and Tier-C target set (this doc created); the research/Lyra forward sequences are superseded by §4 here.

---

1. Where the builder is today (the floor — don't rebuild)

Tier B is shipped and tested (M4 done). The intelligence that decides which chart and why already exists in src/core/chart-builder.ts:

• Smart default mark from the (X, Y) field-type shape — not unconditionally Bar (defaultMark).
• Valid-type-only field-type menus per column + Size discipline (Nominal / Temporal / negative-extent columns are blocked on Size, not merely warned) (validFieldTypes, isChannelTypeAllowed).
• Non-blocking guidance — builderWarnings (6 rules: line/area needs both axes, all-categorical, two-measures-want-scatter, area-split-many-series, crowded category axis, negative-size guard).
• Per-channel transforms — Aggregate / Bin / timeUnit, chart-level Sort / Stack, and a field-less "Count of records" measure.
• Swap X/Y, debounced live preview, validation gate (≥1 channel mapped).

Everything in §3 below is backlog — verified not yet built: BuilderWarning has no fix field, there is no per-chart export, no top-level transform, no data-table preview, no channels beyond X/Y/Color/Size, no styling/scale controls.

---

2. The target experience (Tier C, concretely)

Reading the brief's four words against the research:

Brief word	What it means here	The levers (from §3)
Rapid	shortest path from "a dataset" to "a chart I'd keep"	smart defaults (have), intent front door, starter examples, field shelf
Intuitive	matches how people think ("I have fields; what shows my point?")	intent front door, field shelf, data preview, value-or-field channels
Guidance & recommendations on the fly	the app proposes and corrects, not just validates	actionable hints, intent front door (the defining Tier-C feature)
Moderately capable	covers the common data-shaping + a few high-value encodings; not every knob	filter / calculate, a small set of promoted controls — and a hard stop short of Lyra's everything-inspector

The defining shift from B→C is the intent-first front door: a "what do you want to show?" entry (FT / Datawrapper intent categories) that recommends a mark + channel layout from intent × column-types, instead of starting the user at a blank mark picker. It is the feature the brief most directly asks for, and it is the one piece the M4 research deferred. Choosing Tier C is choosing to pull it forward.

Tier C extends §06 — it does not replace the mark-first builder. The front door is an on-ramp; the user can still ignore it and drive the channels directly, and can always drop to Monaco. This preserves Astrolabe's core invariant: the JSON spec is the source of truth; the builder is a view that emits it (the Lyra anti-lesson — never let the GUI become the document).

---

3. The consolidated enhancement set

Organized into build phases by dependency and value. Each item carries its source, value/effort, code home, and spec impact. Phases 1–3 are the committed Tier-C scope; Phase 4 is explicitly beyond "moderately capable" and gated on a later decision.

Phase 1 — Guidance + I/O (no new interaction model) — do first

High value-to-effort, mostly pure-core + thin UI, no architectural change. These make the current builder dramatically better and de-risk the bigger phases.

1A · Actionable hints — done (2026-06-10) Turn advisory warnings into one-click fixes. Source: Lyra §3.1 (Hints carries an action). Several existing warnings have an obvious remedy:

• "draws one mark per row → " [Aggregate as Sum] (set the measure's aggregate)
• "long labels → " [Swap X/Y] (the action already exists — just wire it)
• "two measures usually read as a scatter → " [Switch to Point]
• "area split into many series → " [Stack] or [Remove colour]

Implementation: extended BuilderWarning with fixes?: BuilderWarningFix[] ({ label, apply }, pure, unit-tested in chart-builder.test.ts); the modal renders each as a ghost button wired to a new applyWarningFix store action. Wired fixes: [Aggregate as Sum] + [Swap X/Y] (one-mark-per-row), [Swap X/Y] (high-cardinality axis), [Switch to Point] (two measures), [Stack] + [Remove colour] (area split — and a stacked area is no longer flagged, so [Stack] resolves it). Council run (Carbon Actionable notification + APG Alert): ghost buttons, remedy-in-button, polite "Applied: …" announcement, focus moved off the removed button — resolution recorded in architecture/10 §5. §06 "Guidance" amended to document the one-click fixes.

1B · Per-chart export — done (2026-06-10) Today export was workspace-backup only (§08); there was no way to get one chart out. Source: Lyra §3.8. Shipped as an Export disclosure in the Live Preview header:

• Copy spec (clipboard) + Download .vl.json of the currently-shown text.
• Download PNG / SVG of the live chart via a new RenderHandle.toImageURL wrapping view.toImageURL (PNG at 2× → blob: URL, revoked after download; SVG → data: URL), so the embedding boundary holds — no component touches the raw view.
• Filenames from the snippet name, filesystem-safe and script-preserving (pure core/chart-export.ts, tested). Standalone HTML left out (the deferred optional).

Home decision: the preview header, not the plan's original "library row / editor toolbar" suggestion — the image formats need the live rendered view, and "export this chart" reads best beside the chart. The widget is a disclosure-of-action-buttons (not an ARIA menu), mirroring SettingsPopover and sharing its single-open registry. Spec impact: new §08 "Per-chart export" section; §04 cross-reference; architecture/05 §2 handle note.

1C · Filter (+ Calculate) dataset transforms — closes the loop the builder's own warnings open The transform layer the builder doesn't touch: top-level transform: []. Source: Lyra §3.12. The builder already tells users to filter in three warnings (chart-builder.ts:475,476,511) while offering no way to do it.

• Filter (highest) — a guarded field + operator + value predicate shelf (Voyager-style, no expression needed for the common case); power form is a raw datum.… expression validated with Vega's parseExpr (see 1E). VL applies top-level transforms before encoding aggregation, so "filter raw rows, then aggregate" is the natural default; filtering on an aggregated value (HAVING) is the advanced case — defer.
• Calculate / derived field (second) — transform: [{calculate, as}]; the new field then appears in column dropdowns like any other.
• Lookup / join a second dataset — larger data-model change (one dataset per snippet today) → defer to Phase 4.

Home: a new "Data" section in the builder's left pane, above the channels (here are your rows [+ Filter] [+ Calculate] → now encode them), paired with 1D. Spec impact: new §06 "Data / transforms" subsection — this is genuinely new behaviour, write it.

1D · Data-table preview — closes a confirmed gap; pairs with 1C Neither the builder nor the Datasets manager ever shows the actual rows. Source: Lyra §3.2. A compact, read-only first-N-rows grid with a per-column type chip in each header lets users sanity-check inferred types before building — exactly when inference is most likely to surprise. Type + cardinality + extent already come from profile.ts; we only need the row sample. Homes: a collapsible "Data" strip in the builder's left pane (under the dataset name) and/or the Datasets manager. Keep it read-only (editing data is out of scope). Spec impact: §06 + §05 (Datasets) additions.

1E · Inline expression validation + field autocomplete — build with 1C, not standalone When 1C's expression mode lands, validate the Vega/VL expression string with the library's own parseExpr (Lyra §3.6) and surface errors inline; autocomplete the dataset's own column names (we have the schema from profile.ts) (Lyra §3.10). Record the parseExpr technique in architecture/08. Spec impact: folded into 1C.

Phase 2 — Interaction substrate (enables Tier C)

Two changes to how the user touches fields. They have standalone value but their main job is to be the substrate Phase 3 (and any future added channels) stands on — sequence them here so Tier C lands cleanly.

2A · Value-or-field channels (the Property model) — capability gain; medium Source: Lyra §3.3 (Property.tsx — one droppable control that is either a literal value or a bound field). Today a channel is field-only. Let a channel also hold a constant value (fixed colour / size) with one consistent control and a chip showing the binding kind. VL encodes exactly this (field vs value vs datum). Generalizes cleanly to any channel we add later. Spec impact: §06 "Encoding channels" — a channel may carry a constant.

2B · Field shelf + in-place type cycling — the largest interaction shift; the Tier-C/facet substrate Source: Lyra §3.4 (drop-zones) + Voyager (field list with type chips) + Lyra §3.5 (FieldType — the type icon is the control, click cycles N→O→Q→T within the valid set). Flip from channel-first ("pick a channel, then its column") to field-first ("here are your columns — drag/click onto channels"), matching how people think. This is the natural way to assign many fields across many channels, so it is the interaction substrate for Tier C and faceting, not a standalone task. Spec impact: §06 layout revision (field shelf alongside the channel rows).

Phase 3 — Tier C (the intent-first front door) — the defining feature of this push

3A · Intent-first front door — the B→C step Source: research §5/§8 Tier C (FT Visual Vocabulary + Datawrapper intent taxonomy). A "what do you want to show?" entry mapping intent × column types → recommended mark + channel layout:

Intent (FT / Datawrapper)	Our expression (within 5 marks / our channels)
Magnitude / Comparison	Bar (x=N, y=Q; horizontal for long labels)
Ranking	Bar, sorted by value
Change over time	Line (x=T, y=Q; color=N for series)
Correlation	Point (x=Q, y=Q); Circle + size=Q for a 3rd measure
Distribution	Bar of binned counts (histogram) — uses Bin
Part-to-whole	Stacked / 100% bar (uses Stack); true pie needs theta → Phase 4
Deviation	diverging signed Bar

Honest coverage gaps stay honest (Spatial / Flow excluded; Part-to-whole partial until theta). The front door is an on-ramp, not a gate — it pre-populates the mark-first builder, which the user can then adjust or ignore. Munzner's typology and Wilke's directory become seatable council sources at this point (research §2). Built on the 2B field shelf. Run the front-door copy + flow through /council. Spec impact: substantial §06 amendment — a new "Intent" front-door subsection; the M4 spec note explicitly flagged this as the deferred tier, so this is the planned amendment, not drift.

3B · Starter examples gallery — cheap; pairs with 3A Source: Lyra §3.7. A small set of curated starter snippets, one per covered FT intent (Magnitude/Bar, Change-over-time/Line, Correlation/Point, Distribution/histogram, Part-to-whole/stacked). Improves first-run, doubles as living documentation of what the app does well. Natural home: the snippet library. Spec impact: §02 (library seed content).

Builder UX & perf — in-flight fixes (2026-06-10, from dogfooding the Superstore dataset)

Pre-existing builder rough edges surfaced while testing on a 10k-row / ~24-col dataset. Fixed in this batch (not part of 1A–3B, but the same surface):

• Modal is a near-fullscreen work surface — new xlarge shell tier (ModalShell, min(1800px, 96vw) × min(1100px, 92vh)); the Chart Builder no longer wastes screen.
• Panes scroll internally, modal keeps its shape — the .builder grid fills the body (grid-template-rows: minmax(0,1fr)), the config pane and the preview each scroll in their own viewport, so a tall one-mark-per-row chart scrolls inside the preview instead of pushing Create/Cancel below the fold.
• Backdrop click no longer discards in-progress work — dismissOnBackdrop: false on the builder (registry flag); Escape and × still close.
• Data-aware default pre-population — the builder no longer blindly takes the first two columns (which opened Superstore on a 9994-bar degenerate chart). When the dataset is profiled, defaultBuilderConfig picks a "safest bet": a low-cardinality category vs a count of records (tidy bar), else a time series of the first measure, else a scatter — each guaranteed to render. Falls back to positional when there are no stats. Pure + tested. This composes with (isn't replaced by) the future intent-first front door — the builder always needs a sane opening state.
• Render-timing diagnostics — BuilderPreview logs parse · prepare · destroy · embed · paint · total (+ mark, row count) to the console (dev always; prod only when slow). The paint phase (a double-rAF after embed()) captures the real freeze.

Perf finding — confirmed and fixed. Diagnostics on the Superstore dataset: embed 237ms · paint 6458ms for the default one-bar-per-row chart (9994 rows), vs paint 6ms once grouped to a few categories. The freeze was entirely SVG layout/paint (one DOM node per mark), not chart compilation. Fix shipped: the builder preview now renders with canvas (renderSpec(…, { renderer: 'canvas' })); SVG stays the default for the editor's LivePreview and for image export. Contract divergence recorded in architecture/05 §2.

Canvas max-dimension guard (measured, not guessed). Canvas (unlike SVG) has a hard max side (~32k px), so a chart that resolves taller than that fails to allocate (the broken-image icon). The cause is physical render size, not cardinality — a vertical bar with thousands of X bands renders fine (width is container-bounded); only an unbounded band axis (e.g. a horizontal bar's Y) overflows. So renderSpec now runs a headless ('none') layout probe for canvas charts, reads the chart's resolved height, and throws ChartTooLargeError(heightPx, limitPx) when it exceeds MAX_CANVAS_PX ÷ dpr. The builder catches it and shows the real numbers ("would be ~200,000px tall — larger than the browser can draw on a canvas (~16,383px max here); aggregate or filter"). The earlier band-count proxy (previewBandCount/MAX_PREVIEW_BANDS) was removed — readability (cardinality) stays a builderWarnings concern; the render-size limit is now measured at its true cause.

Phase 4 — Beyond "moderately capable" (gated — decide later)

These exceed the stated ceiling. List them so they have a home, but do not commit them in this push — revisit once Phases 1–3 land and we see real usage. Each must clear the guardrail: promote a control only when it is both common AND awkward in JSON.

• More channels — theta (unlocks pie/donut → true part-to-whole), opacity, shape. (theta is the most defensible — it closes a real coverage gap.) Ride on 2A/2B.
• Faceting (Row / Column → small multiples) — research §8 B8. The clean way to compare many categories. Design crux: VL facets default to shared scales (keep that default); expose an "independent axes" toggle (resolve.scale) only as advanced. Verify against the preview's "container" fit modes (per-cell sizing on facets is finicky).
• Light styling / scale controls — colour-scheme picker (categorical / sequential / diverging), measure-axis zero / log toggle, custom axis title, legend title / hide. Implement as auto-derived override panels that start empty (inheriting VL defaults) and emit JSON only when touched; clearing a channel drops its overrides (Lyra §3.9 cleanupUnused — no orphaned scale/axis in the output spec).
• Builder undo/redo / "reset to smart defaults" — Lyra §3.11. Low priority (the modal is short-lived; the smart default already gives a sane start).
• Lookup / join a second dataset — Lyra §3.12; data-model change (multi-dataset snippets). Larger, separate effort.

---

4. Recommended build order

Phase 1  1A actionable hints      ✓ done
         1B per-chart export      ✓ done
         1C filter (+ calculate)  ← next: closes the loop on warnings the builder already emits
         1D data preview          ← pairs with 1C
         1E expr-validate + autocomplete (with 1C)
Phase 2  2A value-or-field channels (Property model)
         2B field shelf + in-place type cycling   ← Tier-C substrate
Phase 3  3A intent-first front door (Tier C)       ← built on 2B; the defining feature
         3B starter examples
Phase 4  (gated) theta/facets/styling-overrides/undo/lookup — decide after Phase 3

Also shipped (builder UX/perf, from dogfooding): near-fullscreen modal, internal-scroll
panes, backdrop-dismiss guard, render diagnostics, canvas preview + canvas-size guard,
data-aware default pre-population. See "Builder UX & perf" above.

Rationale for the order: Phase 1 is the cheapest large quality jump and needs no new interaction model, so it ships value while the bigger design settles. Phase 2 is pure substrate — low user-visible payoff alone, but Phase 3 is much cleaner on top of it than bolted onto the channel-first UI. Phase 3 delivers the brief's headline ("recommendations on the fly"). Phase 4 is deliberately deferred to protect the "moderately capable" ceiling.

---

5. Constraints & anti-scope (the ceiling)

What "moderately capable" rules out — load-bearing, from lyra-review.md §2.1/§4:

• No everything-inspector. Lyra surfaces ~50 direct controls per primitive because the GUI was its document. We split the work on purpose: a small guarded builder + a first-class Monaco editor for the long tail. Styling/scale/axis breadth for its own sake belongs in Monaco, not the builder.
• The promotion test: a control enters the builder only when it is both common AND awkward in JSON. Otherwise it stays in Monaco.
• JSON stays the source of truth. The builder emits spec; it is never the document (the Lyra one-way-export trap that forbids round-trips).
• No interaction-by-demonstration, no direct-manipulation canvas, no general data-pipeline editor. If we ever add interactivity, expose VL params/selections as a small guarded action — never port Lyra's signal generator.
• Take ideas from the reference clones, read no code into the repo (AGENTS.md: no shared lib; patterns adapted, not imported).

---

6. Cross-cutting notes

• Spec deltas: 1A (minor §06), 1B (§08 + §02/§03), 1C (new §06 transforms subsection), 1D (§06 + §05), 2A/2B (§06 layout), 3A (substantial §06 intent front-door amendment), 3B (§02). Per "spec follows code now": build the decision, then amend the spec to match — don't let code and §06 drift.
• Council: auto-fires on guidance copy and new interactive-widget keyboard/focus work — so 1A (hint affordance + copy) and 3A (front-door flow + copy) both go through /council before committing. It advises; architecture 09/10 decide.
• Verification: pure rules get chart-builder.test.ts cases; every UI/affordance change gets a manual pass against the live builder — a green build proves nothing about what the user sees (AGENTS.md; docs/manual-verification.md). Items 1A and 1C are mostly src/core/, squarely the "core-first, tested hardest" rule.
• Source of truth going forward: this doc. chart-builder-research.md §8 and lyra-review.md §5 remain the research record; their forward sequences are superseded by §4 here.