astrolabe/docs/exploration/chart-builder-research.md

Chart Builder — Design Research (M4)

Status: research complete; informs the M4 build (spec §06). Decision: build Tier B — "smart + guarded" (mark-first, still §06-shaped). Why this doc exists: the Chart Builder is the point where Astrolabe stops being a pass-through JSON editor and starts making chart-shaped suggestions/defaults. "Which chart, and why" becomes a decision the app owns, so we researched it deliberately before building. This is the record of what we studied and what we took from each source — the citations behind every default and guardrail in src/core/chart-builder.ts.

---

1. Scope of the builder (the constraint everything maps into)

Spec §06: compose a Vega-Lite chart from a dataset with one mark ∈ {Bar, Line, Point, Area, Circle}, mapping columns to four channels (X, Y, Color, Size), each carrying a field type ∈ {Quantitative, Nominal, Ordinal, Temporal}, plus optional pixel width/height → a complete spec saved as a snippet that references the dataset by name. Column types are inferred upstream as number | string | date | boolean (src/core/type-inference.ts).

No transforms (no binning, aggregation, stacking, regression), no second axis, no geo. That narrow surface is the lens through which every source below was read: "what does this canon tell us to do within Bar/Line/Point/Area/Circle and X/Y/Color/Size?"

2. The sources

Two kinds: formal CS (how recommendation engines actually rank charts) and chart-choice canon (how practitioners pick). They were chosen for being cloneable/grep-able offline (the council's working model) and authoritative for "which chart," which our other seats (Carbon/GOV.UK/APG/NN/g) don't cover.

Source	What it is	Local path
Draco (uwdata)	Visualization design knowledge as ASP constraints — the formal "what makes a good chart," with hard (validity) + soft (preference) rules and weights, some learned from human perception experiments (Kim 2018, Saket 2018).	reference/draco
Voyager (vega)	UW IDL's recommendation/exploration tool on CompassQL — the interaction model (field shelves, auto-add, type chips) and effectiveness-ranked encoding suggestions.	reference/voyager
FT Visual Vocabulary (Financial Times)	A poster/taxonomy mapping what you want to show (9 data-relationship categories) → chart types. Seated in the council.	reference/chart-doctor/visual-vocabulary/
Datawrapper	Practitioner chart-choice in plain language; intent-first ("the chart's main statement becomes a compass"). Seated (distilled).	reference/principles/datawrapper.md

Theoretical basis, not seated (deliberately): Munzner, Visualization Analysis and Design (marks & channels; the channel-effectiveness rankings — magnitude: position → length → angle → area …; identity: spatial region → hue → shape; expressiveness & effectiveness principles) and Wilke, Fundamentals of Data Visualization (clauswilke/dataviz; example directory by intent + "ugly/bad/wrong" pedagogy). They are the why beneath Draco and Voyager — Draco's soft weights are an operationalization of exactly these Mackinlay/APT/Munzner effectiveness rankings — but they restate the same rules the seated sources already give us, so seating them would add overlap, not coverage. Cited here as grounding; revisit if we ever build the intent-first "Tier C" front door, where Munzner's typology and Wilke's directory would earn their place.

3. What we take from each source

From Draco — validity guardrails + a preference ranking (the rigorous core)

Draco models a chart as ASP facts and rejects/ranks them with hard (∞ cost) and soft (weighted) constraints (asp/optimize.lp). We can't ship an ASP solver in a browser, but the rules are a lookup table. The portable subset:

• Hard validity (block in the UI): reference/draco/asp/hard.lp
  • Quantitative on a string/boolean column — illegal (:6). Temporal only on a datetime column (:7).
  • Size encoding a Nominal field — illegal ("size implies order; nominal is misleading", :53). Size cannot encode negative values (:56). Size only on point/text marks (:110).
  • Bar/Area must include a zero baseline on the measure axis (:103-104).
  • Bar needs a categorical axis — both x and y continuous on a bar is malformed (:97); Line/Area need both x and y, and not both discrete (:91,:94).
  • Same field on x and y — illegal (:122). >20 categorical colors — illegal (:172).
• Soft preference (the weights, asp/weights.lp + asp/soft.lp):
  • Channel-by-type appropriateness (lower = better): continuous data is free on x/y, costs to put on color (10) or size (1); nominal cheapest on y then x then color; ordered data expensive on size. → fill X/Y before Color/Size.
  • Mark by data shape: continuous×continuous → point (line/area heavily penalized); continuous×discrete aggregated → bar; discrete×discrete → point/rect.
  • Prefer time on x (temporal_y, :147); never type a number as nominal (number_nominal, weight 10); the loudest nudge is an all-discrete chart with no measure (only_discrete, weight 30).

The hand-tuned weights.lp is the portable "common-sense" set; the learned weights_learned.lp corroborates direction, not magnitude.

From Voyager — the interaction model + the valid-type table

• getValidTypes (src/components/data-pane/field-list.tsx:140-155) — adopted almost verbatim: number→{quantitative, nominal}, integer→{quantitative, nominal}, datetime→{temporal}, string→{nominal}, boolean→{nominal}. The type toggle shows only when ≥2 valid types exist. (We extend slightly — see §4 — to also offer Ordinal, which Voyager deliberately omits, encoding.ts:131-134.)
• Auto-add / "auto" mark (models/shelf/index.ts:72-81): Voyager lets a field be added with channel:'?' and asks CompassQL to place it by effectiveness. The small builder analogue is a non-empty smart default (defaultBuilderConfig) so the preview is never blank.
• Type chips + swap: per-field type indicator with a click-to-change popover, and a cheap x↔y swap (Voyager's SPEC_FIELD_MOVE is remove-both + re-add).
• Out of scope (Voyager scope creep we reject): wildcard shelves, the full Related Views gallery, faceting (row/column), and embedding CompassQL/compassql@0.20.2 itself. We hand-roll a small decision table in src/core/ instead of pulling the engine.

From FT Visual Vocabulary — the intent→chart taxonomy (and our coverage gaps)

reference/chart-doctor/visual-vocabulary/README.md (taxonomy is prose). Nine categories; mapped to our five marks:

FT category	What it shows	Our expression
Magnitude	size comparisons	Bar (x=N, y=Q; horizontal x=Q, y=N for long labels) — primary
Ranking	position in an ordered list	Bar, sorted by value (the sort is the feature)
Change over Time	trends	Line (x=T, y=Q; color=N for series); Bar/Area alternatives, single series
Correlation	relationship of 2+ measures	Point (x=Q, y=Q); Circle/bubble + size=Q for a third measure
Deviation	+/− from a reference	Bar with signed Q (diverging bar only)
Distribution	spread/frequency	weak: raw Point strip, or Bar of pre-binned counts (no bin transform)
Part-to-whole	component shares	none well — redirect to Magnitude/Bar; we can't show true proportions
Spatial	geography	none — exclude
Flow	movement between states	none — exclude

Coverage: strong on Magnitude, Ranking, Change-over-Time, Correlation; partial on Deviation/Distribution; none on Part-to-whole/Spatial/Flow. Honest gaps, not silent degradation.

From Datawrapper — plain-language rules + intent labels

reference/principles/datawrapper.md. Corroborates the same default-mark-by-intent table (comparison→Bar, time→Line, correlation→Point/bubble) and supplies friendlier intent words (Developments over time / Shares / Comparison / Correlation). Bindable rules: bar is the safe default; bar over column on small screens; line for continuous time, columns for a few points; circles are hard to compare precisely; size encodes a quantity; area = single total (warn on multi-series).

4. The convergent rules — what all four agree on (high-confidence)

These are not a judgment call; the formal engines and the practitioner canon land on the same place. They are the spec for src/core/chart-builder.ts:

1. Column type → valid field types (Voyager getValidTypes; Draco hard.lp:6-7): number→{Quantitative (default), Ordinal, Nominal}; date→{Temporal only}; string→{Nominal (default), Ordinal}; boolean→{Nominal}. Never offer Q for string/boolean, never Temporal for a non-date. (We add Ordinal where it's a defensible user assertion of order; Voyager omits it for UX simplicity — our deliberate superset.)
2. Default mark from the (X, Y) shape (Draco mark-by-shape; Voyager effectiveness; FT; Datawrapper): temporal × quantitative → Line; quantitative × quantitative → Point; (nominal/ordinal) × quantitative → Bar; both-discrete → Point (Bar/Line/Area are invalid with no continuous axis); single axis or unknown → Bar.
3. Channel priority + Size discipline (Draco hard.lp:53,56,110 + non-positional pref): fill X/Y before Color/Size; Color before Size. Size is only valid for Quantitative/Ordinal positive measures on Point/Circle marks — disabled for Nominal, Temporal, and negative data (not merely discouraged).
4. Bar/Area zero-baseline; Line exempt (Draco hard.lp:103-104; FT; ONS/Vox sources FT links). We expose no axis-truncation control, so Vega-Lite's own defaults already give zero-baseline bars and free-baseline lines — the rule is satisfied by not adding an override, nothing to emit.
5. Chart-choice polish (FT; Datawrapper): sort bars when ranking; horizontal bar for long category labels; Size encodes a quantity, Color a category; Area is for a single series (warn against color-splitting into many).

5. The decision: Tier B — "smart + guarded"

Three tiers were on the table. Tier B was chosen (2026-06-05).

• Tier A — spec-literal: Bar default, four channel dropdowns, type override, smart pre-population. Matches §06 verbatim but uses almost none of the research; stays a "dumb" composer.
• Tier B — smart + guarded (chosen): Tier A + default mark from the (X, Y) type shape (not always Bar) + valid-type-only menus + inline non-blocking warnings from the Draco rules + swap-X/Y + Size disabled for Nominal/Temporal/negative. Still mark-first and §06-shaped, but genuinely intelligent. Requires a small §06 amendment (documented in the spec).
• Tier C — intent-first aid: Tier B + a "what do you want to show?" front door (FT/Datawrapper intents → recommended mark + channel layout from intent × column types). Highest "which chart & why" value; biggest UI; clearly extends §06. Deferred — if revisited, this is where Munzner's typology and Wilke's directory would be seated.

6. How it maps to implementation

The convergent rules become pure functions in src/core/chart-builder.ts (tested in chart-builder.test.ts), consumed by the builder store/modal:

• validFieldTypes(columnType) → the type menu (rule 1); defaultFieldType = its head.
• defaultMark(xType, yType) → smart default mark (rule 2); used by defaultBuilderConfig.
• isChannelTypeAllowed(channel, type) → Size discipline gate (rule 3).
• builderWarnings(config) → inline non-blocking hints (rules 3–5: line/area need both axes, area + many series, two measures better as a scatter, etc.).
• buildChartSpec / buildSnippetSpecText → assemble the final spec; zero-baseline is Vega-Lite-default (rule 4), so nothing is emitted for it.

7. Anti-recommendations (what a naive builder would happily produce, and we don't)

The highest-value guardrails — encodings a naive UI emits that the canon rejects:

• A categorical column on Size (Draco hard :53) — blocked, not warned.
• A truncated-axis bar — prevented by never exposing an axis override (Draco :103).
• A high-cardinality category on Color → unreadable legend (soft w=10; >20 hard).
• A Line between two raw measures instead of a scatter (Draco soft w=20) — warned.
• An all-categorical chart with no measure (Draco soft w=30, the loudest) — warned.
• A number typed Nominal (Draco soft w=10) — discouraged via default = Quantitative.

8. Future enhancements (backlog)

The Tier-B build is the floor, not the ceiling. The enhancements below were surfaced by the research. Status as of 2026-06-06.

Forward plan moved (2026-06-10): the prioritized, sequenced enhancement plan now lives in chart-builder-enhancement-scope.md, which merges this backlog with the lyra-review.md §5 ideas and sets the Tier-C target. This section remains the research record (the citations behind each item); consult the scope doc for what to build next and in what order.

A · Cheap wins inside the current 5-mark / 4-channel scope

• A1 · Sort-on-ranking (done) — chart-level Sort control (Asc/Desc/None) sorts the categorical axis by the measure (FT: "bars display ranks much more easily when sorted"). Appears only for a category-vs-measure pair.
• A2 · Bar orientation (partly done) — the Swap X/Y control is the manual path to a horizontal bar, and the crowded-axis hint (A3, below) now auto-suggests it for the un-aggregated case. A general "long labels → go horizontal" suggestion on any vertical bar is still deferred (needs a label-length / cardinality signal); a blanket warning was rejected — it would fire on every ordinary vertical bar.
• A3 · Crowded-axis & high-cardinality warnings (done) —
  • Done: the un-aggregated crowded axis — a bar/line/area with a category axis and a raw measure draws one mark (and one label) per row, so over CROWDED_CATEGORY_ROWS (30) rows it warns and points to aggregating, or a horizontal bar. Row-count-based: detects exactly the mark-count == row-count case (URL/non-tabular → rowCount null → skipped).
  • Done (via the profiling extension below): an aggregated axis that still has many distinct categories (CROWDED_CATEGORY_DISTINCT = 30) and an unreadable Color legend (CROWDED_LEGEND_DISTINCT = 12, discrete colour only) now warn from per-column cardinality. (Number-typed-Nominal remains a possible future nudge; not yet flagged.)
• A4 · Data-aware Size guard (done) — Size mapped to a field whose profiled numeric extent goes negative warns (Draco hard.lp:56; size implies positive magnitude), on top of the type-level Size discipline.

Done — profiling extension (the A3 / A4 enabler). profile.ts now derives, in the same sample pass that feeds inferColumnType, a per-column capped distinct count (DISTINCT_CAP = 50, with a distinctCapped overflow flag) and a numeric extent (min/max, numeric columns only), surfaced on DatasetProfile.columnStats and the Dataset record. builderWarnings(config, rowCount, columns) consumes them for the legend/axis crowding hints and the negative-value Size guard. Caveats handled: URL / non-tabular data has no rows → columnStats is [] and the dependent hints skip; datasets stored before the field default to [] via dataset-migrations / import normalization (no forced re-profile on read — they pick up stats on next save). Thresholds live in chart-builder.ts constants alongside CROWDED_CATEGORY_ROWS.

B · Transform-enabled coverage (new core capability + §06 extension) (done)

• B5 · Aggregation (done) — per-channel sum / mean / median / min / max, plus a field-less "Count of records" measure (Voyager's count(*)). The priority item.
• B6 · Binning (done) — bin on a quantitative field → true histograms (closes the Distribution gap); mutually exclusive with aggregate on the same field.
• B7 · Stacking (done) — stack (zero / normalize) for bar/area + a Color series → part-to-whole (closes that gap; enables 100%-stacked).
• Temporal granularity (done) — Vega-Lite timeUnit (Year / Quarter / Month / Week / Day / Hour, plus combined units) on a Temporal field; defaults to None (raw).
• B8 · Faceting (Row / Column → small multiples) (next increment, after A+B + UI land) — two more channels that multiply the chart into a trellis, the clean way to compare many categories (Voyager has it; FT/Datawrapper recommend small multiples; we currently can't express them). Still mark-first, so a Tier-B extension. Axis alignment is the design crux: Vega-Lite facets default to shared scales (aligned axes) — keep that as the default; expose an "independent axes" toggle (resolve.scale) only as an advanced option. Verify faceting against the preview's "container" fit modes before trusting it (per-cell sizing on facets is finicky). Sequenced as additive after the current build.

C · Intent-first front door (Tier C) (deferred) — see §5. "What do you want to show?" → recommend mark + channels from the FT/Datawrapper taxonomy × column types; where Munzner + Wilke would be seated.

D · Plumbing — D9 URL hash routing for the open builder (owned by M6, spec §01E); D10 a GOV.UK/NN-g copy pass over the guidance-hint wording (the M4 council seating's residual one-off debt).

---

Citations are to files under /Users/oleh/code/reference/. The seated chart-choice canon (FT clone + Datawrapper distill) lives in the council roster (.claude/skills/council/SKILL.md); Draco/Voyager are reference clones, not council seats — they're engineering sources, not user-facing design authorities.