mirror of
https://github.com/olehomelchenko/astrolabe.git
synced 2026-08-08 02:02:33 +00:00
Add aggregation, binning, granularity, sort, and stacking to the Chart Builder
- Per-channel transforms: aggregate (sum/mean/median/min/max), quantitative bin, and temporal timeUnit granularity; bin and aggregate are mutually exclusive. A field-less "Count of records" measure (Voyager's count(*)). - Chart-level sort (rank a categorical axis by its measure) and stacking (zero / 100% normalize), each shown only when it applies. - Field type is a fixed N|O|Q|T segmented control with the column's invalid types disabled; SegmentedControl gains APG-correct disabled options. - A crowded-category-axis warning (a raw measure drawing one mark per row over a large dataset) and a disabled-Create hint (says why it's disabled). - Drop the Create success toast — the new snippet is immediately visible. - Docs: spec §06, research-doc §8 backlog (incl. the cardinality/extent profiling TODO), architecture 01 (stable-selector rule) and 05 (builder-local preview), and a profiling breadcrumb.
This commit is contained in:
@@ -284,6 +284,11 @@ the reference.
|
||||
|
||||
**Goal:** no-JSON chart composition from a dataset → a new snippet.
|
||||
|
||||
> **Enhancement backlog** beyond the Tier-B floor (aggregation, binning, stacking,
|
||||
> temporal granularity, sort/orientation, cardinality-based warnings, Tier C
|
||||
> intent-first) lives in [`docs/chart-builder-research.md`](chart-builder-research.md) §8
|
||||
> — its single home, so these stop living in chat.
|
||||
|
||||
**Core**
|
||||
|
||||
- `chart-builder.ts` — pure spec assembler: (mark ∈ Bar/Line/Point/Area/Circle) +
|
||||
|
||||
@@ -87,6 +87,26 @@ const { activeModal, uiTheme } = useAppStore(
|
||||
);
|
||||
```
|
||||
|
||||
**`useShallow` only helps when the elements are stable.** It shallow-compares the
|
||||
result — array elements (or object values) by `Object.is`. A selector that
|
||||
**computes** a fresh collection of fresh objects each call (e.g.
|
||||
`useShallow((s) => buildWarnings(s.config))`) defeats it: every element is a new
|
||||
reference, so the result never compares equal, `useSyncExternalStore` re-renders
|
||||
forever, and React throws _"Maximum update depth exceeded"_ (a white screen). A
|
||||
selector must return a **primitive** or a **stored reference** — never a freshly
|
||||
built array/object. Derive computed collections in the component with `useMemo`
|
||||
over a stable slice instead:
|
||||
|
||||
```tsx
|
||||
const config = useChartBuilderStore((s) => s.config); // stored ref, stable between updates
|
||||
const warnings = useMemo(() => builderWarnings(config), [config]); // recompute only on change
|
||||
```
|
||||
|
||||
This is a render-time loop, so core/store unit tests stay green and miss it. A bare
|
||||
`react-dom/client` + `react`'s `act` mount test catches it with **no test-library
|
||||
dependency** — mount the component in the looping config and assert it doesn't throw
|
||||
(prove the guard by reverting the fix first). See `ChartBuilderModal.test.tsx`.
|
||||
|
||||
### Reading/writing outside components
|
||||
|
||||
Services, orchestration, infrastructure, and tests use the store object directly —
|
||||
|
||||
@@ -352,6 +352,18 @@ blank mid-edit.
|
||||
- **Don't** render synchronously on every keystroke.
|
||||
- **Don't** await a render inside an input/keydown handler.
|
||||
|
||||
### A second preview surface: the Chart Builder
|
||||
|
||||
The editor's `LivePreview` is **bound to the snippet editor** — it reads `SnippetStore`
|
||||
(shown spec), `AppStore` (fit mode/theme), and `PreviewStore` (shared error). The
|
||||
**Chart Builder modal** needs a preview of a _different_ spec source (its config), so it
|
||||
does **not** reuse `LivePreview`; it runs its own small debounced render over the same
|
||||
`chart-renderer.renderSpec` + `prepareSpecForRender`, with **local** error state (never the
|
||||
shared `PreviewStore`, which would cross-talk with the editor). Two preview surfaces, one
|
||||
renderer service. Builder flow: `chart-builder.ts` (pure spec assembler) → `ChartBuilderStore`
|
||||
(config + create) → `ChartBuilderModal`'s `BuilderPreview`. Reach for a reusable preview
|
||||
component only if a _third_ surface appears.
|
||||
|
||||
---
|
||||
|
||||
## 6. Rendering Contract Lives Upstream (reference)
|
||||
|
||||
@@ -198,6 +198,74 @@ The highest-value guardrails — encodings a naive UI emits that the canon rejec
|
||||
- An **all-categorical chart with no measure** (Draco soft w=30, the loudest) — warned.
|
||||
- A **number typed Nominal** (Draco soft w=10) — discouraged via default = Quantitative.
|
||||
|
||||
## 8. Future enhancements (backlog)
|
||||
|
||||
The Tier-B build is the floor, not the ceiling. The enhancements below were surfaced by
|
||||
the research; this is their single home (the milestone plan's M4 row points here). Status
|
||||
as of 2026-06-06.
|
||||
|
||||
**A · Cheap wins inside the current 5-mark / 4-channel scope**
|
||||
|
||||
- **A1 · Sort-on-ranking** _(done)_ — chart-level Sort control (Asc/Desc/None) sorts the
|
||||
categorical axis by the measure (FT: "bars display ranks much more easily when sorted").
|
||||
Appears only for a category-vs-measure pair.
|
||||
- **A2 · Bar orientation** _(partly done)_ — the **Swap X/Y** control is the manual path to
|
||||
a horizontal bar, and the crowded-axis hint (A3, below) now auto-suggests it for the
|
||||
un-aggregated case. A general "long labels → go horizontal" suggestion on _any_ vertical
|
||||
bar is still deferred (needs a label-length / cardinality signal); a blanket warning was
|
||||
rejected — it would fire on every ordinary vertical bar.
|
||||
- **A3 · Crowded-axis & high-cardinality warnings** _(partly done)_ —
|
||||
- _Done:_ the **un-aggregated crowded axis** — a bar/line/area with a category axis and a
|
||||
**raw** measure draws one mark (and one label) per row, so over `CROWDED_CATEGORY_ROWS`
|
||||
(30) rows it warns and points to aggregating, or a horizontal bar. Row-count-based:
|
||||
`builderWarnings(config, rowCount)`, with `rowCount` from the loaded dataset; detects
|
||||
exactly the mark-count == row-count case (URL/non-tabular → `rowCount` null → skipped).
|
||||
- _Remaining (needs the profiling extension below):_ an **aggregated** axis that still has
|
||||
many distinct **categories**, an unreadable **Color legend** (>10/>20 categories), and
|
||||
**number-typed-Nominal**. All need per-column **cardinality**, which the profile lacks.
|
||||
- **A4 · Data-aware Size guard** _(deferred — needs the profiling extension)_ — exclude
|
||||
**negative**-valued columns from Size (Draco `hard.lp:56`; size implies positive
|
||||
magnitude). Today only the type-level Size discipline is enforced.
|
||||
|
||||
> **TODO — profiling extension (the A3-remaining + A4 enabler).** `profile.ts` computes only
|
||||
> `rowCount` / `columnCount` / `columnTypes`. Extend it, in the **same sample pass** that
|
||||
> already feeds `inferColumnType` (so it's nearly free), to also derive per column: a
|
||||
> **capped distinct count** (cardinality — cap at ~50; a sampled count is enough for a
|
||||
> ">N categories" threshold, no full scan) and a **numeric extent** (min/max → sign).
|
||||
> Surface them on `DatasetProfile` (alongside `columnTypes`). Then: `builderWarnings` consumes
|
||||
> cardinality (legend/axis crowding) and the Size gate consumes sign (A4). **Caveats:**
|
||||
> URL / non-tabular datasets have no rows at profile time → these fields are null and the
|
||||
> dependent warnings simply skip; and stored datasets predate the field, so this needs a
|
||||
> recompute-on-read or a `dataset-migrations` bump (see architecture 02 / 06). Keep
|
||||
> thresholds in `chart-builder.ts` constants like `CROWDED_CATEGORY_ROWS`.
|
||||
|
||||
**B · Transform-enabled coverage (new core capability + §06 extension)** _(done)_
|
||||
|
||||
- **B5 · Aggregation** _(done)_ — per-channel `sum` / `mean` / `median` / `min` / `max`, plus
|
||||
a field-less "Count of records" measure (Voyager's `count(*)`). The priority item.
|
||||
- **B6 · Binning** _(done)_ — `bin` on a quantitative field → true histograms (closes the
|
||||
Distribution gap); mutually exclusive with aggregate on the same field.
|
||||
- **B7 · Stacking** _(done)_ — `stack` (`zero` / `normalize`) for bar/area + a Color series →
|
||||
part-to-whole (closes that gap; enables 100%-stacked).
|
||||
- **Temporal granularity** _(done)_ — Vega-Lite `timeUnit` (Year / Quarter / Month / Week /
|
||||
Day / Hour, plus combined units) on a Temporal field; defaults to None (raw).
|
||||
- **B8 · Faceting (Row / Column → small multiples)** _(next increment, after A+B + UI land)_
|
||||
— two more channels that multiply the chart into a trellis, the clean way to compare many
|
||||
categories (Voyager has it; FT/Datawrapper recommend small multiples; we currently can't
|
||||
express them). Still mark-first, so a Tier-B extension. **Axis alignment** is the design
|
||||
crux: Vega-Lite facets default to **shared scales** (aligned axes) — keep that as the
|
||||
default; expose an "independent axes" toggle (`resolve.scale`) only as an advanced option.
|
||||
**Verify** faceting against the preview's `"container"` fit modes before trusting it
|
||||
(per-cell sizing on facets is finicky). Sequenced as additive after the current build.
|
||||
|
||||
**C · Intent-first front door (Tier C)** _(deferred)_ — see §5. "What do you want to
|
||||
show?" → recommend mark + channels from the FT/Datawrapper taxonomy × column types; where
|
||||
Munzner + Wilke would be seated.
|
||||
|
||||
**D · Plumbing** — **D9** URL hash routing for the open builder (owned by **M6**, spec
|
||||
§01E); **D10** a GOV.UK/NN-g copy pass over the guidance-hint wording (the M4 council
|
||||
seating's residual one-off debt).
|
||||
|
||||
---
|
||||
|
||||
_Citations are to files under `/Users/oleh/code/reference/`. The seated chart-choice
|
||||
|
||||
@@ -33,12 +33,30 @@ A two-pane modal:
|
||||
- Optionally overrides the channel's **field type**. The override appears only once a column is selected, and offers only the **types valid for that column** (Tier B valid-type locking) — a string/boolean column never offers Quantitative, and only a date column offers Temporal. Concretely: number → {Quantitative (default), Ordinal, Nominal}; date → {Temporal}; text → {Nominal (default), Ordinal}; boolean → {Nominal}. When a column admits only one valid type, no override control is shown.
|
||||
- When a column is chosen, its field type defaults from the dataset's inferred column type (numeric → Quantitative, date → Temporal, otherwise Nominal); the user may change it within the valid set above.
|
||||
- **Size discipline:** the **Size** channel accepts only Quantitative or Ordinal columns — size implies an ordered magnitude, so categorical (Nominal) and Temporal columns are not offered for Size (they remain available on X/Y/Color). A column that can't go on Size is shown disabled there with a brief reason.
|
||||
- The column dropdown also offers a field-less **"Count of records"** measure (Vega-Lite `count`) — a quantitative count of the rows, with no column.
|
||||
- Clearing a channel back to "None" leaves it out of the produced spec.
|
||||
- A **Swap X/Y** control exchanges the X and Y mappings (field and type) in one click, for quickly flipping the axes of the pre-populated default without re-selecting both columns.
|
||||
|
||||
The field-type override is presented as a fixed **`N | O | Q | T`** segmented control (abbreviations with full-name tooltips, after _Datasets_' Nominal/Ordinal/Quantitative/Temporal), always showing all four with the column's invalid types **disabled** rather than hidden — so the control keeps one shape on every channel.
|
||||
|
||||
### Transforms (per channel)
|
||||
|
||||
Once a column is mapped, the channel offers the transforms that apply to its field type — and only those:
|
||||
|
||||
- **Aggregate** (a measure / Quantitative field): one of `Sum`, `Mean`, `Median`, `Min`, `Max`, or `None`. (The field-less `Count` measure is chosen via the "Count of records" column option above.)
|
||||
- **Bin** (a Quantitative field): bins the values into ranges — e.g. a Quantitative X binned with a Count Y is a histogram. Binning and aggregating the same field are mutually exclusive (setting one clears the other).
|
||||
- **Granularity** (a Temporal field): a Vega-Lite `timeUnit` — Year, Year-Quarter, Year-Month, Year-Month-Day, Quarter, Month, Week, Day of month, Day of week, Hour — or `None` (raw timestamps). Defaults to **None** (no silent change to what the raw data shows).
|
||||
|
||||
### Sort and stacking (chart-level)
|
||||
|
||||
These controls appear only when they apply:
|
||||
|
||||
- **Sort** (when X and Y form a category-vs-measure pair): sorts the categorical axis by the measure — `Ascending`, `Descending`, or `None` — the standard way to rank a bar chart.
|
||||
- **Stacking** (a Bar or Area mark with a Color series): `Stacked` (absolute) or `100%` (normalized, part-to-whole). Bars/areas without a Color series, or other marks, show no stacking control.
|
||||
|
||||
### Default pre-population
|
||||
|
||||
- On open, the first detected column is assigned to **X** and the second (if any) to **Y**, each with its derived field type. Remaining channels start unmapped. The mark starts at the smart default for that X/Y shape (see _Mark type_), not unconditionally Bar.
|
||||
- On open, the first detected column is assigned to **X** and the second (if any) to **Y**, each with its derived field type and no transforms. Remaining channels start unmapped. The mark starts at the smart default for that X/Y shape (see _Mark type_), not unconditionally Bar.
|
||||
|
||||
### Guidance (non-blocking)
|
||||
|
||||
@@ -48,6 +66,7 @@ The builder surfaces short, plain-language hints for configurations that render
|
||||
- A **Bar/Line/Area** whose X and Y are both categories (nothing to measure).
|
||||
- **Two measures** on a non-scatter mark (a scatter — Point/Circle — usually reads better).
|
||||
- An **Area** chart split into multiple colour series (per-series change is hard to see).
|
||||
- A **Bar/Line/Area** that pairs a category axis with a **raw (un-aggregated) measure** over a many-row dataset — it draws one mark, and one axis label, per row, so the category axis becomes an unreadable picket fence. The hint suggests aggregating the measure (one mark per category) or, for a bar, flipping to a horizontal bar (Swap X/Y) where long labels stay readable (FT Visual Vocabulary / Datawrapper). Only the un-aggregated case (mark-count = row-count) is detected; flagging an _aggregated_ axis that still has many distinct categories needs per-column distinct counts the profiler does not yet compute (a known gap).
|
||||
|
||||
A clean configuration shows no hints.
|
||||
|
||||
@@ -72,12 +91,11 @@ A clean configuration shows no hints.
|
||||
|
||||
Selecting "Create Snippet" produces the final artifact:
|
||||
|
||||
- Builds a complete Vega-Lite spec containing: the schema reference, a named data reference to the dataset, the chosen mark (with tooltips enabled), the mapped encodings (each with its field and field type), and any explicit width/height.
|
||||
- Builds a complete Vega-Lite spec containing: the schema reference, a named data reference to the dataset, the chosen mark (with tooltips enabled), the mapped encodings (each with its field and field type, plus any aggregate / bin / `timeUnit` transform), chart-level sort and stacking where set, and any explicit width/height.
|
||||
- Channels left unmapped are omitted; if no encodings exist the spec omits the encoding block entirely (prevented by validation here).
|
||||
- Creates a new snippet from that spec with an auto-generated descriptive name, adds it to the snippet library, and records that it was built from the dataset.
|
||||
- Links the snippet to the dataset by recording the dataset reference, so the bidirectional snippet↔dataset relationship is established (see _Datasets_).
|
||||
- Raises a success toast naming the created snippet.
|
||||
- Closes the builder; the newly created snippet becomes the active snippet in the library/editor.
|
||||
- Closes the builder; the newly created snippet becomes the active snippet in the library/editor. **No success toast** — the result is immediately visible (the new snippet opens in the editor), so a toast would be noise (architecture 10 §1, "toast only what the user can't already see"). This refines the earlier blanket "every action toasts" rule, consistent with the Extract-to-dataset / publish reconciliation.
|
||||
|
||||
## Closing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user