Chart builder: filter/calculate transforms, data preview, and inline expression validation

docs/chart-builder-enhancement-scope.md

@@ -25,6 +25,51 @@
 Newest first. The at-a-glance build-order tracker is §4; per-item detail is §3. This log is
 the quick "where are we" — read it first.
 
+- **2026-06-11** — **1C + 1D + 1E shipped (the Data section). Up next: Phase 2 (2A value-or-field channels, then 2B field shelf).**
+  - A new **Data section** at the top of the builder's left pane — "here are your rows;
+    shape them, then encode them" — emits the spec's top-level `transform` array.
+  - **1C · Filters** — a guarded **field + operator + value** predicate shelf. Operators
+    narrow by field type (`validFilterOps`): a measure/temporal field offers ordering
+    (`< ≤ > ≥`) + `is between`; a category offers `is` / `is not` / `is one of`. Values
+    coerce by type (quantitative → number; others → string, so ISO dates sort right).
+    `notEqual` emits a `{ not: { …equal } }` wrapper. A reversible **expression** power-mode
+    takes a raw `datum.…` predicate. Incomplete filters are skipped so the preview keeps
+    rendering. Multiple filters AND together. Pure core (`buildTransforms`,
+    `validFilterOps`, `filterOpArity`), tested.
+  - **1C · Calculated fields** — `{ calculate, as }` derived columns. A named field appears
+    in the channel dropdowns via `effectiveColumns` (defaults Quantitative); emitted
+    **before** filters (a row-wise calculate is order-independent, so calc-first is
+    equivalent and lets filters reference derived fields). Removing/renaming a referenced
+    field clears the dangling channel (`pruneEncodings`, in the store on calc edit/remove).
+  - **1E · Expression validation** — new pure core `expr-validate.ts` using Vega's own
+    `parseExpression` (already in the `vega` chunk, so ~zero bundle cost): inline syntax
+    errors on both expression inputs, plus a soft **unknown-field** warning when a
+    `datum.<field>` reference doesn't match a column (`referencedFields` walks the AST).
+    Field discoverability is served by **dataset-derived placeholder examples** (e.g.
+    `datum.revenue * 2`); a full Monaco-style completion popup is **deferred** (a bare
+    `<input>` doesn't warrant it — noted, not built).
+  - **1D · Data preview** — a collapsible, read-only first-N-rows table with a per-column
+    **type chip** in each header, to sanity-check inferred types before building (reuses
+    core `tabularRows`). Default collapsed; the scroll region is keyboard-reachable
+    (`tabIndex=0` + labelled group — avoids the Datasets-manager a11y gap).
+  - **Spec §06** gained a "Data (filters, calculated fields, preview)" section; the Layout
+    and Output blocks cross-reference it.
+  - **Council not yet run** on the new error/disclosure copy (the soft auto-fire surface:
+    expression-error + unknown-field copy, the preview disclosure). Conventions were matched
+    to the existing warnings region (arch 10 §5) and `SettingsPopover` disclosure; flag for a
+    council pass on review if desired.
+  - **Expression reference:** a contextual link to the Vega expression-language docs is
+    shown when an expression input is in play (a calculated field, or a filter in
+    expression mode) — the place the user needs to know the available functions/operators.
+  - **Verified:** `typecheck` + `test` (full suite green; +103 new core/store cases, +4 modal
+    smoke tests) + `eslint` + `build` (PWA, 45 precache entries). **Owed:** a manual/visual
+    pass against the live builder (filter shelf, calc → channel, expr errors, preview
+    table) — tests don't cover what the surface looks/feels like.
+  - **Surfaced direction (now parked):** the 1D preview shows **raw source** rows; a
+    transform-aware **data inspector** (resolved rows, à la vega-editor, in the builder
+    _and_ below the main Live Preview) is the wanted evolution — documented in
+    **[`docs/data-inspector-exploration.md`](./data-inspector-exploration.md)**, deferred.
+
 - **2026-06-10** — **Up next: 1C (filter + calculate transforms), paired with 1D (data preview).**
   - **1B · Per-chart export** shipped: an **Export** disclosure in the Live Preview header
     (distinct from the workspace Export) — **Copy spec** + **Download JSON** (`.vl.json`) of
@@ -34,17 +79,15 @@ the quick "where are we" — read it first.
     `view`. Filenames derive from the snippet name — filesystem-safe, script-preserving
     (pure `core/chart-export.ts`, tested). Home: the **preview header**, not the plan's
     "library row / editor toolbar" — image export needs the live view; the
-    disclosure-of-controls (not an ARIA menu) mirrors `SettingsPopover`.
-    - **Export options (from first-round feedback):** PNG **Resolution** `1×/2×/3×` is a
-      multiplier of `devicePixelRatio`, so the default `1×` is Retina-crisp — the soft-1×
-      export was a dpr bug (raw `toImageURL` scaleFactor ignores dpr). **Background**
-      `Theme`(default)/`White`/`None` fixes transparent PNGs (the chart config is
-      transparent so the on-screen pane colour shows; export composites the chosen colour
-      under the PNG / adds an SVG `<rect>`). **Referenced data** `Inline`(default)/`Keep
+    disclosure-of-controls (not an ARIA menu) mirrors `SettingsPopover`. - **Export options (from first-round feedback):** PNG **Resolution** `1×/2×/3×` is a
+    multiplier of `devicePixelRatio`, so the default `1×` is Retina-crisp — the soft-1×
+    export was a dpr bug (raw `toImageURL` scaleFactor ignores dpr). **Background**
+    `Theme`(default)/`White`/`None` fixes transparent PNGs (the chart config is
+    transparent so the on-screen pane colour shows; export composites the chosen colour
+    under the PNG / adds an SVG `<rect>`). **Referenced data** `Inline`(default)/`Keep
 refs` (shown only when the spec references saved datasets) inlines dataset values so
-      the exported spec renders standalone (`inlineReferencedDatasets`, tested).
-    - Spec §08 gained a _Per-chart export_ section (with the options); §04 cross-references
-      it; `architecture/05` §2 records the handle's dpr-aware scale + background compositing.
+    the exported spec renders standalone (`inlineReferencedDatasets`, tested). - Spec §08 gained a _Per-chart export_ section (with the options); §04 cross-references
+    it; `architecture/05` §2 records the handle's dpr-aware scale + background compositing.
   - **1A · Actionable hints** shipped: one-click fixes on guidance warnings
     (`BuilderWarning.fixes` + `applyWarningFix`), council-reviewed, with focus/announce a11y.
   - **Builder UX/perf batch** (from dogfooding the Superstore dataset) shipped: near-fullscreen
@@ -152,7 +195,7 @@ chart" reads best beside the chart. The widget is a disclosure-of-action-buttons
 ARIA menu), mirroring `SettingsPopover` and sharing its single-open registry. _Spec impact:
 new §08 "Per-chart export" section; §04 cross-reference; `architecture/05` §2 handle note._
 
-**1C · Filter (+ Calculate) dataset transforms** — _closes the loop the builder's own warnings open_
+**1C · Filter (+ Calculate) dataset transforms** — _done (2026-06-11; see status log)_
 The transform layer the builder doesn't touch: top-level `transform: []`. Source: Lyra
 §3.12. The builder _already tells users to filter_ in three warnings
 (`chart-builder.ts:475,476,511`) while offering no way to do it.
@@ -171,7 +214,7 @@ Home: a new **"Data" section in the builder's left pane, above the channels** (`
 your rows [+ Filter] [+ Calculate] → now encode them`), paired with 1D. _Spec impact: new
 §06 "Data / transforms" subsection — this is genuinely new behaviour, write it._
 
-**1D · Data-table preview** — _closes a confirmed gap; pairs with 1C_
+**1D · Data-table preview** — _done (2026-06-11; see status log)_
 Neither the builder nor the Datasets manager ever shows the **actual rows**. Source: Lyra
 §3.2. A compact, **read-only** first-N-rows grid with a **per-column type chip** in each
 header lets users sanity-check inferred types _before_ building — exactly when inference is
@@ -180,7 +223,7 @@ only need the row sample. Homes: a collapsible "Data" strip in the builder's lef
 (under the dataset name) and/or the Datasets manager. Keep it read-only (editing data is
 out of scope). _Spec impact: §06 + §05 (Datasets) additions._
 
-**1E · Inline expression validation + field autocomplete** — _build with 1C, not standalone_
+**1E · Inline expression validation + field autocomplete** — _done (2026-06-11; completion popup deferred)_
 When 1C's expression mode lands, validate the Vega/VL expression string with the library's
 own `parseExpr` (Lyra §3.6) and surface errors inline; autocomplete the dataset's own
 column names (we have the schema from `profile.ts`) (Lyra §3.10). Record the `parseExpr`
@@ -306,6 +349,12 @@ guardrail: _promote a control only when it is **both common AND awkward in JSON*
   is short-lived; the smart default already gives a sane start).
 - **Lookup / join a second dataset** — Lyra §3.12; data-model change (multi-dataset
   snippets). Larger, separate effort.
+- **Transform-aware data inspector** — evolve 1D from raw-source rows to the _resolved,
+  post-transform_ data (filtered rows + calculated columns), and generalize it to a
+  togglable data panel below the **main** Live Preview (vega-editor's "Data Viewer", a
+  debugging aid for any snippet, not only builder output). Reads runtime rows via
+  `view.data(name)` through the `RenderHandle`. Cross-cutting (editor + builder), so it
+  has its own home: **[`docs/data-inspector-exploration.md`](./data-inspector-exploration.md)**.
 
 ---
 
@@ -314,10 +363,10 @@ guardrail: _promote a control only when it is **both common AND awkward in JSON*
 ```
 Phase 1  1A actionable hints      ✓ done
          1B per-chart export      ✓ done
-         1C filter (+ calculate)  ← next: closes the loop on warnings the builder already emits
-         1D data preview          ← pairs with 1C
-         1E expr-validate + autocomplete (with 1C)
-Phase 2  2A value-or-field channels (Property model)
+         1C filter (+ calculate)  ✓ done
+         1D data preview          ✓ done
+         1E expr-validate         ✓ done (syntax + unknown-field; completion popup deferred)
+Phase 2  2A value-or-field channels (Property model)  ← next
          2B field shelf + in-place type cycling   ← Tier-C substrate
 Phase 3  3A intent-first front door (Tier C)       ← built on 2B; the defining feature
          3B starter examples