diff --git a/docs/embedding-vega-lite.md b/docs/embedding-vega-lite.md new file mode 100644 index 0000000..436eeb7 --- /dev/null +++ b/docs/embedding-vega-lite.md @@ -0,0 +1,447 @@ +# Embedding Vega-Lite in a real app: the parts the docs don't warn you about + +Vega-Lite is a joy to author and a little treacherous to embed. The grammar is +well documented; the _runtime_ — view lifecycle, sizing, fonts, export, theming — +is where you lose an afternoon to a blank chart with no error in the console. + +This is a field guide from building a browser app that renders arbitrary, +user-authored Vega-Lite specs live: type JSON on the left, see the chart on the +right, export it, theme it, keep it responsive. Everything below is something that +actually cost us time, with the fix and — more importantly — _why_ it happens, so +you can recognize the next variant of it. + +It assumes `vega-embed`. If you hand-roll `compile → parse → new View()`, the same +issues apply; you just own more of the plumbing. + +--- + +## 1. The view is the bug surface, not the spec + +Every successful `vegaEmbed()` hands back a `result.view` — a live Vega `View`. It +owns timers, signal listeners, event handlers, and DOM. Render a new spec into the +same node without disposing the old view and the old one **leaks**: its listeners +keep firing, resources accumulate, and a long editing session slowly degrades. + +```ts +let current: Awaited> | null = null; + +async function rerender(node: HTMLElement, spec, config) { + current?.view.finalize(); // tear down the previous view FIRST + node.replaceChildren(); // drop any DOM the previous embed left behind + current = await vegaEmbed(node, spec, { config }); +} +``` + +Two rules that pay for themselves: + +- **`finalize()` before every re-embed, and on unmount.** This is the single most + important discipline. `finalize()` is not optional cleanup; it's how you avoid a + zombie view. +- **Keep all `vegaEmbed()` calls behind one small module.** Components ask it to + "draw this spec into this node" and get back a handle with `destroy()`, + `toImageURL()`, `resize()`. Nothing else imports `vega-embed` or touches a `View` + directly. This one boundary is what makes every other fix in this article land in + exactly one place. + +--- + +## 2. Container sizing breaks in two completely different ways + +`width: "container"` / `height: "container"` is how Vega-Lite does responsive +sizing. It's also the source of the two most baffling bugs we hit, and they look +nothing alike. + +### Gotcha A — the chart collapses to zero width + +Symptom: `width: "container"` charts render a sliver; `height: "container"` is +often fine. Classic "width is broken, height works" head-scratcher. + +Cause: `vega-embed` injects `.vega-embed { display: inline-block }` into `` +at runtime. Because it's injected late, it **wins the cascade** over a class you put +on that same element. `inline-block` shrink-wraps horizontally, and `"container"` +width reads `host.clientWidth` — which is now ~0. (Height survives because a tall +parent still gives the box a `clientHeight`.) + +A nasty wrinkle: `vega-embed` only adds its responsive `chart-wrapper` element — +the thing its own `width: 100%` rule targets — **when the actions menu is enabled**. +If you pass `actions: false` (you probably do; see §7), that path is dead and your +element is branded `inline-block` directly, with nothing fixing it. + +Fix: embed into a dedicated **inner host** with a _static_ className, nested inside +an outer frame you control. Size the inner host with a **two-class selector** so you +out-specify `.vega-embed`: + +```css +/* one class loses to .vega-embed; two classes win */ +.fitWidth .host { + width: 100%; +} +``` + +Keep the inner host's class static so React (or whatever owns the DOM) never +re-reconciles it and stomps Vega's runtime classes. + +### Gotcha B — the chart never follows a resize + +Symptom: a responsive chart sizes correctly on first render, then ignores the pane +being dragged wider. + +Cause: Vega-Lite compiles `"container"` sizing into width/height signals that +re-read `containerSize()` **only on a `window:resize` event**. Two consequences +people rediscover the hard way: + +1. `view.resize()` does **not** re-measure. It re-runs layout with the _stale_ size. +2. A pane drag (a splitter, a layout change) fires no `window:resize`, so nothing + re-fits on its own. + +Fix: observe the host with a `ResizeObserver` and synthesize the event Vega is +actually listening for. + +```ts +const ro = new ResizeObserver(() => { + window.dispatchEvent(new Event('resize')); // the mechanism, not a hack +}); +ro.observe(host); +``` + +This is the documented mechanism, not a workaround — it's literally what the Vega +editor does. `ResizeObserver` callbacks are frame-batched, so it tracks a drag +smoothly with no debounce. Bonus: only the container-bound dimension carries the +resize handler, so a width-only chart re-fits width and leaves height natural for +free, with zero bookkeeping. + +--- + +## 3. Fonts must finish loading _before_ you render — any renderer + +This one is invisible until you ship a custom font. The chart renders, the text +looks slightly wrong (spacing off, labels colliding or over-padded), and it +_sometimes_ fixes itself on the next edit. + +Cause: Vega measures every text label with canvas `measureText` **regardless of +renderer** — SVG, canvas, even the headless `'none'` renderer runs a layout pass. If +a web font is still loading when you embed, the entire chart is laid out with +_fallback_ font metrics. When the real font swaps in, the glyphs change but the +layout was already computed against the wrong widths. + +Fix: gate the render on the fonts the spec actually references. + +```ts +async function ensureFontsLoaded(families: string[]) { + if (!document.fonts?.load) return; // no-op in tests / old browsers + const loads = families.flatMap((f) => + ['400', '600', '700'].map((w) => document.fonts.load(`${w} 16px ${f}`)), + ); + // allSettled, not all: a missing face (offline, 404, a system family with no + // @font-face) is EXPECTED — degrade to fallback metrics, never fail the chart. + await Promise.race([ + Promise.allSettled(loads), + new Promise((r) => setTimeout(r, 3000)), // bound a slow first fetch + ]); +} +``` + +Two judgment calls worth copying: use `allSettled` (a font failing to load is not a +chart error — it's a render-with-fallback), and cap the wait with a timeout so a +slow network never freezes the preview. A cached face resolves near-instantly; the +timeout only ever bounds the very first fetch of an uncached subset. + +--- + +## 4. SVG vs canvas is a real performance cliff, and canvas has a silent ceiling + +The default `renderer: 'svg'` is the right call almost always — crisp at any zoom, +inspectable, copyable, themeable. But SVG renders **one DOM node per mark**. A chart +with thousands of marks (say one bar per row of a 10k-row dataset) costs _seconds_ +of main-thread layout and paint per render. We measured ~6.5s of paint on ~10k rows — +and the freeze lands _after_ the chart first appears, because the browser paints the +SVG tree lazily. The tab locks up holding a chart that looks done. + +Switch many-mark charts to `renderer: 'canvas'`: a single node, painted in +milliseconds. The raster trade-off (not crisp on zoom) is invisible for an ephemeral +preview, and — crucially — image export is renderer-agnostic (§7), so you lose +nothing downstream. + +But canvas has its own trap: a **hard maximum dimension**. Browsers cap a canvas +backing store at ~32,767px per side (less on Safari, which is also area-bound). Past +that, the canvas fails to allocate and draws **nothing** — no error, no exception, +just a blank surface and sometimes a null 2D context. A tall categorical chart +(hundreds of natural-height rows) blows past this easily. + +Fix: before committing to canvas, run a headless layout probe and read the resolved +size. The `'none'` renderer computes layout without allocating a canvas: + +```ts +const probe = await vegaEmbed(detachedDiv, spec, { renderer: 'none', config }); +const height = probe.view.height(); +probe.view.finalize(); + +const limit = 32767 / (window.devicePixelRatio || 1); // backing store is dpr× +if (height > limit) throw new ChartTooLargeError(height, limit); +``` + +Now you can tell the user the _real_ cause ("this chart is 50,000px tall") instead +of handing back a blank box. SVG has no such cap — it just gets slow — so the probe +is canvas-only. + +--- + +## 5. Exporting an image has three sharp edges + +You'd think `view.toImageURL()` is the export story. It isn't, quite. + +**Retina blur.** `toImageURL`'s `scaleFactor` ignores `devicePixelRatio`. A naive +"1×" PNG export comes out at half resolution on a 2× display — soft, obviously +wrong next to the crisp on-screen chart. Multiply the scale by dpr yourself: + +```ts +const dpr = window.devicePixelRatio || 1; +const canvas = await view.toCanvas(scale * dpr); // "1×" now matches the screen +``` + +**Transparent background.** If your theme sets `background: 'transparent'` (you +probably do, so the chart shows the pane color through it — §6), every export is +also transparent. Usually not what someone wants in a PNG. Composite an opaque color +under the canvas, and inject a full-bleed `` as the first child of the root +`` for the vector path: + +```ts +svg = svg.replace(/(]*>)/, `$1`); +``` + +**SVG drops your fonts.** `view.toSVG()` serializes only the `font-family` _name_. +Open that SVG anywhere the font isn't installed and it falls back to a system font. +If the font is one your users uploaded, embed it as a base64 `@font-face` rule inside +a ``); +``` + +PNG needs none of this — the raster already baked the glyphs in. Only the vector +format leaks the font dependency. + +One nice property to lean on: export is **renderer-agnostic**. `view.toCanvas()` and +`view.toSVG()` draw to their own off-screen surface, independent of how the chart is +displayed. So you can show an SVG chart on screen and still export a high-res PNG, or +show a canvas preview (§4) and still export a clean SVG. + +--- + +## 6. Theme is a config you merge at embed time — and Vega is picky about it + +A Vega-Lite **config** object styles every chart globally: fonts, axis colors, +background, the categorical palette. The right model is to keep the config _out_ of +the user's stored spec and inject it at embed time, so the same spec re-themes for +free when the UI flips light/dark: + +```ts +await vegaEmbed(node, spec, { config: chartConfigFor(theme) }); +``` + +Three things that bit us: + +- **The spec wins, key by key.** Vega-Lite merges your injected `config` _under_ the + spec's own `config` (`mergeConfig(opt.config, spec.config)`). That's the behavior + you want — a snippet can override or opt out locally — but know it: you can't force + a style the spec contradicts. +- **Don't rebuild a config from a fixed schema.** If you let users edit a config + through structured controls, mutate the config object _in place_; don't reconstruct + it from a known set of keys. Preset themes (and Vega proper) carry Vega-_layer_ + keys — `symbol`, `shape`, `path`, `group` — that aren't in the Vega-Lite `Config` + type but are forwarded to Vega at render. A rebuild silently drops them. +- **A bare scheme name passes compile but fails at render.** Writing + `range: { category: "tableau20" }` (a bare string) survives Vega-Lite _compilation_ + and then Vega rejects it at _render_ with "Unrecognized scale range value" — and + blanks the chart. The accepted form is the object: `range: { category: { scheme: +"tableau20" } }`. This compile-passes/render-fails split is a recurring Vega theme; + when a chart goes blank with a console error but no compile error, suspect a value + that's structurally valid JSON but semantically wrong for the runtime. + +If you want themes that follow the system light/dark, keep exactly **one** function +that maps `(selection, uiTheme) → config`. Every render resolves through it; nothing +else decides a chart's styling. (We also offer the `vega-themes` package's presets +verbatim — it's already in your tree as a `vega-embed` dependency, so the famous +FiveThirtyEight / Excel / Carbon looks are free.) + +--- + +## 7. `actions: false` does more than hide a menu + +You'll almost certainly want `actions: false` — the built-in "Save as / View Source / +Open in Vega Editor" overlay doesn't belong on most embeds, and you'll provide your +own export. Just know two side effects: + +- As noted in §2, it removes the responsive `chart-wrapper`, so you own host sizing. +- You also give up the built-in PNG/SVG export, so build your own through the view + (§5). That's a feature, not a cost — you get dpr-correct, background-filled, + font-embedded exports the built-in menu never gave you. + +And for tooltips: pass `tooltip: { disableDefaultStyle: true }` so `vega-tooltip` +doesn't inject its own light/dark stylesheet. The tooltip element (`#vg-tooltip- +element`) is appended to ``, so once the default style is gone you style it +entirely from your own CSS — and because it lives under ``, it inherits your +`[data-theme]` cascade for free. `vega-tooltip` still handles positioning and the +`.visible` toggle; you just supply the look. + +--- + +## 8. Field names with dots are not what you think + +If you construct specs from data-derived column names (a chart builder, an +auto-encoding helper), this _will_ bite you. Vega-Lite treats `.`, `[`, and `]` +inside a `field:` as **nested-property accessors**: `field: "user.age"` reads +`row.user.age`, not a column literally named `"user.age"`. Real-world CSVs have +columns like `Price ($)` or `2021.Q3` all the time. + +```ts +const escapeField = (name: string) => name.replace(/([.[\]])/g, '\\$1'); +encoding.x = { field: escapeField(columnName), type: 'quantitative' }; +``` + +Escape every data-derived name before it lands in any field-position key — `field`, +`as`, `groupby`, tooltip fields, the lot. (For specs a user hand-authored, escaping +is their responsibility; don't rewrite their `field:` values.) + +--- + +## 9. Never mutate the spec you render + +Rendering should be a pure function of (spec, config). If your pipeline rewrites the +spec on the way to the view — resolving dataset references to inline values, applying +a responsive sizing mode, escaping fields — do it on a **deep copy**: + +```ts +const prepared = structuredClone(userSpec); +// ...mutate `prepared` freely: resolve refs, set width:"container", etc. +await vegaEmbed(node, prepared, { config }); +// userSpec is untouched — what the user sees in the editor is still what they wrote. +``` + +The moment rendering mutates the stored spec, you get spooky action: a fit-mode +toggle permanently rewrites the user's `width`, an export inlines a 2MB dataset into +the document they're editing. Keep the transform pure and copy-first, and it stays +unit-testable without a DOM as a bonus. + +A related subtlety: a "fit to container" mode that sets `width: "container"` should +also _delete_ the spec's explicit `height` (and vice-versa) so the unconstrained +dimension recomputes naturally. Which means a surface that lets the user type an +explicit width/height must opt _out_ of fit mode while they do — the two fight over +the same keys. + +--- + +## 10. Live editing: debounce the input, guard the output + +For a live preview that re-renders as the user types, two independent concerns: + +**Debounce edit → state, not state → render.** Re-rendering must never compete with +typing. Debounce the editor's text changes (we make the delay user-configurable, +~500–5000ms); render only after a pause. But render _immediately_ for non-typing +changes — loading a different spec, a theme flip, a fit-mode toggle. The debounce +exists for keystroke churn and nothing else; detect "this was a keystroke" by +elimination (the text changed but the document identity didn't). + +**Guard against out-of-order renders.** `vegaEmbed`/`runAsync` is async, so a slow +render can resolve _after_ a newer one already mounted. A bare debounce doesn't cover +this. Stamp each render with a generation token and let only the latest one win: + +```ts +let generation = 0; +async function render() { + const mine = ++generation; + const handle = await renderSpec(node, spec, config); + if (mine !== generation) { + handle.destroy(); + return; + } // a newer render superseded us + current = handle; +} +``` + +And keep the _last good chart on screen_ while the next render computes — overlay a +subtle busy indicator rather than blanking the pane. A pane that flickers to empty on +every keystroke feels broken even when it's fast. + +--- + +## 11. Errors: a blank spec is not an error, and recovery should be automatic + +Three stages fail, and you want them distinguishable in the message: JSON parse +("Invalid JSON: …"), spec preparation ("Dataset not found: …"), and embed itself +("Rendering error: …", the Vega-Lite compile or Vega runtime failure). Funnel all +three to one error field the preview reads. + +The behaviors that make it feel solid: + +- **Empty/blank text renders nothing** — a clean pane, not an error. Finalize the + current view, clear the error, stop. +- **Every successful render clears the error.** Recovery is then automatic: the next + valid edit re-renders and wipes the message. No retry button, no reload. +- **Keep the last good chart visible under a parse error** if you can, so a + half-typed keystroke doesn't strobe the whole pane. +- **Wrap `runAsync`/embed in try/catch and finalize on failure.** Vega won't catch + runtime errors for you, and a half-initialized view leaks if you don't finalize it. +- **Don't dump a raw stack trace.** Give the reason plus a hint ("check your JSON and + that the spec is valid Vega-Lite"). + +--- + +## 12. If you also embed an editor (Monaco) — wire it yourself + +Optional, but if you're putting users in front of raw spec JSON you'll want schema +validation and autocomplete. The non-obvious parts: + +- **Bundle the schema; never fetch it.** `import schema from +'vega-lite/vega-lite-schema.json'` and register it once, globally, via the JSON + language service (`setDiagnosticsOptions`). Version-locked to your installed + Vega-Lite, offline-safe, no runtime network call. Set `enableSchemaRequest: false` + so the worker can't go fetch an unbundled `$schema` URL behind your back. +- **Bind by `fileMatch`, not by the doc's `$schema` value.** If you key validation + off the `$schema` line, a spec without one gets zero validation and zero + completions. Match your model URIs instead so it always works. +- **Monaco workers are on you.** With a CDN loader they're automatic; self-hosted, + you must set `MonacoEnvironment.getWorker` to return the JSON worker for label + `'json'` and the editor worker otherwise. No worker means no squiggles and no + completions — and no error telling you why. +- **`quickSuggestions: { strings: true }`.** Vega-Lite enum values (`"bar"`, + `"quantitative"`) live _inside JSON strings_, where Monaco disables autocomplete by + default. Without this, completions silently never appear. +- **Two layers, two tiers.** The Monaco worker gives inline squiggles; a separate + `ajv` pass can feed a richer error pane. Sort findings into _fatal_ (syntax / compile + / runtime errors that suppress the chart) and _advisory_ (schema-validation warnings + that don't). Vega-Lite emits plenty of benign warnings; treating them as fatal hides + specs that render fine. +- **ajv has its own gotchas:** construct it with `strict: false`, add the draft-06 + meta-schema (the VL schema is draft-06; ajv 8 defaults newer), register a no-op + `color-hex` format, and **compile the validator once at module load** — the schema + is multi-megabyte and compiling per keystroke is a real perf sink. + +--- + +## The short version + +If you skim one thing, skim this: + +| Trap | Fix | +| --------------------------------------- | ----------------------------------------------------------------------------------------------------------- | +| Re-embedding leaks the old view | `view.finalize()` before every re-embed and on unmount | +| `width:"container"` collapses to ~0 | Inner host + out-specify `.vega-embed { inline-block }` with a 2-class selector | +| Chart won't follow a resize | `ResizeObserver` → `window.dispatchEvent(new Event('resize'))`, not `view.resize()` | +| Custom font lays out wrong | `document.fonts.load(...)` (allSettled + timeout) before embed; metrics are measured regardless of renderer | +| Many-mark SVG freezes the tab | Switch to `renderer: 'canvas'`; probe size first — canvas fails silently past ~32k px | +| Export looks soft on Retina | Multiply scale by `devicePixelRatio` | +| Export is transparent / loses fonts | Composite a bg color; embed `@font-face` (CDATA) in the SVG | +| Bare scheme name blanks the chart | Use `range: { category: { scheme: "…" } }`, not a bare string | +| Dotted column names misread | Escape `.[]` in every data-derived `field:` | +| Stale async render clobbers a fresh one | Render-generation token; only the latest wins | +| Rendering mutates the user's spec | Transform on a `structuredClone` copy | + +None of these are exotic. They're the gap between "it works in the demo" and "it +holds up under a 10k-row dataset, a custom font, a dragged pane, and a Retina +export." Vega-Lite is excellent; it just expects you to know where its runtime edges +are. Now you do.