astrolabe/docs/architecture/08-vega-editor-techniques.md

08 · Borrowed Techniques from vega/editor

The official Vega-Lite editor (vega/editor) solves the exact "edit a Vega-Lite spec as JSON, validate it, render it live" problem Astrolabe sits on top of — minus the snippet/dataset library. This doc distills the techniques worth borrowing and the gotchas worth avoiding, so we don't rediscover them from scratch in M1/M2.

It is a reference, not a contract. The behavioral contract is still docs/spec/; the patterns are still docs 01–07. This doc is the bridge: "here is how the canonical implementation does the editor/renderer plumbing, and here is what we keep vs. improve."

Source of these findings

A read-only clone of vega/editor lives at /Users/oleh/code/reference/vega-editor (shallow clone of main, HEAD 4fdbb59). Re-clone with git clone --depth 1 https://github.com/vega/editor. Citations below are file:line into that tree.

Stack delta (read this first — it changes how directly we can borrow)

	vega/editor	Astrolabe
UI framework	React	React (moved off Preact before build start)
State	Redux-ish single State in React context (useState)	Zustand stores — not Redux
Monaco	@monaco-editor/react + @monaco-editor/loader (CDN-loaded Monaco, workers auto-wired)	raw monaco-editor via Vite (we must wire workers ourselves)
Rendering	hand-rolled vegaLite.compile → vega.parse → new vega.View().runAsync()	vegaEmbed() (wraps that same pipeline)
Schema validation	Monaco JSON worker + standalone ajv ^8 (two independent layers)	same two-layer model planned

Because both apps are now React, vega/editor's component lifecycle patterns port more or less directly — the friction is only in (a) state (their Redux-flat-state → our Zustand stores) and (b) Monaco worker wiring (their CDN loader → our explicit Vite workers).

---

Decision · Monaco integration (self-hosted, raw API)

Decided. Astrolabe uses raw monaco-editor from npm, bundled and self-hosted, with workers wired explicitly via Vite ?worker — not vega/editor's @monaco-editor/react + @monaco-editor/loader (CDN) setup. Two independent axes:

Axis A — sourcing: self-hosted/bundled, not CDN. (Forced by Astrolabe's values.) vega/editor's @monaco-editor/loader fetches Monaco's AMD bundle from a CDN at runtime. For us that breaks three things at once: (1) offline — the CDN bundle is outside Vite's module graph, so vite-plugin-pwa/Workbox never precaches it and offline silently fails; bundled npm assets are hashed files in dist/ that Workbox precaches automatically; (2) privacy — a third-party fetch on load contradicts SOUL's "the only outbound requests are user-created URL-dataset fetches"; (3) determinism — npm + package-lock is integrity-pinned and reproducible, a runtime CDN resolve is not. This axis is not a close call; vega/editor's CDN choice is right for an online hosted tool and wrong for an offline, installable, private app.

Axis B — React integration: raw API, not @monaco-editor/react. (A lean, not forced.) The wrapper helps with the easy 80% (mount a JSON editor, lifecycle) and adds nothing to the load-bearing 20% this app needs:

• Workers are still ours — the wrapper never manages MonacoEnvironment (see §1 gotcha).
• The M2 schema service (jsonDefaults.setDiagnosticsOptions, fileMatch) is namespace-level; you reach through the wrapper via onMount, so it saves nothing there.
• Its headline value/onChange controlled-input model is a hazard: driving Monaco's content from React state causes cursor jumps and undo-stack churn, against §10's "typing stays fluid" — you end up using it uncontrolled, i.e. the raw pattern anyway.
• Its CDN-by-default is a standing footgun (works in dev online, fails offline in prod unless you remember loader.config({ monaco })).

Against that, raw costs one testable useMonacoEditor hook (~50–80 lines: create in useEffect, dispose on unmount, push value, subscribe to onDidChangeModelContent, resize). That's the same imperative-teardown discipline already adopted for vega-embed in doc 05 (view.finalize()), and consistent with already using raw vegaEmbed() over a React chart wrapper — "thin integration layers we own" (SOUL). Lock-in is low either way, so the final raw-vs-wrapper call is confirmable at the Monaco spike; what is not up for revisiting is self-hosting.

Accepted cost: the explicit worker wiring (§1) is inherent to self-hosting — it is the price of offline, paid in any non-CDN setup, and the wrapper would not remove it.

Entry point: edcore.main, never editor.api (trim languages, not features)

Self-hosting raw Monaco forces a choice of ESM entry point, and the granularity matters:

Import	What you get	Use?
monaco-editor (barrel)	All features + every basic language (sql, abap, solidity, …)	❌ language bloat (~20 dead chunks)
esm/vs/editor/editor.api	The API surface only — zero feature contributions	❌ a text box: no folding, suggest widget, Cmd+Backspace, find, bracket colorization
esm/vs/editor/edcore.main	editor.all (all 59 feature contributions) + API, no languages	✅ full editor UX, JSON-only weight

Import edcore.main and add only the JSON language service (esm/vs/language/json/monaco.contribution). edcore.main ships no .d.ts of its own — add an ambient declare module … { export * from '…/editor.api'; } so types (and Monaco's global MonacoEnvironment augmentation) resolve. Two editor options worth setting because they bite Vega-Lite specs specifically: showFoldingControls: 'always' (fold arrows always visible), and quickSuggestions: { strings: true } (VL enum values like "bar" live inside JSON strings, where Monaco disables auto-suggest by default).

Reaching for editor.api to "drop unused languages" silently strips every editor feature — the languages live elsewhere. This is the concrete case behind AGENTS.md's "trim content, not capability" rule: cut the unwanted content, keep the behavior, and verify the behavior survived by exercising the editor, not by a green build.

---

The single biggest surprise: vega/editor does not use vega-embed for its live preview. It builds the compile→parse→View pipeline by hand; vega-embed is imported only for types and the exported standalone-HTML snippet. This is good news — vega-embed is exactly the wrapper they wrote by hand, so we get it for free. But their hand-rolled version (src/components/renderer/renderer.tsx) is the best available documentation of the lifecycle/cleanup discipline vega-embed still expects from us.

---

1 · Monaco + Vega-Lite schema wiring (M2 — highest from-scratch risk)

All of vega/editor's Monaco setup is one file: src/utils/monaco.ts.

What to borrow:

• Bundle the schema; never fetch it. They import vegaLiteSchema from 'vega-lite/vega-lite-schema.json', resolved by a Vite alias to the package's build/ output (monaco.ts:7-8, vite.config.ts). The schema version is pinned to the installed vega-lite — offline-safe, version-locked, no runtime network call. Astrolabe should do the same.
• Attach via the JSON language service, once, globally: monaco.languages.json.jsonDefaults.setDiagnosticsOptions({ schemas, validate:true, ... }) (monaco.ts:51-57).
• markdownDescription patch (monaco.ts:12-13, utils/markdownProps.ts): recursively copy every schema description → markdownDescription before registering. Monaco renders rich hover docs only from markdownDescription; without this, hovers are plain text. Do it once at setup.
• Replace the built-in JSON formatter with json-stringify-pretty-compact via registerDocumentFormattingEditProvider('json', …) (monaco.ts:60-61,71-80) for Vega's compact array-on-one-line style.
• Editor options worth copying (spec-editor/renderer.tsx:263-274): folding:true, minimap.enabled:false, scrollBeyondLastLine:false, wordWrap:'on', quickSuggestions:true (this is what makes schema completions appear without an explicit trigger), stickyScroll.enabled:false.

Gotchas / where we improve:

• ⚠️ Workers are on us. vega/editor never configures Monaco workers — the CDN loader does. With raw monaco-editor + Vite we must set self.MonacoEnvironment.getWorker to return the json.worker for label 'json' and editor.worker otherwise (via ?worker imports):

  import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
  import JsonWorker from 'monaco-editor/esm/vs/language/json/json.worker?worker';
  self.MonacoEnvironment = {
    getWorker: (_id, label) => (label === 'json' ? new JsonWorker() : new EditorWorker()),
  };
  

  The json.worker runs schema validation + autocomplete. No worker ⇒ no squiggles, no completions. Upside: dropping the CDN loader makes monaco synchronously importable — no await loader.init() dance, just call setDiagnosticsOptions(...) at module load.
• ⚠️ $schema-based binding vs fileMatch. They register schemas under versioned uris (.../vega-lite/v6.json) and bind by matching the doc's $schema value — no fileMatch (monaco.ts:15-46). Consequence: a spec with no $schema gets zero validation/autocomplete. Astrolabe should prefer fileMatch against our model URIs so validation works regardless of whether the user wrote a $schema line.
• ⚠️ Set enableSchemaRequest:false for our offline-first app. They set it true (monaco.ts:54), which lets the worker network-fetch any unbundled $schema URL — failing network calls for an offline app. Register all schema versions locally instead.
• The schema is multi-MB; register it once globally, never per-model.

2 · Live preview with vega-embed (M1 lifecycle, M2 fit-mode)

This is doc 05's territory; these are the concrete details vega/editor's hand-rolled renderer (src/components/renderer/renderer.tsx) reveals.

What to borrow:

• Theme = a vega-themes config object merged into the spec config. There is no automatic light/dark sync in vega/editor — theme is an explicit choice baked in at compile (config-editor/config-editor-header.tsx:5-37). For Astrolabe: pass the chosen theme/config to vegaEmbed, and when our theme changes, re-embed with the new config.
• "width":"container" / "height":"container" is how VL responsiveness works — it compiles to a containerSize signal (renderer.tsx:78-90 detects this). Pair it with a ResizeObserver on the preview pane → view.resize().runAsync(). This is cleaner than vega/editor's window.dispatchEvent(new Event('resize')) hack (renderer.tsx:101-122) and is the mechanism behind our M2 fit-mode contract.
• Reuse the view for cheap changes. They rebuild the View only on spec change; renderer (svg/canvas) and tooltip toggles re-initialize() the existing view (renderer.tsx:367-371).
• Capture warnings separately from errors via a buffering logger (see §4's LocalLogger).

Gotchas / where we improve:

• ⚠️ Finalize before re-embed, or leak. Every spec change must view.finalize() the old view and clear the container before mounting the new one (renderer.tsx:218-226). vegaEmbed returns { view, finalize } — call finalize() before the next embed and on unmount. This is already a Do-rule in doc 05; vega/editor confirms how easy it is to leak otherwise.
• ⚠️ Race on rapid edits. runAsync is async; a stale render can resolve after a newer one mounts. vega/editor mitigates only with debounce. We should add a render-generation token and ignore stale resolves (an improvement over the reference).
• ⚠️ Wrap runAsync in try/catch and finalize on failure — Vega won't catch runtime errors for you, and a half-initialized view leaks if you don't finalize (renderer.tsx:247-259).
• The CSP-safe expression interpreter (vega-interpreter + vega.parse(..., {ast:true})) matters only under a strict no-eval CSP. A local offline app doesn't need it — keep it opt-in.

3 · Two-tier validation & error surfacing (M2, spec §03E)

vega/editor runs two independent schema-validation systems with no reconciliation, and sorts errors into two tiers. Both are worth copying.

The two layers:

1. Monaco JSON worker → inline squiggles, hovers, autocomplete in the editor.
2. ajv ^8 (src/utils/validate.ts) → runs at parse time, feeds the error/log pane. It does not create editor markers.

The two error tiers (keep them separate):

• Fatal / blocking — thrown exceptions: JSON syntax error, VL compile error, Vega runtime error. These set a single error and suppress the chart.
• Advisory — ajv schema-validation findings and $schema version mismatch. These are a warnings list and do not block rendering. (Vega-Lite emits many benign warnings; treating ajv output as fatal would wrongly hide specs that render fine.)

The orchestration is app.tsx:188-291: parseJSONCOrThrow → $schema semver check (warn) → validateVegaLite (ajv, warn) → vegaLite.compile (throw=fatal) → render (renderer.tsx, throw=fatal).

ajv setup specifics that will bite a from-scratch impl (validate.ts:9-17):

• new Ajv({ strict: false }) — the VL/Vega schemas fail ajv strict-mode at compile time otherwise.
• The VL schema is draft-06 → must ajv.addMetaSchema(json-schema-draft-06.json) (ajv 8 defaults to draft-07/2020) or compile throws.
• Register a no-op color-hex format (ajv.addFormat('color-hex', () => true)) plus addFormats(ajv); the schema references formats ajv-formats doesn't cover.
• Compile the validator once at module load and cache it — the schema is huge; compiling per keystroke is a perf killer.

Where we improve: ajv errors are shown as JSON-pointer text (e.g. /encoding/x) with no editor position — vega/editor does not map them to markers. Only JSON syntax errors get a line/col (via jsonc-parser's visitor, utils/jsonc-parser.ts:3-17). If our §03E wants inline ajv markers, we map instancePath → editor offsets ourselves via jsonc-parser's node tree — something the reference does not do.

4 · Data flow & debouncing (M1/M2 — translate to Zustand stores)

vega/editor keeps editorString (the text) as the single source of truth; the parsed spec and compiled Vega spec are derived and recomputed by a subscriber when text/mode/config change (app.tsx:338-365). Errors don't clobber the last-good derived specs.

The Zustand-store translation (this is the shape to build):

text  (store field, debounced writer on editor change)
  └─▶ parsedSpec   (derived: JSONC parse + collect syntax/diagnostic errors)
        └─▶ renderInput (derived: prepareSpecForRender — refs, fit-mode)
              └─▶ effect: deep-equal guard → vegaEmbed(); finalize previous view

What to borrow:

• Debounce only at edit→state, not state→render. vega/editor debounces the editor at 1200 ms (spec-editor/renderer.tsx:66) and guards the render with a deepEqual prop diff (renderer.tsx:340-349). (1200 ms is their number; tune ours — our settings expose a render-debounce preference.)
• A manual-parse escape hatch (Ctrl/Cmd+S re-parses without waiting) maps to a future live-vs-manual preview toggle (renderer.tsx:89-111).
• LocalLogger pattern (utils/logger.ts): a logger that buffers errors/warns/infos/debugs into arrays instead of writing to console. This lets a pure src/core compile/validate step return structured diagnostics with zero browser coupling — e.g. validateSpec(spec) → { errors, warns }. Ideal core-first fit.
• json-stringify-pretty-compact for the format action and prettify-on-load — much nicer than JSON.stringify(…, null, 2) for VL specs.

Persistence note: vega/editor snapshots its whole state to localStorage on every change, stripping non-serializable fields (view, runtime, editor refs) and restoring via { ...DEFAULT_STATE, ...parsed } (context/app-context.tsx). Our debounced auto-save to IndexedDB (doc 01/02) is the better pattern — but the "strip non-serializable, restore with defaults-spread" discipline is worth keeping.

---

Borrow list (where each lands)

Technique	Lands in	Milestone
Bundle VL schema from package build/; setDiagnosticsOptions	src/app/infrastructure/ Monaco setup	M2
markdownDescription patch + compact formatter	Monaco setup	M2
Explicit Vite worker wiring (MonacoEnvironment.getWorker)	Monaco setup	M2
fileMatch schema binding (improvement over $schema-only)	Monaco setup	M2
jsonc-parser tolerant parse + line/col syntax errors	src/core/	M1/M2
ajv wrapper (strict:false, draft-06, color-hex, compile-once) → structured diagnostics	src/core/	M2
LocalLogger-style buffered diagnostics from pure compile	src/core/	M2
Fatal-vs-advisory two-tier error model	rendering/store contract	M1/M2
"container" sizing + ResizeObserver → view.resize()	rendering.ts + LivePreview	M2
finalize()-before-reembed + render-generation guard	LivePreview	M1
theme = vega-themes config merged into vegaEmbed	preview + settings	M5
json-stringify-pretty-compact format action	editor	M2

Where we deliberately do better than the reference

• Wire Monaco workers explicitly (they sidestep it via the CDN loader).
• Map ajv errors to editor positions via jsonc-parser offsets (they show pointer text only).
• Render-generation guard against stale async renders (they rely on debounce alone).
• fileMatch-based schema binding so validation works without a $schema line.
• Debounced auto-save to IndexedDB rather than write-the-whole-state-on-every-change.

Key files in the reference (for deeper reads)

• src/utils/monaco.ts — all Monaco/schema wiring
• src/utils/markdownProps.ts — the markdownDescription patch
• src/utils/validate.ts — ajv setup + cached validators
• src/utils/jsonc-parser.ts — tolerant parse + line/col syntax errors
• src/utils/logger.ts — LocalLogger / DispatchingLogger
• src/components/renderer/renderer.tsx — the hand-rolled View lifecycle (finalize, sizing, errors)
• src/components/app.tsx:188-365 — parse → $schema check → ajv → compile → render orchestration
• src/components/error-pane/renderer.tsx — error/log display
• src/constants/default-state.ts — the full app-state shape
• src/components/input-panel/spec-editor/renderer.tsx — editor component, 1200ms debounce, $schema→mode detection