astrolabe/docs/architecture/08-vega-editor-techniques.md

08 · Borrowed Techniques from vega/editor

The official Vega-Lite editor (vega/editor) solves the exact "edit a Vega-Lite spec as JSON, validate it, render it live" problem Astrolabe sits on top of — minus the snippet/dataset library. This doc distills the techniques worth borrowing and the gotchas worth avoiding, so we don't rediscover them from scratch in M1/M2.

It is a reference, not a contract. The behavioral contract is still docs/spec/; the patterns are still docs 01–07. This doc is the bridge: "here is how the canonical implementation does the editor/renderer plumbing, and here is what we keep vs. improve."

Source of these findings

A read-only clone of vega/editor lives at /Users/oleh/code/reference/vega-editor (shallow clone of main, HEAD 4fdbb59). Re-clone with git clone --depth 1 https://github.com/vega/editor. Citations below are file:line into that tree.

Stack delta (read this first — it changes how directly we can borrow)

	vega/editor	Astrolabe
UI framework	React	React (moved off Preact before build start)
State	Redux-ish single State in React context (useState)	Zustand stores — not Redux
Monaco	@monaco-editor/react + @monaco-editor/loader (CDN-loaded Monaco, workers auto-wired)	raw monaco-editor via Vite (we must wire workers ourselves)
Rendering	hand-rolled vegaLite.compile → vega.parse → new vega.View().runAsync()	vegaEmbed() (wraps that same pipeline)
Schema validation	Monaco JSON worker + standalone ajv ^8 (two independent layers)	same two-layer model planned

Because both apps are now React, vega/editor's component lifecycle patterns port more or less directly — the friction is only in (a) state (their Redux-flat-state → our Zustand stores) and (b) Monaco worker wiring (their CDN loader → our explicit Vite workers).

---

Decision · Monaco integration (self-hosted, raw API)

Decided. Astrolabe uses raw monaco-editor from npm, bundled and self-hosted, with workers wired explicitly via Vite ?worker — not vega/editor's @monaco-editor/react + @monaco-editor/loader (CDN) setup. Two independent axes:

Axis A — sourcing: self-hosted/bundled, not CDN. (Forced by Astrolabe's values.) vega/editor's @monaco-editor/loader fetches Monaco's AMD bundle from a CDN at runtime. For us that breaks three things at once: (1) offline — the CDN bundle is outside Vite's module graph, so vite-plugin-pwa/Workbox never precaches it and offline silently fails; bundled npm assets are hashed files in dist/ that Workbox precaches automatically; (2) privacy — a third-party fetch on load contradicts SOUL's "the only outbound requests are user-created URL-dataset fetches"; (3) determinism — npm + package-lock is integrity-pinned and reproducible, a runtime CDN resolve is not. This axis is not a close call; vega/editor's CDN choice is right for an online hosted tool and wrong for an offline, installable, private app.

Axis B — React integration: raw API, not @monaco-editor/react. (A lean, not forced.) The wrapper helps with the easy 80% (mount a JSON editor, lifecycle) and adds nothing to the load-bearing 20% this app needs:

• Workers are still ours — the wrapper never manages MonacoEnvironment (see §1 gotcha).
• The M2 schema service (jsonDefaults.setDiagnosticsOptions, fileMatch) is namespace-level; you reach through the wrapper via onMount, so it saves nothing there.
• Its headline value/onChange controlled-input model is a hazard: driving Monaco's content from React state causes cursor jumps and undo-stack churn, against §10's "typing stays fluid" — you end up using it uncontrolled, i.e. the raw pattern anyway.
• Its CDN-by-default is a standing footgun (works in dev online, fails offline in prod unless you remember loader.config({ monaco })).

Against that, raw costs one testable useMonacoEditor hook (~50–80 lines: create in useEffect, dispose on unmount, push value, subscribe to onDidChangeModelContent, resize). That's the same imperative-teardown discipline already adopted for vega-embed in doc 05 (view.finalize()), and consistent with already using raw vegaEmbed() over a React chart wrapper — "thin integration layers we own" (SOUL). Lock-in is low either way, so the final raw-vs-wrapper call is confirmable at the Monaco spike; what is not up for revisiting is self-hosting.

Accepted cost: the explicit worker wiring (§1) is inherent to self-hosting — it is the price of offline, paid in any non-CDN setup, and the wrapper would not remove it.

Entry point: edcore.main, never editor.api (trim languages, not features)

Self-hosting raw Monaco forces a choice of ESM entry point, and the granularity matters:

Import	What you get	Use?
monaco-editor (barrel)	All features + every basic language (sql, abap, solidity, …)	❌ language bloat (~20 dead chunks)
esm/vs/editor/editor.api	The API surface only — zero feature contributions	❌ a text box: no folding, suggest widget, Cmd+Backspace, find, bracket colorization
esm/vs/editor/edcore.main	editor.all (all 59 feature contributions) + API, no languages	✅ full editor UX, JSON-only weight

Import edcore.main and add only the JSON language service (esm/vs/language/json/monaco.contribution). edcore.main ships no .d.ts of its own — add an ambient declare module … { export * from '…/editor.api'; } so types (and Monaco's global MonacoEnvironment augmentation) resolve. Two editor options worth setting because they bite Vega-Lite specs specifically: showFoldingControls: 'always' (fold arrows always visible), and quickSuggestions: { strings: true } (VL enum values like "bar" live inside JSON strings, where Monaco disables auto-suggest by default).

Reaching for editor.api to "drop unused languages" silently strips every editor feature — the languages live elsewhere. This is the concrete case behind AGENTS.md's "trim content, not capability" rule: cut the unwanted content, keep the behavior, and verify the behavior survived by exercising the editor, not by a green build.

---

The single biggest surprise: vega/editor does not use vega-embed for its live preview. It builds the compile→parse→View pipeline by hand; vega-embed is imported only for types and the exported standalone-HTML snippet. This is good news — vega-embed is exactly the wrapper they wrote by hand, so we get it for free. But their hand-rolled version (src/components/renderer/renderer.tsx) is the best available documentation of the lifecycle/cleanup discipline vega-embed still expects from us.

---

1 · Monaco + Vega-Lite schema wiring (M2 — highest from-scratch risk)

All of vega/editor's Monaco setup is one file: src/utils/monaco.ts.

What to borrow:

• Bundle the schema; never fetch it. They import vegaLiteSchema from 'vega-lite/vega-lite-schema.json', resolved by a Vite alias to the package's build/ output (monaco.ts:7-8, vite.config.ts). The schema version is pinned to the installed vega-lite — offline-safe, version-locked, no runtime network call. Astrolabe should do the same.
• Attach via the JSON language service, once, globally: monaco.languages.json.jsonDefaults.setDiagnosticsOptions({ schemas, validate:true, ... }) (monaco.ts:51-57).
• markdownDescription patch (monaco.ts:12-13, utils/markdownProps.ts): recursively copy every schema description → markdownDescription before registering. Monaco renders rich hover docs only from markdownDescription; without this, hovers are plain text. Do it once at setup.
• Replace the built-in JSON formatter with json-stringify-pretty-compact via registerDocumentFormattingEditProvider('json', …) (monaco.ts:60-61,71-80) for Vega's compact array-on-one-line style.
• Editor options worth copying (spec-editor/renderer.tsx:263-274): folding:true, minimap.enabled:false, scrollBeyondLastLine:false, wordWrap:'on', quickSuggestions:true (this is what makes schema completions appear without an explicit trigger), stickyScroll.enabled:false.

Gotchas / where we improve:

• ⚠️ Workers are on us. vega/editor never configures Monaco workers — the CDN loader does. With raw monaco-editor + Vite we must set self.MonacoEnvironment.getWorker to return the json.worker for label 'json' and editor.worker otherwise (via ?worker imports):

  import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
  import JsonWorker from 'monaco-editor/esm/vs/language/json/json.worker?worker';
  self.MonacoEnvironment = {
    getWorker: (_id, label) => (label === 'json' ? new JsonWorker() : new EditorWorker()),
  };
  

  The json.worker runs schema validation + autocomplete. No worker ⇒ no squiggles, no completions. Upside: dropping the CDN loader makes monaco synchronously importable — no await loader.init() dance, just call setDiagnosticsOptions(...) at module load.
• ⚠️ $schema-based binding vs fileMatch. They register schemas under versioned uris (.../vega-lite/v6.json) and bind by matching the doc's $schema value — no fileMatch (monaco.ts:15-46). Consequence: a spec with no $schema gets zero validation/autocomplete. Astrolabe should prefer fileMatch against our model URIs so validation works regardless of whether the user wrote a $schema line.
• ⚠️ Set enableSchemaRequest:false for our offline-first app. They set it true (monaco.ts:54), which lets the worker network-fetch any unbundled $schema URL — failing network calls for an offline app. Register all schema versions locally instead.
• The schema is multi-MB; register it once globally, never per-model.

2 · Live preview with vega-embed (M1 lifecycle, M2 fit-mode)

This is doc 05's territory; these are the concrete details vega/editor's hand-rolled renderer (src/components/renderer/renderer.tsx) reveals.

What to borrow:

• Theme = a vega-themes config object merged into the spec config. There is no automatic light/dark sync in vega/editor — theme is an explicit choice baked in at compile (config-editor/config-editor-header.tsx:5-37). For Astrolabe: pass the chosen theme/config to vegaEmbed, and when our theme changes, re-embed with the new config.
• "width":"container" / "height":"container" is how VL responsiveness works — it compiles width/height to signals that re-read containerSize() only on a window:resize event (renderer.tsx:78-90 detects container sizing). Two things a from-scratch impl will get wrong (we did): (1) view.resize().runAsync() does not re-measure — it re-runs layout with the stale size; (2) a pane drag fires no window resize, so nothing re-fits on its own. The fix is exactly vega/editor's window.dispatchEvent(new Event('resize')) (renderer.tsx:101-122) — not a hack, the actual mechanism — driven by a ResizeObserver on the pane. It also leaves a non-container dimension natural for free. Full write-up in doc 05 §8.
• Reuse the view for cheap changes. They rebuild the View only on spec change; renderer (svg/canvas) and tooltip toggles re-initialize() the existing view (renderer.tsx:367-371).
• Capture warnings separately from errors via a buffering logger (see §4's LocalLogger).

Gotchas / where we improve:

• ⚠️ Finalize before re-embed, or leak. Every spec change must view.finalize() the old view and clear the container before mounting the new one (renderer.tsx:218-226). vegaEmbed returns { view, finalize } — call finalize() before the next embed and on unmount. This is already a Do-rule in doc 05; vega/editor confirms how easy it is to leak otherwise.
• ⚠️ Race on rapid edits. runAsync is async; a stale render can resolve after a newer one mounts. vega/editor mitigates only with debounce. We should add a render-generation token and ignore stale resolves (an improvement over the reference).
• ⚠️ Wrap runAsync in try/catch and finalize on failure — Vega won't catch runtime errors for you, and a half-initialized view leaks if you don't finalize (renderer.tsx:247-259).
• The CSP-safe expression interpreter (vega-interpreter + vega.parse(..., {ast:true})) matters only under a strict no-eval CSP. A local offline app doesn't need it — keep it opt-in.

3 · Two-tier validation & error surfacing (M2, spec §03E)

vega/editor runs two independent schema-validation systems with no reconciliation, and sorts errors into two tiers. Both are worth copying.

The two layers:

1. Monaco JSON worker → inline squiggles, hovers, autocomplete in the editor.
2. ajv ^8 (src/utils/validate.ts) → runs at parse time, feeds the error/log pane. It does not create editor markers.

The two error tiers (keep them separate):

• Fatal / blocking — thrown exceptions: JSON syntax error, VL compile error, Vega runtime error. These set a single error and suppress the chart.
• Advisory — ajv schema-validation findings and $schema version mismatch. These are a warnings list and do not block rendering. (Vega-Lite emits many benign warnings; treating ajv output as fatal would wrongly hide specs that render fine.)

The orchestration is app.tsx:188-291: parseJSONCOrThrow → $schema semver check (warn) → validateVegaLite (ajv, warn) → vegaLite.compile (throw=fatal) → render (renderer.tsx, throw=fatal).

ajv setup specifics that will bite a from-scratch impl (validate.ts:9-17):

• new Ajv({ strict: false }) — the VL/Vega schemas fail ajv strict-mode at compile time otherwise.
• The VL schema is draft-06 → must ajv.addMetaSchema(json-schema-draft-06.json) (ajv 8 defaults to draft-07/2020) or compile throws.
• Register a no-op color-hex format (ajv.addFormat('color-hex', () => true)) plus addFormats(ajv); the schema references formats ajv-formats doesn't cover.
• Compile the validator once at module load and cache it — the schema is huge; compiling per keystroke is a perf killer.

Where we improve: ajv errors are shown as JSON-pointer text (e.g. /encoding/x) with no editor position — vega/editor does not map them to markers. Only JSON syntax errors get a line/col (via jsonc-parser's visitor, utils/jsonc-parser.ts:3-17). If our §03E wants inline ajv markers, we map instancePath → editor offsets ourselves via jsonc-parser's node tree — something the reference does not do.

4 · Data flow & debouncing (M1/M2 — translate to Zustand stores)

vega/editor keeps editorString (the text) as the single source of truth; the parsed spec and compiled Vega spec are derived and recomputed by a subscriber when text/mode/config change (app.tsx:338-365). Errors don't clobber the last-good derived specs.

The Zustand-store translation (this is the shape to build):

text  (store field, debounced writer on editor change)
  └─▶ parsedSpec   (derived: JSONC parse + collect syntax/diagnostic errors)
        └─▶ renderInput (derived: prepareSpecForRender — refs, fit-mode)
              └─▶ effect: deep-equal guard → vegaEmbed(); finalize previous view

What to borrow:

• Debounce only at edit→state, not state→render. vega/editor debounces the editor at 1200 ms (spec-editor/renderer.tsx:66) and guards the render with a deepEqual prop diff (renderer.tsx:340-349). (1200 ms is their number; tune ours — our settings expose a render-debounce preference.)
• A manual-parse escape hatch (Ctrl/Cmd+S re-parses without waiting) maps to a future live-vs-manual preview toggle (renderer.tsx:89-111).
• LocalLogger pattern (utils/logger.ts): a logger that buffers errors/warns/infos/debugs into arrays instead of writing to console. This lets a pure src/core compile/validate step return structured diagnostics with zero browser coupling — e.g. validateSpec(spec) → { errors, warns }. Ideal core-first fit.
• json-stringify-pretty-compact for the format action and prettify-on-load — much nicer than JSON.stringify(…, null, 2) for VL specs.

Persistence note: vega/editor snapshots its whole state to localStorage on every change, stripping non-serializable fields (view, runtime, editor refs) and restoring via { ...DEFAULT_STATE, ...parsed } (context/app-context.tsx). Our debounced auto-save to IndexedDB (doc 01/02) is the better pattern — but the "strip non-serializable, restore with defaults-spread" discipline is worth keeping.

---

5 · Editor augmentation (our layer over the borrowed base)

Beyond schema validation/completion (§1), the spec editor adds structural refactors and data-aware hints — the edits that are awkward in raw JSON and out of reach of the single-view visual builder. All transform logic is pure src/core/; the Monaco glue is thin app-layer services.

Core (pure, portable):

• spec-transforms — wrap a view in layer/hconcat/vconcat/facet/repeat; collapse a single-child composition (unwrapSingleton). Object-in/object-out.
• spec-data — the Vega-Lite data model: classify a data block (classifyData, mirroring isNamedData), the library reference name (libraryRefName), and the data binding in scope at a cursor path (dataBindingAtPath, honoring a view's data inheritance from its parent).
• spec-cursor — findViewRange (cursor offset → the enclosing view's byte range), pathAtOffset (cursor → JSON path), and valueKeyAtOffset/stringValueAtOffset (the JSON context at the cursor), over jsonc-parser.
• spec-fields — field names a spec's transforms introduce (their as), scoped to a cursor's ancestor chain (derivedFieldNamesAtPath).
• spec-inline-data — the rows a specific data binding carries for profiling (rowsForDataBinding: inline values, or a self-defined datasets entry).
• spec-insert — the composition the cursor is in (compositionTargetAt), inserting a view at an index (insertView) and reordering siblings (moveView swaps a neighbour, moveViewTo slides to any index), plus elementOffset to re-find a view after the edit. It also owns the shared SpecPath walkers (valueAtPath/arrayAtPath/isPrefixPath) — a module navigating a path reuses these rather than re-inlining the array/object descent.
• spec-view-tree — the whole composition as a recursive tree (viewTree), each node carrying its operator, orientation and byte range. Read at once (vs. spec-insert's one-array edits) to drive the composition wireframe — a schematic of the multi-view structure in a preview-toolbar disclosure (CompositionWireframe); clicking a box reveals that view's range in the editor via AppStore.requestRevealView, and on the draft it is drag-editable (reorder + restructure). Interaction contract in arch 10.
• spec-restructure — the path-targeted cross-container moves behind the wireframe's drag. wrapViews(target, source, axis, side) pairs the dragged source beside the drop target in a new concat (placed where the target was, the source removed), enforcing three invariants: flatten a bare same-orientation concat nested directly in a concat (so a with-axis drop reads as a plain insert, not redundant nesting), collapse the source's emptied container (unwrap a one-child, drop a zero-child, recursing up the chain), and data-pin a source's inherited data before it changes ancestor (dataBindingAtPath, so it never silently rebinds). Degenerate drops — onto itself, its own ancestor/descendant, or the root — return null.

Services (app, store-aware via getState): spec-transform-actions (the wrap/simplify/add-view operations and their surfaces), spec-dataset-hints (completion, hover, inlay providers), active-dataset (dataInfoAt(text, offset) — the columns/types/stats plus derived fields the draft sees at the cursor). SpecEditor does the wiring.

Decision rules:

• Provider lifetime — global-once vs per-editor. Language providers that need no editor handle (code actions, completion, hover, inlay) register once for json, like the schema and formatter; per-editor registration would duplicate them on remount. Pieces that need the editor handle — the addAction context/F1 commands, and the CodeLens whose command runs executeEdits — are installed per editor and disposed with it.
• Transform scope. A transform targets, in order: an explicit selection → the view the cursor sits in (findViewRange) → the whole document. A "view" is a composition-array element or a facet/repeat spec child; a flat unit spec has no inner view, so it scopes to the whole document. jsonc-parser is error-tolerant, so scoping holds mid-edit; the path logic stays in core and only the Monaco Range is built in the service.
• One edit path. The lightbulb returns a WorkspaceEdit (no editor handle); the toolbar and palette use executeEdits. Both build the replacement through the same serialize-and-reindent step, bracketed by pushUndoStop, so ⌘Z restores the prior text.
• Wireframe restructuring is one core op + the editor's undo. Every drag resolves to either moveViewTo (reorder within one container) or wrapViews (everything else — wrap, cross-container move, insert); the wireframe only requests it (AppStore.requestComposeMove/requestComposeWrap) and SpecEditor applies it through the same executeEdits + pushUndoStop path as the other transforms, so a drag is one ⌘Z and the editor stays the single text source. wrapViews covers wrap and insert with one operation because it flattens bare same-orientation nesting afterward; the drag's edge-zone interaction model lives in arch 10.
• Composition CodeLens is cursor-scoped. It follows the view the cursor sits in — ＋ Add view above/below at the view's edges and ↑/↓ Move to reorder among its siblings — rather than one fixed button per composition array; an empty composition shows a single ＋ Add view, and the F1 palette mirrors all four for the keyboard. The provider reads editor.getPosition() and refreshes via an onDidChange emitter fired on cursor moves (keyed to the enclosing view, so typing inside one view doesn't churn the lenses). After an edit the cursor follows the affected view (elementOffset), so a repeated click keeps acting on it instead of the neighbour that slid into place. Cursor-scoping does not strand a load-bearing action (arch 10 — revealed actions): editing a composition puts the cursor in a view exactly when add/reorder is wanted, and the palette is the ever-present path for the keyboard.
• Field source for hints (view-scoped). dataInfoAt(text, offset) resolves the data binding of the cursor's nearest enclosing view (dataBindingAtPath — a child inherits a parent's data unless it declares its own), then its columns: a named library dataset (matched case-insensitively, like the renderer), else the binding's inline rows profiled on the fly (rowsForDataBinding + core/profile) — a "ghost dataset" with nothing stored. Derived fields come from that view's and its ancestors' transforms only (derivedFieldNamesAtPath), so a sibling view's calculate does not leak in. Profiling is memoized per (draft text, enclosing view), so the inlay provider's many per-line queries profile once. A composed spec whose views bind different datasets therefore gets the right columns per view. Out of scope: data-dependent derived columns (pivot/lookup output) and url/CSV-string inline data, which need the pipeline run or format-aware parsing.
• No unknown-field diagnostic. Hints are additive and forgiving, so over- or under-listing costs nothing; a "field not in data" squiggle would false-positive on every derived or data-dependent field, so there is deliberately none.
• Code-action menu icons are kind-derived (a wrench for the refactor.* kinds) — Monaco's CodeAction carries no icon field. Custom iconography lives only where it is supported: CodeLens titles ($(codicon)), completion-item kinds, and glyph-margin decorations.

jsonc-parser is a direct dependency (Monaco bundles its own copy internally but does not re-export it). A standalone editor-augmentation-demo.html loads Monaco from a CDN to exercise these provider surfaces in isolation.

Borrow list (where each lands)

Technique	Lands in	Milestone
Bundle VL schema from package build/; setDiagnosticsOptions	src/app/infrastructure/ Monaco setup	M2
markdownDescription patch + compact formatter	Monaco setup	M2
Explicit Vite worker wiring (MonacoEnvironment.getWorker)	Monaco setup	M2
fileMatch schema binding (improvement over $schema-only)	Monaco setup	M2
jsonc-parser tolerant parse + line/col syntax errors	src/core/	M1/M2
ajv wrapper (strict:false, draft-06, color-hex, compile-once) → structured diagnostics	src/core/	M2
LocalLogger-style buffered diagnostics from pure compile	src/core/	M2
Fatal-vs-advisory two-tier error model	rendering/store contract	M1/M2
"container" sizing + ResizeObserver → synthetic window:resize (not view.resize())	chart-renderer + LivePreview	M2
finalize()-before-reembed + render-generation guard	LivePreview	M1
theme = vega-themes config merged into vegaEmbed	preview + settings	M5
json-stringify-pretty-compact format action	editor	M2

Where we deliberately do better than the reference

• Wire Monaco workers explicitly (they sidestep it via the CDN loader).
• Map ajv errors to editor positions via jsonc-parser offsets (they show pointer text only).
• Render-generation guard against stale async renders (they rely on debounce alone).
• fileMatch-based schema binding so validation works without a $schema line.
• Debounced auto-save to IndexedDB rather than write-the-whole-state-on-every-change.

Key files in the reference (for deeper reads)

• src/utils/monaco.ts — all Monaco/schema wiring
• src/utils/markdownProps.ts — the markdownDescription patch
• src/utils/validate.ts — ajv setup + cached validators
• src/utils/jsonc-parser.ts — tolerant parse + line/col syntax errors
• src/utils/logger.ts — LocalLogger / DispatchingLogger
• src/components/renderer/renderer.tsx — the hand-rolled View lifecycle (finalize, sizing, errors)
• src/components/app.tsx:188-365 — parse → $schema check → ajv → compile → render orchestration
• src/components/error-pane/renderer.tsx — error/log display
• src/constants/default-state.ts — the full app-state shape
• src/components/input-panel/spec-editor/renderer.tsx — editor component, 1200ms debounce, $schema→mode detection