# 08 · Borrowed Techniques from vega/editor

> The official Vega-Lite editor ([vega/editor](https://github.com/vega/editor)) solves the
> exact "edit a Vega-Lite spec as JSON, validate it, render it live" problem Astrolabe sits
> on top of — minus the snippet/dataset library. This doc distills the techniques worth
> borrowing and the gotchas worth avoiding, so we don't rediscover them from scratch in
> M1/M2.
>
> It is a **reference**, not a contract. The behavioral contract is still [`docs/spec/`](../spec/);
> the patterns are still docs [01](01-state-and-stores.md)–[07](07-naming-and-relationships.md).
> This doc is the bridge: "here is how the canonical implementation does the editor/renderer
> plumbing, and here is what we keep vs. improve."

## Source of these findings

A read-only clone of vega/editor lives at `/Users/oleh/code/reference/vega-editor` (shallow
clone of `main`, HEAD `4fdbb59`). Re-clone with
`git clone --depth 1 https://github.com/vega/editor`. Citations below are `file:line` into
that tree.

## Stack delta (read this first — it changes how directly we can borrow)

|                   | vega/editor                                                                                  | Astrolabe                                                             |
| ----------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| UI framework      | **React**                                                                                    | **React** (moved off Preact before build start)                       |
| State             | Redux-ish single `State` in React context (`useState`)                                       | Zustand **stores** — _not_ Redux                                      |
| Monaco            | `@monaco-editor/react` + `@monaco-editor/loader` (CDN-loaded Monaco, **workers auto-wired**) | **raw `monaco-editor`** via Vite (**we must wire workers ourselves**) |
| Rendering         | **hand-rolled** `vegaLite.compile` → `vega.parse` → `new vega.View().runAsync()`             | **`vegaEmbed()`** (wraps that same pipeline)                          |
| Schema validation | Monaco JSON worker **+** standalone `ajv ^8` (two independent layers)                        | same two-layer model planned                                          |

Because both apps are now React, vega/editor's **component lifecycle patterns port more or
less directly** — the friction is only in (a) state (their Redux-flat-state → our Zustand
stores) and (b) Monaco worker wiring (their CDN loader → our explicit Vite workers).

---

## Decision · Monaco integration (self-hosted, raw API)

> **Decided.** Astrolabe uses **raw `monaco-editor` from npm, bundled and self-hosted**, with
> workers wired explicitly via Vite `?worker` — **not** vega/editor's
> `@monaco-editor/react` + `@monaco-editor/loader` (CDN) setup. Two independent axes:

**Axis A — sourcing: self-hosted/bundled, not CDN. (Forced by Astrolabe's values.)**
vega/editor's `@monaco-editor/loader` fetches Monaco's AMD bundle from a CDN at runtime. For
us that breaks three things at once: (1) **offline** — the CDN bundle is outside Vite's module
graph, so `vite-plugin-pwa`/Workbox never precaches it and offline silently fails; bundled npm
assets are hashed files in `dist/` that Workbox precaches automatically; (2) **privacy** — a
third-party fetch on load contradicts SOUL's "the only outbound requests are user-created
URL-dataset fetches"; (3) **determinism** — npm + `package-lock` is integrity-pinned and
reproducible, a runtime CDN resolve is not. This axis is not a close call; vega/editor's CDN
choice is right _for an online hosted tool_ and wrong for an offline, installable, private app.

**Axis B — React integration: raw API, not `@monaco-editor/react`. (A lean, not forced.)**
The wrapper helps with the easy 80% (mount a JSON editor, lifecycle) and adds nothing to the
load-bearing 20% this app needs:

- **Workers** are still ours — the wrapper never manages `MonacoEnvironment` (see §1 gotcha).
- The **M2 schema service** (`jsonDefaults.setDiagnosticsOptions`, `fileMatch`) is namespace-level;
  you reach _through_ the wrapper via `onMount`, so it saves nothing there.
- Its headline **`value`/`onChange` controlled-input model is a hazard**: driving Monaco's
  content from React state causes cursor jumps and undo-stack churn, against §10's "typing
  stays fluid" — you end up using it uncontrolled, i.e. the raw pattern anyway.
- Its **CDN-by-default** is a standing footgun (works in dev online, fails offline in prod
  unless you remember `loader.config({ monaco })`).

Against that, raw costs **one testable `useMonacoEditor` hook** (~50–80 lines: create in
`useEffect`, `dispose` on unmount, push value, subscribe to `onDidChangeModelContent`, resize).
That's the **same imperative-teardown discipline already adopted for `vega-embed`** in doc 05
(`view.finalize()`), and consistent with already using raw `vegaEmbed()` over a React chart
wrapper — "thin integration layers we own" (SOUL). Lock-in is low either way, so the final
raw-vs-wrapper call is confirmable at the Monaco spike; what is **not** up for revisiting is
self-hosting.

**Accepted cost:** the explicit worker wiring (§1) is inherent to self-hosting — it is the
price of offline, paid in any non-CDN setup, and the wrapper would not remove it.

### Entry point: `edcore.main`, never `editor.api` (trim languages, not features)

Self-hosting raw Monaco forces a choice of ESM entry point, and the granularity matters:

| Import                      | What you get                                                        | Use?                                                                                   |
| --------------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `monaco-editor` (barrel)    | All features **+ every basic language** (sql, abap, solidity, …)    | ❌ language bloat (~20 dead chunks)                                                    |
| `esm/vs/editor/editor.api`  | The API surface only — **zero feature contributions**               | ❌ a text box: no folding, suggest widget, `Cmd+Backspace`, find, bracket colorization |
| `esm/vs/editor/edcore.main` | `editor.all` (all 59 feature contributions) + API, **no languages** | ✅ full editor UX, JSON-only weight                                                    |

Import **`edcore.main`** and add only the JSON language service
(`esm/vs/language/json/monaco.contribution`). `edcore.main` ships no `.d.ts` of its own —
add an ambient `declare module … { export * from '…/editor.api'; }` so types (and Monaco's
global `MonacoEnvironment` augmentation) resolve. Two editor options worth setting because
they bite Vega-Lite specs specifically: `showFoldingControls: 'always'` (fold arrows always
visible), and `quickSuggestions: { strings: true }` (VL enum values like `"bar"` live inside
JSON strings, where Monaco disables auto-suggest by default).

> Reaching for `editor.api` to "drop unused languages" silently strips every editor feature —
> the languages live elsewhere. This is the concrete case behind AGENTS.md's **"trim content,
> not capability"** rule: cut the unwanted _content_, keep the _behavior_, and verify the
> behavior survived by exercising the editor, not by a green build.

---

> **The single biggest surprise:** vega/editor does **not** use `vega-embed` for its live
> preview. It builds the compile→parse→View pipeline by hand; `vega-embed` is imported only
> for types and the exported standalone-HTML snippet. This is _good news_ — `vega-embed` is
> exactly the wrapper they wrote by hand, so we get it for free. But their hand-rolled
> version (`src/components/renderer/renderer.tsx`) is the best available documentation of the
> lifecycle/cleanup discipline `vega-embed` still expects from us.

---

## 1 · Monaco + Vega-Lite schema wiring (M2 — highest from-scratch risk)

All of vega/editor's Monaco setup is one file: `src/utils/monaco.ts`.

**What to borrow:**

- **Bundle the schema; never fetch it.** They `import vegaLiteSchema from 'vega-lite/vega-lite-schema.json'`,
  resolved by a Vite alias to the package's `build/` output (`monaco.ts:7-8`, `vite.config.ts`).
  The schema version is pinned to the installed `vega-lite` — offline-safe, version-locked,
  no runtime network call. Astrolabe should do the same.
- **Attach via the JSON language service**, once, globally:
  `monaco.languages.json.jsonDefaults.setDiagnosticsOptions({ schemas, validate:true, ... })`
  (`monaco.ts:51-57`).
- **`markdownDescription` patch** (`monaco.ts:12-13`, `utils/markdownProps.ts`): recursively
  copy every schema `description` → `markdownDescription` before registering. Monaco renders
  rich hover docs only from `markdownDescription`; without this, hovers are plain text. Do it
  once at setup.
- **Replace the built-in JSON formatter** with `json-stringify-pretty-compact` via
  `registerDocumentFormattingEditProvider('json', …)` (`monaco.ts:60-61,71-80`) for Vega's
  compact array-on-one-line style.
- **Editor options worth copying** (`spec-editor/renderer.tsx:263-274`): `folding:true`,
  `minimap.enabled:false`, `scrollBeyondLastLine:false`, `wordWrap:'on'`,
  `quickSuggestions:true` (this is what makes schema completions appear without an explicit
  trigger), `stickyScroll.enabled:false`.

**Gotchas / where we improve:**

- ⚠️ **Workers are on us.** vega/editor never configures Monaco workers — the CDN loader does.
  With raw `monaco-editor` + Vite we **must** set `self.MonacoEnvironment.getWorker` to return
  the `json.worker` for label `'json'` and `editor.worker` otherwise (via `?worker` imports):
  ```ts
  import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
  import JsonWorker from 'monaco-editor/esm/vs/language/json/json.worker?worker';
  self.MonacoEnvironment = {
    getWorker: (_id, label) => (label === 'json' ? new JsonWorker() : new EditorWorker()),
  };
  ```
  The `json.worker` runs schema validation + autocomplete. **No worker ⇒ no squiggles, no
  completions.** Upside: dropping the CDN loader makes `monaco` synchronously importable — no
  `await loader.init()` dance, just call `setDiagnosticsOptions(...)` at module load.
- ⚠️ **`$schema`-based binding vs `fileMatch`.** They register schemas under versioned `uri`s
  (`.../vega-lite/v6.json`) and bind by matching the doc's `$schema` value — **no `fileMatch`**
  (`monaco.ts:15-46`). Consequence: a spec with **no `$schema` gets zero validation/autocomplete.**
  Astrolabe should prefer `fileMatch` against our model URIs so validation works regardless of
  whether the user wrote a `$schema` line.
- ⚠️ **Set `enableSchemaRequest:false`** for our offline-first app. They set it `true`
  (`monaco.ts:54`), which lets the worker network-fetch any unbundled `$schema` URL — failing
  network calls for an offline app. Register all schema versions locally instead.
- The schema is multi-MB; register it **once globally**, never per-model.

## 2 · Live preview with `vega-embed` (M1 lifecycle, M2 fit-mode)

This is doc [05](05-rendering-theming-preview.md)'s territory; these are the concrete details
vega/editor's hand-rolled renderer (`src/components/renderer/renderer.tsx`) reveals.

**What to borrow:**

- **Theme = a `vega-themes` config object merged into the spec config.** There is no automatic
  light/dark sync in vega/editor — theme is an explicit choice baked in at compile
  (`config-editor/config-editor-header.tsx:5-37`). For Astrolabe: pass the chosen `theme`/`config`
  to `vegaEmbed`, and when our theme changes, re-embed with the new config.
- **`"width":"container"` / `"height":"container"` is how VL responsiveness works** — it
  compiles `width`/`height` to signals that re-read `containerSize()` **only on a
  `window:resize` event** (`renderer.tsx:78-90` detects container sizing). Two things a
  from-scratch impl _will_ get wrong (we did): (1) `view.resize().runAsync()` does **not**
  re-measure — it re-runs layout with the stale size; (2) a pane drag fires no window resize, so
  nothing re-fits on its own. The fix is exactly vega/editor's
  `window.dispatchEvent(new Event('resize'))` (`renderer.tsx:101-122`) — **not a hack, the
  actual mechanism** — driven by a `ResizeObserver` on the pane. It also leaves a non-container
  dimension natural for free. Full write-up in doc [05](05-rendering-theming-preview.md) §8.
- **Reuse the view for cheap changes.** They rebuild the `View` only on spec change; renderer
  (svg/canvas) and tooltip toggles re-`initialize()` the existing view (`renderer.tsx:367-371`).
- **Capture warnings separately from errors** via a buffering logger (see §4's `LocalLogger`).

**Gotchas / where we improve:**

- ⚠️ **Finalize before re-embed, or leak.** Every spec change must `view.finalize()` the old
  view _and_ clear the container before mounting the new one (`renderer.tsx:218-226`). `vegaEmbed`
  returns `{ view, finalize }` — call `finalize()` before the next embed and on unmount. This is
  already a Do-rule in doc 05; vega/editor confirms how easy it is to leak otherwise.
- ⚠️ **Race on rapid edits.** `runAsync` is async; a stale render can resolve after a newer one
  mounts. vega/editor mitigates only with debounce. **We should add a render-generation token**
  and ignore stale resolves (an improvement over the reference).
- ⚠️ **Wrap `runAsync` in try/catch and finalize on failure** — Vega won't catch runtime errors
  for you, and a half-initialized view leaks if you don't finalize (`renderer.tsx:247-259`).
- The **CSP-safe expression interpreter** (`vega-interpreter` + `vega.parse(..., {ast:true})`)
  matters only under a strict no-`eval` CSP. A local offline app doesn't need it — keep it
  opt-in.

## 3 · Two-tier validation & error surfacing (M2, spec §03E)

vega/editor runs **two independent schema-validation systems** with no reconciliation, and
sorts errors into two tiers. Both are worth copying.

**The two layers:**

1. **Monaco JSON worker** → inline **squiggles, hovers, autocomplete** in the editor.
2. **`ajv ^8`** (`src/utils/validate.ts`) → runs at parse time, feeds the **error/log pane**.
   It does _not_ create editor markers.

**The two error tiers (keep them separate):**

- **Fatal / blocking** — thrown exceptions: JSON syntax error, VL compile error, Vega runtime
  error. These set a single `error` and suppress the chart.
- **Advisory** — ajv schema-validation findings and `$schema` version mismatch. These are a
  warnings list and do **not** block rendering. (Vega-Lite emits many benign warnings; treating
  ajv output as fatal would wrongly hide specs that render fine.)

The orchestration is `app.tsx:188-291`: `parseJSONCOrThrow` → `$schema` semver check (warn) →
`validateVegaLite` (ajv, warn) → `vegaLite.compile` (throw=fatal) → render (`renderer.tsx`,
throw=fatal).

**ajv setup specifics that _will_ bite a from-scratch impl** (`validate.ts:9-17`):

- `new Ajv({ strict: false })` — the VL/Vega schemas fail ajv strict-mode at **compile** time
  otherwise.
- The VL schema is **draft-06** → must `ajv.addMetaSchema(json-schema-draft-06.json)` (ajv 8
  defaults to draft-07/2020) or `compile` throws.
- Register a no-op `color-hex` format (`ajv.addFormat('color-hex', () => true)`) plus
  `addFormats(ajv)`; the schema references formats ajv-formats doesn't cover.
- **Compile the validator once at module load and cache it** — the schema is huge; compiling
  per keystroke is a perf killer.

**Where we improve:** ajv errors are shown as JSON-pointer text (e.g. `/encoding/x`) with **no
editor position** — vega/editor does not map them to markers. Only JSON _syntax_ errors get a
line/col (via jsonc-parser's visitor, `utils/jsonc-parser.ts:3-17`). If our §03E wants inline
ajv markers, we map `instancePath` → editor offsets ourselves via jsonc-parser's node tree —
something the reference does _not_ do.

## 4 · Data flow & debouncing (M1/M2 — translate to Zustand stores)

vega/editor keeps **`editorString` (the text) as the single source of truth**; the parsed spec
and compiled Vega spec are _derived_ and recomputed by a subscriber when text/mode/config change
(`app.tsx:338-365`). Errors don't clobber the last-good derived specs.

**The Zustand-store translation (this is the shape to build):**

```
text  (store field, debounced writer on editor change)
  └─▶ parsedSpec   (derived: JSONC parse + collect syntax/diagnostic errors)
        └─▶ renderInput (derived: prepareSpecForRender — refs, fit-mode)
              └─▶ effect: deep-equal guard → vegaEmbed(); finalize previous view
```

**What to borrow:**

- **Debounce only at edit→state**, not state→render. vega/editor debounces the editor at
  **1200 ms** (`spec-editor/renderer.tsx:66`) and guards the render with a `deepEqual` prop
  diff (`renderer.tsx:340-349`). (1200 ms is _their_ number; tune ours — our settings expose a
  render-debounce preference.)
- **A manual-parse escape hatch** (Ctrl/Cmd+S re-parses without waiting) maps to a future
  live-vs-manual preview toggle (`renderer.tsx:89-111`).
- **`LocalLogger` pattern** (`utils/logger.ts`): a logger that buffers `errors/warns/infos/debugs`
  into arrays instead of writing to console. This lets a **pure** `src/core` compile/validate
  step _return_ structured diagnostics with zero browser coupling — e.g.
  `validateSpec(spec) → { errors, warns }`. Ideal core-first fit.
- **`json-stringify-pretty-compact`** for the format action and prettify-on-load — much nicer
  than `JSON.stringify(…, null, 2)` for VL specs.

**Persistence note:** vega/editor snapshots its whole state to localStorage on _every_ change,
stripping non-serializable fields (`view`, `runtime`, editor refs) and restoring via
`{ ...DEFAULT_STATE, ...parsed }` (`context/app-context.tsx`). Our **debounced auto-save to
IndexedDB** (doc 01/02) is the better pattern — but the "strip non-serializable, restore with
defaults-spread" discipline is worth keeping.

---

## Borrow list (where each lands)

| Technique                                                                                 | Lands in                               | Milestone |
| ----------------------------------------------------------------------------------------- | -------------------------------------- | --------- |
| Bundle VL schema from package `build/`; `setDiagnosticsOptions`                           | `src/app/infrastructure/` Monaco setup | M2        |
| `markdownDescription` patch + compact formatter                                           | Monaco setup                           | M2        |
| Explicit Vite worker wiring (`MonacoEnvironment.getWorker`)                               | Monaco setup                           | M2        |
| `fileMatch` schema binding (improvement over `$schema`-only)                              | Monaco setup                           | M2        |
| jsonc-parser tolerant parse + line/col syntax errors                                      | `src/core/`                            | M1/M2     |
| ajv wrapper (`strict:false`, draft-06, color-hex, compile-once) → structured diagnostics  | `src/core/`                            | M2        |
| `LocalLogger`-style buffered diagnostics from pure compile                                | `src/core/`                            | M2        |
| Fatal-vs-advisory two-tier error model                                                    | rendering/store contract               | M1/M2     |
| `"container"` sizing + `ResizeObserver` → synthetic `window:resize` (not `view.resize()`) | `chart-renderer` + LivePreview         | M2        |
| `finalize()`-before-reembed + **render-generation guard**                                 | LivePreview                            | M1        |
| theme = `vega-themes` config merged into `vegaEmbed`                                      | preview + settings                     | M5        |
| `json-stringify-pretty-compact` format action                                             | editor                                 | M2        |

## Where we deliberately do better than the reference

- **Wire Monaco workers explicitly** (they sidestep it via the CDN loader).
- **Map ajv errors to editor positions** via jsonc-parser offsets (they show pointer text only).
- **Render-generation guard** against stale async renders (they rely on debounce alone).
- **`fileMatch`-based schema binding** so validation works without a `$schema` line.
- **Debounced auto-save to IndexedDB** rather than write-the-whole-state-on-every-change.

## Key files in the reference (for deeper reads)

- `src/utils/monaco.ts` — all Monaco/schema wiring
- `src/utils/markdownProps.ts` — the `markdownDescription` patch
- `src/utils/validate.ts` — ajv setup + cached validators
- `src/utils/jsonc-parser.ts` — tolerant parse + line/col syntax errors
- `src/utils/logger.ts` — `LocalLogger` / `DispatchingLogger`
- `src/components/renderer/renderer.tsx` — the hand-rolled View lifecycle (finalize, sizing, errors)
- `src/components/app.tsx:188-365` — parse → $schema check → ajv → compile → render orchestration
- `src/components/error-pane/renderer.tsx` — error/log display
- `src/constants/default-state.ts` — the full app-state shape
- `src/components/input-panel/spec-editor/renderer.tsx` — editor component, 1200ms debounce, $schema→mode detection