mirror of
https://github.com/olehomelchenko/astrolabe.git
synced 2026-08-08 02:02:33 +00:00
311 lines
21 KiB
Markdown
311 lines
21 KiB
Markdown
# 08 · Borrowed Techniques from vega/editor
|
||
|
||
> The official Vega-Lite editor ([vega/editor](https://github.com/vega/editor)) solves the
|
||
> exact "edit a Vega-Lite spec as JSON, validate it, render it live" problem Astrolabe sits
|
||
> on top of — minus the snippet/dataset library. This doc distills the techniques worth
|
||
> borrowing and the gotchas worth avoiding, so we don't rediscover them from scratch in
|
||
> M1/M2.
|
||
>
|
||
> It is a **reference**, not a contract. The behavioral contract is still [`docs/spec/`](../spec/);
|
||
> the patterns are still docs [01](01-state-and-stores.md)–[07](07-naming-and-relationships.md).
|
||
> This doc is the bridge: "here is how the canonical implementation does the editor/renderer
|
||
> plumbing, and here is what we keep vs. improve."
|
||
|
||
## Source of these findings
|
||
|
||
A read-only clone of vega/editor lives at `/Users/oleh/code/reference/vega-editor` (shallow
|
||
clone of `main`, HEAD `4fdbb59`). Re-clone with
|
||
`git clone --depth 1 https://github.com/vega/editor`. Citations below are `file:line` into
|
||
that tree.
|
||
|
||
## Stack delta (read this first — it changes how directly we can borrow)
|
||
|
||
| | vega/editor | Astrolabe |
|
||
| ----------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
|
||
| UI framework | **React** | **React** (moved off Preact before build start) |
|
||
| State | Redux-ish single `State` in React context (`useState`) | Zustand **stores** — _not_ Redux |
|
||
| Monaco | `@monaco-editor/react` + `@monaco-editor/loader` (CDN-loaded Monaco, **workers auto-wired**) | **raw `monaco-editor`** via Vite (**we must wire workers ourselves**) |
|
||
| Rendering | **hand-rolled** `vegaLite.compile` → `vega.parse` → `new vega.View().runAsync()` | **`vegaEmbed()`** (wraps that same pipeline) |
|
||
| Schema validation | Monaco JSON worker **+** standalone `ajv ^8` (two independent layers) | same two-layer model planned |
|
||
|
||
Because both apps are now React, vega/editor's **component lifecycle patterns port more or
|
||
less directly** — the friction is only in (a) state (their Redux-flat-state → our Zustand
|
||
stores) and (b) Monaco worker wiring (their CDN loader → our explicit Vite workers).
|
||
|
||
---
|
||
|
||
## Decision · Monaco integration (self-hosted, raw API)
|
||
|
||
> **Decided.** Astrolabe uses **raw `monaco-editor` from npm, bundled and self-hosted**, with
|
||
> workers wired explicitly via Vite `?worker` — **not** vega/editor's
|
||
> `@monaco-editor/react` + `@monaco-editor/loader` (CDN) setup. Two independent axes:
|
||
|
||
**Axis A — sourcing: self-hosted/bundled, not CDN. (Forced by Astrolabe's values.)**
|
||
vega/editor's `@monaco-editor/loader` fetches Monaco's AMD bundle from a CDN at runtime. For
|
||
us that breaks three things at once: (1) **offline** — the CDN bundle is outside Vite's module
|
||
graph, so `vite-plugin-pwa`/Workbox never precaches it and offline silently fails; bundled npm
|
||
assets are hashed files in `dist/` that Workbox precaches automatically; (2) **privacy** — a
|
||
third-party fetch on load contradicts SOUL's "the only outbound requests are user-created
|
||
URL-dataset fetches"; (3) **determinism** — npm + `package-lock` is integrity-pinned and
|
||
reproducible, a runtime CDN resolve is not. This axis is not a close call; vega/editor's CDN
|
||
choice is right _for an online hosted tool_ and wrong for an offline, installable, private app.
|
||
|
||
**Axis B — React integration: raw API, not `@monaco-editor/react`. (A lean, not forced.)**
|
||
The wrapper helps with the easy 80% (mount a JSON editor, lifecycle) and adds nothing to the
|
||
load-bearing 20% this app needs:
|
||
|
||
- **Workers** are still ours — the wrapper never manages `MonacoEnvironment` (see §1 gotcha).
|
||
- The **M2 schema service** (`jsonDefaults.setDiagnosticsOptions`, `fileMatch`) is namespace-level;
|
||
you reach _through_ the wrapper via `onMount`, so it saves nothing there.
|
||
- Its headline **`value`/`onChange` controlled-input model is a hazard**: driving Monaco's
|
||
content from React state causes cursor jumps and undo-stack churn, against §10's "typing
|
||
stays fluid" — you end up using it uncontrolled, i.e. the raw pattern anyway.
|
||
- Its **CDN-by-default** is a standing footgun (works in dev online, fails offline in prod
|
||
unless you remember `loader.config({ monaco })`).
|
||
|
||
Against that, raw costs **one testable `useMonacoEditor` hook** (~50–80 lines: create in
|
||
`useEffect`, `dispose` on unmount, push value, subscribe to `onDidChangeModelContent`, resize).
|
||
That's the **same imperative-teardown discipline already adopted for `vega-embed`** in doc 05
|
||
(`view.finalize()`), and consistent with already using raw `vegaEmbed()` over a React chart
|
||
wrapper — "thin integration layers we own" (SOUL). Lock-in is low either way, so the final
|
||
raw-vs-wrapper call is confirmable at the Monaco spike; what is **not** up for revisiting is
|
||
self-hosting.
|
||
|
||
**Accepted cost:** the explicit worker wiring (§1) is inherent to self-hosting — it is the
|
||
price of offline, paid in any non-CDN setup, and the wrapper would not remove it.
|
||
|
||
### Entry point: `edcore.main`, never `editor.api` (trim languages, not features)
|
||
|
||
Self-hosting raw Monaco forces a choice of ESM entry point, and the granularity matters:
|
||
|
||
| Import | What you get | Use? |
|
||
| --------------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
||
| `monaco-editor` (barrel) | All features **+ every basic language** (sql, abap, solidity, …) | ❌ language bloat (~20 dead chunks) |
|
||
| `esm/vs/editor/editor.api` | The API surface only — **zero feature contributions** | ❌ a text box: no folding, suggest widget, `Cmd+Backspace`, find, bracket colorization |
|
||
| `esm/vs/editor/edcore.main` | `editor.all` (all 59 feature contributions) + API, **no languages** | ✅ full editor UX, JSON-only weight |
|
||
|
||
Import **`edcore.main`** and add only the JSON language service
|
||
(`esm/vs/language/json/monaco.contribution`). `edcore.main` ships no `.d.ts` of its own —
|
||
add an ambient `declare module … { export * from '…/editor.api'; }` so types (and Monaco's
|
||
global `MonacoEnvironment` augmentation) resolve. Two editor options worth setting because
|
||
they bite Vega-Lite specs specifically: `showFoldingControls: 'always'` (fold arrows always
|
||
visible), and `quickSuggestions: { strings: true }` (VL enum values like `"bar"` live inside
|
||
JSON strings, where Monaco disables auto-suggest by default).
|
||
|
||
> Reaching for `editor.api` to "drop unused languages" silently strips every editor feature —
|
||
> the languages live elsewhere. This is the concrete case behind AGENTS.md's **"trim content,
|
||
> not capability"** rule: cut the unwanted _content_, keep the _behavior_, and verify the
|
||
> behavior survived by exercising the editor, not by a green build.
|
||
|
||
---
|
||
|
||
> **The single biggest surprise:** vega/editor does **not** use `vega-embed` for its live
|
||
> preview. It builds the compile→parse→View pipeline by hand; `vega-embed` is imported only
|
||
> for types and the exported standalone-HTML snippet. This is _good news_ — `vega-embed` is
|
||
> exactly the wrapper they wrote by hand, so we get it for free. But their hand-rolled
|
||
> version (`src/components/renderer/renderer.tsx`) is the best available documentation of the
|
||
> lifecycle/cleanup discipline `vega-embed` still expects from us.
|
||
|
||
---
|
||
|
||
## 1 · Monaco + Vega-Lite schema wiring (M2 — highest from-scratch risk)
|
||
|
||
All of vega/editor's Monaco setup is one file: `src/utils/monaco.ts`.
|
||
|
||
**What to borrow:**
|
||
|
||
- **Bundle the schema; never fetch it.** They `import vegaLiteSchema from 'vega-lite/vega-lite-schema.json'`,
|
||
resolved by a Vite alias to the package's `build/` output (`monaco.ts:7-8`, `vite.config.ts`).
|
||
The schema version is pinned to the installed `vega-lite` — offline-safe, version-locked,
|
||
no runtime network call. Astrolabe should do the same.
|
||
- **Attach via the JSON language service**, once, globally:
|
||
`monaco.languages.json.jsonDefaults.setDiagnosticsOptions({ schemas, validate:true, ... })`
|
||
(`monaco.ts:51-57`).
|
||
- **`markdownDescription` patch** (`monaco.ts:12-13`, `utils/markdownProps.ts`): recursively
|
||
copy every schema `description` → `markdownDescription` before registering. Monaco renders
|
||
rich hover docs only from `markdownDescription`; without this, hovers are plain text. Do it
|
||
once at setup.
|
||
- **Replace the built-in JSON formatter** with `json-stringify-pretty-compact` via
|
||
`registerDocumentFormattingEditProvider('json', …)` (`monaco.ts:60-61,71-80`) for Vega's
|
||
compact array-on-one-line style.
|
||
- **Editor options worth copying** (`spec-editor/renderer.tsx:263-274`): `folding:true`,
|
||
`minimap.enabled:false`, `scrollBeyondLastLine:false`, `wordWrap:'on'`,
|
||
`quickSuggestions:true` (this is what makes schema completions appear without an explicit
|
||
trigger), `stickyScroll.enabled:false`.
|
||
|
||
**Gotchas / where we improve:**
|
||
|
||
- ⚠️ **Workers are on us.** vega/editor never configures Monaco workers — the CDN loader does.
|
||
With raw `monaco-editor` + Vite we **must** set `self.MonacoEnvironment.getWorker` to return
|
||
the `json.worker` for label `'json'` and `editor.worker` otherwise (via `?worker` imports):
|
||
```ts
|
||
import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
|
||
import JsonWorker from 'monaco-editor/esm/vs/language/json/json.worker?worker';
|
||
self.MonacoEnvironment = {
|
||
getWorker: (_id, label) => (label === 'json' ? new JsonWorker() : new EditorWorker()),
|
||
};
|
||
```
|
||
The `json.worker` runs schema validation + autocomplete. **No worker ⇒ no squiggles, no
|
||
completions.** Upside: dropping the CDN loader makes `monaco` synchronously importable — no
|
||
`await loader.init()` dance, just call `setDiagnosticsOptions(...)` at module load.
|
||
- ⚠️ **`$schema`-based binding vs `fileMatch`.** They register schemas under versioned `uri`s
|
||
(`.../vega-lite/v6.json`) and bind by matching the doc's `$schema` value — **no `fileMatch`**
|
||
(`monaco.ts:15-46`). Consequence: a spec with **no `$schema` gets zero validation/autocomplete.**
|
||
Astrolabe should prefer `fileMatch` against our model URIs so validation works regardless of
|
||
whether the user wrote a `$schema` line.
|
||
- ⚠️ **Set `enableSchemaRequest:false`** for our offline-first app. They set it `true`
|
||
(`monaco.ts:54`), which lets the worker network-fetch any unbundled `$schema` URL — failing
|
||
network calls for an offline app. Register all schema versions locally instead.
|
||
- The schema is multi-MB; register it **once globally**, never per-model.
|
||
|
||
## 2 · Live preview with `vega-embed` (M1 lifecycle, M2 fit-mode)
|
||
|
||
This is doc [05](05-rendering-theming-preview.md)'s territory; these are the concrete details
|
||
vega/editor's hand-rolled renderer (`src/components/renderer/renderer.tsx`) reveals.
|
||
|
||
**What to borrow:**
|
||
|
||
- **Theme = a `vega-themes` config object merged into the spec config.** There is no automatic
|
||
light/dark sync in vega/editor — theme is an explicit choice baked in at compile
|
||
(`config-editor/config-editor-header.tsx:5-37`). For Astrolabe: pass the chosen `theme`/`config`
|
||
to `vegaEmbed`, and when our theme changes, re-embed with the new config.
|
||
- **`"width":"container"` / `"height":"container"` is how VL responsiveness works** — it
|
||
compiles to a `containerSize` signal (`renderer.tsx:78-90` detects this). Pair it with a
|
||
**`ResizeObserver`** on the preview pane → `view.resize().runAsync()`. This is cleaner than
|
||
vega/editor's `window.dispatchEvent(new Event('resize'))` hack (`renderer.tsx:101-122`) and is
|
||
the mechanism behind our M2 fit-mode contract.
|
||
- **Reuse the view for cheap changes.** They rebuild the `View` only on spec change; renderer
|
||
(svg/canvas) and tooltip toggles re-`initialize()` the existing view (`renderer.tsx:367-371`).
|
||
- **Capture warnings separately from errors** via a buffering logger (see §4's `LocalLogger`).
|
||
|
||
**Gotchas / where we improve:**
|
||
|
||
- ⚠️ **Finalize before re-embed, or leak.** Every spec change must `view.finalize()` the old
|
||
view _and_ clear the container before mounting the new one (`renderer.tsx:218-226`). `vegaEmbed`
|
||
returns `{ view, finalize }` — call `finalize()` before the next embed and on unmount. This is
|
||
already a Do-rule in doc 05; vega/editor confirms how easy it is to leak otherwise.
|
||
- ⚠️ **Race on rapid edits.** `runAsync` is async; a stale render can resolve after a newer one
|
||
mounts. vega/editor mitigates only with debounce. **We should add a render-generation token**
|
||
and ignore stale resolves (an improvement over the reference).
|
||
- ⚠️ **Wrap `runAsync` in try/catch and finalize on failure** — Vega won't catch runtime errors
|
||
for you, and a half-initialized view leaks if you don't finalize (`renderer.tsx:247-259`).
|
||
- The **CSP-safe expression interpreter** (`vega-interpreter` + `vega.parse(..., {ast:true})`)
|
||
matters only under a strict no-`eval` CSP. A local offline app doesn't need it — keep it
|
||
opt-in.
|
||
|
||
## 3 · Two-tier validation & error surfacing (M2, spec §03E)
|
||
|
||
vega/editor runs **two independent schema-validation systems** with no reconciliation, and
|
||
sorts errors into two tiers. Both are worth copying.
|
||
|
||
**The two layers:**
|
||
|
||
1. **Monaco JSON worker** → inline **squiggles, hovers, autocomplete** in the editor.
|
||
2. **`ajv ^8`** (`src/utils/validate.ts`) → runs at parse time, feeds the **error/log pane**.
|
||
It does _not_ create editor markers.
|
||
|
||
**The two error tiers (keep them separate):**
|
||
|
||
- **Fatal / blocking** — thrown exceptions: JSON syntax error, VL compile error, Vega runtime
|
||
error. These set a single `error` and suppress the chart.
|
||
- **Advisory** — ajv schema-validation findings and `$schema` version mismatch. These are a
|
||
warnings list and do **not** block rendering. (Vega-Lite emits many benign warnings; treating
|
||
ajv output as fatal would wrongly hide specs that render fine.)
|
||
|
||
The orchestration is `app.tsx:188-291`: `parseJSONCOrThrow` → `$schema` semver check (warn) →
|
||
`validateVegaLite` (ajv, warn) → `vegaLite.compile` (throw=fatal) → render (`renderer.tsx`,
|
||
throw=fatal).
|
||
|
||
**ajv setup specifics that _will_ bite a from-scratch impl** (`validate.ts:9-17`):
|
||
|
||
- `new Ajv({ strict: false })` — the VL/Vega schemas fail ajv strict-mode at **compile** time
|
||
otherwise.
|
||
- The VL schema is **draft-06** → must `ajv.addMetaSchema(json-schema-draft-06.json)` (ajv 8
|
||
defaults to draft-07/2020) or `compile` throws.
|
||
- Register a no-op `color-hex` format (`ajv.addFormat('color-hex', () => true)`) plus
|
||
`addFormats(ajv)`; the schema references formats ajv-formats doesn't cover.
|
||
- **Compile the validator once at module load and cache it** — the schema is huge; compiling
|
||
per keystroke is a perf killer.
|
||
|
||
**Where we improve:** ajv errors are shown as JSON-pointer text (e.g. `/encoding/x`) with **no
|
||
editor position** — vega/editor does not map them to markers. Only JSON _syntax_ errors get a
|
||
line/col (via jsonc-parser's visitor, `utils/jsonc-parser.ts:3-17`). If our §03E wants inline
|
||
ajv markers, we map `instancePath` → editor offsets ourselves via jsonc-parser's node tree —
|
||
something the reference does _not_ do.
|
||
|
||
## 4 · Data flow & debouncing (M1/M2 — translate to Zustand stores)
|
||
|
||
vega/editor keeps **`editorString` (the text) as the single source of truth**; the parsed spec
|
||
and compiled Vega spec are _derived_ and recomputed by a subscriber when text/mode/config change
|
||
(`app.tsx:338-365`). Errors don't clobber the last-good derived specs.
|
||
|
||
**The Zustand-store translation (this is the shape to build):**
|
||
|
||
```
|
||
text (store field, debounced writer on editor change)
|
||
└─▶ parsedSpec (derived: JSONC parse + collect syntax/diagnostic errors)
|
||
└─▶ renderInput (derived: prepareSpecForRender — refs, fit-mode)
|
||
└─▶ effect: deep-equal guard → vegaEmbed(); finalize previous view
|
||
```
|
||
|
||
**What to borrow:**
|
||
|
||
- **Debounce only at edit→state**, not state→render. vega/editor debounces the editor at
|
||
**1200 ms** (`spec-editor/renderer.tsx:66`) and guards the render with a `deepEqual` prop
|
||
diff (`renderer.tsx:340-349`). (1200 ms is _their_ number; tune ours — our settings expose a
|
||
render-debounce preference.)
|
||
- **A manual-parse escape hatch** (Ctrl/Cmd+S re-parses without waiting) maps to a future
|
||
live-vs-manual preview toggle (`renderer.tsx:89-111`).
|
||
- **`LocalLogger` pattern** (`utils/logger.ts`): a logger that buffers `errors/warns/infos/debugs`
|
||
into arrays instead of writing to console. This lets a **pure** `src/core` compile/validate
|
||
step _return_ structured diagnostics with zero browser coupling — e.g.
|
||
`validateSpec(spec) → { errors, warns }`. Ideal core-first fit.
|
||
- **`json-stringify-pretty-compact`** for the format action and prettify-on-load — much nicer
|
||
than `JSON.stringify(…, null, 2)` for VL specs.
|
||
|
||
**Persistence note:** vega/editor snapshots its whole state to localStorage on _every_ change,
|
||
stripping non-serializable fields (`view`, `runtime`, editor refs) and restoring via
|
||
`{ ...DEFAULT_STATE, ...parsed }` (`context/app-context.tsx`). Our **debounced auto-save to
|
||
IndexedDB** (doc 01/02) is the better pattern — but the "strip non-serializable, restore with
|
||
defaults-spread" discipline is worth keeping.
|
||
|
||
---
|
||
|
||
## Borrow list (where each lands)
|
||
|
||
| Technique | Lands in | Milestone |
|
||
| ---------------------------------------------------------------------------------------- | -------------------------------------- | --------- |
|
||
| Bundle VL schema from package `build/`; `setDiagnosticsOptions` | `src/app/infrastructure/` Monaco setup | M2 |
|
||
| `markdownDescription` patch + compact formatter | Monaco setup | M2 |
|
||
| Explicit Vite worker wiring (`MonacoEnvironment.getWorker`) | Monaco setup | M2 |
|
||
| `fileMatch` schema binding (improvement over `$schema`-only) | Monaco setup | M2 |
|
||
| jsonc-parser tolerant parse + line/col syntax errors | `src/core/` | M1/M2 |
|
||
| ajv wrapper (`strict:false`, draft-06, color-hex, compile-once) → structured diagnostics | `src/core/` | M2 |
|
||
| `LocalLogger`-style buffered diagnostics from pure compile | `src/core/` | M2 |
|
||
| Fatal-vs-advisory two-tier error model | rendering/store contract | M1/M2 |
|
||
| `"container"` sizing + `ResizeObserver` → `view.resize()` | `rendering.ts` + LivePreview | M2 |
|
||
| `finalize()`-before-reembed + **render-generation guard** | LivePreview | M1 |
|
||
| theme = `vega-themes` config merged into `vegaEmbed` | preview + settings | M5 |
|
||
| `json-stringify-pretty-compact` format action | editor | M2 |
|
||
|
||
## Where we deliberately do better than the reference
|
||
|
||
- **Wire Monaco workers explicitly** (they sidestep it via the CDN loader).
|
||
- **Map ajv errors to editor positions** via jsonc-parser offsets (they show pointer text only).
|
||
- **Render-generation guard** against stale async renders (they rely on debounce alone).
|
||
- **`fileMatch`-based schema binding** so validation works without a `$schema` line.
|
||
- **Debounced auto-save to IndexedDB** rather than write-the-whole-state-on-every-change.
|
||
|
||
## Key files in the reference (for deeper reads)
|
||
|
||
- `src/utils/monaco.ts` — all Monaco/schema wiring
|
||
- `src/utils/markdownProps.ts` — the `markdownDescription` patch
|
||
- `src/utils/validate.ts` — ajv setup + cached validators
|
||
- `src/utils/jsonc-parser.ts` — tolerant parse + line/col syntax errors
|
||
- `src/utils/logger.ts` — `LocalLogger` / `DispatchingLogger`
|
||
- `src/components/renderer/renderer.tsx` — the hand-rolled View lifecycle (finalize, sizing, errors)
|
||
- `src/components/app.tsx:188-365` — parse → $schema check → ajv → compile → render orchestration
|
||
- `src/components/error-pane/renderer.tsx` — error/log display
|
||
- `src/constants/default-state.ts` — the full app-state shape
|
||
- `src/components/input-panel/spec-editor/renderer.tsx` — editor component, 1200ms debounce, $schema→mode detection
|