mirror of
https://github.com/olehomelchenko/astrolabe.git
synced 2026-08-08 02:02:33 +00:00
Initial scaffold: spec, architecture playbook, and M0 skeleton
This commit is contained in:
@@ -0,0 +1,50 @@
|
||||
# Astrolabe — Architecture Playbook
|
||||
|
||||
> These documents capture the **architectural patterns** Astrolabe is built on. They are
|
||||
> self-contained: everything needed to implement a pattern lives here, in Astrolabe's own
|
||||
> domain terms (snippets, datasets, settings, Vega-Lite specs). You do not need any other
|
||||
> repository to work from them.
|
||||
>
|
||||
> They are the architectural counterpart to [`docs/spec/`](../spec/): the **spec** says
|
||||
> *what the app does* (behavior, acceptance points); this **playbook** says *how we build
|
||||
> it* (state, persistence, modals, routing, rendering, inference, relationships).
|
||||
|
||||
## How to use this playbook
|
||||
|
||||
- Building a feature? Read the relevant spec section first (the *what*), then the matching
|
||||
playbook doc (the *how*), then implement core-first per [`../IMPLEMENTATION-PLAN.md`](../IMPLEMENTATION-PLAN.md).
|
||||
- Each doc states the pattern, the **rationale** (what problem it solves, what it prevents),
|
||||
TypeScript sketches in Astrolabe terms, and Do/Don't rules.
|
||||
- The sketches are *illustrative*, not finished code. Adapt them; keep the principles.
|
||||
|
||||
## The documents
|
||||
|
||||
| # | Doc | Covers |
|
||||
|---|-----|--------|
|
||||
| 01 | [State & Stores](01-state-and-stores.md) | Zustand stores; one source of truth; selector derivations; central `useAppStore` vs per-feature stores; testable action functions; debounced auto-save. |
|
||||
| 02 | [Persistence](02-persistence.md) | The infrastructure-adapter boundary; promise-wrapped IndexedDB wrapper; lazy data loading; per-record schema versioning + migration; localStorage prefs with fallback; storage tiers + quota monitoring. |
|
||||
| 03 | [Modal System](03-modal-system.md) | Registry + coordinator + shell; one modal at a time; unsaved-change detection via snapshot; focus trap; backdrop/Escape/close dismissal. |
|
||||
| 04 | [Routing & Events](04-routing-and-events.md) | URL hash as view-state (restore/sync, Back/Forward); global keyboard routing; Escape priority chain; the single-source `isInInteractiveContext()` helper (Monaco-aware). |
|
||||
| 05 | [Rendering, Theming & Preview](05-rendering-theming-preview.md) | vega-embed integration (`actions:false`, `view.finalize()`); field-name escaping; theme→config mapping; debounced non-blocking renderer; resilient error display. |
|
||||
| 06 | [Type Inference & Profiling](06-type-inference.md) | Pure, portable column-type inference (number/text/date/boolean) and the dataset profile shape. |
|
||||
| 07 | [Naming & Relationships](07-naming-and-relationships.md) | Unique-name enforcement + import auto-suffix; the bidirectional snippet↔dataset name link; rename propagation into specs. |
|
||||
| 08 | [vega/editor Techniques](08-vega-editor-techniques.md) | Reference brief: borrowable Monaco-schema wiring, vega-embed lifecycle, two-tier validation, and data-flow/debounce techniques distilled from the official Vega-Lite editor — plus where we do better. |
|
||||
|
||||
## The non-negotiable layering (every doc assumes this)
|
||||
|
||||
- **`src/core/`** — portable, pure logic. No browser APIs, no React, no Monaco. Spec
|
||||
operations live here and are unit-tested hardest. (Docs 06, 07, parts of 05 land here.)
|
||||
- **`src/app/stores/`** — Zustand stores. (Doc 01.)
|
||||
- **`src/app/infrastructure/`** — the *only* place that touches `indexedDB`, `localStorage`,
|
||||
or `window.location`. Everything else goes through these typed adapters. (Docs 02, 04.)
|
||||
- **`src/app/services/` & `orchestration/`** — coordination that composes stores +
|
||||
infrastructure + core (lifecycle, routing sync, dependency upkeep). (Docs 03, 04, 07.)
|
||||
- **`src/app/components/`** — React + CSS Modules. Thin; pushes logic down into stores/core
|
||||
so it stays testable. (Docs 03, 05.)
|
||||
|
||||
## Why a playbook at all
|
||||
|
||||
Patterns written down once, in one place, stop two classes of problem: drift (the same
|
||||
decision re-litigated inconsistently across features) and rediscovery (re-deriving why
|
||||
something is the way it is). When a pattern here proves wrong, change the doc — don't fork
|
||||
the convention silently. This is the same discipline `docs/spec/` applies to behavior.
|
||||
@@ -0,0 +1,432 @@
|
||||
# State Management & Stores
|
||||
|
||||
How Astrolabe holds and shares application state. The whole app is built on
|
||||
**Zustand**: small, standalone stores created with `create()`, each exposing
|
||||
state fields and the actions that mutate them. Components subscribe to the exact
|
||||
slices they read; non-component code (services, infrastructure, orchestration)
|
||||
reads and writes the same stores directly. This document defines how we use
|
||||
Zustand, where state lives, and the rules that keep state predictable as the app
|
||||
grows.
|
||||
|
||||
Why Zustand: it is idiomatic React (just a hook), it has a first-class **outside-React**
|
||||
API (`getState`/`setState`/`subscribe`) that fits our "logic lives in core/services,
|
||||
not components" architecture, and it carries no build-time magic. The principles below
|
||||
(one source of truth, derive-don't-duplicate, actions outside components, thin components)
|
||||
are the durable part — they would survive a change of library.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Primitives
|
||||
|
||||
A store is a module that calls `create<State>()` once and exports the resulting
|
||||
hook. The state object holds both **data fields** and **action functions**.
|
||||
|
||||
```ts
|
||||
// src/app/stores/AppStore.ts
|
||||
import { create } from 'zustand';
|
||||
import type { UiTheme } from '@core/theme'; // defined in core; charts key off it too
|
||||
|
||||
export type ModalName = 'datasets' | 'settings' | 'about' | 'donate' | 'chartBuilder' | 'extract';
|
||||
|
||||
export interface AppState {
|
||||
uiTheme: UiTheme;
|
||||
activeModal: ModalName | null;
|
||||
setTheme: (theme: UiTheme) => void;
|
||||
// Low-level primitive. High-level open/close (snapshot, URL sync, discard
|
||||
// prompt) is the modal coordinator's job — see docs/architecture/03.
|
||||
setActiveModal: (modal: ModalName | null) => void;
|
||||
}
|
||||
|
||||
export const useAppStore = create<AppState>((set) => ({
|
||||
uiTheme: 'light',
|
||||
activeModal: null,
|
||||
setTheme: (uiTheme) => set({ uiTheme }),
|
||||
setActiveModal: (activeModal) => set({ activeModal }),
|
||||
}));
|
||||
```
|
||||
|
||||
Three ways to touch a store:
|
||||
|
||||
- **`set(partial)`** — update state (shallow-merges). Inside actions, the only place
|
||||
that mutates state.
|
||||
- **`get()`** — read current state inside actions without subscribing.
|
||||
- **the hook `useAppStore(selector)`** — read state *in a React component*, subscribing
|
||||
to exactly what the selector returns.
|
||||
|
||||
### Reading in components — always select narrowly
|
||||
|
||||
Call the hook with a **selector** that returns the smallest thing you need. The
|
||||
component re-renders only when that selected value changes (default `Object.is`
|
||||
comparison).
|
||||
|
||||
```tsx
|
||||
import { useAppStore } from '../stores/AppStore';
|
||||
|
||||
export function ThemeBadge() {
|
||||
const theme = useAppStore((s) => s.uiTheme); // re-renders only when uiTheme changes
|
||||
return <span>{theme}</span>;
|
||||
}
|
||||
```
|
||||
|
||||
When you select **multiple fields or a fresh object/array**, wrap the selector in
|
||||
`useShallow` so a new-but-equal result doesn't cause an extra render:
|
||||
|
||||
```tsx
|
||||
import { useShallow } from 'zustand/react/shallow';
|
||||
|
||||
const { activeModal, uiTheme } = useAppStore(
|
||||
useShallow((s) => ({ activeModal: s.activeModal, uiTheme: s.uiTheme })),
|
||||
);
|
||||
```
|
||||
|
||||
### Reading/writing outside components
|
||||
|
||||
Services, orchestration, infrastructure, and tests use the store object directly —
|
||||
no React involved. This is the property that lets our logic live outside components:
|
||||
|
||||
```ts
|
||||
openModal('settings'); // via the modal coordinator (doc 03)
|
||||
const theme = useAppStore.getState().uiTheme; // snapshot read
|
||||
const unsub = useAppStore.subscribe((s, prev) => { /* react to changes */ });
|
||||
```
|
||||
|
||||
> Rule: in components, **select narrowly** (and `useShallow` for object/array
|
||||
> selections). Outside components, use `getState()` for a snapshot, `subscribe()`
|
||||
> to react.
|
||||
|
||||
---
|
||||
|
||||
## 2. One Source of Truth per Fact — Derive, Don't Duplicate
|
||||
|
||||
Every fact lives in exactly one state field. Anything that can be *calculated*
|
||||
from other state is computed **in a selector at read time**, never stored as a
|
||||
second field you keep in sync by hand.
|
||||
|
||||
The failure mode this avoids: two fields that must agree (`snippets` and
|
||||
`snippetCount`, or `activeSnippetId` and `activeSnippet`) drift apart because one
|
||||
update path forgets the other. If the derived value is computed from the source on
|
||||
read, drift is structurally impossible.
|
||||
|
||||
```ts
|
||||
// State holds only the sources:
|
||||
// snippets: Snippet[]
|
||||
// activeSnippetId: string | null
|
||||
|
||||
// Derive in the component's selector — not a stored field:
|
||||
const activeSnippet = useSnippetStore((s) =>
|
||||
s.snippets.find((x) => x.id === s.activeSnippetId) ?? null,
|
||||
);
|
||||
const snippetCount = useSnippetStore((s) => s.snippets.length);
|
||||
```
|
||||
|
||||
For a derivation that is **expensive** or reused in many places, expose it as a
|
||||
selector function (memoize if profiling shows it matters) rather than caching it
|
||||
into state:
|
||||
|
||||
```ts
|
||||
// src/app/stores/snippet-selectors.ts
|
||||
export const selectActiveSnippet = (s: SnippetState) =>
|
||||
s.snippets.find((x) => x.id === s.activeSnippetId) ?? null;
|
||||
|
||||
// in a component:
|
||||
const active = useSnippetStore(selectActiveSnippet);
|
||||
```
|
||||
|
||||
> Rule: if you can compute it, do not store it. Add a new state field only for a
|
||||
> value that is *input* the app receives, not output it derives.
|
||||
|
||||
---
|
||||
|
||||
## 3. Where State Lives: Central vs. Per-Feature Stores
|
||||
|
||||
Each store is its own `create()` module. We split by *concern*, not by component
|
||||
tree.
|
||||
|
||||
### Per-feature stores
|
||||
|
||||
Each cohesive feature owns a store holding its durable domain state.
|
||||
|
||||
- **`useSnippetStore`** — the snippet library: `snippets`, `activeSnippetId`, the
|
||||
working `draftSpec`, and its actions.
|
||||
- **`useDatasetStore`** — loaded datasets, the active dataset, inferred fields.
|
||||
- **`useSettingsStore`** — user preferences (editor options, render debounce,
|
||||
date format, theme); mirrors what gets persisted to `localStorage`.
|
||||
|
||||
### The central `useAppStore`
|
||||
|
||||
`useAppStore` holds only *cross-cutting, ephemeral UI state* that no single
|
||||
feature owns — which modal is open, the runtime theme, transient render flags.
|
||||
|
||||
### How to decide
|
||||
|
||||
| Put it in a **feature store** when… | Put it in **`useAppStore`** when… |
|
||||
| --------------------------------------------- | --------------------------------------------- |
|
||||
| It's domain data (snippets, datasets, specs) | It's transient UI chrome (open modal, theme) |
|
||||
| It outlives a single interaction | It belongs to no single feature |
|
||||
| It gets persisted | Multiple unrelated features read/write it |
|
||||
|
||||
> Rule: keep `useAppStore` small. When a chunk of it only ever serves one feature,
|
||||
> that's the signal to extract a feature store. A bloated central store is the
|
||||
> thing this split exists to prevent.
|
||||
|
||||
---
|
||||
|
||||
## 4. Actions: Mutations Live in the Store, Not Components
|
||||
|
||||
Components **render** and **dispatch**; they do not contain mutation logic. Every
|
||||
state change goes through a named action defined on the store (via `set`/`get`).
|
||||
Multi-step logic that coordinates several stores or touches infrastructure can
|
||||
live in a `src/app/services/*` module that calls store actions.
|
||||
|
||||
```ts
|
||||
// src/app/stores/SnippetStore.ts
|
||||
import { create } from 'zustand';
|
||||
import type { Snippet } from '@core/snippet';
|
||||
|
||||
interface SnippetState {
|
||||
snippets: Snippet[];
|
||||
activeSnippetId: string | null;
|
||||
draftSpec: string; // Monaco editor buffer (Vega-Lite JSON)
|
||||
|
||||
create: (name: string) => string;
|
||||
select: (id: string) => void;
|
||||
remove: (id: string) => void;
|
||||
updateDraft: (spec: string) => void;
|
||||
reset: () => void;
|
||||
}
|
||||
|
||||
export const useSnippetStore = create<SnippetState>((set, get) => ({
|
||||
snippets: [],
|
||||
activeSnippetId: null,
|
||||
draftSpec: '',
|
||||
|
||||
create: (name) => {
|
||||
const snippet: Snippet = { id: crypto.randomUUID(), name, spec: '{}' };
|
||||
set((s) => ({ snippets: [...s.snippets, snippet] }));
|
||||
get().select(snippet.id);
|
||||
return snippet.id;
|
||||
},
|
||||
|
||||
select: (id) =>
|
||||
set((s) => ({
|
||||
activeSnippetId: id,
|
||||
draftSpec: s.snippets.find((x) => x.id === id)?.spec ?? '{}',
|
||||
})),
|
||||
|
||||
remove: (id) =>
|
||||
set((s) => {
|
||||
const snippets = s.snippets.filter((x) => x.id !== id);
|
||||
const activeSnippetId =
|
||||
s.activeSnippetId === id ? (snippets[0]?.id ?? null) : s.activeSnippetId;
|
||||
return { snippets, activeSnippetId };
|
||||
}),
|
||||
|
||||
updateDraft: (draftSpec) => set({ draftSpec }),
|
||||
|
||||
reset: () => set({ snippets: [], activeSnippetId: null, draftSpec: '' }),
|
||||
}));
|
||||
```
|
||||
|
||||
The component is thin — it selects state and calls actions:
|
||||
|
||||
```tsx
|
||||
import { useShallow } from 'zustand/react/shallow';
|
||||
import { useSnippetStore } from '../stores/SnippetStore';
|
||||
|
||||
export function SnippetList() {
|
||||
const { snippets, activeSnippetId } = useSnippetStore(
|
||||
useShallow((s) => ({ snippets: s.snippets, activeSnippetId: s.activeSnippetId })),
|
||||
);
|
||||
const select = useSnippetStore((s) => s.select);
|
||||
const remove = useSnippetStore((s) => s.remove);
|
||||
|
||||
return (
|
||||
<ul>
|
||||
{snippets.map((s) => (
|
||||
<li key={s.id} aria-current={s.id === activeSnippetId} onClick={() => select(s.id)}>
|
||||
{s.name}
|
||||
<button onClick={(e) => { e.stopPropagation(); remove(s.id); }}>✕</button>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
> Note: action identities are stable, so selecting them (`s.select`) never causes
|
||||
> re-renders — select actions individually rather than bundling them into a
|
||||
> `useShallow` object.
|
||||
|
||||
### Why mutations live in the store
|
||||
|
||||
- **Testable without a DOM.** Actions are plain functions over state. A Vitest test
|
||||
calls `useStore.getState().create('x')` and asserts on `getState()` — no
|
||||
rendering, no React.
|
||||
- **One place to change behavior.** "Deleting the active snippet falls back to the
|
||||
first remaining one" is a rule that lives in `remove`, not scattered across every
|
||||
delete button.
|
||||
- **Readable components.** A component that only wires events to named actions reads
|
||||
like a description of the UI, not a tangle of state juggling.
|
||||
|
||||
```ts
|
||||
// SnippetStore.test.ts — no browser needed
|
||||
import { useSnippetStore } from './SnippetStore';
|
||||
|
||||
beforeEach(() => useSnippetStore.getState().reset());
|
||||
|
||||
test('deleting the active snippet selects the next one', () => {
|
||||
const store = useSnippetStore.getState();
|
||||
const a = store.create('A');
|
||||
const b = store.create('B');
|
||||
store.select(a);
|
||||
store.remove(a);
|
||||
expect(useSnippetStore.getState().activeSnippetId).toBe(b);
|
||||
});
|
||||
```
|
||||
|
||||
> Rule: no `setState` calls inside component bodies for shared state — call an
|
||||
> action. Local, throwaway UI state (a dropdown's open flag) may stay in component
|
||||
> `useState`; anything another component reads belongs in a store behind an action.
|
||||
|
||||
---
|
||||
|
||||
## 5. Effects: Persistence and External Sync
|
||||
|
||||
Cross-cutting reactions — persisting state, mirroring the theme onto the document,
|
||||
pushing the draft into Vega for rendering — are wired once at app startup with
|
||||
`store.subscribe(...)`, in the orchestration/startup layer, not in components.
|
||||
Subscribers read state and write to `src/app/infrastructure/` adapters (IndexedDB,
|
||||
`localStorage`, URL hash).
|
||||
|
||||
### Theme → document (the minimal example, already wired)
|
||||
|
||||
```ts
|
||||
// src/main.tsx
|
||||
const applyTheme = (t: string) => { document.documentElement.dataset.theme = t; };
|
||||
applyTheme(useAppStore.getState().uiTheme);
|
||||
useAppStore.subscribe((s, prev) => {
|
||||
if (s.uiTheme !== prev.uiTheme) applyTheme(s.uiTheme);
|
||||
});
|
||||
```
|
||||
|
||||
The store stays DOM-free; the adapter (the `applyTheme` subscriber) lives at the edge.
|
||||
|
||||
### Debounced auto-save of the draft spec
|
||||
|
||||
The Monaco editor writes every keystroke into `draftSpec`. We do **not** persist on
|
||||
every keystroke. A startup subscriber observes the draft and debounces the expensive
|
||||
work:
|
||||
|
||||
```ts
|
||||
// src/app/orchestration/persistence.ts
|
||||
import { useSnippetStore } from '../stores/SnippetStore';
|
||||
import { saveSnippet } from '../infrastructure/snippet-store'; // IndexedDB adapter
|
||||
|
||||
export function wireDraftAutoSave(): void {
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
|
||||
useSnippetStore.subscribe((s, prev) => {
|
||||
if (s.draftSpec === prev.draftSpec) return; // only react to draft edits
|
||||
const id = s.activeSnippetId;
|
||||
if (!id) return;
|
||||
|
||||
clearTimeout(timer);
|
||||
const spec = s.draftSpec;
|
||||
timer = setTimeout(() => {
|
||||
useSnippetStore.setState((cur) => ({
|
||||
snippets: cur.snippets.map((x) => (x.id === id ? { ...x, spec } : x)),
|
||||
}));
|
||||
void saveSnippet(id, spec);
|
||||
}, 400);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
> For selector-based subscriptions (`subscribe(selector, listener)` with an equality
|
||||
> function) add the `subscribeWithSelector` middleware to the store. Plain
|
||||
> `subscribe((state, prev) => …)` as above is enough for most wiring.
|
||||
|
||||
> Rule: components never touch infrastructure adapters directly. Reads/writes to
|
||||
> IndexedDB, `localStorage`, and the URL hash happen in startup subscribers or
|
||||
> actions, so the persistence story is in one place and the UI stays pure.
|
||||
|
||||
---
|
||||
|
||||
## 6. Reading State: Import the Store, Don't Thread It
|
||||
|
||||
Because stores are singletons importable anywhere, a deep leaf component reads the
|
||||
state it needs directly instead of receiving it through five layers of props.
|
||||
|
||||
```tsx
|
||||
// Good: a deeply nested toggle reads + flips the theme itself.
|
||||
import { useAppStore } from '../stores/AppStore';
|
||||
|
||||
export function ThemeToggle() {
|
||||
const theme = useAppStore((s) => s.uiTheme);
|
||||
const setTheme = useAppStore((s) => s.setTheme);
|
||||
return (
|
||||
<button onClick={() => setTheme(theme === 'experimental' ? 'light' : 'experimental')}>
|
||||
{theme === 'experimental' ? '🌙' : '☀️'}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
This is the right default for **global/shared** state. Threading `theme` and
|
||||
`onThemeChange` through `Layout → Header → Toolbar → ThemeToggle` adds noise and
|
||||
couples every intermediate component to data it doesn't use.
|
||||
|
||||
### When to thread props instead
|
||||
|
||||
- The value is **presentational input**, not shared app state. `<Button variant="primary">`
|
||||
takes `variant` as a prop; it should not know about any store.
|
||||
- The component is meant to be **reusable / store-agnostic** (design-system
|
||||
components, list-item renderers given their item via prop).
|
||||
- A parent supplies **per-instance** data, e.g. `<SnippetRow snippet={s} />` inside a
|
||||
`.map()` — the row gets its snippet by prop but still calls
|
||||
`useSnippetStore.getState().remove(...)` (or a selected action) for mutations.
|
||||
|
||||
> Rule of thumb: shared app state → select it from the store at the point of use.
|
||||
> Per-instance or presentational data → pass it as a prop. Passing global state down
|
||||
> as props is the anti-pattern to avoid.
|
||||
|
||||
---
|
||||
|
||||
## 7. Resetting State
|
||||
|
||||
Each store exposes a `reset()` action that returns its fields to initial values
|
||||
(used on "new workspace", sign-out, or test teardown). Because every fact is a
|
||||
single source field with no hand-maintained duplicates, reset is a flat `set(...)`
|
||||
of the initial values; selector-derived values recompute on their own.
|
||||
|
||||
```ts
|
||||
reset: () => set({ snippets: [], activeSnippetId: null, draftSpec: '' });
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rules Summary
|
||||
|
||||
**Do**
|
||||
|
||||
- Keep one state field per fact; derive everything else in selectors, not stored fields.
|
||||
- In components, **select narrowly**; use `useShallow` for object/array selections.
|
||||
Outside components, use `getState()` / `subscribe()`.
|
||||
- Split durable domain state into feature stores (`useSnippetStore`, `useDatasetStore`,
|
||||
`useSettingsStore`); keep `useAppStore` for thin cross-cutting UI state.
|
||||
- Put every shared-state mutation behind a named action on the store so it's testable
|
||||
without a DOM (`getState().action()`).
|
||||
- Do persistence and external sync (IndexedDB, `localStorage`, URL hash, theme) in
|
||||
startup `subscribe` listeners via `infrastructure/` adapters.
|
||||
- Debounce expensive reactions (auto-save, re-render) inside the subscriber.
|
||||
- Import singleton store hooks directly in the leaves that need shared state.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't store derived values as their own fields and sync them by hand.
|
||||
- Don't call `setState` for shared state inside component render bodies — call an action.
|
||||
- Don't select broad objects without `useShallow` (causes needless re-renders).
|
||||
- Don't let `useAppStore` accumulate feature-specific state; extract a store.
|
||||
- Don't touch IndexedDB/`localStorage`/URL adapters from components.
|
||||
- Don't thread global state down through props; don't pass per-instance or
|
||||
presentational data via store imports.
|
||||
@@ -0,0 +1,401 @@
|
||||
# 02 · Persistence Architecture
|
||||
|
||||
How Astrolabe stores data in the browser, and the rules that keep that storage testable, portable, and safe to evolve. This document is the implementation contract for the persistence layer. For the *behavioral* data model (what fields a Snippet or Dataset has, what the tiers hold), see [09 · Data Model & Persistence](../spec/09-data-model.md); this document covers *how the code is structured to implement it*.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Infrastructure-Adapter Principle
|
||||
|
||||
**Rule: nothing outside `src/app/infrastructure/` ever touches `indexedDB`, `localStorage`, `window`, or `location` directly.** Every browser-storage interaction goes through a typed adapter module that exposes plain async functions returning domain objects.
|
||||
|
||||
```
|
||||
src/
|
||||
├── core/ # portable engine — NO browser APIs, NO React
|
||||
├── app/
|
||||
│ ├── stores/ # Zustand stores; calls infrastructure, never IDB
|
||||
│ ├── services/ # business logic; calls infrastructure, never IDB
|
||||
│ └── infrastructure/ # the ONLY place that imports indexedDB/localStorage
|
||||
│ ├── snippet-store.ts # IndexedDB: snippets (metadata + drafts)
|
||||
│ ├── dataset-store.ts # IndexedDB: datasets (heavy payloads)
|
||||
│ ├── settings-store.ts # localStorage: UserSettings
|
||||
│ └── prefs-store.ts # localStorage: app/UI prefs (sort, layout)
|
||||
```
|
||||
|
||||
### Why this boundary exists
|
||||
|
||||
- **Testability.** Stores and services depend on a small typed surface (`getSnippet(id): Promise<Snippet | null>`), not on the IndexedDB request API. Tests mock the adapter, not a browser global. The adapters themselves are tested directly against `fake-indexeddb` / a localStorage stub in Vitest.
|
||||
- **Portability.** `src/core/` stays free of browser APIs so the spec/parse/transform logic can run in Node (tests, future CLI, SSR). The adapters are the seam where the portable core meets the browser.
|
||||
- **Single place for migrations.** Schema upgrades and record migrations live in exactly one module per store. A reader looking for "how does v1 data become v2 data" has one file to open, not a scattered set of `if (record.someOldField)` checks across the UI.
|
||||
- **Failure containment.** Quota errors, corrupt JSON, and missing keys are handled at the boundary and converted into typed results (or sane fallbacks), so the rest of the app never sees a raw `DOMException`.
|
||||
|
||||
> **Do:** `import { saveSnippet } from '@/app/infrastructure/snippet-store'`
|
||||
> **Don't:** `indexedDB.open(...)` or `localStorage.getItem(...)` anywhere in a component, store, or service.
|
||||
|
||||
---
|
||||
|
||||
## 2. IndexedDB Wrapper
|
||||
|
||||
IndexedDB's native API is event-based (`onsuccess`/`onerror`) and verbose. The adapter wraps it into promises and exposes a tiny CRUD surface per object store. Define one shared helper and build typed stores on top of it.
|
||||
|
||||
### 2.1 Opening the database
|
||||
|
||||
Open with an explicit **version number** and an `onupgradeneeded` handler that creates/upgrades object stores. The version is a monotonically increasing integer; bump it whenever the *store layout* changes (a new object store, a new index). It is independent of per-record schema versions (§4).
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/db.ts
|
||||
const DB_NAME = 'astrolabe';
|
||||
const DB_VERSION = 1;
|
||||
|
||||
let dbPromise: Promise<IDBDatabase> | null = null;
|
||||
|
||||
export function openDB(): Promise<IDBDatabase> {
|
||||
// Memoize: opening is idempotent and cheap to share across calls.
|
||||
if (dbPromise) return dbPromise;
|
||||
|
||||
dbPromise = new Promise((resolve, reject) => {
|
||||
const req = indexedDB.open(DB_NAME, DB_VERSION);
|
||||
|
||||
req.onupgradeneeded = (event) => {
|
||||
const db = req.result;
|
||||
const oldVersion = event.oldVersion;
|
||||
|
||||
// Create stores idempotently — guard every create.
|
||||
if (!db.objectStoreNames.contains('snippets')) {
|
||||
db.createObjectStore('snippets', { keyPath: 'id' });
|
||||
}
|
||||
if (!db.objectStoreNames.contains('datasets')) {
|
||||
db.createObjectStore('datasets', { keyPath: 'id' });
|
||||
}
|
||||
|
||||
// Per-version store-layout migrations go here, gated on oldVersion.
|
||||
// if (oldVersion < 2) { /* add index, split a store, ... */ }
|
||||
void oldVersion;
|
||||
};
|
||||
|
||||
req.onsuccess = () => resolve(req.result);
|
||||
req.onerror = () => reject(req.error ?? new Error('Failed to open IndexedDB'));
|
||||
});
|
||||
|
||||
return dbPromise;
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Promise-wrapped CRUD helpers
|
||||
|
||||
Wrap a single IDB request and a whole transaction so callers write linear `async/await` code.
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/db.ts (continued)
|
||||
function wrap<T>(req: IDBRequest<T>): Promise<T> {
|
||||
return new Promise((resolve, reject) => {
|
||||
req.onsuccess = () => resolve(req.result);
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
}
|
||||
|
||||
async function tx<T>(
|
||||
store: string,
|
||||
mode: IDBTransactionMode,
|
||||
run: (s: IDBObjectStore) => IDBRequest<T>
|
||||
): Promise<T> {
|
||||
const db = await openDB();
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const transaction = db.transaction(store, mode);
|
||||
const request = run(transaction.objectStore(store));
|
||||
transaction.oncomplete = () => resolve(request.result);
|
||||
transaction.onerror = () => reject(transaction.error);
|
||||
transaction.onabort = () => reject(transaction.error);
|
||||
});
|
||||
}
|
||||
|
||||
export const get = <T>(store: string, key: IDBValidKey) =>
|
||||
tx<T | undefined>(store, 'readonly', (s) => s.get(key) as IDBRequest<T | undefined>);
|
||||
|
||||
export const getAll = <T>(store: string) =>
|
||||
tx<T[]>(store, 'readonly', (s) => s.getAll() as IDBRequest<T[]>);
|
||||
|
||||
export const put = <T>(store: string, value: T) =>
|
||||
tx<IDBValidKey>(store, 'readwrite', (s) => s.put(value as any));
|
||||
|
||||
export const del = (store: string, key: IDBValidKey) =>
|
||||
tx<undefined>(store, 'readwrite', (s) => s.delete(key) as IDBRequest<undefined>);
|
||||
```
|
||||
|
||||
> **Do:** resolve on `transaction.oncomplete`, not on the request's `onsuccess` — the write is only durable once the transaction commits.
|
||||
> **Don't:** hold an IndexedDB transaction open across an `await` to non-IDB work; transactions auto-close when the microtask queue drains and you'll get `TransactionInactiveError`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Lazy Loading: Metadata vs Heavy Payloads
|
||||
|
||||
A snippet library can grow large, and **datasets can be megabytes each** (CSV text, parsed TopoJSON). Loading every dataset payload at startup just to render a list of names is wasteful and slow. The rule:
|
||||
|
||||
> **Store record metadata separately from large payloads. Load heavy data on demand. Treat `null` as "exists but not loaded yet" — distinct from absent.**
|
||||
|
||||
For Astrolabe this maps cleanly onto the two stores:
|
||||
|
||||
- **`snippets`** — snippet records are small (a spec is JSON text). They load eagerly as a set when the library opens.
|
||||
- **`datasets`** — the `data` payload is the heavy part. The list view needs only the derived summary fields (`name`, `format`, `source`, `rowCount`, `columnCount`, `columns`, `size`, timestamps). Load `data` only when a snippet that references the dataset is actually previewed.
|
||||
|
||||
There are two ways to implement the split; pick per store:
|
||||
|
||||
1. **Two object stores** (`datasets` for metadata, `dataset-payloads` keyed by the same id for `data`) — strongest separation; a `getAll` on metadata never touches payload bytes.
|
||||
2. **One store, lazy field** — keep `data` in the record but set it to `null` on the bulk list load and fetch it per-id on demand.
|
||||
|
||||
Astrolabe uses the **lazy-field** approach for datasets (one store, simpler), with `data === null` signalling "summary loaded, payload not yet."
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/dataset-store.ts
|
||||
import { get, getAll, put } from './db';
|
||||
import { migrateDataset, type Dataset } from './dataset-migrations';
|
||||
|
||||
/** List view: returns every dataset's summary, payload nulled out. */
|
||||
export async function loadDatasetSummaries(): Promise<Dataset[]> {
|
||||
const records = await getAll<Dataset>('datasets');
|
||||
return records.map((r) => ({ ...migrateDataset(r), data: null }));
|
||||
}
|
||||
|
||||
/** Detail/preview: load (or return cached) full payload for one dataset. */
|
||||
export async function ensureDatasetData(dataset: Dataset): Promise<Dataset['data']> {
|
||||
if (dataset.data !== null && dataset.data !== undefined) return dataset.data; // already loaded
|
||||
const record = await get<Dataset>('datasets', dataset.id);
|
||||
dataset.data = record?.data ?? null;
|
||||
return dataset.data;
|
||||
}
|
||||
```
|
||||
|
||||
> **Do:** use `null` for "not loaded" and a real value (including `''` or `[]`) for "loaded but empty." The distinction prevents a re-fetch loop.
|
||||
> **Don't:** overwrite a stored payload with `null` on save. When persisting a record whose `data` is `null` (never loaded into memory), skip writing the payload field and leave the stored bytes intact — otherwise a list-load-then-save round-trip silently destroys data.
|
||||
|
||||
---
|
||||
|
||||
## 4. Per-Record Schema Versioning & Read-Time Migration
|
||||
|
||||
The IndexedDB **database version** (§2.1) governs *store layout*. A separate **per-record `version` field** governs the *shape of an individual record*. Both Snippet and Dataset records carry `version` (and `created` / `modified` timestamps). This lets record shapes evolve without forcing an `onupgradeneeded` database bump for every field rename.
|
||||
|
||||
Migrations are applied **on read** — when a record comes out of the store, run it through a migration function that upgrades it to the current shape before the app sees it. New writes always store the current version.
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/snippet-migrations.ts
|
||||
export const CURRENT_SNIPPET_VERSION = 2;
|
||||
|
||||
export function migrateSnippet(raw: any): Snippet {
|
||||
let r = { ...raw };
|
||||
const v = r.version ?? 1; // records written before versioning existed are v1
|
||||
|
||||
if (v < 2) {
|
||||
// Example: a v1 snippet had a single `spec`; v2 splits draft from published.
|
||||
r.draftSpec = r.draftSpec ?? r.spec;
|
||||
r.tags = r.tags ?? [];
|
||||
r.datasetRefs = r.datasetRefs ?? [];
|
||||
}
|
||||
// if (v < 3) { ... }
|
||||
|
||||
r.version = CURRENT_SNIPPET_VERSION;
|
||||
return r as Snippet;
|
||||
}
|
||||
```
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/snippet-store.ts
|
||||
export async function loadSnippets(): Promise<Snippet[]> {
|
||||
const records = await getAll<any>('snippets');
|
||||
return records.map(migrateSnippet); // upgrade every record at the boundary
|
||||
}
|
||||
|
||||
export async function saveSnippet(s: Snippet): Promise<void> {
|
||||
await put('snippets', { ...s, version: CURRENT_SNIPPET_VERSION, modified: new Date().toISOString() });
|
||||
}
|
||||
```
|
||||
|
||||
### Rationale
|
||||
|
||||
- **Read-time migration is forgiving.** Old records sitting untouched in the store keep working; they upgrade lazily the next time they're loaded and re-saved. There is no big-bang migration step that can fail halfway.
|
||||
- **Tolerate unknown fields.** A migration normalizes *missing/old* fields but must not strip fields it doesn't recognize — a record written by a *newer* build that downgraded must round-trip without data loss. Spread the original (`{ ...raw }`) and only fill in what's missing.
|
||||
- **One function, well tested.** Each migration step is a pure function over a plain object — trivial to unit-test with fixture records from each historical version.
|
||||
|
||||
> **Do:** default `version` to the earliest shape (`1`) when the field is absent.
|
||||
> **Don't:** branch on the presence of individual fields scattered through the app to detect "old data." Centralize that knowledge in the migration function.
|
||||
|
||||
---
|
||||
|
||||
## 5. localStorage Preferences (Settings & App/UI Prefs)
|
||||
|
||||
Small, frequently-read structured records live in `localStorage`, not IndexedDB: **UserSettings** (one record) and **app/UI preferences** (snippet sort, panel layout). Why split them from `UserSettings`? UI prefs change often (drag a panel divider, toggle a sort) and shouldn't force a rewrite of the whole settings blob on every interaction.
|
||||
|
||||
The pattern is **load-with-fallback, write-through on change.**
|
||||
|
||||
- **Load-with-fallback:** merge the parsed stored object over a complete `DEFAULTS` constant. Missing keys (a setting added in a later build) and malformed JSON silently fall back to defaults — the app always gets a fully-populated object and never `undefined`-crashes on a new field.
|
||||
- **Write-through:** every update reads current, applies the change, and writes the whole record back immediately. No dirty-tracking, no flush step.
|
||||
- **Environment-guarded:** `localStorage` is absent or throws in some test/SSR contexts; guard access and degrade to defaults rather than throwing.
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/settings-store.ts
|
||||
const KEY = 'astrolabe:settings';
|
||||
|
||||
export const CURRENT_SETTINGS_VERSION = 1;
|
||||
|
||||
export interface UserSettings {
|
||||
version: number;
|
||||
editor: { fontSize: number; theme: string; minimap: boolean; wordWrap: 'on' | 'off';
|
||||
lineNumbers: 'on' | 'off'; tabSize: number };
|
||||
performance: { renderDebounce: number };
|
||||
ui: { theme: 'light' | 'experimental'; previewFitMode: 'default' | 'width' | 'height' | 'full' };
|
||||
formatting: { dateFormat: 'smart' | 'iso' | 'custom'; customDateFormat: string };
|
||||
}
|
||||
|
||||
// Defaults must match the authoritative spec §07 table exactly — that is the
|
||||
// contract; this is just where it's encoded.
|
||||
const DEFAULTS: UserSettings = {
|
||||
version: CURRENT_SETTINGS_VERSION,
|
||||
editor: { fontSize: 12, theme: 'auto', minimap: false, wordWrap: 'on',
|
||||
lineNumbers: 'on', tabSize: 2 },
|
||||
performance: { renderDebounce: 1500 },
|
||||
ui: { theme: 'light', previewFitMode: 'default' },
|
||||
formatting: { dateFormat: 'smart', customDateFormat: '' },
|
||||
};
|
||||
|
||||
// NOTE — editor.theme default is 'auto': the editor theme follows the app UI
|
||||
// theme (light -> light editor theme, experimental -> dark) via custom Monaco
|
||||
// themes that match the app chrome, unless the user picks an explicit override.
|
||||
// The explicit-override option set (custom themes; whether to include High
|
||||
// Contrast or the stock Monaco themes) is still TBD — see spec §07's provisional
|
||||
// editor-theme note. Resolve the `'auto'` sentinel to a concrete Monaco theme at
|
||||
// editor-config time, keyed off the current UI theme.
|
||||
|
||||
function available(): boolean {
|
||||
try {
|
||||
return typeof localStorage !== 'undefined' && typeof localStorage.getItem === 'function';
|
||||
} catch {
|
||||
return false; // access itself can throw (e.g. blocked storage)
|
||||
}
|
||||
}
|
||||
|
||||
export function loadSettings(): UserSettings {
|
||||
if (!available()) return structuredClone(DEFAULTS);
|
||||
try {
|
||||
const raw = localStorage.getItem(KEY);
|
||||
if (!raw) return structuredClone(DEFAULTS);
|
||||
const p = JSON.parse(raw);
|
||||
// Deep-merge each group over defaults so new keys fall back silently.
|
||||
return {
|
||||
version: CURRENT_SETTINGS_VERSION,
|
||||
editor: { ...DEFAULTS.editor, ...p.editor },
|
||||
performance: { ...DEFAULTS.performance, ...p.performance },
|
||||
ui: { ...DEFAULTS.ui, ...p.ui },
|
||||
formatting: { ...DEFAULTS.formatting, ...p.formatting },
|
||||
};
|
||||
} catch (err) {
|
||||
console.warn('[settings] failed to load, using defaults', err);
|
||||
return structuredClone(DEFAULTS);
|
||||
}
|
||||
}
|
||||
|
||||
export function saveSettings(s: UserSettings): void {
|
||||
if (!available()) return;
|
||||
try {
|
||||
localStorage.setItem(KEY, JSON.stringify({ ...s, version: CURRENT_SETTINGS_VERSION }));
|
||||
} catch (err) {
|
||||
console.warn('[settings] failed to save', err);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
App/UI prefs follow the identical pattern under their own keys, e.g.:
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/prefs-store.ts
|
||||
const SORT_KEY = 'astrolabe:snippet-sort';
|
||||
const LAYOUT_KEY = 'astrolabe:panel-layout';
|
||||
|
||||
const SORT_DEFAULTS = { sortBy: 'modified' as const, sortOrder: 'desc' as const };
|
||||
// loadSort()/saveSort() and loadLayout()/saveLayout() mirror §5's guard+fallback shape.
|
||||
```
|
||||
|
||||
> **Do:** keep a single complete `DEFAULTS` object as the source of truth and merge over it.
|
||||
> **Don't:** read individual keys with bespoke `?? fallback` at each call site; one stale default and the shapes drift.
|
||||
|
||||
---
|
||||
|
||||
## 6. Storage Tiers, Budgets & Quota Monitoring
|
||||
|
||||
Astrolabe has three tiers with different capacities and risk profiles:
|
||||
|
||||
| Tier | Backing | Holds | Budget & behavior |
|
||||
|------|---------|-------|-------------------|
|
||||
| **Snippet store** | IndexedDB `snippets` | All snippet records | Practical budget ~**5 MB**. A storage monitor estimates usage and surfaces a warning as it fills. Snippets are user-authored and irreplaceable, so we fail **loudly**. |
|
||||
| **Dataset store** | IndexedDB `datasets` | All dataset payloads | Separate, **high-capacity**; suited to large payloads. Lazily loaded (§3). |
|
||||
| **Settings & prefs** | localStorage | `UserSettings` + app/UI prefs (§5) | Small; effectively unbounded for this use. |
|
||||
|
||||
Splitting snippets and datasets into separate stores means a few large datasets can't crowd out the snippet budget, and the snippet monitor can report a meaningful "how full is my library" number without summing dataset bytes.
|
||||
|
||||
### Estimating usage
|
||||
|
||||
Use the Storage Manager API where available, with a manual byte-sum fallback for the snippet tier so the ~5 MB budget is always reportable.
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/storage-monitor.ts
|
||||
export interface StorageReport {
|
||||
snippetBytes: number; // estimated bytes used by the snippet tier
|
||||
snippetBudget: number; // 5 MB practical budget
|
||||
ratio: number; // snippetBytes / snippetBudget, clamped to >= 0
|
||||
warn: boolean; // ratio crossed the warning threshold
|
||||
}
|
||||
|
||||
const SNIPPET_BUDGET = 5 * 1024 * 1024;
|
||||
const WARN_AT = 0.8;
|
||||
|
||||
export async function reportSnippetUsage(snippets: Snippet[]): Promise<StorageReport> {
|
||||
// Cheap, deterministic estimate: serialize the records we hold.
|
||||
const snippetBytes = snippets.reduce(
|
||||
(n, s) => n + new Blob([JSON.stringify(s)]).size,
|
||||
0
|
||||
);
|
||||
const ratio = snippetBytes / SNIPPET_BUDGET;
|
||||
const report: StorageReport = {
|
||||
snippetBytes,
|
||||
snippetBudget: SNIPPET_BUDGET,
|
||||
ratio,
|
||||
warn: ratio >= WARN_AT,
|
||||
};
|
||||
if (report.warn) {
|
||||
console.warn(
|
||||
`[storage] snippet tier ${(ratio * 100).toFixed(0)}% of ${SNIPPET_BUDGET} bytes`
|
||||
);
|
||||
}
|
||||
return report;
|
||||
}
|
||||
```
|
||||
|
||||
### Fail loudly, never silently lose data
|
||||
|
||||
When a write would exceed quota, IndexedDB rejects with a `QuotaExceededError`. The adapter must **propagate** this so the UI can tell the user to export and prune — it must never swallow the error and pretend the save succeeded.
|
||||
|
||||
```ts
|
||||
export async function saveSnippet(s: Snippet): Promise<void> {
|
||||
try {
|
||||
await put('snippets', { ...s, version: CURRENT_SNIPPET_VERSION });
|
||||
} catch (err) {
|
||||
if (err instanceof DOMException && err.name === 'QuotaExceededError') {
|
||||
// Surface to the user via the store; do NOT silently drop the write.
|
||||
throw new StorageQuotaError('Snippet storage is full. Export and remove snippets to free space.');
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> **Do:** surface quota warnings *before* the budget is hit (the 80% threshold) and hard errors loudly when a write fails.
|
||||
> **Don't:** wrap a save in a bare `try/catch {}` that logs and returns — that turns "your work wasn't saved" into a silent data-loss bug. The only thing the adapter may safely swallow is a *read* failure, where falling back to defaults/empty is the correct behavior.
|
||||
|
||||
---
|
||||
|
||||
## 7. Checklist for Adding a New Persisted Entity
|
||||
|
||||
1. Define the record type with `id`, `created`, `modified`, and a `version` field.
|
||||
2. Decide the tier: small + critical → IndexedDB store with a monitored budget; large payload → separate high-capacity store with lazy loading (§3); tiny + frequently changing → localStorage pref (§5).
|
||||
3. Add the object store in `openDB`'s `onupgradeneeded`, guarded by `contains(...)`; bump `DB_VERSION` only if you changed store *layout*.
|
||||
4. Add a `migrate<Entity>()` function and call it on every read.
|
||||
5. Expose typed `load*/save*/ensure*` functions from one infrastructure module — and from *only* there.
|
||||
6. If the tier has a budget, hook it into the storage monitor and propagate `QuotaExceededError`.
|
||||
7. Test the adapter against `fake-indexeddb` / a localStorage stub; test the migration with fixtures from each historical version.
|
||||
@@ -0,0 +1,502 @@
|
||||
# 03 · Modal System
|
||||
|
||||
How Astrolabe manages its modals: a single metadata-driven registry, a thin
|
||||
lifecycle coordinator, and one rendering shell. This document is the
|
||||
authoritative architecture for adding, opening, closing, and rendering modals.
|
||||
|
||||
## Goals
|
||||
|
||||
- **One source of truth** for modal metadata — no `switch` statements scattered
|
||||
across the codebase keyed on the active modal.
|
||||
- **At most one modal open at a time** (mandated by the product spec). Opening a
|
||||
modal closes any other; the two never overlap.
|
||||
- **Uniform dismissal**: close button, `Escape`, or backdrop click — never a
|
||||
click inside the body.
|
||||
- **Accessible by default**: focus moves into the modal on open and returns to
|
||||
the trigger on close.
|
||||
- **Unsaved-change safety** for editing modals, with an explicit opt-out for
|
||||
modals that apply changes immediately.
|
||||
|
||||
The system is three layers, each with a single responsibility:
|
||||
|
||||
| Layer | Responsibility | Lives in |
|
||||
|-------|----------------|----------|
|
||||
| **Registry** | Static metadata per modal (title, validity, snapshot, init) | `src/app/modals/modal-registry.ts` |
|
||||
| **Coordinator** | Lifecycle: open, close, URL sync, change detection | `src/app/modals/ModalCoordinator.ts` |
|
||||
| **Shell** | Render exactly one modal; backdrop / Escape / focus trap | `src/app/App.tsx` + a `useFocusTrap` hook |
|
||||
|
||||
---
|
||||
|
||||
## The Modal Set
|
||||
|
||||
Astrolabe has a small, fixed set of modals. Model it as a closed union so the
|
||||
registry, coordinator, and shell are exhaustively type-checked.
|
||||
|
||||
```ts
|
||||
// src/app/modals/types.ts
|
||||
export type ModalName =
|
||||
| 'datasets' // Datasets manager (list / detail / new-dataset form)
|
||||
| 'settings' // Appearance, editor, performance, formatting prefs
|
||||
| 'about' // About & Help (shortcuts, privacy)
|
||||
| 'donate' // Donate
|
||||
| 'chartBuilder' // Visual no-JSON chart composition for a dataset
|
||||
| 'extract'; // Extract inline spec data into a new dataset
|
||||
|
||||
export type ActiveModal = ModalName | null;
|
||||
```
|
||||
|
||||
Two of these — `chartBuilder` and `extract` — are **opened from within
|
||||
workflows** (the Datasets manager and the snippet editor), not from the header
|
||||
toolbar. That is a UI wiring detail, not a structural one: every modal opens
|
||||
through the same coordinator regardless of where the trigger lives.
|
||||
|
||||
---
|
||||
|
||||
## Layer 1 — Registry
|
||||
|
||||
Each modal is registered once with its metadata. The registry is a plain lookup
|
||||
object keyed by `ModalName`; order is irrelevant. Utility queries
|
||||
(title, validity, whether a modal participates in URL state) read from the
|
||||
registry so there is exactly one place to change when behavior shifts.
|
||||
|
||||
### Config shape
|
||||
|
||||
```ts
|
||||
// src/app/modals/modal-registry.ts
|
||||
import type { ComponentType } from 'react';
|
||||
import type { ModalName } from './types';
|
||||
|
||||
export interface ModalConfig {
|
||||
name: ModalName;
|
||||
title: string; // i18n key or literal
|
||||
component: ComponentType<any>; // the body rendered inside the shell
|
||||
|
||||
/** Initialize transient modal state when it opens. `arg` carries an
|
||||
* optional sub-target (e.g. a dataset id for chartBuilder/extract). */
|
||||
init?: (arg?: string) => void;
|
||||
|
||||
/** Serializable snapshot of in-progress edits, used to detect unsaved
|
||||
* changes on close. OMIT for modals that apply immediately (settings,
|
||||
* about, donate) — omission opts out of the discard-confirmation. */
|
||||
getState?: () => Record<string, unknown> | null;
|
||||
|
||||
/** Whether the modal's primary action (Save / Apply) should be blocked
|
||||
* because the current input is invalid. Drives the disabled button. */
|
||||
hasError?: () => boolean;
|
||||
|
||||
/** Human-readable reason for the disabled action, shown as a tooltip. */
|
||||
getError?: () => string | null;
|
||||
|
||||
/** Whether this modal is reflected in the URL hash (back/forward, reload
|
||||
* restore). Datasets and Chart Builder are navigable; Donate is not. */
|
||||
isUrlNavigable?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### Example entries
|
||||
|
||||
```ts
|
||||
import { DatasetsModal } from '../components/DatasetsModal';
|
||||
import { SettingsModal } from '../components/SettingsModal';
|
||||
import { ChartBuilderModal } from '../components/ChartBuilderModal';
|
||||
import { ExtractModal } from '../components/ExtractModal';
|
||||
import { DonateModal } from '../components/DonateModal';
|
||||
import { AboutModal } from '../components/AboutModal';
|
||||
import { useDatasetStore } from '../stores/DatasetStore';
|
||||
import { useChartBuilderStore } from '../stores/ChartBuilderStore';
|
||||
import { useExtractStore } from '../stores/ExtractStore';
|
||||
import { useSettingsStore } from '../stores/SettingsStore';
|
||||
|
||||
// Per-modal transient state lives in the relevant feature store; the registry
|
||||
// reads it via `getState()` (Zustand), never through component hooks.
|
||||
export const MODAL_REGISTRY: Record<ModalName, ModalConfig> = {
|
||||
// Navigable, editing modal — snapshot guards unsaved work.
|
||||
datasets: {
|
||||
name: 'datasets',
|
||||
title: 'modals.datasets.title',
|
||||
component: DatasetsModal,
|
||||
isUrlNavigable: true,
|
||||
init: (datasetId) => useDatasetStore.getState().select(datasetId ?? null),
|
||||
getState: () => {
|
||||
const s = useDatasetStore.getState();
|
||||
return {
|
||||
view: s.view, // 'list' | 'detail' | 'new'
|
||||
draft: s.draftForm, // in-progress new/edit form
|
||||
};
|
||||
},
|
||||
hasError: () => useDatasetStore.getState().formError !== null,
|
||||
getError: () => useDatasetStore.getState().formError,
|
||||
},
|
||||
|
||||
// Opened from a workflow (a specific dataset), navigable, editing.
|
||||
chartBuilder: {
|
||||
name: 'chartBuilder',
|
||||
title: 'modals.chartBuilder.title',
|
||||
component: ChartBuilderModal,
|
||||
isUrlNavigable: true,
|
||||
init: (datasetId) => useChartBuilderStore.getState().initFor(datasetId),
|
||||
getState: () => ({ encoding: useChartBuilderStore.getState().encoding }),
|
||||
hasError: () => !useChartBuilderStore.getState().markType,
|
||||
getError: () =>
|
||||
useChartBuilderStore.getState().markType ? null : 'modals.chartBuilder.pickMark',
|
||||
},
|
||||
|
||||
// Opened from the snippet editor with the inline data to lift out.
|
||||
extract: {
|
||||
name: 'extract',
|
||||
title: 'modals.extract.title',
|
||||
component: ExtractModal,
|
||||
init: (sourceKey) => useExtractStore.getState().initFrom(sourceKey),
|
||||
getState: () => ({ name: useExtractStore.getState().name }),
|
||||
hasError: () => useExtractStore.getState().name.trim() === '',
|
||||
getError: () =>
|
||||
useExtractStore.getState().name.trim() ? null : 'modals.extract.nameRequired',
|
||||
},
|
||||
|
||||
// Applies immediately — no getState, so closing never prompts.
|
||||
settings: { name: 'settings', title: 'modals.settings.title', component: SettingsModal, isUrlNavigable: true, init: () => useSettingsStore.getState().loadFromPrefs() },
|
||||
|
||||
// Pure info modals — no state, no validity, not navigable for donate.
|
||||
about: { name: 'about', title: 'modals.about.title', component: AboutModal, isUrlNavigable: true },
|
||||
donate: { name: 'donate', title: 'modals.donate.title', component: DonateModal },
|
||||
};
|
||||
```
|
||||
|
||||
### Registry queries
|
||||
|
||||
All callers go through these helpers instead of inspecting the active modal
|
||||
directly:
|
||||
|
||||
```ts
|
||||
export const getModalConfig = (name: ActiveModal): ModalConfig | undefined =>
|
||||
name ? MODAL_REGISTRY[name] : undefined;
|
||||
|
||||
export const getModalTitle = (name: ActiveModal): string =>
|
||||
getModalConfig(name)?.title ?? '';
|
||||
|
||||
export const isUrlNavigable = (name: ActiveModal): boolean =>
|
||||
getModalConfig(name)?.isUrlNavigable ?? false;
|
||||
```
|
||||
|
||||
> **Why metadata-driven?** The alternative — branching on the active modal in
|
||||
> the shell, the URL sync, the keyboard handler, and the close logic — spreads
|
||||
> one decision across four files. Each new modal then means four edits and a
|
||||
> chance to forget one. With the registry, a new modal is one entry plus its
|
||||
> component.
|
||||
|
||||
**Do**
|
||||
- Add a modal by appending one `MODAL_REGISTRY` entry and writing its component.
|
||||
- Express validity through `hasError` / `getError` so the shell's action button
|
||||
and tooltip stay generic.
|
||||
- Omit `getState` for any modal that commits changes immediately.
|
||||
|
||||
**Don't**
|
||||
- Don't `switch (activeModal)` outside the shell's body render. Lookups belong
|
||||
in registry helpers.
|
||||
- Don't put rendering or DOM concerns in the registry — it is pure metadata.
|
||||
- Don't read rapidly-changing input state inside `hasError`/`getState` from
|
||||
component render paths; compute them with a selector at the shell boundary (see
|
||||
Shell layer) so a keystroke doesn't re-render the whole app.
|
||||
|
||||
---
|
||||
|
||||
## Layer 2 — Coordinator
|
||||
|
||||
The coordinator owns the modal lifecycle. It mutates a single piece of state —
|
||||
the active modal name — plus a snapshot used for change detection, and keeps the
|
||||
URL in sync. It is framework-light: pure functions over a Zustand store, unit
|
||||
testable without a DOM.
|
||||
|
||||
### State
|
||||
|
||||
The active modal name is **cross-cutting UI chrome**, so it lives on the central
|
||||
`useAppStore` (`activeModal` + the `setActiveModal` primitive — see
|
||||
docs/architecture/01). The coordinator never gets its own store; the only extra
|
||||
piece of state it needs is the change-detection **snapshot**, which is
|
||||
coordinator-internal (no component reads it), so it stays as a module-local
|
||||
variable rather than store state.
|
||||
|
||||
```ts
|
||||
// useAppStore already exposes:
|
||||
// activeModal: ModalName | null
|
||||
// setActiveModal: (modal: ModalName | null) => void
|
||||
```
|
||||
|
||||
### Open / close
|
||||
|
||||
```ts
|
||||
// src/app/modals/ModalCoordinator.ts
|
||||
import { useAppStore } from '../stores/AppStore';
|
||||
import { MODAL_REGISTRY, getModalConfig } from './modal-registry';
|
||||
import { syncModalToUrl, clearModalFromUrl } from './UrlStateSync';
|
||||
|
||||
let confirmDiscard: (msg: string) => Promise<boolean> = async () => true;
|
||||
export const setConfirm = (fn: typeof confirmDiscard) => { confirmDiscard = fn; };
|
||||
|
||||
// Coordinator-internal: the getState() JSON captured at open, compared on close.
|
||||
let stateSnapshot: string | null = null;
|
||||
|
||||
const snapshot = (name: ActiveModal) =>
|
||||
getModalConfig(name)?.getState
|
||||
? JSON.stringify(getModalConfig(name)!.getState!())
|
||||
: null;
|
||||
|
||||
/** Open `name`, optionally with a sub-target (dataset id, source key). */
|
||||
export function openModal(name: ModalName, arg?: string): void {
|
||||
// Opening any modal replaces the previous one — at most one open at a time.
|
||||
useAppStore.getState().setActiveModal(name);
|
||||
getModalConfig(name)?.init?.(arg);
|
||||
stateSnapshot = snapshot(name);
|
||||
syncModalToUrl(name, arg); // no-op when !isUrlNavigable
|
||||
}
|
||||
|
||||
/** Close the active modal. Prompts on unsaved changes unless `force`. */
|
||||
export async function closeModal(force = false): Promise<void> {
|
||||
const name = useAppStore.getState().activeModal;
|
||||
if (!name) return;
|
||||
|
||||
if (!force && hasUnsavedChanges()) {
|
||||
const ok = await confirmDiscard('modals.discardChanges');
|
||||
if (!ok) return;
|
||||
}
|
||||
|
||||
clearModalFromUrl(name);
|
||||
useAppStore.getState().setActiveModal(null);
|
||||
stateSnapshot = null;
|
||||
getModalConfig(name)?.init?.(undefined); // optional: reset transient state
|
||||
}
|
||||
|
||||
/** Cmd/Ctrl+K toggle for the Datasets manager. */
|
||||
export function toggleDatasets(): void {
|
||||
if (useAppStore.getState().activeModal === 'datasets') void closeModal();
|
||||
else openModal('datasets');
|
||||
}
|
||||
```
|
||||
|
||||
### Change detection
|
||||
|
||||
```ts
|
||||
export function hasUnsavedChanges(): boolean {
|
||||
const name = useAppStore.getState().activeModal;
|
||||
if (!name || stateSnapshot === null) return false; // no snapshot ⇒ opted out
|
||||
const current = getModalConfig(name)?.getState?.();
|
||||
if (current == null) return false;
|
||||
return JSON.stringify(current) !== stateSnapshot;
|
||||
}
|
||||
```
|
||||
|
||||
The snapshot is taken once on open and compared on close. Modals without
|
||||
`getState` (settings, about, donate) snapshot to `null`, so `hasUnsavedChanges`
|
||||
short-circuits and they close instantly — correct, because they either apply
|
||||
immediately or hold nothing to lose.
|
||||
|
||||
### Validity passthrough
|
||||
|
||||
```ts
|
||||
export const activeModalHasError = (): boolean =>
|
||||
getModalConfig(useAppStore.getState().activeModal)?.hasError?.() ?? false;
|
||||
|
||||
export const activeModalError = (): string | null =>
|
||||
getModalConfig(useAppStore.getState().activeModal)?.getError?.() ?? null;
|
||||
```
|
||||
|
||||
> **Why a coordinator instead of letting components open/close themselves?**
|
||||
> Centralizing means the "close the previous one", snapshot, URL-sync, and
|
||||
> discard-prompt rules are enforced once. A component that opened a peer modal
|
||||
> directly could bypass the discard check or leave the URL stale.
|
||||
|
||||
**Do**
|
||||
- Route every open/close through `openModal` / `closeModal`.
|
||||
- Take the snapshot in `openModal` (after `init`) and compare in `closeModal`.
|
||||
- Keep the coordinator DOM-free so it can be tested with plain Vitest.
|
||||
|
||||
**Don't**
|
||||
- Don't mutate `activeModal` directly from components or handlers.
|
||||
- Don't skip `closeModal`'s unsaved-change check by toggling state manually;
|
||||
pass `force` only when the user has explicitly saved or confirmed.
|
||||
|
||||
---
|
||||
|
||||
## Layer 3 — Shell
|
||||
|
||||
`App` renders **exactly one** modal — whichever `activeModal` names — inside a
|
||||
single reusable shell. The shell provides the backdrop, header, focus trap, and
|
||||
the generic close/action affordances; the modal's registered `component` fills
|
||||
the body.
|
||||
|
||||
```tsx
|
||||
// src/app/App.tsx (modal portion)
|
||||
import { useAppStore } from '../stores/AppStore';
|
||||
import { getModalConfig, getModalTitle } from '../modals/modal-registry';
|
||||
import { closeModal, activeModalHasError, activeModalError } from '../modals/ModalCoordinator';
|
||||
import { useFocusTrap } from '../hooks/useFocusTrap';
|
||||
|
||||
export function App() {
|
||||
const name = useAppStore((s) => s.activeModal);
|
||||
const config = getModalConfig(name);
|
||||
|
||||
// Move focus into the modal on open, return it to the trigger on close.
|
||||
const modalRef = useFocusTrap<HTMLDivElement>(name !== null);
|
||||
|
||||
// Derived at the shell boundary so per-keystroke store reads don't
|
||||
// re-render the whole app tree.
|
||||
const hasError = activeModalHasError();
|
||||
const errorMsg = activeModalError();
|
||||
|
||||
return (
|
||||
<div className={styles.app}>
|
||||
{/* ...library · editor · preview panes, header... */}
|
||||
|
||||
{config && (
|
||||
<div
|
||||
className={styles.backdrop}
|
||||
onClick={() => void closeModal()} // backdrop dismisses
|
||||
onKeyDown={(e) => { if (e.key === 'Escape') void closeModal(); }}
|
||||
>
|
||||
<div
|
||||
ref={modalRef}
|
||||
className={styles.modal}
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby="modal-title"
|
||||
onClick={(e) => e.stopPropagation()} // inside body never dismisses
|
||||
>
|
||||
<header className={styles.modalHeader}>
|
||||
<h2 id="modal-title">{t(getModalTitle(name))}</h2>
|
||||
<button aria-label={t('buttons.close')} onClick={() => void closeModal()}>×</button>
|
||||
</header>
|
||||
|
||||
<div className={styles.modalBody}>
|
||||
{/* The ONE place the active modal is mapped to a component. */}
|
||||
<config.component />
|
||||
</div>
|
||||
|
||||
{/* Optional generic action row for editing modals. A modal with no
|
||||
primary action (about, donate) can render its own footer/none. */}
|
||||
{config.getState && (
|
||||
<footer className={styles.modalFooter}>
|
||||
<button className="btn-secondary" onClick={() => void closeModal()}>
|
||||
{t('buttons.cancel')}
|
||||
</button>
|
||||
<button
|
||||
className="btn-primary"
|
||||
aria-disabled={hasError || undefined}
|
||||
title={errorMsg ? t(errorMsg) : undefined}
|
||||
onClick={() => { if (!hasError) config.component /* invoke save handler */; }}
|
||||
>
|
||||
{t('buttons.save')}
|
||||
</button>
|
||||
</footer>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Rendering `<config.component />` from the registry is the only modal-name→view
|
||||
mapping in the app. There is no `name === 'datasets' && <DatasetsModal/>` chain.
|
||||
|
||||
### Focus trap
|
||||
|
||||
A small hook saves the previously focused element, focuses the first focusable
|
||||
child on open, wraps `Tab`/`Shift+Tab` within the modal, and restores focus on
|
||||
close.
|
||||
|
||||
```ts
|
||||
// src/app/hooks/useFocusTrap.ts
|
||||
import { useRef, useEffect } from 'react';
|
||||
|
||||
const FOCUSABLE =
|
||||
'a[href], button:not([disabled]), input:not([disabled]), select:not([disabled]), ' +
|
||||
'textarea:not([disabled]), [tabindex]:not([tabindex="-1"])';
|
||||
|
||||
export function useFocusTrap<T extends HTMLElement = HTMLDivElement>(active: boolean) {
|
||||
const ref = useRef<T>(null);
|
||||
const returnTo = useRef<Element | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
const el = ref.current;
|
||||
if (!active || !el) return;
|
||||
|
||||
returnTo.current = document.activeElement;
|
||||
el.querySelector<HTMLElement>(FOCUSABLE)?.focus();
|
||||
|
||||
const onKey = (e: KeyboardEvent) => {
|
||||
if (e.key !== 'Tab') return;
|
||||
const f = el.querySelectorAll<HTMLElement>(FOCUSABLE);
|
||||
if (!f.length) return;
|
||||
const first = f[0], last = f[f.length - 1];
|
||||
if (e.shiftKey && document.activeElement === first) { e.preventDefault(); last.focus(); }
|
||||
else if (!e.shiftKey && document.activeElement === last) { e.preventDefault(); first.focus(); }
|
||||
};
|
||||
|
||||
el.addEventListener('keydown', onKey);
|
||||
return () => {
|
||||
el.removeEventListener('keydown', onKey);
|
||||
(returnTo.current as HTMLElement | null)?.focus(); // restore focus on close
|
||||
};
|
||||
}, [active]);
|
||||
|
||||
return ref;
|
||||
}
|
||||
```
|
||||
|
||||
> **Why one shell instead of each modal rendering its own chrome?** Backdrop
|
||||
> behavior, the focus trap, `aria-modal`, Escape handling, and the close button
|
||||
> are identical for every modal and easy to get subtly wrong (e.g. a backdrop
|
||||
> that dismisses on inner clicks). Centralizing guarantees consistency and means
|
||||
> accessibility is fixed once.
|
||||
|
||||
**Do**
|
||||
- Render the active modal via `<config.component />` — the single mapping point.
|
||||
- Put `onClick={closeModal}` on the backdrop and `stopPropagation` on the body.
|
||||
- Compute `hasError`/`getError`/preview reads with a selector at the shell level.
|
||||
- Gate the generic Save button on `hasError` and surface `getError` as its
|
||||
tooltip.
|
||||
|
||||
**Don't**
|
||||
- Don't render two modals simultaneously, and don't stack a second backdrop.
|
||||
- Don't attach the focus trap to the backdrop — attach it to the modal body so
|
||||
the backdrop click stays outside the trap.
|
||||
- Don't dismiss on clicks inside the body, and don't let Escape fire when no
|
||||
modal is open (the handler only exists while a modal renders).
|
||||
|
||||
---
|
||||
|
||||
## URL & Keyboard Integration
|
||||
|
||||
The coordinator is the join point for navigation:
|
||||
|
||||
- `openModal` calls `syncModalToUrl`; navigable modals write a hash
|
||||
(`#datasets`, `#datasets/dataset-<id>`, `#datasets/dataset-<id>/build`,
|
||||
`#settings`). Non-navigable modals (donate) write nothing.
|
||||
- `closeModal` calls `clearModalFromUrl`, returning to the underlying workspace
|
||||
hash.
|
||||
- On load, the URL restorer reads the hash and calls `openModal(name, arg)` to
|
||||
rehydrate the right modal and sub-target.
|
||||
- The global key handler maps `Cmd/Ctrl+K` → `toggleDatasets()`,
|
||||
`Cmd/Ctrl+,` → `openModal('settings')`, and `Escape` → `closeModal()` (the
|
||||
Escape binding is a no-op when `activeModal` is `null`).
|
||||
|
||||
Because all of these call the same coordinator functions, browser
|
||||
Back/Forward, keyboard shortcuts, and in-app triggers stay consistent — they
|
||||
share the open/close/snapshot/URL logic rather than reimplementing it.
|
||||
|
||||
---
|
||||
|
||||
## Adding a Modal: Checklist
|
||||
|
||||
1. Add the name to the `ModalName` union.
|
||||
2. Add one `MODAL_REGISTRY` entry (title, component; `getState`/`hasError`/
|
||||
`getError` if it edits; `isUrlNavigable` + `init(arg)` if navigable).
|
||||
3. Write the body component; it reads/writes its feature store (e.g.
|
||||
`useDatasetStore`, `useChartBuilderStore`) via a narrow selector.
|
||||
4. If navigable, add its hash form to the URL sync and restore logic.
|
||||
5. If it has a keyboard shortcut or workflow trigger, wire that to
|
||||
`openModal(name, arg)` — never to `activeModal` directly.
|
||||
|
||||
No edits to the shell render, the close logic, or the change-detection code are
|
||||
needed: those are generic and driven entirely by the registry.
|
||||
@@ -0,0 +1,463 @@
|
||||
# 04 · Routing & Global Events
|
||||
|
||||
Two small, related subsystems govern how the app talks to the browser shell:
|
||||
|
||||
1. **URL hash as view-state** — the current view (selected snippet, open dataset
|
||||
modal, etc.) lives in `location.hash`. It is read on load to restore state,
|
||||
written on navigation, and Back/Forward step between prior states. Result:
|
||||
every meaningful view is shareable, bookmarkable, and reload-safe.
|
||||
2. **Global event / keyboard routing** — a single router owns the
|
||||
document-level `keydown` / `paste` / `click` listeners. It runs an Escape
|
||||
priority chain, dispatches shortcuts, and consults a single
|
||||
`isInInteractiveContext()` helper so global shortcuts and paste handlers
|
||||
never fire while the user is typing in an input or the Monaco editor.
|
||||
|
||||
Both are layered the same way:
|
||||
|
||||
```
|
||||
src/app/infrastructure/url-hash.ts adapter: owns window.location & history
|
||||
src/app/orchestration/UrlStateSync.ts mediator: hash <-> Zustand stores
|
||||
src/app/orchestration/EventRouter.ts mediator: DOM events -> store actions
|
||||
src/app/orchestration/focus-utils.ts single-source isInInteractiveContext()
|
||||
```
|
||||
|
||||
Infrastructure modules touch browser globals; orchestration modules touch the
|
||||
Zustand stores. Components never read `location.hash` or attach
|
||||
`window.addEventListener` themselves — they go through these mediators.
|
||||
|
||||
---
|
||||
|
||||
## 1. URL Hash as View-State
|
||||
|
||||
### 1.1 The hash grammar
|
||||
|
||||
The hash is the serialized view. Astrolabe's forms:
|
||||
|
||||
| State | Hash |
|
||||
| ----------------------------- | --------------------------------- |
|
||||
| Default snippets view | _(empty / absent)_ |
|
||||
| A selected snippet | `#snippet-<id>` |
|
||||
| Datasets manager (list) | `#datasets` |
|
||||
| A specific dataset | `#datasets/dataset-<id>` |
|
||||
| New-dataset form | `#datasets/new` |
|
||||
| Chart Builder for a dataset | `#datasets/dataset-<id>/build` |
|
||||
|
||||
Snippet `id` is an opaque string; dataset `id` is the numeric dataset id
|
||||
rendered as a decimal string. The hash is the **only** persisted view-routing
|
||||
state — there is no in-memory "current route" that can drift from it.
|
||||
|
||||
### 1.2 The adapter: `infrastructure/url-hash.ts`
|
||||
|
||||
This is the only file that reads or writes `window.location` / `history`. It
|
||||
exposes a parse function (hash string → typed `ViewState`), a serialize
|
||||
function (`ViewState` → hash string), and write helpers. Keep it pure-ish:
|
||||
parsing is a total function with no side effects; writing is the only place
|
||||
`history.replaceState` is called.
|
||||
|
||||
```ts
|
||||
// src/app/infrastructure/url-hash.ts
|
||||
export type ViewState =
|
||||
| { kind: 'snippets' } // empty hash
|
||||
| { kind: 'snippet'; snippetId: string } // #snippet-<id>
|
||||
| { kind: 'datasets' } // #datasets
|
||||
| { kind: 'dataset'; datasetId: number } // #datasets/dataset-<id>
|
||||
| { kind: 'dataset-new' } // #datasets/new
|
||||
| { kind: 'dataset-build'; datasetId: number }; // .../build
|
||||
|
||||
export function parseHash(rawHash: string): ViewState {
|
||||
const hash = rawHash.replace(/^#/, '');
|
||||
if (hash === '') return { kind: 'snippets' };
|
||||
|
||||
const snippet = /^snippet-(.+)$/.exec(hash);
|
||||
if (snippet) return { kind: 'snippet', snippetId: snippet[1] };
|
||||
|
||||
const parts = hash.split('/').filter(Boolean);
|
||||
if (parts[0] === 'datasets') {
|
||||
if (parts.length === 1) return { kind: 'datasets' };
|
||||
if (parts[1] === 'new') return { kind: 'dataset-new' };
|
||||
const m = /^dataset-(\d+)$/.exec(parts[1]);
|
||||
if (m) {
|
||||
const id = Number(m[1]);
|
||||
if (parts[2] === 'build') return { kind: 'dataset-build', datasetId: id };
|
||||
return { kind: 'dataset', datasetId: id };
|
||||
}
|
||||
}
|
||||
// Unknown hash -> fall back to default rather than throwing.
|
||||
return { kind: 'snippets' };
|
||||
}
|
||||
|
||||
export function serializeHash(view: ViewState): string {
|
||||
switch (view.kind) {
|
||||
case 'snippets': return '';
|
||||
case 'snippet': return `#snippet-${view.snippetId}`;
|
||||
case 'datasets': return '#datasets';
|
||||
case 'dataset': return `#datasets/dataset-${view.datasetId}`;
|
||||
case 'dataset-new': return '#datasets/new';
|
||||
case 'dataset-build': return `#datasets/dataset-${view.datasetId}/build`;
|
||||
}
|
||||
}
|
||||
|
||||
export function readView(): ViewState {
|
||||
return parseHash(window.location.hash);
|
||||
}
|
||||
|
||||
/** Write without adding a history entry (in-place correction, restore). */
|
||||
export function replaceView(view: ViewState): void {
|
||||
const url = new URL(window.location.href);
|
||||
url.hash = serializeHash(view);
|
||||
url.search = '';
|
||||
window.history.replaceState({}, '', url.toString());
|
||||
}
|
||||
|
||||
/** Write and add a history entry (user navigation -> Back works). */
|
||||
export function pushView(view: ViewState): void {
|
||||
const url = new URL(window.location.href);
|
||||
url.hash = serializeHash(view);
|
||||
url.search = '';
|
||||
window.history.pushState({}, '', url.toString());
|
||||
}
|
||||
```
|
||||
|
||||
**`pushState` vs `replaceState` is the lever that makes Back/Forward feel
|
||||
right.** Use `pushView` for deliberate user navigation (selecting a snippet,
|
||||
opening a dataset) so each becomes a Back-able step. Use `replaceView` for
|
||||
restoring on load and for correcting a stale/invalid hash, where you do not want
|
||||
to litter history.
|
||||
|
||||
### 1.3 The mediator: `orchestration/UrlStateSync.ts`
|
||||
|
||||
`UrlStateSync` is the bridge between the hash and the Zustand stores. It does
|
||||
three jobs:
|
||||
|
||||
- **On load — restore:** read the view, validate referenced ids against the
|
||||
stores, and drive the stores to match. If an id no longer exists, fall back
|
||||
to the default view and `replaceView` to clean the URL.
|
||||
- **Hash → state (Back/Forward):** listen for `hashchange` and reconcile the
|
||||
stores to the new view. This is what makes the browser buttons work.
|
||||
- **State → hash:** expose typed `navigate*` helpers the rest of the app calls
|
||||
when the user moves around. These `pushView` (or `replaceView`).
|
||||
|
||||
Guard against feedback loops: writing the hash fires no `hashchange` when you
|
||||
use the History API the way above, but a defensive `applying` flag keeps the
|
||||
`hashchange` reconciler from re-triggering navigation it just caused.
|
||||
|
||||
```ts
|
||||
// src/app/orchestration/UrlStateSync.ts
|
||||
import { useSnippetStore } from '../stores/SnippetStore';
|
||||
import { useDatasetStore } from '../stores/DatasetStore';
|
||||
import { useAppStore } from '../stores/AppStore'; // activeModal, etc.
|
||||
import { readView, replaceView, pushView, type ViewState } from '../infrastructure/url-hash';
|
||||
|
||||
let applying = false; // suppress re-entrancy while we drive the stores
|
||||
let started = false;
|
||||
|
||||
// Restore/reconcile is the one path that writes `activeModal` with the bare
|
||||
// `setActiveModal` primitive instead of the coordinator's openModal/closeModal:
|
||||
// we are reflecting the URL *into* the stores, so we must NOT re-sync the URL or
|
||||
// run the unsaved-change discard prompt (the `applying` guard blocks re-entrancy).
|
||||
/** Make the stores reflect `view`. Falls back + cleans URL on dead ids. */
|
||||
function applyView(view: ViewState): void {
|
||||
applying = true;
|
||||
try {
|
||||
switch (view.kind) {
|
||||
case 'snippets':
|
||||
useAppStore.getState().setActiveModal(null);
|
||||
return;
|
||||
case 'snippet': {
|
||||
const snippet = useSnippetStore.getState().byId(view.snippetId);
|
||||
if (!snippet) { replaceView({ kind: 'snippets' }); return; }
|
||||
useAppStore.getState().setActiveModal(null);
|
||||
useSnippetStore.getState().select(view.snippetId);
|
||||
return;
|
||||
}
|
||||
case 'datasets':
|
||||
useAppStore.getState().setActiveModal('datasets');
|
||||
return;
|
||||
case 'dataset':
|
||||
case 'dataset-build': {
|
||||
const ds = useDatasetStore.getState().byId(view.datasetId);
|
||||
if (!ds) { replaceView({ kind: 'datasets' }); return; }
|
||||
useAppStore.getState().setActiveModal('datasets');
|
||||
useDatasetStore.getState().select(view.datasetId);
|
||||
if (view.kind === 'dataset-build') useAppStore.getState().setActiveModal('chartBuilder');
|
||||
return;
|
||||
}
|
||||
case 'dataset-new':
|
||||
useAppStore.getState().setActiveModal('datasets');
|
||||
useDatasetStore.getState().beginNew();
|
||||
return;
|
||||
}
|
||||
} finally {
|
||||
applying = false;
|
||||
}
|
||||
}
|
||||
|
||||
export function startUrlStateSync(): void {
|
||||
if (started) return;
|
||||
started = true;
|
||||
|
||||
// 1. Restore from the URL on load.
|
||||
applyView(readView());
|
||||
|
||||
// 2. Back/Forward -> reconcile stores.
|
||||
window.addEventListener('hashchange', () => {
|
||||
if (applying) return;
|
||||
applyView(readView());
|
||||
});
|
||||
|
||||
// 3. State -> hash. A store subscription keeps the URL honest if any code path
|
||||
// changes the active view without calling a navigate* helper. Optional;
|
||||
// explicit navigate* calls are the primary writer.
|
||||
useAppStore.subscribe((state, prev) => {
|
||||
if (applying) return;
|
||||
// derive ViewState from state and replaceView(...) here if desired
|
||||
});
|
||||
}
|
||||
|
||||
// --- State -> hash: the API the app calls on user navigation -------------
|
||||
export const navigate = {
|
||||
toSnippet: (id: string) => pushView({ kind: 'snippet', snippetId: id }),
|
||||
toSnippets: () => pushView({ kind: 'snippets' }),
|
||||
toDatasets: () => pushView({ kind: 'datasets' }),
|
||||
toDataset: (id: number) => pushView({ kind: 'dataset', datasetId: id }),
|
||||
toNewDataset: () => pushView({ kind: 'dataset-new' }),
|
||||
toChartBuilder: (id: number) => pushView({ kind: 'dataset-build', datasetId: id }),
|
||||
};
|
||||
```
|
||||
|
||||
**Do**
|
||||
|
||||
- Restore on load with `replaceView`; navigate at runtime with `pushView`.
|
||||
- Validate every id from the hash against the stores; fall back + clean URL on
|
||||
a miss (deleted/shared-stale ids are normal, not exceptional).
|
||||
- Keep `parseHash` / `serializeHash` pure and round-trippable — unit-test that
|
||||
`parseHash(serializeHash(v)) === v` for every `ViewState`.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't read or write `location.hash` from components — call `navigate.*`.
|
||||
- Don't `pushState` on load-restore (pollutes Back history).
|
||||
- Don't throw on an unrecognized hash; degrade to the default view.
|
||||
|
||||
---
|
||||
|
||||
## 2. Global Event / Keyboard Routing
|
||||
|
||||
### 2.1 The router: `orchestration/EventRouter.ts`
|
||||
|
||||
One module binds the document-level listeners (`keydown`, `paste`, `click`) and
|
||||
routes them. Centralizing this keeps ordering explicit and gives one place to
|
||||
reason about priority. The router owns two things in particular:
|
||||
|
||||
- the **Escape priority chain**, and
|
||||
- **shortcut dispatch**, gated by `isInInteractiveContext()`.
|
||||
|
||||
```ts
|
||||
// src/app/orchestration/EventRouter.ts
|
||||
import { useAppStore } from '../stores/AppStore';
|
||||
import { useSnippetStore } from '../stores/SnippetStore';
|
||||
import { navigate } from './UrlStateSync';
|
||||
import { openModal, closeModal, toggleDatasets } from '../modals/ModalCoordinator';
|
||||
import { isInInteractiveContext } from './focus-utils';
|
||||
|
||||
let started = false;
|
||||
|
||||
export function startEventRouter(): void {
|
||||
if (started) return;
|
||||
started = true;
|
||||
window.addEventListener('keydown', onKeyDown);
|
||||
window.addEventListener('paste', onPaste);
|
||||
}
|
||||
|
||||
export function stopEventRouter(): void {
|
||||
window.removeEventListener('keydown', onKeyDown);
|
||||
window.removeEventListener('paste', onPaste);
|
||||
started = false;
|
||||
}
|
||||
|
||||
const isMac = /Mac|iPhone|iPad|iPod/.test(navigator.platform);
|
||||
|
||||
function onKeyDown(e: KeyboardEvent): void {
|
||||
// --- Escape: highest priority, runs even inside editors/inputs ----------
|
||||
if (e.key === 'Escape') {
|
||||
if (handleEscapeChain()) e.preventDefault();
|
||||
return;
|
||||
}
|
||||
|
||||
const mod = isMac ? e.metaKey : e.ctrlKey;
|
||||
|
||||
// --- Shortcuts: never fire while typing in an input or Monaco ----------
|
||||
if (isInInteractiveContext()) return;
|
||||
|
||||
// Cmd/Ctrl + Shift + N -> new snippet
|
||||
if (mod && e.shiftKey && e.key.toLowerCase() === 'n') {
|
||||
e.preventDefault();
|
||||
const created = useSnippetStore.getState().create();
|
||||
navigate.toSnippet(created.id);
|
||||
return;
|
||||
}
|
||||
// Cmd/Ctrl + K -> toggle Datasets manager (coordinator owns open/close + URL)
|
||||
if (mod && !e.shiftKey && e.key.toLowerCase() === 'k') {
|
||||
e.preventDefault();
|
||||
toggleDatasets();
|
||||
return;
|
||||
}
|
||||
// Cmd/Ctrl + S -> publish current draft
|
||||
if (mod && e.key.toLowerCase() === 's') {
|
||||
e.preventDefault(); // override the browser "save page" dialog
|
||||
useSnippetStore.getState().publishDraft();
|
||||
return;
|
||||
}
|
||||
// Cmd/Ctrl + , -> settings (through the coordinator: snapshot + URL sync)
|
||||
if (mod && e.key === ',') {
|
||||
e.preventDefault();
|
||||
openModal('settings');
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
/** Returns true if it consumed the Escape (caller should preventDefault). */
|
||||
function handleEscapeChain(): boolean {
|
||||
// 1. Toast/message box would go here if it grew a blocking variant.
|
||||
// 2. Active modal — route through the coordinator so the unsaved-change
|
||||
// discard prompt runs and the URL is cleared. NEVER setActiveModal(null)
|
||||
// here: that would silently drop in-progress dataset/chart-builder edits.
|
||||
if (useAppStore.getState().activeModal) {
|
||||
void closeModal();
|
||||
return true;
|
||||
}
|
||||
// 3. Open menu / popover.
|
||||
if (useAppStore.getState().openMenu) {
|
||||
useAppStore.getState().setOpenMenu(null);
|
||||
return true;
|
||||
}
|
||||
// 4. Active selection (e.g. selected snippet in the library).
|
||||
if (useSnippetStore.getState().selectionId) {
|
||||
useSnippetStore.getState().clearSelection();
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function onPaste(e: ClipboardEvent): void {
|
||||
// Paste-to-import (e.g. paste a Vega-Lite spec) must NOT hijack a paste the
|
||||
// user makes inside the editor or an input.
|
||||
if (isInInteractiveContext()) return;
|
||||
// ... route clipboard text to the import handler ...
|
||||
}
|
||||
```
|
||||
|
||||
**The Escape chain is an explicit, ordered ladder, top-down.** Each rung
|
||||
returns as soon as it consumes the event, so only the topmost active layer
|
||||
reacts. Order matters: a blocking message box outranks a modal, a modal
|
||||
outranks an open menu, a menu outranks a selection. Add new dismissible layers
|
||||
by inserting a rung at the right priority — never by sprinkling
|
||||
`document.addEventListener('keydown', …Escape…)` in a component.
|
||||
|
||||
**Shortcuts override browser defaults.** Each handled combo calls
|
||||
`e.preventDefault()` so Cmd/Ctrl+S does not trigger "save page", Cmd/Ctrl+K
|
||||
does not focus the browser search bar, etc.
|
||||
|
||||
Note the asymmetry: **Escape is checked before the interactive-context gate**
|
||||
(you want Escape to dismiss a modal even while focus is in the editor), whereas
|
||||
all other shortcuts are checked **after** the gate (so they don't fire mid-typing).
|
||||
|
||||
### 2.2 The single-source helper: `orchestration/focus-utils.ts`
|
||||
|
||||
There is exactly **one** function that answers "is the user currently typing in
|
||||
an editable surface?" Every shortcut path and the paste handler call it. Never
|
||||
inline element-type checks — one place to get it right, one place to fix it
|
||||
when the DOM changes.
|
||||
|
||||
> **Monaco difference (important):** Astrolabe's spec editor is **Monaco**, not
|
||||
> CodeMirror. Monaco renders into a `.monaco-editor` container and keeps focus
|
||||
> on a hidden `<textarea class="inputarea">` inside it. The detector must match
|
||||
> Monaco's DOM — a `.monaco-editor` ancestor (and/or the inputarea) — **not** a
|
||||
> `.cm-editor` / `.cm-content` selector. If you copy a CodeMirror check here it
|
||||
> will silently fail and global shortcuts will fire while the user edits a spec.
|
||||
|
||||
```ts
|
||||
// src/app/orchestration/focus-utils.ts
|
||||
|
||||
/**
|
||||
* True when focus is in an editable surface where global shortcuts and
|
||||
* paste-to-import must be suppressed: <input>, <textarea>, <select>,
|
||||
* contenteditable, or the Monaco editor.
|
||||
*
|
||||
* This is the SINGLE source of truth — do not inline these checks elsewhere.
|
||||
*/
|
||||
export function isInInteractiveContext(): boolean {
|
||||
const el = document.activeElement as HTMLElement | null;
|
||||
if (!el) return false;
|
||||
|
||||
const tag = el.tagName.toLowerCase();
|
||||
if (tag === 'input' || tag === 'textarea' || tag === 'select') return true;
|
||||
if (el.isContentEditable) return true;
|
||||
|
||||
// Monaco renders into a .monaco-editor container; its focused element is a
|
||||
// hidden <textarea class="inputarea"> (already caught above) but guard the
|
||||
// container explicitly so focus on any inner node still counts.
|
||||
if (el.closest?.('.monaco-editor')) return true;
|
||||
|
||||
return false;
|
||||
}
|
||||
```
|
||||
|
||||
**Do**
|
||||
|
||||
- Route all global keyboard/paste/click through `EventRouter`; bind listeners
|
||||
in exactly one place, started once at app init.
|
||||
- Express Escape as an ordered chain that returns on first consumption.
|
||||
- Call `isInInteractiveContext()` everywhere a global handler might collide
|
||||
with typing; keep it the only definition.
|
||||
- `preventDefault()` on every shortcut the app claims, so it overrides the
|
||||
browser default.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't add ad-hoc `window`/`document` keydown listeners in components.
|
||||
- Don't inline `tagName === 'textarea'` / editor-class checks at call sites —
|
||||
call the helper.
|
||||
- Don't match a CodeMirror selector for the editor; Astrolabe is Monaco.
|
||||
- Don't gate Escape behind `isInInteractiveContext()` — Escape should still
|
||||
close a modal while the editor has focus.
|
||||
|
||||
---
|
||||
|
||||
## 3. Wiring at startup
|
||||
|
||||
Both subsystems start once, after the stores are hydrated from persistence, in
|
||||
the app's init/orchestration step:
|
||||
|
||||
```ts
|
||||
// src/app/orchestration/bootstrap.ts (sketch)
|
||||
import { startUrlStateSync } from './UrlStateSync';
|
||||
import { startEventRouter } from './EventRouter';
|
||||
|
||||
export function initApp(): void {
|
||||
// ... load settings + hydrate snippet/dataset stores from IndexedDB/localStorage ...
|
||||
startUrlStateSync(); // restore view from hash, then keep hash <-> stores in sync
|
||||
startEventRouter(); // bind global keyboard/paste routing
|
||||
}
|
||||
```
|
||||
|
||||
Order: hydrate stores first (so hash-restore can resolve ids), then
|
||||
`startUrlStateSync` (it reads the hash and may drive the stores), then
|
||||
`startEventRouter`. Each `start*` is idempotent and has a matching `stop*` for
|
||||
teardown in tests.
|
||||
|
||||
---
|
||||
|
||||
## 4. Testing notes
|
||||
|
||||
- **`parseHash` / `serializeHash`:** pure, so test directly. Cover every
|
||||
`ViewState`, the empty hash, and at least one malformed hash → default.
|
||||
Assert the round-trip identity.
|
||||
- **`isInInteractiveContext`:** happy-dom test (the project's Vitest env). Mount
|
||||
an `<input>`, a `contenteditable` div, and a `<div class="monaco-editor"><textarea/></div>`;
|
||||
focus each and assert `true`; assert `false` for a focused `<button>`.
|
||||
- **Escape chain:** with stores in known states, dispatch a synthetic Escape
|
||||
and assert only the top active layer changed.
|
||||
- **Restore-on-load with dead id:** seed an empty store, set
|
||||
`location.hash = '#snippet-gone'`, call `startUrlStateSync()`, assert the
|
||||
view fell back to default and the hash was cleaned.
|
||||
@@ -0,0 +1,448 @@
|
||||
# Rendering, Theming & Live Preview
|
||||
|
||||
How Astrolabe turns a user-authored Vega-Lite specification into a live chart in
|
||||
the preview pane. This covers four mechanics: **embedding** a spec via
|
||||
`vega-embed`, **theming** so charts match the active UI theme, **debounced
|
||||
re-rendering** so typing stays smooth, and **error handling** so a broken spec
|
||||
produces a readable message and self-heals. It deliberately stops at the
|
||||
embedding boundary — the *content* of the spec (resolving named-dataset
|
||||
references, applying fit-mode sizing) is prepared upstream by a pure transform;
|
||||
see §6.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Embedding Boundary
|
||||
|
||||
The preview is a thin imperative layer wrapping the `vega-embed` library, driven
|
||||
by reactive store state. The flow is always the same:
|
||||
|
||||
```
|
||||
spec text ──parse──▶ Vega-Lite spec object
|
||||
│
|
||||
▼
|
||||
prepareSpecForRender(spec, { fitMode }) ← pure, src/core/rendering.ts
|
||||
│ (operates on a COPY; never mutates the stored spec)
|
||||
▼
|
||||
render(node, preparedSpec, config) ← src/app, this doc
|
||||
│
|
||||
vega-embed ─▶ View ─▶ SVG in the DOM node
|
||||
```
|
||||
|
||||
`vega-embed` is the only place in the app that touches the chart DOM. Everything
|
||||
above it is data; everything below it is a Vega `View` we own and must tear down.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** keep all `vega-embed` calls behind one small renderer module. Components
|
||||
ask the renderer to draw a spec into a node; they never import `vega-embed`
|
||||
directly.
|
||||
- **Do** treat the renderer as imperative glue driven by store state (via a
|
||||
`subscribe` listener), not as reactive state itself.
|
||||
- **Don't** scatter `vegaEmbed(...)` calls across components.
|
||||
|
||||
---
|
||||
|
||||
## 2. vega-embed Integration
|
||||
|
||||
A single async `render` function embeds a prepared spec into a DOM node. Three
|
||||
non-negotiable embed options, plus disciplined teardown of the previous view:
|
||||
|
||||
```ts
|
||||
// src/app/services/chart-renderer.ts (sketch)
|
||||
import vegaEmbed, { type Result as EmbedResult } from 'vega-embed';
|
||||
import type { Config, TopLevelSpec } from 'vega-lite';
|
||||
|
||||
export interface RenderHandle {
|
||||
/** Finalize the underlying Vega view and release its resources. */
|
||||
destroy(): void;
|
||||
}
|
||||
|
||||
export async function renderSpec(
|
||||
node: HTMLElement,
|
||||
spec: TopLevelSpec,
|
||||
config: Config,
|
||||
): Promise<RenderHandle> {
|
||||
const result: EmbedResult = await vegaEmbed(node, spec, {
|
||||
actions: false, // no built-in export/source/editor menu — clean chart
|
||||
renderer: 'svg', // crisp, inspectable, copyable output
|
||||
config, // theme config (see §3)
|
||||
});
|
||||
|
||||
return {
|
||||
destroy() {
|
||||
// Frees timers, listeners, and the canvas/SVG the view created.
|
||||
result.view.finalize();
|
||||
node.replaceChildren(); // drop any leftover DOM the embed inserted
|
||||
},
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### The view lifecycle is the bug surface
|
||||
|
||||
Every successful `vegaEmbed` returns a `result.view` (a live Vega `View`
|
||||
instance). It owns timers, signal listeners, and DOM. If you embed a new spec
|
||||
into the same node *without* finalizing the old view, the old one leaks — its
|
||||
listeners keep firing and resources accumulate over a long editing session.
|
||||
|
||||
The renderer that drives re-rendering must therefore hold the previous handle and
|
||||
destroy it before (or while) creating the next:
|
||||
|
||||
```ts
|
||||
let current: RenderHandle | null = null;
|
||||
|
||||
async function rerender(node: HTMLElement, spec: TopLevelSpec, config: Config) {
|
||||
current?.destroy(); // tear down the previous view first
|
||||
current = await renderSpec(node, spec, config);
|
||||
}
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** pass `actions: false`. Astrolabe owns its own export/copy affordances;
|
||||
the library's overlay menu does not belong on the preview.
|
||||
- **Do** call `view.finalize()` on every previous view before rendering a new
|
||||
one, and on component unmount.
|
||||
- **Do** keep exactly one live view per preview node.
|
||||
- **Don't** re-embed into a node whose previous view you have not finalized.
|
||||
- **Don't** keep a reference to a finalized view; null it out.
|
||||
|
||||
---
|
||||
|
||||
## 3. Theme Follows the UI Theme
|
||||
|
||||
A Vega-Lite **config** object styles every chart globally — fonts, axis colors,
|
||||
background, the categorical color range, default mark colors. Astrolabe ships one
|
||||
config per UI theme so charts visually belong to the app rather than looking like
|
||||
stock Vega-Lite.
|
||||
|
||||
```ts
|
||||
// src/core/vega-themes.ts (sketch)
|
||||
import type { Config } from 'vega-lite';
|
||||
|
||||
export const lightChartConfig: Config = {
|
||||
background: 'transparent',
|
||||
font: '"Inter", sans-serif',
|
||||
title: { fontSize: 15, fontWeight: 600, color: '#1c1c1e' },
|
||||
axis: {
|
||||
domainColor: '#1c1c1e',
|
||||
gridColor: '#e4e4e7',
|
||||
gridDash: [3, 3],
|
||||
labelColor: '#52525b',
|
||||
titleColor: '#1c1c1e',
|
||||
labelFontSize: 11,
|
||||
titleFontSize: 12,
|
||||
},
|
||||
range: {
|
||||
category: ['#2f6df6', '#f5a524', '#17b890', '#e5484d', '#8b5cf6', '#0ea5e9'],
|
||||
},
|
||||
view: { stroke: 'transparent' },
|
||||
};
|
||||
|
||||
export const experimentalChartConfig: Config = {
|
||||
background: 'transparent',
|
||||
font: '"Inter", sans-serif',
|
||||
title: { fontSize: 15, fontWeight: 600, color: '#f4f4f5' },
|
||||
axis: {
|
||||
domainColor: '#a1a1aa',
|
||||
gridColor: '#3f3f46',
|
||||
gridDash: [3, 3],
|
||||
labelColor: '#a1a1aa',
|
||||
titleColor: '#f4f4f5',
|
||||
labelFontSize: 11,
|
||||
titleFontSize: 12,
|
||||
},
|
||||
range: {
|
||||
category: ['#5b8def', '#f5a524', '#2dd4a7', '#f0666b', '#a78bfa', '#38bdf8'],
|
||||
},
|
||||
view: { stroke: 'transparent' },
|
||||
};
|
||||
```
|
||||
|
||||
One mapping, in one place, is the single source of truth for theme → config:
|
||||
|
||||
```ts
|
||||
// src/core/vega-themes.ts
|
||||
import type { UiTheme } from './theme'; // core-local — never import from src/app
|
||||
|
||||
const CHART_CONFIG: Record<UiTheme, Config> = {
|
||||
light: lightChartConfig,
|
||||
experimental: experimentalChartConfig,
|
||||
};
|
||||
|
||||
export function chartConfigFor(theme: UiTheme): Config {
|
||||
return CHART_CONFIG[theme];
|
||||
}
|
||||
```
|
||||
|
||||
The renderer reads the active UI theme (from the store) and passes the matching config into
|
||||
`renderSpec`. When the theme changes, the same subscriber that drives
|
||||
re-rendering picks up the new config and the chart restyles automatically.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** keep `chartConfigFor` as the *only* place that maps a UI theme to a Vega
|
||||
config. Adding a UI theme = adding one config and one map entry.
|
||||
- **Do** set chart `background: 'transparent'` so the pane's own background shows
|
||||
through and theme switches look seamless.
|
||||
- **Don't** inline colors or fonts into individual specs to "match the theme" —
|
||||
that is the config's job, and per-spec styling drifts from the app.
|
||||
- **Don't** let the user's stored spec carry a `config`; the theme config is
|
||||
applied at embed time via the embed options, leaving the spec theme-agnostic.
|
||||
|
||||
---
|
||||
|
||||
## 4. Field-Name Escaping
|
||||
|
||||
Vega-Lite treats `.`, `[`, and `]` inside a `field:` string as **nested-property
|
||||
accessors**: `field: "user.age"` reads `row.user.age`, not a column literally
|
||||
named `"user.age"`. Astrolabe renders arbitrary user data whose column names may
|
||||
contain those characters, so any column name placed into a `field:` (or `as:`,
|
||||
`groupby:`, tooltip `field:`, etc.) must be escaped first.
|
||||
|
||||
```ts
|
||||
// src/core/rendering.ts (sketch)
|
||||
/** Escape `.`/`[`/`]` so Vega-Lite treats the string as a literal field name. */
|
||||
export function escapeVegaField(name: string): string {
|
||||
return name.replace(/([.[\]])/g, '\\$1');
|
||||
}
|
||||
```
|
||||
|
||||
```ts
|
||||
// usage when constructing/normalizing an encoding that references a column:
|
||||
encoding.x = { field: escapeVegaField(columnName), type: 'quantitative' };
|
||||
```
|
||||
|
||||
This matters wherever Astrolabe *constructs* spec fragments from data-derived
|
||||
column names — most notably the chart builder (see *Chart Builder* spec) and any
|
||||
helper that injects an encoding. For specs the user authored by hand, escaping is
|
||||
the user's responsibility; Astrolabe does not rewrite hand-authored `field:`
|
||||
values.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** route every data-derived column name through `escapeVegaField` before it
|
||||
lands in a `field:` (or any field-position key).
|
||||
- **Don't** ever pass a raw column name to `field:`. If the name came from data,
|
||||
it is unescaped until proven otherwise.
|
||||
|
||||
---
|
||||
|
||||
## 5. Debounced Preview
|
||||
|
||||
Rendering must never compete with typing. The preview re-renders only after the
|
||||
user pauses, the pending render is cancelled on each new keystroke, and a render
|
||||
in flight never blocks the editor.
|
||||
|
||||
The debounce delay is **user-configurable** via the `performance.renderDebounce`
|
||||
setting (range ~500–5000 ms). Read it live so changes take effect without reload.
|
||||
|
||||
```ts
|
||||
// src/app/services/debounced-renderer.ts (sketch)
|
||||
export interface DebouncedRenderer {
|
||||
/** Schedule a render after the debounce window; resets the timer. */
|
||||
schedule(): void;
|
||||
/** Render now, skipping the debounce (e.g. on fit-mode change or theme flip). */
|
||||
flush(): void;
|
||||
/** Cancel a pending render without rendering. */
|
||||
cancel(): void;
|
||||
}
|
||||
|
||||
export function createDebouncedRenderer(opts: {
|
||||
/** Current debounce delay in ms; read fresh each schedule so settings apply live. */
|
||||
delayMs: () => number;
|
||||
/** Performs one render. Reads the current spec/theme; awaits the embed. */
|
||||
render: () => Promise<void>;
|
||||
/** Toggle the non-blocking busy indicator. */
|
||||
setBusy: (busy: boolean) => void;
|
||||
}): DebouncedRenderer {
|
||||
let timer: ReturnType<typeof setTimeout> | null = null;
|
||||
let generation = 0; // guards against a stale in-flight render finishing late
|
||||
|
||||
const run = async () => {
|
||||
timer = null;
|
||||
const mine = ++generation;
|
||||
opts.setBusy(true);
|
||||
try {
|
||||
await opts.render();
|
||||
} finally {
|
||||
// Only the most recent render clears the indicator.
|
||||
if (mine === generation) opts.setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
return {
|
||||
schedule() {
|
||||
if (timer) clearTimeout(timer); // cancel the pending render
|
||||
timer = setTimeout(run, opts.delayMs());
|
||||
},
|
||||
flush() {
|
||||
if (timer) { clearTimeout(timer); timer = null; }
|
||||
void run();
|
||||
},
|
||||
cancel() {
|
||||
if (timer) { clearTimeout(timer); timer = null; }
|
||||
generation++; // abandon any in-flight result
|
||||
},
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Wiring it to the store
|
||||
|
||||
Startup subscribers observe the inputs that affect the picture — the current spec
|
||||
text, the active fit mode, the UI theme — and call `schedule()` (debounced) for
|
||||
spec edits, or `flush()` for instantaneous controls like a fit-mode toggle:
|
||||
|
||||
```ts
|
||||
// wired once at startup
|
||||
useEditorStore.subscribe((s, prev) => {
|
||||
if (s.currentSpecText !== prev.currentSpecText) renderer.schedule(); // react to edits
|
||||
});
|
||||
|
||||
useSettingsStore.subscribe((s, prev) => {
|
||||
if (s.previewFitMode !== prev.previewFitMode || s.uiTheme !== prev.uiTheme) {
|
||||
renderer.flush(); // immediate, no debounce
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### Busy indicator
|
||||
|
||||
`setBusy(true/false)` toggles store state that the preview reads to overlay a
|
||||
**subtle, non-blocking** spinner/shimmer. It sits *over* the existing chart so the
|
||||
last good render stays visible while the next one computes — the pane never goes
|
||||
blank mid-edit.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** read `renderDebounce` fresh on each `schedule()` (via the `delayMs()`
|
||||
thunk) so a settings change applies immediately.
|
||||
- **Do** cancel the pending timer on every new input before scheduling the next.
|
||||
- **Do** guard against out-of-order completion (the `generation` counter): a slow
|
||||
render that resolves after a newer one must not clobber the indicator or view.
|
||||
- **Do** keep the busy indicator non-blocking and overlaid; never clear the chart
|
||||
to show "rendering…".
|
||||
- **Don't** render synchronously on every keystroke.
|
||||
- **Don't** await a render inside an input/keydown handler.
|
||||
|
||||
---
|
||||
|
||||
## 6. Rendering Contract Lives Upstream (reference)
|
||||
|
||||
Before a spec reaches `renderSpec`, it passes through a **pure** transform in
|
||||
`src/core/rendering.ts`:
|
||||
|
||||
```ts
|
||||
prepareSpecForRender(spec, { fitMode }): TopLevelSpec
|
||||
```
|
||||
|
||||
It does two deterministic things, on a **deep copy** of the spec:
|
||||
|
||||
1. **Dataset reference resolution** — replaces any named-data reference with the
|
||||
referenced dataset's actual contents (inline values, raw CSV/TSV text, or a
|
||||
URL reference), recursing into layered/concat/child sub-specs.
|
||||
2. **Fit-mode sizing** — rewrites `width`/`height` per the active fit mode using
|
||||
Vega-Lite's `"container"` keyword (Original = untouched; Width/Height/Full set
|
||||
the corresponding dimension(s) to `"container"`), recursing the same way.
|
||||
|
||||
This is *content* preparation, not embedding, and it is fully covered by the
|
||||
*Live Preview* spec. The only invariant this doc cares about:
|
||||
|
||||
> `prepareSpecForRender` runs on a copy and returns a new spec. The renderer
|
||||
> embeds that returned spec. **The user's stored spec is never mutated by
|
||||
> rendering.**
|
||||
|
||||
The container-relative fit modes (Width/Height/Full) depend on `renderer: 'svg'`
|
||||
plus `"container"` sizing to follow the pane; when the pane resizes, re-running
|
||||
`prepareSpecForRender` + re-embedding (a `flush()`) re-fits the chart.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** call `prepareSpecForRender` between parse and embed, every render.
|
||||
- **Don't** put reference resolution or fit-mode logic in the renderer — it is
|
||||
pure core logic and must be unit-testable without a DOM.
|
||||
- **Don't** mutate the input spec anywhere in the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 7. Error Handling
|
||||
|
||||
A spec that cannot be rendered must produce a **readable** message in the preview
|
||||
area and recover on its own once the spec is valid again. Errors arise at three
|
||||
stages, all funneled to one error field the preview reads:
|
||||
|
||||
| Stage | Failure | Surfaced as |
|
||||
|---|---|---|
|
||||
| Parse | Invalid JSON | "Invalid JSON: …" |
|
||||
| Prepare (`prepareSpecForRender`) | Referenced dataset missing/unfetchable | "Dataset not found: …" |
|
||||
| Embed (`vega-embed`) | Vega-Lite compile / data error | "Rendering error: …" |
|
||||
|
||||
```ts
|
||||
// inside render(), driven by the debounced renderer
|
||||
async function render(): Promise<void> {
|
||||
const text = useEditorStore.getState().currentSpecText.trim();
|
||||
|
||||
// Empty/blank is NOT an error — render nothing, clean pane.
|
||||
if (!text) {
|
||||
current?.destroy();
|
||||
current = null;
|
||||
usePreviewStore.getState().setError(null);
|
||||
return;
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(text);
|
||||
} catch (e) {
|
||||
usePreviewStore.getState().setError(`Invalid JSON: ${(e as Error).message}`);
|
||||
return; // keep the last good chart underneath the error, or show the message
|
||||
}
|
||||
|
||||
try {
|
||||
const { previewFitMode, uiTheme } = useSettingsStore.getState();
|
||||
const prepared = prepareSpecForRender(parsed, { fitMode: previewFitMode });
|
||||
const config = chartConfigFor(uiTheme);
|
||||
current?.destroy();
|
||||
current = await renderSpec(node, prepared, config);
|
||||
usePreviewStore.getState().setError(null); // success clears any prior error
|
||||
} catch (e) {
|
||||
usePreviewStore.getState().setError(
|
||||
`Rendering error: ${(e as Error).message}. ` +
|
||||
`Check your JSON syntax and that the spec is valid Vega-Lite.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The preview component renders the chart node when `error` is `null`, and the
|
||||
error panel when it is set. Because **every successful render clears the error**,
|
||||
recovery is automatic: the next valid edit re-renders and wipes the message — no
|
||||
manual retry, no reload.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Do** treat empty/blank spec text as "render nothing" — finalize the current
|
||||
view, clear the error, show a clean empty pane.
|
||||
- **Do** clear the error state on every successful render.
|
||||
- **Do** make messages legible and actionable (the underlying reason plus a hint
|
||||
to check JSON/Vega-Lite validity), never a raw stack trace dump.
|
||||
- **Do** distinguish the failing stage in the message (Invalid JSON vs Dataset
|
||||
not found vs Rendering error).
|
||||
- **Don't** show a broken/partial chart — replace the chart area with the
|
||||
message.
|
||||
- **Don't** require a manual "retry"; validity restores the chart on its own.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Concern | Mechanism | Source of truth |
|
||||
|---|---|---|
|
||||
| Embedding | One `renderSpec` over `vega-embed`, `actions: false`, `renderer: 'svg'` | `src/app/services/chart-renderer.ts` |
|
||||
| View teardown | `view.finalize()` before each re-render and on unmount | the renderer's `RenderHandle` |
|
||||
| Theming | Vega `Config` per UI theme, applied at embed time | `chartConfigFor()` in `src/core/vega-themes.ts` |
|
||||
| Field names | `escapeVegaField` on every data-derived `field:` | `src/core/rendering.ts` |
|
||||
| Debounce | `createDebouncedRenderer`, delay from `renderDebounce` setting | `src/app/services/debounced-renderer.ts` |
|
||||
| Spec prep | `prepareSpecForRender` (pure, on a copy) | `src/core/rendering.ts` (see *Live Preview*) |
|
||||
| Errors | One error field, cleared on success, empty = nothing | `PreviewStore.error` |
|
||||
@@ -0,0 +1,346 @@
|
||||
# Type Inference & Data Profiling
|
||||
|
||||
How Astrolabe looks at a tabular dataset and figures out, for each column, what
|
||||
kind of data it holds — `number`, `string`, `date`, or `boolean` — and how it
|
||||
rolls those facts up into the **profile** stored on a dataset record.
|
||||
|
||||
This is pure, portable logic. It lives in `src/core/`, touches no browser APIs
|
||||
and no React, takes plain values in and returns plain data out, and is covered
|
||||
by Vitest unit tests. Anything that needs a profile (the create form, the edit
|
||||
flow, the detail panel) calls into this module; nothing here reaches back out.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why infer types at all
|
||||
|
||||
A dataset is just rows of values. The UI wants to *describe* it without
|
||||
re-parsing the payload every time: how many rows and columns, what the columns
|
||||
are called, and roughly what each column contains. The inferred type drives the
|
||||
small type indicator next to each column name in the dataset detail panel and
|
||||
the meta line in the list. It is a **display hint**, not a contract — nothing
|
||||
downstream coerces values based on it, and Vega-Lite does its own type handling
|
||||
at render time. Because it is only a hint, a wrong guess is cheap, and the rules
|
||||
below favour being simple and predictable over being clever.
|
||||
|
||||
We support exactly **four** inferred types:
|
||||
|
||||
| Type | Meaning |
|
||||
| --------- | ---------------------------------------------------- |
|
||||
| `number` | Every non-empty value is numeric. |
|
||||
| `boolean` | Every non-empty value is `true`/`false` (any case). |
|
||||
| `date` | Every non-empty value parses as a date. |
|
||||
| `string` | The fallback — anything that isn't one of the above. |
|
||||
|
||||
There is deliberately no integer/float split, no datetime-vs-date distinction,
|
||||
and no JSON type. Those distinctions add branches and edge cases without
|
||||
changing what the user sees. Keep it at four.
|
||||
|
||||
---
|
||||
|
||||
## 2. Inferring one column
|
||||
|
||||
Given the values of a single column, decide its type.
|
||||
|
||||
### The shape of the algorithm
|
||||
|
||||
1. **Drop the empties.** Filter out `null`, `undefined`, and empty/whitespace-only
|
||||
strings before doing anything. Empty cells carry no type signal — a column of
|
||||
numbers with a few blanks is still a number column.
|
||||
2. **All-empty → `string`.** If nothing survives the filter (the column is
|
||||
entirely empty, or there are zero rows), default to `string`. There is no
|
||||
evidence for any other type.
|
||||
3. **Run the type checks in precedence order.** For each candidate type, ask:
|
||||
*does **every** surviving value match this type?* The first candidate for
|
||||
which the answer is yes wins. This is the **"all values match → that type,
|
||||
else fall back"** rule: one stray value that doesn't fit knocks the column
|
||||
down to the next candidate, and ultimately to `string`.
|
||||
|
||||
### Precedence order matters
|
||||
|
||||
The order of the checks is not arbitrary — it exists because the value-sets
|
||||
overlap, and we want the most specific interpretation that fits.
|
||||
|
||||
1. **boolean** first. The strings `"true"`/`"false"` are not numbers and not
|
||||
dates, so booleans never collide with the other checks — but putting them
|
||||
first keeps a `0`/`1`-free true/false column out of `string`. (We do *not*
|
||||
treat `0`/`1` as boolean; that's a number column.)
|
||||
2. **number** second. `Number("2024")` is a perfectly good number, so a column
|
||||
of bare years would read as `number` — which is the honest answer. Numbers
|
||||
are checked before dates so that plain numeric columns never get
|
||||
mis-classified as dates by an over-eager date parser.
|
||||
3. **date** third. Date parsing is the loosest, most permissive check, so it
|
||||
goes last among the positive checks. By the time we reach it we already know
|
||||
the column isn't all-boolean and isn't all-numeric.
|
||||
4. **string** is the fallback when no positive check matches every value.
|
||||
|
||||
> Mnemonic: **boolean → number → date → string**, narrowest evidence to widest.
|
||||
|
||||
### What counts as each type
|
||||
|
||||
- **numeric**: trim the string form; reject empty; `Number(trimmed)` must be
|
||||
finite and not `NaN`. (Native `number` values pass directly.) Reject blank and
|
||||
whitespace so `Number("") === 0` doesn't sneak through.
|
||||
- **boolean**: native `boolean` values pass; otherwise the trimmed,
|
||||
lower-cased string must be exactly `"true"` or `"false"`.
|
||||
- **date**: guard *before* parsing. Require the trimmed value to look
|
||||
date-shaped (a leading `YYYY-MM-DD` or `YYYY/MM/DD`, or `M/D/YYYY`) **and**
|
||||
then confirm `Date.parse` returns a finite timestamp. The shape guard is
|
||||
essential: `Date.parse` will happily accept `"42"` or `"March"` on some
|
||||
engines, which would swallow number and string columns. Never rely on
|
||||
`Date.parse` alone.
|
||||
|
||||
### Sketch
|
||||
|
||||
```ts
|
||||
// src/core/type-inference.ts
|
||||
export type ColumnType = 'number' | 'string' | 'date' | 'boolean';
|
||||
|
||||
const isEmpty = (v: unknown): boolean =>
|
||||
v === null || v === undefined || (typeof v === 'string' && v.trim() === '');
|
||||
|
||||
const isNumeric = (v: unknown): boolean => {
|
||||
if (typeof v === 'number') return Number.isFinite(v);
|
||||
if (typeof v !== 'string') return false;
|
||||
const t = v.trim();
|
||||
if (t === '') return false;
|
||||
const n = Number(t);
|
||||
return !Number.isNaN(n) && Number.isFinite(n);
|
||||
};
|
||||
|
||||
const isBoolean = (v: unknown): boolean => {
|
||||
if (typeof v === 'boolean') return true;
|
||||
if (typeof v !== 'string') return false;
|
||||
const t = v.trim().toLowerCase();
|
||||
return t === 'true' || t === 'false';
|
||||
};
|
||||
|
||||
// Shape guard first, then confirm it actually parses.
|
||||
const DATE_SHAPE = /^\d{4}[-/]\d{2}[-/]\d{2}|^\d{1,2}\/\d{1,2}\/\d{4}/;
|
||||
const isDate = (v: unknown): boolean => {
|
||||
if (typeof v !== 'string') return false;
|
||||
const t = v.trim();
|
||||
return DATE_SHAPE.test(t) && !Number.isNaN(Date.parse(t));
|
||||
};
|
||||
|
||||
/**
|
||||
* Infer one of four column types from a sample of column values.
|
||||
* Empty cells are ignored; an all-empty column is `string`.
|
||||
* Precedence: boolean → number → date → string.
|
||||
*/
|
||||
export function inferColumnType(values: readonly unknown[]): ColumnType {
|
||||
const present = values.filter((v) => !isEmpty(v));
|
||||
if (present.length === 0) return 'string';
|
||||
|
||||
if (present.every(isBoolean)) return 'boolean';
|
||||
if (present.every(isNumeric)) return 'number';
|
||||
if (present.every(isDate)) return 'date';
|
||||
return 'string';
|
||||
}
|
||||
```
|
||||
|
||||
### Robustness notes
|
||||
|
||||
- **Mixed columns** fall through to `string` automatically — the `every` check
|
||||
fails on the first non-conforming value, so a column of mostly-numbers with
|
||||
one label is `string`, which is the safe, honest answer.
|
||||
- **Whitespace** is trimmed in every check, so `" 42 "` reads as numeric and
|
||||
`" "` is treated as empty.
|
||||
- **Empty columns** (all cells blank, or a zero-row dataset) return `string` by
|
||||
the all-empty rule — never throw, never guess.
|
||||
- **Large columns**: see §4. `inferColumnType` itself just consumes whatever
|
||||
array it's handed; the caller decides whether to sample.
|
||||
|
||||
### Do / Don't
|
||||
|
||||
- **Do** ignore empty cells before classifying.
|
||||
- **Do** keep the precedence boolean → number → date → string.
|
||||
- **Do** guard date detection with a shape regex before trusting `Date.parse`.
|
||||
- **Don't** classify a column unless *every* present value matches — one
|
||||
outlier means `string`.
|
||||
- **Don't** add more types (integer, float, datetime, json). Four, no more.
|
||||
- **Don't** let `Number("")`, `Date.parse("42")`, or `0`/`1` leak into the wrong
|
||||
bucket.
|
||||
|
||||
---
|
||||
|
||||
## 3. Profiling a dataset
|
||||
|
||||
A **profile** is the set of derived summary fields stored on a dataset record so
|
||||
the UI can describe it without re-parsing the payload. Per the data model, a
|
||||
profiled dataset carries:
|
||||
|
||||
| Field | Type | Meaning |
|
||||
| ------------- | --------------------------------- | -------------------------------------- |
|
||||
| `rowCount` | `number \| null` | Data rows, or `null` when N/A. |
|
||||
| `columnCount` | `number \| null` | Columns, or `null` when N/A. |
|
||||
| `columns` | `string[]` | Column names, in order. |
|
||||
| `columnTypes` | `Array<{ name; type }>` | Per-column inferred type (see §2). |
|
||||
| `size` | `number` | Approximate payload size in bytes. |
|
||||
|
||||
`null` row/column counts and an empty `columns`/`columnTypes` are how the UI
|
||||
shows **"N/A"** — see §3.2.
|
||||
|
||||
### 3.1 What gets profiled
|
||||
|
||||
Profiling applies only to **tabular inline data**:
|
||||
|
||||
- **JSON** that is an array of objects.
|
||||
- **CSV** (comma-separated, header row).
|
||||
- **TSV** (tab-separated, header row).
|
||||
|
||||
Everything else is **not profiled**:
|
||||
|
||||
- **URL datasets** — the library holds only the link, not the data, so there is
|
||||
nothing to scan. Counts are `null` / N/A.
|
||||
- **Non-tabular data** — a single JSON object, TopoJSON, or anything we can't
|
||||
read as rows-of-columns. Counts are `null` / N/A.
|
||||
|
||||
For the not-profiled cases, `size` is still computed (it's just the byte length
|
||||
of the stored payload), but `rowCount` and `columnCount` are `null`, and
|
||||
`columns`/`columnTypes` are empty.
|
||||
|
||||
### 3.2 The algorithm
|
||||
|
||||
1. **Compute `size`** from the raw payload regardless of whether it's tabular —
|
||||
byte length of the text (CSV/TSV) or of the JSON-serialized value.
|
||||
2. **Decide if it's tabular.** Map `(format, parsed shape)` to a row set:
|
||||
- `csv` / `tsv` → parse into rows-of-objects using the matching delimiter.
|
||||
- `json` that is a non-empty **array of objects** → use it directly.
|
||||
- anything else (`topojson`, a lone JSON object, an empty array) → not
|
||||
tabular; return the N/A profile (`rowCount: null`, `columnCount: null`,
|
||||
`columns: []`, `columnTypes: []`, plus `size`).
|
||||
3. **Derive columns** from the union of keys across the rows (or the CSV/TSV
|
||||
header), preserving first-seen order.
|
||||
4. **Infer each column's type** by collecting that column's values across the
|
||||
rows and calling `inferColumnType` (§2), sampling per §4.
|
||||
5. **Assemble** `rowCount`, `columnCount`, `columns`, `columnTypes`, `size`.
|
||||
|
||||
### Sketch
|
||||
|
||||
```ts
|
||||
// src/core/profile.ts
|
||||
import { inferColumnType, type ColumnType } from './type-inference';
|
||||
|
||||
export interface DatasetProfile {
|
||||
rowCount: number | null;
|
||||
columnCount: number | null;
|
||||
columns: string[];
|
||||
columnTypes: Array<{ name: string; type: ColumnType }>;
|
||||
size: number;
|
||||
}
|
||||
|
||||
const NA = (size: number): DatasetProfile => ({
|
||||
rowCount: null,
|
||||
columnCount: null,
|
||||
columns: [],
|
||||
columnTypes: [],
|
||||
size,
|
||||
});
|
||||
|
||||
/** Profile a dataset payload. `rows` is the tabular form (CSV/TSV/JSON-array)
|
||||
* already parsed to rows-of-objects, or null for non-tabular / URL data. */
|
||||
export function profileData(
|
||||
rows: ReadonlyArray<Record<string, unknown>> | null,
|
||||
size: number,
|
||||
): DatasetProfile {
|
||||
if (!rows || rows.length === 0) return NA(size);
|
||||
|
||||
// Column order = first-seen order across all rows.
|
||||
const columns: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const row of rows) {
|
||||
for (const key of Object.keys(row)) {
|
||||
if (!seen.has(key)) {
|
||||
seen.add(key);
|
||||
columns.push(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (columns.length === 0) return NA(size);
|
||||
|
||||
const sample = sampleRows(rows);
|
||||
const columnTypes = columns.map((name) => ({
|
||||
name,
|
||||
type: inferColumnType(sample.map((r) => r[name])),
|
||||
}));
|
||||
|
||||
return {
|
||||
rowCount: rows.length,
|
||||
columnCount: columns.length,
|
||||
columns,
|
||||
columnTypes,
|
||||
size,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Parsing CSV/TSV text and detecting the payload shape happen *upstream* of
|
||||
`profileData`; this function takes already-parsed rows so it stays pure and
|
||||
trivially testable. The caller passes `null` for URL and non-tabular datasets.
|
||||
|
||||
---
|
||||
|
||||
## 4. Sampling vs. full scan
|
||||
|
||||
`rowCount`/`columnCount`/`size` always reflect the **whole** dataset — they're
|
||||
cheap (a length and a byte count). Only **type inference** has a per-value cost,
|
||||
and it's the one place a huge dataset could hurt.
|
||||
|
||||
So: infer types from a **bounded sample** of rows, not the full column. A fixed
|
||||
cap (e.g. the first ~200 rows) keeps profiling fast and predictable on large
|
||||
pasted datasets while still being more than enough signal to classify a column.
|
||||
|
||||
```ts
|
||||
const SAMPLE_SIZE = 200;
|
||||
const sampleRows = <T>(rows: ReadonlyArray<T>): ReadonlyArray<T> =>
|
||||
rows.length <= SAMPLE_SIZE ? rows : rows.slice(0, SAMPLE_SIZE);
|
||||
```
|
||||
|
||||
Trade-off to be aware of: a column that is numeric for its first 200 rows but
|
||||
turns to text later will be mis-typed as `number`. That's an accepted cost — the
|
||||
type is a display hint, the mistake is cheap, and the speed win on large
|
||||
datasets is worth it. Sampling the head (rather than randomly) keeps results
|
||||
**deterministic**, which matters for tests and for not surprising the user when
|
||||
the same paste profiles the same way twice.
|
||||
|
||||
### Do / Don't
|
||||
|
||||
- **Do** count rows/columns and size over the full payload.
|
||||
- **Do** cap type-inference sampling at a fixed head slice for determinism.
|
||||
- **Don't** randomly sample — non-deterministic profiles break tests and confuse
|
||||
users.
|
||||
- **Don't** scan every value of a million-row paste to guess a type.
|
||||
|
||||
---
|
||||
|
||||
## 5. Testing
|
||||
|
||||
Both functions are pure, so tests are plain input/output assertions in Vitest —
|
||||
no mocks, no DOM, no fixtures beyond literal arrays.
|
||||
|
||||
Cover at least:
|
||||
|
||||
- **`inferColumnType`**: each type detected from a clean column; mixed columns
|
||||
fall to `string`; empty/whitespace cells ignored; all-empty and zero-length →
|
||||
`string`; precedence (a `["true","false"]` column is `boolean` not `string`; a
|
||||
`["2024","2025"]` column is `number` not `date`); date shape guard rejects
|
||||
`"42"` and `"hello"` even though one engine's `Date.parse` might accept them;
|
||||
`0`/`1` are `number`, not `boolean`.
|
||||
- **`profileData`**: a JSON-array dataset profiles fully; `null` rows (URL) and
|
||||
an empty array (non-tabular) return the N/A profile but still carry `size`;
|
||||
column order follows first-seen key order across ragged rows; sampling cap is
|
||||
respected (a dataset longer than the cap still profiles, using only the head).
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
- Four types only: **boolean → number → date → string**, checked in that order.
|
||||
- **All present values must match** a type or the column falls through; empty
|
||||
cells are ignored; an all-empty column is `string`.
|
||||
- Guard date detection with a shape regex before trusting `Date.parse`.
|
||||
- A **profile** carries `rowCount`, `columnCount`, `columns`, `columnTypes`,
|
||||
`size`; URL and non-tabular datasets get a **null/N-A** profile (still sized).
|
||||
- Counts and size scan the whole payload; **type inference samples the head** for
|
||||
speed and determinism.
|
||||
- All of it is **pure `src/core/` logic, unit-tested with Vitest**.
|
||||
@@ -0,0 +1,432 @@
|
||||
# Naming & Relationships
|
||||
|
||||
How Astrolabe keeps entity **names unique** within a collection, and how it
|
||||
tracks the **bidirectional links** between snippets and datasets so they stay
|
||||
consistent as entities are created, imported, and renamed.
|
||||
|
||||
Two concerns live here, and they reinforce each other:
|
||||
|
||||
1. **Name uniqueness** — every dataset has a unique name. Names are the primary
|
||||
key users see and the key snippets reference, so duplicates would be
|
||||
ambiguous. We reject duplicate names on create/rename, and auto-suffix
|
||||
collisions during bulk import.
|
||||
2. **Relationship tracking** — a snippet references datasets *by name* through
|
||||
its `datasetRefs: string[]` field. This is a bidirectional, name-based link:
|
||||
from a snippet you read its refs; from a dataset you scan snippets to find
|
||||
who uses it. Renaming a dataset must propagate to every snippet that points
|
||||
at it, in both the spec and the `datasetRefs` array, or the links rot.
|
||||
|
||||
The hard, testable logic is **pure** and lives in `src/core/`. The parts that
|
||||
read and mutate stores live in `src/app/services/`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why names, not IDs, are the link
|
||||
|
||||
Datasets carry a numeric `id`, but snippets reference them **by name** because
|
||||
that is what Vega-Lite uses: a spec resolves data through a named-data
|
||||
reference, `{ "data": { "name": "MyDataset" } }`. The name *is* the contract
|
||||
between a spec and the dataset library. Storing a numeric id in the spec would
|
||||
mean the spec is no longer a standalone, paste-anywhere Vega-Lite document.
|
||||
|
||||
The consequence: names must be unique (two datasets named `Sales` would make
|
||||
`{ "data": { "name": "Sales" } }` ambiguous), and renaming a dataset is a
|
||||
**graph operation**, not a single field write — every reference to the old name
|
||||
must move with it.
|
||||
|
||||
---
|
||||
|
||||
## 2. Name uniqueness (pure — `src/core/naming.ts`)
|
||||
|
||||
### 2.1 Uniqueness check
|
||||
|
||||
Comparisons are **case-insensitive** (`Sales` and `sales` collide), so a single
|
||||
display name maps to a single dataset regardless of how a user types a
|
||||
reference. The check takes an optional `excludeId` so a rename can ignore the
|
||||
record being renamed (renaming `Sales` to `Sales` is not a collision with
|
||||
itself).
|
||||
|
||||
```ts
|
||||
// src/core/naming.ts
|
||||
|
||||
/** Case-insensitive set of names already in use, minus an optional excluded id. */
|
||||
export function isNameTaken(
|
||||
desired: string,
|
||||
datasets: ReadonlyArray<{ id: number; name: string }>,
|
||||
excludeId?: number,
|
||||
): boolean {
|
||||
const lower = desired.trim().toLowerCase();
|
||||
return datasets.some((d) => d.id !== excludeId && d.name.toLowerCase() === lower);
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Making a unique name
|
||||
|
||||
When a desired name is taken — during import, "extract inline data", or
|
||||
"build chart" — we do **not** overwrite the existing dataset. We derive the
|
||||
next free name by appending a numeric suffix: `Name` → `Name 2` → `Name 3`.
|
||||
The function takes the set of existing names so it has no store dependency and
|
||||
is trivially unit-testable.
|
||||
|
||||
```ts
|
||||
// src/core/naming.ts
|
||||
|
||||
/**
|
||||
* Returns `desired` if free, else the first available `${desired} ${n}` (n >= 2).
|
||||
* `existingNames` is the set of names already in the collection.
|
||||
* Comparison is case-insensitive; the returned name preserves `desired`'s casing.
|
||||
*/
|
||||
export function makeUniqueName(desired: string, existingNames: Iterable<string>): string {
|
||||
const taken = new Set<string>();
|
||||
for (const n of existingNames) taken.add(n.toLowerCase());
|
||||
|
||||
const base = desired.trim();
|
||||
if (!taken.has(base.toLowerCase())) return base;
|
||||
|
||||
let n = 2;
|
||||
while (taken.has(`${base} ${n}`.toLowerCase())) n++;
|
||||
return `${base} ${n}`;
|
||||
}
|
||||
```
|
||||
|
||||
> If a base name already ends in a number (`Q1 2024`), the suffix still appends
|
||||
> (`Q1 2024 2`). That is intentional: we never parse meaning out of the name,
|
||||
> we only guarantee a free slot. Keep this dumb and predictable.
|
||||
|
||||
**Do**
|
||||
|
||||
- Use `isNameTaken` to reject duplicate create/rename in the UI before saving,
|
||||
and surface an error toast.
|
||||
- Use `makeUniqueName` for every non-interactive path (import, extract, build)
|
||||
where blocking the user would be worse than a silent, reported rename.
|
||||
- Pass `excludeId` on rename so an unchanged or case-only edit is allowed.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't compare names case-sensitively anywhere — pick `toLowerCase()` once and
|
||||
use it consistently.
|
||||
- Don't let `makeUniqueName` mutate a store or read store state; it takes plain
|
||||
data and returns a string.
|
||||
|
||||
---
|
||||
|
||||
## 3. The bidirectional snippet ↔ dataset link
|
||||
|
||||
```
|
||||
datasetRefs: ["Sales", "Regions"] (forward, on the snippet)
|
||||
Snippet ───────────────────────────────────────────────────► Dataset "Sales"
|
||||
▲ │
|
||||
└──────────── scan all snippets for "Sales" in datasetRefs ◄──────┘
|
||||
(reverse, derived)
|
||||
```
|
||||
|
||||
- **Forward** (snippet → datasets): read `snippet.datasetRefs`. Cheap, stored.
|
||||
- **Reverse** (dataset → snippets): there is no stored back-pointer. We compute
|
||||
it by scanning snippets. Keeping it *derived* means it can never disagree with
|
||||
the forward links — there is one source of truth.
|
||||
|
||||
`datasetRefs` is **derived from the spec**, not hand-maintained. It is
|
||||
recomputed whenever a snippet is published (its draft spec is promoted), so it
|
||||
always mirrors the dataset names actually referenced in the published spec.
|
||||
|
||||
### 3.1 Extracting referenced names from a spec (pure — `src/core/spec-refs.ts`)
|
||||
|
||||
A Vega-Lite spec can reference named data in several places: the top-level
|
||||
`data`, per-layer `data`, `data` inside `spec`/`facet`/`hconcat`/`vconcat`, and
|
||||
named entries in top-level `datasets`. Rather than enumerate Vega-Lite's grammar,
|
||||
we walk the spec recursively and collect every `{ data: { name } }` we find.
|
||||
This is pure, deterministic, and the most heavily unit-tested function here.
|
||||
|
||||
```ts
|
||||
// src/core/spec-refs.ts
|
||||
|
||||
type Json = unknown;
|
||||
|
||||
/** Collects every dataset name referenced by `{ data: { name } }` anywhere in the spec. */
|
||||
export function extractDatasetRefs(spec: Json): string[] {
|
||||
const names = new Set<string>();
|
||||
|
||||
const walk = (node: Json): void => {
|
||||
if (Array.isArray(node)) {
|
||||
for (const item of node) walk(item);
|
||||
return;
|
||||
}
|
||||
if (node && typeof node === 'object') {
|
||||
const obj = node as Record<string, Json>;
|
||||
const data = obj.data as Record<string, Json> | undefined;
|
||||
if (data && typeof data === 'object' && typeof data.name === 'string') {
|
||||
names.add(data.name);
|
||||
}
|
||||
for (const key of Object.keys(obj)) walk(obj[key]);
|
||||
}
|
||||
};
|
||||
|
||||
walk(typeof spec === 'string' ? safeParse(spec) : spec);
|
||||
return [...names];
|
||||
}
|
||||
|
||||
function safeParse(s: string): Json {
|
||||
try {
|
||||
return JSON.parse(s);
|
||||
} catch {
|
||||
return null; // an unparseable draft simply has no resolvable refs
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```ts
|
||||
// src/core/spec-refs.ts — thin wrapper used at publish time
|
||||
|
||||
/** The list stored on snippet.datasetRefs. Sorted + de-duped for stable diffs. */
|
||||
export function recomputeDatasetRefs(spec: Json): string[] {
|
||||
return extractDatasetRefs(spec).sort();
|
||||
}
|
||||
```
|
||||
|
||||
> A `spec` may be an object or a string (see the Data Model). Normalize once,
|
||||
> at the boundary, so the recursive walk never has to care.
|
||||
|
||||
**Do**
|
||||
|
||||
- Treat `extractDatasetRefs` as the single source of truth for "what does this
|
||||
spec reference". The reverse-lookup and rename paths both depend on it
|
||||
agreeing with what the renderer actually resolves.
|
||||
- Recompute and store `datasetRefs` on **publish**, not on every keystroke —
|
||||
the draft can be transiently invalid, and only the published spec is shared.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't let two code paths each have their own idea of "referenced names".
|
||||
Renamer and ref-recomputer must use the same extractor.
|
||||
|
||||
---
|
||||
|
||||
## 4. Reverse lookup: who uses this dataset?
|
||||
|
||||
The Dataset Manager shows a **usage badge** and a **Linked Snippets** list; the
|
||||
Snippet Library shows a snippet's linked datasets. Both come from one selector
|
||||
scan — no stored back-pointer to drift.
|
||||
|
||||
```ts
|
||||
// src/app/services/RelationshipService.ts
|
||||
|
||||
import { useSnippetStore } from '../stores/SnippetStore';
|
||||
import type { Snippet } from '../../core/types';
|
||||
|
||||
/** Snippets whose datasetRefs include `name` (case-insensitive). */
|
||||
export function findSnippetsReferencingDataset(name: string): Snippet[] {
|
||||
const lower = name.toLowerCase();
|
||||
return useSnippetStore.getState().snippets.filter((s) =>
|
||||
s.datasetRefs.some((ref) => ref.toLowerCase() === lower),
|
||||
);
|
||||
}
|
||||
|
||||
/** Count for the usage badge. */
|
||||
export function datasetUsageCount(name: string): number {
|
||||
return findSnippetsReferencingDataset(name).length;
|
||||
}
|
||||
```
|
||||
|
||||
Because this reads `useSnippetStore.getState().snippets`, exposing it as a
|
||||
selector for the UI makes the badge and Linked Snippets list reactive for free —
|
||||
they update the moment any snippet is published with changed refs.
|
||||
|
||||
**Do**
|
||||
|
||||
- Keep reverse lookup a pure scan over the store. It is O(snippets) but the
|
||||
collections are small (library budget ~5 MB); clarity beats an index.
|
||||
- Expose it as a selector where the UI needs reactivity.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't add a `referencedBy` array to datasets. A stored reverse pointer is a
|
||||
second source of truth that *will* fall out of sync with `datasetRefs`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Import: auto-suffix collisions, then report
|
||||
|
||||
On import we never overwrite an existing dataset. A dataset whose name collides
|
||||
is renamed to a unique name via `makeUniqueName`, and **every rename is
|
||||
collected and reported to the user** (toast / summary) so the change is never
|
||||
silent. Crucially, names are reserved *as we go* — within a single import, two
|
||||
incoming `Sales` datasets become `Sales 2` and `Sales 3`, not two `Sales 2`.
|
||||
|
||||
```ts
|
||||
// src/app/services/ImportService.ts
|
||||
|
||||
import { makeUniqueName } from '../../core/naming';
|
||||
import type { Dataset } from '../../core/types';
|
||||
|
||||
export interface DatasetRename {
|
||||
from: string;
|
||||
to: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns incoming datasets with collision-free names, plus the renames applied.
|
||||
* `existing` are names already in the library; `incoming` are datasets to add.
|
||||
*/
|
||||
export function dedupeIncomingDatasetNames(
|
||||
existing: ReadonlyArray<string>,
|
||||
incoming: ReadonlyArray<Dataset>,
|
||||
): { datasets: Dataset[]; renames: DatasetRename[] } {
|
||||
const reserved = new Set(existing.map((n) => n.toLowerCase()));
|
||||
const renames: DatasetRename[] = [];
|
||||
|
||||
const datasets = incoming.map((d) => {
|
||||
const unique = makeUniqueName(d.name, reserved);
|
||||
reserved.add(unique.toLowerCase()); // reserve so later imports don't collide
|
||||
if (unique !== d.name) renames.push({ from: d.name, to: unique });
|
||||
return unique === d.name ? d : { ...d, name: unique };
|
||||
});
|
||||
|
||||
return { datasets, renames };
|
||||
}
|
||||
```
|
||||
|
||||
> If imported snippets reference the renamed dataset, their `datasetRefs` and
|
||||
> specs must be rewritten to the new name too — reuse the rename machinery in
|
||||
> §6 over the imported snippet set, or run `renameDatasetEverywhere` per applied
|
||||
> rename after the import is committed.
|
||||
|
||||
**Do**
|
||||
|
||||
- Reserve each chosen name immediately so collisions *within* one import are
|
||||
also resolved.
|
||||
- Return the rename list and show it; a silent rename looks like data loss.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't overwrite or merge a same-named existing dataset on import. Suffix and
|
||||
keep both — the user decides what to delete.
|
||||
|
||||
---
|
||||
|
||||
## 6. Rename propagation: keep the link consistent
|
||||
|
||||
Renaming a dataset is the operation that ties §2–§5 together. A rename must, in
|
||||
one atomic step:
|
||||
|
||||
1. Update the dataset's own `name`.
|
||||
2. For **every snippet referencing the old name**: rewrite the named-data
|
||||
references inside its spec (`{ "data": { "name": "old" } }` →
|
||||
`{ "data": { "name": "new" } }`) — in **both** `spec` and `draftSpec`.
|
||||
3. Recompute that snippet's `datasetRefs` from the rewritten spec, so the
|
||||
forward link mirrors reality and the reverse scan stays correct.
|
||||
|
||||
The spec rewrite is pure; the orchestration reads and writes stores.
|
||||
|
||||
```ts
|
||||
// src/core/spec-refs.ts — pure rewrite
|
||||
|
||||
/** Returns a copy of `spec` with every data.name === oldName replaced by newName. */
|
||||
export function renameDatasetInSpec(spec: Json, oldName: string, newName: string): Json {
|
||||
const obj = typeof spec === 'string' ? safeParse(spec) : spec;
|
||||
|
||||
const rewrite = (node: Json): Json => {
|
||||
if (Array.isArray(node)) return node.map(rewrite);
|
||||
if (node && typeof node === 'object') {
|
||||
const out: Record<string, Json> = {};
|
||||
for (const [k, v] of Object.entries(node as Record<string, Json>)) {
|
||||
if (
|
||||
k === 'data' &&
|
||||
v && typeof v === 'object' &&
|
||||
(v as Record<string, Json>).name === oldName
|
||||
) {
|
||||
out[k] = { ...(v as object), name: newName };
|
||||
} else {
|
||||
out[k] = rewrite(v);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
return node;
|
||||
};
|
||||
|
||||
const rewritten = rewrite(obj);
|
||||
// Preserve the original spec's stored shape (string vs object).
|
||||
return typeof spec === 'string' ? JSON.stringify(rewritten, null, 2) : rewritten;
|
||||
}
|
||||
```
|
||||
|
||||
```ts
|
||||
// src/app/services/RelationshipService.ts — store coordination
|
||||
|
||||
import { isNameTaken, makeUniqueName } from '../../core/naming';
|
||||
import { renameDatasetInSpec, recomputeDatasetRefs } from '../../core/spec-refs';
|
||||
import { useDatasetStore } from '../stores/DatasetStore';
|
||||
import { useSnippetStore } from '../stores/SnippetStore';
|
||||
|
||||
/**
|
||||
* Renames a dataset and propagates the rename to every referencing snippet
|
||||
* (spec, draftSpec, and datasetRefs). Returns the snippets that changed.
|
||||
* Caller is responsible for collision policy on `newName` (reject vs suffix).
|
||||
*/
|
||||
export function renameDatasetEverywhere(oldName: string, newName: string): { updated: number } {
|
||||
if (oldName === newName) return { updated: 0 };
|
||||
|
||||
// 1. Rename the dataset record itself.
|
||||
const dataset = useDatasetStore.getState().datasets.find((d) => d.name === oldName);
|
||||
if (!dataset) return { updated: 0 };
|
||||
useDatasetStore.getState().update(dataset.id, { name: newName });
|
||||
|
||||
// 2 + 3. Rewrite every referencing snippet's specs and refs.
|
||||
let updated = 0;
|
||||
for (const snippet of useSnippetStore.getState().snippets) {
|
||||
if (!snippet.datasetRefs.some((r) => r.toLowerCase() === oldName.toLowerCase())) continue;
|
||||
|
||||
const spec = renameDatasetInSpec(snippet.spec, oldName, newName);
|
||||
const draftSpec = renameDatasetInSpec(snippet.draftSpec, oldName, newName);
|
||||
useSnippetStore.getState().update(snippet.id, {
|
||||
spec,
|
||||
draftSpec,
|
||||
datasetRefs: recomputeDatasetRefs(spec),
|
||||
});
|
||||
updated++;
|
||||
}
|
||||
|
||||
return { updated };
|
||||
}
|
||||
```
|
||||
|
||||
> **Collision on rename.** The UI rename form rejects a name already in use via
|
||||
> `isNameTaken(newName, datasets, dataset.id)`. Programmatic renames (e.g. an
|
||||
> import flow) instead resolve with `makeUniqueName` before calling
|
||||
> `renameDatasetEverywhere`. The propagation function itself does not invent a
|
||||
> name — it assumes `newName` is the agreed target.
|
||||
|
||||
**Do**
|
||||
|
||||
- Rewrite `spec` **and** `draftSpec`. A user mid-edit must not see their draft
|
||||
silently break because the dataset was renamed underneath them.
|
||||
- Recompute `datasetRefs` from the rewritten spec rather than string-replacing
|
||||
the array — the spec is the source of truth, the array is its mirror.
|
||||
- Use the §4 reverse lookup to find affected snippets, so "who references this"
|
||||
has exactly one implementation.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Don't update `datasetRefs` without also rewriting the spec — the rendered
|
||||
named-data reference would still point at the old, now-missing name.
|
||||
- Don't rename the dataset and skip propagation "for now". A half-applied rename
|
||||
is the exact inconsistency this whole document exists to prevent.
|
||||
|
||||
---
|
||||
|
||||
## 7. Where things live
|
||||
|
||||
| Concern | Location | Pure? | Tested |
|
||||
|---|---|---|---|
|
||||
| `makeUniqueName`, `isNameTaken` | `src/core/naming.ts` | yes | unit |
|
||||
| `extractDatasetRefs`, `recomputeDatasetRefs` | `src/core/spec-refs.ts` | yes | unit |
|
||||
| `renameDatasetInSpec` | `src/core/spec-refs.ts` | yes | unit |
|
||||
| `findSnippetsReferencingDataset`, usage count | `src/app/services/RelationshipService.ts` | no (reads store) | integration |
|
||||
| `renameDatasetEverywhere` | `src/app/services/RelationshipService.ts` | no (mutates stores) | integration |
|
||||
| `dedupeIncomingDatasetNames` | `src/app/services/ImportService.ts` | nearly (uses `makeUniqueName`) | unit/integration |
|
||||
|
||||
The dividing line: anything that takes plain data and returns plain data is
|
||||
**core** and unit-tested in isolation; anything that reaches into a Zustand store
|
||||
is an **app service**. The rename rule of thumb — *the spec is the source of
|
||||
truth, `datasetRefs` mirrors it, the reverse lookup is derived* — is what keeps
|
||||
the bidirectional link from ever needing manual repair.
|
||||
@@ -0,0 +1,286 @@
|
||||
# 08 · Borrowed Techniques from vega/editor
|
||||
|
||||
> The official Vega-Lite editor ([vega/editor](https://github.com/vega/editor)) solves the
|
||||
> exact "edit a Vega-Lite spec as JSON, validate it, render it live" problem Astrolabe sits
|
||||
> on top of — minus the snippet/dataset library. This doc distills the techniques worth
|
||||
> borrowing and the gotchas worth avoiding, so we don't rediscover them from scratch in
|
||||
> M1/M2.
|
||||
>
|
||||
> It is a **reference**, not a contract. The behavioral contract is still [`docs/spec/`](../spec/);
|
||||
> the patterns are still docs [01](01-state-and-stores.md)–[07](07-naming-and-relationships.md).
|
||||
> This doc is the bridge: "here is how the canonical implementation does the editor/renderer
|
||||
> plumbing, and here is what we keep vs. improve."
|
||||
|
||||
## Source of these findings
|
||||
|
||||
A read-only clone of vega/editor lives at `/Users/oleh/code/reference/vega-editor` (shallow
|
||||
clone of `main`, HEAD `4fdbb59`). Re-clone with
|
||||
`git clone --depth 1 https://github.com/vega/editor`. Citations below are `file:line` into
|
||||
that tree.
|
||||
|
||||
## Stack delta (read this first — it changes how directly we can borrow)
|
||||
|
||||
| | vega/editor | Astrolabe |
|
||||
|---|---|---|
|
||||
| UI framework | **React** | **React** (moved off Preact before build start) |
|
||||
| State | Redux-ish single `State` in React context (`useState`) | Zustand **stores** — *not* Redux |
|
||||
| Monaco | `@monaco-editor/react` + `@monaco-editor/loader` (CDN-loaded Monaco, **workers auto-wired**) | **raw `monaco-editor`** via Vite (**we must wire workers ourselves**) |
|
||||
| Rendering | **hand-rolled** `vegaLite.compile` → `vega.parse` → `new vega.View().runAsync()` | **`vegaEmbed()`** (wraps that same pipeline) |
|
||||
| Schema validation | Monaco JSON worker **+** standalone `ajv ^8` (two independent layers) | same two-layer model planned |
|
||||
|
||||
Because both apps are now React, vega/editor's **component lifecycle patterns port more or
|
||||
less directly** — the friction is only in (a) state (their Redux-flat-state → our Zustand
|
||||
stores) and (b) Monaco worker wiring (their CDN loader → our explicit Vite workers).
|
||||
|
||||
---
|
||||
|
||||
## Decision · Monaco integration (self-hosted, raw API)
|
||||
|
||||
> **Decided.** Astrolabe uses **raw `monaco-editor` from npm, bundled and self-hosted**, with
|
||||
> workers wired explicitly via Vite `?worker` — **not** vega/editor's
|
||||
> `@monaco-editor/react` + `@monaco-editor/loader` (CDN) setup. Two independent axes:
|
||||
|
||||
**Axis A — sourcing: self-hosted/bundled, not CDN. (Forced by Astrolabe's values.)**
|
||||
vega/editor's `@monaco-editor/loader` fetches Monaco's AMD bundle from a CDN at runtime. For
|
||||
us that breaks three things at once: (1) **offline** — the CDN bundle is outside Vite's module
|
||||
graph, so `vite-plugin-pwa`/Workbox never precaches it and offline silently fails; bundled npm
|
||||
assets are hashed files in `dist/` that Workbox precaches automatically; (2) **privacy** — a
|
||||
third-party fetch on load contradicts SOUL's "the only outbound requests are user-created
|
||||
URL-dataset fetches"; (3) **determinism** — npm + `package-lock` is integrity-pinned and
|
||||
reproducible, a runtime CDN resolve is not. This axis is not a close call; vega/editor's CDN
|
||||
choice is right *for an online hosted tool* and wrong for an offline, installable, private app.
|
||||
|
||||
**Axis B — React integration: raw API, not `@monaco-editor/react`. (A lean, not forced.)**
|
||||
The wrapper helps with the easy 80% (mount a JSON editor, lifecycle) and adds nothing to the
|
||||
load-bearing 20% this app needs:
|
||||
- **Workers** are still ours — the wrapper never manages `MonacoEnvironment` (see §1 gotcha).
|
||||
- The **M2 schema service** (`jsonDefaults.setDiagnosticsOptions`, `fileMatch`) is namespace-level;
|
||||
you reach *through* the wrapper via `onMount`, so it saves nothing there.
|
||||
- Its headline **`value`/`onChange` controlled-input model is a hazard**: driving Monaco's
|
||||
content from React state causes cursor jumps and undo-stack churn, against §10's "typing
|
||||
stays fluid" — you end up using it uncontrolled, i.e. the raw pattern anyway.
|
||||
- Its **CDN-by-default** is a standing footgun (works in dev online, fails offline in prod
|
||||
unless you remember `loader.config({ monaco })`).
|
||||
|
||||
Against that, raw costs **one testable `useMonacoEditor` hook** (~50–80 lines: create in
|
||||
`useEffect`, `dispose` on unmount, push value, subscribe to `onDidChangeModelContent`, resize).
|
||||
That's the **same imperative-teardown discipline already adopted for `vega-embed`** in doc 05
|
||||
(`view.finalize()`), and consistent with already using raw `vegaEmbed()` over a React chart
|
||||
wrapper — "thin integration layers we own" (SOUL). Lock-in is low either way, so the final
|
||||
raw-vs-wrapper call is confirmable at the Monaco spike; what is **not** up for revisiting is
|
||||
self-hosting.
|
||||
|
||||
**Accepted cost:** the explicit worker wiring (§1) is inherent to self-hosting — it is the
|
||||
price of offline, paid in any non-CDN setup, and the wrapper would not remove it.
|
||||
|
||||
---
|
||||
|
||||
> **The single biggest surprise:** vega/editor does **not** use `vega-embed` for its live
|
||||
> preview. It builds the compile→parse→View pipeline by hand; `vega-embed` is imported only
|
||||
> for types and the exported standalone-HTML snippet. This is *good news* — `vega-embed` is
|
||||
> exactly the wrapper they wrote by hand, so we get it for free. But their hand-rolled
|
||||
> version (`src/components/renderer/renderer.tsx`) is the best available documentation of the
|
||||
> lifecycle/cleanup discipline `vega-embed` still expects from us.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Monaco + Vega-Lite schema wiring (M2 — highest from-scratch risk)
|
||||
|
||||
All of vega/editor's Monaco setup is one file: `src/utils/monaco.ts`.
|
||||
|
||||
**What to borrow:**
|
||||
|
||||
- **Bundle the schema; never fetch it.** They `import vegaLiteSchema from 'vega-lite/vega-lite-schema.json'`,
|
||||
resolved by a Vite alias to the package's `build/` output (`monaco.ts:7-8`, `vite.config.ts`).
|
||||
The schema version is pinned to the installed `vega-lite` — offline-safe, version-locked,
|
||||
no runtime network call. Astrolabe should do the same.
|
||||
- **Attach via the JSON language service**, once, globally:
|
||||
`monaco.languages.json.jsonDefaults.setDiagnosticsOptions({ schemas, validate:true, ... })`
|
||||
(`monaco.ts:51-57`).
|
||||
- **`markdownDescription` patch** (`monaco.ts:12-13`, `utils/markdownProps.ts`): recursively
|
||||
copy every schema `description` → `markdownDescription` before registering. Monaco renders
|
||||
rich hover docs only from `markdownDescription`; without this, hovers are plain text. Do it
|
||||
once at setup.
|
||||
- **Replace the built-in JSON formatter** with `json-stringify-pretty-compact` via
|
||||
`registerDocumentFormattingEditProvider('json', …)` (`monaco.ts:60-61,71-80`) for Vega's
|
||||
compact array-on-one-line style.
|
||||
- **Editor options worth copying** (`spec-editor/renderer.tsx:263-274`): `folding:true`,
|
||||
`minimap.enabled:false`, `scrollBeyondLastLine:false`, `wordWrap:'on'`,
|
||||
`quickSuggestions:true` (this is what makes schema completions appear without an explicit
|
||||
trigger), `stickyScroll.enabled:false`.
|
||||
|
||||
**Gotchas / where we improve:**
|
||||
|
||||
- ⚠️ **Workers are on us.** vega/editor never configures Monaco workers — the CDN loader does.
|
||||
With raw `monaco-editor` + Vite we **must** set `self.MonacoEnvironment.getWorker` to return
|
||||
the `json.worker` for label `'json'` and `editor.worker` otherwise (via `?worker` imports):
|
||||
```ts
|
||||
import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
|
||||
import JsonWorker from 'monaco-editor/esm/vs/language/json/json.worker?worker';
|
||||
self.MonacoEnvironment = {
|
||||
getWorker: (_id, label) => (label === 'json' ? new JsonWorker() : new EditorWorker()),
|
||||
};
|
||||
```
|
||||
The `json.worker` runs schema validation + autocomplete. **No worker ⇒ no squiggles, no
|
||||
completions.** Upside: dropping the CDN loader makes `monaco` synchronously importable — no
|
||||
`await loader.init()` dance, just call `setDiagnosticsOptions(...)` at module load.
|
||||
- ⚠️ **`$schema`-based binding vs `fileMatch`.** They register schemas under versioned `uri`s
|
||||
(`.../vega-lite/v6.json`) and bind by matching the doc's `$schema` value — **no `fileMatch`**
|
||||
(`monaco.ts:15-46`). Consequence: a spec with **no `$schema` gets zero validation/autocomplete.**
|
||||
Astrolabe should prefer `fileMatch` against our model URIs so validation works regardless of
|
||||
whether the user wrote a `$schema` line.
|
||||
- ⚠️ **Set `enableSchemaRequest:false`** for our offline-first app. They set it `true`
|
||||
(`monaco.ts:54`), which lets the worker network-fetch any unbundled `$schema` URL — failing
|
||||
network calls for an offline app. Register all schema versions locally instead.
|
||||
- The schema is multi-MB; register it **once globally**, never per-model.
|
||||
|
||||
## 2 · Live preview with `vega-embed` (M1 lifecycle, M2 fit-mode)
|
||||
|
||||
This is doc [05](05-rendering-theming-preview.md)'s territory; these are the concrete details
|
||||
vega/editor's hand-rolled renderer (`src/components/renderer/renderer.tsx`) reveals.
|
||||
|
||||
**What to borrow:**
|
||||
|
||||
- **Theme = a `vega-themes` config object merged into the spec config.** There is no automatic
|
||||
light/dark sync in vega/editor — theme is an explicit choice baked in at compile
|
||||
(`config-editor/config-editor-header.tsx:5-37`). For Astrolabe: pass the chosen `theme`/`config`
|
||||
to `vegaEmbed`, and when our theme changes, re-embed with the new config.
|
||||
- **`"width":"container"` / `"height":"container"` is how VL responsiveness works** — it
|
||||
compiles to a `containerSize` signal (`renderer.tsx:78-90` detects this). Pair it with a
|
||||
**`ResizeObserver`** on the preview pane → `view.resize().runAsync()`. This is cleaner than
|
||||
vega/editor's `window.dispatchEvent(new Event('resize'))` hack (`renderer.tsx:101-122`) and is
|
||||
the mechanism behind our M2 fit-mode contract.
|
||||
- **Reuse the view for cheap changes.** They rebuild the `View` only on spec change; renderer
|
||||
(svg/canvas) and tooltip toggles re-`initialize()` the existing view (`renderer.tsx:367-371`).
|
||||
- **Capture warnings separately from errors** via a buffering logger (see §4's `LocalLogger`).
|
||||
|
||||
**Gotchas / where we improve:**
|
||||
|
||||
- ⚠️ **Finalize before re-embed, or leak.** Every spec change must `view.finalize()` the old
|
||||
view *and* clear the container before mounting the new one (`renderer.tsx:218-226`). `vegaEmbed`
|
||||
returns `{ view, finalize }` — call `finalize()` before the next embed and on unmount. This is
|
||||
already a Do-rule in doc 05; vega/editor confirms how easy it is to leak otherwise.
|
||||
- ⚠️ **Race on rapid edits.** `runAsync` is async; a stale render can resolve after a newer one
|
||||
mounts. vega/editor mitigates only with debounce. **We should add a render-generation token**
|
||||
and ignore stale resolves (an improvement over the reference).
|
||||
- ⚠️ **Wrap `runAsync` in try/catch and finalize on failure** — Vega won't catch runtime errors
|
||||
for you, and a half-initialized view leaks if you don't finalize (`renderer.tsx:247-259`).
|
||||
- The **CSP-safe expression interpreter** (`vega-interpreter` + `vega.parse(..., {ast:true})`)
|
||||
matters only under a strict no-`eval` CSP. A local offline app doesn't need it — keep it
|
||||
opt-in.
|
||||
|
||||
## 3 · Two-tier validation & error surfacing (M2, spec §03E)
|
||||
|
||||
vega/editor runs **two independent schema-validation systems** with no reconciliation, and
|
||||
sorts errors into two tiers. Both are worth copying.
|
||||
|
||||
**The two layers:**
|
||||
|
||||
1. **Monaco JSON worker** → inline **squiggles, hovers, autocomplete** in the editor.
|
||||
2. **`ajv ^8`** (`src/utils/validate.ts`) → runs at parse time, feeds the **error/log pane**.
|
||||
It does *not* create editor markers.
|
||||
|
||||
**The two error tiers (keep them separate):**
|
||||
|
||||
- **Fatal / blocking** — thrown exceptions: JSON syntax error, VL compile error, Vega runtime
|
||||
error. These set a single `error` and suppress the chart.
|
||||
- **Advisory** — ajv schema-validation findings and `$schema` version mismatch. These are a
|
||||
warnings list and do **not** block rendering. (Vega-Lite emits many benign warnings; treating
|
||||
ajv output as fatal would wrongly hide specs that render fine.)
|
||||
|
||||
The orchestration is `app.tsx:188-291`: `parseJSONCOrThrow` → `$schema` semver check (warn) →
|
||||
`validateVegaLite` (ajv, warn) → `vegaLite.compile` (throw=fatal) → render (`renderer.tsx`,
|
||||
throw=fatal).
|
||||
|
||||
**ajv setup specifics that *will* bite a from-scratch impl** (`validate.ts:9-17`):
|
||||
|
||||
- `new Ajv({ strict: false })` — the VL/Vega schemas fail ajv strict-mode at **compile** time
|
||||
otherwise.
|
||||
- The VL schema is **draft-06** → must `ajv.addMetaSchema(json-schema-draft-06.json)` (ajv 8
|
||||
defaults to draft-07/2020) or `compile` throws.
|
||||
- Register a no-op `color-hex` format (`ajv.addFormat('color-hex', () => true)`) plus
|
||||
`addFormats(ajv)`; the schema references formats ajv-formats doesn't cover.
|
||||
- **Compile the validator once at module load and cache it** — the schema is huge; compiling
|
||||
per keystroke is a perf killer.
|
||||
|
||||
**Where we improve:** ajv errors are shown as JSON-pointer text (e.g. `/encoding/x`) with **no
|
||||
editor position** — vega/editor does not map them to markers. Only JSON *syntax* errors get a
|
||||
line/col (via jsonc-parser's visitor, `utils/jsonc-parser.ts:3-17`). If our §03E wants inline
|
||||
ajv markers, we map `instancePath` → editor offsets ourselves via jsonc-parser's node tree —
|
||||
something the reference does *not* do.
|
||||
|
||||
## 4 · Data flow & debouncing (M1/M2 — translate to Zustand stores)
|
||||
|
||||
vega/editor keeps **`editorString` (the text) as the single source of truth**; the parsed spec
|
||||
and compiled Vega spec are *derived* and recomputed by a subscriber when text/mode/config change
|
||||
(`app.tsx:338-365`). Errors don't clobber the last-good derived specs.
|
||||
|
||||
**The Zustand-store translation (this is the shape to build):**
|
||||
|
||||
```
|
||||
text (store field, debounced writer on editor change)
|
||||
└─▶ parsedSpec (derived: JSONC parse + collect syntax/diagnostic errors)
|
||||
└─▶ renderInput (derived: prepareSpecForRender — refs, fit-mode)
|
||||
└─▶ effect: deep-equal guard → vegaEmbed(); finalize previous view
|
||||
```
|
||||
|
||||
**What to borrow:**
|
||||
|
||||
- **Debounce only at edit→state**, not state→render. vega/editor debounces the editor at
|
||||
**1200 ms** (`spec-editor/renderer.tsx:66`) and guards the render with a `deepEqual` prop
|
||||
diff (`renderer.tsx:340-349`). (1200 ms is *their* number; tune ours — our settings expose a
|
||||
render-debounce preference.)
|
||||
- **A manual-parse escape hatch** (Ctrl/Cmd+S re-parses without waiting) maps to a future
|
||||
live-vs-manual preview toggle (`renderer.tsx:89-111`).
|
||||
- **`LocalLogger` pattern** (`utils/logger.ts`): a logger that buffers `errors/warns/infos/debugs`
|
||||
into arrays instead of writing to console. This lets a **pure** `src/core` compile/validate
|
||||
step *return* structured diagnostics with zero browser coupling — e.g.
|
||||
`validateSpec(spec) → { errors, warns }`. Ideal core-first fit.
|
||||
- **`json-stringify-pretty-compact`** for the format action and prettify-on-load — much nicer
|
||||
than `JSON.stringify(…, null, 2)` for VL specs.
|
||||
|
||||
**Persistence note:** vega/editor snapshots its whole state to localStorage on *every* change,
|
||||
stripping non-serializable fields (`view`, `runtime`, editor refs) and restoring via
|
||||
`{ ...DEFAULT_STATE, ...parsed }` (`context/app-context.tsx`). Our **debounced auto-save to
|
||||
IndexedDB** (doc 01/02) is the better pattern — but the "strip non-serializable, restore with
|
||||
defaults-spread" discipline is worth keeping.
|
||||
|
||||
---
|
||||
|
||||
## Borrow list (where each lands)
|
||||
|
||||
| Technique | Lands in | Milestone |
|
||||
|---|---|---|
|
||||
| Bundle VL schema from package `build/`; `setDiagnosticsOptions` | `src/app/infrastructure/` Monaco setup | M2 |
|
||||
| `markdownDescription` patch + compact formatter | Monaco setup | M2 |
|
||||
| Explicit Vite worker wiring (`MonacoEnvironment.getWorker`) | Monaco setup | M2 |
|
||||
| `fileMatch` schema binding (improvement over `$schema`-only) | Monaco setup | M2 |
|
||||
| jsonc-parser tolerant parse + line/col syntax errors | `src/core/` | M1/M2 |
|
||||
| ajv wrapper (`strict:false`, draft-06, color-hex, compile-once) → structured diagnostics | `src/core/` | M2 |
|
||||
| `LocalLogger`-style buffered diagnostics from pure compile | `src/core/` | M2 |
|
||||
| Fatal-vs-advisory two-tier error model | rendering/store contract | M1/M2 |
|
||||
| `"container"` sizing + `ResizeObserver` → `view.resize()` | `rendering.ts` + LivePreview | M2 |
|
||||
| `finalize()`-before-reembed + **render-generation guard** | LivePreview | M1 |
|
||||
| theme = `vega-themes` config merged into `vegaEmbed` | preview + settings | M5 |
|
||||
| `json-stringify-pretty-compact` format action | editor | M2 |
|
||||
|
||||
## Where we deliberately do better than the reference
|
||||
|
||||
- **Wire Monaco workers explicitly** (they sidestep it via the CDN loader).
|
||||
- **Map ajv errors to editor positions** via jsonc-parser offsets (they show pointer text only).
|
||||
- **Render-generation guard** against stale async renders (they rely on debounce alone).
|
||||
- **`fileMatch`-based schema binding** so validation works without a `$schema` line.
|
||||
- **Debounced auto-save to IndexedDB** rather than write-the-whole-state-on-every-change.
|
||||
|
||||
## Key files in the reference (for deeper reads)
|
||||
|
||||
- `src/utils/monaco.ts` — all Monaco/schema wiring
|
||||
- `src/utils/markdownProps.ts` — the `markdownDescription` patch
|
||||
- `src/utils/validate.ts` — ajv setup + cached validators
|
||||
- `src/utils/jsonc-parser.ts` — tolerant parse + line/col syntax errors
|
||||
- `src/utils/logger.ts` — `LocalLogger` / `DispatchingLogger`
|
||||
- `src/components/renderer/renderer.tsx` — the hand-rolled View lifecycle (finalize, sizing, errors)
|
||||
- `src/components/app.tsx:188-365` — parse → $schema check → ajv → compile → render orchestration
|
||||
- `src/components/error-pane/renderer.tsx` — error/log display
|
||||
- `src/constants/default-state.ts` — the full app-state shape
|
||||
- `src/components/input-panel/spec-editor/renderer.tsx` — editor component, 1200ms debounce, $schema→mode detection
|
||||
Reference in New Issue
Block a user