astrolabe/docs/IMPLEMENTATION-PLAN.md

Astrolabe — Incremental Implementation Plan

A spec-driven rebuild of Astrolabe on Syto's architecture. The authoritative behavioral contract is docs/spec/ (sections 00–10). This document sequences the build into the quickest path to a usable MVP, then layers the rest.

Method per milestone: build core-first (portable, pure, tested) → wire UI → cover with tests → manual smoke check against the spec's acceptance points. "Test the Core, Trust the UI": high coverage on src/core/, lighter on components.

---

Architectural ground rules

These are decided and apply to every milestone. The how behind each is written up self-containedly in docs/architecture/ — read the matching doc before implementing.

• src/core/ is portable — no browser APIs, no React, no Monaco. Pure spec operations (detection, profiling, reference resolution, fit transforms, validation, import normalization). This is what we test hardest and what could power a future headless renderer/CLI.
• src/app/ holds React + Zustand UI. State lives in Zustand stores (useAppStore, plus per-feature stores); browser specifics live in src/app/infrastructure/ adapters (IndexedDB, localStorage, URL hash) so the rest of the app never touches window/indexedDB directly.
• Modals via a registry + coordinator + shell (see Architecture 03), not ad-hoc conditional rendering.
• CSS Modules + design tokens (styles/tokens.css); themes flip [data-theme]. Vega theme follows the UI theme. The design language behind the tokens — type, spacing, color roles, components, themes — is defined in Architecture 09 and established in M1.5.
• Editor: Monaco, self-hosted from npm + raw monaco-editor API (not the CDN loader / @monaco-editor/react wrapper — decided; rationale in Architecture 08). Workers are wired explicitly via Vite ?worker. The Vega-Lite JSON-schema service is what gives autocomplete + validation; mine vega-editor for how it wires the schema.
• No shared library with Syto. Patterns are copied/adapted, never imported.

---

Milestone map

#	Milestone	Outcome	Spec
M0	Skeleton ✅	Repo builds, tests run, empty shell renders	—
M1	MVP core loop	Author a Vega-Lite snippet, see it render live, it persists	§02, §03A–C, §04, §09A
M1.5	Visual design foundation ✅	Apply the design language: tokens, IBM Plex, restyled M1 surfaces, chart theme	arch 09
M2	Editor robustness ✅	Draft/Published, validation, schema autocomplete, fit modes	§03D–E, §04, §07(editor)
M3	Datasets ✅	Named reusable data + reference resolution in preview	§05, §03F, §09B
M4	Chart Builder ✅	No-JSON chart composition from a dataset	§06
M5	Settings + Import/Export ✅	Preferences + workspace backup/transfer	§07, §08, §09C
M6	Shell polish	Resize/toggle panes, routing, shortcuts, toasts, a11y, offline	§01, §10

MVP boundary = end of M1 (a genuinely usable single-user chart authoring loop). M1.5 makes it look right; M2 makes it robust; M3–M6 make it complete. Ship/dogfood after M1, iterate.

---

M0 · Skeleton ✅ (done)

Vite + React + Zustand + TypeScript + Vitest (happy-dom) + vite-plugin-pwa. src/core ↔ src/app split, useAppStore, design tokens, three-pane placeholder shell, first core module (format-detection) with tests.

Verified: npm run typecheck, npm test (14 passing), npm run build (PWA SW generated).

---

M1 · MVP core loop → the quickest usable Astrolabe

Goal: select/create a snippet, edit its spec JSON, watch a live Vega-Lite preview, and have it survive reload. Single source kind: inline-data specs only (datasets come in M3). No draft/published yet — edits save directly.

Core (src/core/)

• snippet.ts — the Snippet type (spec §09A) + factory (createSnippet, default sample bar-chart template, auto-generated date/time name).
• rendering.ts — prepareSpecForRender(spec, { fitMode }) skeleton; in M1 it's near pass-through (reference resolution is a no-op until M3, fit-mode is M2). Establish the "transform a copy, never mutate stored spec" contract now.

Infrastructure (src/app/infrastructure/)

• idb.ts — thin IndexedDB wrapper (open, get/put/delete/getAll by store). (see Architecture 02 · Persistence)
• snippet-store.ts — persist snippets (object store snippets).

App

• stores/SnippetStore.ts — useSnippetStore with snippets, activeSnippetId, selector-derived activeSnippet; load-on-startup; create/select/delete/update actions (debounced auto-save of edits, spec §03B). Seed one sample snippet on first run. (Superseded later: an empty library now shows the onboarding canvas instead of a placeholder seed — spec §02 → First-Run & Empty Workspace.)
• components/SnippetLibrary.tsx — list + "Create New" pinned item + select/delete.
• components/SpecEditor.tsx — Monaco JSON editor bound to active snippet's spec; debounced write-back to the store. (Worker wiring via Vite ?worker imports — mine vega-editor's Monaco setup.)
• components/LivePreview.tsx — render current spec via vega-embed (actions: false), debounced; clean empty pane when no/blank spec; basic error text.
• Fill the three panes in App.tsx with these.

Tests (core-first)

• snippet.test.ts — factory defaults, sample template validity, unique naming.
• rendering.test.ts — copy-not-mutate invariant; pass-through shape.
• A store test for create/select/delete/auto-save reducer logic (logic extracted from the component so it's testable without DOM).

Manual checks

• Fresh load shows the onboarding canvas (welcome + Create + live example gallery); Create or an example's Add lands you in the editor with a rendered chart. (M1 originally seeded a sample snippet; replaced by the onboarding canvas — spec §02.)
• Type in the editor → preview updates after the debounce; bad JSON → editor keeps working, preview shows an error, recovers when fixed.
• Reload → snippets and selection persist.

---

M1.5 · Visual design foundation ✅ (done) → make the MVP look like itself

Goal: apply our design language so the running MVP looks deliberate, and every later milestone builds on settled tokens instead of placeholders. The expensive part (the design decisions) is already done — this milestone is application, not invention. See Architecture 09 · Visual Design Language and the companion visual-specimen.html.

Styles

• Port the settled specimen tokens into styles/tokens.css (IBM-Plex type scale, 8px-based spacing, role-based color, square chrome, motion); light + dark themes via [data-theme].
• Self-host IBM Plex Sans + Mono in styles/base.css via @fontsource (offline/PWA — never a CDN).

App

• Restyle the four M1 surfaces against the tokens: App shell, SnippetLibrary, SpecEditor (Monaco theme follows [data-theme]), LivePreview. Tokens only — no raw hexes, no hardcoded hues in components.
• Establish the reusable component conventions (buttons, fields, list rows, status, focus ring) that M2–M6 reuse.
• Header theme toggle (pulled forward from M5): a one-click light⇄dark control, persisted via the ui.theme settings key (a minimal forward-compatible settings-store adapter the full M5 UserSettings store will absorb). Hydrated before first paint (no FOUC). Justified: the theme system was already complete, so dogfooding dark mode through M2–M4 beat waiting for the full settings UI.

Core

• Align src/core/vega-themes.ts: chart Config per theme + a categorical range.category palette (clone carbon-design-system/carbon-charts for the sequence — see Architecture 09 §8).

Tests

• Light: the design is mostly visual — a token/theme smoke check, trust the eye.

Manual checks

• The real app looks deliberate in both themes; theme flip repaints UI + chart.
• Keyboard focus ring visible; text/UI contrast passes AA in light and dark.
• No placeholder styling remains on the M1 surfaces.

Verified: typecheck + test (61 passing, incl. vega-themes.test.ts) + build (Plex woff2, all script subsets, bundled & precached via the PWA globPatterns). Both themes screenshotted via the real app (chrome + Monaco + chart all repaint on theme flip); focus ring visible. Notes from the build-out: the placeholder 'experimental' theme was renamed to 'dark' (the settled name); the swappable [data-accent] layer landed with deep teal as the robust default (no switcher UI until M5); Monaco's fontFamily is set to Plex Mono explicitly since it can't read the CSS token.

---

M2 · Editor robustness ✅ (done)

Goal: the editor becomes trustworthy — draft vs published, schema-aware assistance, and the fit-mode rendering contract.

Core

• rendering.ts — implement fit-mode transform (Original/Width/Height/Full → Vega-Lite "container"), recursing into layered/concat/child specs (spec §04 Rendering Contract, step 2).
• vega-lite-schema.ts — provide the Vega-Lite JSON schema for Monaco's validation/autocomplete. Delivered early in M1.5 as infrastructure/monaco-schema.ts (bundled schema, offline, markdownDescription hover docs); no further work needed in M2.

App

• Snippet gains spec (published) + draftSpec (working) per §09A; editing touches draftSpec only.
• SpecEditor header: Draft/Published toggle; Publish (promotes draft, recomputes dataset refs — refs land in M3) + Revert (confirm dialog).
• Library list item: draft-vs-published status indicator.
• Monaco wired with the Vega-Lite schema → squiggles + autocomplete; inline error surface in the editor pane (§03E).
• Preview Fit control (4 modes), persisted (previewFitMode).

Tests

• Fit-mode transforms for each mode incl. nested specs; copy-not-mutate.
• Draft/publish/revert reducer logic; "has unpublished changes" derivation.

Manual checks

• Edit draft, see status flip to "draft"; Publish → status clears; Revert → draft restored with confirmation.
• Invalid spec shows inline error; autocomplete suggests Vega-Lite properties.
• Each fit mode resizes the chart as specified; choice survives reload.

Verified: typecheck + test (83 passing — rendering fit-mode incl. nested layer/concat/facet specs, SnippetStore draft/publish/revert/editorView, settings-store previewFitMode round-trip) + build (PWA, 41 precache entries) + eslint clean. Implementation notes: editing now writes the draft only (commitDraft no longer touches spec); publish/revert live in SnippetStore, with a bufferEpoch counter so programmatic buffer reloads (select/create/revert) refresh Monaco without fighting the cursor mid-typing. The Draft/Published view is a store-level editorView; the published view is read-only and the preview renders whichever version is shown (selectShownText). The editor (§03E) and preview (§04) share one render error via a small PreviewStore. previewFitMode was pulled into AppStore + the settings adapter, hydrated/persisted by a new orchestration/preferences.ts mirroring the theme slice. Publish/Revert success toasts stay deferred to M6 (TODO breadcrumbs at the call sites), matching the existing delete-toast convention.

Fit-mode rendering needed a layout fix: vega-embed brands the embed host with its own .vega-embed { display: inline-block } (injected at runtime, wins the cascade), which shrink-wrapped the host so width: "container" collapsed (Height survived only via the old min-height: 100%). Fix: embed into a static-class inner host (React never reconciles its className, so Vega's runtime classes survive) inside a React-owned frame that carries the fit-sizing class via two-class selectors that out-specify .vega-embed. All four fit modes user-verified in the running app.

---

M3 · Datasets ✅ (done)

Goal: named, reusable data that snippets reference by name; preview resolves the reference.

Core

• profiling.ts — row/column counts, column names, per-column type inference (number/text/date/boolean). (see Architecture 06 · Type Inference)
• rendering.ts — implement dataset reference resolution (§04 Rendering Contract, step 1): {data:{name}} → inline values / raw text+format / URL+format, recursing into sub-specs; "dataset not found" error.
• dataset.ts — Dataset type (§09B); name uniqueness helpers; rename-propagation into referencing specs. (see Architecture 07 · Naming & Relationships)

Infrastructure

• dataset-store.ts — separate high-capacity IndexedDB store (§09E).

App

• stores/DatasetStore.ts + Datasets modal (list/detail panes, create form, edit, delete, copy-reference) via the modal registry/coordinator.
• Snippet datasetRefs maintained on publish; library shows dataset icon + Linked Datasets; dataset detail shows Linked Snippets (bidirectional name link, §09F).
• Extract-to-Dataset flow from the editor (§03F).
• URL-sourced datasets fetched at render time.

Tests

• Reference resolution per source/format incl. nested; not-found error.
• Profiling/type inference across mixed columns, nulls, booleans.
• Rename propagation; name-uniqueness + import-style auto-suffix.

Manual checks

• Create a dataset, reference it by name in a snippet → preview renders.
• Extract inline data → spec rewritten to a reference, dataset appears, links show both ways.
• Delete/rename a referenced dataset behaves per spec.

---

M4 · Chart Builder ✅ (done)

Goal: no-JSON chart composition from a dataset → a new snippet.

Enhancement backlog beyond the Tier-B floor (aggregation, binning, stacking, temporal granularity, sort/orientation, cardinality-based warnings, Tier C intent-first) lives in docs/chart-builder-research.md §8 — its single home, so these stop living in chat.

Core

• chart-builder.ts — pure spec assembler: (mark ∈ Bar/Line/Point/Area/Circle) + channels (X/Y/Color/Size) with field types (Quantitative/Nominal/Ordinal/Temporal)
  • optional width/height → complete Vega-Lite spec with tooltips + named data ref (§06 Output). Field-type defaults from inferred column type.

App

• Chart Builder modal (config pane + live preview pane), launched from a selected dataset; default pre-population (first col→X, second→Y); validation (≥1 channel); Create Snippet → new linked snippet becomes active.

Tests

• Spec assembly: mark/channel/type permutations, unmapped channels omitted, width/height inclusion, field-type derivation, validation gate.

Manual checks

• Build a bar chart from a dataset in a few clicks; preview live-updates; Create → new snippet opens and renders.

Council — FT Visual Vocabulary + Datawrapper are now seated (chart-choice canon): this is where Astrolabe stops being a pass-through JSON editor and starts making chart-shaped suggestions/defaults, so "which chart, and why" becomes a decision the app owns — the one thing Carbon's data-viz styling doesn't cover. Rather than a styling-only seating, we ran a full research-first pass (FT + Datawrapper + the formal engines Draco and Voyager), recorded in docs/chart-builder-research.md, and chose the Tier B "smart + guarded" design: smart default mark for the data shape, valid-type-locked field-type menus, Size-channel discipline, and non-blocking guidance. The convergent rules and citations live in that doc; the spec (§06) was amended to match.

Verified: typecheck + test green — the pure chart-builder.ts assembler (chart-builder.test.ts: mark/channel/type permutations, unmapped-channel omission, validation gate). The Chart Builder modal is wired through the registry, launched from a selected dataset (DatasetsModal → openModal('chartBuilder')). Data-aware cardinality/extent profiling + chart warnings landed (A3/A4).

---

M4.5 · Snippet-library consolidation ✅ (done)

Why out of band: a spec-vs-implementation audit after M4 found §02 features that no later milestone owned — the Selected-Snippet Metadata Panel (inline name + comment editing, timestamps, linked datasets) and the Duplicate operation. Without them a snippet could only ever carry its auto-generated date-time name (no rename, no annotation, no copy), a sharp edge for a snippet manager. Closed before M5 since the spec text already existed and the work was core-first and cheap.

Core

• snippet.ts — duplicateSnippet(source, {now,id}): independent copy carrying both spec versions, comment, tags, and dataset refs; "(copy)" name; fresh identity/timestamps; cloned mutable members.

App

• SnippetStore — renameSnippet, setComment (both advance modified per §02 → Sort, no editor-buffer touch), duplicateActiveSnippet (flushes the live buffer first, prepends the copy, makes it active).
• SnippetLibrary — the metadata panel below the list: Name + Comment auto-save (debounced while typing, flushed on blur), read-only Created/Modified, Linked Datasets list, and Duplicate / Delete. Duplicate raises a success toast (the copy isn't self-evident, unlike Create); §02-compliant.

Also fixed (§03C divergence): the preview debounced every change, so a snippet load / Draft↔Published switch incurred a 300 ms blank instead of the spec's immediate render. LivePreview now renders immediately on bufferEpoch/editorView change and debounces only keystroke (shownText-only) changes.

Deferred at the time to M5/M6 (per §02): Search, Sort controls + persistence, two distinct empty-state messages, Storage Monitor — all now delivered in M6.

Verified: typecheck + test (287 passing — snippet duplicate factory, SnippetStore rename/comment/duplicate, a SnippetLibrary render test guarding the auto-save effect against a render loop) + eslint clean + build (PWA, 41 precache entries).

---

M5 · Settings + Import/Export ✅ (done)

Goal: preferences and whole-workspace backup/transfer.

Core

• settings.ts — UserSettings shape + defaults + load-with-fallback (§07, §09C); unknown/missing values fall back silently.
• import-normalize.ts — accept envelope / bare array / single snippet / foreign shapes; field mapping (content→spec, draft→draftSpec, createdAt→created); tag "imported"; merge rules (append, id-collision reassign, dataset-name auto-suffix, datasets-before-snippets) (§08).
• export-envelope.ts — build the {version, exportedAt, exportedBy, snippets, datasets} envelope.

Infrastructure

• settings-store.ts (localStorage): extend the minimal M1.5 adapter (which already persists ui.theme) to the full UserSettings record; ux-prefs for sort
  • panel layout (§09D).

App

• Distributed settings — not a modal (design review, see spec §07 + arch 10). Each cluster is a per-pane disclosure popover that applies live: Editor settings in the editor toolbar, render debounce in the preview, date format in the library; theme stays the header toggle. A shared SettingsPopover primitive (gear + non-modal popover, APG disclosure) backs all three. Wire render-debounce + editor options + date-format through.
• Header Import/Export (direct file dialog / download, no modal).
• Date formatting util (smart/iso/custom) used by the library list + metadata panel.

Tests

• Import normalization across all accepted shapes; merge/collision/rename logic; quota-overage messaging path. Envelope round-trip (export→import idempotence).
• Settings load-with-fallback for partial/unknown records.

Manual checks

• Change theme/debounce/date-format → takes effect; Cancel reverts; Reset confirms.
• Export → reimport into a populated workspace merges without overwrite; renames reported.

Verified: typecheck + test green — import-normalize (accepted shapes + merge/ collision/rename), export-envelope (round-trip), settings (load-with-fallback), UserSettingsStore, ux-prefs. Distributed per-pane settings shipped as SettingsPopover disclosures (editor toolbar, preview, library) applying live; header Import/Export wired through services/transfer.ts (→ normalizeImport / envelope build), no modal.

---

M6 · Shell polish & non-functional

Goal: the workspace feels finished and meets §10.

• Panes: drag-resize handles with min widths; widths persist ✅ (pulled forward after M2). Per-pane show/hide toggle strip + visibility persist + proportional redistribution on hide (§01A, §09D) ✅.
• Routing: URL hash view-state (#snippet-<id>, #datasets/...) with Back/Forward; restore on load (§01E) ✅.
• Shortcuts: Cmd/Ctrl+Shift+N / +K / +S / +, / Esc via a single key router (§01D) ✅.
• Toasts: success/error/warning/info, stacking, auto-dismiss, reduced-motion; appear/disappear with a brief fade (§01F) ✅ (fade-out two-phase dismiss landed last).
• Library search / sort / empty states (§02, §09D): live search across name/comment/draft spec; Sort by Modified/Created/Name/Size with a disclosure + flip-on-reselect, persisted to astrolabe:ux-prefs; the two distinct empty states ✅ (council-guided — see arch 10).
• Storage monitor for the snippet tier (§02) ✅ (role="meter" fill bar, escalating ok/warning/critical at 0.8/0.95).
• Live-preview busy indicator (§04, §10): ~non-blocking overlay + aria-busy for renders past 1s ✅.
• Import atomicity (§08): roll back on storage-quota failure so no partial import is committed; actionable error ✅ (effective at the service boundary; true cross-record IDB-transaction atomicity would require exposing a raw transaction from db.ts — see arch 02).
• A11y: modal focus trap + return, labelled icon buttons, contrast in both themes (§10) — ✅ in place.
• About & Privacy and Donate modals (§01) ✅ (Donate URL is a placeholder DONATE_URL pending the real link).
• Offline/installable: the manifest now ships a full SVG icon set (favicon / maskable / monochrome) + theme_color, and the SW precaches the shell — the app is installable. ⏳ Remaining — manual verification in a running/installed app (checklist: manual-verification.md); iOS home-screen still wants a PNG apple-touch-icon (logged residual).
• Council — seat web.dev for the PWA/offline/storage surfaces none of the seated members cover: the service-worker update-available prompt (registerType: 'prompt'), storage persistence (navigator.storage.persist()), and the quota estimate (StorageManager.estimate()) ✅ seated + backfilled (update-prompt toast + persist() request; estimate already wired). ⏳ Remaining gap: manifest ships no icons → not yet installable ✅ SVG icon set added + wired (favicon / maskable / monochrome). See /council and arch 10.

Manual checks: keyboard-only run-through; reload restores view from URL; offline reload works; install as standalone; reduced-motion honored. ⏳ Still owed (plus visual verification of the new library/monitor/modals/busy-indicator surfaces).

---

Cross-cutting, do-as-you-go

• Build to the design language: the foundation lands in M1.5; from M2 on, every new component uses the Architecture 09 tokens and conventions — no placeholder styling, no raw hues. Staying on it is the do-as-you-go part.
• i18n (optional, deferred): if translation is wanted, split a portable i18n registry (no React) from the app-layer bindings, mirroring the core ↔ app boundary. M1–M6 can ship English-only with date formatting locale-aware (§10). Don't retrofit later if avoidable — keep user-facing strings centralized from M1.
• Versioning: simplified semver 0.x.y, package.json → __APP_VERSION__ (already wired). Bump per shippable milestone.
• Docs trio: keep SOUL.md / AGENTS.md / CLAUDE.md current as the app grows.

---

Open divergences (spec ↔ code)

Surfaced by the full-docs consistency review — each needs a deliberate resolution, not a silent drift:

• Storage-monitor scope. Spec §02 frames the indicator as the snippet storage budget specifically; the shipped estimate instead reported whole-origin usage/quota. ✅ Resolved — rather than pick "snippet budget" vs. "whole-origin", the monitor was redesigned into a composition breakdown (snippets · datasets · app) that drops the unreliable browser quota entirely and shows real measured sizes. Spec §02 + §10 and arch 02 §6 updated; council resolution recorded in arch 10.

---

Architecture reference

The how behind each milestone is documented self-containedly in docs/architecture/ — no external repo needed:

Need	Doc
Zustand stores, selector derivations, debounced auto-save	01 · State & Stores
IndexedDB wrapper, lazy loading, migrations, localStorage prefs, storage tiers	02 · Persistence
Modal registry + coordinator + shell, unsaved-change detection, focus trap	03 · Modal System
URL hash view-state, keyboard routing, interactive-context detection	04 · Routing & Events
vega-embed integration, theming, debounced preview, error display	05 · Rendering, Theming & Preview
Column type inference + dataset profiling	06 · Type Inference & Profiling
Unique names + import auto-suffix, snippet↔dataset links, rename propagation	07 · Naming & Relationships
Monaco setup, Vega-Lite schema service, editor patterns mined from vega/editor	08 · Vega Editor Techniques
Design language: tokens, type, spacing, color roles, components, themes	09 · Visual Design Language