astrolabe/docs/IMPLEMENTATION-PLAN.md

# Astrolabe — Incremental Implementation Plan

> A spec-driven rebuild of Astrolabe on Syto's architecture. The authoritative
> behavioral contract is `docs/spec/` (sections 00–10). This document sequences
> the build into the **quickest path to a usable MVP**, then layers the rest.
>
> **Method per milestone:** build core-first (portable, pure, tested) → wire UI →
> cover with tests → manual smoke check against the spec's acceptance points.
> "Test the Core, Trust the UI": high coverage on `src/core/`, lighter on components.

---

## Architectural ground rules

These are decided and apply to every milestone. The **how** behind each is written up
self-containedly in [`docs/architecture/`](architecture/00-overview.md) — read the matching
doc before implementing.

- **`src/core/` is portable** — no browser APIs, no React, no Monaco. Pure spec
  operations (detection, profiling, reference resolution, fit transforms,
  validation, import normalization). This is what we test hardest and what could
  power a future headless renderer/CLI.
- **`src/app/`** holds React + Zustand UI. State lives in Zustand **stores**
  (`useAppStore`, plus per-feature stores); browser specifics live in
  **`src/app/infrastructure/`** adapters (IndexedDB, localStorage, URL hash) so
  the rest of the app never touches `window`/`indexedDB` directly.
- **Modals via a registry + coordinator + shell** (see [Architecture 03](architecture/03-modal-system.md)),
  not ad-hoc conditional rendering.
- **CSS Modules + design tokens** (`styles/tokens.css`); themes flip
  `[data-theme]`. Vega theme follows the UI theme. The design language behind the
  tokens — type, spacing, color roles, components, themes — is defined in
  [Architecture 09](architecture/09-visual-design.md) and established in M1.5.
- **Editor: Monaco**, **self-hosted from npm + raw `monaco-editor` API** (not the
  CDN loader / `@monaco-editor/react` wrapper — decided; rationale in
  [Architecture 08](architecture/08-vega-editor-techniques.md#decision--monaco-integration-self-hosted-raw-api)).
  Workers are wired explicitly via Vite `?worker`. The Vega-Lite JSON-schema
  service is what gives autocomplete + validation; mine vega-editor for how it
  wires the schema.
- **No shared library with Syto.** Patterns are copied/adapted, never imported.

---

## Milestone map

| #        | Milestone                   | Outcome                                                                        | Spec                                        |
| -------- | --------------------------- | ------------------------------------------------------------------------------ | ------------------------------------------- |
| **M0**   | Skeleton ✅                 | Repo builds, tests run, empty shell renders                                    | —                                           |
| **M1**   | **MVP core loop**           | Author a Vega-Lite snippet, see it render live, it persists                    | §02, §03A–C, §04, §09A                      |
| **M1.5** | Visual design foundation ✅ | Apply the design language: tokens, IBM Plex, restyled M1 surfaces, chart theme | [arch 09](architecture/09-visual-design.md) |
| **M2**   | Editor robustness           | Draft/Published, validation, schema autocomplete, fit modes                    | §03D–E, §04, §07(editor)                    |
| **M3**   | Datasets                    | Named reusable data + reference resolution in preview                          | §05, §03F, §09B                             |
| **M4**   | Chart Builder               | No-JSON chart composition from a dataset                                       | §06                                         |
| **M5**   | Settings + Import/Export    | Preferences + workspace backup/transfer                                        | §07, §08, §09C                              |
| **M6**   | Shell polish                | Resize/toggle panes, routing, shortcuts, toasts, a11y, offline                 | §01, §10                                    |

**MVP boundary = end of M1** (a genuinely usable single-user chart authoring loop).
M1.5 makes it _look right_; M2 makes it _robust_; M3–M6 make it _complete_.
Ship/dogfood after M1, iterate.

---

## M0 · Skeleton ✅ (done)

Vite + React + Zustand + TypeScript + Vitest (happy-dom) + vite-plugin-pwa.
`src/core` ↔ `src/app` split, `useAppStore`, design tokens, three-pane placeholder
shell, first core module (`format-detection`) with tests.

**Verified:** `npm run typecheck`, `npm test` (14 passing), `npm run build` (PWA SW generated).

---

## M1 · MVP core loop → _the quickest usable Astrolabe_

**Goal:** select/create a snippet, edit its spec JSON, watch a live Vega-Lite
preview, and have it survive reload. Single source kind: inline-data specs only
(datasets come in M3). No draft/published yet — edits save directly.

**Core (`src/core/`)**

- `snippet.ts` — the Snippet type (spec §09A) + factory (`createSnippet`,
  default sample bar-chart template, auto-generated date/time name).
- `rendering.ts` — `prepareSpecForRender(spec, { fitMode })` skeleton; in M1 it's
  near pass-through (reference resolution is a no-op until M3, fit-mode is M2).
  Establish the "transform a copy, never mutate stored spec" contract now.

**Infrastructure (`src/app/infrastructure/`)**

- `idb.ts` — thin IndexedDB wrapper (open, get/put/delete/getAll by store).
  _(see [Architecture 02 · Persistence](architecture/02-persistence.md))_
- `snippet-store.ts` — persist snippets (object store `snippets`).

**App**

- `stores/SnippetStore.ts` — `useSnippetStore` with `snippets`, `activeSnippetId`,
  selector-derived `activeSnippet`; load-on-startup; create/select/delete/update actions
  (debounced auto-save of edits, spec §03B). Seed one sample snippet on first run.
- `components/SnippetLibrary.tsx` — list + "Create New" pinned item + select/delete.
- `components/SpecEditor.tsx` — Monaco JSON editor bound to active snippet's spec;
  debounced write-back to the store. (Worker wiring via Vite `?worker` imports —
  mine vega-editor's Monaco setup.)
- `components/LivePreview.tsx` — render current spec via `vega-embed` (actions:
  false), debounced; clean empty pane when no/blank spec; basic error text.
- Fill the three panes in `App.tsx` with these.

**Tests (core-first)**

- `snippet.test.ts` — factory defaults, sample template validity, unique naming.
- `rendering.test.ts` — copy-not-mutate invariant; pass-through shape.
- A store test for create/select/delete/auto-save reducer logic (logic extracted
  from the component so it's testable without DOM).

**Manual checks**

- Fresh load seeds a sample snippet that renders a bar chart.
- Type in the editor → preview updates after the debounce; bad JSON → editor keeps
  working, preview shows an error, recovers when fixed.
- Reload → snippets and selection persist.

---

## M1.5 · Visual design foundation ✅ (done) → _make the MVP look like itself_

**Goal:** apply our design language so the running MVP looks deliberate, and every
later milestone builds on settled tokens instead of placeholders. The expensive part
(the design decisions) is already done — this milestone is _application_, not
invention. See [Architecture 09 · Visual Design Language](architecture/09-visual-design.md)
and the companion `visual-specimen.html`.

**Styles**

- Port the settled specimen tokens into `styles/tokens.css` (IBM-Plex type scale,
  8px-based spacing, role-based color, square chrome, motion); light + dark themes
  via `[data-theme]`.
- Self-host **IBM Plex Sans + Mono** in `styles/base.css` via `@fontsource`
  (offline/PWA — never a CDN).

**App**

- Restyle the four M1 surfaces against the tokens: App shell, SnippetLibrary,
  SpecEditor (Monaco theme follows `[data-theme]`), LivePreview. Tokens only — no
  raw hexes, no hardcoded hues in components.
- Establish the reusable component conventions (buttons, fields, list rows, status,
  focus ring) that M2–M6 reuse.
- **Header theme toggle** (pulled forward from M5): a one-click light⇄dark control,
  persisted via the `ui.theme` settings key (a minimal forward-compatible
  `settings-store` adapter the full M5 UserSettings store will absorb). Hydrated
  before first paint (no FOUC). Justified: the theme system was already complete,
  so dogfooding dark mode through M2–M4 beat waiting for the Settings modal.

**Core**

- Align `src/core/vega-themes.ts`: chart `Config` per theme + a categorical
  `range.category` palette (clone `carbon-design-system/carbon-charts` for the
  sequence — see Architecture 09 §7).

**Tests**

- Light: the design is mostly visual — a token/theme smoke check, trust the eye.

**Manual checks**

- The real app looks deliberate in both themes; theme flip repaints UI + chart.
- Keyboard focus ring visible; text/UI contrast passes AA in light and dark.
- No placeholder styling remains on the M1 surfaces.

**Verified:** `typecheck` + `test` (61 passing, incl. `vega-themes.test.ts`) +
`build` (Plex woff2, all script subsets, bundled & precached via the PWA
`globPatterns`). Both themes screenshotted via the real
app (chrome + Monaco + chart all repaint on theme flip); focus ring visible.
Notes from the build-out: the placeholder `'experimental'` theme was renamed to
`'dark'` (the settled name); the swappable `[data-accent]` layer landed with
indigo as the robust default (no switcher UI until M5); Monaco's `fontFamily` is
set to Plex Mono explicitly since it can't read the CSS token.

---

## M2 · Editor robustness

**Goal:** the editor becomes trustworthy — draft vs published, schema-aware
assistance, and the fit-mode rendering contract.

**Core**

- `rendering.ts` — implement **fit-mode** transform (Original/Width/Height/Full →
  Vega-Lite `"container"`), recursing into layered/concat/child specs (spec §04
  Rendering Contract, step 2).
- `vega-lite-schema.ts` — provide the Vega-Lite JSON schema for Monaco's
  validation/autocomplete (mine vega-editor for sourcing/versioning the schema).

**App**

- Snippet gains `spec` (published) + `draftSpec` (working) per §09A; editing
  touches `draftSpec` only.
- `SpecEditor` header: Draft/Published toggle; **Publish** (promotes draft, recomputes
  dataset refs — refs land in M3) + **Revert** (confirm dialog).
- Library list item: draft-vs-published **status indicator**.
- Monaco wired with the Vega-Lite schema → squiggles + autocomplete; inline error
  surface in the editor pane (§03E).
- Preview **Fit control** (4 modes), persisted (`previewFitMode`).

**Tests**

- Fit-mode transforms for each mode incl. nested specs; copy-not-mutate.
- Draft/publish/revert reducer logic; "has unpublished changes" derivation.

**Manual checks**

- Edit draft, see status flip to "draft"; Publish → status clears; Revert →
  draft restored with confirmation.
- Invalid spec shows inline error; autocomplete suggests Vega-Lite properties.
- Each fit mode resizes the chart as specified; choice survives reload.

---

## M3 · Datasets

**Goal:** named, reusable data that snippets reference by name; preview resolves
the reference.

**Core**

- `profiling.ts` — row/column counts, column names, **per-column type inference**
  (number/text/date/boolean). _(see [Architecture 06 · Type Inference](architecture/06-type-inference.md))_
- `rendering.ts` — implement **dataset reference resolution** (§04 Rendering
  Contract, step 1): `{data:{name}}` → inline values / raw text+format / URL+format,
  recursing into sub-specs; "dataset not found" error.
- `dataset.ts` — Dataset type (§09B); name uniqueness helpers; rename-propagation
  into referencing specs. _(see [Architecture 07 · Naming & Relationships](architecture/07-naming-and-relationships.md))_

**Infrastructure**

- `dataset-store.ts` — separate high-capacity IndexedDB store (§09E).

**App**

- `stores/DatasetStore.ts` + Datasets **modal** (list/detail panes, create form,
  edit, delete, copy-reference) via the modal registry/coordinator.
- Snippet `datasetRefs` maintained on publish; library shows dataset icon +
  Linked Datasets; dataset detail shows Linked Snippets (bidirectional name link, §09F).
- **Extract-to-Dataset** flow from the editor (§03F).
- URL-sourced datasets fetched at render time.

**Tests**

- Reference resolution per source/format incl. nested; not-found error.
- Profiling/type inference across mixed columns, nulls, booleans.
- Rename propagation; name-uniqueness + import-style auto-suffix.

**Manual checks**

- Create a dataset, reference it by name in a snippet → preview renders.
- Extract inline data → spec rewritten to a reference, dataset appears, links show both ways.
- Delete/rename a referenced dataset behaves per spec.

---

## M4 · Chart Builder

**Goal:** no-JSON chart composition from a dataset → a new snippet.

**Core**

- `chart-builder.ts` — pure spec assembler: (mark ∈ Bar/Line/Point/Area/Circle) +
  channels (X/Y/Color/Size) with field types (Quantitative/Nominal/Ordinal/Temporal)
  - optional width/height → complete Vega-Lite spec with tooltips + named data ref
    (§06 Output). Field-type defaults from inferred column type.

**App**

- Chart Builder **modal** (config pane + live preview pane), launched from a
  selected dataset; default pre-population (first col→X, second→Y); validation
  (≥1 channel); Create Snippet → new linked snippet becomes active.

**Tests**

- Spec assembly: mark/channel/type permutations, unmapped channels omitted,
  width/height inclusion, field-type derivation, validation gate.

**Manual checks**

- Build a bar chart from a dataset in a few clicks; preview live-updates;
  Create → new snippet opens and renders.

---

## M5 · Settings + Import/Export

**Goal:** preferences and whole-workspace backup/transfer.

**Core**

- `settings.ts` — UserSettings shape + defaults + load-with-fallback (§07, §09C);
  unknown/missing values fall back silently.
- `import-normalize.ts` — accept envelope / bare array / single snippet / foreign
  shapes; field mapping (`content`→spec, `draft`→draftSpec, `createdAt`→created);
  tag `"imported"`; merge rules (append, id-collision reassign, dataset-name
  auto-suffix, datasets-before-snippets) (§08).
- `export-envelope.ts` — build the `{version, exportedAt, exportedBy, snippets, datasets}` envelope.

**Infrastructure**

- `settings-store.ts` (localStorage): **extend** the minimal M1.5 adapter (which
  already persists `ui.theme`) to the full UserSettings record; `ux-prefs` for sort
  - panel layout (§09D).

**App**

- Settings **modal** (Appearance/Editor/Performance/Formatting), Apply/Cancel/Reset,
  dirty indicator; wire render-debounce + theme + date-format through to the app.
  Theme already switches (M1.5 header toggle + chart/editor themes) — M5 surfaces
  it inside the modal too (sharing the same `ui.theme`) and wires the rest.
- Header **Import**/**Export** (direct file dialog / download, no modal).
- Date formatting util (smart/iso/custom) used by the library list.

**Tests**

- Import normalization across all accepted shapes; merge/collision/rename logic;
  quota-overage messaging path. Envelope round-trip (export→import idempotence).
- Settings load-with-fallback for partial/unknown records.

**Manual checks**

- Change theme/debounce/date-format → takes effect; Cancel reverts; Reset confirms.
- Export → reimport into a populated workspace merges without overwrite; renames reported.

---

## M6 · Shell polish & non-functional

**Goal:** the workspace feels finished and meets §10.

- **Panes:** drag-resize handles with min widths; per-pane show/hide toggle strip;
  widths + visibility persist (§01A, §09D).
- **Routing:** URL hash view-state (`#snippet-<id>`, `#datasets/...`) with Back/Forward;
  restore on load (§01E). _(see [Architecture 04 · Routing & Events](architecture/04-routing-and-events.md))_
- **Shortcuts:** Cmd/Ctrl+Shift+N / +K / +S / +, / Esc via a single key router
  (§01D). _(see [Architecture 04 · Routing & Events](architecture/04-routing-and-events.md))_
- **Toasts:** success/error/warning/info, stacking, auto-dismiss, reduced-motion (§01F).
- **Storage monitor** for the snippet tier (§02).
- **A11y:** modal focus trap + return, labelled icon buttons, contrast in both themes (§10).
- **Offline/installable:** verify the SW + manifest give a working offline + installed app.
- **About & Privacy** and **Donate** modals.

**Manual checks:** keyboard-only run-through; reload restores view from URL;
offline reload works; install as standalone; reduced-motion honored.

---

## Cross-cutting, do-as-you-go

- **Build to the design language:** the foundation lands in M1.5; from M2 on, every
  new component uses the [Architecture 09](architecture/09-visual-design.md) tokens
  and conventions — no placeholder styling, no raw hues. Staying on it is the
  do-as-you-go part.
- **i18n** (optional, deferred): if translation is wanted, split a portable i18n
  registry (no React) from the app-layer bindings, mirroring the `core` ↔ `app`
  boundary. M1–M6 can ship English-only with date formatting locale-aware (§10).
  Don't retrofit later if avoidable — keep user-facing strings centralized from M1.
- **Versioning:** simplified semver `0.x.y`, `package.json` → `__APP_VERSION__`
  (already wired). Bump per shippable milestone.
- **Docs trio:** keep `SOUL.md` / `AGENTS.md` / `CLAUDE.md` current as the app grows.

---

## Architecture reference

The **how** behind each milestone is documented self-containedly in
[`docs/architecture/`](architecture/00-overview.md) — no external repo needed:

| Need                                                                           | Doc                                                                               |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- |
| Zustand stores, selector derivations, debounced auto-save                      | [01 · State & Stores](architecture/01-state-and-stores.md)                        |
| IndexedDB wrapper, lazy loading, migrations, localStorage prefs, storage tiers | [02 · Persistence](architecture/02-persistence.md)                                |
| Modal registry + coordinator + shell, unsaved-change detection, focus trap     | [03 · Modal System](architecture/03-modal-system.md)                              |
| URL hash view-state, keyboard routing, interactive-context detection           | [04 · Routing & Events](architecture/04-routing-and-events.md)                    |
| vega-embed integration, theming, debounced preview, error display              | [05 · Rendering, Theming & Preview](architecture/05-rendering-theming-preview.md) |
| Column type inference + dataset profiling                                      | [06 · Type Inference & Profiling](architecture/06-type-inference.md)              |
| Unique names + import auto-suffix, snippet↔dataset links, rename propagation   | [07 · Naming & Relationships](architecture/07-naming-and-relationships.md)        |
| Monaco setup, Vega-Lite schema service, editor patterns mined from vega/editor | [08 · Vega Editor Techniques](architecture/08-vega-editor-techniques.md)          |
| Design language: tokens, type, spacing, color roles, components, themes        | [09 · Visual Design Language](architecture/09-visual-design.md)                   |