oleh/astrolabe
1
0
Fork 0
mirror of https://github.com/olehomelchenko/astrolabe.git synced 2026-08-08 02:02:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initial scaffold: spec, architecture playbook, and M0 skeleton

Browse Source
This commit is contained in:
Oleh Omelchenko
2026-06-04 22:14:33 +03:00
commit 056644450c
51 changed files with 13754 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/projects/-Users-oleh-code-astrolabe/memory/MEMORY.md
+2
View File
@@ -0,0 +1,2 @@
- [vega-editor reference clone](vega-editor-reference-clone.md) — local vega/editor at /Users/oleh/code/reference/vega-editor for Monaco/render/validation techniques
- [Stack: React + Zustand](stack-moving-to-react.md) — final UI stack is React + Zustand (migrated off Preact + signals at M0); doc sweep done
.claude/projects/-Users-oleh-code-astrolabe/memory/stack-moving-to-react.md
+29
View File
@@ -0,0 +1,29 @@
---
name: stack-moving-to-react
description: Astrolabe's UI stack is React + Zustand (migrated from Preact + signals at M0)
metadata:
type: project
---
The UI stack is **React + Zustand** (user decision, 2026-06-03, executed at M0 before feature
work). It started as Preact + `@preact/signals`; a brief intermediate step migrated to React +
`@preact/signals-react`, but the final call is **React + Zustand** — signals were dropped.
**Why React:** the Preact pain was React-ecosystem friction (real-React-only libraries not
cooperating with `preact/compat`) — *not* the signals model. Real React removes that whole
class of problem and makes borrowing from the reference [[vega-editor-reference-clone]] (a
React app) port directly.
**Why Zustand over keeping signals:** signals were never the problem, but switching framework
was the one cheap moment (M0, ~nothing implemented) to also pick the lowest-future-migration-risk
state lib. Zustand is idiomatic React, has first-class outside-React access
(`getState`/`setState`/`subscribe`) that fits the "logic lives in core/services, not components"
architecture, and carries no build-time transform. The signals→Zustand cost was only rewriting
unimplemented docs.
**How to apply:** Stores are `create<State>()` modules exporting a `useXStore` hook (state +
actions in one object); components read via `useXStore(selector)` (+ `useShallow` for object
selections); non-component code uses `getState()/setState()/subscribe()`; derive in selectors,
never store derived fields. See `docs/architecture/01-state-and-stores.md`. The repo-wide
Preact/signals→React/Zustand doc sweep is **done** — no stale "Preact"/"signals" wording should
remain except where it describes the sibling project Syto, Vega's own signals, or plain English.
.claude/projects/-Users-oleh-code-astrolabe/memory/vega-editor-reference-clone.md
+27
View File
@@ -0,0 +1,27 @@
---
name: vega-editor-reference-clone
description: Location and nature of the local vega/editor clone used as a technique reference for Astrolabe's Monaco/vega-embed/validation work
metadata:
type: reference
---
The canonical Vega-Lite editor (vega/editor) is cloned locally at
`/Users/oleh/code/reference/vega-editor` (shallow clone of `main`, HEAD 4fdbb59). It is the
reference for the "editor + renderer" wiring Astrolabe's M1/M2 call out ("mine vega-editor
for how it wires the schema"). Re-clone with `git clone --depth 1 https://github.com/vega/editor`.
Key divergences to remember when borrowing — it is **React + Redux/context**, uses
`@monaco-editor/react` + `@monaco-editor/loader` (CDN-loaded Monaco, no explicit worker
config), and — surprisingly — **does NOT use vega-embed for its live preview** (it hand-rolls
`vegaLite.compile` → `vega.parse` → `new vega.View().runAsync()`; vega-embed is imported only
for types + the exported standalone HTML). Astrolabe is React (see [[stack-moving-to-react]]) +
Zustand stores + raw `monaco-editor` + `vegaEmbed()`. Since both are now React, vega/editor's
component lifecycle patterns port fairly directly; the friction is state (their flat Redux →
our Zustand stores) and Monaco worker wiring (their CDN loader → our explicit Vite workers).
The Monaco choice (self-hosted from npm + raw API, not the CDN loader / `@monaco-editor/react`
route) is a recorded decision — see `docs/architecture/08` § Decision · Monaco integration.
The highest-value files: `src/utils/monaco.ts` (schema wiring), `src/utils/validate.ts` (ajv),
`src/utils/jsonc-parser.ts`, `src/utils/logger.ts` (LocalLogger), `src/components/renderer/renderer.tsx`
(View lifecycle), `src/components/app.tsx:188-365` (parse→validate→compile→render flow),
`src/constants/default-state.ts` (state shape). Relevant to [[]] M1/M2 of the implementation plan.
.claude/skills/alignment/SKILL.md
+185
View File
@@ -0,0 +1,185 @@
---
name: alignment
description: Review staged or uncommitted code to ensure quality, test coverage, and alignment with project specifications
disable-model-invocation: true
---
# Code Alignment
Review staged or uncommitted code to ensure quality, test coverage, and alignment with the
project's spec (`docs/spec/`) and architecture playbook (`docs/architecture/`).
## Scope
Determine the review scope using `git diff` (unstaged) and `git diff --staged` (staged).
Review all changes in scope. If changes span multiple patterns below, apply all relevant sections.
## General Instructions
### Process
1. **Git**: **NEVER** stage (`git add`) or commit (`git commit`) — that is the USER's
responsibility. If the reviewed changes span multiple independent concerns (a feature + an
unrelated fix, a refactor + a new capability), suggest splitting them into separate commits
and mention the logical boundaries.
2. **Verification**: After changes, run `npm run typecheck` and `npm test`; run `npm run build`
if the change could affect the build. If tests fail, fix the issue if straightforward; ask
the user only if non-trivial or ambiguous.
3. **Fix directly; don't ask first.** When you find an issue covered by these instructions,
fix it in place rather than reporting it and waiting. Ask the user only when the fix is
genuinely ambiguous or several valid approaches exist with real trade-offs. When guidelines
conflict, prefer in this order: **SOUL.md philosophy > `docs/spec/` behavioral contract >
`docs/architecture/` patterns > local cleanup**. These instructions are not strictly
prohibitive — if a guideline has a valid reason to be bypassed, mention it in the summary.
### Code Quality
4. **Code Cleanup**: Remove leftover code, unnecessary defensive programming, and
over-engineering from iterative development — dead code, try/catch around internal calls
that can't throw, abstraction layers wrapping a single implementation. Proceed with caution;
ask if unsure.
5. **Styles and UI**: When altering CSS or layout, follow or generalize existing patterns
(CSS Modules + design tokens in `styles/tokens.css`) rather than writing from scratch. Don't
fix whitespace/formatting (trailing newlines etc.) — Prettier owns that.
6. **Code Comments**: Comments should not duplicate what the code already says. Remove
parroting comments. Ensure comments capture non-obvious *why* — design decisions,
constraints, gotchas. Flag missing comments where a reader would reasonably ask "why is this
done this way?"
7. **Workarounds**: Flag code that works around a problem rather than solving it (`// HACK`,
silent catch-and-ignore, feature detection for internal bugs). A justified workaround
(upstream bug, browser quirk) needs a comment explaining why and a tracking reference; an
unjustified one should be replaced with a proper fix.
8. **Pre-existing & out-of-scope issues — leave a breadcrumb.** For anything you notice but
don't fix (pre-existing patterns the new code follows; observations the change exposes but
that are out of scope), mark it with a `// TODO:` at the relevant code site explaining
*what* could be improved and *why* (1–3 lines). **If an observation is important enough to
mention in the summary, it is important enough to deserve a `// TODO:` at the code location**
— otherwise the next reader has no way to recover the context.
### Architecture & Project-Specific Checks
9. **Portable core boundary**: `src/core/` must stay pure — no browser APIs (`window`,
`document`, `indexedDB`, `localStorage`), no React, no Monaco, no `vega-embed`. Flag any such
import. Pure spec logic (detection, profiling, reference resolution, fit transforms,
validation, import normalization) belongs in `src/core/` and must be unit-tested. See
`docs/architecture/00-overview.md` for the layering.
10. **Infrastructure-adapter boundary**: Only `src/app/infrastructure/` touches `indexedDB`,
`localStorage`, or `window.location`. Flag direct access elsewhere — route it through an
adapter (`docs/architecture/02-persistence.md`, `04-routing-and-events.md`).
11. **Rendering safety** (`docs/architecture/05-rendering-theming-preview.md`): the
reference-resolution/fit-mode transform must run on a **copy** of the spec — never mutate the
stored spec; a previous `vega-embed` view must be `.finalize()`d before re-render (no leaks);
user-derived field names must be escaped before going into `field:`; an invalid/unrenderable
spec must fail safe (readable error, no crash), and a blank spec renders nothing.
12. **Persistence safety**: records that may need migration carry a `version` field; reads
apply migrations; destructive actions (delete, revert, reset) confirm; storage failures
warn rather than silently lose data.
13. **Self-containment**: documentation and comments must not add pointers that require an
external repository to follow. Knowledge gets captured locally (`docs/spec/`,
`docs/architecture/`), not linked out.
14. **User-facing copy**: keep user-visible strings centralized and written for users (sentence
case, active voice, no "please", no exclamation marks in errors). If/when an i18n layer
exists, route strings through it instead of hardcoding.
### Output
15. **Summary**: respond with a summary of changes — choices made due to these instructions,
choices where multiple approaches existed, and non-obvious architectural assumptions the
user should know but might not spot in the diff. If the summary mentions an observation you
chose not to fix (rule #8), confirm a `// TODO:` breadcrumb was placed at the code site.
---
## Pattern A: New Functionality
### Testing
- Unit tests for new `src/core/` logic (test the core hardest).
- Lighter component/interaction tests for new UI.
- Tests pass before proceeding.
### Documentation
Update relevant docs if the feature is significant:
- **`docs/spec/`** — if product behavior changed (this is a contract; change deliberately).
- **`docs/architecture/`** — if a new pattern, navigation map, or decision rule emerged.
- **`docs/IMPLEMENTATION-PLAN.md`** — mark milestone progress.
Use the `/doc-update` skill for session-discovered gaps. The list is not exclusive.
### Dependencies
If `package.json` changed:
- Flag each new dependency; explain what it does and why it's needed.
- Could a small custom implementation avoid it? Note the trade-off.
- Prefer dependencies that solve genuinely hard problems (parsing, rendering) over those that
save boilerplate.
### Alignment Check
- **SOUL.md** — philosophy (must not violate without good reason).
- **`docs/spec/`** — behavioral contract.
- **`docs/architecture/`** — the relevant pattern doc.
---
## Pattern B: Bug Fixes
### Testing
- Add a regression test that reproduces the bug and verifies the fix.
- Interaction test if the bug affected UI behavior.
### Documentation
Usually not required unless the bug revealed incorrect docs, or the fix changes documented
(spec) behavior.
### Alignment Check
- **SOUL.md** philosophy; **`docs/spec/`** behavioral contract; **`docs/architecture/`** patterns.
---
## Pattern C: Refactoring
### Impact Analysis
1. **Search for usages** of modified functions/types across the codebase (Grep).
2. **Identify call sites** (components, stores, services, infrastructure, tests).
3. **Check exports** used by other modules.
4. **Review dependencies** — what the code depends on and what depends on it.
### Testing
- Update existing tests to the new structure; verify all call sites.
- Run `npm test` and `npm run typecheck`.
### Documentation
Update `docs/architecture/` if a pattern, module responsibility, or navigation map changed.
Update JSDoc/inline comments if signatures or behavior changed.
### Alignment Check
- **SOUL.md** (simplicity, no parallel systems); **`docs/architecture/`** (consistent with the
documented patterns); **`docs/spec/`** (behavior unchanged unless intended).
### Common Refactoring Checks
- Function signatures → all call sites updated.
- Type definitions → search type usages.
- Imports → correct after file moves.
- Stores → all consumers verified.
- Component props → all usages checked.
- Constants/enums → all references updated.
---
## Reference Documents
| Document | Purpose |
| --- | --- |
| [SOUL.md](../../../SOUL.md) | Project philosophy and core values |
| [AGENTS.md](../../../AGENTS.md) | AI onboarding and project context |
| [docs/spec/](../../../docs/spec/) | Behavioral contract — *what* the app does |
| [docs/architecture/](../../../docs/architecture/00-overview.md) | Architecture playbook — *how* it's built |
| [docs/IMPLEMENTATION-PLAN.md](../../../docs/IMPLEMENTATION-PLAN.md) | Milestone sequence and scope |
.claude/skills/doc-update/SKILL.md
+111
View File
@@ -0,0 +1,111 @@
---
name: doc-update
description: Update project documentation based on knowledge gaps discovered during the current session
disable-model-invocation: false
---
# Documentation Update from Session Context
Review the current session to identify knowledge gaps that caused suboptimal codebase
navigation, then update the relevant documentation.
## The quality bar
Every addition must pass this test: **"Would this save a future session at least 5 minutes
of exploration?"**
Documentation serves two purposes — know **where to look** and know **what to do**. Both
are valuable, but at different levels of detail:
- **Navigation map** (good): "Preview flow: `LivePreview.tsx` → `prepareSpecForRender()` (core) → `vega-embed`" — lists the files and their roles so you don't read a dozen files to find the right four.
- **Decision rule** (good): "The fit-mode/reference-resolution transform runs on a *copy* of the spec — never mutate the stored spec" — captures a non-obvious convention.
- **Code walkthrough** (bad): "SnippetStore.updateDraft sets draftSpec, which a startup subscriber watches, debounces, then calls snippetStore.put… " — restates the code, goes stale on any rename.
**Navigation maps** use file/module names (stable) to show flow direction. **Decision
rules** capture "when/why" constraints. **Code walkthroughs** restate implementation
details — that's what reading the code is for.
For documentation organization, see **[CLAUDE.md](../../../CLAUDE.md)** and the doc index in
**[AGENTS.md](../../../AGENTS.md)**.
## The three documentation layers (know which one a gap belongs to)
- **`docs/spec/`** — the *what*: behavioral contract (what the app does, acceptance points).
This is a **contract**. Only change it when product behavior genuinely changes, and do so
deliberately — never as a casual "fill a doc gap" edit. A how-detail does NOT belong here.
- **`docs/architecture/`** — the *how*: the patterns behind each layer (state, persistence,
modals, routing, rendering, inference, relationships). Most navigation maps and decision
rules land here.
- **`docs/IMPLEMENTATION-PLAN.md`** — the *when*: milestone sequence and scope.
## Process
### 1. Analyze the session
Look back through the conversation and identify:
- **Missing navigation maps**: Where did you read many files to discover which 3–4 files a
flow actually involves? A one-line map of file roles would have saved that.
- **Missing rules**: What conventions or constraints were discovered that a new session
would violate or re-discover?
- **Non-obvious "when/why" knowledge**: What decisions require understanding intent, not
just implementation?
Produce a brief list of gaps before proceeding. For each, state what's needed in one
sentence — a navigation map ("X flow: file → file → file") or a decision rule ("X must/must
not do Y"). If you can't state it concisely, it may be too implementation-specific to document.
### 2. Categorize and target
Map each gap to the right document:
| Gap type | Target document |
| --- | --- |
| Product behavior, capabilities, acceptance points | `docs/spec/` (the relevant 00–10 section) — **contract; change deliberately** |
| State / Zustand stores | `docs/architecture/01-state-and-stores.md` |
| Persistence, IndexedDB, localStorage, migrations | `docs/architecture/02-persistence.md` |
| Modals, dialog lifecycle | `docs/architecture/03-modal-system.md` |
| URL routing, keyboard/events | `docs/architecture/04-routing-and-events.md` |
| Rendering, theming, vega-embed, preview | `docs/architecture/05-rendering-theming-preview.md` |
| Type inference, dataset profiling | `docs/architecture/06-type-inference.md` |
| Names, snippet↔dataset links, rename propagation | `docs/architecture/07-naming-and-relationships.md` |
| Milestone scope, build order | `docs/IMPLEMENTATION-PLAN.md` |
| Project philosophy / identity | `SOUL.md` |
| Onboarding, conventions, stack | `AGENTS.md` / `CLAUDE.md` |
If a gap fits no existing document, consider a new section in the closest one; prefer
extending over creating. A brand-new architecture topic can become `docs/architecture/08-*.md`
(add it to `docs/architecture/00-overview.md`).
### 3. Read, locate, and check for bloat
For each target document:
- Confirm the gap isn't already covered (if partially covered, extend rather than duplicate).
- Find the right insertion point.
- **Check section length**: if a section is already long (>50 lines), tighten or consolidate
before adding. Documentation that only grows becomes noise.
### 4. Apply updates
- **Rules and constraints over descriptions**: "X must do Y because Z" beats "X works by A, B, C".
- **Stability over specifics**: no line numbers, no file counts, no volatile details.
- **Proportional**: a missing sentence doesn't need a new section; a missing concept does.
- **Match existing style**: follow surrounding formatting, heading levels, tone.
- **Self-contained**: never add a pointer that requires an external repository to follow.
- **Consolidate while adding**: net size increase should be minimal.
### 5. Update index (if needed)
Only update `CLAUDE.md` (or `docs/architecture/00-overview.md`) if a new document was created
or a major new section was added that should be discoverable. Not for minor additions.
## What NOT to document
- **Code walkthroughs**: prose that restates the code; goes stale on any rename. (Navigation
maps that list file roles are fine.)
- **Obvious-from-code patterns**: if reading the file makes it clear, don't add docs.
- **Session-specific context**: current task details, debugging steps taken.
- **Speculative patterns**: only document conventions confirmed across multiple instances.
- **Implementation details that change with refactoring**: if renaming a variable would
invalidate the doc, it's too specific.
.claude/skills/release/SKILL.md
+94
View File
@@ -0,0 +1,94 @@
---
name: release
description: Bump the app version, update the changelog, and prepare a git tag for release
disable-model-invocation: false
---
# Release
Bump the app version, update the changelog, and prepare a git tag.
## Process
### 1. Determine what changed since the last version
Run `git log` from the last version tag (or all history if no tags exist) and review the
changes. Categorize:
- **Features**: new user-facing capabilities
- **Fixes**: bug fixes
- **Improvements**: performance, UX polish, refactoring that affects behavior
- **Internal**: refactoring, docs, tests, build (don't list individually — summarize if substantial)
### 2. Determine bump type
Read the current version from `package.json`. The project uses **simplified semver during
pre-1.0**:
| Bump | When | Example |
| --- | --- | --- |
| **Minor** (`0.x.0`) | New features, UI changes, behavior changes | `0.1.0` → `0.2.0` |
| **Patch** (`0.x.y`) | Bug fixes, polish, performance, internal | `0.1.0` → `0.1.1` |
| **Major** (`1.0.0`) | Only when declaring public stability (user decision) | — |
Present the categorized changes and your recommended bump type to the user **for confirmation
before proceeding**.
### 3. Update version
Update the `version` field in `package.json`. This is the **single source of truth** — Vite
injects it as `__APP_VERSION__` at build time (shown in the header badge, and in Settings /
the export envelope once those exist).
### 4. Update the changelog
Maintain `docs/CHANGELOG.md`. If it doesn't exist yet, create it with a top-level `# Changelog`
heading. Add a new entry under a month heading:
```markdown
## June 2026
### v0.2.0
- **New feature** — description…
### v0.1.1
- **Bug fix** — description…
```
- Group by feature/change, not by commit.
- Lead with the name in bold, then a dash and description.
- Most important changes first; summarize related commits into coherent items.
### 5. Cross-check user-facing docs
Scan hand-maintained user-facing surfaces against the changes landing in this release and fix
drift **before** the version-bump commit (the CHANGELOG entry is not the place to quietly slip
in doc fixes):
- `README.md` — the status line and any feature claims still accurate?
- Any in-app help / about / onboarding content that exists at release time.
(When narrative content pages are added later, list them here so this net catches accumulated
drift across many changes.)
### 6. Suggest the git tag
After the user stages and commits the version bump, suggest:
```bash
git tag v{version}
git push --tags
```
If/when a deploy pipeline is wired, note here what publishing the tag triggers.
## Rules
- **Never bump the version without user confirmation** on the bump type.
- **Never stage or commit** — the user handles git operations.
- The export-format `version` (the import/export envelope) is **independent** of the app
version — only bump it when the export schema actually changes.
- Keep commit subjects single-line; no `Co-Authored-By` trailers (the user handles the commit
regardless).
.gitignore
+9
View File
@@ -0,0 +1,9 @@
node_modules
dist
dist-ssr
dev-dist
*.local
.DS_Store
*.log
coverage
.vite
.prettierrc
+6
View File
@@ -0,0 +1,6 @@
{
"singleQuote": true,
"trailingComma": "all",
"printWidth": 100,
"semi": true
}
AGENTS.md
+107
View File
@@ -0,0 +1,107 @@
# Astrolabe Project
> **Purpose**: Onboarding document for AI agents (and humans) working on Astrolabe.
---
## Project Overview
**Astrolabe** is a browser-based snippet manager for Vega-Lite visualizations. A user keeps
a local library of **snippets** (saved Vega-Lite specs), edits each as JSON with live
validation and a live chart preview, and reuses **datasets** across many snippets. Fully
local, offline-capable, no account.
It is a **spec-driven rebuild** on an architecture adapted from its sibling project Syto.
The authoritative behavioral contract is **`docs/spec/`** (sections 00–10). Implement *to
the spec*; do not port legacy code.
### Technical Stack
| Layer | Technology |
|-------|------------|
| Build | Vite, TypeScript, Vitest (happy-dom) |
| UI | React, Zustand, CSS Modules |
| Editor | Monaco (JSON + Vega-Lite schema service) |
| Charts | Vega-Lite + vega-embed |
| Storage | IndexedDB (snippets, datasets), localStorage (settings/prefs), URL hash (view state) |
| Offline | `vite-plugin-pwa` (Workbox), `registerType: 'prompt'` |
---
## Architecture (non-negotiable)
- **`src/core/` is portable** — no browser APIs, no React, no Monaco. All spec operations
live here and are tested hardest.
- **`src/app/`** — React + Zustand UI. State in Zustand **stores**; browser specifics in
**`src/app/infrastructure/`** adapters (IndexedDB / localStorage / URL hash). The rest of
the app never touches `window`/`indexedDB` directly.
- **Modals** go through a registry + coordinator + shell, not ad-hoc rendering.
- **CSS Modules + design tokens** (`styles/tokens.css`); themes flip `[data-theme]`.
- **No shared library with Syto** — patterns are adapted, never imported.
See [`docs/architecture/`](docs/architecture/00-overview.md) for the patterns behind each
layer (state, persistence, modals, routing, rendering, inference, relationships) and
`docs/IMPLEMENTATION-PLAN.md` for the milestone sequence. Both are **self-contained** — no
external repo is needed to work from them.
---
## Directory Structure
```
src/
├── core/ # Portable spec engine (no browser/React/Monaco)
├── app/
│ ├── components/ # React UI (CSS Modules co-located)
│ ├── stores/ # Zustand stores
│ ├── services/ # Business logic
│ └── infrastructure/ # IndexedDB, localStorage, URL hash adapters
styles/ # Global CSS (tokens, base)
docs/
├── spec/ # Authoritative behavioral specification (00–10) — the WHAT
├── architecture/ # Architecture playbook (00–08) — the HOW (self-contained)
└── IMPLEMENTATION-PLAN.md
```
---
## Development
```bash
npm run dev # Dev server
npm run build # Typecheck + production build (+ PWA)
npm run typecheck # tsc --noEmit
npm test # Vitest (run once)
npm run test:watch # Vitest watch
npm run format # Prettier
```
### AI Developer Protocol
- **No git on your own initiative** — don't `add`/`commit`/`push` unless explicitly invited.
- **Verify** — run `npm run typecheck` and `npm test` after changes.
- **Spec is the contract** — when in doubt, read `docs/spec/`. If the spec is wrong or
silent, raise it; change the spec deliberately rather than drifting from it.
- **Core-first** — for each feature, build the pure `src/core/` logic with tests before UI.
### Project skills
Invoke with `/<name>` (defined in `.claude/skills/`):
- **`/alignment`** — review uncommitted/staged changes for quality, test coverage, and
alignment with `docs/spec/` + `docs/architecture/`. Fixes issues directly and leaves
`// TODO:` breadcrumbs at code sites for out-of-scope observations.
- **`/doc-update`** — capture session-discovered knowledge gaps into the right doc layer
(`docs/spec/` for behavior, `docs/architecture/` for patterns).
- **`/release`** — bump version, update the changelog, prepare a git tag.
### Versioning
Simplified semver `0.x.y` (pre-1.0): minor for features/behavior, patch for fixes. Single
source of truth is `version` in `package.json`, injected as `__APP_VERSION__`.
### Testing Philosophy
High coverage on `src/core/` (parsing, detection, profiling, reference resolution, fit
transforms, import normalization). Lighter on components. Extract testable logic out of
components into core/stores where practical.
CLAUDE.md
+32
View File
@@ -0,0 +1,32 @@
# Claude Context — Astrolabe
See @AGENTS.md for project overview, architecture rules, and the AI developer protocol.
## Documentation Index
- **[SOUL.md](SOUL.md)** — project philosophy and identity. _Read first._
- **[docs/spec/](docs/spec/)** — authoritative behavioral specification (sections 00–10):
the **what**. This is the contract; implement to it.
- **[docs/architecture/](docs/architecture/00-overview.md)** — architecture playbook
(00–08): the **how** (state, persistence, modals, routing, rendering, inference,
relationships, vega-editor techniques). Self-contained — no external repo needed.
- **[docs/IMPLEMENTATION-PLAN.md](docs/IMPLEMENTATION-PLAN.md)** — incremental milestone
plan (M0–M6), MVP boundary, per-milestone tests + manual checks, and an architecture
reference index.
- **[AGENTS.md](AGENTS.md)** — onboarding, stack, directory map, scripts, conventions.
## Quick Orientation
- Astrolabe is a **spec-driven rebuild** — the behavior is fixed in `docs/spec/`; the
architecture is adapted from Syto. Implement to the spec; don't port legacy code.
- **`src/core/` is portable and tested hardest.** Browser specifics live in
`src/app/infrastructure/`. UI is React + Zustand.
- **Editor is Monaco**, charts render via **vega-embed**, storage is **IndexedDB**.
- Work milestone by milestone (see the plan): core-first, then UI, then tests, then a
manual smoke check against the spec's acceptance points.
## Conventions
- No git actions unless explicitly invited.
- Run `npm run typecheck` and `npm test` after changes.
- Single-line commit subjects; no Co-Authored-By trailers.
README.md
+33
View File
@@ -0,0 +1,33 @@
# Astrolabe
A browser-based **snippet manager for [Vega-Lite](https://vega.github.io/vega-lite/)
visualizations**. Author chart specs as JSON, watch them render live, and keep a personal,
searchable library — fully local, offline-capable, no account.
> Astrolabe is a **spec-driven rebuild** on an architecture adapted from its sibling
> project Syto. The authoritative behavioral contract lives in [`docs/spec/`](docs/spec/).
> See [SOUL.md](SOUL.md) for the philosophy and [docs/IMPLEMENTATION-PLAN.md](docs/IMPLEMENTATION-PLAN.md)
> for the build sequence.
## Stack
Vite · TypeScript · React + Zustand · Monaco · Vega-Lite + vega-embed · IndexedDB · PWA (offline + installable).
## Develop
```bash
npm install
npm run dev # dev server
npm run build # typecheck + production build (+ service worker)
npm run typecheck
npm test # Vitest
```
## Status
**M0 — Skeleton.** Toolchain green (typecheck, tests, build, PWA). The three-pane shell
renders; features land milestone by milestone per the implementation plan (MVP at end of M1).
## License
TBD.
SOUL.md
+102
View File
@@ -0,0 +1,102 @@
# Astrolabe — What This Project Is About
## The Problem
People who work with [Vega-Lite](https://vega.github.io/vega-lite/) directly — analysts,
educators, chart authors — don't have a fast, private place to *keep* their charts. The
official Vega-Lite editor is great for a single spec in a tab, but it forgets everything
when you close it. Notebooks bury charts in code. BI tools hide the spec behind a GUI and
lock you into an account.
Astrolabe fills this gap: **a local-first workspace where you author Vega-Lite specs as
JSON, see them render live, and keep a personal, searchable library of them — with no
account, no server, and full offline use.**
## The Core Idea
The central artifact is the **snippet**: a saved Vega-Lite specification plus metadata.
Everything else — the editor, the live preview, the dataset library, the chart builder —
exists to author, organize, and reuse snippets. Astrolabe does not abstract Vega-Lite
away; a snippet *is* a Vega-Lite spec. The chart builder offers a no-JSON on-ramp, but the
JSON is always the source of truth and always editable.
Reusable **datasets** are stored once and referenced by name from many snippets, so the
data lives in one place and the specs stay lean.
## Core Values
### 1. Local-Only by Default
Everything runs in the browser. Snippets, datasets, and settings never leave the machine.
No accounts, no uploads, no tracking. The only outbound requests are user-created
URL-dataset fetches.
### 2. Vega-Lite Native, Not Vega-Lite Hidden
The product domain *is* Vega-Lite. We validate, render, and reason about specs as
Vega-Lite, and we surface its real vocabulary (marks, encodings, field types). We don't
invent a parallel chart abstraction. The chart builder is an on-ramp, not a replacement
for the spec.
### 3. Experiment Safely
A snippet carries a stable **published** spec and an editable **draft**. You can tinker
freely without losing a known-good version. Auto-save protects in-progress work; publish
promotes it deliberately.
### 4. Beginner On-Ramp, Power-User Ceiling
The chart builder lets someone produce a chart without writing JSON. The editor — with
schema-aware autocomplete and live validation — lets a power user do anything Vega-Lite
can. Neither caps the other.
### 5. Own Your Data
Fully local and offline-capable, with import/export for backup and transfer. Your library
is a file you control, not a row in someone's database.
### 6. Predictable, Not Clever
When a behavior could go several ways, pick the one closest to the user's existing mental
model (the Vega-Lite editor, JSON tooling, file-based apps). Least surprise beats most
clever.
## What We're Not
- **Not a BI/dashboarding tool.** A snippet is *one* visualization, not a composed report
with cross-filters and layout. Dashboards are a different product.
- **Not a data-wrangling tool.** Datasets are stored and referenced, not cleaned or
transformed. (That's [Syto](https://github.com/) territory — Astrolabe's sibling in
architecture and quality bar, but a separate product with separate goals.)
- **Not a collaboration platform.** No multi-user, no sync, no comments. Import/export
moves data between machines.
- **Not a server app.** No backend, no rendering service, no account system.
## Technical Philosophy
### Spec-Driven, Clean Implementation
The behavioral contract lives in `docs/spec/`. Astrolabe is a deliberate rebuild on a
robust architecture (adapted from Syto): we implement *to the spec*, not by porting old
code. When the spec and convenience conflict, the spec wins or the spec changes — never
silent drift.
### Portable Core, Thin Browser Shell
`src/core/` is pure and portable — no browser APIs, no UI framework. Spec operations
(detection, profiling, reference resolution, fit transforms, validation, import
normalization) live there and are tested hardest. The UI is a thin, replaceable shell over
that core.
### Leverage Existing Libraries
Vega-Lite renders. Monaco edits. vega-embed mounts charts. React + Zustand drive the UI.
We wrap these with thin integration layers rather than reinventing them. Custom code
focuses on what's unique to Astrolabe: the snippet/dataset model, the rendering contract,
and the workspace that ties it together.
### No Parallel Systems
Each fact lives in one place. A snippet↔dataset link, a setting, a schema — one source of
truth, others derived. If you're writing the same logic twice, one should import or be
generated from the other.
### Test the Core, Trust the UI
High coverage on the portable engine (where a bug corrupts data or breaks rendering);
lighter coverage on components (where a bug is a cosmetic annoyance).
## The Name
An **astrolabe** is an ancient instrument for locating and predicting the positions of
stars — a tool for finding your way by the sky. The app helps you find your way through a
library of visualizations: keep them, locate them, and see where each one points.
docs/IMPLEMENTATION-PLAN.md
+288
View File
@@ -0,0 +1,288 @@
# Astrolabe — Incremental Implementation Plan
> A spec-driven rebuild of Astrolabe on Syto's architecture. The authoritative
> behavioral contract is `docs/spec/` (sections 00–10). This document sequences
> the build into the **quickest path to a usable MVP**, then layers the rest.
>
> **Method per milestone:** build core-first (portable, pure, tested) → wire UI →
> cover with tests → manual smoke check against the spec's acceptance points.
> "Test the Core, Trust the UI": high coverage on `src/core/`, lighter on components.
---
## Architectural ground rules
These are decided and apply to every milestone. The **how** behind each is written up
self-containedly in [`docs/architecture/`](architecture/00-overview.md) — read the matching
doc before implementing.
- **`src/core/` is portable** — no browser APIs, no React, no Monaco. Pure spec
operations (detection, profiling, reference resolution, fit transforms,
validation, import normalization). This is what we test hardest and what could
power a future headless renderer/CLI.
- **`src/app/`** holds React + Zustand UI. State lives in Zustand **stores**
(`useAppStore`, plus per-feature stores); browser specifics live in
**`src/app/infrastructure/`** adapters (IndexedDB, localStorage, URL hash) so
the rest of the app never touches `window`/`indexedDB` directly.
- **Modals via a registry + coordinator + shell** (see [Architecture 03](architecture/03-modal-system.md)),
not ad-hoc conditional rendering.
- **CSS Modules + design tokens** (`styles/tokens.css`); themes flip
`[data-theme]`. Vega theme follows the UI theme.
- **Editor: Monaco**, **self-hosted from npm + raw `monaco-editor` API** (not the
CDN loader / `@monaco-editor/react` wrapper — decided; rationale in
[Architecture 08](architecture/08-vega-editor-techniques.md#decision--monaco-integration-self-hosted-raw-api)).
Workers are wired explicitly via Vite `?worker`. The Vega-Lite JSON-schema
service is what gives autocomplete + validation; mine vega-editor for how it
wires the schema.
- **No shared library with Syto.** Patterns are copied/adapted, never imported.
---
## Milestone map
| # | Milestone | Outcome | Spec |
|---|-----------|---------|------|
| **M0** | Skeleton ✅ | Repo builds, tests run, empty shell renders | — |
| **M1** | **MVP core loop** | Author a Vega-Lite snippet, see it render live, it persists | §02, §03A–C, §04, §09A |
| **M2** | Editor robustness | Draft/Published, validation, schema autocomplete, fit modes | §03D–E, §04, §07(editor) |
| **M3** | Datasets | Named reusable data + reference resolution in preview | §05, §03F, §09B |
| **M4** | Chart Builder | No-JSON chart composition from a dataset | §06 |
| **M5** | Settings + Import/Export | Preferences + workspace backup/transfer | §07, §08, §09C |
| **M6** | Shell polish | Resize/toggle panes, routing, shortcuts, toasts, a11y, offline | §01, §10 |
**MVP boundary = end of M1** (a genuinely usable single-user chart authoring loop).
M2 makes it *robust*; M3–M6 make it *complete*. Ship/dogfood after M1, iterate.
---
## M0 · Skeleton ✅ (done)
Vite + React + Zustand + TypeScript + Vitest (happy-dom) + vite-plugin-pwa.
`src/core` ↔ `src/app` split, `useAppStore`, design tokens, three-pane placeholder
shell, first core module (`format-detection`) with tests.
**Verified:** `npm run typecheck`, `npm test` (14 passing), `npm run build` (PWA SW generated).
---
## M1 · MVP core loop → *the quickest usable Astrolabe*
**Goal:** select/create a snippet, edit its spec JSON, watch a live Vega-Lite
preview, and have it survive reload. Single source kind: inline-data specs only
(datasets come in M3). No draft/published yet — edits save directly.
**Core (`src/core/`)**
- `snippet.ts` — the Snippet type (spec §09A) + factory (`createSnippet`,
default sample bar-chart template, auto-generated date/time name).
- `rendering.ts` — `prepareSpecForRender(spec, { fitMode })` skeleton; in M1 it's
near pass-through (reference resolution is a no-op until M3, fit-mode is M2).
Establish the "transform a copy, never mutate stored spec" contract now.
**Infrastructure (`src/app/infrastructure/`)**
- `idb.ts` — thin IndexedDB wrapper (open, get/put/delete/getAll by store).
*(see [Architecture 02 · Persistence](architecture/02-persistence.md))*
- `snippet-store.ts` — persist snippets (object store `snippets`).
**App**
- `stores/SnippetStore.ts` — `useSnippetStore` with `snippets`, `activeSnippetId`,
selector-derived `activeSnippet`; load-on-startup; create/select/delete/update actions
(debounced auto-save of edits, spec §03B). Seed one sample snippet on first run.
- `components/SnippetLibrary.tsx` — list + "Create New" pinned item + select/delete.
- `components/SpecEditor.tsx` — Monaco JSON editor bound to active snippet's spec;
debounced write-back to the store. (Worker wiring via Vite `?worker` imports —
mine vega-editor's Monaco setup.)
- `components/LivePreview.tsx` — render current spec via `vega-embed` (actions:
false), debounced; clean empty pane when no/blank spec; basic error text.
- Fill the three panes in `App.tsx` with these.
**Tests (core-first)**
- `snippet.test.ts` — factory defaults, sample template validity, unique naming.
- `rendering.test.ts` — copy-not-mutate invariant; pass-through shape.
- A store test for create/select/delete/auto-save reducer logic (logic extracted
from the component so it's testable without DOM).
**Manual checks**
- Fresh load seeds a sample snippet that renders a bar chart.
- Type in the editor → preview updates after the debounce; bad JSON → editor keeps
working, preview shows an error, recovers when fixed.
- Reload → snippets and selection persist.
---
## M2 · Editor robustness
**Goal:** the editor becomes trustworthy — draft vs published, schema-aware
assistance, and the fit-mode rendering contract.
**Core**
- `rendering.ts` — implement **fit-mode** transform (Original/Width/Height/Full →
Vega-Lite `"container"`), recursing into layered/concat/child specs (spec §04
Rendering Contract, step 2).
- `vega-lite-schema.ts` — provide the Vega-Lite JSON schema for Monaco's
validation/autocomplete (mine vega-editor for sourcing/versioning the schema).
**App**
- Snippet gains `spec` (published) + `draftSpec` (working) per §09A; editing
touches `draftSpec` only.
- `SpecEditor` header: Draft/Published toggle; **Publish** (promotes draft, recomputes
dataset refs — refs land in M3) + **Revert** (confirm dialog).
- Library list item: draft-vs-published **status indicator**.
- Monaco wired with the Vega-Lite schema → squiggles + autocomplete; inline error
surface in the editor pane (§03E).
- Preview **Fit control** (4 modes), persisted (`previewFitMode`).
**Tests**
- Fit-mode transforms for each mode incl. nested specs; copy-not-mutate.
- Draft/publish/revert reducer logic; "has unpublished changes" derivation.
**Manual checks**
- Edit draft, see status flip to "draft"; Publish → status clears; Revert →
draft restored with confirmation.
- Invalid spec shows inline error; autocomplete suggests Vega-Lite properties.
- Each fit mode resizes the chart as specified; choice survives reload.
---
## M3 · Datasets
**Goal:** named, reusable data that snippets reference by name; preview resolves
the reference.
**Core**
- `profiling.ts` — row/column counts, column names, **per-column type inference**
(number/text/date/boolean). *(see [Architecture 06 · Type Inference](architecture/06-type-inference.md))*
- `rendering.ts` — implement **dataset reference resolution** (§04 Rendering
Contract, step 1): `{data:{name}}` → inline values / raw text+format / URL+format,
recursing into sub-specs; "dataset not found" error.
- `dataset.ts` — Dataset type (§09B); name uniqueness helpers; rename-propagation
into referencing specs. *(see [Architecture 07 · Naming & Relationships](architecture/07-naming-and-relationships.md))*
**Infrastructure**
- `dataset-store.ts` — separate high-capacity IndexedDB store (§09E).
**App**
- `stores/DatasetStore.ts` + Datasets **modal** (list/detail panes, create form,
edit, delete, copy-reference) via the modal registry/coordinator.
- Snippet `datasetRefs` maintained on publish; library shows dataset icon +
Linked Datasets; dataset detail shows Linked Snippets (bidirectional name link, §09F).
- **Extract-to-Dataset** flow from the editor (§03F).
- URL-sourced datasets fetched at render time.
**Tests**
- Reference resolution per source/format incl. nested; not-found error.
- Profiling/type inference across mixed columns, nulls, booleans.
- Rename propagation; name-uniqueness + import-style auto-suffix.
**Manual checks**
- Create a dataset, reference it by name in a snippet → preview renders.
- Extract inline data → spec rewritten to a reference, dataset appears, links show both ways.
- Delete/rename a referenced dataset behaves per spec.
---
## M4 · Chart Builder
**Goal:** no-JSON chart composition from a dataset → a new snippet.
**Core**
- `chart-builder.ts` — pure spec assembler: (mark ∈ Bar/Line/Point/Area/Circle) +
channels (X/Y/Color/Size) with field types (Quantitative/Nominal/Ordinal/Temporal)
+ optional width/height → complete Vega-Lite spec with tooltips + named data ref
(§06 Output). Field-type defaults from inferred column type.
**App**
- Chart Builder **modal** (config pane + live preview pane), launched from a
selected dataset; default pre-population (first col→X, second→Y); validation
(≥1 channel); Create Snippet → new linked snippet becomes active.
**Tests**
- Spec assembly: mark/channel/type permutations, unmapped channels omitted,
width/height inclusion, field-type derivation, validation gate.
**Manual checks**
- Build a bar chart from a dataset in a few clicks; preview live-updates;
Create → new snippet opens and renders.
---
## M5 · Settings + Import/Export
**Goal:** preferences and whole-workspace backup/transfer.
**Core**
- `settings.ts` — UserSettings shape + defaults + load-with-fallback (§07, §09C);
unknown/missing values fall back silently.
- `import-normalize.ts` — accept envelope / bare array / single snippet / foreign
shapes; field mapping (`content`→spec, `draft`→draftSpec, `createdAt`→created);
tag `"imported"`; merge rules (append, id-collision reassign, dataset-name
auto-suffix, datasets-before-snippets) (§08).
- `export-envelope.ts` — build the `{version, exportedAt, exportedBy, snippets, datasets}` envelope.
**Infrastructure**
- `settings-store.ts` (localStorage); `ux-prefs` for sort + panel layout (§09D).
**App**
- Settings **modal** (Appearance/Editor/Performance/Formatting), Apply/Cancel/Reset,
dirty indicator; wire render-debounce + theme + date-format through to the app.
- Header **Import**/**Export** (direct file dialog / download, no modal).
- Date formatting util (smart/iso/custom) used by the library list.
**Tests**
- Import normalization across all accepted shapes; merge/collision/rename logic;
quota-overage messaging path. Envelope round-trip (export→import idempotence).
- Settings load-with-fallback for partial/unknown records.
**Manual checks**
- Change theme/debounce/date-format → takes effect; Cancel reverts; Reset confirms.
- Export → reimport into a populated workspace merges without overwrite; renames reported.
---
## M6 · Shell polish & non-functional
**Goal:** the workspace feels finished and meets §10.
- **Panes:** drag-resize handles with min widths; per-pane show/hide toggle strip;
widths + visibility persist (§01A, §09D).
- **Routing:** URL hash view-state (`#snippet-<id>`, `#datasets/...`) with Back/Forward;
restore on load (§01E). *(see [Architecture 04 · Routing & Events](architecture/04-routing-and-events.md))*
- **Shortcuts:** Cmd/Ctrl+Shift+N / +K / +S / +, / Esc via a single key router
(§01D). *(see [Architecture 04 · Routing & Events](architecture/04-routing-and-events.md))*
- **Toasts:** success/error/warning/info, stacking, auto-dismiss, reduced-motion (§01F).
- **Storage monitor** for the snippet tier (§02).
- **A11y:** modal focus trap + return, labelled icon buttons, contrast in both themes (§10).
- **Offline/installable:** verify the SW + manifest give a working offline + installed app.
- **About & Privacy** and **Donate** modals.
**Manual checks:** keyboard-only run-through; reload restores view from URL;
offline reload works; install as standalone; reduced-motion honored.
---
## Cross-cutting, do-as-you-go
- **i18n** (optional, deferred): if translation is wanted, split a portable i18n
registry (no React) from the app-layer bindings, mirroring the `core` ↔ `app`
boundary. M1–M6 can ship English-only with date formatting locale-aware (§10).
Don't retrofit later if avoidable — keep user-facing strings centralized from M1.
- **Versioning:** simplified semver `0.x.y`, `package.json` → `__APP_VERSION__`
(already wired). Bump per shippable milestone.
- **Docs trio:** keep `SOUL.md` / `AGENTS.md` / `CLAUDE.md` current as the app grows.
---
## Architecture reference
The **how** behind each milestone is documented self-containedly in
[`docs/architecture/`](architecture/00-overview.md) — no external repo needed:
| Need | Doc |
|------|-----|
| Zustand stores, selector derivations, debounced auto-save | [01 · State & Stores](architecture/01-state-and-stores.md) |
| IndexedDB wrapper, lazy loading, migrations, localStorage prefs, storage tiers | [02 · Persistence](architecture/02-persistence.md) |
| Modal registry + coordinator + shell, unsaved-change detection, focus trap | [03 · Modal System](architecture/03-modal-system.md) |
| URL hash view-state, keyboard routing, interactive-context detection | [04 · Routing & Events](architecture/04-routing-and-events.md) |
| vega-embed integration, theming, debounced preview, error display | [05 · Rendering, Theming & Preview](architecture/05-rendering-theming-preview.md) |
| Column type inference + dataset profiling | [06 · Type Inference & Profiling](architecture/06-type-inference.md) |
| Unique names + import auto-suffix, snippet↔dataset links, rename propagation | [07 · Naming & Relationships](architecture/07-naming-and-relationships.md) |
| Monaco setup, Vega-Lite schema service, editor patterns mined from vega/editor | [08 · Vega Editor Techniques](architecture/08-vega-editor-techniques.md) |
docs/WHY-A-SEPARATE-REBUILD.md
+204
View File
@@ -0,0 +1,204 @@
# Astrolabe → Syto Integration Analysis
> **Question:** Can Astrolabe (a browser-based Vega-Lite snippet manager) be integrated into
> Syto's functionality? This document compares the Astrolabe specification (`docs/spec/`)
> against Syto in its current state, and recommends an integration path.
>
> **Short answer:** Not as a wholesale port, and not as the "snippet manager" it is today —
> that framing collides with Syto's stated non-goals. But the *valuable parts* of Astrolabe
> (the Chart Builder, the generic spec→render pipeline, the schema-assisted JSON editor) map
> cleanly onto a Syto-native **"chart a model"** feature, and most of the supporting tech already
> exists in the codebase. The recommendation is **harvest, don't port** — and the framing decision
> needs a `SOUL.md` ruling first.
---
## 1. Executive Summary
| | |
|---|---|
| **Conceptual fit** | Partial. Astrolabe and Syto are both local-first, browser-only, Vega-Lite-using, three-pane-ish workspaces. But Astrolabe's *primary entity* (a saved chart spec) is a thing Syto deliberately does not have. |
| **Strategic fit** | **Conflicted.** `SOUL.md` explicitly lists "Not a BI/visualization platform — charts are for exploration during wrangling, not final output" as a non-goal, and "Do One Thing Well." A *snippet library* is chart-authoring-as-product. This is the central tension and must be resolved before any code. |
| **Technical fit** | **Good for the rendering/editing layer, poor for the data-model and shell layers.** Syto already ships Vega-Lite, vega-embed, CodeMirror 6, IndexedDB persistence, a settings system, URL-hash routing, and a far stronger type/schema engine than Astrolabe's profiler. The friction is in the *parallel systems* a verbatim port would introduce. |
| **Recommended path** | **Option B (harvest into a native "Visualize" feature).** Reuse Astrolabe's Chart Builder and rendering contract; bind them to Syto **Models** instead of a new "dataset" entity; drop the snippet-as-primary-entity, the draft/published workflow, the separate dataset library, and the separate import/export envelope. |
---
## 2. The Two Products Side by Side
| Dimension | **Astrolabe** | **Syto** |
|---|---|---|
| Core artifact | A **snippet** = a saved Vega-Lite spec + metadata | A **workflow** = a declarative transform pipeline over a Source |
| Primary verb | *Author & organize charts* | *Clean & reshape tabular data* |
| Data unit | **Dataset** (named blob: JSON/CSV/TSV/TopoJSON, inline or URL) | **Source** (immutable imported table) → **Model** (derived table) |
| Persistence | Snippets (~5 MB tier) + Datasets (high-capacity tier), both local | Sources + Models in IndexedDB (lazy row data), prefs in localStorage |
| Editor | JSON editor w/ Vega-Lite schema autocomplete + live validation | CodeMirror 6 — used for transform JSON + the expression language |
| Rendering | Renders *arbitrary user specs* via reference-resolution + fit-mode transforms | Renders *programmatically generated* EDA specs (`charts.ts`) |
| Shell | 3 panes: library · editor · preview, + modals | Ribbon + sidebar + data table + slide-panel/modal dialogs |
| Routing | URL hash: `#snippet-<id>`, `#datasets/...` | URL hash: active source/model/dialog |
| Export | One JSON envelope of all snippets + datasets | Workflow v2 JSON (transforms, topo-sorted) |
| Stack stance | Implementation-agnostic spec | Preact + Signals + Arquero + CSS Modules, fixed |
**The key observation:** Astrolabe's "dataset" is conceptually Syto's "Source," and the thing you
most want to chart in Syto — a cleaned, transformed **Model** — has *no equivalent in Astrolabe at
all*. Astrolabe charts static blobs; Syto produces living, recomputed tables. A naive port would
bolt a second, weaker data-library (Astrolabe datasets) next to Syto's existing one (Sources/Models),
which directly violates SOUL's **"No Parallel Systems"** value.
---
## 3. The Strategic Tension (resolve this first)
This is not a technical blocker; it is a product-identity decision, and per project convention
(`SOUL.md` is the arbiter for contract/design decisions) it must be settled before implementation.
**What `SOUL.md` currently says:**
- *"Do One Thing Well… It's not trying to become a spreadsheet, a statistical package, a visualization tool, or a database. The EDA features… exist to help users understand their data before transforming it — not to replace dedicated analysis tools."*
- *"Not a BI/visualization platform: Charts are for exploration during wrangling, not final output. Dashboards and reporting are a separate concern."*
A **snippet manager** — a personal, searchable, import/exportable *library of saved charts* — is
squarely "charts as final output" and "a visualization tool." Porting Astrolabe as-is would
contradict two written non-goals.
**However**, there is a reading that is fully *aligned* with the rest of SOUL:
- *"Beginner-Friendly, Not Beginner-Limited"* and *"Progressive Complexity"* — today a user can clean data but has **no way to turn the result into a shareable picture.** A chart is the natural last step of a wrangling session.
- *"Leverage Existing Libraries — Vega-Lite handles charts."* The infrastructure is already paid for.
- Astrolabe's **Chart Builder** (pick a mark, map columns → spec) is the *exact* beginner-friendly, no-JSON affordance Syto favors, and the live JSON editor is the power-user escape hatch.
**The decision to make:** Is "produce a chart as the output of a workflow" *part of* doing the one
thing well (wrangling ends in a usable artifact), or is it the BI/viz scope SOUL rejects?
Two coherent resolutions:
- **(A) Amend SOUL** to permit *single-chart output of a model* (not dashboards, not a chart library-as-product) — and integrate as a native feature (§6, Option B).
- **(B) Keep it separate** — Astrolabe stays its own thing, or lives as a sibling `/tools/` mini-app that merely *shares code* with Syto (§6, Option C). The main app's non-goals stay intact.
I recommend (A) with a tightly-scoped amendment, because the value lands precisely where Syto is
currently weakest (no output artifact), and because doing it natively avoids the parallel-systems
trap. But this is the user's call to make against SOUL.
---
## 4. Feature-by-Feature Reuse Map
Legend: 🟢 already exists / strong reuse · 🟡 partial, needs adaptation · 🔴 net-new build
| Astrolabe feature | Syto today | Verdict | Notes |
|---|---|---|---|
| **Vega-Lite rendering** | `charts.ts` + `vega-embed@7` render programmatic specs into DOM refs | 🟡 | Engine present; needs a *generic* "render this arbitrary spec" path + error surface. The hardcoded EDA specs don't help directly, but the rendering primitive does. |
| **Dataset-reference resolution** (`{data:{name}}` → contents, recursing into layers) | none | 🔴 | New, but small and pure — and in Syto it resolves to a **Model's data**, not a separate dataset store. |
| **Fit-mode transforms** (Original/Width/Height/Full via `"container"`) | none | 🔴 | Small, pure, copy-on-render spec rewrite. Directly portable. |
| **JSON spec editor** | CodeMirror 6 (`CodeMirrorEditor.tsx`, `JsonEditorModal.tsx`) + lint infra (`linters/`) | 🟡 | Editor & lint plumbing exist. Missing: a **Vega-Lite schema service** for autocomplete + validation. (Note: Astrolabe's "minimap" and "VS Light/Dark/High-Contrast" editor themes are Monaco-isms; Syto is on CodeMirror — those exact settings don't carry over.) |
| **Chart Builder** (mark + X/Y/Color/Size + field types → spec) | none | 🟡→🔴 | The single most valuable, most SOUL-aligned piece. Build it against a **Model's columns** using Syto's existing schema types. High reuse of the *dialog* pattern (registry + slide-panel/modal + debounced preview). |
| **Column profiling / type inference** | `schema-engine.ts` (integer/float/date/datetime/boolean/json) | 🟢 | Syto's engine **supersedes** Astrolabe's (number/string/date/boolean). Astrolabe→Vega field-type mapping (numeric→Quantitative, date→Temporal, else Nominal) layers on top trivially. |
| **Datasets library + manager modal** | Sources/Models already *are* the data library | 🔴 *(avoid)* | Do **not** build. Reuse Sources/Models. Building it = parallel systems. |
| **Snippet library** (search/sort/CRUD, draft vs published, status, tags, storage monitor) | none | 🔴 | The genuinely new persistent entity. Only needed if going full snippet-manager (not recommended). Draft/Published has no analog in Syto's undo/redo model. |
| **Settings** (editor/performance/formatting) | `ux-settings.ts` + settings dialog | 🟡 | System exists; add render-debounce + a couple of fields. Editor-theme/minimap fields are Monaco-shaped and mostly drop. |
| **Import/Export envelope** (snippets+datasets JSON) | Workflow v2 export/import | 🔴 *(avoid)* | A second export format competing with workflow v2. If charts are part of a workflow, they belong *in* the workflow spec or alongside it — not in a rival envelope. |
| **App shell / 3-pane layout** | Ribbon + sidebar + table + slide-panel | 🔴 *(avoid)* | Don't graft Astrolabe's shell. A chart view is a *mode/panel within* Syto's shell. |
| **URL-hash routing** | Hash routing for source/model/dialog | 🟡 | Reusable, but Astrolabe's `#snippet-…`/`#datasets/…` scheme would **collide**; must namespace under Syto's existing scheme. |
| **Keyboard shortcuts** | `EventRouter` owns Ctrl+S (save), Escape priority chain, etc. | 🟡 | **Collisions:** Astrolabe binds Ctrl+S (Publish) and Ctrl+K (Datasets). Syto already owns Ctrl+S. Must reconcile, not adopt verbatim. |
| **Offline / PWA / installable** | `vite-plugin-pwa` already configured | 🟢 | Free. |
| **i18n** | i18next, en/uk, namespaced | 🟢 | New strings go in a namespace; framework is there. |
| **Toasts** | Notification system exists | 🟢 | Reuse. |
**Reuse tally:** the *rendering, editing, persistence, settings, schema, i18n, PWA, and toast*
substrate is largely present. The *data-model, shell, routing-scheme, and lifecycle* layers of
Astrolabe are either redundant with Syto or actively conflicting and should be dropped.
---
## 5. Technical Friction Points (if ported verbatim)
1. **Parallel data library.** Astrolabe datasets vs Syto Sources/Models — two stores, two
profilers, two "named data" concepts. Violates *No Parallel Systems*. (The fix: charts reference
Models.)
2. **Parallel persistence + export.** A second IndexedDB store layout and a second JSON envelope
alongside workflow v2. Two backup formats for users to confuse.
3. **Draft/Published has no home.** Syto's non-destructive model is *pipeline steps + undo/redo*,
not a per-document draft/published toggle. Astrolabe's central editing model would be a third,
unrelated state concept.
4. **Shell mismatch.** Astrolabe's library·editor·preview triptych is a *whole app*. Syto's shell is
ribbon-driven with slide-panel dialogs. They don't compose; one must yield.
5. **Routing & shortcut collisions.** Hash schemes overlap; Ctrl+S/Ctrl+K already bound.
6. **Editor-feature gap.** Syto is on CodeMirror (no minimap, different theme model); Astrolabe's
settings assume Monaco. And neither today has a **Vega-Lite schema service** — that autocomplete/
validation is net-new work on either stack.
7. **TopoJSON / arbitrary-JSON data.** Syto Sources are *tabular*. Astrolabe datasets include
TopoJSON and arbitrary JSON. Charting a Model covers the tabular case; map/topology charts would
be out of scope unless Sources grow a non-tabular kind.
None of these are unsolvable — but every one of them is *work created by the port itself*, not by
the user value. That's the signature of "harvest, don't port."
---
## 6. Integration Options
### Option A — Full port (snippet manager inside Syto)
Bring Astrolabe over more-or-less intact: snippet library, dataset manager, draft/published, its
shell, its export.
- **Pros:** Fastest way to "have Astrolabe." Feature-complete chart authoring.
- **Cons:** Maximal parallel-systems debt (§5). Directly contradicts SOUL non-goals. Two data
libraries, two export formats, shell/routing/shortcut conflicts. **Not recommended.**
### Option B — Harvest into a native "Visualize" feature ✅ *recommended*
Add charting as the natural *output* step of a workflow, reusing Syto's own primitives:
- A **"Chart" / "Visualize"** action on a **Model** opens a **Chart Builder** (Astrolabe's mark +
X/Y/Color/Size + field-type controls), populated from the Model's columns and `schema-engine`
types.
- It produces a Vega-Lite spec rendered live via the existing `vega-embed`, using a ported
**reference-resolution + fit-mode** rendering contract where the named data resolves to the
**Model's rows**.
- Power users get the **JSON spec editor** (CodeMirror, with a Vega-Lite schema service added) as the
escape hatch — consistent with *Beginner-Friendly, Not Beginner-Limited*.
- The chart (its spec) is persisted **attached to the Model** (or to the workflow), not as a separate
snippet entity. Export rides along with workflow v2 (or a sibling field), not a rival envelope.
- **Dropped from Astrolabe:** separate dataset library, draft/published, snippet search/sort/tags,
storage monitor, its shell, its import/export, its routing scheme.
- **Pros:** No parallel systems. Lands value exactly where Syto is weak (no output artifact). Maximal
reuse of existing infra. Defensible against SOUL with a *narrow* amendment ("single-chart output of
a model," not dashboards/library).
- **Cons:** Requires the SOUL decision (§3). Loses Astrolabe features that depend on the
snippet/dataset model (TopoJSON/URL datasets, multi-snippet library). Net-new: schema service,
builder dialog, render contract.
### Option C — Sibling `/tools/` mini-app
Port Astrolabe as a self-contained app under `/tools/astrolabe/`, sharing only *code* (vega render
helpers, CodeMirror wrapper, i18n) with the main app — no AppStore/DialogStore coupling.
- **Pros:** Keeps the main app's non-goals pristine (it's a separate utility, like other tools).
Lower conceptual conflict. Astrolabe keeps its own model.
- **Cons:** Syto's `/tools/` layer is designed for *small, single-purpose* utilities; Astrolabe is a
full application — a stretch for that slot. Still carries Astrolabe's whole parallel data model,
just quarantined. "Integration" here means "co-located," not "unified" — limited synergy.
---
## 7. Recommendation
1. **Make the SOUL call first (§3).** Decide whether single-chart *output of a model* is in scope.
If **no**, stop here or pursue Option C as a quarantined sibling. If **yes**, amend SOUL with a
tight scope statement and proceed to Option B.
2. **Pursue Option B.** Harvest the three high-value, well-aligned pieces:
- the **Chart Builder** (bound to a Model, driven by `schema-engine` types),
- the **rendering contract** (reference-resolution + fit modes, resolving to Model data),
- the **schema-assisted JSON editor** (CodeMirror + a new Vega-Lite schema service).
3. **Drop the parallel-systems pieces:** separate dataset library, draft/published, snippet
library + storage monitor, separate import/export envelope, Astrolabe's shell and routing scheme.
4. **Reconcile, don't adopt,** the cross-cutting surfaces: fold settings into `ux-settings`,
namespace any new hash state under Syto's scheme, resolve the Ctrl+S/Ctrl+K shortcut collisions.
This delivers the genuinely useful core of Astrolabe — turning cleaned data into a chart, with a
beginner path and a power-user path — while staying true to *Do One Thing Well* and *No Parallel
Systems*, and reusing the infrastructure Syto has already built.
---
## 8. Open Questions for the User
- **SOUL scope:** Is "a chart as the output of a workflow" inside Syto's mission, or out? (Blocks everything.)
- **Persistence model:** Should a chart spec live *on a Model*, *in the workflow v2 export*, or as a new top-level entity?
- **Non-tabular data:** Do we ever need TopoJSON / arbitrary-JSON charts (maps), which Syto Sources can't currently hold? If not, that simplifies scope considerably.
- **Editor depth:** Is full Vega-Lite schema autocomplete/validation in scope, or is a plain JSON editor + live error surface enough for v1?
docs/architecture/00-overview.md
+50
View File
@@ -0,0 +1,50 @@
# Astrolabe — Architecture Playbook
> These documents capture the **architectural patterns** Astrolabe is built on. They are
> self-contained: everything needed to implement a pattern lives here, in Astrolabe's own
> domain terms (snippets, datasets, settings, Vega-Lite specs). You do not need any other
> repository to work from them.
>
> They are the architectural counterpart to [`docs/spec/`](../spec/): the **spec** says
> *what the app does* (behavior, acceptance points); this **playbook** says *how we build
> it* (state, persistence, modals, routing, rendering, inference, relationships).
## How to use this playbook
- Building a feature? Read the relevant spec section first (the *what*), then the matching
playbook doc (the *how*), then implement core-first per [`../IMPLEMENTATION-PLAN.md`](../IMPLEMENTATION-PLAN.md).
- Each doc states the pattern, the **rationale** (what problem it solves, what it prevents),
TypeScript sketches in Astrolabe terms, and Do/Don't rules.
- The sketches are *illustrative*, not finished code. Adapt them; keep the principles.
## The documents
| # | Doc | Covers |
|---|-----|--------|
| 01 | [State & Stores](01-state-and-stores.md) | Zustand stores; one source of truth; selector derivations; central `useAppStore` vs per-feature stores; testable action functions; debounced auto-save. |
| 02 | [Persistence](02-persistence.md) | The infrastructure-adapter boundary; promise-wrapped IndexedDB wrapper; lazy data loading; per-record schema versioning + migration; localStorage prefs with fallback; storage tiers + quota monitoring. |
| 03 | [Modal System](03-modal-system.md) | Registry + coordinator + shell; one modal at a time; unsaved-change detection via snapshot; focus trap; backdrop/Escape/close dismissal. |
| 04 | [Routing & Events](04-routing-and-events.md) | URL hash as view-state (restore/sync, Back/Forward); global keyboard routing; Escape priority chain; the single-source `isInInteractiveContext()` helper (Monaco-aware). |
| 05 | [Rendering, Theming & Preview](05-rendering-theming-preview.md) | vega-embed integration (`actions:false`, `view.finalize()`); field-name escaping; theme→config mapping; debounced non-blocking renderer; resilient error display. |
| 06 | [Type Inference & Profiling](06-type-inference.md) | Pure, portable column-type inference (number/text/date/boolean) and the dataset profile shape. |
| 07 | [Naming & Relationships](07-naming-and-relationships.md) | Unique-name enforcement + import auto-suffix; the bidirectional snippet↔dataset name link; rename propagation into specs. |
| 08 | [vega/editor Techniques](08-vega-editor-techniques.md) | Reference brief: borrowable Monaco-schema wiring, vega-embed lifecycle, two-tier validation, and data-flow/debounce techniques distilled from the official Vega-Lite editor — plus where we do better. |
## The non-negotiable layering (every doc assumes this)
- **`src/core/`** — portable, pure logic. No browser APIs, no React, no Monaco. Spec
operations live here and are unit-tested hardest. (Docs 06, 07, parts of 05 land here.)
- **`src/app/stores/`** — Zustand stores. (Doc 01.)
- **`src/app/infrastructure/`** — the *only* place that touches `indexedDB`, `localStorage`,
or `window.location`. Everything else goes through these typed adapters. (Docs 02, 04.)
- **`src/app/services/` & `orchestration/`** — coordination that composes stores +
infrastructure + core (lifecycle, routing sync, dependency upkeep). (Docs 03, 04, 07.)
- **`src/app/components/`** — React + CSS Modules. Thin; pushes logic down into stores/core
so it stays testable. (Docs 03, 05.)
## Why a playbook at all
Patterns written down once, in one place, stop two classes of problem: drift (the same
decision re-litigated inconsistently across features) and rediscovery (re-deriving why
something is the way it is). When a pattern here proves wrong, change the doc — don't fork
the convention silently. This is the same discipline `docs/spec/` applies to behavior.
docs/architecture/01-state-and-stores.md
+432
View File
@@ -0,0 +1,432 @@
# State Management & Stores
How Astrolabe holds and shares application state. The whole app is built on
**Zustand**: small, standalone stores created with `create()`, each exposing
state fields and the actions that mutate them. Components subscribe to the exact
slices they read; non-component code (services, infrastructure, orchestration)
reads and writes the same stores directly. This document defines how we use
Zustand, where state lives, and the rules that keep state predictable as the app
grows.
Why Zustand: it is idiomatic React (just a hook), it has a first-class **outside-React**
API (`getState`/`setState`/`subscribe`) that fits our "logic lives in core/services,
not components" architecture, and it carries no build-time magic. The principles below
(one source of truth, derive-don't-duplicate, actions outside components, thin components)
are the durable part — they would survive a change of library.
---
## 1. The Primitives
A store is a module that calls `create<State>()` once and exports the resulting
hook. The state object holds both **data fields** and **action functions**.
```ts
// src/app/stores/AppStore.ts
import { create } from 'zustand';
import type { UiTheme } from '@core/theme'; // defined in core; charts key off it too
export type ModalName = 'datasets' | 'settings' | 'about' | 'donate' | 'chartBuilder' | 'extract';
export interface AppState {
uiTheme: UiTheme;
activeModal: ModalName | null;
setTheme: (theme: UiTheme) => void;
// Low-level primitive. High-level open/close (snapshot, URL sync, discard
// prompt) is the modal coordinator's job — see docs/architecture/03.
setActiveModal: (modal: ModalName | null) => void;
}
export const useAppStore = create<AppState>((set) => ({
uiTheme: 'light',
activeModal: null,
setTheme: (uiTheme) => set({ uiTheme }),
setActiveModal: (activeModal) => set({ activeModal }),
}));
```
Three ways to touch a store:
- **`set(partial)`** — update state (shallow-merges). Inside actions, the only place
that mutates state.
- **`get()`** — read current state inside actions without subscribing.
- **the hook `useAppStore(selector)`** — read state *in a React component*, subscribing
to exactly what the selector returns.
### Reading in components — always select narrowly
Call the hook with a **selector** that returns the smallest thing you need. The
component re-renders only when that selected value changes (default `Object.is`
comparison).
```tsx
import { useAppStore } from '../stores/AppStore';
export function ThemeBadge() {
const theme = useAppStore((s) => s.uiTheme); // re-renders only when uiTheme changes
return <span>{theme}</span>;
}
```
When you select **multiple fields or a fresh object/array**, wrap the selector in
`useShallow` so a new-but-equal result doesn't cause an extra render:
```tsx
import { useShallow } from 'zustand/react/shallow';
const { activeModal, uiTheme } = useAppStore(
useShallow((s) => ({ activeModal: s.activeModal, uiTheme: s.uiTheme })),
);
```
### Reading/writing outside components
Services, orchestration, infrastructure, and tests use the store object directly —
no React involved. This is the property that lets our logic live outside components:
```ts
openModal('settings'); // via the modal coordinator (doc 03)
const theme = useAppStore.getState().uiTheme; // snapshot read
const unsub = useAppStore.subscribe((s, prev) => { /* react to changes */ });
```
> Rule: in components, **select narrowly** (and `useShallow` for object/array
> selections). Outside components, use `getState()` for a snapshot, `subscribe()`
> to react.
---
## 2. One Source of Truth per Fact — Derive, Don't Duplicate
Every fact lives in exactly one state field. Anything that can be *calculated*
from other state is computed **in a selector at read time**, never stored as a
second field you keep in sync by hand.
The failure mode this avoids: two fields that must agree (`snippets` and
`snippetCount`, or `activeSnippetId` and `activeSnippet`) drift apart because one
update path forgets the other. If the derived value is computed from the source on
read, drift is structurally impossible.
```ts
// State holds only the sources:
// snippets: Snippet[]
// activeSnippetId: string | null
// Derive in the component's selector — not a stored field:
const activeSnippet = useSnippetStore((s) =>
s.snippets.find((x) => x.id === s.activeSnippetId) ?? null,
);
const snippetCount = useSnippetStore((s) => s.snippets.length);
```
For a derivation that is **expensive** or reused in many places, expose it as a
selector function (memoize if profiling shows it matters) rather than caching it
into state:
```ts
// src/app/stores/snippet-selectors.ts
export const selectActiveSnippet = (s: SnippetState) =>
s.snippets.find((x) => x.id === s.activeSnippetId) ?? null;
// in a component:
const active = useSnippetStore(selectActiveSnippet);
```
> Rule: if you can compute it, do not store it. Add a new state field only for a
> value that is *input* the app receives, not output it derives.
---
## 3. Where State Lives: Central vs. Per-Feature Stores
Each store is its own `create()` module. We split by *concern*, not by component
tree.
### Per-feature stores
Each cohesive feature owns a store holding its durable domain state.
- **`useSnippetStore`** — the snippet library: `snippets`, `activeSnippetId`, the
working `draftSpec`, and its actions.
- **`useDatasetStore`** — loaded datasets, the active dataset, inferred fields.
- **`useSettingsStore`** — user preferences (editor options, render debounce,
date format, theme); mirrors what gets persisted to `localStorage`.
### The central `useAppStore`
`useAppStore` holds only *cross-cutting, ephemeral UI state* that no single
feature owns — which modal is open, the runtime theme, transient render flags.
### How to decide
| Put it in a **feature store** when… | Put it in **`useAppStore`** when… |
| --------------------------------------------- | --------------------------------------------- |
| It's domain data (snippets, datasets, specs) | It's transient UI chrome (open modal, theme) |
| It outlives a single interaction | It belongs to no single feature |
| It gets persisted | Multiple unrelated features read/write it |
> Rule: keep `useAppStore` small. When a chunk of it only ever serves one feature,
> that's the signal to extract a feature store. A bloated central store is the
> thing this split exists to prevent.
---
## 4. Actions: Mutations Live in the Store, Not Components
Components **render** and **dispatch**; they do not contain mutation logic. Every
state change goes through a named action defined on the store (via `set`/`get`).
Multi-step logic that coordinates several stores or touches infrastructure can
live in a `src/app/services/*` module that calls store actions.
```ts
// src/app/stores/SnippetStore.ts
import { create } from 'zustand';
import type { Snippet } from '@core/snippet';
interface SnippetState {
snippets: Snippet[];
activeSnippetId: string | null;
draftSpec: string; // Monaco editor buffer (Vega-Lite JSON)
create: (name: string) => string;
select: (id: string) => void;
remove: (id: string) => void;
updateDraft: (spec: string) => void;
reset: () => void;
}
export const useSnippetStore = create<SnippetState>((set, get) => ({
snippets: [],
activeSnippetId: null,
draftSpec: '',
create: (name) => {
const snippet: Snippet = { id: crypto.randomUUID(), name, spec: '{}' };
set((s) => ({ snippets: [...s.snippets, snippet] }));
get().select(snippet.id);
return snippet.id;
},
select: (id) =>
set((s) => ({
activeSnippetId: id,
draftSpec: s.snippets.find((x) => x.id === id)?.spec ?? '{}',
})),
remove: (id) =>
set((s) => {
const snippets = s.snippets.filter((x) => x.id !== id);
const activeSnippetId =
s.activeSnippetId === id ? (snippets[0]?.id ?? null) : s.activeSnippetId;
return { snippets, activeSnippetId };
}),
updateDraft: (draftSpec) => set({ draftSpec }),
reset: () => set({ snippets: [], activeSnippetId: null, draftSpec: '' }),
}));
```
The component is thin — it selects state and calls actions:
```tsx
import { useShallow } from 'zustand/react/shallow';
import { useSnippetStore } from '../stores/SnippetStore';
export function SnippetList() {
const { snippets, activeSnippetId } = useSnippetStore(
useShallow((s) => ({ snippets: s.snippets, activeSnippetId: s.activeSnippetId })),
);
const select = useSnippetStore((s) => s.select);
const remove = useSnippetStore((s) => s.remove);
return (
<ul>
{snippets.map((s) => (
<li key={s.id} aria-current={s.id === activeSnippetId} onClick={() => select(s.id)}>
{s.name}
<button onClick={(e) => { e.stopPropagation(); remove(s.id); }}>✕</button>
</li>
))}
</ul>
);
}
```
> Note: action identities are stable, so selecting them (`s.select`) never causes
> re-renders — select actions individually rather than bundling them into a
> `useShallow` object.
### Why mutations live in the store
- **Testable without a DOM.** Actions are plain functions over state. A Vitest test
calls `useStore.getState().create('x')` and asserts on `getState()` — no
rendering, no React.
- **One place to change behavior.** "Deleting the active snippet falls back to the
first remaining one" is a rule that lives in `remove`, not scattered across every
delete button.
- **Readable components.** A component that only wires events to named actions reads
like a description of the UI, not a tangle of state juggling.
```ts
// SnippetStore.test.ts — no browser needed
import { useSnippetStore } from './SnippetStore';
beforeEach(() => useSnippetStore.getState().reset());
test('deleting the active snippet selects the next one', () => {
const store = useSnippetStore.getState();
const a = store.create('A');
const b = store.create('B');
store.select(a);
store.remove(a);
expect(useSnippetStore.getState().activeSnippetId).toBe(b);
});
```
> Rule: no `setState` calls inside component bodies for shared state — call an
> action. Local, throwaway UI state (a dropdown's open flag) may stay in component
> `useState`; anything another component reads belongs in a store behind an action.
---
## 5. Effects: Persistence and External Sync
Cross-cutting reactions — persisting state, mirroring the theme onto the document,
pushing the draft into Vega for rendering — are wired once at app startup with
`store.subscribe(...)`, in the orchestration/startup layer, not in components.
Subscribers read state and write to `src/app/infrastructure/` adapters (IndexedDB,
`localStorage`, URL hash).
### Theme → document (the minimal example, already wired)
```ts
// src/main.tsx
const applyTheme = (t: string) => { document.documentElement.dataset.theme = t; };
applyTheme(useAppStore.getState().uiTheme);
useAppStore.subscribe((s, prev) => {
if (s.uiTheme !== prev.uiTheme) applyTheme(s.uiTheme);
});
```
The store stays DOM-free; the adapter (the `applyTheme` subscriber) lives at the edge.
### Debounced auto-save of the draft spec
The Monaco editor writes every keystroke into `draftSpec`. We do **not** persist on
every keystroke. A startup subscriber observes the draft and debounces the expensive
work:
```ts
// src/app/orchestration/persistence.ts
import { useSnippetStore } from '../stores/SnippetStore';
import { saveSnippet } from '../infrastructure/snippet-store'; // IndexedDB adapter
export function wireDraftAutoSave(): void {
let timer: ReturnType<typeof setTimeout> | undefined;
useSnippetStore.subscribe((s, prev) => {
if (s.draftSpec === prev.draftSpec) return; // only react to draft edits
const id = s.activeSnippetId;
if (!id) return;
clearTimeout(timer);
const spec = s.draftSpec;
timer = setTimeout(() => {
useSnippetStore.setState((cur) => ({
snippets: cur.snippets.map((x) => (x.id === id ? { ...x, spec } : x)),
}));
void saveSnippet(id, spec);
}, 400);
});
}
```
> For selector-based subscriptions (`subscribe(selector, listener)` with an equality
> function) add the `subscribeWithSelector` middleware to the store. Plain
> `subscribe((state, prev) => …)` as above is enough for most wiring.
> Rule: components never touch infrastructure adapters directly. Reads/writes to
> IndexedDB, `localStorage`, and the URL hash happen in startup subscribers or
> actions, so the persistence story is in one place and the UI stays pure.
---
## 6. Reading State: Import the Store, Don't Thread It
Because stores are singletons importable anywhere, a deep leaf component reads the
state it needs directly instead of receiving it through five layers of props.
```tsx
// Good: a deeply nested toggle reads + flips the theme itself.
import { useAppStore } from '../stores/AppStore';
export function ThemeToggle() {
const theme = useAppStore((s) => s.uiTheme);
const setTheme = useAppStore((s) => s.setTheme);
return (
<button onClick={() => setTheme(theme === 'experimental' ? 'light' : 'experimental')}>
{theme === 'experimental' ? '🌙' : '☀️'}
</button>
);
}
```
This is the right default for **global/shared** state. Threading `theme` and
`onThemeChange` through `Layout → Header → Toolbar → ThemeToggle` adds noise and
couples every intermediate component to data it doesn't use.
### When to thread props instead
- The value is **presentational input**, not shared app state. `<Button variant="primary">`
takes `variant` as a prop; it should not know about any store.
- The component is meant to be **reusable / store-agnostic** (design-system
components, list-item renderers given their item via prop).
- A parent supplies **per-instance** data, e.g. `<SnippetRow snippet={s} />` inside a
`.map()` — the row gets its snippet by prop but still calls
`useSnippetStore.getState().remove(...)` (or a selected action) for mutations.
> Rule of thumb: shared app state → select it from the store at the point of use.
> Per-instance or presentational data → pass it as a prop. Passing global state down
> as props is the anti-pattern to avoid.
---
## 7. Resetting State
Each store exposes a `reset()` action that returns its fields to initial values
(used on "new workspace", sign-out, or test teardown). Because every fact is a
single source field with no hand-maintained duplicates, reset is a flat `set(...)`
of the initial values; selector-derived values recompute on their own.
```ts
reset: () => set({ snippets: [], activeSnippetId: null, draftSpec: '' });
```
---
## Rules Summary
**Do**
- Keep one state field per fact; derive everything else in selectors, not stored fields.
- In components, **select narrowly**; use `useShallow` for object/array selections.
Outside components, use `getState()` / `subscribe()`.
- Split durable domain state into feature stores (`useSnippetStore`, `useDatasetStore`,
`useSettingsStore`); keep `useAppStore` for thin cross-cutting UI state.
- Put every shared-state mutation behind a named action on the store so it's testable
without a DOM (`getState().action()`).
- Do persistence and external sync (IndexedDB, `localStorage`, URL hash, theme) in
startup `subscribe` listeners via `infrastructure/` adapters.
- Debounce expensive reactions (auto-save, re-render) inside the subscriber.
- Import singleton store hooks directly in the leaves that need shared state.
**Don't**
- Don't store derived values as their own fields and sync them by hand.
- Don't call `setState` for shared state inside component render bodies — call an action.
- Don't select broad objects without `useShallow` (causes needless re-renders).
- Don't let `useAppStore` accumulate feature-specific state; extract a store.
- Don't touch IndexedDB/`localStorage`/URL adapters from components.
- Don't thread global state down through props; don't pass per-instance or
presentational data via store imports.
docs/architecture/02-persistence.md
+401
View File
@@ -0,0 +1,401 @@
# 02 · Persistence Architecture
How Astrolabe stores data in the browser, and the rules that keep that storage testable, portable, and safe to evolve. This document is the implementation contract for the persistence layer. For the *behavioral* data model (what fields a Snippet or Dataset has, what the tiers hold), see [09 · Data Model & Persistence](../spec/09-data-model.md); this document covers *how the code is structured to implement it*.
---
## 1. The Infrastructure-Adapter Principle
**Rule: nothing outside `src/app/infrastructure/` ever touches `indexedDB`, `localStorage`, `window`, or `location` directly.** Every browser-storage interaction goes through a typed adapter module that exposes plain async functions returning domain objects.
```
src/
├── core/ # portable engine — NO browser APIs, NO React
├── app/
│ ├── stores/ # Zustand stores; calls infrastructure, never IDB
│ ├── services/ # business logic; calls infrastructure, never IDB
│ └── infrastructure/ # the ONLY place that imports indexedDB/localStorage
│ ├── snippet-store.ts # IndexedDB: snippets (metadata + drafts)
│ ├── dataset-store.ts # IndexedDB: datasets (heavy payloads)
│ ├── settings-store.ts # localStorage: UserSettings
│ └── prefs-store.ts # localStorage: app/UI prefs (sort, layout)
```
### Why this boundary exists
- **Testability.** Stores and services depend on a small typed surface (`getSnippet(id): Promise<Snippet | null>`), not on the IndexedDB request API. Tests mock the adapter, not a browser global. The adapters themselves are tested directly against `fake-indexeddb` / a localStorage stub in Vitest.
- **Portability.** `src/core/` stays free of browser APIs so the spec/parse/transform logic can run in Node (tests, future CLI, SSR). The adapters are the seam where the portable core meets the browser.
- **Single place for migrations.** Schema upgrades and record migrations live in exactly one module per store. A reader looking for "how does v1 data become v2 data" has one file to open, not a scattered set of `if (record.someOldField)` checks across the UI.
- **Failure containment.** Quota errors, corrupt JSON, and missing keys are handled at the boundary and converted into typed results (or sane fallbacks), so the rest of the app never sees a raw `DOMException`.
> **Do:** `import { saveSnippet } from '@/app/infrastructure/snippet-store'`
> **Don't:** `indexedDB.open(...)` or `localStorage.getItem(...)` anywhere in a component, store, or service.
---
## 2. IndexedDB Wrapper
IndexedDB's native API is event-based (`onsuccess`/`onerror`) and verbose. The adapter wraps it into promises and exposes a tiny CRUD surface per object store. Define one shared helper and build typed stores on top of it.
### 2.1 Opening the database
Open with an explicit **version number** and an `onupgradeneeded` handler that creates/upgrades object stores. The version is a monotonically increasing integer; bump it whenever the *store layout* changes (a new object store, a new index). It is independent of per-record schema versions (§4).
```ts
// src/app/infrastructure/db.ts
const DB_NAME = 'astrolabe';
const DB_VERSION = 1;
let dbPromise: Promise<IDBDatabase> | null = null;
export function openDB(): Promise<IDBDatabase> {
// Memoize: opening is idempotent and cheap to share across calls.
if (dbPromise) return dbPromise;
dbPromise = new Promise((resolve, reject) => {
const req = indexedDB.open(DB_NAME, DB_VERSION);
req.onupgradeneeded = (event) => {
const db = req.result;
const oldVersion = event.oldVersion;
// Create stores idempotently — guard every create.
if (!db.objectStoreNames.contains('snippets')) {
db.createObjectStore('snippets', { keyPath: 'id' });
}
if (!db.objectStoreNames.contains('datasets')) {
db.createObjectStore('datasets', { keyPath: 'id' });
}
// Per-version store-layout migrations go here, gated on oldVersion.
// if (oldVersion < 2) { /* add index, split a store, ... */ }
void oldVersion;
};
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error ?? new Error('Failed to open IndexedDB'));
});
return dbPromise;
}
```
### 2.2 Promise-wrapped CRUD helpers
Wrap a single IDB request and a whole transaction so callers write linear `async/await` code.
```ts
// src/app/infrastructure/db.ts (continued)
function wrap<T>(req: IDBRequest<T>): Promise<T> {
return new Promise((resolve, reject) => {
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
}
async function tx<T>(
store: string,
mode: IDBTransactionMode,
run: (s: IDBObjectStore) => IDBRequest<T>
): Promise<T> {
const db = await openDB();
return new Promise<T>((resolve, reject) => {
const transaction = db.transaction(store, mode);
const request = run(transaction.objectStore(store));
transaction.oncomplete = () => resolve(request.result);
transaction.onerror = () => reject(transaction.error);
transaction.onabort = () => reject(transaction.error);
});
}
export const get = <T>(store: string, key: IDBValidKey) =>
tx<T | undefined>(store, 'readonly', (s) => s.get(key) as IDBRequest<T | undefined>);
export const getAll = <T>(store: string) =>
tx<T[]>(store, 'readonly', (s) => s.getAll() as IDBRequest<T[]>);
export const put = <T>(store: string, value: T) =>
tx<IDBValidKey>(store, 'readwrite', (s) => s.put(value as any));
export const del = (store: string, key: IDBValidKey) =>
tx<undefined>(store, 'readwrite', (s) => s.delete(key) as IDBRequest<undefined>);
```
> **Do:** resolve on `transaction.oncomplete`, not on the request's `onsuccess` — the write is only durable once the transaction commits.
> **Don't:** hold an IndexedDB transaction open across an `await` to non-IDB work; transactions auto-close when the microtask queue drains and you'll get `TransactionInactiveError`.
---
## 3. Lazy Loading: Metadata vs Heavy Payloads
A snippet library can grow large, and **datasets can be megabytes each** (CSV text, parsed TopoJSON). Loading every dataset payload at startup just to render a list of names is wasteful and slow. The rule:
> **Store record metadata separately from large payloads. Load heavy data on demand. Treat `null` as "exists but not loaded yet" — distinct from absent.**
For Astrolabe this maps cleanly onto the two stores:
- **`snippets`** — snippet records are small (a spec is JSON text). They load eagerly as a set when the library opens.
- **`datasets`** — the `data` payload is the heavy part. The list view needs only the derived summary fields (`name`, `format`, `source`, `rowCount`, `columnCount`, `columns`, `size`, timestamps). Load `data` only when a snippet that references the dataset is actually previewed.
There are two ways to implement the split; pick per store:
1. **Two object stores** (`datasets` for metadata, `dataset-payloads` keyed by the same id for `data`) — strongest separation; a `getAll` on metadata never touches payload bytes.
2. **One store, lazy field** — keep `data` in the record but set it to `null` on the bulk list load and fetch it per-id on demand.
Astrolabe uses the **lazy-field** approach for datasets (one store, simpler), with `data === null` signalling "summary loaded, payload not yet."
```ts
// src/app/infrastructure/dataset-store.ts
import { get, getAll, put } from './db';
import { migrateDataset, type Dataset } from './dataset-migrations';
/** List view: returns every dataset's summary, payload nulled out. */
export async function loadDatasetSummaries(): Promise<Dataset[]> {
const records = await getAll<Dataset>('datasets');
return records.map((r) => ({ ...migrateDataset(r), data: null }));
}
/** Detail/preview: load (or return cached) full payload for one dataset. */
export async function ensureDatasetData(dataset: Dataset): Promise<Dataset['data']> {
if (dataset.data !== null && dataset.data !== undefined) return dataset.data; // already loaded
const record = await get<Dataset>('datasets', dataset.id);
dataset.data = record?.data ?? null;
return dataset.data;
}
```
> **Do:** use `null` for "not loaded" and a real value (including `''` or `[]`) for "loaded but empty." The distinction prevents a re-fetch loop.
> **Don't:** overwrite a stored payload with `null` on save. When persisting a record whose `data` is `null` (never loaded into memory), skip writing the payload field and leave the stored bytes intact — otherwise a list-load-then-save round-trip silently destroys data.
---
## 4. Per-Record Schema Versioning & Read-Time Migration
The IndexedDB **database version** (§2.1) governs *store layout*. A separate **per-record `version` field** governs the *shape of an individual record*. Both Snippet and Dataset records carry `version` (and `created` / `modified` timestamps). This lets record shapes evolve without forcing an `onupgradeneeded` database bump for every field rename.
Migrations are applied **on read** — when a record comes out of the store, run it through a migration function that upgrades it to the current shape before the app sees it. New writes always store the current version.
```ts
// src/app/infrastructure/snippet-migrations.ts
export const CURRENT_SNIPPET_VERSION = 2;
export function migrateSnippet(raw: any): Snippet {
let r = { ...raw };
const v = r.version ?? 1; // records written before versioning existed are v1
if (v < 2) {
// Example: a v1 snippet had a single `spec`; v2 splits draft from published.
r.draftSpec = r.draftSpec ?? r.spec;
r.tags = r.tags ?? [];
r.datasetRefs = r.datasetRefs ?? [];
}
// if (v < 3) { ... }
r.version = CURRENT_SNIPPET_VERSION;
return r as Snippet;
}
```
```ts
// src/app/infrastructure/snippet-store.ts
export async function loadSnippets(): Promise<Snippet[]> {
const records = await getAll<any>('snippets');
return records.map(migrateSnippet); // upgrade every record at the boundary
}
export async function saveSnippet(s: Snippet): Promise<void> {
await put('snippets', { ...s, version: CURRENT_SNIPPET_VERSION, modified: new Date().toISOString() });
}
```
### Rationale
- **Read-time migration is forgiving.** Old records sitting untouched in the store keep working; they upgrade lazily the next time they're loaded and re-saved. There is no big-bang migration step that can fail halfway.
- **Tolerate unknown fields.** A migration normalizes *missing/old* fields but must not strip fields it doesn't recognize — a record written by a *newer* build that downgraded must round-trip without data loss. Spread the original (`{ ...raw }`) and only fill in what's missing.
- **One function, well tested.** Each migration step is a pure function over a plain object — trivial to unit-test with fixture records from each historical version.
> **Do:** default `version` to the earliest shape (`1`) when the field is absent.
> **Don't:** branch on the presence of individual fields scattered through the app to detect "old data." Centralize that knowledge in the migration function.
---
## 5. localStorage Preferences (Settings & App/UI Prefs)
Small, frequently-read structured records live in `localStorage`, not IndexedDB: **UserSettings** (one record) and **app/UI preferences** (snippet sort, panel layout). Why split them from `UserSettings`? UI prefs change often (drag a panel divider, toggle a sort) and shouldn't force a rewrite of the whole settings blob on every interaction.
The pattern is **load-with-fallback, write-through on change.**
- **Load-with-fallback:** merge the parsed stored object over a complete `DEFAULTS` constant. Missing keys (a setting added in a later build) and malformed JSON silently fall back to defaults — the app always gets a fully-populated object and never `undefined`-crashes on a new field.
- **Write-through:** every update reads current, applies the change, and writes the whole record back immediately. No dirty-tracking, no flush step.
- **Environment-guarded:** `localStorage` is absent or throws in some test/SSR contexts; guard access and degrade to defaults rather than throwing.
```ts
// src/app/infrastructure/settings-store.ts
const KEY = 'astrolabe:settings';
export const CURRENT_SETTINGS_VERSION = 1;
export interface UserSettings {
version: number;
editor: { fontSize: number; theme: string; minimap: boolean; wordWrap: 'on' | 'off';
lineNumbers: 'on' | 'off'; tabSize: number };
performance: { renderDebounce: number };
ui: { theme: 'light' | 'experimental'; previewFitMode: 'default' | 'width' | 'height' | 'full' };
formatting: { dateFormat: 'smart' | 'iso' | 'custom'; customDateFormat: string };
}
// Defaults must match the authoritative spec §07 table exactly — that is the
// contract; this is just where it's encoded.
const DEFAULTS: UserSettings = {
version: CURRENT_SETTINGS_VERSION,
editor: { fontSize: 12, theme: 'auto', minimap: false, wordWrap: 'on',
lineNumbers: 'on', tabSize: 2 },
performance: { renderDebounce: 1500 },
ui: { theme: 'light', previewFitMode: 'default' },
formatting: { dateFormat: 'smart', customDateFormat: '' },
};
// NOTE — editor.theme default is 'auto': the editor theme follows the app UI
// theme (light -> light editor theme, experimental -> dark) via custom Monaco
// themes that match the app chrome, unless the user picks an explicit override.
// The explicit-override option set (custom themes; whether to include High
// Contrast or the stock Monaco themes) is still TBD — see spec §07's provisional
// editor-theme note. Resolve the `'auto'` sentinel to a concrete Monaco theme at
// editor-config time, keyed off the current UI theme.
function available(): boolean {
try {
return typeof localStorage !== 'undefined' && typeof localStorage.getItem === 'function';
} catch {
return false; // access itself can throw (e.g. blocked storage)
}
}
export function loadSettings(): UserSettings {
if (!available()) return structuredClone(DEFAULTS);
try {
const raw = localStorage.getItem(KEY);
if (!raw) return structuredClone(DEFAULTS);
const p = JSON.parse(raw);
// Deep-merge each group over defaults so new keys fall back silently.
return {
version: CURRENT_SETTINGS_VERSION,
editor: { ...DEFAULTS.editor, ...p.editor },
performance: { ...DEFAULTS.performance, ...p.performance },
ui: { ...DEFAULTS.ui, ...p.ui },
formatting: { ...DEFAULTS.formatting, ...p.formatting },
};
} catch (err) {
console.warn('[settings] failed to load, using defaults', err);
return structuredClone(DEFAULTS);
}
}
export function saveSettings(s: UserSettings): void {
if (!available()) return;
try {
localStorage.setItem(KEY, JSON.stringify({ ...s, version: CURRENT_SETTINGS_VERSION }));
} catch (err) {
console.warn('[settings] failed to save', err);
}
}
```
App/UI prefs follow the identical pattern under their own keys, e.g.:
```ts
// src/app/infrastructure/prefs-store.ts
const SORT_KEY = 'astrolabe:snippet-sort';
const LAYOUT_KEY = 'astrolabe:panel-layout';
const SORT_DEFAULTS = { sortBy: 'modified' as const, sortOrder: 'desc' as const };
// loadSort()/saveSort() and loadLayout()/saveLayout() mirror §5's guard+fallback shape.
```
> **Do:** keep a single complete `DEFAULTS` object as the source of truth and merge over it.
> **Don't:** read individual keys with bespoke `?? fallback` at each call site; one stale default and the shapes drift.
---
## 6. Storage Tiers, Budgets & Quota Monitoring
Astrolabe has three tiers with different capacities and risk profiles:
| Tier | Backing | Holds | Budget & behavior |
|------|---------|-------|-------------------|
| **Snippet store** | IndexedDB `snippets` | All snippet records | Practical budget ~**5 MB**. A storage monitor estimates usage and surfaces a warning as it fills. Snippets are user-authored and irreplaceable, so we fail **loudly**. |
| **Dataset store** | IndexedDB `datasets` | All dataset payloads | Separate, **high-capacity**; suited to large payloads. Lazily loaded (§3). |
| **Settings & prefs** | localStorage | `UserSettings` + app/UI prefs (§5) | Small; effectively unbounded for this use. |
Splitting snippets and datasets into separate stores means a few large datasets can't crowd out the snippet budget, and the snippet monitor can report a meaningful "how full is my library" number without summing dataset bytes.
### Estimating usage
Use the Storage Manager API where available, with a manual byte-sum fallback for the snippet tier so the ~5 MB budget is always reportable.
```ts
// src/app/infrastructure/storage-monitor.ts
export interface StorageReport {
snippetBytes: number; // estimated bytes used by the snippet tier
snippetBudget: number; // 5 MB practical budget
ratio: number; // snippetBytes / snippetBudget, clamped to >= 0
warn: boolean; // ratio crossed the warning threshold
}
const SNIPPET_BUDGET = 5 * 1024 * 1024;
const WARN_AT = 0.8;
export async function reportSnippetUsage(snippets: Snippet[]): Promise<StorageReport> {
// Cheap, deterministic estimate: serialize the records we hold.
const snippetBytes = snippets.reduce(
(n, s) => n + new Blob([JSON.stringify(s)]).size,
0
);
const ratio = snippetBytes / SNIPPET_BUDGET;
const report: StorageReport = {
snippetBytes,
snippetBudget: SNIPPET_BUDGET,
ratio,
warn: ratio >= WARN_AT,
};
if (report.warn) {
console.warn(
`[storage] snippet tier ${(ratio * 100).toFixed(0)}% of ${SNIPPET_BUDGET} bytes`
);
}
return report;
}
```
### Fail loudly, never silently lose data
When a write would exceed quota, IndexedDB rejects with a `QuotaExceededError`. The adapter must **propagate** this so the UI can tell the user to export and prune — it must never swallow the error and pretend the save succeeded.
```ts
export async function saveSnippet(s: Snippet): Promise<void> {
try {
await put('snippets', { ...s, version: CURRENT_SNIPPET_VERSION });
} catch (err) {
if (err instanceof DOMException && err.name === 'QuotaExceededError') {
// Surface to the user via the store; do NOT silently drop the write.
throw new StorageQuotaError('Snippet storage is full. Export and remove snippets to free space.');
}
throw err;
}
}
```
> **Do:** surface quota warnings *before* the budget is hit (the 80% threshold) and hard errors loudly when a write fails.
> **Don't:** wrap a save in a bare `try/catch {}` that logs and returns — that turns "your work wasn't saved" into a silent data-loss bug. The only thing the adapter may safely swallow is a *read* failure, where falling back to defaults/empty is the correct behavior.
---
## 7. Checklist for Adding a New Persisted Entity
1. Define the record type with `id`, `created`, `modified`, and a `version` field.
2. Decide the tier: small + critical → IndexedDB store with a monitored budget; large payload → separate high-capacity store with lazy loading (§3); tiny + frequently changing → localStorage pref (§5).
3. Add the object store in `openDB`'s `onupgradeneeded`, guarded by `contains(...)`; bump `DB_VERSION` only if you changed store *layout*.
4. Add a `migrate<Entity>()` function and call it on every read.
5. Expose typed `load*/save*/ensure*` functions from one infrastructure module — and from *only* there.
6. If the tier has a budget, hook it into the storage monitor and propagate `QuotaExceededError`.
7. Test the adapter against `fake-indexeddb` / a localStorage stub; test the migration with fixtures from each historical version.
docs/architecture/03-modal-system.md
+502
View File
@@ -0,0 +1,502 @@
# 03 · Modal System
How Astrolabe manages its modals: a single metadata-driven registry, a thin
lifecycle coordinator, and one rendering shell. This document is the
authoritative architecture for adding, opening, closing, and rendering modals.
## Goals
- **One source of truth** for modal metadata — no `switch` statements scattered
across the codebase keyed on the active modal.
- **At most one modal open at a time** (mandated by the product spec). Opening a
modal closes any other; the two never overlap.
- **Uniform dismissal**: close button, `Escape`, or backdrop click — never a
click inside the body.
- **Accessible by default**: focus moves into the modal on open and returns to
the trigger on close.
- **Unsaved-change safety** for editing modals, with an explicit opt-out for
modals that apply changes immediately.
The system is three layers, each with a single responsibility:
| Layer | Responsibility | Lives in |
|-------|----------------|----------|
| **Registry** | Static metadata per modal (title, validity, snapshot, init) | `src/app/modals/modal-registry.ts` |
| **Coordinator** | Lifecycle: open, close, URL sync, change detection | `src/app/modals/ModalCoordinator.ts` |
| **Shell** | Render exactly one modal; backdrop / Escape / focus trap | `src/app/App.tsx` + a `useFocusTrap` hook |
---
## The Modal Set
Astrolabe has a small, fixed set of modals. Model it as a closed union so the
registry, coordinator, and shell are exhaustively type-checked.
```ts
// src/app/modals/types.ts
export type ModalName =
| 'datasets' // Datasets manager (list / detail / new-dataset form)
| 'settings' // Appearance, editor, performance, formatting prefs
| 'about' // About & Help (shortcuts, privacy)
| 'donate' // Donate
| 'chartBuilder' // Visual no-JSON chart composition for a dataset
| 'extract'; // Extract inline spec data into a new dataset
export type ActiveModal = ModalName | null;
```
Two of these — `chartBuilder` and `extract` — are **opened from within
workflows** (the Datasets manager and the snippet editor), not from the header
toolbar. That is a UI wiring detail, not a structural one: every modal opens
through the same coordinator regardless of where the trigger lives.
---
## Layer 1 — Registry
Each modal is registered once with its metadata. The registry is a plain lookup
object keyed by `ModalName`; order is irrelevant. Utility queries
(title, validity, whether a modal participates in URL state) read from the
registry so there is exactly one place to change when behavior shifts.
### Config shape
```ts
// src/app/modals/modal-registry.ts
import type { ComponentType } from 'react';
import type { ModalName } from './types';
export interface ModalConfig {
name: ModalName;
title: string; // i18n key or literal
component: ComponentType<any>; // the body rendered inside the shell
/** Initialize transient modal state when it opens. `arg` carries an
* optional sub-target (e.g. a dataset id for chartBuilder/extract). */
init?: (arg?: string) => void;
/** Serializable snapshot of in-progress edits, used to detect unsaved
* changes on close. OMIT for modals that apply immediately (settings,
* about, donate) — omission opts out of the discard-confirmation. */
getState?: () => Record<string, unknown> | null;
/** Whether the modal's primary action (Save / Apply) should be blocked
* because the current input is invalid. Drives the disabled button. */
hasError?: () => boolean;
/** Human-readable reason for the disabled action, shown as a tooltip. */
getError?: () => string | null;
/** Whether this modal is reflected in the URL hash (back/forward, reload
* restore). Datasets and Chart Builder are navigable; Donate is not. */
isUrlNavigable?: boolean;
}
```
### Example entries
```ts
import { DatasetsModal } from '../components/DatasetsModal';
import { SettingsModal } from '../components/SettingsModal';
import { ChartBuilderModal } from '../components/ChartBuilderModal';
import { ExtractModal } from '../components/ExtractModal';
import { DonateModal } from '../components/DonateModal';
import { AboutModal } from '../components/AboutModal';
import { useDatasetStore } from '../stores/DatasetStore';
import { useChartBuilderStore } from '../stores/ChartBuilderStore';
import { useExtractStore } from '../stores/ExtractStore';
import { useSettingsStore } from '../stores/SettingsStore';
// Per-modal transient state lives in the relevant feature store; the registry
// reads it via `getState()` (Zustand), never through component hooks.
export const MODAL_REGISTRY: Record<ModalName, ModalConfig> = {
// Navigable, editing modal — snapshot guards unsaved work.
datasets: {
name: 'datasets',
title: 'modals.datasets.title',
component: DatasetsModal,
isUrlNavigable: true,
init: (datasetId) => useDatasetStore.getState().select(datasetId ?? null),
getState: () => {
const s = useDatasetStore.getState();
return {
view: s.view, // 'list' | 'detail' | 'new'
draft: s.draftForm, // in-progress new/edit form
};
},
hasError: () => useDatasetStore.getState().formError !== null,
getError: () => useDatasetStore.getState().formError,
},
// Opened from a workflow (a specific dataset), navigable, editing.
chartBuilder: {
name: 'chartBuilder',
title: 'modals.chartBuilder.title',
component: ChartBuilderModal,
isUrlNavigable: true,
init: (datasetId) => useChartBuilderStore.getState().initFor(datasetId),
getState: () => ({ encoding: useChartBuilderStore.getState().encoding }),
hasError: () => !useChartBuilderStore.getState().markType,
getError: () =>
useChartBuilderStore.getState().markType ? null : 'modals.chartBuilder.pickMark',
},
// Opened from the snippet editor with the inline data to lift out.
extract: {
name: 'extract',
title: 'modals.extract.title',
component: ExtractModal,
init: (sourceKey) => useExtractStore.getState().initFrom(sourceKey),
getState: () => ({ name: useExtractStore.getState().name }),
hasError: () => useExtractStore.getState().name.trim() === '',
getError: () =>
useExtractStore.getState().name.trim() ? null : 'modals.extract.nameRequired',
},
// Applies immediately — no getState, so closing never prompts.
settings: { name: 'settings', title: 'modals.settings.title', component: SettingsModal, isUrlNavigable: true, init: () => useSettingsStore.getState().loadFromPrefs() },
// Pure info modals — no state, no validity, not navigable for donate.
about: { name: 'about', title: 'modals.about.title', component: AboutModal, isUrlNavigable: true },
donate: { name: 'donate', title: 'modals.donate.title', component: DonateModal },
};
```
### Registry queries
All callers go through these helpers instead of inspecting the active modal
directly:
```ts
export const getModalConfig = (name: ActiveModal): ModalConfig | undefined =>
name ? MODAL_REGISTRY[name] : undefined;
export const getModalTitle = (name: ActiveModal): string =>
getModalConfig(name)?.title ?? '';
export const isUrlNavigable = (name: ActiveModal): boolean =>
getModalConfig(name)?.isUrlNavigable ?? false;
```
> **Why metadata-driven?** The alternative — branching on the active modal in
> the shell, the URL sync, the keyboard handler, and the close logic — spreads
> one decision across four files. Each new modal then means four edits and a
> chance to forget one. With the registry, a new modal is one entry plus its
> component.
**Do**
- Add a modal by appending one `MODAL_REGISTRY` entry and writing its component.
- Express validity through `hasError` / `getError` so the shell's action button
and tooltip stay generic.
- Omit `getState` for any modal that commits changes immediately.
**Don't**
- Don't `switch (activeModal)` outside the shell's body render. Lookups belong
in registry helpers.
- Don't put rendering or DOM concerns in the registry — it is pure metadata.
- Don't read rapidly-changing input state inside `hasError`/`getState` from
component render paths; compute them with a selector at the shell boundary (see
Shell layer) so a keystroke doesn't re-render the whole app.
---
## Layer 2 — Coordinator
The coordinator owns the modal lifecycle. It mutates a single piece of state —
the active modal name — plus a snapshot used for change detection, and keeps the
URL in sync. It is framework-light: pure functions over a Zustand store, unit
testable without a DOM.
### State
The active modal name is **cross-cutting UI chrome**, so it lives on the central
`useAppStore` (`activeModal` + the `setActiveModal` primitive — see
docs/architecture/01). The coordinator never gets its own store; the only extra
piece of state it needs is the change-detection **snapshot**, which is
coordinator-internal (no component reads it), so it stays as a module-local
variable rather than store state.
```ts
// useAppStore already exposes:
// activeModal: ModalName | null
// setActiveModal: (modal: ModalName | null) => void
```
### Open / close
```ts
// src/app/modals/ModalCoordinator.ts
import { useAppStore } from '../stores/AppStore';
import { MODAL_REGISTRY, getModalConfig } from './modal-registry';
import { syncModalToUrl, clearModalFromUrl } from './UrlStateSync';
let confirmDiscard: (msg: string) => Promise<boolean> = async () => true;
export const setConfirm = (fn: typeof confirmDiscard) => { confirmDiscard = fn; };
// Coordinator-internal: the getState() JSON captured at open, compared on close.
let stateSnapshot: string | null = null;
const snapshot = (name: ActiveModal) =>
getModalConfig(name)?.getState
? JSON.stringify(getModalConfig(name)!.getState!())
: null;
/** Open `name`, optionally with a sub-target (dataset id, source key). */
export function openModal(name: ModalName, arg?: string): void {
// Opening any modal replaces the previous one — at most one open at a time.
useAppStore.getState().setActiveModal(name);
getModalConfig(name)?.init?.(arg);
stateSnapshot = snapshot(name);
syncModalToUrl(name, arg); // no-op when !isUrlNavigable
}
/** Close the active modal. Prompts on unsaved changes unless `force`. */
export async function closeModal(force = false): Promise<void> {
const name = useAppStore.getState().activeModal;
if (!name) return;
if (!force && hasUnsavedChanges()) {
const ok = await confirmDiscard('modals.discardChanges');
if (!ok) return;
}
clearModalFromUrl(name);
useAppStore.getState().setActiveModal(null);
stateSnapshot = null;
getModalConfig(name)?.init?.(undefined); // optional: reset transient state
}
/** Cmd/Ctrl+K toggle for the Datasets manager. */
export function toggleDatasets(): void {
if (useAppStore.getState().activeModal === 'datasets') void closeModal();
else openModal('datasets');
}
```
### Change detection
```ts
export function hasUnsavedChanges(): boolean {
const name = useAppStore.getState().activeModal;
if (!name || stateSnapshot === null) return false; // no snapshot ⇒ opted out
const current = getModalConfig(name)?.getState?.();
if (current == null) return false;
return JSON.stringify(current) !== stateSnapshot;
}
```
The snapshot is taken once on open and compared on close. Modals without
`getState` (settings, about, donate) snapshot to `null`, so `hasUnsavedChanges`
short-circuits and they close instantly — correct, because they either apply
immediately or hold nothing to lose.
### Validity passthrough
```ts
export const activeModalHasError = (): boolean =>
getModalConfig(useAppStore.getState().activeModal)?.hasError?.() ?? false;
export const activeModalError = (): string | null =>
getModalConfig(useAppStore.getState().activeModal)?.getError?.() ?? null;
```
> **Why a coordinator instead of letting components open/close themselves?**
> Centralizing means the "close the previous one", snapshot, URL-sync, and
> discard-prompt rules are enforced once. A component that opened a peer modal
> directly could bypass the discard check or leave the URL stale.
**Do**
- Route every open/close through `openModal` / `closeModal`.
- Take the snapshot in `openModal` (after `init`) and compare in `closeModal`.
- Keep the coordinator DOM-free so it can be tested with plain Vitest.
**Don't**
- Don't mutate `activeModal` directly from components or handlers.
- Don't skip `closeModal`'s unsaved-change check by toggling state manually;
pass `force` only when the user has explicitly saved or confirmed.
---
## Layer 3 — Shell
`App` renders **exactly one** modal — whichever `activeModal` names — inside a
single reusable shell. The shell provides the backdrop, header, focus trap, and
the generic close/action affordances; the modal's registered `component` fills
the body.
```tsx
// src/app/App.tsx (modal portion)
import { useAppStore } from '../stores/AppStore';
import { getModalConfig, getModalTitle } from '../modals/modal-registry';
import { closeModal, activeModalHasError, activeModalError } from '../modals/ModalCoordinator';
import { useFocusTrap } from '../hooks/useFocusTrap';
export function App() {
const name = useAppStore((s) => s.activeModal);
const config = getModalConfig(name);
// Move focus into the modal on open, return it to the trigger on close.
const modalRef = useFocusTrap<HTMLDivElement>(name !== null);
// Derived at the shell boundary so per-keystroke store reads don't
// re-render the whole app tree.
const hasError = activeModalHasError();
const errorMsg = activeModalError();
return (
<div className={styles.app}>
{/* ...library · editor · preview panes, header... */}
{config && (
<div
className={styles.backdrop}
onClick={() => void closeModal()} // backdrop dismisses
onKeyDown={(e) => { if (e.key === 'Escape') void closeModal(); }}
>
<div
ref={modalRef}
className={styles.modal}
role="dialog"
aria-modal="true"
aria-labelledby="modal-title"
onClick={(e) => e.stopPropagation()} // inside body never dismisses
>
<header className={styles.modalHeader}>
<h2 id="modal-title">{t(getModalTitle(name))}</h2>
<button aria-label={t('buttons.close')} onClick={() => void closeModal()}>×</button>
</header>
<div className={styles.modalBody}>
{/* The ONE place the active modal is mapped to a component. */}
<config.component />
</div>
{/* Optional generic action row for editing modals. A modal with no
primary action (about, donate) can render its own footer/none. */}
{config.getState && (
<footer className={styles.modalFooter}>
<button className="btn-secondary" onClick={() => void closeModal()}>
{t('buttons.cancel')}
</button>
<button
className="btn-primary"
aria-disabled={hasError || undefined}
title={errorMsg ? t(errorMsg) : undefined}
onClick={() => { if (!hasError) config.component /* invoke save handler */; }}
>
{t('buttons.save')}
</button>
</footer>
)}
</div>
</div>
)}
</div>
);
}
```
Rendering `<config.component />` from the registry is the only modal-name→view
mapping in the app. There is no `name === 'datasets' && <DatasetsModal/>` chain.
### Focus trap
A small hook saves the previously focused element, focuses the first focusable
child on open, wraps `Tab`/`Shift+Tab` within the modal, and restores focus on
close.
```ts
// src/app/hooks/useFocusTrap.ts
import { useRef, useEffect } from 'react';
const FOCUSABLE =
'a[href], button:not([disabled]), input:not([disabled]), select:not([disabled]), ' +
'textarea:not([disabled]), [tabindex]:not([tabindex="-1"])';
export function useFocusTrap<T extends HTMLElement = HTMLDivElement>(active: boolean) {
const ref = useRef<T>(null);
const returnTo = useRef<Element | null>(null);
useEffect(() => {
const el = ref.current;
if (!active || !el) return;
returnTo.current = document.activeElement;
el.querySelector<HTMLElement>(FOCUSABLE)?.focus();
const onKey = (e: KeyboardEvent) => {
if (e.key !== 'Tab') return;
const f = el.querySelectorAll<HTMLElement>(FOCUSABLE);
if (!f.length) return;
const first = f[0], last = f[f.length - 1];
if (e.shiftKey && document.activeElement === first) { e.preventDefault(); last.focus(); }
else if (!e.shiftKey && document.activeElement === last) { e.preventDefault(); first.focus(); }
};
el.addEventListener('keydown', onKey);
return () => {
el.removeEventListener('keydown', onKey);
(returnTo.current as HTMLElement | null)?.focus(); // restore focus on close
};
}, [active]);
return ref;
}
```
> **Why one shell instead of each modal rendering its own chrome?** Backdrop
> behavior, the focus trap, `aria-modal`, Escape handling, and the close button
> are identical for every modal and easy to get subtly wrong (e.g. a backdrop
> that dismisses on inner clicks). Centralizing guarantees consistency and means
> accessibility is fixed once.
**Do**
- Render the active modal via `<config.component />` — the single mapping point.
- Put `onClick={closeModal}` on the backdrop and `stopPropagation` on the body.
- Compute `hasError`/`getError`/preview reads with a selector at the shell level.
- Gate the generic Save button on `hasError` and surface `getError` as its
tooltip.
**Don't**
- Don't render two modals simultaneously, and don't stack a second backdrop.
- Don't attach the focus trap to the backdrop — attach it to the modal body so
the backdrop click stays outside the trap.
- Don't dismiss on clicks inside the body, and don't let Escape fire when no
modal is open (the handler only exists while a modal renders).
---
## URL & Keyboard Integration
The coordinator is the join point for navigation:
- `openModal` calls `syncModalToUrl`; navigable modals write a hash
(`#datasets`, `#datasets/dataset-<id>`, `#datasets/dataset-<id>/build`,
`#settings`). Non-navigable modals (donate) write nothing.
- `closeModal` calls `clearModalFromUrl`, returning to the underlying workspace
hash.
- On load, the URL restorer reads the hash and calls `openModal(name, arg)` to
rehydrate the right modal and sub-target.
- The global key handler maps `Cmd/Ctrl+K` → `toggleDatasets()`,
`Cmd/Ctrl+,` → `openModal('settings')`, and `Escape` → `closeModal()` (the
Escape binding is a no-op when `activeModal` is `null`).
Because all of these call the same coordinator functions, browser
Back/Forward, keyboard shortcuts, and in-app triggers stay consistent — they
share the open/close/snapshot/URL logic rather than reimplementing it.
---
## Adding a Modal: Checklist
1. Add the name to the `ModalName` union.
2. Add one `MODAL_REGISTRY` entry (title, component; `getState`/`hasError`/
`getError` if it edits; `isUrlNavigable` + `init(arg)` if navigable).
3. Write the body component; it reads/writes its feature store (e.g.
`useDatasetStore`, `useChartBuilderStore`) via a narrow selector.
4. If navigable, add its hash form to the URL sync and restore logic.
5. If it has a keyboard shortcut or workflow trigger, wire that to
`openModal(name, arg)` — never to `activeModal` directly.
No edits to the shell render, the close logic, or the change-detection code are
needed: those are generic and driven entirely by the registry.
docs/architecture/04-routing-and-events.md
+463
View File
@@ -0,0 +1,463 @@
# 04 · Routing & Global Events
Two small, related subsystems govern how the app talks to the browser shell:
1. **URL hash as view-state** — the current view (selected snippet, open dataset
modal, etc.) lives in `location.hash`. It is read on load to restore state,
written on navigation, and Back/Forward step between prior states. Result:
every meaningful view is shareable, bookmarkable, and reload-safe.
2. **Global event / keyboard routing** — a single router owns the
document-level `keydown` / `paste` / `click` listeners. It runs an Escape
priority chain, dispatches shortcuts, and consults a single
`isInInteractiveContext()` helper so global shortcuts and paste handlers
never fire while the user is typing in an input or the Monaco editor.
Both are layered the same way:
```
src/app/infrastructure/url-hash.ts adapter: owns window.location & history
src/app/orchestration/UrlStateSync.ts mediator: hash <-> Zustand stores
src/app/orchestration/EventRouter.ts mediator: DOM events -> store actions
src/app/orchestration/focus-utils.ts single-source isInInteractiveContext()
```
Infrastructure modules touch browser globals; orchestration modules touch the
Zustand stores. Components never read `location.hash` or attach
`window.addEventListener` themselves — they go through these mediators.
---
## 1. URL Hash as View-State
### 1.1 The hash grammar
The hash is the serialized view. Astrolabe's forms:
| State | Hash |
| ----------------------------- | --------------------------------- |
| Default snippets view | _(empty / absent)_ |
| A selected snippet | `#snippet-<id>` |
| Datasets manager (list) | `#datasets` |
| A specific dataset | `#datasets/dataset-<id>` |
| New-dataset form | `#datasets/new` |
| Chart Builder for a dataset | `#datasets/dataset-<id>/build` |
Snippet `id` is an opaque string; dataset `id` is the numeric dataset id
rendered as a decimal string. The hash is the **only** persisted view-routing
state — there is no in-memory "current route" that can drift from it.
### 1.2 The adapter: `infrastructure/url-hash.ts`
This is the only file that reads or writes `window.location` / `history`. It
exposes a parse function (hash string → typed `ViewState`), a serialize
function (`ViewState` → hash string), and write helpers. Keep it pure-ish:
parsing is a total function with no side effects; writing is the only place
`history.replaceState` is called.
```ts
// src/app/infrastructure/url-hash.ts
export type ViewState =
| { kind: 'snippets' } // empty hash
| { kind: 'snippet'; snippetId: string } // #snippet-<id>
| { kind: 'datasets' } // #datasets
| { kind: 'dataset'; datasetId: number } // #datasets/dataset-<id>
| { kind: 'dataset-new' } // #datasets/new
| { kind: 'dataset-build'; datasetId: number }; // .../build
export function parseHash(rawHash: string): ViewState {
const hash = rawHash.replace(/^#/, '');
if (hash === '') return { kind: 'snippets' };
const snippet = /^snippet-(.+)$/.exec(hash);
if (snippet) return { kind: 'snippet', snippetId: snippet[1] };
const parts = hash.split('/').filter(Boolean);
if (parts[0] === 'datasets') {
if (parts.length === 1) return { kind: 'datasets' };
if (parts[1] === 'new') return { kind: 'dataset-new' };
const m = /^dataset-(\d+)$/.exec(parts[1]);
if (m) {
const id = Number(m[1]);
if (parts[2] === 'build') return { kind: 'dataset-build', datasetId: id };
return { kind: 'dataset', datasetId: id };
}
}
// Unknown hash -> fall back to default rather than throwing.
return { kind: 'snippets' };
}
export function serializeHash(view: ViewState): string {
switch (view.kind) {
case 'snippets': return '';
case 'snippet': return `#snippet-${view.snippetId}`;
case 'datasets': return '#datasets';
case 'dataset': return `#datasets/dataset-${view.datasetId}`;
case 'dataset-new': return '#datasets/new';
case 'dataset-build': return `#datasets/dataset-${view.datasetId}/build`;
}
}
export function readView(): ViewState {
return parseHash(window.location.hash);
}
/** Write without adding a history entry (in-place correction, restore). */
export function replaceView(view: ViewState): void {
const url = new URL(window.location.href);
url.hash = serializeHash(view);
url.search = '';
window.history.replaceState({}, '', url.toString());
}
/** Write and add a history entry (user navigation -> Back works). */
export function pushView(view: ViewState): void {
const url = new URL(window.location.href);
url.hash = serializeHash(view);
url.search = '';
window.history.pushState({}, '', url.toString());
}
```
**`pushState` vs `replaceState` is the lever that makes Back/Forward feel
right.** Use `pushView` for deliberate user navigation (selecting a snippet,
opening a dataset) so each becomes a Back-able step. Use `replaceView` for
restoring on load and for correcting a stale/invalid hash, where you do not want
to litter history.
### 1.3 The mediator: `orchestration/UrlStateSync.ts`
`UrlStateSync` is the bridge between the hash and the Zustand stores. It does
three jobs:
- **On load — restore:** read the view, validate referenced ids against the
stores, and drive the stores to match. If an id no longer exists, fall back
to the default view and `replaceView` to clean the URL.
- **Hash → state (Back/Forward):** listen for `hashchange` and reconcile the
stores to the new view. This is what makes the browser buttons work.
- **State → hash:** expose typed `navigate*` helpers the rest of the app calls
when the user moves around. These `pushView` (or `replaceView`).
Guard against feedback loops: writing the hash fires no `hashchange` when you
use the History API the way above, but a defensive `applying` flag keeps the
`hashchange` reconciler from re-triggering navigation it just caused.
```ts
// src/app/orchestration/UrlStateSync.ts
import { useSnippetStore } from '../stores/SnippetStore';
import { useDatasetStore } from '../stores/DatasetStore';
import { useAppStore } from '../stores/AppStore'; // activeModal, etc.
import { readView, replaceView, pushView, type ViewState } from '../infrastructure/url-hash';
let applying = false; // suppress re-entrancy while we drive the stores
let started = false;
// Restore/reconcile is the one path that writes `activeModal` with the bare
// `setActiveModal` primitive instead of the coordinator's openModal/closeModal:
// we are reflecting the URL *into* the stores, so we must NOT re-sync the URL or
// run the unsaved-change discard prompt (the `applying` guard blocks re-entrancy).
/** Make the stores reflect `view`. Falls back + cleans URL on dead ids. */
function applyView(view: ViewState): void {
applying = true;
try {
switch (view.kind) {
case 'snippets':
useAppStore.getState().setActiveModal(null);
return;
case 'snippet': {
const snippet = useSnippetStore.getState().byId(view.snippetId);
if (!snippet) { replaceView({ kind: 'snippets' }); return; }
useAppStore.getState().setActiveModal(null);
useSnippetStore.getState().select(view.snippetId);
return;
}
case 'datasets':
useAppStore.getState().setActiveModal('datasets');
return;
case 'dataset':
case 'dataset-build': {
const ds = useDatasetStore.getState().byId(view.datasetId);
if (!ds) { replaceView({ kind: 'datasets' }); return; }
useAppStore.getState().setActiveModal('datasets');
useDatasetStore.getState().select(view.datasetId);
if (view.kind === 'dataset-build') useAppStore.getState().setActiveModal('chartBuilder');
return;
}
case 'dataset-new':
useAppStore.getState().setActiveModal('datasets');
useDatasetStore.getState().beginNew();
return;
}
} finally {
applying = false;
}
}
export function startUrlStateSync(): void {
if (started) return;
started = true;
// 1. Restore from the URL on load.
applyView(readView());
// 2. Back/Forward -> reconcile stores.
window.addEventListener('hashchange', () => {
if (applying) return;
applyView(readView());
});
// 3. State -> hash. A store subscription keeps the URL honest if any code path
// changes the active view without calling a navigate* helper. Optional;
// explicit navigate* calls are the primary writer.
useAppStore.subscribe((state, prev) => {
if (applying) return;
// derive ViewState from state and replaceView(...) here if desired
});
}
// --- State -> hash: the API the app calls on user navigation -------------
export const navigate = {
toSnippet: (id: string) => pushView({ kind: 'snippet', snippetId: id }),
toSnippets: () => pushView({ kind: 'snippets' }),
toDatasets: () => pushView({ kind: 'datasets' }),
toDataset: (id: number) => pushView({ kind: 'dataset', datasetId: id }),
toNewDataset: () => pushView({ kind: 'dataset-new' }),
toChartBuilder: (id: number) => pushView({ kind: 'dataset-build', datasetId: id }),
};
```
**Do**
- Restore on load with `replaceView`; navigate at runtime with `pushView`.
- Validate every id from the hash against the stores; fall back + clean URL on
a miss (deleted/shared-stale ids are normal, not exceptional).
- Keep `parseHash` / `serializeHash` pure and round-trippable — unit-test that
`parseHash(serializeHash(v)) === v` for every `ViewState`.
**Don't**
- Don't read or write `location.hash` from components — call `navigate.*`.
- Don't `pushState` on load-restore (pollutes Back history).
- Don't throw on an unrecognized hash; degrade to the default view.
---
## 2. Global Event / Keyboard Routing
### 2.1 The router: `orchestration/EventRouter.ts`
One module binds the document-level listeners (`keydown`, `paste`, `click`) and
routes them. Centralizing this keeps ordering explicit and gives one place to
reason about priority. The router owns two things in particular:
- the **Escape priority chain**, and
- **shortcut dispatch**, gated by `isInInteractiveContext()`.
```ts
// src/app/orchestration/EventRouter.ts
import { useAppStore } from '../stores/AppStore';
import { useSnippetStore } from '../stores/SnippetStore';
import { navigate } from './UrlStateSync';
import { openModal, closeModal, toggleDatasets } from '../modals/ModalCoordinator';
import { isInInteractiveContext } from './focus-utils';
let started = false;
export function startEventRouter(): void {
if (started) return;
started = true;
window.addEventListener('keydown', onKeyDown);
window.addEventListener('paste', onPaste);
}
export function stopEventRouter(): void {
window.removeEventListener('keydown', onKeyDown);
window.removeEventListener('paste', onPaste);
started = false;
}
const isMac = /Mac|iPhone|iPad|iPod/.test(navigator.platform);
function onKeyDown(e: KeyboardEvent): void {
// --- Escape: highest priority, runs even inside editors/inputs ----------
if (e.key === 'Escape') {
if (handleEscapeChain()) e.preventDefault();
return;
}
const mod = isMac ? e.metaKey : e.ctrlKey;
// --- Shortcuts: never fire while typing in an input or Monaco ----------
if (isInInteractiveContext()) return;
// Cmd/Ctrl + Shift + N -> new snippet
if (mod && e.shiftKey && e.key.toLowerCase() === 'n') {
e.preventDefault();
const created = useSnippetStore.getState().create();
navigate.toSnippet(created.id);
return;
}
// Cmd/Ctrl + K -> toggle Datasets manager (coordinator owns open/close + URL)
if (mod && !e.shiftKey && e.key.toLowerCase() === 'k') {
e.preventDefault();
toggleDatasets();
return;
}
// Cmd/Ctrl + S -> publish current draft
if (mod && e.key.toLowerCase() === 's') {
e.preventDefault(); // override the browser "save page" dialog
useSnippetStore.getState().publishDraft();
return;
}
// Cmd/Ctrl + , -> settings (through the coordinator: snapshot + URL sync)
if (mod && e.key === ',') {
e.preventDefault();
openModal('settings');
return;
}
}
/** Returns true if it consumed the Escape (caller should preventDefault). */
function handleEscapeChain(): boolean {
// 1. Toast/message box would go here if it grew a blocking variant.
// 2. Active modal — route through the coordinator so the unsaved-change
// discard prompt runs and the URL is cleared. NEVER setActiveModal(null)
// here: that would silently drop in-progress dataset/chart-builder edits.
if (useAppStore.getState().activeModal) {
void closeModal();
return true;
}
// 3. Open menu / popover.
if (useAppStore.getState().openMenu) {
useAppStore.getState().setOpenMenu(null);
return true;
}
// 4. Active selection (e.g. selected snippet in the library).
if (useSnippetStore.getState().selectionId) {
useSnippetStore.getState().clearSelection();
return true;
}
return false;
}
function onPaste(e: ClipboardEvent): void {
// Paste-to-import (e.g. paste a Vega-Lite spec) must NOT hijack a paste the
// user makes inside the editor or an input.
if (isInInteractiveContext()) return;
// ... route clipboard text to the import handler ...
}
```
**The Escape chain is an explicit, ordered ladder, top-down.** Each rung
returns as soon as it consumes the event, so only the topmost active layer
reacts. Order matters: a blocking message box outranks a modal, a modal
outranks an open menu, a menu outranks a selection. Add new dismissible layers
by inserting a rung at the right priority — never by sprinkling
`document.addEventListener('keydown', …Escape…)` in a component.
**Shortcuts override browser defaults.** Each handled combo calls
`e.preventDefault()` so Cmd/Ctrl+S does not trigger "save page", Cmd/Ctrl+K
does not focus the browser search bar, etc.
Note the asymmetry: **Escape is checked before the interactive-context gate**
(you want Escape to dismiss a modal even while focus is in the editor), whereas
all other shortcuts are checked **after** the gate (so they don't fire mid-typing).
### 2.2 The single-source helper: `orchestration/focus-utils.ts`
There is exactly **one** function that answers "is the user currently typing in
an editable surface?" Every shortcut path and the paste handler call it. Never
inline element-type checks — one place to get it right, one place to fix it
when the DOM changes.
> **Monaco difference (important):** Astrolabe's spec editor is **Monaco**, not
> CodeMirror. Monaco renders into a `.monaco-editor` container and keeps focus
> on a hidden `<textarea class="inputarea">` inside it. The detector must match
> Monaco's DOM — a `.monaco-editor` ancestor (and/or the inputarea) — **not** a
> `.cm-editor` / `.cm-content` selector. If you copy a CodeMirror check here it
> will silently fail and global shortcuts will fire while the user edits a spec.
```ts
// src/app/orchestration/focus-utils.ts
/**
* True when focus is in an editable surface where global shortcuts and
* paste-to-import must be suppressed: <input>, <textarea>, <select>,
* contenteditable, or the Monaco editor.
*
* This is the SINGLE source of truth — do not inline these checks elsewhere.
*/
export function isInInteractiveContext(): boolean {
const el = document.activeElement as HTMLElement | null;
if (!el) return false;
const tag = el.tagName.toLowerCase();
if (tag === 'input' || tag === 'textarea' || tag === 'select') return true;
if (el.isContentEditable) return true;
// Monaco renders into a .monaco-editor container; its focused element is a
// hidden <textarea class="inputarea"> (already caught above) but guard the
// container explicitly so focus on any inner node still counts.
if (el.closest?.('.monaco-editor')) return true;
return false;
}
```
**Do**
- Route all global keyboard/paste/click through `EventRouter`; bind listeners
in exactly one place, started once at app init.
- Express Escape as an ordered chain that returns on first consumption.
- Call `isInInteractiveContext()` everywhere a global handler might collide
with typing; keep it the only definition.
- `preventDefault()` on every shortcut the app claims, so it overrides the
browser default.
**Don't**
- Don't add ad-hoc `window`/`document` keydown listeners in components.
- Don't inline `tagName === 'textarea'` / editor-class checks at call sites —
call the helper.
- Don't match a CodeMirror selector for the editor; Astrolabe is Monaco.
- Don't gate Escape behind `isInInteractiveContext()` — Escape should still
close a modal while the editor has focus.
---
## 3. Wiring at startup
Both subsystems start once, after the stores are hydrated from persistence, in
the app's init/orchestration step:
```ts
// src/app/orchestration/bootstrap.ts (sketch)
import { startUrlStateSync } from './UrlStateSync';
import { startEventRouter } from './EventRouter';
export function initApp(): void {
// ... load settings + hydrate snippet/dataset stores from IndexedDB/localStorage ...
startUrlStateSync(); // restore view from hash, then keep hash <-> stores in sync
startEventRouter(); // bind global keyboard/paste routing
}
```
Order: hydrate stores first (so hash-restore can resolve ids), then
`startUrlStateSync` (it reads the hash and may drive the stores), then
`startEventRouter`. Each `start*` is idempotent and has a matching `stop*` for
teardown in tests.
---
## 4. Testing notes
- **`parseHash` / `serializeHash`:** pure, so test directly. Cover every
`ViewState`, the empty hash, and at least one malformed hash → default.
Assert the round-trip identity.
- **`isInInteractiveContext`:** happy-dom test (the project's Vitest env). Mount
an `<input>`, a `contenteditable` div, and a `<div class="monaco-editor"><textarea/></div>`;
focus each and assert `true`; assert `false` for a focused `<button>`.
- **Escape chain:** with stores in known states, dispatch a synthetic Escape
and assert only the top active layer changed.
- **Restore-on-load with dead id:** seed an empty store, set
`location.hash = '#snippet-gone'`, call `startUrlStateSync()`, assert the
view fell back to default and the hash was cleaned.
docs/architecture/05-rendering-theming-preview.md
+448
View File
@@ -0,0 +1,448 @@
# Rendering, Theming & Live Preview
How Astrolabe turns a user-authored Vega-Lite specification into a live chart in
the preview pane. This covers four mechanics: **embedding** a spec via
`vega-embed`, **theming** so charts match the active UI theme, **debounced
re-rendering** so typing stays smooth, and **error handling** so a broken spec
produces a readable message and self-heals. It deliberately stops at the
embedding boundary — the *content* of the spec (resolving named-dataset
references, applying fit-mode sizing) is prepared upstream by a pure transform;
see §6.
---
## 1. The Embedding Boundary
The preview is a thin imperative layer wrapping the `vega-embed` library, driven
by reactive store state. The flow is always the same:
```
spec text ──parse──▶ Vega-Lite spec object
│
▼
prepareSpecForRender(spec, { fitMode }) ← pure, src/core/rendering.ts
│ (operates on a COPY; never mutates the stored spec)
▼
render(node, preparedSpec, config) ← src/app, this doc
│
vega-embed ─▶ View ─▶ SVG in the DOM node
```
`vega-embed` is the only place in the app that touches the chart DOM. Everything
above it is data; everything below it is a Vega `View` we own and must tear down.
### Rules
- **Do** keep all `vega-embed` calls behind one small renderer module. Components
ask the renderer to draw a spec into a node; they never import `vega-embed`
directly.
- **Do** treat the renderer as imperative glue driven by store state (via a
`subscribe` listener), not as reactive state itself.
- **Don't** scatter `vegaEmbed(...)` calls across components.
---
## 2. vega-embed Integration
A single async `render` function embeds a prepared spec into a DOM node. Three
non-negotiable embed options, plus disciplined teardown of the previous view:
```ts
// src/app/services/chart-renderer.ts (sketch)
import vegaEmbed, { type Result as EmbedResult } from 'vega-embed';
import type { Config, TopLevelSpec } from 'vega-lite';
export interface RenderHandle {
/** Finalize the underlying Vega view and release its resources. */
destroy(): void;
}
export async function renderSpec(
node: HTMLElement,
spec: TopLevelSpec,
config: Config,
): Promise<RenderHandle> {
const result: EmbedResult = await vegaEmbed(node, spec, {
actions: false, // no built-in export/source/editor menu — clean chart
renderer: 'svg', // crisp, inspectable, copyable output
config, // theme config (see §3)
});
return {
destroy() {
// Frees timers, listeners, and the canvas/SVG the view created.
result.view.finalize();
node.replaceChildren(); // drop any leftover DOM the embed inserted
},
};
}
```
### The view lifecycle is the bug surface
Every successful `vegaEmbed` returns a `result.view` (a live Vega `View`
instance). It owns timers, signal listeners, and DOM. If you embed a new spec
into the same node *without* finalizing the old view, the old one leaks — its
listeners keep firing and resources accumulate over a long editing session.
The renderer that drives re-rendering must therefore hold the previous handle and
destroy it before (or while) creating the next:
```ts
let current: RenderHandle | null = null;
async function rerender(node: HTMLElement, spec: TopLevelSpec, config: Config) {
current?.destroy(); // tear down the previous view first
current = await renderSpec(node, spec, config);
}
```
### Rules
- **Do** pass `actions: false`. Astrolabe owns its own export/copy affordances;
the library's overlay menu does not belong on the preview.
- **Do** call `view.finalize()` on every previous view before rendering a new
one, and on component unmount.
- **Do** keep exactly one live view per preview node.
- **Don't** re-embed into a node whose previous view you have not finalized.
- **Don't** keep a reference to a finalized view; null it out.
---
## 3. Theme Follows the UI Theme
A Vega-Lite **config** object styles every chart globally — fonts, axis colors,
background, the categorical color range, default mark colors. Astrolabe ships one
config per UI theme so charts visually belong to the app rather than looking like
stock Vega-Lite.
```ts
// src/core/vega-themes.ts (sketch)
import type { Config } from 'vega-lite';
export const lightChartConfig: Config = {
background: 'transparent',
font: '"Inter", sans-serif',
title: { fontSize: 15, fontWeight: 600, color: '#1c1c1e' },
axis: {
domainColor: '#1c1c1e',
gridColor: '#e4e4e7',
gridDash: [3, 3],
labelColor: '#52525b',
titleColor: '#1c1c1e',
labelFontSize: 11,
titleFontSize: 12,
},
range: {
category: ['#2f6df6', '#f5a524', '#17b890', '#e5484d', '#8b5cf6', '#0ea5e9'],
},
view: { stroke: 'transparent' },
};
export const experimentalChartConfig: Config = {
background: 'transparent',
font: '"Inter", sans-serif',
title: { fontSize: 15, fontWeight: 600, color: '#f4f4f5' },
axis: {
domainColor: '#a1a1aa',
gridColor: '#3f3f46',
gridDash: [3, 3],
labelColor: '#a1a1aa',
titleColor: '#f4f4f5',
labelFontSize: 11,
titleFontSize: 12,
},
range: {
category: ['#5b8def', '#f5a524', '#2dd4a7', '#f0666b', '#a78bfa', '#38bdf8'],
},
view: { stroke: 'transparent' },
};
```
One mapping, in one place, is the single source of truth for theme → config:
```ts
// src/core/vega-themes.ts
import type { UiTheme } from './theme'; // core-local — never import from src/app
const CHART_CONFIG: Record<UiTheme, Config> = {
light: lightChartConfig,
experimental: experimentalChartConfig,
};
export function chartConfigFor(theme: UiTheme): Config {
return CHART_CONFIG[theme];
}
```
The renderer reads the active UI theme (from the store) and passes the matching config into
`renderSpec`. When the theme changes, the same subscriber that drives
re-rendering picks up the new config and the chart restyles automatically.
### Rules
- **Do** keep `chartConfigFor` as the *only* place that maps a UI theme to a Vega
config. Adding a UI theme = adding one config and one map entry.
- **Do** set chart `background: 'transparent'` so the pane's own background shows
through and theme switches look seamless.
- **Don't** inline colors or fonts into individual specs to "match the theme" —
that is the config's job, and per-spec styling drifts from the app.
- **Don't** let the user's stored spec carry a `config`; the theme config is
applied at embed time via the embed options, leaving the spec theme-agnostic.
---
## 4. Field-Name Escaping
Vega-Lite treats `.`, `[`, and `]` inside a `field:` string as **nested-property
accessors**: `field: "user.age"` reads `row.user.age`, not a column literally
named `"user.age"`. Astrolabe renders arbitrary user data whose column names may
contain those characters, so any column name placed into a `field:` (or `as:`,
`groupby:`, tooltip `field:`, etc.) must be escaped first.
```ts
// src/core/rendering.ts (sketch)
/** Escape `.`/`[`/`]` so Vega-Lite treats the string as a literal field name. */
export function escapeVegaField(name: string): string {
return name.replace(/([.[\]])/g, '\\$1');
}
```
```ts
// usage when constructing/normalizing an encoding that references a column:
encoding.x = { field: escapeVegaField(columnName), type: 'quantitative' };
```
This matters wherever Astrolabe *constructs* spec fragments from data-derived
column names — most notably the chart builder (see *Chart Builder* spec) and any
helper that injects an encoding. For specs the user authored by hand, escaping is
the user's responsibility; Astrolabe does not rewrite hand-authored `field:`
values.
### Rules
- **Do** route every data-derived column name through `escapeVegaField` before it
lands in a `field:` (or any field-position key).
- **Don't** ever pass a raw column name to `field:`. If the name came from data,
it is unescaped until proven otherwise.
---
## 5. Debounced Preview
Rendering must never compete with typing. The preview re-renders only after the
user pauses, the pending render is cancelled on each new keystroke, and a render
in flight never blocks the editor.
The debounce delay is **user-configurable** via the `performance.renderDebounce`
setting (range ~500–5000 ms). Read it live so changes take effect without reload.
```ts
// src/app/services/debounced-renderer.ts (sketch)
export interface DebouncedRenderer {
/** Schedule a render after the debounce window; resets the timer. */
schedule(): void;
/** Render now, skipping the debounce (e.g. on fit-mode change or theme flip). */
flush(): void;
/** Cancel a pending render without rendering. */
cancel(): void;
}
export function createDebouncedRenderer(opts: {
/** Current debounce delay in ms; read fresh each schedule so settings apply live. */
delayMs: () => number;
/** Performs one render. Reads the current spec/theme; awaits the embed. */
render: () => Promise<void>;
/** Toggle the non-blocking busy indicator. */
setBusy: (busy: boolean) => void;
}): DebouncedRenderer {
let timer: ReturnType<typeof setTimeout> | null = null;
let generation = 0; // guards against a stale in-flight render finishing late
const run = async () => {
timer = null;
const mine = ++generation;
opts.setBusy(true);
try {
await opts.render();
} finally {
// Only the most recent render clears the indicator.
if (mine === generation) opts.setBusy(false);
}
};
return {
schedule() {
if (timer) clearTimeout(timer); // cancel the pending render
timer = setTimeout(run, opts.delayMs());
},
flush() {
if (timer) { clearTimeout(timer); timer = null; }
void run();
},
cancel() {
if (timer) { clearTimeout(timer); timer = null; }
generation++; // abandon any in-flight result
},
};
}
```
### Wiring it to the store
Startup subscribers observe the inputs that affect the picture — the current spec
text, the active fit mode, the UI theme — and call `schedule()` (debounced) for
spec edits, or `flush()` for instantaneous controls like a fit-mode toggle:
```ts
// wired once at startup
useEditorStore.subscribe((s, prev) => {
if (s.currentSpecText !== prev.currentSpecText) renderer.schedule(); // react to edits
});
useSettingsStore.subscribe((s, prev) => {
if (s.previewFitMode !== prev.previewFitMode || s.uiTheme !== prev.uiTheme) {
renderer.flush(); // immediate, no debounce
}
});
```
### Busy indicator
`setBusy(true/false)` toggles store state that the preview reads to overlay a
**subtle, non-blocking** spinner/shimmer. It sits *over* the existing chart so the
last good render stays visible while the next one computes — the pane never goes
blank mid-edit.
### Rules
- **Do** read `renderDebounce` fresh on each `schedule()` (via the `delayMs()`
thunk) so a settings change applies immediately.
- **Do** cancel the pending timer on every new input before scheduling the next.
- **Do** guard against out-of-order completion (the `generation` counter): a slow
render that resolves after a newer one must not clobber the indicator or view.
- **Do** keep the busy indicator non-blocking and overlaid; never clear the chart
to show "rendering…".
- **Don't** render synchronously on every keystroke.
- **Don't** await a render inside an input/keydown handler.
---
## 6. Rendering Contract Lives Upstream (reference)
Before a spec reaches `renderSpec`, it passes through a **pure** transform in
`src/core/rendering.ts`:
```ts
prepareSpecForRender(spec, { fitMode }): TopLevelSpec
```
It does two deterministic things, on a **deep copy** of the spec:
1. **Dataset reference resolution** — replaces any named-data reference with the
referenced dataset's actual contents (inline values, raw CSV/TSV text, or a
URL reference), recursing into layered/concat/child sub-specs.
2. **Fit-mode sizing** — rewrites `width`/`height` per the active fit mode using
Vega-Lite's `"container"` keyword (Original = untouched; Width/Height/Full set
the corresponding dimension(s) to `"container"`), recursing the same way.
This is *content* preparation, not embedding, and it is fully covered by the
*Live Preview* spec. The only invariant this doc cares about:
> `prepareSpecForRender` runs on a copy and returns a new spec. The renderer
> embeds that returned spec. **The user's stored spec is never mutated by
> rendering.**
The container-relative fit modes (Width/Height/Full) depend on `renderer: 'svg'`
plus `"container"` sizing to follow the pane; when the pane resizes, re-running
`prepareSpecForRender` + re-embedding (a `flush()`) re-fits the chart.
### Rules
- **Do** call `prepareSpecForRender` between parse and embed, every render.
- **Don't** put reference resolution or fit-mode logic in the renderer — it is
pure core logic and must be unit-testable without a DOM.
- **Don't** mutate the input spec anywhere in the pipeline.
---
## 7. Error Handling
A spec that cannot be rendered must produce a **readable** message in the preview
area and recover on its own once the spec is valid again. Errors arise at three
stages, all funneled to one error field the preview reads:
| Stage | Failure | Surfaced as |
|---|---|---|
| Parse | Invalid JSON | "Invalid JSON: …" |
| Prepare (`prepareSpecForRender`) | Referenced dataset missing/unfetchable | "Dataset not found: …" |
| Embed (`vega-embed`) | Vega-Lite compile / data error | "Rendering error: …" |
```ts
// inside render(), driven by the debounced renderer
async function render(): Promise<void> {
const text = useEditorStore.getState().currentSpecText.trim();
// Empty/blank is NOT an error — render nothing, clean pane.
if (!text) {
current?.destroy();
current = null;
usePreviewStore.getState().setError(null);
return;
}
let parsed: unknown;
try {
parsed = JSON.parse(text);
} catch (e) {
usePreviewStore.getState().setError(`Invalid JSON: ${(e as Error).message}`);
return; // keep the last good chart underneath the error, or show the message
}
try {
const { previewFitMode, uiTheme } = useSettingsStore.getState();
const prepared = prepareSpecForRender(parsed, { fitMode: previewFitMode });
const config = chartConfigFor(uiTheme);
current?.destroy();
current = await renderSpec(node, prepared, config);
usePreviewStore.getState().setError(null); // success clears any prior error
} catch (e) {
usePreviewStore.getState().setError(
`Rendering error: ${(e as Error).message}. ` +
`Check your JSON syntax and that the spec is valid Vega-Lite.`,
);
}
}
```
The preview component renders the chart node when `error` is `null`, and the
error panel when it is set. Because **every successful render clears the error**,
recovery is automatic: the next valid edit re-renders and wipes the message — no
manual retry, no reload.
### Rules
- **Do** treat empty/blank spec text as "render nothing" — finalize the current
view, clear the error, show a clean empty pane.
- **Do** clear the error state on every successful render.
- **Do** make messages legible and actionable (the underlying reason plus a hint
to check JSON/Vega-Lite validity), never a raw stack trace dump.
- **Do** distinguish the failing stage in the message (Invalid JSON vs Dataset
not found vs Rendering error).
- **Don't** show a broken/partial chart — replace the chart area with the
message.
- **Don't** require a manual "retry"; validity restores the chart on its own.
---
## Summary
| Concern | Mechanism | Source of truth |
|---|---|---|
| Embedding | One `renderSpec` over `vega-embed`, `actions: false`, `renderer: 'svg'` | `src/app/services/chart-renderer.ts` |
| View teardown | `view.finalize()` before each re-render and on unmount | the renderer's `RenderHandle` |
| Theming | Vega `Config` per UI theme, applied at embed time | `chartConfigFor()` in `src/core/vega-themes.ts` |
| Field names | `escapeVegaField` on every data-derived `field:` | `src/core/rendering.ts` |
| Debounce | `createDebouncedRenderer`, delay from `renderDebounce` setting | `src/app/services/debounced-renderer.ts` |
| Spec prep | `prepareSpecForRender` (pure, on a copy) | `src/core/rendering.ts` (see *Live Preview*) |
| Errors | One error field, cleared on success, empty = nothing | `PreviewStore.error` |
docs/architecture/06-type-inference.md
+346
View File
@@ -0,0 +1,346 @@
# Type Inference & Data Profiling
How Astrolabe looks at a tabular dataset and figures out, for each column, what
kind of data it holds — `number`, `string`, `date`, or `boolean` — and how it
rolls those facts up into the **profile** stored on a dataset record.
This is pure, portable logic. It lives in `src/core/`, touches no browser APIs
and no React, takes plain values in and returns plain data out, and is covered
by Vitest unit tests. Anything that needs a profile (the create form, the edit
flow, the detail panel) calls into this module; nothing here reaches back out.
---
## 1. Why infer types at all
A dataset is just rows of values. The UI wants to *describe* it without
re-parsing the payload every time: how many rows and columns, what the columns
are called, and roughly what each column contains. The inferred type drives the
small type indicator next to each column name in the dataset detail panel and
the meta line in the list. It is a **display hint**, not a contract — nothing
downstream coerces values based on it, and Vega-Lite does its own type handling
at render time. Because it is only a hint, a wrong guess is cheap, and the rules
below favour being simple and predictable over being clever.
We support exactly **four** inferred types:
| Type | Meaning |
| --------- | ---------------------------------------------------- |
| `number` | Every non-empty value is numeric. |
| `boolean` | Every non-empty value is `true`/`false` (any case). |
| `date` | Every non-empty value parses as a date. |
| `string` | The fallback — anything that isn't one of the above. |
There is deliberately no integer/float split, no datetime-vs-date distinction,
and no JSON type. Those distinctions add branches and edge cases without
changing what the user sees. Keep it at four.
---
## 2. Inferring one column
Given the values of a single column, decide its type.
### The shape of the algorithm
1. **Drop the empties.** Filter out `null`, `undefined`, and empty/whitespace-only
strings before doing anything. Empty cells carry no type signal — a column of
numbers with a few blanks is still a number column.
2. **All-empty → `string`.** If nothing survives the filter (the column is
entirely empty, or there are zero rows), default to `string`. There is no
evidence for any other type.
3. **Run the type checks in precedence order.** For each candidate type, ask:
*does **every** surviving value match this type?* The first candidate for
which the answer is yes wins. This is the **"all values match → that type,
else fall back"** rule: one stray value that doesn't fit knocks the column
down to the next candidate, and ultimately to `string`.
### Precedence order matters
The order of the checks is not arbitrary — it exists because the value-sets
overlap, and we want the most specific interpretation that fits.
1. **boolean** first. The strings `"true"`/`"false"` are not numbers and not
dates, so booleans never collide with the other checks — but putting them
first keeps a `0`/`1`-free true/false column out of `string`. (We do *not*
treat `0`/`1` as boolean; that's a number column.)
2. **number** second. `Number("2024")` is a perfectly good number, so a column
of bare years would read as `number` — which is the honest answer. Numbers
are checked before dates so that plain numeric columns never get
mis-classified as dates by an over-eager date parser.
3. **date** third. Date parsing is the loosest, most permissive check, so it
goes last among the positive checks. By the time we reach it we already know
the column isn't all-boolean and isn't all-numeric.
4. **string** is the fallback when no positive check matches every value.
> Mnemonic: **boolean → number → date → string**, narrowest evidence to widest.
### What counts as each type
- **numeric**: trim the string form; reject empty; `Number(trimmed)` must be
finite and not `NaN`. (Native `number` values pass directly.) Reject blank and
whitespace so `Number("") === 0` doesn't sneak through.
- **boolean**: native `boolean` values pass; otherwise the trimmed,
lower-cased string must be exactly `"true"` or `"false"`.
- **date**: guard *before* parsing. Require the trimmed value to look
date-shaped (a leading `YYYY-MM-DD` or `YYYY/MM/DD`, or `M/D/YYYY`) **and**
then confirm `Date.parse` returns a finite timestamp. The shape guard is
essential: `Date.parse` will happily accept `"42"` or `"March"` on some
engines, which would swallow number and string columns. Never rely on
`Date.parse` alone.
### Sketch
```ts
// src/core/type-inference.ts
export type ColumnType = 'number' | 'string' | 'date' | 'boolean';
const isEmpty = (v: unknown): boolean =>
v === null || v === undefined || (typeof v === 'string' && v.trim() === '');
const isNumeric = (v: unknown): boolean => {
if (typeof v === 'number') return Number.isFinite(v);
if (typeof v !== 'string') return false;
const t = v.trim();
if (t === '') return false;
const n = Number(t);
return !Number.isNaN(n) && Number.isFinite(n);
};
const isBoolean = (v: unknown): boolean => {
if (typeof v === 'boolean') return true;
if (typeof v !== 'string') return false;
const t = v.trim().toLowerCase();
return t === 'true' || t === 'false';
};
// Shape guard first, then confirm it actually parses.
const DATE_SHAPE = /^\d{4}[-/]\d{2}[-/]\d{2}|^\d{1,2}\/\d{1,2}\/\d{4}/;
const isDate = (v: unknown): boolean => {
if (typeof v !== 'string') return false;
const t = v.trim();
return DATE_SHAPE.test(t) && !Number.isNaN(Date.parse(t));
};
/**
* Infer one of four column types from a sample of column values.
* Empty cells are ignored; an all-empty column is `string`.
* Precedence: boolean → number → date → string.
*/
export function inferColumnType(values: readonly unknown[]): ColumnType {
const present = values.filter((v) => !isEmpty(v));
if (present.length === 0) return 'string';
if (present.every(isBoolean)) return 'boolean';
if (present.every(isNumeric)) return 'number';
if (present.every(isDate)) return 'date';
return 'string';
}
```
### Robustness notes
- **Mixed columns** fall through to `string` automatically — the `every` check
fails on the first non-conforming value, so a column of mostly-numbers with
one label is `string`, which is the safe, honest answer.
- **Whitespace** is trimmed in every check, so `" 42 "` reads as numeric and
`" "` is treated as empty.
- **Empty columns** (all cells blank, or a zero-row dataset) return `string` by
the all-empty rule — never throw, never guess.
- **Large columns**: see §4. `inferColumnType` itself just consumes whatever
array it's handed; the caller decides whether to sample.
### Do / Don't
- **Do** ignore empty cells before classifying.
- **Do** keep the precedence boolean → number → date → string.
- **Do** guard date detection with a shape regex before trusting `Date.parse`.
- **Don't** classify a column unless *every* present value matches — one
outlier means `string`.
- **Don't** add more types (integer, float, datetime, json). Four, no more.
- **Don't** let `Number("")`, `Date.parse("42")`, or `0`/`1` leak into the wrong
bucket.
---
## 3. Profiling a dataset
A **profile** is the set of derived summary fields stored on a dataset record so
the UI can describe it without re-parsing the payload. Per the data model, a
profiled dataset carries:
| Field | Type | Meaning |
| ------------- | --------------------------------- | -------------------------------------- |
| `rowCount` | `number \| null` | Data rows, or `null` when N/A. |
| `columnCount` | `number \| null` | Columns, or `null` when N/A. |
| `columns` | `string[]` | Column names, in order. |
| `columnTypes` | `Array<{ name; type }>` | Per-column inferred type (see §2). |
| `size` | `number` | Approximate payload size in bytes. |
`null` row/column counts and an empty `columns`/`columnTypes` are how the UI
shows **"N/A"** — see §3.2.
### 3.1 What gets profiled
Profiling applies only to **tabular inline data**:
- **JSON** that is an array of objects.
- **CSV** (comma-separated, header row).
- **TSV** (tab-separated, header row).
Everything else is **not profiled**:
- **URL datasets** — the library holds only the link, not the data, so there is
nothing to scan. Counts are `null` / N/A.
- **Non-tabular data** — a single JSON object, TopoJSON, or anything we can't
read as rows-of-columns. Counts are `null` / N/A.
For the not-profiled cases, `size` is still computed (it's just the byte length
of the stored payload), but `rowCount` and `columnCount` are `null`, and
`columns`/`columnTypes` are empty.
### 3.2 The algorithm
1. **Compute `size`** from the raw payload regardless of whether it's tabular —
byte length of the text (CSV/TSV) or of the JSON-serialized value.
2. **Decide if it's tabular.** Map `(format, parsed shape)` to a row set:
- `csv` / `tsv` → parse into rows-of-objects using the matching delimiter.
- `json` that is a non-empty **array of objects** → use it directly.
- anything else (`topojson`, a lone JSON object, an empty array) → not
tabular; return the N/A profile (`rowCount: null`, `columnCount: null`,
`columns: []`, `columnTypes: []`, plus `size`).
3. **Derive columns** from the union of keys across the rows (or the CSV/TSV
header), preserving first-seen order.
4. **Infer each column's type** by collecting that column's values across the
rows and calling `inferColumnType` (§2), sampling per §4.
5. **Assemble** `rowCount`, `columnCount`, `columns`, `columnTypes`, `size`.
### Sketch
```ts
// src/core/profile.ts
import { inferColumnType, type ColumnType } from './type-inference';
export interface DatasetProfile {
rowCount: number | null;
columnCount: number | null;
columns: string[];
columnTypes: Array<{ name: string; type: ColumnType }>;
size: number;
}
const NA = (size: number): DatasetProfile => ({
rowCount: null,
columnCount: null,
columns: [],
columnTypes: [],
size,
});
/** Profile a dataset payload. `rows` is the tabular form (CSV/TSV/JSON-array)
* already parsed to rows-of-objects, or null for non-tabular / URL data. */
export function profileData(
rows: ReadonlyArray<Record<string, unknown>> | null,
size: number,
): DatasetProfile {
if (!rows || rows.length === 0) return NA(size);
// Column order = first-seen order across all rows.
const columns: string[] = [];
const seen = new Set<string>();
for (const row of rows) {
for (const key of Object.keys(row)) {
if (!seen.has(key)) {
seen.add(key);
columns.push(key);
}
}
}
if (columns.length === 0) return NA(size);
const sample = sampleRows(rows);
const columnTypes = columns.map((name) => ({
name,
type: inferColumnType(sample.map((r) => r[name])),
}));
return {
rowCount: rows.length,
columnCount: columns.length,
columns,
columnTypes,
size,
};
}
```
Parsing CSV/TSV text and detecting the payload shape happen *upstream* of
`profileData`; this function takes already-parsed rows so it stays pure and
trivially testable. The caller passes `null` for URL and non-tabular datasets.
---
## 4. Sampling vs. full scan
`rowCount`/`columnCount`/`size` always reflect the **whole** dataset — they're
cheap (a length and a byte count). Only **type inference** has a per-value cost,
and it's the one place a huge dataset could hurt.
So: infer types from a **bounded sample** of rows, not the full column. A fixed
cap (e.g. the first ~200 rows) keeps profiling fast and predictable on large
pasted datasets while still being more than enough signal to classify a column.
```ts
const SAMPLE_SIZE = 200;
const sampleRows = <T>(rows: ReadonlyArray<T>): ReadonlyArray<T> =>
rows.length <= SAMPLE_SIZE ? rows : rows.slice(0, SAMPLE_SIZE);
```
Trade-off to be aware of: a column that is numeric for its first 200 rows but
turns to text later will be mis-typed as `number`. That's an accepted cost — the
type is a display hint, the mistake is cheap, and the speed win on large
datasets is worth it. Sampling the head (rather than randomly) keeps results
**deterministic**, which matters for tests and for not surprising the user when
the same paste profiles the same way twice.
### Do / Don't
- **Do** count rows/columns and size over the full payload.
- **Do** cap type-inference sampling at a fixed head slice for determinism.
- **Don't** randomly sample — non-deterministic profiles break tests and confuse
users.
- **Don't** scan every value of a million-row paste to guess a type.
---
## 5. Testing
Both functions are pure, so tests are plain input/output assertions in Vitest —
no mocks, no DOM, no fixtures beyond literal arrays.
Cover at least:
- **`inferColumnType`**: each type detected from a clean column; mixed columns
fall to `string`; empty/whitespace cells ignored; all-empty and zero-length →
`string`; precedence (a `["true","false"]` column is `boolean` not `string`; a
`["2024","2025"]` column is `number` not `date`); date shape guard rejects
`"42"` and `"hello"` even though one engine's `Date.parse` might accept them;
`0`/`1` are `number`, not `boolean`.
- **`profileData`**: a JSON-array dataset profiles fully; `null` rows (URL) and
an empty array (non-tabular) return the N/A profile but still carry `size`;
column order follows first-seen key order across ragged rows; sampling cap is
respected (a dataset longer than the cap still profiles, using only the head).
---
## Summary
- Four types only: **boolean → number → date → string**, checked in that order.
- **All present values must match** a type or the column falls through; empty
cells are ignored; an all-empty column is `string`.
- Guard date detection with a shape regex before trusting `Date.parse`.
- A **profile** carries `rowCount`, `columnCount`, `columns`, `columnTypes`,
`size`; URL and non-tabular datasets get a **null/N-A** profile (still sized).
- Counts and size scan the whole payload; **type inference samples the head** for
speed and determinism.
- All of it is **pure `src/core/` logic, unit-tested with Vitest**.
docs/architecture/07-naming-and-relationships.md
+432
View File
@@ -0,0 +1,432 @@
# Naming & Relationships
How Astrolabe keeps entity **names unique** within a collection, and how it
tracks the **bidirectional links** between snippets and datasets so they stay
consistent as entities are created, imported, and renamed.
Two concerns live here, and they reinforce each other:
1. **Name uniqueness** — every dataset has a unique name. Names are the primary
key users see and the key snippets reference, so duplicates would be
ambiguous. We reject duplicate names on create/rename, and auto-suffix
collisions during bulk import.
2. **Relationship tracking** — a snippet references datasets *by name* through
its `datasetRefs: string[]` field. This is a bidirectional, name-based link:
from a snippet you read its refs; from a dataset you scan snippets to find
who uses it. Renaming a dataset must propagate to every snippet that points
at it, in both the spec and the `datasetRefs` array, or the links rot.
The hard, testable logic is **pure** and lives in `src/core/`. The parts that
read and mutate stores live in `src/app/services/`.
---
## 1. Why names, not IDs, are the link
Datasets carry a numeric `id`, but snippets reference them **by name** because
that is what Vega-Lite uses: a spec resolves data through a named-data
reference, `{ "data": { "name": "MyDataset" } }`. The name *is* the contract
between a spec and the dataset library. Storing a numeric id in the spec would
mean the spec is no longer a standalone, paste-anywhere Vega-Lite document.
The consequence: names must be unique (two datasets named `Sales` would make
`{ "data": { "name": "Sales" } }` ambiguous), and renaming a dataset is a
**graph operation**, not a single field write — every reference to the old name
must move with it.
---
## 2. Name uniqueness (pure — `src/core/naming.ts`)
### 2.1 Uniqueness check
Comparisons are **case-insensitive** (`Sales` and `sales` collide), so a single
display name maps to a single dataset regardless of how a user types a
reference. The check takes an optional `excludeId` so a rename can ignore the
record being renamed (renaming `Sales` to `Sales` is not a collision with
itself).
```ts
// src/core/naming.ts
/** Case-insensitive set of names already in use, minus an optional excluded id. */
export function isNameTaken(
desired: string,
datasets: ReadonlyArray<{ id: number; name: string }>,
excludeId?: number,
): boolean {
const lower = desired.trim().toLowerCase();
return datasets.some((d) => d.id !== excludeId && d.name.toLowerCase() === lower);
}
```
### 2.2 Making a unique name
When a desired name is taken — during import, "extract inline data", or
"build chart" — we do **not** overwrite the existing dataset. We derive the
next free name by appending a numeric suffix: `Name` → `Name 2` → `Name 3`.
The function takes the set of existing names so it has no store dependency and
is trivially unit-testable.
```ts
// src/core/naming.ts
/**
* Returns `desired` if free, else the first available `${desired} ${n}` (n >= 2).
* `existingNames` is the set of names already in the collection.
* Comparison is case-insensitive; the returned name preserves `desired`'s casing.
*/
export function makeUniqueName(desired: string, existingNames: Iterable<string>): string {
const taken = new Set<string>();
for (const n of existingNames) taken.add(n.toLowerCase());
const base = desired.trim();
if (!taken.has(base.toLowerCase())) return base;
let n = 2;
while (taken.has(`${base} ${n}`.toLowerCase())) n++;
return `${base} ${n}`;
}
```
> If a base name already ends in a number (`Q1 2024`), the suffix still appends
> (`Q1 2024 2`). That is intentional: we never parse meaning out of the name,
> we only guarantee a free slot. Keep this dumb and predictable.
**Do**
- Use `isNameTaken` to reject duplicate create/rename in the UI before saving,
and surface an error toast.
- Use `makeUniqueName` for every non-interactive path (import, extract, build)
where blocking the user would be worse than a silent, reported rename.
- Pass `excludeId` on rename so an unchanged or case-only edit is allowed.
**Don't**
- Don't compare names case-sensitively anywhere — pick `toLowerCase()` once and
use it consistently.
- Don't let `makeUniqueName` mutate a store or read store state; it takes plain
data and returns a string.
---
## 3. The bidirectional snippet ↔ dataset link
```
datasetRefs: ["Sales", "Regions"] (forward, on the snippet)
Snippet ───────────────────────────────────────────────────► Dataset "Sales"
▲ │
└──────────── scan all snippets for "Sales" in datasetRefs ◄──────┘
(reverse, derived)
```
- **Forward** (snippet → datasets): read `snippet.datasetRefs`. Cheap, stored.
- **Reverse** (dataset → snippets): there is no stored back-pointer. We compute
it by scanning snippets. Keeping it *derived* means it can never disagree with
the forward links — there is one source of truth.
`datasetRefs` is **derived from the spec**, not hand-maintained. It is
recomputed whenever a snippet is published (its draft spec is promoted), so it
always mirrors the dataset names actually referenced in the published spec.
### 3.1 Extracting referenced names from a spec (pure — `src/core/spec-refs.ts`)
A Vega-Lite spec can reference named data in several places: the top-level
`data`, per-layer `data`, `data` inside `spec`/`facet`/`hconcat`/`vconcat`, and
named entries in top-level `datasets`. Rather than enumerate Vega-Lite's grammar,
we walk the spec recursively and collect every `{ data: { name } }` we find.
This is pure, deterministic, and the most heavily unit-tested function here.
```ts
// src/core/spec-refs.ts
type Json = unknown;
/** Collects every dataset name referenced by `{ data: { name } }` anywhere in the spec. */
export function extractDatasetRefs(spec: Json): string[] {
const names = new Set<string>();
const walk = (node: Json): void => {
if (Array.isArray(node)) {
for (const item of node) walk(item);
return;
}
if (node && typeof node === 'object') {
const obj = node as Record<string, Json>;
const data = obj.data as Record<string, Json> | undefined;
if (data && typeof data === 'object' && typeof data.name === 'string') {
names.add(data.name);
}
for (const key of Object.keys(obj)) walk(obj[key]);
}
};
walk(typeof spec === 'string' ? safeParse(spec) : spec);
return [...names];
}
function safeParse(s: string): Json {
try {
return JSON.parse(s);
} catch {
return null; // an unparseable draft simply has no resolvable refs
}
}
```
```ts
// src/core/spec-refs.ts — thin wrapper used at publish time
/** The list stored on snippet.datasetRefs. Sorted + de-duped for stable diffs. */
export function recomputeDatasetRefs(spec: Json): string[] {
return extractDatasetRefs(spec).sort();
}
```
> A `spec` may be an object or a string (see the Data Model). Normalize once,
> at the boundary, so the recursive walk never has to care.
**Do**
- Treat `extractDatasetRefs` as the single source of truth for "what does this
spec reference". The reverse-lookup and rename paths both depend on it
agreeing with what the renderer actually resolves.
- Recompute and store `datasetRefs` on **publish**, not on every keystroke —
the draft can be transiently invalid, and only the published spec is shared.
**Don't**
- Don't let two code paths each have their own idea of "referenced names".
Renamer and ref-recomputer must use the same extractor.
---
## 4. Reverse lookup: who uses this dataset?
The Dataset Manager shows a **usage badge** and a **Linked Snippets** list; the
Snippet Library shows a snippet's linked datasets. Both come from one selector
scan — no stored back-pointer to drift.
```ts
// src/app/services/RelationshipService.ts
import { useSnippetStore } from '../stores/SnippetStore';
import type { Snippet } from '../../core/types';
/** Snippets whose datasetRefs include `name` (case-insensitive). */
export function findSnippetsReferencingDataset(name: string): Snippet[] {
const lower = name.toLowerCase();
return useSnippetStore.getState().snippets.filter((s) =>
s.datasetRefs.some((ref) => ref.toLowerCase() === lower),
);
}
/** Count for the usage badge. */
export function datasetUsageCount(name: string): number {
return findSnippetsReferencingDataset(name).length;
}
```
Because this reads `useSnippetStore.getState().snippets`, exposing it as a
selector for the UI makes the badge and Linked Snippets list reactive for free —
they update the moment any snippet is published with changed refs.
**Do**
- Keep reverse lookup a pure scan over the store. It is O(snippets) but the
collections are small (library budget ~5 MB); clarity beats an index.
- Expose it as a selector where the UI needs reactivity.
**Don't**
- Don't add a `referencedBy` array to datasets. A stored reverse pointer is a
second source of truth that *will* fall out of sync with `datasetRefs`.
---
## 5. Import: auto-suffix collisions, then report
On import we never overwrite an existing dataset. A dataset whose name collides
is renamed to a unique name via `makeUniqueName`, and **every rename is
collected and reported to the user** (toast / summary) so the change is never
silent. Crucially, names are reserved *as we go* — within a single import, two
incoming `Sales` datasets become `Sales 2` and `Sales 3`, not two `Sales 2`.
```ts
// src/app/services/ImportService.ts
import { makeUniqueName } from '../../core/naming';
import type { Dataset } from '../../core/types';
export interface DatasetRename {
from: string;
to: string;
}
/**
* Returns incoming datasets with collision-free names, plus the renames applied.
* `existing` are names already in the library; `incoming` are datasets to add.
*/
export function dedupeIncomingDatasetNames(
existing: ReadonlyArray<string>,
incoming: ReadonlyArray<Dataset>,
): { datasets: Dataset[]; renames: DatasetRename[] } {
const reserved = new Set(existing.map((n) => n.toLowerCase()));
const renames: DatasetRename[] = [];
const datasets = incoming.map((d) => {
const unique = makeUniqueName(d.name, reserved);
reserved.add(unique.toLowerCase()); // reserve so later imports don't collide
if (unique !== d.name) renames.push({ from: d.name, to: unique });
return unique === d.name ? d : { ...d, name: unique };
});
return { datasets, renames };
}
```
> If imported snippets reference the renamed dataset, their `datasetRefs` and
> specs must be rewritten to the new name too — reuse the rename machinery in
> §6 over the imported snippet set, or run `renameDatasetEverywhere` per applied
> rename after the import is committed.
**Do**
- Reserve each chosen name immediately so collisions *within* one import are
also resolved.
- Return the rename list and show it; a silent rename looks like data loss.
**Don't**
- Don't overwrite or merge a same-named existing dataset on import. Suffix and
keep both — the user decides what to delete.
---
## 6. Rename propagation: keep the link consistent
Renaming a dataset is the operation that ties §2–§5 together. A rename must, in
one atomic step:
1. Update the dataset's own `name`.
2. For **every snippet referencing the old name**: rewrite the named-data
references inside its spec (`{ "data": { "name": "old" } }` →
`{ "data": { "name": "new" } }`) — in **both** `spec` and `draftSpec`.
3. Recompute that snippet's `datasetRefs` from the rewritten spec, so the
forward link mirrors reality and the reverse scan stays correct.
The spec rewrite is pure; the orchestration reads and writes stores.
```ts
// src/core/spec-refs.ts — pure rewrite
/** Returns a copy of `spec` with every data.name === oldName replaced by newName. */
export function renameDatasetInSpec(spec: Json, oldName: string, newName: string): Json {
const obj = typeof spec === 'string' ? safeParse(spec) : spec;
const rewrite = (node: Json): Json => {
if (Array.isArray(node)) return node.map(rewrite);
if (node && typeof node === 'object') {
const out: Record<string, Json> = {};
for (const [k, v] of Object.entries(node as Record<string, Json>)) {
if (
k === 'data' &&
v && typeof v === 'object' &&
(v as Record<string, Json>).name === oldName
) {
out[k] = { ...(v as object), name: newName };
} else {
out[k] = rewrite(v);
}
}
return out;
}
return node;
};
const rewritten = rewrite(obj);
// Preserve the original spec's stored shape (string vs object).
return typeof spec === 'string' ? JSON.stringify(rewritten, null, 2) : rewritten;
}
```
```ts
// src/app/services/RelationshipService.ts — store coordination
import { isNameTaken, makeUniqueName } from '../../core/naming';
import { renameDatasetInSpec, recomputeDatasetRefs } from '../../core/spec-refs';
import { useDatasetStore } from '../stores/DatasetStore';
import { useSnippetStore } from '../stores/SnippetStore';
/**
* Renames a dataset and propagates the rename to every referencing snippet
* (spec, draftSpec, and datasetRefs). Returns the snippets that changed.
* Caller is responsible for collision policy on `newName` (reject vs suffix).
*/
export function renameDatasetEverywhere(oldName: string, newName: string): { updated: number } {
if (oldName === newName) return { updated: 0 };
// 1. Rename the dataset record itself.
const dataset = useDatasetStore.getState().datasets.find((d) => d.name === oldName);
if (!dataset) return { updated: 0 };
useDatasetStore.getState().update(dataset.id, { name: newName });
// 2 + 3. Rewrite every referencing snippet's specs and refs.
let updated = 0;
for (const snippet of useSnippetStore.getState().snippets) {
if (!snippet.datasetRefs.some((r) => r.toLowerCase() === oldName.toLowerCase())) continue;
const spec = renameDatasetInSpec(snippet.spec, oldName, newName);
const draftSpec = renameDatasetInSpec(snippet.draftSpec, oldName, newName);
useSnippetStore.getState().update(snippet.id, {
spec,
draftSpec,
datasetRefs: recomputeDatasetRefs(spec),
});
updated++;
}
return { updated };
}
```
> **Collision on rename.** The UI rename form rejects a name already in use via
> `isNameTaken(newName, datasets, dataset.id)`. Programmatic renames (e.g. an
> import flow) instead resolve with `makeUniqueName` before calling
> `renameDatasetEverywhere`. The propagation function itself does not invent a
> name — it assumes `newName` is the agreed target.
**Do**
- Rewrite `spec` **and** `draftSpec`. A user mid-edit must not see their draft
silently break because the dataset was renamed underneath them.
- Recompute `datasetRefs` from the rewritten spec rather than string-replacing
the array — the spec is the source of truth, the array is its mirror.
- Use the §4 reverse lookup to find affected snippets, so "who references this"
has exactly one implementation.
**Don't**
- Don't update `datasetRefs` without also rewriting the spec — the rendered
named-data reference would still point at the old, now-missing name.
- Don't rename the dataset and skip propagation "for now". A half-applied rename
is the exact inconsistency this whole document exists to prevent.
---
## 7. Where things live
| Concern | Location | Pure? | Tested |
|---|---|---|---|
| `makeUniqueName`, `isNameTaken` | `src/core/naming.ts` | yes | unit |
| `extractDatasetRefs`, `recomputeDatasetRefs` | `src/core/spec-refs.ts` | yes | unit |
| `renameDatasetInSpec` | `src/core/spec-refs.ts` | yes | unit |
| `findSnippetsReferencingDataset`, usage count | `src/app/services/RelationshipService.ts` | no (reads store) | integration |
| `renameDatasetEverywhere` | `src/app/services/RelationshipService.ts` | no (mutates stores) | integration |
| `dedupeIncomingDatasetNames` | `src/app/services/ImportService.ts` | nearly (uses `makeUniqueName`) | unit/integration |
The dividing line: anything that takes plain data and returns plain data is
**core** and unit-tested in isolation; anything that reaches into a Zustand store
is an **app service**. The rename rule of thumb — *the spec is the source of
truth, `datasetRefs` mirrors it, the reverse lookup is derived* — is what keeps
the bidirectional link from ever needing manual repair.
docs/architecture/08-vega-editor-techniques.md
+286
View File
@@ -0,0 +1,286 @@
# 08 · Borrowed Techniques from vega/editor
> The official Vega-Lite editor ([vega/editor](https://github.com/vega/editor)) solves the
> exact "edit a Vega-Lite spec as JSON, validate it, render it live" problem Astrolabe sits
> on top of — minus the snippet/dataset library. This doc distills the techniques worth
> borrowing and the gotchas worth avoiding, so we don't rediscover them from scratch in
> M1/M2.
>
> It is a **reference**, not a contract. The behavioral contract is still [`docs/spec/`](../spec/);
> the patterns are still docs [01](01-state-and-stores.md)–[07](07-naming-and-relationships.md).
> This doc is the bridge: "here is how the canonical implementation does the editor/renderer
> plumbing, and here is what we keep vs. improve."
## Source of these findings
A read-only clone of vega/editor lives at `/Users/oleh/code/reference/vega-editor` (shallow
clone of `main`, HEAD `4fdbb59`). Re-clone with
`git clone --depth 1 https://github.com/vega/editor`. Citations below are `file:line` into
that tree.
## Stack delta (read this first — it changes how directly we can borrow)
| | vega/editor | Astrolabe |
|---|---|---|
| UI framework | **React** | **React** (moved off Preact before build start) |
| State | Redux-ish single `State` in React context (`useState`) | Zustand **stores** — *not* Redux |
| Monaco | `@monaco-editor/react` + `@monaco-editor/loader` (CDN-loaded Monaco, **workers auto-wired**) | **raw `monaco-editor`** via Vite (**we must wire workers ourselves**) |
| Rendering | **hand-rolled** `vegaLite.compile` → `vega.parse` → `new vega.View().runAsync()` | **`vegaEmbed()`** (wraps that same pipeline) |
| Schema validation | Monaco JSON worker **+** standalone `ajv ^8` (two independent layers) | same two-layer model planned |
Because both apps are now React, vega/editor's **component lifecycle patterns port more or
less directly** — the friction is only in (a) state (their Redux-flat-state → our Zustand
stores) and (b) Monaco worker wiring (their CDN loader → our explicit Vite workers).
---
## Decision · Monaco integration (self-hosted, raw API)
> **Decided.** Astrolabe uses **raw `monaco-editor` from npm, bundled and self-hosted**, with
> workers wired explicitly via Vite `?worker` — **not** vega/editor's
> `@monaco-editor/react` + `@monaco-editor/loader` (CDN) setup. Two independent axes:
**Axis A — sourcing: self-hosted/bundled, not CDN. (Forced by Astrolabe's values.)**
vega/editor's `@monaco-editor/loader` fetches Monaco's AMD bundle from a CDN at runtime. For
us that breaks three things at once: (1) **offline** — the CDN bundle is outside Vite's module
graph, so `vite-plugin-pwa`/Workbox never precaches it and offline silently fails; bundled npm
assets are hashed files in `dist/` that Workbox precaches automatically; (2) **privacy** — a
third-party fetch on load contradicts SOUL's "the only outbound requests are user-created
URL-dataset fetches"; (3) **determinism** — npm + `package-lock` is integrity-pinned and
reproducible, a runtime CDN resolve is not. This axis is not a close call; vega/editor's CDN
choice is right *for an online hosted tool* and wrong for an offline, installable, private app.
**Axis B — React integration: raw API, not `@monaco-editor/react`. (A lean, not forced.)**
The wrapper helps with the easy 80% (mount a JSON editor, lifecycle) and adds nothing to the
load-bearing 20% this app needs:
- **Workers** are still ours — the wrapper never manages `MonacoEnvironment` (see §1 gotcha).
- The **M2 schema service** (`jsonDefaults.setDiagnosticsOptions`, `fileMatch`) is namespace-level;
you reach *through* the wrapper via `onMount`, so it saves nothing there.
- Its headline **`value`/`onChange` controlled-input model is a hazard**: driving Monaco's
content from React state causes cursor jumps and undo-stack churn, against §10's "typing
stays fluid" — you end up using it uncontrolled, i.e. the raw pattern anyway.
- Its **CDN-by-default** is a standing footgun (works in dev online, fails offline in prod
unless you remember `loader.config({ monaco })`).
Against that, raw costs **one testable `useMonacoEditor` hook** (~50–80 lines: create in
`useEffect`, `dispose` on unmount, push value, subscribe to `onDidChangeModelContent`, resize).
That's the **same imperative-teardown discipline already adopted for `vega-embed`** in doc 05
(`view.finalize()`), and consistent with already using raw `vegaEmbed()` over a React chart
wrapper — "thin integration layers we own" (SOUL). Lock-in is low either way, so the final
raw-vs-wrapper call is confirmable at the Monaco spike; what is **not** up for revisiting is
self-hosting.
**Accepted cost:** the explicit worker wiring (§1) is inherent to self-hosting — it is the
price of offline, paid in any non-CDN setup, and the wrapper would not remove it.
---
> **The single biggest surprise:** vega/editor does **not** use `vega-embed` for its live
> preview. It builds the compile→parse→View pipeline by hand; `vega-embed` is imported only
> for types and the exported standalone-HTML snippet. This is *good news* — `vega-embed` is
> exactly the wrapper they wrote by hand, so we get it for free. But their hand-rolled
> version (`src/components/renderer/renderer.tsx`) is the best available documentation of the
> lifecycle/cleanup discipline `vega-embed` still expects from us.
---
## 1 · Monaco + Vega-Lite schema wiring (M2 — highest from-scratch risk)
All of vega/editor's Monaco setup is one file: `src/utils/monaco.ts`.
**What to borrow:**
- **Bundle the schema; never fetch it.** They `import vegaLiteSchema from 'vega-lite/vega-lite-schema.json'`,
resolved by a Vite alias to the package's `build/` output (`monaco.ts:7-8`, `vite.config.ts`).
The schema version is pinned to the installed `vega-lite` — offline-safe, version-locked,
no runtime network call. Astrolabe should do the same.
- **Attach via the JSON language service**, once, globally:
`monaco.languages.json.jsonDefaults.setDiagnosticsOptions({ schemas, validate:true, ... })`
(`monaco.ts:51-57`).
- **`markdownDescription` patch** (`monaco.ts:12-13`, `utils/markdownProps.ts`): recursively
copy every schema `description` → `markdownDescription` before registering. Monaco renders
rich hover docs only from `markdownDescription`; without this, hovers are plain text. Do it
once at setup.
- **Replace the built-in JSON formatter** with `json-stringify-pretty-compact` via
`registerDocumentFormattingEditProvider('json', …)` (`monaco.ts:60-61,71-80`) for Vega's
compact array-on-one-line style.
- **Editor options worth copying** (`spec-editor/renderer.tsx:263-274`): `folding:true`,
`minimap.enabled:false`, `scrollBeyondLastLine:false`, `wordWrap:'on'`,
`quickSuggestions:true` (this is what makes schema completions appear without an explicit
trigger), `stickyScroll.enabled:false`.
**Gotchas / where we improve:**
- ⚠️ **Workers are on us.** vega/editor never configures Monaco workers — the CDN loader does.
With raw `monaco-editor` + Vite we **must** set `self.MonacoEnvironment.getWorker` to return
the `json.worker` for label `'json'` and `editor.worker` otherwise (via `?worker` imports):
```ts
import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
import JsonWorker from 'monaco-editor/esm/vs/language/json/json.worker?worker';
self.MonacoEnvironment = {
getWorker: (_id, label) => (label === 'json' ? new JsonWorker() : new EditorWorker()),
};
```
The `json.worker` runs schema validation + autocomplete. **No worker ⇒ no squiggles, no
completions.** Upside: dropping the CDN loader makes `monaco` synchronously importable — no
`await loader.init()` dance, just call `setDiagnosticsOptions(...)` at module load.
- ⚠️ **`$schema`-based binding vs `fileMatch`.** They register schemas under versioned `uri`s
(`.../vega-lite/v6.json`) and bind by matching the doc's `$schema` value — **no `fileMatch`**
(`monaco.ts:15-46`). Consequence: a spec with **no `$schema` gets zero validation/autocomplete.**
Astrolabe should prefer `fileMatch` against our model URIs so validation works regardless of
whether the user wrote a `$schema` line.
- ⚠️ **Set `enableSchemaRequest:false`** for our offline-first app. They set it `true`
(`monaco.ts:54`), which lets the worker network-fetch any unbundled `$schema` URL — failing
network calls for an offline app. Register all schema versions locally instead.
- The schema is multi-MB; register it **once globally**, never per-model.
## 2 · Live preview with `vega-embed` (M1 lifecycle, M2 fit-mode)
This is doc [05](05-rendering-theming-preview.md)'s territory; these are the concrete details
vega/editor's hand-rolled renderer (`src/components/renderer/renderer.tsx`) reveals.
**What to borrow:**
- **Theme = a `vega-themes` config object merged into the spec config.** There is no automatic
light/dark sync in vega/editor — theme is an explicit choice baked in at compile
(`config-editor/config-editor-header.tsx:5-37`). For Astrolabe: pass the chosen `theme`/`config`
to `vegaEmbed`, and when our theme changes, re-embed with the new config.
- **`"width":"container"` / `"height":"container"` is how VL responsiveness works** — it
compiles to a `containerSize` signal (`renderer.tsx:78-90` detects this). Pair it with a
**`ResizeObserver`** on the preview pane → `view.resize().runAsync()`. This is cleaner than
vega/editor's `window.dispatchEvent(new Event('resize'))` hack (`renderer.tsx:101-122`) and is
the mechanism behind our M2 fit-mode contract.
- **Reuse the view for cheap changes.** They rebuild the `View` only on spec change; renderer
(svg/canvas) and tooltip toggles re-`initialize()` the existing view (`renderer.tsx:367-371`).
- **Capture warnings separately from errors** via a buffering logger (see §4's `LocalLogger`).
**Gotchas / where we improve:**
- ⚠️ **Finalize before re-embed, or leak.** Every spec change must `view.finalize()` the old
view *and* clear the container before mounting the new one (`renderer.tsx:218-226`). `vegaEmbed`
returns `{ view, finalize }` — call `finalize()` before the next embed and on unmount. This is
already a Do-rule in doc 05; vega/editor confirms how easy it is to leak otherwise.
- ⚠️ **Race on rapid edits.** `runAsync` is async; a stale render can resolve after a newer one
mounts. vega/editor mitigates only with debounce. **We should add a render-generation token**
and ignore stale resolves (an improvement over the reference).
- ⚠️ **Wrap `runAsync` in try/catch and finalize on failure** — Vega won't catch runtime errors
for you, and a half-initialized view leaks if you don't finalize (`renderer.tsx:247-259`).
- The **CSP-safe expression interpreter** (`vega-interpreter` + `vega.parse(..., {ast:true})`)
matters only under a strict no-`eval` CSP. A local offline app doesn't need it — keep it
opt-in.
## 3 · Two-tier validation & error surfacing (M2, spec §03E)
vega/editor runs **two independent schema-validation systems** with no reconciliation, and
sorts errors into two tiers. Both are worth copying.
**The two layers:**
1. **Monaco JSON worker** → inline **squiggles, hovers, autocomplete** in the editor.
2. **`ajv ^8`** (`src/utils/validate.ts`) → runs at parse time, feeds the **error/log pane**.
It does *not* create editor markers.
**The two error tiers (keep them separate):**
- **Fatal / blocking** — thrown exceptions: JSON syntax error, VL compile error, Vega runtime
error. These set a single `error` and suppress the chart.
- **Advisory** — ajv schema-validation findings and `$schema` version mismatch. These are a
warnings list and do **not** block rendering. (Vega-Lite emits many benign warnings; treating
ajv output as fatal would wrongly hide specs that render fine.)
The orchestration is `app.tsx:188-291`: `parseJSONCOrThrow` → `$schema` semver check (warn) →
`validateVegaLite` (ajv, warn) → `vegaLite.compile` (throw=fatal) → render (`renderer.tsx`,
throw=fatal).
**ajv setup specifics that *will* bite a from-scratch impl** (`validate.ts:9-17`):
- `new Ajv({ strict: false })` — the VL/Vega schemas fail ajv strict-mode at **compile** time
otherwise.
- The VL schema is **draft-06** → must `ajv.addMetaSchema(json-schema-draft-06.json)` (ajv 8
defaults to draft-07/2020) or `compile` throws.
- Register a no-op `color-hex` format (`ajv.addFormat('color-hex', () => true)`) plus
`addFormats(ajv)`; the schema references formats ajv-formats doesn't cover.
- **Compile the validator once at module load and cache it** — the schema is huge; compiling
per keystroke is a perf killer.
**Where we improve:** ajv errors are shown as JSON-pointer text (e.g. `/encoding/x`) with **no
editor position** — vega/editor does not map them to markers. Only JSON *syntax* errors get a
line/col (via jsonc-parser's visitor, `utils/jsonc-parser.ts:3-17`). If our §03E wants inline
ajv markers, we map `instancePath` → editor offsets ourselves via jsonc-parser's node tree —
something the reference does *not* do.
## 4 · Data flow & debouncing (M1/M2 — translate to Zustand stores)
vega/editor keeps **`editorString` (the text) as the single source of truth**; the parsed spec
and compiled Vega spec are *derived* and recomputed by a subscriber when text/mode/config change
(`app.tsx:338-365`). Errors don't clobber the last-good derived specs.
**The Zustand-store translation (this is the shape to build):**
```
text (store field, debounced writer on editor change)
└─▶ parsedSpec (derived: JSONC parse + collect syntax/diagnostic errors)
└─▶ renderInput (derived: prepareSpecForRender — refs, fit-mode)
└─▶ effect: deep-equal guard → vegaEmbed(); finalize previous view
```
**What to borrow:**
- **Debounce only at edit→state**, not state→render. vega/editor debounces the editor at
**1200 ms** (`spec-editor/renderer.tsx:66`) and guards the render with a `deepEqual` prop
diff (`renderer.tsx:340-349`). (1200 ms is *their* number; tune ours — our settings expose a
render-debounce preference.)
- **A manual-parse escape hatch** (Ctrl/Cmd+S re-parses without waiting) maps to a future
live-vs-manual preview toggle (`renderer.tsx:89-111`).
- **`LocalLogger` pattern** (`utils/logger.ts`): a logger that buffers `errors/warns/infos/debugs`
into arrays instead of writing to console. This lets a **pure** `src/core` compile/validate
step *return* structured diagnostics with zero browser coupling — e.g.
`validateSpec(spec) → { errors, warns }`. Ideal core-first fit.
- **`json-stringify-pretty-compact`** for the format action and prettify-on-load — much nicer
than `JSON.stringify(…, null, 2)` for VL specs.
**Persistence note:** vega/editor snapshots its whole state to localStorage on *every* change,
stripping non-serializable fields (`view`, `runtime`, editor refs) and restoring via
`{ ...DEFAULT_STATE, ...parsed }` (`context/app-context.tsx`). Our **debounced auto-save to
IndexedDB** (doc 01/02) is the better pattern — but the "strip non-serializable, restore with
defaults-spread" discipline is worth keeping.
---
## Borrow list (where each lands)
| Technique | Lands in | Milestone |
|---|---|---|
| Bundle VL schema from package `build/`; `setDiagnosticsOptions` | `src/app/infrastructure/` Monaco setup | M2 |
| `markdownDescription` patch + compact formatter | Monaco setup | M2 |
| Explicit Vite worker wiring (`MonacoEnvironment.getWorker`) | Monaco setup | M2 |
| `fileMatch` schema binding (improvement over `$schema`-only) | Monaco setup | M2 |
| jsonc-parser tolerant parse + line/col syntax errors | `src/core/` | M1/M2 |
| ajv wrapper (`strict:false`, draft-06, color-hex, compile-once) → structured diagnostics | `src/core/` | M2 |
| `LocalLogger`-style buffered diagnostics from pure compile | `src/core/` | M2 |
| Fatal-vs-advisory two-tier error model | rendering/store contract | M1/M2 |
| `"container"` sizing + `ResizeObserver` → `view.resize()` | `rendering.ts` + LivePreview | M2 |
| `finalize()`-before-reembed + **render-generation guard** | LivePreview | M1 |
| theme = `vega-themes` config merged into `vegaEmbed` | preview + settings | M5 |
| `json-stringify-pretty-compact` format action | editor | M2 |
## Where we deliberately do better than the reference
- **Wire Monaco workers explicitly** (they sidestep it via the CDN loader).
- **Map ajv errors to editor positions** via jsonc-parser offsets (they show pointer text only).
- **Render-generation guard** against stale async renders (they rely on debounce alone).
- **`fileMatch`-based schema binding** so validation works without a `$schema` line.
- **Debounced auto-save to IndexedDB** rather than write-the-whole-state-on-every-change.
## Key files in the reference (for deeper reads)
- `src/utils/monaco.ts` — all Monaco/schema wiring
- `src/utils/markdownProps.ts` — the `markdownDescription` patch
- `src/utils/validate.ts` — ajv setup + cached validators
- `src/utils/jsonc-parser.ts` — tolerant parse + line/col syntax errors
- `src/utils/logger.ts` — `LocalLogger` / `DispatchingLogger`
- `src/components/renderer/renderer.tsx` — the hand-rolled View lifecycle (finalize, sizing, errors)
- `src/components/app.tsx:188-365` — parse → $schema check → ajv → compile → render orchestration
- `src/components/error-pane/renderer.tsx` — error/log display
- `src/constants/default-state.ts` — the full app-state shape
- `src/components/input-panel/spec-editor/renderer.tsx` — editor component, 1200ms debounce, $schema→mode detection
docs/spec/00-product-overview.md
+60
View File
@@ -0,0 +1,60 @@
# 00 · Product Overview
This document set is a UX/behavioral specification for **Astrolabe**, a browser-based snippet manager for [Vega-Lite](https://vega.github.io/vega-lite/) visualizations. It describes *what the app does* from the user's perspective — its capabilities, workflows, and structural layout — so the app can be recreated on any web/HTML/TS stack. It deliberately avoids prescribing *how* anything is built: no frameworks, libraries, storage technologies, code structure, or concrete visual styling are mandated. Implementers are free to choose those.
## What Astrolabe Is
Astrolabe is a local-first tool for authoring, organizing, and previewing Vega-Lite charts. A user keeps a personal library of **snippets** (saved chart specifications), edits each one as JSON with live validation, and sees the result render in real time beside the editor. Reusable **datasets** can be stored once and referenced by many snippets. Everything lives in the user's browser — there is no account, no server, and no network dependency after first load.
## Who It Is For
People who work with Vega-Lite directly and want a fast, private workspace to draft, iterate on, and keep many visualizations: data practitioners, analysts, educators, and chart authors. Familiarity with Vega-Lite's JSON spec format is assumed; the app does not abstract Vega-Lite away (though the *Chart Builder* offers a no-JSON starting point).
## Core Value
- **Iterate quickly** — edit JSON and watch the chart update live, with schema-aware assistance and instant error feedback.
- **Stay organized** — a searchable, sortable library of named, annotated snippets.
- **Experiment safely** — a draft/published model lets users tinker without losing a known-good version.
- **Reuse data** — datasets stored once, referenced anywhere, in multiple formats and from inline data or remote URLs.
- **Own your data** — fully local, private, and offline-capable, with import/export for backup and transfer.
## Scope & Principles
- **Local-first** — all data is stored in the browser and survives reload; the app works fully offline and is installable as a standalone app.
- **Single-screen workspace** — a three-pane layout (library · editor · preview) plus modals for cross-cutting tools (datasets, chart builder, settings, help).
- **Vega-Lite native** — snippets *are* Vega-Lite specs; the app validates, renders, and reasons about them as such.
- **Keyboard-friendly and shareable** — common actions have shortcuts, and the current location is reflected in a shareable URL.
## Non-Goals
- No user accounts, authentication, or cross-device sync (use *Import & Export* to move data).
- No server-side storage, rendering, or processing.
- No collaboration or multi-user features.
- No general BI/dashboarding — a snippet is a single Vega-Lite visualization, not a composed report.
## Key Concepts (Glossary)
- **Snippet** — a saved Vega-Lite specification plus metadata (name, comment, timestamps, tags, dataset references). The primary user-authored entity. See *Snippet Library* and *Data Model & Persistence*.
- **Spec** — the Vega-Lite JSON specification that defines one visualization.
- **Draft vs Published** — each snippet holds a stable **published** spec and an editable **draft**; edits affect only the draft until the user publishes. See *Spec Editor & Draft/Published Workflow*.
- **Dataset** — a named, reusable data source (JSON/CSV/TSV/TopoJSON; inline or URL) that snippets reference by name. See *Datasets*.
- **Dataset reference** — a Vega-Lite named-data reference, e.g. `{ "data": { "name": "MyDataset" } }`, linking a spec to a stored dataset.
- **Live preview** — the rendered chart, updated automatically as the spec changes. See *Live Preview*.
## How This Specification Is Organized
| # | Section | Covers |
|---|---------|--------|
| 00 | Product Overview | This document — purpose, scope, glossary. |
| 01 | Application Shell & Navigation | Layout, panes, header, modals, keyboard shortcuts, URL state, toasts, offline/installable. |
| 02 | Snippet Library | Browsing, search, sort, metadata, create/duplicate/delete, storage monitor. |
| 03 | Spec Editor & Draft/Published Workflow | Editing, auto-save, auto-render, draft/publish/revert, extract-to-dataset. |
| 04 | Live Preview | Rendering, reference resolution, fit modes, error display. |
| 05 | Datasets | Dataset manager, formats, sources, profiling, references, linking. |
| 06 | Chart Builder | Visual no-JSON chart composition from a dataset. |
| 07 | Settings | Appearance, editor, performance, and formatting preferences. |
| 08 | Import & Export | Backup/transfer file format, import normalization and merging. |
| 09 | Data Model & Persistence | Entity field definitions, storage tiers, relationships. |
| 10 | Non-Functional Requirements | Platform, performance, accessibility, reliability, privacy. |
Read 00 first for orientation, then any section independently. Sections cross-reference one another by title where behavior spans more than one area.
docs/spec/01-application-shell.md
+115
View File
@@ -0,0 +1,115 @@
# 01 · Application Shell & Navigation
This section describes the overall workspace structure, the header toolbar, the modal system, keyboard shortcuts, URL-based navigation, transient notifications, and the offline/installable nature of the app. Feature-specific behavior lives in the sections referenced inline.
## A. Workspace Layout
Astrolabe is a single-screen workspace. Below a fixed top header sits a three-pane working area, each pane dedicated to one part of the snippet-editing workflow:
- **Snippet library** (left) — browse, search, select, and manage saved snippets (see *Snippet Library*).
- **Spec editor** (center) — edit the Vega-Lite spec of the selected snippet (see *Spec Editor & Draft/Published Workflow*).
- **Live preview** (right) — render the current spec (see *Live Preview*).
Behavior:
- All three panes are visible by default, laid out side by side in the order library, editor, preview.
- Adjacent panes are separated by a vertical drag handle. The user can drag a handle left/right to resize the two panes it sits between; the rest of the layout is unaffected.
- Each pane enforces a minimum width while resizing, so a pane cannot be dragged to nothing.
- Each pane can be individually shown or hidden via a persistent **toggle strip** (a narrow vertical strip of toggle buttons, one per pane, each indicating whether its pane is currently shown). The toggle strip also contains a shortcut button that opens the Datasets manager.
- When a pane is hidden, the remaining visible panes expand to fill the freed space, redistributing proportionally to their remembered widths. When a previously hidden pane is shown again, it returns at its remembered width.
- Hiding all panes is permitted; the toggle strip remains available to bring panes back.
- Pane widths and per-pane visibility persist locally across sessions and are restored on next load. The app remembers a pane's preferred width even while it is hidden, so re-showing it restores that width rather than an arbitrary one.
## B. Header / Toolbar
A fixed header spans the top of the app.
- **Left side**: the app icon, the app title ("Astrolabe"), and a version badge showing the current app version.
- **Right side**: a row of text entry points. Each opens a destination:
| Entry point | Opens |
|---|---|
| Import | A file-picker dialog to choose a previously exported file; the chosen file is imported (see *Import & Export*). |
| Export | Immediately produces a downloaded file containing all snippets and datasets (see *Import & Export*). |
| Datasets | The Datasets manager modal (see *Datasets*). |
| Settings | The Settings modal (see *Settings*). |
| About & Privacy | The About & Help modal (keyboard shortcuts, about, and privacy information). |
| Donate | The Donate modal. |
Notes:
- Import and Export act directly (file dialog / file download); they do not open in-app modals.
- The Datasets, Settings, About & Privacy, and Donate entry points each open a modal (see *Modal System*).
## C. Modal System
The app shows at most one modal at a time. The modal set is: Datasets, Settings, About & Help, Donate, Chart Builder, and Extract-to-Dataset.
- Opening any modal closes whichever modal was previously open; the two never overlap.
- Every modal can be dismissed by: clicking its close button, pressing **Escape**, or clicking the backdrop outside the modal body.
- Clicking inside the modal body does not dismiss it.
- The Chart Builder and Extract-to-Dataset modals are opened from within the Datasets / snippet workflows (see *Chart Builder* and *Datasets*), not from the header.
- Dismissing a modal returns the user to the underlying workspace unchanged.
## D. Keyboard Shortcuts
Shortcuts are platform-aware: the modifier is **Cmd** on Mac and **Ctrl** on other platforms (shown below as Cmd/Ctrl).
| Shortcut | Action |
|---|---|
| Cmd/Ctrl + Shift + N | Create a new snippet (see *Snippet Library*) |
| Cmd/Ctrl + K | Toggle the Datasets manager open/closed |
| Cmd/Ctrl + S | Publish the current snippet's draft (see *Spec Editor & Draft/Published Workflow*) |
| Cmd/Ctrl + , | Open the Settings modal |
| Escape | Close the active modal |
Notes:
- Cmd/Ctrl + K is a toggle: if the Datasets manager is already open it closes it; otherwise it opens it.
- Escape only acts when a modal is open; with no modal open it does nothing.
- The shortcut actions override the browser's default behavior for those key combinations.
## E. Navigation & Shareable URL State
The app reflects its current location in the URL hash so that reloading restores the same view and the browser's Back/Forward buttons move between prior states. The user can copy the URL to share or bookmark a specific location.
States and their hash forms:
| State | Hash |
|---|---|
| A selected snippet | `#snippet-<id>` |
| Datasets manager (list) | `#datasets` |
| A specific dataset | `#datasets/dataset-<id>` |
| New-dataset form | `#datasets/new` |
| Chart Builder for a dataset | `#datasets/dataset-<id>/build` |
Behavior:
- Selecting a snippet updates the URL to that snippet; reloading reopens that snippet.
- Opening the Datasets manager updates the URL to `#datasets`; opening a specific dataset, the new-dataset form, or the Chart Builder for a dataset updates the URL to the corresponding form above.
- Browser Back/Forward navigate between these states (e.g. closing a modal via Back returns to the previously selected snippet).
- On load, the app reads the hash and restores the corresponding state (selected snippet, Datasets list, a specific dataset, the new-dataset form, or the Chart Builder).
- An empty/absent hash opens the default snippets view with no modal.
## F. Toast Notifications
Transient toast messages appear in a corner of the screen to confirm actions or report problems, without interrupting the workflow.
- Four kinds, each visually distinct: **success**, **error**, **warning**, **info**.
- Each toast auto-dismisses after a few seconds, and can also be dismissed manually via its close control.
- Multiple toasts stack rather than replacing one another, and appear/disappear with a brief fade.
Events that raise toasts include:
- Snippet actions: creating, duplicating, deleting, publishing a draft, and reverting/discarding a draft.
- Dataset actions: creating, deleting, and extracting inline data into a dataset.
- Import/Export: results of an import (success/partial/failure) and confirmation of an export.
- Errors: spec/data validation failures and local-storage capacity warnings.
## G. Offline & Installable
Astrolabe is local-first and usable without a network connection.
- After the first successful load, the app works fully offline; the interface and previously loaded content remain available with no connection.
- All snippets, datasets, and settings are stored locally and remain accessible offline (see *Data Model*).
- The app is installable as a standalone application from a supporting browser and, once installed, launches in its own window.
docs/spec/02-snippet-library.md
+82
View File
@@ -0,0 +1,82 @@
# 02 · Snippet Library
The Snippet Library is the left pane and the primary entry point to the app. A **snippet** is a saved Vega-Lite specification together with metadata (name, comment, timestamps, tags, references to external datasets). The library lets the user browse, search, sort, select, and manage their snippets. Editing the specification, the draft-vs-published workflow, the live preview, and dataset management are covered elsewhere (see *Spec Editor & Draft/Published Workflow*, *Live Preview*, *Datasets*); this section covers only the library and management surface.
## The List
The list shows every saved snippet and is always visible. A persistent "Create New Snippet" affordance sits at the top of the list, above all snippets, so the user can always start a new snippet regardless of scroll position.
- The list shows all snippets, ordered newest-modified first by default (see *Sort*).
- A "Create New Snippet" item is pinned at the top of the list; activating it creates and selects a new snippet (see *Snippet Operations*).
- Selecting a snippet makes it the **active snippet**: it loads into the editor and preview, becomes highlighted in the list, and the URL updates to reflect the selected snippet so the state is shareable and survives a page reload (see *Application Shell & Navigation*).
- Exactly one snippet is active at a time.
- When no snippets match the current search, the list shows an empty-state message ("No snippets match your search"); when there are genuinely no snippets, it shows "No snippets found".
- On first run, when no snippets exist, the app seeds one sample bar-chart snippet so the user starts with a working example.
## List Item
Each list item is a compact row summarizing one snippet, designed for fast scanning.
- Shows the snippet **name**.
- Shows a **last-modified date**, rendered relatively for recent items ("Today", "Yesterday", "Nd ago" within the past week) and as a full date beyond that, formatted per the user's date-format setting (see *Settings*). When sorting by Created, the item shows the created date instead of the modified date.
- Shows the snippet **size** (in KB), but only once the snippet reaches at least about 1 KB; smaller snippets omit the size to reduce clutter.
- Shows a **status indicator** distinguishing a snippet that has unpublished draft changes from one that is fully published (the indicator communicates "draft" vs "published"). The publish and revert actions themselves live in *Spec Editor & Draft/Published Workflow*.
- Shows a small **dataset icon** when the snippet references one or more external datasets (see *Datasets*); the icon is omitted otherwise.
- The active snippet is visually highlighted.
## Search
A live search box lets the user narrow the list as they type. It exists so users with many snippets can find one by name, by note, or by something inside the specification itself.
- The search box filters the list immediately on each keystroke.
- Matching is case-insensitive and spans the snippet **name**, the snippet **comment**, and the **specification content** (the current working/draft spec text), so a search for a field name, mark type, or dataset name in the spec will surface matching snippets.
- The search has a clear control that empties the box and returns focus to it, restoring the full list.
- Search affects only which snippets are shown; it does not change the active snippet or any data.
## Sort
The user chooses how the list is ordered. The choice persists across sessions so the library always opens the way the user left it.
- Sort fields: **Modified**, **Created**, **Name**, **Size**.
- An ascending/descending toggle controls direction; the current field and direction are indicated (e.g. a directional arrow on the active field).
- Selecting the already-active sort field flips the direction; selecting a different field switches to it and resets to descending.
- Default ordering is **Modified, descending** (newest changes first).
- Name sorts alphabetically; Size sorts by stored snippet size; Created and Modified sort chronologically.
- The **Modified** time advances on every save — including silent draft auto-saves (see *Spec Editor & Draft/Published Workflow*) and inline name/comment edits — so under the default Modified-descending sort the active snippet continually rises to the top while it is being edited.
- The selected sort field and direction persist across sessions.
## Selected-Snippet Metadata Panel
When a snippet is active, a metadata panel (within the left pane) exposes its editable properties and key facts. It exists so the user can rename, annotate, and inspect a snippet without leaving the library.
- Shows and lets the user edit the **Name** inline; edits save automatically.
- Shows and lets the user edit a multiline **Comment** (free-form notes); edits save automatically.
- Shows read-only **Created** and **Modified** timestamps, formatted per the user's date-format setting (see *Settings*).
- When the snippet references external datasets, shows a **Linked Datasets** list of the referenced dataset names, each with a dataset icon (see *Datasets*). The list is omitted when there are no references.
- The panel also exposes the Duplicate and Delete operations for the active snippet (see *Snippet Operations*).
## Snippet Operations
The library provides the lifecycle operations for snippets. Each operation gives clear feedback via a toast notification (see *Application Shell & Navigation*).
- **Create New**: starts a new snippet from a small sample Vega-Lite bar-chart template (a few inline category/value rows), assigns it an auto-generated default name (see *Naming & Tags*), saves it, and makes it the active snippet.
- **Duplicate**: creates an independent copy of the active snippet with a name suffixed "(copy)". The copy carries over the specification, comment, tags, and dataset references, gets fresh created/modified timestamps and a new identity, and becomes the active snippet. A success toast confirms the duplication.
- **Delete**: permanently removes the active snippet after the user confirms a warning that the action cannot be undone. After deletion no snippet is active. A toast confirms the deletion.
- These operations never affect other snippets.
## Naming & Tags
New snippets get a sensible default name, and a tag field exists on each snippet for categorization, though tags are not a primary user surface.
- A new snippet receives an auto-generated default name based on the current date and time, so it is uniquely identifiable until the user renames it (renaming happens in the metadata panel).
- Each snippet stores a list of **tags**. Tags are persisted and carried through duplication; for example, snippets brought in via import are tagged "imported" (see *Import & Export*).
- There is no dedicated tag-management UI; tags are stored on the data model but are not surfaced as a primary browsing or editing control.
## Storage Monitor
A small indicator at the bottom of the library shows how much of the snippet storage budget is in use, warning the user before they run out of room. This concerns snippet storage specifically; datasets are stored separately with far greater capacity (see *Datasets* / *Data Model*).
- Displays current usage against the total budget (used vs. total), where the practical snippet budget is about 5 MB.
- A fill indicator reflects the percentage used.
- The indicator enters escalating warning states as usage climbs (a cautionary state past roughly 70% and a critical state past roughly 90%).
- When storage is full, a save may fail; the system warns the user that the snippet could not be saved rather than silently losing data, so the user can delete snippets to free space.
docs/spec/03-editor-and-drafts.md
+78
View File
@@ -0,0 +1,78 @@
# 03 · Spec Editor & Draft/Published Workflow
The center pane is where the user reads and edits the active snippet's Vega-Lite specification. It is a code editor for JSON paired with a Draft/Published workflow: edits are made against a working draft that auto-saves silently, drive the *Live Preview* automatically, and become the snippet's stable version only when explicitly published. The pane is empty when no snippet is selected; it loads the active snippet's spec when one is chosen in the *Snippet Library*.
## A. The Spec Editor
The editor presents the active snippet's spec as formatted JSON with full code-editing affordances tuned for Vega-Lite.
- The editor displays the spec as indented, readable JSON with syntax highlighting.
- As the user types, the spec is validated against the Vega-Lite schema; problems are surfaced as inline indicators at the offending locations (e.g. squiggles/markers), without blocking continued editing.
- The editor offers schema-driven autocomplete/suggestions while typing (property names and allowed values from the Vega-Lite schema).
- Pasting content and typing trigger automatic reformatting so the JSON stays consistently indented.
- The editor's appearance and behavior — font size, editor theme, minimap visibility, word wrap, line numbers, and tab size — are configurable and read from *Settings*; this section does not redefine their defaults.
- The editor always edits a single active snippet. Selecting a different snippet in the *Snippet Library*, or toggling the Draft/Published view, replaces the editor content with the corresponding spec.
## B. Auto-Save of the Draft
Edits persist automatically so the user never loses work and never needs an explicit "save" action for ordinary editing.
- A short moment after the user stops typing, the current editor content is parsed and stored as the snippet's working draft, silently and with no notification.
- Auto-save only commits when the editor content is valid JSON; if the content is momentarily unparseable, the save is skipped and retried after the next pause in typing, so a half-typed spec never overwrites the stored draft.
- Auto-save writes the **draft** only. It never alters the published version (see *D. Draft vs Published*).
- Auto-save is distinct from Publish: auto-save preserves in-progress work; Publish promotes that work to stable.
## C. Auto-Render to Preview
Edits flow to the *Live Preview* automatically, so the user sees results without invoking a render.
- A brief moment after the user stops typing, the current spec is sent to the *Live Preview* for rendering.
- The delay before rendering is a configurable debounce (see *Settings* / *Live Preview*), letting the user trade responsiveness against churn while typing heavy specs.
- Rendering also occurs immediately when a snippet is first loaded into the editor or the Draft/Published view is switched.
- Rendering specifics (dataset reference resolution, fit modes) belong to *Live Preview*; the editor's role is to supply the current spec text on each settle.
## D. Draft vs Published Workflow
Every snippet carries two versions of its spec: a **published** (stable) version and a **working draft**. This separation is the central editing model. A view toggle in the pane header switches which version the editor shows.
- The header offers a Draft/Published toggle; the currently active view is visually indicated.
- **Draft view** shows the working draft and is the editable surface — all typing, auto-save, and auto-render act on the draft.
- **Published view** shows the last published version, for reference; the published version is never modified by ordinary editing.
- Editing the draft never touches the published version until the user publishes.
- The *Snippet Library* status indicator reflects whether a snippet currently has unpublished draft changes (draft differs from published); this section only produces that difference, it does not render the indicator.
### Publish
- A **Publish** action promotes the current draft to become the published version (the two are made identical).
- Publish is also triggered by the keyboard shortcut Cmd/Ctrl+S.
- On publish, the snippet's dataset references are recomputed from the now-published spec (see *Datasets* for reference linking).
- A success toast confirms the snippet was published.
- Publish is unavailable when no snippet is active.
### Revert
- A **Revert** action discards all draft changes and restores the draft to match the last published version.
- Revert requires explicit confirmation before discarding, warning that the action cannot be undone.
- On confirmation, the editor reloads with the published spec and a toast confirms the draft was reverted.
- Revert is unavailable when no snippet is active.
## E. Inline Error Surface
When the spec cannot be parsed or cannot be rendered, the editor pane shows the problem clearly while keeping the user in place to fix it.
- When the spec is invalid JSON, or is valid JSON but fails to render as Vega-Lite (including an unresolved dataset reference), a clear, readable error message appears in the editor pane, near the editor area.
- The error message is plainly legible (monospaced, distinct from normal content) and conveys what went wrong.
- The editor remains fully usable while an error is shown, so the user can edit to fix it; the error clears automatically once a subsequent edit renders successfully.
- This is the editor-side error affordance only; how a valid spec is drawn lives in *Live Preview*.
## F. Extract Inline Data to a Dataset
When a snippet's spec embeds its data inline, the user can lift that data out into a reusable, named dataset and have the spec reference it instead. This keeps specs lean and lets the same data serve multiple snippets (see *Datasets*).
- When the active snippet's draft spec contains inline data, an **Extract to Dataset** action is available in the pane header; it is hidden when the spec has no inline data.
- Choosing it opens a modal that shows a read-only preview of the inline data and asks the user for a dataset name (required).
- The user enters a name and confirms creation. Names must be non-empty and unique; if the name is blank or already in use, the modal shows an inline error and the action does not proceed.
- On success, the system: saves the inline data as a new dataset (preserving its detected format), rewrites the snippet's draft spec so the inline data is replaced by a reference to the dataset by name, links the dataset to the snippet, and reloads the editor to show the rewritten spec.
- A toast confirms the dataset was created, and the modal closes.
- The user can cancel the modal at any time, leaving the spec unchanged.
- Dataset-side specifics (formats, storage, the bidirectional snippet↔dataset link) are described in *Datasets*.
docs/spec/04-live-preview.md
+81
View File
@@ -0,0 +1,81 @@
# 04 · Live Preview
The right pane renders the active snippet's current specification as a live Vega-Lite visualization. It mirrors whatever the editor currently shows and updates on its own as the user types, giving immediate visual feedback without any explicit "run" action.
## Purpose & Live Updating
- Renders the active snippet's current spec as a Vega-Lite visualization.
- Always reflects the version currently shown in the editor: while the user edits the draft, the preview renders the draft; once published/viewing the published version, it renders that (see *Spec Editor & Draft/Published Workflow*).
- Updates automatically as the user edits, after a brief render debounce so rapid keystrokes do not trigger constant re-rendering. The debounce delay is user-configurable (see *Settings*).
- A subtle busy indication may appear over the preview while a render is in progress; it clears when rendering completes.
- When no snippet is active, or the editor content is empty/blank, the preview renders nothing (a clean, empty pane) rather than showing an error.
## Dataset Reference Resolution
When a spec uses inline data, the preview renders it directly. When a spec instead references a named dataset from the library, the preview resolves that reference and renders using the stored dataset's contents (see *Datasets*).
- A spec may point at a dataset from the library by name instead of embedding the data inline.
- Before rendering, the preview substitutes the referenced dataset's stored contents into the spec.
- URL-sourced datasets are fetched as needed at render time.
- If a referenced dataset cannot be found or fetched, the preview shows a readable error (see *Error Display*) rather than a broken chart.
## Fit / Sizing Modes
The preview pane header has a "Fit" control offering exactly four modes that determine how the chart is sized within the pane. The chosen mode applies immediately and re-renders the current chart.
- **Original** — renders the chart at its natural size as defined by the spec. If the chart is larger than the pane, it overflows and the pane provides scrolling to reach the rest.
- **Width** — fits the chart's width to the pane (the width becomes responsive to the pane); the height is left to the chart's own natural sizing.
- **Height** — fits the chart's height to the pane; the width is left to the chart's own natural sizing.
- **Full** — fits the chart to the pane in both dimensions, so it occupies the full available width and height.
The exact spec transform each mode performs is defined in *Rendering Contract* below.
Behavior of the selected mode:
- The control shows the four modes with the active one visibly indicated.
- The selected mode persists across sessions, stored in *Settings* as `previewFitMode`.
- The default is the natural Original mode.
## Rendering Contract
Before the chart is drawn, the spec shown in the editor is transformed into the spec actually rendered. Two deterministic transforms are applied in order. They are specified here because reproducing them faithfully is what makes references and fit modes behave correctly; the result is observable as the rendered chart.
**1. Dataset reference resolution.** Any named-data reference (`data` with a `name`) is replaced in-place with the referenced dataset's actual contents, shaped by the dataset's source and format (see *Datasets*):
| Dataset source / format | The reference's `data` becomes |
|---|---|
| URL (any format) | a URL reference to the dataset's address, tagged with its format |
| Inline JSON | the parsed values, inlined |
| Inline CSV / TSV | the raw text, inlined, tagged with its format (CSV or TSV) |
| Inline TopoJSON | the value, inlined, tagged as TopoJSON |
- Resolution recurses into nested sub-specs (layered and concatenated specs, and a parent spec's child `spec`), so references anywhere in the spec are resolved.
- If a referenced dataset does not exist, rendering fails with a "dataset not found" error (see *Error Display*).
**2. Fit-mode sizing.** The selected fit mode rewrites the spec's sizing using Vega-Lite's responsive `"container"` sizing keyword, recursing into the same nested sub-specs:
| Mode | Transform |
|---|---|
| Original | spec sizing left untouched (the spec's own `width`/`height`, or Vega-Lite defaults, apply) |
| Width | set `width` to `"container"`; remove any explicit `height` |
| Height | set `height` to `"container"`; remove any explicit `width` |
| Full | set both `width` and `height` to `"container"` |
- For the responsive (non-Original) modes the chart's container-relative dimension follows the pane size, while the unconstrained dimension is recomputed naturally — this is why Width/Height do not preserve the original aspect ratio.
- The transform operates on a copy; the user's stored spec is never modified by rendering.
The preview renders the resulting spec without the charting library's built-in action/export menu, so the output is a clean chart with no overlaid controls.
## Error Display
When a spec cannot be rendered, the preview replaces the chart area with a clear, readable error message rather than a broken or partial visualization, and recovers on its own once the spec becomes valid again.
- Invalid JSON, incomplete specs, Vega-Lite errors, and data problems (e.g. a missing or unfetchable dataset) all surface as a legible error message.
- The message identifies it as a rendering error and includes the underlying reason, with a hint to check the JSON syntax and the Vega-Lite specification.
- As soon as the spec becomes valid again, the error clears automatically and the chart renders without any manual retry.
- Empty/blank specs are not treated as errors — they simply render nothing.
## Responsiveness
- The preview re-fits when the pane is resized, re-applying the current fit mode so the chart continues to honor the chosen sizing (see panes in *Application Shell & Navigation*).
- Resizing does not require a manual refresh; the displayed chart adapts to the new pane dimensions.
docs/spec/05-datasets.md
+107
View File
@@ -0,0 +1,107 @@
# 05 · Datasets
The **Dataset Manager** is a modal for creating and managing named, reusable datasets that snippets can reference by name. It is the home of the dataset library: a place separate from snippets where data lives once and is shared across many visualizations.
## Purpose & Model
Datasets are named blobs of data stored in the user's local library, independent of any single snippet. A snippet references a dataset by name rather than embedding the data inline, so the same data can power many snippets and be edited in one place.
- Datasets persist locally across sessions in a high-capacity local store, far larger than the budget available to snippets — large datasets belong here, not inline in specs.
- A snippet references a dataset using a Vega-Lite named-data reference, e.g. `{ "data": { "name": "MyDataset" } }`. When the *Live Preview* renders a spec, it resolves any such named reference against the dataset library (see *Live Preview*).
- See *Data Model* for the stored shape of a dataset.
## Opening & Navigation
- Opened from a header control or via the keyboard shortcut Cmd/Ctrl+K.
- The current view and the selected dataset are reflected in the URL, so a selected dataset produces a shareable/back-navigable location (see *Application Shell & Navigation*).
- Closing the modal clears the current selection and any open create form.
## Layout
A two-pane modal:
- **List pane** (left): a "New Dataset" action plus the list of all datasets, sorted most-recently-modified first.
- **Detail pane** (right): shows the selected dataset's details, the create form when creating, or an empty prompt ("Select a dataset or create a new one") when nothing is selected.
### List item
Each list item shows:
- The dataset **name**.
- A **meta line** combining: source ("URL" prefix for URL datasets), row count when known, the **format label** (JSON / CSV / TSV / TOPOJSON), and **size** (human-readable, e.g. B / KB / MB). For URL datasets where counts are not yet known, only the source and format label are shown.
- A **usage badge** when one or more snippets reference the dataset, indicating how many.
Clicking an item selects it and shows its detail. Per-item actions (delete, plus copy-reference and build-chart) live in the detail pane for the selected dataset.
## Source Types
A dataset has one of two source types, chosen when creating it:
- **Inline** — the data itself is pasted in and stored directly in the library.
- **URL** — the dataset stores a remote URL (http/https). The data is not copied locally; it is fetched on demand when a referencing spec is rendered (see *Live Preview*).
For inline datasets the library holds the full data and can profile it. For URL datasets the library holds only the link, so row/column/size figures are typically not computed up front and show as "N/A".
## Supported Formats
Four data formats are supported, named in the UI and stored on the dataset:
- **JSON** — an array of objects (most common, profilable) or a single object.
- **CSV** — comma-separated with a header row.
- **TSV** — tab-separated with a header row.
- **TopoJSON** — topology/map data (a JSON object whose type marks it as a topology).
### Auto-detection
When the user pastes inline data, the app auto-detects the format and reports a **confidence** level (high / medium / low):
- Valid JSON parses to JSON, or to TopoJSON when it is a topology object — high confidence.
- Otherwise, multi-line text with a header row is detected as TSV (when tab-separated) or CSV (when comma-separated) — medium confidence.
- Unrecognized input yields no format (low confidence); saving is blocked with a message asking the user to check the input.
The detected format and source are shown as badges in the create form so the user can confirm or override the source (Inline/URL) before saving. For URL datasets the format is inferred from the URL's file extension (`.csv`, `.tsv`, `.json`, `.topojson`) and shown as a hint.
## Profiling
For tabular inline data (JSON array-of-objects, CSV, TSV) the app computes and stores a profile:
- **Row count** and **column count**.
- The list of **column names**.
- An **inferred type per column**: number, text/string, date, or boolean. Type inference looks at the column's values: all-numeric becomes number, all `true`/`false` becomes boolean, otherwise string; empty cells are ignored.
- **Size** in bytes of the stored data.
A **truncated data preview** of the raw data is also retained for display. URL datasets and non-tabular data are not profiled (counts show "N/A").
## Detail Panel
The detail pane for a selected dataset shows:
- **Name**.
- **Comment** (optional free-text notes), when present.
- **Overview**: statistics (rows, columns, size), the **column list** with each column's name and inferred type shown with a simple type indicator, and created/modified timestamps.
- **Preview**: a truncated rendering of the data (raw text for CSV/TSV/URL, pretty-printed for JSON/TopoJSON).
- **Linked Snippets**: the list of snippets that reference this dataset by name. This is the dataset side of bidirectional dataset↔snippet linking (see *Snippet Library*).
## Actions
Each action raises a confirming toast (or an error toast on failure).
- **Copy Reference** — copies the by-name reference object to the clipboard, ready to paste into a spec:
`{ "data": { "name": "MyDataset" } }`
- **New / Create New** — opens the create form in the detail pane with fields: **name** (required, unique), **source** toggle (Inline / URL), the **data** (a paste area for inline, a URL field for URL source), and an optional **comment**. Save is disabled until a name and valid data/URL are present. On success the new dataset is selected.
- **Edit** — rename, edit the comment, and update the data (re-paste inline data or refresh the URL). Updating inline data re-profiles it; the modified timestamp advances.
- **Delete** — asks for confirmation ("Delete \"Name\"? This cannot be undone."), then removes the dataset and clears the selection.
## Build Chart From Dataset
From a selected dataset the user can launch the visual *Chart Builder* (see *Chart Builder*) pre-targeted at that dataset, producing a new snippet whose spec references the dataset by name.
## Extract Inline Data → Dataset
The reverse flow starts in the editor: a user can extract inline `data.values` out of a spec into a new named dataset (see *Spec Editor & Draft/Published Workflow*). The result appears here as a new dataset, and the originating snippet's spec is rewritten to reference it by name.
## Naming & Uniqueness
- Dataset names must be **unique**. Attempting to create a dataset with a name already in use is rejected with an error toast.
- During bulk operations such as import, conflicting names are automatically suffixed to remain unique rather than overwriting existing datasets (see *Import & Export*).
- Renaming a dataset that is referenced by snippets keeps references consistent by updating the matching named-data references in affected specs.
docs/spec/06-chart-builder.md
+70
View File
@@ -0,0 +1,70 @@
# 06 · Chart Builder
The Chart Builder is a visual, no-JSON way to compose a Vega-Lite chart from a selected dataset. The user picks a mark type and maps the dataset's columns to encoding channels; the builder produces a complete Vega-Lite spec and saves it as a new snippet that references the dataset. It is intended for users who want to start a chart quickly without hand-writing JSON in the *Spec Editor & Draft/Published Workflow*.
## Opening
- Launched from a selected dataset in the *Datasets* manager via that dataset's "build chart" action.
- Opens as a modal dialog over the application; the URL reflects the dataset's "build" action so the open builder is shareable/restorable (see *Application Shell & Navigation*).
- On open, the builder loads the selected dataset, displays its name, and pre-populates sensible defaults (see below). If no dataset is available, it shows a "No dataset loaded" message and offers no controls.
## Layout
A two-pane modal:
- **Left — configuration:** dataset name, mark type selector, one row per encoding channel, optional width/height inputs, and a "Create Snippet" action.
- **Right — live preview:** a rendered chart that updates as the configuration changes, with a placeholder/error area.
## Inputs and Controls
### Mark type
- Single selection from an exact set of five mark types: **Bar, Line, Point, Area, Circle**.
- Defaults to **Bar**.
- Exactly one mark type is active at any time; selecting one updates the preview.
### Encoding channels
- Exactly four channels are offered, in this order: **X, Y, Color, Size**.
- For each channel the user:
- Picks a dataset column from a dropdown of the dataset's detected columns (see *Datasets* for column detection). A "None" option leaves the channel unmapped. Each column option shows a small type indicator alongside the column name.
- Optionally overrides the channel's **field type**, chosen from an exact set: **Quantitative, Nominal, Ordinal, Temporal**. The type override only appears once a column is selected for that channel.
- When a column is chosen, its field type defaults from the dataset's inferred column type (numeric → Quantitative, date → Temporal, otherwise Nominal); the user may change it afterward.
- Clearing a channel back to "None" leaves it out of the produced spec.
### Default pre-population
- On open, the first detected column is assigned to **X** and the second (if any) to **Y**, each with its derived field type. Remaining channels start unmapped. Mark type starts at Bar.
### Dimensions (optional)
- Optional numeric **Width** and **Height** inputs in pixels.
- When left empty, the chart uses default/responsive sizing (consistent with *Live Preview*); when provided, the values are written into the spec.
## Live Preview
- The right pane renders the chart described by the current mark, encodings, and dimensions, resolving the dataset reference to its actual data (same rendering behavior as *Live Preview*).
- Updates are debounced: changes to mark, encodings, or dimensions trigger a re-render after a short pause rather than on every keystroke.
- While no encoding is mapped, the pane shows a placeholder instructing the user to configure at least one encoding.
- If the spec fails to render, the pane shows an inline error message describing the problem instead of a chart.
## Validation
- A chart requires **at least one** channel mapped to a column.
- While no channel is mapped, the "Create Snippet" action is disabled and the preview shows the configuration prompt.
## Output / Create
Selecting "Create Snippet" produces the final artifact:
- Builds a complete Vega-Lite spec containing: the schema reference, a named data reference to the dataset, the chosen mark (with tooltips enabled), the mapped encodings (each with its field and field type), and any explicit width/height.
- Channels left unmapped are omitted; if no encodings exist the spec omits the encoding block entirely (prevented by validation here).
- Creates a new snippet from that spec with an auto-generated descriptive name, adds it to the snippet library, and records that it was built from the dataset.
- Links the snippet to the dataset by recording the dataset reference, so the bidirectional snippet↔dataset relationship is established (see *Datasets*).
- Raises a success toast naming the created snippet.
- Closes the builder; the newly created snippet becomes the active snippet in the library/editor.
## Closing
- The builder can be dismissed without creating anything (close control / modal dismissal).
- Closing resets all builder state (dataset, mark type, encodings, dimensions, preview) so a later open starts fresh, and any pending preview render is cancelled.
docs/spec/07-settings.md
+79
View File
@@ -0,0 +1,79 @@
# 07 · Settings
Astrolabe provides a **Settings** modal where users tune appearance, the spec editor, preview performance, and date formatting. All settings persist locally and apply across sessions on the same device. Settings load at startup; any unknown or missing value falls back to its factory default, so older or partial saved settings never break the app.
## Opening the modal
- An entry in the application header opens the Settings modal.
- The keyboard shortcut **Cmd/Ctrl+,** also opens it.
- The modal is grouped into clearly titled sections: Appearance, Editor, Performance, and Formatting.
- The modal can be dismissed with a Cancel action or the standard modal-close affordance; dismissing without applying discards any pending edits and restores the last saved values.
## Settings
### Appearance
Controls the overall UI theme. Choosing the experimental Dark theme switches the whole application chrome to a dark presentation.
| Setting | Options | Default |
| -------- | ----------------------------- | ------- |
| UI theme | Light, Experimental Dark | Light |
### Editor
These settings configure the spec editor used to edit Vega-Lite specs (see *Spec Editor & Draft/Published Workflow*). They take effect in the editing surface for the snippet spec.
| Setting | Options / Range | Default |
| ------------- | ------------------------------------- | -------- |
| Font size | 10–18 px (integer) | 12 px |
| Editor theme | Auto + explicit overrides (provisional — see note) | Auto |
| Minimap | On / Off | Off |
| Word wrap | On / Off | On |
| Line numbers | On / Off | On |
| Tab size | Integer number of spaces | 2 |
- Font size is chosen along a 10–18 range; the current value is shown alongside the control.
- Editor theme controls the syntax/color presentation inside the editor. **Provisional (to be finalized as we implement the editor):** the default is **Auto**, which derives the editor theme from the app UI theme (light app theme → light editor theme, experimental → dark), using custom Monaco themes that match the app chrome. The user may override Auto with an explicit editor theme; the exact override list (custom themes, and whether to include High Contrast or the stock Monaco themes) is deferred. Stored as `editor.theme` with an `'auto'` sentinel for the follow-the-app default.
- Minimap toggles the condensed overview strip beside the editor.
- Word wrap toggles soft wrapping of long lines.
- Line numbers toggles the line-number gutter.
- Tab size sets the indentation width applied while editing.
### Performance
| Setting | Range | Default |
| --------------- | -------------------- | -------- |
| Render debounce | 500–5000 ms | 1500 ms |
- Render debounce is the delay after the user stops typing before the preview re-renders (see *Live Preview*).
- Tradeoff: a lower value makes the preview feel snappier and more immediate but re-renders more often and uses more CPU; a higher value keeps the app calmer and lighter but makes the preview feel laggier behind the spec.
- The current value is shown alongside the control.
### Formatting
Governs how dates are rendered throughout the app, for example the timestamps shown in the *Snippet Library* list.
| Setting | Options | Default |
| ------------------- | ----------------------------------------- | ------- |
| Date format | Smart, ISO 8601, Custom | Smart |
| Custom date format | Free-text format string | (empty) |
- **Smart**: relative, human-friendly rendering (e.g. "Today", "Yesterday", "3d ago", falling back to a full date for older items).
- **ISO 8601**: a full ISO 8601 timestamp.
- **Custom**: dates render using the user-supplied format string.
- The custom format string field is only relevant when Date format is set to Custom; it is shown only in that case (placeholder guidance such as `yyyy-MM-dd HH:mm`).
## Related persisted preferences (documented elsewhere)
The following preferences also persist locally across sessions but are managed outside this modal and are documented in their own sections:
- **Preview fit mode** — how the preview is sized/fit; see *Live Preview*.
- **Snippet sort preference** — the snippet list's sort field and direction; see *Snippet Library*.
## Behaviors
- **Apply / save**: An explicit Apply action writes all changes; they take effect immediately (e.g. the UI theme switches at once).
- **Dirty indication**: While the form differs from the last saved state, the modal shows an "Unsaved changes" indicator.
- **Cancel / dismiss**: Closing without applying reverts the form to the last saved values and leaves stored settings untouched.
- **Reset to defaults**: A Reset action restores every setting to its factory default. It requires explicit confirmation before applying, then saves the defaults.
- **Startup load**: Settings are read on startup and applied to the UI and editor; missing or unrecognized values silently use their defaults.
docs/spec/08-import-export.md
+83
View File
@@ -0,0 +1,83 @@
# 08 · Import & Export
Astrolabe lets a user back up or transfer their entire workspace as a single JSON file, and bring data back in by importing such a file. Both actions are triggered from header controls labelled **Import** and **Export**. Import always merges with existing data; it never replaces what is already stored.
## Export
Export produces one downloadable JSON file containing every snippet (see *Snippet Library*) and every dataset (see *Datasets*), wrapped in an envelope carrying format metadata.
- **Trigger**: the **Export** header control runs the export immediately (no intermediate dialog).
- **Contents**: all snippets and all datasets currently stored, plus envelope metadata.
- **Empty workspace**: if there are no snippets, the user is informed ("No snippets to export") and no file is downloaded — even if datasets exist.
- **Filename**: `astrolabe-project-YYYY-MM-DD.json`, where the date is today's date (export day).
- **Feedback**: on success a toast reports the counts, e.g. "Exported 4 snippets and 2 datasets" (the dataset clause is omitted when there are no datasets; singular/plural wording adapts to the counts).
### Export envelope shape
The downloaded file is a single JSON object: an envelope with a format `version`, an export timestamp, an exporter tag, and the two data arrays.
```json
{
"version": "1.0",
"exportedAt": "2026-06-03T12:00:00.000Z",
"exportedBy": "Astrolabe",
"snippets": [ /* full snippet objects (see Data Model) */ ],
"datasets": [ /* full dataset objects (see Data Model) */ ]
}
```
- `version` — export format version (currently `"1.0"`).
- `exportedAt` — ISO 8601 timestamp of the export.
- `exportedBy` — fixed identifier `"Astrolabe"`.
- `snippets` / `datasets` — arrays of complete records as defined in *Data Model*, each including its record `version` field. (This is the per-record schema version, not the envelope `version` above.)
## Import
Import lets the user pick a JSON file from their device; its contents are normalized, merged into the current workspace, and saved.
- **Trigger**: the **Import** header control opens a file picker restricted to JSON files. After a file is chosen (or the picker cancelled) the control is ready to be used again immediately.
### Accepted inputs
The importer recognizes several shapes so that both Astrolabe exports and looser snippet files work:
- **Astrolabe export envelope** — an object with a `version` and a `snippets` array; an optional `datasets` array is imported too.
- **Bare array of snippets** — a top-level JSON array is treated as a list of snippets (no datasets).
- **Single snippet object** — any other object is treated as one snippet.
- **Older / foreign snippet shapes** — snippets that do not match the current model are normalized onto it:
- Alternative field names are mapped: `content` → spec, `draft` → draft spec, `createdAt` → creation timestamp.
- Missing timestamps are generated at import time (creation and modification set to now, or derived from the source timestamp when present).
- Missing identifiers, names, comments, tags, dataset references, metadata, and record `version` are filled with defaults (a missing `version` is treated as the earliest shape and migrated up on read — see *Data Model*).
- Such normalized imports are tagged `"imported"` so the user can find them.
A snippet is treated as already in current Astrolabe format when it carries an ISO-style creation timestamp; in that case its existing fields (id, name, timestamps, spec, draft spec, comment, tags, dataset references, metadata) are preserved as-is, with sensible fallbacks for any missing field.
### Merge behavior
- Imported snippets are **appended** to the existing library; nothing is overwritten or removed.
- **ID collisions** (an incoming snippet whose id already exists) are resolved by assigning the incoming snippet a fresh unique id; the original snippet keeps its id.
- Datasets are imported **before** snippets so that snippet dataset references can resolve.
### Dataset conflicts
When an imported dataset's name already exists in the library, it is auto-renamed to a unique name rather than overwriting the existing one (see *Datasets*).
- A numeric suffix is appended to the original name; further suffixes are added until the name is unique.
- The renamed datasets are reported to the user via a warning toast listing each `original -> new` rename.
- If a single dataset fails to import, it is skipped and the rest of the import continues.
### Storage limit handling
Snippet storage has an approximate 5 MB budget (see *Snippet Library* storage monitor).
- If the incoming snippets would push total snippet storage over the budget, the user is warned about the overage amount, but the app still attempts to save the import.
- If the save ultimately fails because the storage quota is exceeded, the user is told to delete some snippets and try again, and no partial snippet import is committed.
- The storage check applies to snippets; datasets are stored separately and saved during the dataset phase above.
### Feedback
- **Success**: a toast reports how many snippets (and datasets, when any) were imported, e.g. "Imported 4 snippets and 2 datasets".
- **Renames**: when datasets were renamed, the success message is shown as a warning toast that also lists the renames.
- **Empty file**: if no snippets are found in the file, the user is informed ("No snippets found in file") and nothing is imported.
- **Quota failure**: a clear error advising the user to delete snippets and retry.
- **Invalid file**: a non-JSON or unparseable file produces a clear error ("Failed to import. Please check that the file is valid JSON."); an unreadable file produces a read error. In all error cases the existing workspace is left unchanged.
docs/spec/09-data-model.md
+107
View File
@@ -0,0 +1,107 @@
# 09 · Data Model & Persistence
This section defines the persistent entities of Astrolabe and how they relate. It is the authoritative data contract: an implementer recreating the app should store equivalent records with these fields and meanings. Types are given abstractly (string, number, boolean, ISO-timestamp string, string[], object, "JSON value") so they map onto any stack. "JSON value" means any valid JSON shape — object, array, string, number, boolean, or null.
All data lives entirely in the browser. There is no server, account, or sync. Records survive page reload and remain available offline (see *Application Shell & Navigation*). To move data between browsers or devices, use *Import & Export*.
## A. Snippet
A **Snippet** is a saved Vega-Lite specification together with its metadata. Snippets are the primary user-authored entity, listed and managed in the *Snippet Library*.
| Field | Type | Meaning |
|-------|------|---------|
| `id` | string | Unique, stable identifier for the snippet. |
| `version` | number | Schema version of this record, used for read-time migration (see *Schema versioning* below). |
| `name` | string | Human-readable title shown in the library. |
| `created` | ISO-timestamp string | When the snippet was first created. |
| `modified` | ISO-timestamp string | When the snippet was last saved. |
| `spec` | JSON value | The **published** Vega-Lite spec. May be an object or a string. This is the version rendered and shared by default. |
| `draftSpec` | JSON value | The **working draft** Vega-Lite spec being edited. May be an object or a string. |
| `comment` | string | Free-form user note about the snippet. |
| `tags` | string[] | User-assigned labels for filtering and organization. |
| `datasetRefs` | string[] | Names of *Datasets* referenced by this spec (see relationships below). |
| `meta` | object | Free-form, extensible metadata bag for app- or feature-specific data. |
### Dual spec / draftSpec model
A snippet carries two specs at once. `draftSpec` is the editable working copy; `spec` is the last published copy. Editing affects only `draftSpec` until the user publishes, at which point `draftSpec` is promoted to `spec`. This separation backs the draft/published workflow described in *Spec Editor & Draft/Published Workflow* — it lets users experiment freely while keeping a known-good published version, and drives indicators for unpublished changes.
### datasetRefs
`datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see *Cross-entity relationships*). It is maintained to mirror the dataset names actually referenced in the spec.
## B. Dataset
A **Dataset** is a named, reusable data source that snippets can reference by name instead of inlining data. Datasets are managed in the *Datasets* manager and support multiple formats and two source kinds.
| Field | Type | Meaning |
|-------|------|---------|
| `id` | number | Unique numeric identifier. |
| `version` | number | Schema version of this record, used for read-time migration (see *Schema versioning* below). |
| `name` | string | Unique, human-readable name; the key snippets reference via `datasetRefs`. |
| `data` | JSON value | The payload. For `source = url`: the URL string. For `source = inline`: the raw CSV/TSV text, or the parsed JSON/TopoJSON value. |
| `format` | string | One of `json`, `csv`, `tsv`, `topojson`. |
| `source` | string | One of `inline` (data embedded in the record) or `url` (data fetched from a remote address). |
| `comment` | string | Free-form user note about the dataset. |
| `rowCount` | number or null | Number of data rows, or null when unknown/not applicable. |
| `columnCount` | number or null | Number of columns, or null when unknown/not applicable. |
| `columns` | string[] | Column names, in order. |
| `columnTypes` | array of `{ name, type }` | Per-column inferred type. `name` is the column; `type` is one of `number`, `string`, `date`, `boolean`. |
| `size` | number | Approximate payload size in bytes. |
| `created` | ISO-timestamp string | When the dataset was first added. |
| `modified` | ISO-timestamp string | When the dataset was last changed. |
The `rowCount`, `columnCount`, `columns`, `columnTypes`, and `size` fields are derived summaries computed when data is added or updated; they support previews and type display without re-parsing the full payload.
### Schema versioning
Both **Snippet** and **Dataset** records carry a numeric `version` recording the shape of that individual record. When a record is read from storage it is migrated up to the current shape before the app uses it; new writes always store the current version. A record written before versioning existed (no `version` field) is treated as version `1`. This is distinct from the storage container's own layout version, and from the *Import & Export* envelope `version` (which describes the file format, not a record). Records exported via *Import & Export* include their `version`.
## C. UserSettings
**UserSettings** holds persisted user preferences as a single structured record. The semantics and UX of each option are covered in *Settings*; the shape below is the storage contract.
| Field | Type | Meaning |
|-------|------|---------|
| `version` | number | Schema version of the settings record, used for migration. |
| `editor.fontSize` | number | Editor font size. |
| `editor.theme` | string | Editor color theme identifier. |
| `editor.minimap` | boolean | Whether the editor minimap is shown. |
| `editor.wordWrap` | string | `on` or `off`. |
| `editor.lineNumbers` | string | `on` or `off`. |
| `editor.tabSize` | number | Spaces per indentation level. |
| `performance.renderDebounce` | number | Delay (ms) before re-rendering the preview after edits. |
| `ui.theme` | string | App theme: `light` or `experimental`. |
| `ui.previewFitMode` | string | Preview sizing: `default`, `width`, `height`, or `full`. |
| `formatting.dateFormat` | string | Date display mode: `smart`, `iso`, or `custom`. |
| `formatting.customDateFormat` | string | Pattern used when `dateFormat = custom`. |
A reference shape:
UserSettings = { version, editor: { fontSize, theme, minimap, wordWrap, lineNumbers, tabSize }, performance: { renderDebounce }, ui: { theme, previewFitMode }, formatting: { dateFormat, customDateFormat } }
## D. App / UI preferences (persisted separately)
Some preferences persist independently of *UserSettings* so they can update frequently without rewriting the settings record. They are stored locally and restored on load.
- **Snippet sort preference** — how the *Snippet Library* list is ordered. `sortBy` is one of `name`, `modified`, `created`; `sortOrder` is `asc` or `desc`. Default is `modified` / `desc` (most recently changed first).
- **Panel layout** — the resizable three-panel arrangement: per-pane widths and per-pane visibility (which panels are shown or hidden). Restored so the workspace reopens as the user left it.
## E. Persistence & limits
| Tier | What it holds | Capacity & behavior |
|------|---------------|---------------------|
| Snippet store | All *Snippet* records | Local, with a practical budget of about 5 MB. A storage monitor tracks usage and surfaces warnings as the budget fills (see *Snippet Library*). |
| Dataset store | All *Dataset* records | Local, in a separate, much higher-capacity store, suited to larger payloads. |
| Settings & preferences | *UserSettings* plus the app/UI preferences in (D) | Local, small. |
Everything stays in the browser — no server or account is involved. All tiers survive reload and function offline. Because capacity is finite and per-browser, *Import & Export* is the supported path for backup and for moving data between browsers or devices.
## F. Cross-entity relationships
Snippets and datasets are linked **bidirectionally by dataset name**: `snippet.datasetRefs` holds dataset names, and each such name matches a `dataset.name`.
- From a snippet, `datasetRefs` yields its linked datasets.
- From a dataset, scanning snippets for its `name` in `datasetRefs` yields the snippets that reference it.
This name-based link is what the *Snippet Library* and *Datasets* surfaces use to show linkage in both directions. The actual resolution of a referenced dataset into spec data at render time is covered in *Live Preview*.
docs/spec/10-non-functional.md
+54
View File
@@ -0,0 +1,54 @@
# 10 · Non-Functional Requirements
This section defines quality attributes the rebuild must satisfy — performance, accessibility, reliability, privacy, and platform posture — independent of any single feature. Feature behavior lives in the other sections; this one constrains *how well* that behavior must work.
## Platform & Form Factor
- **Target**: modern evergreen desktop browsers. The app is a single-page application that loads once and then runs locally.
- **Desktop-first**: the primary experience is the three-pane workspace (see *Application Shell & Navigation*), designed for wide viewports. Each pane has a minimum usable width and stops shrinking below it.
- **Small screens**: the three-pane layout is not expected to reach full parity on narrow/mobile viewports. A graceful fallback (e.g. collapsing to fewer visible panes via the toggle strip, or a single-column arrangement) is acceptable; an unusable or broken layout is not.
- **Offline & installable**: after first load the app must function fully offline, and must be installable as a standalone application that launches in its own window (see *Application Shell & Navigation*).
## Embedding & Environment Assumptions
Astrolabe is specified as a standalone single-page app that owns its whole viewport. A team integrating these capabilities into a larger product should know which shared environment surfaces the app currently reserves, so they can decide how to reconcile each with the host. (Surfacing the assumption is the spec's job; choosing the reconciliation is the integrator's.)
- **Global keyboard shortcuts** — the shortcuts in *Application Shell & Navigation* are bound document-wide and override the browser default, regardless of which element has focus or which modal is open. In a host app they may collide with the host's own bindings.
- **URL hash as view state** — the app stores its current view (selected snippet, open dataset, chart-builder target) in the URL hash and reads it on load (see *Navigation & Shareable URL State*). A host that owns routing will need to share or namespace the hash.
- **Local browser storage** — all state persists to local browser storage across the tiers in *Data Model & Persistence*; storage keys are not namespaced against a co-resident host app.
- **Full-window workspace** — the layout assumes a wide, app-owned viewport (header, three panes, and modals). Hosting it within a smaller region falls under the small-screen fallback above.
## Performance & Responsiveness
- **Live editing stays fluid**: typing in the editor must remain smooth regardless of spec size; rendering must never block input.
- **Debounced rendering**: preview rendering is deferred until the user pauses typing, by a user-configurable delay (see *Settings* / *Live Preview*), so rapid keystrokes do not cause continuous re-rendering.
- **Non-blocking renders**: while a render is in progress the UI stays interactive; a busy indication may overlay the preview but must not freeze editing or navigation.
- **Auto-save is cheap and silent**: persisting the working draft must not interrupt typing or cause visible stalls (see *Spec Editor & Draft/Published Workflow*).
- **Scales with the library**: search, sort, and list rendering must stay responsive with a large number of snippets, and large datasets must be handled by the high-capacity dataset store rather than inflating snippet storage (see *Data Model & Persistence*).
## Accessibility
- **Keyboard operable**: all primary actions are reachable from the keyboard — the global shortcuts (see *Application Shell & Navigation*) plus standard tab/focus traversal of controls, lists, and forms.
- **Modal focus management**: opening a modal moves focus into it and returns focus sensibly on close; **Escape** closes the active modal; focus is contained within an open modal.
- **Labelled controls**: form fields, toggles, and icon-only buttons carry accessible names so assistive technology can announce them.
- **Reduced motion**: animations and transitions (toast fades, etc.) are suppressed when the user's system requests reduced motion.
- **Contrast**: text and interactive elements meet legible contrast in every offered UI theme; a theme that cannot meet contrast in part of the UI is not considered complete (see *Settings*).
## Reliability & Data Safety
- **No silent data loss**: edits are auto-saved as drafts; a known-good published version is always preserved separately (see *Spec Editor & Draft/Published Workflow*).
- **Confirm destructive actions**: deleting snippets or datasets, reverting a draft, and resetting settings require explicit confirmation.
- **Warn before storage failure**: snippet storage usage is surfaced with escalating warnings as it fills, and the user is told when a save fails rather than losing data silently (see *Snippet Library*).
- **Non-destructive import**: importing always merges with existing data and never overwrites or removes it; on failure the existing workspace is left unchanged (see *Import & Export*).
- **Resilient rendering**: an invalid or unrenderable spec produces a readable error and recovers automatically when fixed; it never leaves the app in a broken state (see *Live Preview*).
- **State survives reload**: the current selection/view is restored from the URL, and all data persists across reloads and sessions (see *Application Shell & Navigation*, *Data Model & Persistence*).
## Privacy & Security
- **Local-only data**: all snippets, datasets, and settings stay in the browser. No user content is transmitted to any server, and the app requires no account or login.
- **User-initiated network only**: the only outbound requests for user content are fetches of URL-sourced datasets or remote data referenced by a spec, which the user explicitly created (see *Datasets*). The app performs no background upload of user content.
- **Client-side rendering of untrusted input**: specs and data are user-authored and rendered locally; rendering must fail safely on malformed input rather than crashing the app.
## Internationalization
- **Locale-aware formatting where it exists**: date rendering follows the user's chosen format mode (see *Settings*). Full UI translation is out of scope unless explicitly added later; the spec does not require multiple UI languages.
docs/spec/README.md
+31
View File
@@ -0,0 +1,31 @@
# Astrolabe — Product Specification
A UX/behavioral specification of **Astrolabe**, a browser-based snippet manager for [Vega-Lite](https://vega.github.io/vega-lite/) visualizations. It describes *what the app does* from the user's perspective so it can be recreated on any web/HTML/TS stack.
## How to read this spec
- Start with [00 · Product Overview](00-product-overview.md) for orientation and the glossary.
- Each subsequent file is one feature area and can be read on its own; they cross-reference each other by title.
- Every section describes intended behavior plus testable acceptance points ("The user can…", "When X, the system…").
## What this spec deliberately omits
- **Implementation.** No frameworks, libraries, languages, storage technologies, or code architecture are prescribed. Storage is described behaviorally (e.g. "persists locally across sessions", capacity tiers), not by naming a technology.
- **Visual design.** Structural layout (panes, regions, modal vs inline, where controls live) is specified; concrete styling, colors, and the app's visual aesthetic are left to the implementer.
- **Domain exception.** Vega-Lite and its vocabulary (specs, marks, encoding channels, field types) and data-format names (JSON, CSV, TSV, TopoJSON) *are* named — they are the product domain, not implementation choices.
## Contents
| # | Section |
|---|---------|
| 00 | [Product Overview](00-product-overview.md) |
| 01 | [Application Shell & Navigation](01-application-shell.md) |
| 02 | [Snippet Library](02-snippet-library.md) |
| 03 | [Spec Editor & Draft/Published Workflow](03-editor-and-drafts.md) |
| 04 | [Live Preview](04-live-preview.md) |
| 05 | [Datasets](05-datasets.md) |
| 06 | [Chart Builder](06-chart-builder.md) |
| 07 | [Settings](07-settings.md) |
| 08 | [Import & Export](08-import-export.md) |
| 09 | [Data Model & Persistence](09-data-model.md) |
| 10 | [Non-Functional Requirements](10-non-functional.md) |
index.html
+13
View File
@@ -0,0 +1,13 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Astrolabe</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
package-lock.json
Generated
+7733
View File
File diff suppressed because it is too large Load Diff
package.json
+36
View File
@@ -0,0 +1,36 @@
{
"name": "astrolabe",
"version": "0.1.0",
"private": true,
"type": "module",
"description": "A browser-based snippet manager for Vega-Lite visualizations.",
"scripts": {
"dev": "vite",
"build": "tsc --noEmit && vite build",
"preview": "vite preview",
"typecheck": "tsc --noEmit",
"test": "vitest run",
"test:watch": "vitest",
"format": "prettier --write ."
},
"dependencies": {
"monaco-editor": "^0.54.0",
"react": "^19.2.7",
"react-dom": "^19.2.7",
"vega": "^6.2.0",
"vega-embed": "^7.1.0",
"vega-lite": "^6.4.2",
"zustand": "^5.0.14"
},
"devDependencies": {
"@types/react": "^19.2.16",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^5.2.0",
"happy-dom": "^20.0.0",
"prettier": "^3.6.2",
"typescript": "^5.9.2",
"vite": "^7.1.0",
"vite-plugin-pwa": "^1.0.3",
"vitest": "^3.2.4"
}
}
public/favicon.svg
+7
View File
@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
<circle cx="16" cy="16" r="14" fill="none" stroke="#1a1a2e" stroke-width="2" />
<circle cx="16" cy="16" r="6" fill="none" stroke="#1a1a2e" stroke-width="1.5" />
<line x1="16" y1="2" x2="16" y2="30" stroke="#1a1a2e" stroke-width="1.5" />
<line x1="2" y1="16" x2="30" y2="16" stroke="#1a1a2e" stroke-width="1.5" />
<circle cx="16" cy="16" r="1.5" fill="#1a1a2e" />
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 441 B

src/app/App.module.css
+58
View File
@@ -0,0 +1,58 @@
.app {
display: flex;
flex-direction: column;
height: 100%;
}
.header {
display: flex;
align-items: center;
gap: var(--space-3);
height: var(--header-height);
padding: 0 var(--space-4);
border-bottom: 1px solid var(--color-border);
background: var(--color-surface);
flex: 0 0 auto;
}
.title {
font-weight: 600;
font-size: 16px;
}
.version {
font-size: 11px;
color: var(--color-text-muted);
border: 1px solid var(--color-border);
border-radius: var(--radius);
padding: 1px var(--space-2);
}
.spacer {
flex: 1;
}
.panes {
display: flex;
flex: 1 1 auto;
min-height: 0;
}
.pane {
flex: 1;
min-width: 0;
padding: var(--space-4);
overflow: auto;
border-right: 1px solid var(--color-border);
}
.pane:last-child {
border-right: none;
}
.paneLabel {
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--color-text-muted);
}
src/app/App.tsx
+33
View File
@@ -0,0 +1,33 @@
import styles from './App.module.css';
/**
* Application shell — the three-pane workspace from spec §01A
* (library · editor · preview) under a fixed header.
*
* This is the skeleton: panes are placeholders. Each milestone fills one in
* (see docs/IMPLEMENTATION-PLAN.md). Resizing, toggling, modals, routing, and
* shortcuts arrive in later milestones.
*/
export function App() {
return (
<div className={styles.app}>
<header className={styles.header}>
<span className={styles.title}>Astrolabe</span>
<span className={styles.version}>v{__APP_VERSION__}</span>
<span className={styles.spacer} />
</header>
<main className={styles.panes}>
<section className={styles.pane} aria-label="Snippet library">
<div className={styles.paneLabel}>Library</div>
</section>
<section className={styles.pane} aria-label="Spec editor">
<div className={styles.paneLabel}>Editor</div>
</section>
<section className={styles.pane} aria-label="Live preview">
<div className={styles.paneLabel}>Preview</div>
</section>
</main>
</div>
);
}
src/app/stores/AppStore.ts
+41
View File
@@ -0,0 +1,41 @@
import { create } from 'zustand';
import type { UiTheme } from '@core/theme';
/**
* Centralized cross-cutting application state, as a Zustand store. Keep this
* lean — durable, feature-specific state (snippets, datasets, settings) lands
* in its own store module (e.g. stores/SnippetStore) as the app grows.
*
* Usable inside React via the `useAppStore` hook (with a selector) and outside
* React via `useAppStore.getState()` / `.setState()` / `.subscribe()` — see
* docs/architecture/01-state-and-stores.md.
*/
export type { UiTheme };
/** Which modal, if any, is currently open. At most one at a time (spec §01C). */
export type ModalName = 'datasets' | 'settings' | 'about' | 'donate' | 'chartBuilder' | 'extract';
export interface AppState {
/** Active UI theme; mirrored onto <html data-theme> by a subscriber. */
uiTheme: UiTheme;
/** The currently open modal, or null. */
activeModal: ModalName | null;
setTheme: (theme: UiTheme) => void;
/**
* Low-level modal setter — the single primitive that mutates `activeModal`.
* High-level open/close (snapshot for unsaved-change detection, URL sync,
* discard confirmation) lives in the modal coordinator (docs/architecture/03),
* which calls this; arrives with the modal system in M3.
*/
setActiveModal: (modal: ModalName | null) => void;
}
export const useAppStore = create<AppState>((set) => ({
uiTheme: 'light',
activeModal: null,
setTheme: (uiTheme) => set({ uiTheme }),
setActiveModal: (activeModal) => set({ activeModal }),
}));
src/core/format-detection.test.ts
+53
View File
@@ -0,0 +1,53 @@
import { describe, it, expect } from 'vitest';
import { detectFormat, detectFormatFromUrl } from './format-detection';
describe('detectFormat', () => {
it('detects a JSON array of objects with high confidence', () => {
expect(detectFormat('[{"a":1},{"a":2}]')).toEqual({ format: 'json', confidence: 'high' });
});
it('detects a single JSON object as json', () => {
expect(detectFormat('{"a":1}')).toEqual({ format: 'json', confidence: 'high' });
});
it('detects a TopoJSON topology object', () => {
const topo = JSON.stringify({ type: 'Topology', objects: {}, arcs: [] });
expect(detectFormat(topo)).toEqual({ format: 'topojson', confidence: 'high' });
});
it('detects CSV from a comma-separated header + row with medium confidence', () => {
expect(detectFormat('a,b,c\n1,2,3')).toEqual({ format: 'csv', confidence: 'medium' });
});
it('detects TSV from a tab-separated header + row with medium confidence', () => {
expect(detectFormat('a\tb\tc\n1\t2\t3')).toEqual({ format: 'tsv', confidence: 'medium' });
});
it('prefers TSV over CSV when both delimiters appear in the header', () => {
expect(detectFormat('a\tb,c\n1\t2,3').format).toBe('tsv');
});
it('returns low confidence / null for unrecognized input', () => {
expect(detectFormat('just a sentence')).toEqual({ format: null, confidence: 'low' });
});
it('returns low confidence / null for empty input', () => {
expect(detectFormat(' ')).toEqual({ format: null, confidence: 'low' });
});
});
describe('detectFormatFromUrl', () => {
it.each([
['https://example.com/data.csv', 'csv'],
['https://example.com/data.tsv', 'tsv'],
['https://example.com/data.json', 'json'],
['https://example.com/world.topojson', 'topojson'],
['https://example.com/data.csv?v=2', 'csv'],
])('infers format from %s', (url, expected) => {
expect(detectFormatFromUrl(url)).toBe(expected);
});
it('returns null when no known extension is present', () => {
expect(detectFormatFromUrl('https://example.com/data')).toBeNull();
});
});
src/core/format-detection.ts
+61
View File
@@ -0,0 +1,61 @@
/**
* Format auto-detection for pasted dataset data.
*
* Portable core: no browser APIs, no React — usable in Node and tests.
* Implements the detection rules from spec §05 (Datasets → Auto-detection):
*
* - Valid JSON parses to `json`, or `topojson` when it is a topology object — high confidence.
* - Otherwise multi-line text with a header row is `tsv` (tab-separated) or `csv` (comma) — medium.
* - Unrecognized input yields `null` format — low confidence (saving should be blocked upstream).
*/
export type DataFormat = 'json' | 'csv' | 'tsv' | 'topojson';
export type DetectionConfidence = 'high' | 'medium' | 'low';
export interface FormatDetection {
format: DataFormat | null;
confidence: DetectionConfidence;
}
function isTopology(value: unknown): boolean {
return (
typeof value === 'object' &&
value !== null &&
(value as { type?: unknown }).type === 'Topology'
);
}
/** Detect the format of pasted inline data. */
export function detectFormat(raw: string): FormatDetection {
const text = raw.trim();
if (text === '') return { format: null, confidence: 'low' };
// 1. Try JSON first — highest confidence signal.
try {
const parsed = JSON.parse(text);
return {
format: isTopology(parsed) ? 'topojson' : 'json',
confidence: 'high',
};
} catch {
// not JSON — fall through to delimited detection
}
// 2. Delimited text: needs at least a header row plus one data row.
const lines = text.split(/\r?\n/).filter((l) => l.length > 0);
if (lines.length >= 2) {
const header = lines[0];
if (header.includes('\t')) return { format: 'tsv', confidence: 'medium' };
if (header.includes(',')) return { format: 'csv', confidence: 'medium' };
}
// 3. Unrecognized.
return { format: null, confidence: 'low' };
}
/** Infer a dataset's format from a URL's file extension (spec §05). */
export function detectFormatFromUrl(url: string): DataFormat | null {
const match = url.toLowerCase().match(/\.(csv|tsv|json|topojson)(?:[?#]|$)/);
if (!match) return null;
return match[1] as DataFormat;
}
src/core/theme.ts
+10
View File
@@ -0,0 +1,10 @@
/**
* Theme identity — portable core. No browser APIs, no React.
*
* `UiTheme` is the app-wide theme token. It lives in `src/core/` (not in a
* store) because pure core logic needs it too: the Vega chart-config mapping
* (`chartConfigFor`, see docs/architecture/05) keys off the same union, and core
* must never import from `src/app/`. The store and the document `data-theme`
* mirror this value; this is its single definition.
*/
export type UiTheme = 'light' | 'experimental';
src/main.tsx
+16
View File
@@ -0,0 +1,16 @@
import { createRoot } from 'react-dom/client';
import { App } from './app/App';
import { useAppStore } from './app/stores/AppStore';
import '../styles/base.css';
// Mirror the UI theme onto <html data-theme>: apply the initial value before
// first paint, then keep it in sync. (Store stays DOM-free; the adapter is here.)
const applyTheme = (theme: string) => {
document.documentElement.dataset.theme = theme;
};
applyTheme(useAppStore.getState().uiTheme);
useAppStore.subscribe((state, prev) => {
if (state.uiTheme !== prev.uiTheme) applyTheme(state.uiTheme);
});
createRoot(document.getElementById('app')!).render(<App />);
src/vite-env.d.ts
Vendored
+9
View File
@@ -0,0 +1,9 @@
/// <reference types="vite/client" />
/// <reference types="vite-plugin-pwa/client" />
declare const __APP_VERSION__: string;
declare module '*.module.css' {
const classes: { readonly [key: string]: string };
export default classes;
}
styles/base.css
+28
View File
@@ -0,0 +1,28 @@
@import './tokens.css';
* {
box-sizing: border-box;
}
html,
body,
#app {
height: 100%;
margin: 0;
}
body {
font-family: var(--font-sans);
font-size: var(--font-size-base);
color: var(--color-text);
background: var(--color-bg);
}
@media (prefers-reduced-motion: reduce) {
*,
*::before,
*::after {
animation-duration: 0.01ms !important;
transition-duration: 0.01ms !important;
}
}
styles/tokens.css
+47
View File
@@ -0,0 +1,47 @@
/*
* Design tokens. Borrowed in spirit from Syto: a single source of truth for
* colors/spacing/typography, themable by overriding the custom properties on
* a [data-theme] root. Concrete values are placeholders pending the design pass.
*/
:root {
/* Palette */
--color-bg: #ffffff;
--color-surface: #f7f7fa;
--color-border: #e2e2ea;
--color-text: #1a1a2e;
--color-text-muted: #6b6b80;
--color-accent: #3b5bdb;
--color-accent-contrast: #ffffff;
/* Status */
--color-success: #2f9e44;
--color-error: #e03131;
--color-warning: #f08c00;
--color-info: #1971c2;
/* Typography */
--font-sans: system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif;
--font-mono: ui-monospace, 'SF Mono', 'Cascadia Code', Menlo, monospace;
--font-size-base: 14px;
/* Spacing scale */
--space-1: 4px;
--space-2: 8px;
--space-3: 12px;
--space-4: 16px;
--space-6: 24px;
/* Layout */
--header-height: 48px;
--radius: 6px;
}
[data-theme='experimental'] {
--color-bg: #16161f;
--color-surface: #1f1f2c;
--color-border: #2c2c3a;
--color-text: #e8e8f0;
--color-text-muted: #9a9ab0;
--color-accent: #748ffc;
--color-accent-contrast: #0b0b12;
}
tsconfig.json
+32
View File
@@ -0,0 +1,32 @@
{
"compilerOptions": {
"target": "ES2022",
"useDefineForClassFields": true,
"module": "ESNext",
"lib": ["ES2022", "DOM", "DOM.Iterable"],
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": false,
"resolveJsonModule": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"noUncheckedSideEffectImports": true,
"types": ["vite/client", "vite-plugin-pwa/client"],
"paths": {
"@/*": ["./src/*"],
"@core/*": ["./src/core/*"]
},
"baseUrl": "."
},
"include": ["src", "vite.config.ts"]
}
vite.config.ts
+41
View File
@@ -0,0 +1,41 @@
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { VitePWA } from 'vite-plugin-pwa';
import { readFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
const pkg = JSON.parse(readFileSync(new URL('./package.json', import.meta.url), 'utf-8'));
export default defineConfig({
resolve: {
alias: {
'@': fileURLToPath(new URL('./src', import.meta.url)),
'@core': fileURLToPath(new URL('./src/core', import.meta.url)),
},
},
define: {
__APP_VERSION__: JSON.stringify(pkg.version),
},
plugins: [
react(),
VitePWA({
registerType: 'prompt',
includeAssets: ['favicon.svg'],
manifest: {
name: 'Astrolabe',
short_name: 'Astrolabe',
description: 'A browser-based snippet manager for Vega-Lite visualizations.',
theme_color: '#1a1a2e',
background_color: '#ffffff',
display: 'standalone',
icons: [],
},
}),
],
// Vitest config (shares this file)
test: {
globals: true,
environment: 'happy-dom',
include: ['src/**/*.test.{ts,tsx}'],
},
} as Parameters<typeof defineConfig>[0]);