# Astrolabe → Syto Integration Analysis

> **Question:** Can Astrolabe (a browser-based Vega-Lite snippet manager) be integrated into
> Syto's functionality? This document compares the Astrolabe specification (`docs/spec/`)
> against Syto in its current state, and recommends an integration path.
>
> **Short answer:** Not as a wholesale port, and not as the "snippet manager" it is today —
> that framing collides with Syto's stated non-goals. But the _valuable parts_ of Astrolabe
> (the Chart Builder, the generic spec→render pipeline, the schema-assisted JSON editor) map
> cleanly onto a Syto-native **"chart a model"** feature, and most of the supporting tech already
> exists in the codebase. The recommendation is **harvest, don't port** — and the framing decision
> needs a `SOUL.md` ruling first.

---

## 1. Executive Summary

|                      |                                                                                                                                                                                                                                                                                                                                                     |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Conceptual fit**   | Partial. Astrolabe and Syto are both local-first, browser-only, Vega-Lite-using, three-pane-ish workspaces. But Astrolabe's _primary entity_ (a saved chart spec) is a thing Syto deliberately does not have.                                                                                                                                       |
| **Strategic fit**    | **Conflicted.** `SOUL.md` explicitly lists "Not a BI/visualization platform — charts are for exploration during wrangling, not final output" as a non-goal, and "Do One Thing Well." A _snippet library_ is chart-authoring-as-product. This is the central tension and must be resolved before any code.                                           |
| **Technical fit**    | **Good for the rendering/editing layer, poor for the data-model and shell layers.** Syto already ships Vega-Lite, vega-embed, CodeMirror 6, IndexedDB persistence, a settings system, URL-hash routing, and a far stronger type/schema engine than Astrolabe's profiler. The friction is in the _parallel systems_ a verbatim port would introduce. |
| **Recommended path** | **Option B (harvest into a native "Visualize" feature).** Reuse Astrolabe's Chart Builder and rendering contract; bind them to Syto **Models** instead of a new "dataset" entity; drop the snippet-as-primary-entity, the draft/published workflow, the separate dataset library, and the separate import/export envelope.                          |

---

## 2. The Two Products Side by Side

| Dimension     | **Astrolabe**                                                                 | **Syto**                                                             |
| ------------- | ----------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| Core artifact | A **snippet** = a saved Vega-Lite spec + metadata                             | A **workflow** = a declarative transform pipeline over a Source      |
| Primary verb  | _Author & organize charts_                                                    | _Clean & reshape tabular data_                                       |
| Data unit     | **Dataset** (named blob: JSON/CSV/TSV/TopoJSON, inline or URL)                | **Source** (immutable imported table) → **Model** (derived table)    |
| Persistence   | Snippets (~5 MB tier) + Datasets (high-capacity tier), both local             | Sources + Models in IndexedDB (lazy row data), prefs in localStorage |
| Editor        | JSON editor w/ Vega-Lite schema autocomplete + live validation                | CodeMirror 6 — used for transform JSON + the expression language     |
| Rendering     | Renders _arbitrary user specs_ via reference-resolution + fit-mode transforms | Renders _programmatically generated_ EDA specs (`charts.ts`)         |
| Shell         | 3 panes: library · editor · preview, + modals                                 | Ribbon + sidebar + data table + slide-panel/modal dialogs            |
| Routing       | URL hash: `#snippet-<id>`, `#datasets/...`                                    | URL hash: active source/model/dialog                                 |
| Export        | One JSON envelope of all snippets + datasets                                  | Workflow v2 JSON (transforms, topo-sorted)                           |
| Stack stance  | Implementation-agnostic spec                                                  | Preact + Signals + Arquero + CSS Modules, fixed                      |

**The key observation:** Astrolabe's "dataset" is conceptually Syto's "Source," and the thing you
most want to chart in Syto — a cleaned, transformed **Model** — has _no equivalent in Astrolabe at
all_. Astrolabe charts static blobs; Syto produces living, recomputed tables. A naive port would
bolt a second, weaker data-library (Astrolabe datasets) next to Syto's existing one (Sources/Models),
which directly violates SOUL's **"No Parallel Systems"** value.

---

## 3. The Strategic Tension (resolve this first)

This is not a technical blocker; it is a product-identity decision, and per project convention
(`SOUL.md` is the arbiter for contract/design decisions) it must be settled before implementation.

**What `SOUL.md` currently says:**

- _"Do One Thing Well… It's not trying to become a spreadsheet, a statistical package, a visualization tool, or a database. The EDA features… exist to help users understand their data before transforming it — not to replace dedicated analysis tools."_
- _"Not a BI/visualization platform: Charts are for exploration during wrangling, not final output. Dashboards and reporting are a separate concern."_

A **snippet manager** — a personal, searchable, import/exportable _library of saved charts_ — is
squarely "charts as final output" and "a visualization tool." Porting Astrolabe as-is would
contradict two written non-goals.

**However**, there is a reading that is fully _aligned_ with the rest of SOUL:

- _"Beginner-Friendly, Not Beginner-Limited"_ and _"Progressive Complexity"_ — today a user can clean data but has **no way to turn the result into a shareable picture.** A chart is the natural last step of a wrangling session.
- _"Leverage Existing Libraries — Vega-Lite handles charts."_ The infrastructure is already paid for.
- Astrolabe's **Chart Builder** (pick a mark, map columns → spec) is the _exact_ beginner-friendly, no-JSON affordance Syto favors, and the live JSON editor is the power-user escape hatch.

**The decision to make:** Is "produce a chart as the output of a workflow" _part of_ doing the one
thing well (wrangling ends in a usable artifact), or is it the BI/viz scope SOUL rejects?

Two coherent resolutions:

- **(A) Amend SOUL** to permit _single-chart output of a model_ (not dashboards, not a chart library-as-product) — and integrate as a native feature (§6, Option B).
- **(B) Keep it separate** — Astrolabe stays its own thing, or lives as a sibling `/tools/` mini-app that merely _shares code_ with Syto (§6, Option C). The main app's non-goals stay intact.

I recommend (A) with a tightly-scoped amendment, because the value lands precisely where Syto is
currently weakest (no output artifact), and because doing it natively avoids the parallel-systems
trap. But this is the user's call to make against SOUL.

---

## 4. Feature-by-Feature Reuse Map

Legend: 🟢 already exists / strong reuse · 🟡 partial, needs adaptation · 🔴 net-new build

| Astrolabe feature                                                                         | Syto today                                                                             | Verdict      | Notes                                                                                                                                                                                                                                                              |
| ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Vega-Lite rendering**                                                                   | `charts.ts` + `vega-embed@7` render programmatic specs into DOM refs                   | 🟡           | Engine present; needs a _generic_ "render this arbitrary spec" path + error surface. The hardcoded EDA specs don't help directly, but the rendering primitive does.                                                                                                |
| **Dataset-reference resolution** (`{data:{name}}` → contents, recursing into layers)      | none                                                                                   | 🔴           | New, but small and pure — and in Syto it resolves to a **Model's data**, not a separate dataset store.                                                                                                                                                             |
| **Fit-mode transforms** (Original/Width/Height/Full via `"container"`)                    | none                                                                                   | 🔴           | Small, pure, copy-on-render spec rewrite. Directly portable.                                                                                                                                                                                                       |
| **JSON spec editor**                                                                      | CodeMirror 6 (`CodeMirrorEditor.tsx`, `JsonEditorModal.tsx`) + lint infra (`linters/`) | 🟡           | Editor & lint plumbing exist. Missing: a **Vega-Lite schema service** for autocomplete + validation. (Note: Astrolabe's "minimap" and "VS Light/Dark/High-Contrast" editor themes are Monaco-isms; Syto is on CodeMirror — those exact settings don't carry over.) |
| **Chart Builder** (mark + X/Y/Color/Size + field types → spec)                            | none                                                                                   | 🟡→🔴        | The single most valuable, most SOUL-aligned piece. Build it against a **Model's columns** using Syto's existing schema types. High reuse of the _dialog_ pattern (registry + slide-panel/modal + debounced preview).                                               |
| **Column profiling / type inference**                                                     | `schema-engine.ts` (integer/float/date/datetime/boolean/json)                          | 🟢           | Syto's engine **supersedes** Astrolabe's (number/string/date/boolean). Astrolabe→Vega field-type mapping (numeric→Quantitative, date→Temporal, else Nominal) layers on top trivially.                                                                              |
| **Datasets library + manager modal**                                                      | Sources/Models already _are_ the data library                                          | 🔴 _(avoid)_ | Do **not** build. Reuse Sources/Models. Building it = parallel systems.                                                                                                                                                                                            |
| **Snippet library** (search/sort/CRUD, draft vs published, status, tags, storage monitor) | none                                                                                   | 🔴           | The genuinely new persistent entity. Only needed if going full snippet-manager (not recommended). Draft/Published has no analog in Syto's undo/redo model.                                                                                                         |
| **Settings** (editor/performance/formatting)                                              | `ux-settings.ts` + settings dialog                                                     | 🟡           | System exists; add render-debounce + a couple of fields. Editor-theme/minimap fields are Monaco-shaped and mostly drop.                                                                                                                                            |
| **Import/Export envelope** (snippets+datasets JSON)                                       | Workflow v2 export/import                                                              | 🔴 _(avoid)_ | A second export format competing with workflow v2. If charts are part of a workflow, they belong _in_ the workflow spec or alongside it — not in a rival envelope.                                                                                                 |
| **App shell / 3-pane layout**                                                             | Ribbon + sidebar + table + slide-panel                                                 | 🔴 _(avoid)_ | Don't graft Astrolabe's shell. A chart view is a _mode/panel within_ Syto's shell.                                                                                                                                                                                 |
| **URL-hash routing**                                                                      | Hash routing for source/model/dialog                                                   | 🟡           | Reusable, but Astrolabe's `#snippet-…`/`#datasets/…` scheme would **collide**; must namespace under Syto's existing scheme.                                                                                                                                        |
| **Keyboard shortcuts**                                                                    | `EventRouter` owns Ctrl+S (save), Escape priority chain, etc.                          | 🟡           | **Collisions:** Astrolabe binds Ctrl+S (Publish) and Ctrl+K (Datasets). Syto already owns Ctrl+S. Must reconcile, not adopt verbatim.                                                                                                                              |
| **Offline / PWA / installable**                                                           | `vite-plugin-pwa` already configured                                                   | 🟢           | Free.                                                                                                                                                                                                                                                              |
| **i18n**                                                                                  | i18next, en/uk, namespaced                                                             | 🟢           | New strings go in a namespace; framework is there.                                                                                                                                                                                                                 |
| **Toasts**                                                                                | Notification system exists                                                             | 🟢           | Reuse.                                                                                                                                                                                                                                                             |

**Reuse tally:** the _rendering, editing, persistence, settings, schema, i18n, PWA, and toast_
substrate is largely present. The _data-model, shell, routing-scheme, and lifecycle_ layers of
Astrolabe are either redundant with Syto or actively conflicting and should be dropped.

---

## 5. Technical Friction Points (if ported verbatim)

1. **Parallel data library.** Astrolabe datasets vs Syto Sources/Models — two stores, two
   profilers, two "named data" concepts. Violates _No Parallel Systems_. (The fix: charts reference
   Models.)
2. **Parallel persistence + export.** A second IndexedDB store layout and a second JSON envelope
   alongside workflow v2. Two backup formats for users to confuse.
3. **Draft/Published has no home.** Syto's non-destructive model is _pipeline steps + undo/redo_,
   not a per-document draft/published toggle. Astrolabe's central editing model would be a third,
   unrelated state concept.
4. **Shell mismatch.** Astrolabe's library·editor·preview triptych is a _whole app_. Syto's shell is
   ribbon-driven with slide-panel dialogs. They don't compose; one must yield.
5. **Routing & shortcut collisions.** Hash schemes overlap; Ctrl+S/Ctrl+K already bound.
6. **Editor-feature gap.** Syto is on CodeMirror (no minimap, different theme model); Astrolabe's
   settings assume Monaco. And neither today has a **Vega-Lite schema service** — that autocomplete/
   validation is net-new work on either stack.
7. **TopoJSON / arbitrary-JSON data.** Syto Sources are _tabular_. Astrolabe datasets include
   TopoJSON and arbitrary JSON. Charting a Model covers the tabular case; map/topology charts would
   be out of scope unless Sources grow a non-tabular kind.

None of these are unsolvable — but every one of them is _work created by the port itself_, not by
the user value. That's the signature of "harvest, don't port."

---

## 6. Integration Options

### Option A — Full port (snippet manager inside Syto)

Bring Astrolabe over more-or-less intact: snippet library, dataset manager, draft/published, its
shell, its export.

- **Pros:** Fastest way to "have Astrolabe." Feature-complete chart authoring.
- **Cons:** Maximal parallel-systems debt (§5). Directly contradicts SOUL non-goals. Two data
  libraries, two export formats, shell/routing/shortcut conflicts. **Not recommended.**

### Option B — Harvest into a native "Visualize" feature ✅ _recommended_

Add charting as the natural _output_ step of a workflow, reusing Syto's own primitives:

- A **"Chart" / "Visualize"** action on a **Model** opens a **Chart Builder** (Astrolabe's mark +
  X/Y/Color/Size + field-type controls), populated from the Model's columns and `schema-engine`
  types.
- It produces a Vega-Lite spec rendered live via the existing `vega-embed`, using a ported
  **reference-resolution + fit-mode** rendering contract where the named data resolves to the
  **Model's rows**.
- Power users get the **JSON spec editor** (CodeMirror, with a Vega-Lite schema service added) as the
  escape hatch — consistent with _Beginner-Friendly, Not Beginner-Limited_.
- The chart (its spec) is persisted **attached to the Model** (or to the workflow), not as a separate
  snippet entity. Export rides along with workflow v2 (or a sibling field), not a rival envelope.
- **Dropped from Astrolabe:** separate dataset library, draft/published, snippet search/sort/tags,
  storage monitor, its shell, its import/export, its routing scheme.
- **Pros:** No parallel systems. Lands value exactly where Syto is weak (no output artifact). Maximal
  reuse of existing infra. Defensible against SOUL with a _narrow_ amendment ("single-chart output of
  a model," not dashboards/library).
- **Cons:** Requires the SOUL decision (§3). Loses Astrolabe features that depend on the
  snippet/dataset model (TopoJSON/URL datasets, multi-snippet library). Net-new: schema service,
  builder dialog, render contract.

### Option C — Sibling `/tools/` mini-app

Port Astrolabe as a self-contained app under `/tools/astrolabe/`, sharing only _code_ (vega render
helpers, CodeMirror wrapper, i18n) with the main app — no AppStore/DialogStore coupling.

- **Pros:** Keeps the main app's non-goals pristine (it's a separate utility, like other tools).
  Lower conceptual conflict. Astrolabe keeps its own model.
- **Cons:** Syto's `/tools/` layer is designed for _small, single-purpose_ utilities; Astrolabe is a
  full application — a stretch for that slot. Still carries Astrolabe's whole parallel data model,
  just quarantined. "Integration" here means "co-located," not "unified" — limited synergy.

---

## 7. Recommendation

1. **Make the SOUL call first (§3).** Decide whether single-chart _output of a model_ is in scope.
   If **no**, stop here or pursue Option C as a quarantined sibling. If **yes**, amend SOUL with a
   tight scope statement and proceed to Option B.
2. **Pursue Option B.** Harvest the three high-value, well-aligned pieces:
   - the **Chart Builder** (bound to a Model, driven by `schema-engine` types),
   - the **rendering contract** (reference-resolution + fit modes, resolving to Model data),
   - the **schema-assisted JSON editor** (CodeMirror + a new Vega-Lite schema service).
3. **Drop the parallel-systems pieces:** separate dataset library, draft/published, snippet
   library + storage monitor, separate import/export envelope, Astrolabe's shell and routing scheme.
4. **Reconcile, don't adopt,** the cross-cutting surfaces: fold settings into `ux-settings`,
   namespace any new hash state under Syto's scheme, resolve the Ctrl+S/Ctrl+K shortcut collisions.

This delivers the genuinely useful core of Astrolabe — turning cleaned data into a chart, with a
beginner path and a power-user path — while staying true to _Do One Thing Well_ and _No Parallel
Systems_, and reusing the infrastructure Syto has already built.

---

## 8. Open Questions for the User

- **SOUL scope:** Is "a chart as the output of a workflow" inside Syto's mission, or out? (Blocks everything.)
- **Persistence model:** Should a chart spec live _on a Model_, _in the workflow v2 export_, or as a new top-level entity?
- **Non-tabular data:** Do we ever need TopoJSON / arbitrary-JSON charts (maps), which Syto Sources can't currently hold? If not, that simplifies scope considerably.
- **Editor depth:** Is full Vega-Lite schema autocomplete/validation in scope, or is a plain JSON editor + live error surface enough for v1?
