# 09 · Data Model & Persistence

This section defines the persistent entities of Astrolabe and how they relate. It is the authoritative data contract: an implementer recreating the app should store equivalent records with these fields and meanings. Types are given abstractly (string, number, boolean, ISO-timestamp string, string[], object, "JSON value") so they map onto any stack. "JSON value" means any valid JSON shape — object, array, string, number, boolean, or null.

All data lives entirely in the browser. There is no server, account, or sync. Records survive page reload and remain available offline (see _Application Shell & Navigation_). To move data between browsers or devices, use _Import & Export_.

## A. Snippet

A **Snippet** is a saved Vega-Lite specification together with its metadata. Snippets are the primary user-authored entity, listed and managed in the _Snippet Library_.

| Field         | Type                 | Meaning                                                                                                                                                                                                                                                                                                                                                                |
| ------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id`          | string               | Unique, stable identifier for the snippet.                                                                                                                                                                                                                                                                                                                             |
| `version`     | number               | Schema version of this record, used for read-time migration (see _Schema versioning_ below).                                                                                                                                                                                                                                                                           |
| `name`        | string               | Human-readable title shown in the library.                                                                                                                                                                                                                                                                                                                             |
| `nameSource`  | `'auto' \| 'user'`?  | Name provenance: `auto` names keep tracking the spec on publish; `user` names are frozen (see _Snippet Library → Naming & Tags_). Optional — absent on records predating the field, which are treated as `user` unless the name is provably app-picked: the recognizable timestamp default, or identical to what the app derives from the record's own published spec. |
| `created`     | ISO-timestamp string | When the snippet was first created.                                                                                                                                                                                                                                                                                                                                    |
| `modified`    | ISO-timestamp string | When the snippet was last saved.                                                                                                                                                                                                                                                                                                                                       |
| `spec`        | JSON value           | The **published** Vega-Lite spec. May be an object or a string. This is the version rendered and shared by default.                                                                                                                                                                                                                                                    |
| `draftSpec`   | JSON value           | The **working draft** Vega-Lite spec being edited. May be an object or a string.                                                                                                                                                                                                                                                                                       |
| `comment`     | string               | Free-form user note about the snippet.                                                                                                                                                                                                                                                                                                                                 |
| `tags`        | string[]             | User-assigned labels for filtering and organization.                                                                                                                                                                                                                                                                                                                   |
| `datasetRefs` | string[]             | Names of _Datasets_ referenced by this spec (see relationships below).                                                                                                                                                                                                                                                                                                 |
| `meta`        | object               | Free-form, extensible metadata bag for app- or feature-specific data.                                                                                                                                                                                                                                                                                                  |

### Dual spec / draftSpec model

A snippet carries two specs at once. `draftSpec` is the editable working copy; `spec` is the last published copy. Editing affects only `draftSpec` until the user publishes, at which point `draftSpec` is promoted to `spec`. This separation backs the draft/published workflow described in _Spec Editor & Draft/Published Workflow_ — it lets users experiment freely while keeping a known-good published version, and drives indicators for unpublished changes.

### datasetRefs

`datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see _Cross-entity relationships_). It mirrors the dataset names referenced by the **draft** spec — the version the user is editing — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft means a snippet's linked datasets reflect what the editor currently shows, not only the last published version; recomputation only ever runs on a valid (parseable) spec, so a half-typed draft never disturbs the links.

## B. Dataset

A **Dataset** is a named, reusable data source that snippets can reference by name instead of inlining data. Datasets are managed in the _Datasets_ manager and support multiple formats and two source kinds.

| Field         | Type                      | Meaning                                                                                                                                                                               |
| ------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id`          | number                    | Unique numeric identifier.                                                                                                                                                            |
| `version`     | number                    | Schema version of this record, used for read-time migration (see _Schema versioning_ below).                                                                                          |
| `name`        | string                    | Unique, human-readable name; the key snippets reference via `datasetRefs`.                                                                                                            |
| `data`        | JSON value                | The payload, shaped by format: raw CSV/TSV text, or the parsed JSON/TopoJSON value. For `source = url` this is the **fetched snapshot**, or `null` before the first successful fetch. |
| `format`      | string                    | One of `json`, `csv`, `tsv`, `topojson`.                                                                                                                                              |
| `source`      | string                    | One of `inline` (data pasted into the record) or `url` (data fetched once from a remote address and snapshotted into the record).                                                     |
| `url`         | string (url only)         | The remote address a `url` dataset was fetched from, retained so it can be re-fetched ("Refresh"). Absent for inline datasets.                                                        |
| `fetchedAt`   | ISO-timestamp or null     | For `url` datasets: when the snapshot was last fetched, or `null` if never fetched. Absent for inline datasets.                                                                       |
| `comment`     | string                    | Free-form user note about the dataset.                                                                                                                                                |
| `rowCount`    | number or null            | Number of data rows, or null when unknown/not applicable.                                                                                                                             |
| `columnCount` | number or null            | Number of columns, or null when unknown/not applicable.                                                                                                                               |
| `columns`     | string[]                  | Column names, in order.                                                                                                                                                               |
| `columnTypes` | array of `{ name, type }` | Per-column inferred type. `name` is the column; `type` is one of `number`, `string`, `date`, `boolean`.                                                                               |
| `size`        | number                    | Approximate payload size in bytes.                                                                                                                                                    |
| `created`     | ISO-timestamp string      | When the dataset was first added.                                                                                                                                                     |
| `modified`    | ISO-timestamp string      | When the dataset was last changed.                                                                                                                                                    |

The `rowCount`, `columnCount`, `columns`, `columnTypes`, and `size` fields are derived summaries computed when data is added or updated — including when a `url` dataset is fetched or refreshed; a fetched URL snapshot profiles exactly like inline data. They support previews and type display without re-parsing the full payload.

### Schema versioning

Both **Snippet** and **Dataset** records carry a numeric `version` recording the shape of that individual record. When a record is read from storage it is migrated up to the current shape before the app uses it; new writes always store the current version. A record written before versioning existed (no `version` field) is treated as version `1`. This is distinct from the storage container's own layout version, and from the _Import & Export_ envelope `version` (which describes the file format, not a record). Records exported via _Import & Export_ include their `version`.

The current **Dataset** version is `2`. The v1→v2 migration reflects the URL-snapshot model: a v1 `url` dataset stored its address in `data`, so migration moves that address into the new `url` field and clears `data` to `null` — the record becomes an _unfetched reference_ that renders against its live URL until the user refreshes it, at which point the fetched snapshot is stored and profiled.

## C. UserSettings

**UserSettings** holds persisted user preferences as a single structured record. The semantics and UX of each option are covered in _Settings_; the shape below is the storage contract.

| Field                         | Type    | Meaning                                                                                      |
| ----------------------------- | ------- | -------------------------------------------------------------------------------------------- |
| `version`                     | number  | Schema version of the settings record, used for migration.                                   |
| `editor.fontSize`             | number  | Editor font size.                                                                            |
| `editor.theme`                | string  | Editor color theme identifier.                                                               |
| `editor.minimap`              | boolean | Whether the editor minimap is shown.                                                         |
| `editor.wordWrap`             | string  | `on` or `off`.                                                                               |
| `editor.lineNumbers`          | string  | `on` or `off`.                                                                               |
| `editor.tabSize`              | number  | Spaces per indentation level.                                                                |
| `performance.renderDebounce`  | number  | Delay (ms) before re-rendering the preview after edits.                                      |
| `ui.theme`                    | string  | App theme: `light` or `dark`.                                                                |
| `ui.previewFitMode`           | string  | Preview sizing: `default`, `width`, `height`, or `full`.                                     |
| `ui.chartTheme`               | string  | Chart theme: `astrolabe`, `stock`, a preset id, or `custom:<id>` naming a _CustomTheme_ (G). |
| `formatting.dateFormat`       | string  | Date display mode: `smart`, `iso`, or `custom`.                                              |
| `formatting.customDateFormat` | string  | Pattern used when `dateFormat = custom`.                                                     |

A reference shape:

UserSettings = { version, editor: { fontSize, theme, minimap, wordWrap, lineNumbers, tabSize }, performance: { renderDebounce }, ui: { theme, previewFitMode, chartTheme }, formatting: { dateFormat, customDateFormat } }

## D. App / UI preferences (persisted separately)

Some preferences persist independently of _UserSettings_ so they can update frequently without rewriting the settings record. They are stored locally and restored on load.

- **Snippet sort preference** — how the _Snippet Library_ list is ordered. `sortBy` is one of `name`, `modified`, `created`; `sortOrder` is `asc` or `desc`. Default is `modified` / `desc` (most recently changed first).
- **Panel layout** — the resizable three-panel arrangement: per-pane widths and per-pane visibility (which panels are shown or hidden). Restored so the workspace reopens as the user left it.

## E. Persistence & limits

| Tier                   | What it holds                                     | Capacity & behavior                                                                                                                             |
| ---------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| Snippet store          | All _Snippet_ records                             | Local, with a practical budget of about 5 MB. A storage monitor tracks usage and surfaces warnings as the budget fills (see _Snippet Library_). |
| Dataset store          | All _Dataset_ records                             | Local, in a separate, much higher-capacity store, suited to larger payloads.                                                                    |
| Theme store            | All _CustomTheme_ records (G)                     | Local, separate store; records are small (a config object plus metadata).                                                                       |
| Font store             | All _FontAsset_ records (H)                       | Local, separate store; holds raw font-file bytes, so it is sized like the dataset tier (per-face cap ~10 MB).                                   |
| Settings & preferences | _UserSettings_ plus the app/UI preferences in (D) | Local, small.                                                                                                                                   |

Everything stays in the browser — no server or account is involved. All tiers survive reload and function offline. Because capacity is finite and per-browser, _Import & Export_ is the supported path for backup and for moving data between browsers or devices.

## F. Cross-entity relationships

Snippets and datasets are linked **bidirectionally by dataset name**: `snippet.datasetRefs` holds dataset names, and each such name matches a `dataset.name`.

- From a snippet, `datasetRefs` yields its linked datasets.
- From a dataset, scanning snippets for its `name` in `datasetRefs` yields the snippets that reference it.

This name-based link is what the _Snippet Library_ and _Datasets_ surfaces use to show linkage in both directions. The actual resolution of a referenced dataset into spec data at render time is covered in _Live Preview_.

## G. CustomTheme

A **CustomTheme** is a user-named Vega-Lite config saved in the library and offered by the _Live Preview → Chart theme_ picker alongside the built-in themes and presets. It is created and edited in the _Theme Builder_ (see _Live Preview_).

| Field      | Type                 | Meaning                                                                                       |
| ---------- | -------------------- | --------------------------------------------------------------------------------------------- |
| `id`       | number               | Unique numeric identifier. The picker/persistence selection id is the string `custom:<id>`.   |
| `version`  | number               | Schema version of this record, used for read-time migration (see _Schema versioning_ above).  |
| `name`     | string               | Unique, human-readable name shown in the picker (case-insensitive uniqueness, like datasets). |
| `config`   | object               | The Vega-Lite config injected at render time when this theme is selected.                     |
| `created`  | ISO-timestamp string | When the theme was first created.                                                             |
| `modified` | ISO-timestamp string | When the theme was last changed.                                                              |

Selection is keyed by `id` (not name) so renaming a theme never invalidates the persisted `ui.chartTheme`. A persisted `custom:<id>` whose record no longer exists is not an error: charts render with the house style until the record appears (themes hydrate asynchronously), and deleting the actively-selected theme resets the selection to `astrolabe` explicitly. Custom themes travel in the _Import & Export_ envelope alongside snippets and datasets (spec §08).

## H. FontAsset

A **FontAsset** is a user-uploaded font face stored once and reused across themes and snippets. It is added and managed in the _Theme Builder → Type_ panel (see _Live Preview_), and referenced from a config's font slots by its `family` — exactly as a dataset is referenced by name. The raw bytes are registered as a live `FontFace` so a chart measures and renders the real face.

| Field      | Type                       | Meaning                                                                                                                       |
| ---------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `id`       | number                     | Unique numeric identifier (store key).                                                                                        |
| `version`  | number                     | Schema version of this record, for read-time migration (see _Schema versioning_ above).                                       |
| `family`   | string                     | Unique CSS family name — the key configs reference (case-insensitive uniqueness, like datasets/themes).                       |
| `data`     | bytes                      | The raw font-file bytes (registered as a `FontFace`). In an export envelope these are **base64-encoded** (spec §08).          |
| `format`   | `woff2`/`woff`/`ttf`/`otf` | Container format, from the file extension.                                                                                    |
| `fileName` | string                     | Original file name, kept for display and provenance.                                                                          |
| `source`   | `file`/`google`            | Provenance. Only `file` (an upload) ships today; `google` is reserved for a later keyless-catalog tier.                       |
| `axes`     | array, optional            | Variation axes for a variable font (parsed from `fvar`); drives the `FontFace` weight/width ranges. Absent for a static face. |
| `size`     | number                     | Byte length of `data`.                                                                                                        |
| `created`  | ISO-timestamp string       | When the font was first added.                                                                                                |
| `modified` | ISO-timestamp string       | When the font was last changed (e.g. renamed).                                                                                |

A font is identified by its `family`. No theme or snippet stores a font field: the faces a config uses are derived by scanning its font slots (the config is the source of truth), so a snippet can use a font with no theme to carry it. Fonts travel in the _Import & Export_ envelope alongside snippets, datasets, and themes; on import a family clash **skips** the incoming face rather than renaming it (spec §08 → Name conflicts).
