astrolabe/docs/spec/09-data-model.md

# 09 · Data Model & Persistence

This section defines the persistent entities of Astrolabe and how they relate. It is the authoritative data contract: an implementer recreating the app should store equivalent records with these fields and meanings. Types are given abstractly (string, number, boolean, ISO-timestamp string, string[], object, "JSON value") so they map onto any stack. "JSON value" means any valid JSON shape — object, array, string, number, boolean, or null.

All data lives entirely in the browser. There is no server, account, or sync. Records survive page reload and remain available offline (see _Application Shell & Navigation_). To move data between browsers or devices, use _Import & Export_.

## A. Snippet

A **Snippet** is a saved Vega-Lite specification together with its metadata. Snippets are the primary user-authored entity, listed and managed in the _Snippet Library_.

| Field         | Type                 | Meaning                                                                                                             |
| ------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `id`          | string               | Unique, stable identifier for the snippet.                                                                          |
| `version`     | number               | Schema version of this record, used for read-time migration (see _Schema versioning_ below).                        |
| `name`        | string               | Human-readable title shown in the library.                                                                          |
| `created`     | ISO-timestamp string | When the snippet was first created.                                                                                 |
| `modified`    | ISO-timestamp string | When the snippet was last saved.                                                                                    |
| `spec`        | JSON value           | The **published** Vega-Lite spec. May be an object or a string. This is the version rendered and shared by default. |
| `draftSpec`   | JSON value           | The **working draft** Vega-Lite spec being edited. May be an object or a string.                                    |
| `comment`     | string               | Free-form user note about the snippet.                                                                              |
| `tags`        | string[]             | User-assigned labels for filtering and organization.                                                                |
| `datasetRefs` | string[]             | Names of _Datasets_ referenced by this spec (see relationships below).                                              |
| `meta`        | object               | Free-form, extensible metadata bag for app- or feature-specific data.                                               |

### Dual spec / draftSpec model

A snippet carries two specs at once. `draftSpec` is the editable working copy; `spec` is the last published copy. Editing affects only `draftSpec` until the user publishes, at which point `draftSpec` is promoted to `spec`. This separation backs the draft/published workflow described in _Spec Editor & Draft/Published Workflow_ — it lets users experiment freely while keeping a known-good published version, and drives indicators for unpublished changes.

### datasetRefs

`datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see _Cross-entity relationships_). It is maintained to mirror the dataset names actually referenced in the spec.

## B. Dataset

A **Dataset** is a named, reusable data source that snippets can reference by name instead of inlining data. Datasets are managed in the _Datasets_ manager and support multiple formats and two source kinds.

| Field         | Type                      | Meaning                                                                                                                          |
| ------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `id`          | number                    | Unique numeric identifier.                                                                                                       |
| `version`     | number                    | Schema version of this record, used for read-time migration (see _Schema versioning_ below).                                     |
| `name`        | string                    | Unique, human-readable name; the key snippets reference via `datasetRefs`.                                                       |
| `data`        | JSON value                | The payload. For `source = url`: the URL string. For `source = inline`: the raw CSV/TSV text, or the parsed JSON/TopoJSON value. |
| `format`      | string                    | One of `json`, `csv`, `tsv`, `topojson`.                                                                                         |
| `source`      | string                    | One of `inline` (data embedded in the record) or `url` (data fetched from a remote address).                                     |
| `comment`     | string                    | Free-form user note about the dataset.                                                                                           |
| `rowCount`    | number or null            | Number of data rows, or null when unknown/not applicable.                                                                        |
| `columnCount` | number or null            | Number of columns, or null when unknown/not applicable.                                                                          |
| `columns`     | string[]                  | Column names, in order.                                                                                                          |
| `columnTypes` | array of `{ name, type }` | Per-column inferred type. `name` is the column; `type` is one of `number`, `string`, `date`, `boolean`.                          |
| `size`        | number                    | Approximate payload size in bytes.                                                                                               |
| `created`     | ISO-timestamp string      | When the dataset was first added.                                                                                                |
| `modified`    | ISO-timestamp string      | When the dataset was last changed.                                                                                               |

The `rowCount`, `columnCount`, `columns`, `columnTypes`, and `size` fields are derived summaries computed when data is added or updated; they support previews and type display without re-parsing the full payload.

### Schema versioning

Both **Snippet** and **Dataset** records carry a numeric `version` recording the shape of that individual record. When a record is read from storage it is migrated up to the current shape before the app uses it; new writes always store the current version. A record written before versioning existed (no `version` field) is treated as version `1`. This is distinct from the storage container's own layout version, and from the _Import & Export_ envelope `version` (which describes the file format, not a record). Records exported via _Import & Export_ include their `version`.

## C. UserSettings

**UserSettings** holds persisted user preferences as a single structured record. The semantics and UX of each option are covered in _Settings_; the shape below is the storage contract.

| Field                         | Type    | Meaning                                                    |
| ----------------------------- | ------- | ---------------------------------------------------------- |
| `version`                     | number  | Schema version of the settings record, used for migration. |
| `editor.fontSize`             | number  | Editor font size.                                          |
| `editor.theme`                | string  | Editor color theme identifier.                             |
| `editor.minimap`              | boolean | Whether the editor minimap is shown.                       |
| `editor.wordWrap`             | string  | `on` or `off`.                                             |
| `editor.lineNumbers`          | string  | `on` or `off`.                                             |
| `editor.tabSize`              | number  | Spaces per indentation level.                              |
| `performance.renderDebounce`  | number  | Delay (ms) before re-rendering the preview after edits.    |
| `ui.theme`                    | string  | App theme: `light` or `dark`.                              |
| `ui.previewFitMode`           | string  | Preview sizing: `default`, `width`, `height`, or `full`.   |
| `formatting.dateFormat`       | string  | Date display mode: `smart`, `iso`, or `custom`.            |
| `formatting.customDateFormat` | string  | Pattern used when `dateFormat = custom`.                   |

A reference shape:

UserSettings = { version, editor: { fontSize, theme, minimap, wordWrap, lineNumbers, tabSize }, performance: { renderDebounce }, ui: { theme, previewFitMode }, formatting: { dateFormat, customDateFormat } }

## D. App / UI preferences (persisted separately)

Some preferences persist independently of _UserSettings_ so they can update frequently without rewriting the settings record. They are stored locally and restored on load.

- **Snippet sort preference** — how the _Snippet Library_ list is ordered. `sortBy` is one of `name`, `modified`, `created`; `sortOrder` is `asc` or `desc`. Default is `modified` / `desc` (most recently changed first).
- **Panel layout** — the resizable three-panel arrangement: per-pane widths and per-pane visibility (which panels are shown or hidden). Restored so the workspace reopens as the user left it.

## E. Persistence & limits

| Tier                   | What it holds                                     | Capacity & behavior                                                                                                                             |
| ---------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| Snippet store          | All _Snippet_ records                             | Local, with a practical budget of about 5 MB. A storage monitor tracks usage and surfaces warnings as the budget fills (see _Snippet Library_). |
| Dataset store          | All _Dataset_ records                             | Local, in a separate, much higher-capacity store, suited to larger payloads.                                                                    |
| Settings & preferences | _UserSettings_ plus the app/UI preferences in (D) | Local, small.                                                                                                                                   |

Everything stays in the browser — no server or account is involved. All tiers survive reload and function offline. Because capacity is finite and per-browser, _Import & Export_ is the supported path for backup and for moving data between browsers or devices.

## F. Cross-entity relationships

Snippets and datasets are linked **bidirectionally by dataset name**: `snippet.datasetRefs` holds dataset names, and each such name matches a `dataset.name`.

- From a snippet, `datasetRefs` yields its linked datasets.
- From a dataset, scanning snippets for its `name` in `datasetRefs` yields the snippets that reference it.

This name-based link is what the _Snippet Library_ and _Datasets_ surfaces use to show linkage in both directions. The actual resolution of a referenced dataset into spec data at render time is covered in _Live Preview_.