# 09 · Data Model & Persistence This section defines the persistent entities of Astrolabe and how they relate. It is the authoritative data contract: an implementer recreating the app should store equivalent records with these fields and meanings. Types are given abstractly (string, number, boolean, ISO-timestamp string, string[], object, "JSON value") so they map onto any stack. "JSON value" means any valid JSON shape — object, array, string, number, boolean, or null. All data lives entirely in the browser. There is no server, account, or sync. Records survive page reload and remain available offline (see _Application Shell & Navigation_). To move data between browsers or devices, use _Import & Export_. ## A. Snippet A **Snippet** is a saved Vega-Lite specification together with its metadata. Snippets are the primary user-authored entity, listed and managed in the _Snippet Library_. | Field | Type | Meaning | | ------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | string | Unique, stable identifier for the snippet. | | `version` | number | Schema version of this record, used for read-time migration (see _Schema versioning_ below). | | `name` | string | Human-readable title shown in the library. | | `nameSource` | `'auto' \| 'user'`? | Name provenance: `auto` names keep tracking the spec on publish; `user` names are frozen (see _Snippet Library → Naming & Tags_). Optional — absent on records predating the field, which are treated as `user` unless the name is provably app-picked: the recognizable timestamp default, or identical to what the app derives from the record's own published spec. | | `created` | ISO-timestamp string | When the snippet was first created. | | `modified` | ISO-timestamp string | When the snippet was last saved. | | `spec` | JSON value | The **published** Vega-Lite spec. May be an object or a string. This is the version rendered and shared by default. | | `draftSpec` | JSON value | The **working draft** Vega-Lite spec being edited. May be an object or a string. | | `comment` | string | Free-form user note about the snippet. | | `tags` | string[] | User-assigned labels for filtering and organization. | | `datasetRefs` | string[] | Names of _Datasets_ referenced by this spec (see relationships below). | | `meta` | object | Free-form, extensible metadata bag for app- or feature-specific data. | ### Dual spec / draftSpec model A snippet carries two specs at once. `draftSpec` is the editable working copy; `spec` is the last published copy. Editing affects only `draftSpec` until the user publishes, at which point `draftSpec` is promoted to `spec`. This separation backs the draft/published workflow described in _Spec Editor & Draft/Published Workflow_ — it lets users experiment freely while keeping a known-good published version, and drives indicators for unpublished changes. ### datasetRefs `datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see _Cross-entity relationships_). It mirrors the dataset names referenced by the **draft** spec — the version the user is editing — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft means a snippet's linked datasets reflect what the editor currently shows, not only the last published version; recomputation only ever runs on a valid (parseable) spec, so a half-typed draft never disturbs the links. ## B. Dataset A **Dataset** is a named, reusable data source that snippets can reference by name instead of inlining data. Datasets are managed in the _Datasets_ manager and support multiple formats and two source kinds. | Field | Type | Meaning | | ------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | number | Unique numeric identifier. | | `version` | number | Schema version of this record, used for read-time migration (see _Schema versioning_ below). | | `name` | string | Unique, human-readable name; the key snippets reference via `datasetRefs`. | | `data` | JSON value | The payload, shaped by format: raw CSV/TSV text, or the parsed JSON/TopoJSON value. For `source = url` this is the **fetched snapshot**, or `null` before the first successful fetch. | | `format` | string | One of `json`, `csv`, `tsv`, `topojson`. | | `source` | string | One of `inline` (data pasted into the record) or `url` (data fetched once from a remote address and snapshotted into the record). | | `url` | string (url only) | The remote address a `url` dataset was fetched from, retained so it can be re-fetched ("Refresh"). Absent for inline datasets. | | `fetchedAt` | ISO-timestamp or null | For `url` datasets: when the snapshot was last fetched, or `null` if never fetched. Absent for inline datasets. | | `comment` | string | Free-form user note about the dataset. | | `rowCount` | number or null | Number of data rows, or null when unknown/not applicable. | | `columnCount` | number or null | Number of columns, or null when unknown/not applicable. | | `columns` | string[] | Column names, in order. | | `columnTypes` | array of `{ name, type }` | Per-column inferred type. `name` is the column; `type` is one of `number`, `string`, `date`, `boolean`. | | `size` | number | Approximate payload size in bytes. | | `created` | ISO-timestamp string | When the dataset was first added. | | `modified` | ISO-timestamp string | When the dataset was last changed. | The `rowCount`, `columnCount`, `columns`, `columnTypes`, and `size` fields are derived summaries computed when data is added or updated — including when a `url` dataset is fetched or refreshed; a fetched URL snapshot profiles exactly like inline data. They support previews and type display without re-parsing the full payload. ### Schema versioning Both **Snippet** and **Dataset** records carry a numeric `version` recording the shape of that individual record. When a record is read from storage it is migrated up to the current shape before the app uses it; new writes always store the current version. A record written before versioning existed (no `version` field) is treated as version `1`. This is distinct from the storage container's own layout version, and from the _Import & Export_ envelope `version` (which describes the file format, not a record). Records exported via _Import & Export_ include their `version`. The current **Dataset** version is `2`. The v1→v2 migration reflects the URL-snapshot model: a v1 `url` dataset stored its address in `data`, so migration moves that address into the new `url` field and clears `data` to `null` — the record becomes an _unfetched reference_ that renders against its live URL until the user refreshes it, at which point the fetched snapshot is stored and profiled. ## C. UserSettings **UserSettings** holds persisted user preferences as a single structured record. The semantics and UX of each option are covered in _Settings_; the shape below is the storage contract. | Field | Type | Meaning | | ----------------------------- | ------- | -------------------------------------------------------------------------------------------- | | `version` | number | Schema version of the settings record, used for migration. | | `editor.fontSize` | number | Editor font size. | | `editor.theme` | string | Editor color theme identifier. | | `editor.minimap` | boolean | Whether the editor minimap is shown. | | `editor.wordWrap` | string | `on` or `off`. | | `editor.lineNumbers` | string | `on` or `off`. | | `editor.tabSize` | number | Spaces per indentation level. | | `performance.renderDebounce` | number | Delay (ms) before re-rendering the preview after edits. | | `ui.theme` | string | App theme: `light` or `dark`. | | `ui.previewFitMode` | string | Preview sizing: `default`, `width`, `height`, or `full`. | | `ui.chartTheme` | string | Chart theme: `astrolabe`, `stock`, a preset id, or `custom:` naming a _CustomTheme_ (G). | | `formatting.dateFormat` | string | Date display mode: `smart`, `iso`, or `custom`. | | `formatting.customDateFormat` | string | Pattern used when `dateFormat = custom`. | A reference shape: UserSettings = { version, editor: { fontSize, theme, minimap, wordWrap, lineNumbers, tabSize }, performance: { renderDebounce }, ui: { theme, previewFitMode, chartTheme }, formatting: { dateFormat, customDateFormat } } ## D. App / UI preferences (persisted separately) Some preferences persist independently of _UserSettings_ so they can update frequently without rewriting the settings record. They are stored locally and restored on load. - **Snippet sort preference** — how the _Snippet Library_ list is ordered. `sortBy` is one of `name`, `modified`, `created`; `sortOrder` is `asc` or `desc`. Default is `modified` / `desc` (most recently changed first). - **Panel layout** — the resizable three-panel arrangement: per-pane widths and per-pane visibility (which panels are shown or hidden). Restored so the workspace reopens as the user left it. ## E. Persistence & limits | Tier | What it holds | Capacity & behavior | | ---------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | Snippet store | All _Snippet_ records | Local, with a practical budget of about 5 MB. A storage monitor tracks usage and surfaces warnings as the budget fills (see _Snippet Library_). | | Dataset store | All _Dataset_ records | Local, in a separate, much higher-capacity store, suited to larger payloads. | | Theme store | All _CustomTheme_ records (G) | Local, separate store; records are small (a config object plus metadata). | | Font store | All _FontAsset_ records (H) | Local, separate store; holds raw font-file bytes, so it is sized like the dataset tier (per-face cap ~10 MB). | | Settings & preferences | _UserSettings_ plus the app/UI preferences in (D) | Local, small. | Everything stays in the browser — no server or account is involved. All tiers survive reload and function offline. Because capacity is finite and per-browser, _Import & Export_ is the supported path for backup and for moving data between browsers or devices. ## F. Cross-entity relationships Snippets and datasets are linked **bidirectionally by dataset name**: `snippet.datasetRefs` holds dataset names, and each such name matches a `dataset.name`. - From a snippet, `datasetRefs` yields its linked datasets. - From a dataset, scanning snippets for its `name` in `datasetRefs` yields the snippets that reference it. This name-based link is what the _Snippet Library_ and _Datasets_ surfaces use to show linkage in both directions. The actual resolution of a referenced dataset into spec data at render time is covered in _Live Preview_. ## G. CustomTheme A **CustomTheme** is a user-named Vega-Lite config saved in the library and offered by the _Live Preview → Chart theme_ picker alongside the built-in themes and presets. It is created and edited in the _Theme Builder_ (see _Live Preview_). | Field | Type | Meaning | | ---------- | -------------------- | --------------------------------------------------------------------------------------------- | | `id` | number | Unique numeric identifier. The picker/persistence selection id is the string `custom:`. | | `version` | number | Schema version of this record, used for read-time migration (see _Schema versioning_ above). | | `name` | string | Unique, human-readable name shown in the picker (case-insensitive uniqueness, like datasets). | | `config` | object | The Vega-Lite config injected at render time when this theme is selected. | | `created` | ISO-timestamp string | When the theme was first created. | | `modified` | ISO-timestamp string | When the theme was last changed. | Selection is keyed by `id` (not name) so renaming a theme never invalidates the persisted `ui.chartTheme`. A persisted `custom:` whose record no longer exists is not an error: charts render with the house style until the record appears (themes hydrate asynchronously), and deleting the actively-selected theme resets the selection to `astrolabe` explicitly. Custom themes travel in the _Import & Export_ envelope alongside snippets and datasets (spec §08). ## H. FontAsset A **FontAsset** is a user-uploaded font face stored once and reused across themes and snippets. It is added and managed in the _Theme Builder → Type_ panel (see _Live Preview_), and referenced from a config's font slots by its `family` — exactly as a dataset is referenced by name. The raw bytes are registered as a live `FontFace` so a chart measures and renders the real face. | Field | Type | Meaning | | ---------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `id` | number | Unique numeric identifier (store key). | | `version` | number | Schema version of this record, for read-time migration (see _Schema versioning_ above). | | `family` | string | Unique CSS family name — the key configs reference (case-insensitive uniqueness, like datasets/themes). | | `data` | bytes | The raw font-file bytes (registered as a `FontFace`). In an export envelope these are **base64-encoded** (spec §08). | | `format` | `woff2`/`woff`/`ttf`/`otf` | Container format, from the file extension. | | `fileName` | string | Original file name, kept for display and provenance. | | `source` | `file`/`google` | Provenance. Only `file` (an upload) ships today; `google` is reserved for a later keyless-catalog tier. | | `axes` | array, optional | Variation axes for a variable font (parsed from `fvar`); drives the `FontFace` weight/width ranges. Absent for a static face. | | `size` | number | Byte length of `data`. | | `created` | ISO-timestamp string | When the font was first added. | | `modified` | ISO-timestamp string | When the font was last changed (e.g. renamed). | A font is identified by its `family`. No theme or snippet stores a font field: the faces a config uses are derived by scanning its font slots (the config is the source of truth), so a snippet can use a font with no theme to carry it. Fonts travel in the _Import & Export_ envelope alongside snippets, datasets, and themes; on import a family clash **skips** the incoming face rather than renaming it (spec §08 → Name conflicts).