astrolabe/docs/spec/09-data-model.md

09 · Data Model & Persistence

This section defines the persistent entities of Astrolabe and how they relate. It is the authoritative data contract: an implementer recreating the app should store equivalent records with these fields and meanings. Types are given abstractly (string, number, boolean, ISO-timestamp string, string[], object, "JSON value") so they map onto any stack. "JSON value" means any valid JSON shape — object, array, string, number, boolean, or null.

All data lives entirely in the browser. There is no server, account, or sync. Records survive page reload and remain available offline (see Application Shell & Navigation). To move data between browsers or devices, use Import & Export.

A. Snippet

A Snippet is a saved Vega-Lite specification together with its metadata. Snippets are the primary user-authored entity, listed and managed in the Snippet Library.

Field	Type	Meaning
id	string	Unique, stable identifier for the snippet.
version	number	Schema version of this record, used for read-time migration (see Schema versioning below).
name	string	Human-readable title shown in the library.
created	ISO-timestamp string	When the snippet was first created.
modified	ISO-timestamp string	When the snippet was last saved.
spec	JSON value	The published Vega-Lite spec. May be an object or a string. This is the version rendered and shared by default.
draftSpec	JSON value	The working draft Vega-Lite spec being edited. May be an object or a string.
comment	string	Free-form user note about the snippet.
tags	string[]	User-assigned labels for filtering and organization.
datasetRefs	string[]	Names of Datasets referenced by this spec (see relationships below).
meta	object	Free-form, extensible metadata bag for app- or feature-specific data.

Dual spec / draftSpec model

A snippet carries two specs at once. draftSpec is the editable working copy; spec is the last published copy. Editing affects only draftSpec until the user publishes, at which point draftSpec is promoted to spec. This separation backs the draft/published workflow described in Spec Editor & Draft/Published Workflow — it lets users experiment freely while keeping a known-good published version, and drives indicators for unpublished changes.

datasetRefs

datasetRefs records the names of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see Cross-entity relationships). It mirrors the dataset names referenced by the draft spec — the version the user is editing — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft means a snippet's linked datasets reflect what the editor currently shows, not only the last published version; recomputation only ever runs on a valid (parseable) spec, so a half-typed draft never disturbs the links.

B. Dataset

A Dataset is a named, reusable data source that snippets can reference by name instead of inlining data. Datasets are managed in the Datasets manager and support multiple formats and two source kinds.

Field	Type	Meaning
id	number	Unique numeric identifier.
version	number	Schema version of this record, used for read-time migration (see Schema versioning below).
name	string	Unique, human-readable name; the key snippets reference via datasetRefs.
data	JSON value	The payload, shaped by format: raw CSV/TSV text, or the parsed JSON/TopoJSON value. For source = url this is the fetched snapshot, or null before the first successful fetch.
format	string	One of json, csv, tsv, topojson.
source	string	One of inline (data pasted into the record) or url (data fetched once from a remote address and snapshotted into the record).
url	string (url only)	The remote address a url dataset was fetched from, retained so it can be re-fetched ("Refresh"). Absent for inline datasets.
fetchedAt	ISO-timestamp or null	For url datasets: when the snapshot was last fetched, or null if never fetched. Absent for inline datasets.
comment	string	Free-form user note about the dataset.
rowCount	number or null	Number of data rows, or null when unknown/not applicable.
columnCount	number or null	Number of columns, or null when unknown/not applicable.
columns	string[]	Column names, in order.
columnTypes	array of { name, type }	Per-column inferred type. name is the column; type is one of number, string, date, boolean.
size	number	Approximate payload size in bytes.
created	ISO-timestamp string	When the dataset was first added.
modified	ISO-timestamp string	When the dataset was last changed.

The rowCount, columnCount, columns, columnTypes, and size fields are derived summaries computed when data is added or updated — including when a url dataset is fetched or refreshed; a fetched URL snapshot profiles exactly like inline data. They support previews and type display without re-parsing the full payload.

Schema versioning

Both Snippet and Dataset records carry a numeric version recording the shape of that individual record. When a record is read from storage it is migrated up to the current shape before the app uses it; new writes always store the current version. A record written before versioning existed (no version field) is treated as version 1. This is distinct from the storage container's own layout version, and from the Import & Export envelope version (which describes the file format, not a record). Records exported via Import & Export include their version.

The current Dataset version is 2. The v1→v2 migration reflects the URL-snapshot model: a v1 url dataset stored its address in data, so migration moves that address into the new url field and clears data to null — the record becomes an unfetched reference that renders against its live URL until the user refreshes it, at which point the fetched snapshot is stored and profiled.

C. UserSettings

UserSettings holds persisted user preferences as a single structured record. The semantics and UX of each option are covered in Settings; the shape below is the storage contract.

Field	Type	Meaning
version	number	Schema version of the settings record, used for migration.
editor.fontSize	number	Editor font size.
editor.theme	string	Editor color theme identifier.
editor.minimap	boolean	Whether the editor minimap is shown.
editor.wordWrap	string	on or off.
editor.lineNumbers	string	on or off.
editor.tabSize	number	Spaces per indentation level.
performance.renderDebounce	number	Delay (ms) before re-rendering the preview after edits.
ui.theme	string	App theme: light or dark.
ui.previewFitMode	string	Preview sizing: default, width, height, or full.
ui.chartTheme	string	Chart theme: astrolabe, stock, a preset id, or custom:<id> naming a CustomTheme (G).
formatting.dateFormat	string	Date display mode: smart, iso, or custom.
formatting.customDateFormat	string	Pattern used when dateFormat = custom.

A reference shape:

UserSettings = { version, editor: { fontSize, theme, minimap, wordWrap, lineNumbers, tabSize }, performance: { renderDebounce }, ui: { theme, previewFitMode, chartTheme }, formatting: { dateFormat, customDateFormat } }

D. App / UI preferences (persisted separately)

Some preferences persist independently of UserSettings so they can update frequently without rewriting the settings record. They are stored locally and restored on load.

• Snippet sort preference — how the Snippet Library list is ordered. sortBy is one of name, modified, created; sortOrder is asc or desc. Default is modified / desc (most recently changed first).
• Panel layout — the resizable three-panel arrangement: per-pane widths and per-pane visibility (which panels are shown or hidden). Restored so the workspace reopens as the user left it.

E. Persistence & limits

Tier	What it holds	Capacity & behavior
Snippet store	All Snippet records	Local, with a practical budget of about 5 MB. A storage monitor tracks usage and surfaces warnings as the budget fills (see Snippet Library).
Dataset store	All Dataset records	Local, in a separate, much higher-capacity store, suited to larger payloads.
Theme store	All CustomTheme records (G)	Local, separate store; records are small (a config object plus metadata).
Settings & preferences	UserSettings plus the app/UI preferences in (D)	Local, small.

Everything stays in the browser — no server or account is involved. All tiers survive reload and function offline. Because capacity is finite and per-browser, Import & Export is the supported path for backup and for moving data between browsers or devices.

F. Cross-entity relationships

Snippets and datasets are linked bidirectionally by dataset name: snippet.datasetRefs holds dataset names, and each such name matches a dataset.name.

• From a snippet, datasetRefs yields its linked datasets.
• From a dataset, scanning snippets for its name in datasetRefs yields the snippets that reference it.

This name-based link is what the Snippet Library and Datasets surfaces use to show linkage in both directions. The actual resolution of a referenced dataset into spec data at render time is covered in Live Preview.

G. CustomTheme

A CustomTheme is a user-named Vega-Lite config saved in the library and offered by the Live Preview → Chart theme picker alongside the built-in themes and presets. It is created and edited in the Theme Builder (see Live Preview).

Field	Type	Meaning
id	number	Unique numeric identifier. The picker/persistence selection id is the string custom:<id>.
version	number	Schema version of this record, used for read-time migration (see Schema versioning above).
name	string	Unique, human-readable name shown in the picker (case-insensitive uniqueness, like datasets).
config	object	The Vega-Lite config injected at render time when this theme is selected.
created	ISO-timestamp string	When the theme was first created.
modified	ISO-timestamp string	When the theme was last changed.

Selection is keyed by id (not name) so renaming a theme never invalidates the persisted ui.chartTheme. A persisted custom:<id> whose record no longer exists is not an error: charts render with the house style until the record appears (themes hydrate asynchronously), and deleting the actively-selected theme resets the selection to astrolabe explicitly. Custom themes are not yet included in the Import & Export envelope (planned; see docs/chart-theming-scope.md).