oleh/astrolabe
1
0
Fork 0
mirror of https://github.com/olehomelchenko/astrolabe.git synced 2026-08-08 02:02:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
74b810f8c9b2ce3266dda8e9bd11f4cb74fff2d2
astrolabe/docs/spec/09-data-model.md
T

151 lines
23 KiB
Markdown
Raw Blame History

# 09 · Data Model & Persistence
This section defines the persistent entities of Astrolabe and how they relate. It is the data record of the shipped app: an implementer recreating the app should store equivalent records with these fields and meanings. Types are given abstractly (string, number, boolean, ISO-timestamp string, string[], object, "JSON value") so they map onto any stack. "JSON value" means any valid JSON shape — object, array, string, number, boolean, or null.
All data lives entirely in the browser. There is no server, account, or sync. Records survive page reload and remain available offline (see _Application Shell & Navigation_). To move data between browsers or devices, use _Import & Export_.
## A. Snippet
A **Snippet** is a saved Vega-Lite specification together with its metadata. Snippets are the primary user-authored entity, listed and managed in the _Snippet Library_.
| Field | Type | Meaning |
| ------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | string | Unique, stable identifier for the snippet. |
| `version` | number | Schema version of this record, used for read-time migration (see _Schema versioning_ below). |
| `name` | string | Human-readable title shown in the library. |
| `nameSource` | `'auto' \| 'user'`? | Name provenance: `auto` names keep tracking the spec on publish; `user` names are frozen (see _Snippet Library → Naming & Tags_). Optional — absent on records predating the field, which are treated as `user` unless the name is provably app-picked: the recognizable timestamp default, or identical to what the app derives from the record's own published spec. |
| `created` | ISO-timestamp string | When the snippet was first created. |
| `modified` | ISO-timestamp string | When the snippet was last saved. |
| `spec` | JSON value | The **published** Vega-Lite spec. May be an object or a string. This is the version rendered and shared by default. |
| `draftSpec` | JSON value | The **working draft** Vega-Lite spec being edited. May be an object or a string. |
| `comment` | string | Free-form user note about the snippet. |
| `tags` | string[] | User-assigned labels for filtering and organization. |
| `datasetRefs` | string[] | Names of _Datasets_ referenced by this spec (see relationships below). |
| `meta` | object | Free-form, extensible metadata bag for app- or feature-specific data. |
### Dual spec / draftSpec model
A snippet carries two specs at once. `draftSpec` is the editable working copy; `spec` is the last published copy. Editing affects only `draftSpec` until the user publishes, at which point `draftSpec` is promoted to `spec`. This separation backs the draft/published workflow described in _Spec Editor & Draft/Published Workflow_ — it lets users experiment freely while keeping a known-good published version, and drives indicators for unpublished changes.
### datasetRefs
`datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see _Cross-entity relationships_). It mirrors the dataset names referenced by the **draft** spec — the version the user is editing — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft means a snippet's linked datasets reflect what the editor currently shows, not only the last published version; recomputation only ever runs on a valid (parseable) spec, so a half-typed draft never disturbs the links.
## B. Dataset
A **Dataset** is a named, reusable data source that snippets can reference by name instead of inlining data. Datasets are managed in the _Datasets_ manager and support multiple formats and two source kinds.
| Field | Type | Meaning |
| ------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | number | Unique numeric identifier. |
| `version` | number | Schema version of this record, used for read-time migration (see _Schema versioning_ below). |
| `name` | string | Unique, human-readable name; the key snippets reference via `datasetRefs`. |
| `data` | JSON value | The payload, shaped by format: raw CSV/TSV text, or the parsed JSON/TopoJSON value. For `source = url` this is the **fetched snapshot**, or `null` before the first successful fetch. |
| `format` | string | One of `json`, `csv`, `tsv`, `topojson`. |
| `source` | string | One of `inline` (data pasted into the record) or `url` (data fetched once from a remote address and snapshotted into the record). |
| `url` | string (url only) | The remote address a `url` dataset was fetched from, retained so it can be re-fetched ("Refresh"). Absent for inline datasets. |
| `fetchedAt` | ISO-timestamp or null | For `url` datasets: when the snapshot was last fetched, or `null` if never fetched. Absent for inline datasets. |
| `comment` | string | Free-form user note about the dataset. |
| `rowCount` | number or null | Number of data rows, or null when unknown/not applicable. |
| `columnCount` | number or null | Number of columns, or null when unknown/not applicable. |
| `columns` | string[] | Column names, in order. |
| `columnTypes` | array of `{ name, type }` | Per-column inferred type. `name` is the column; `type` is one of `number`, `string`, `date`, `boolean`. |
| `size` | number | Approximate payload size in bytes. |
| `created` | ISO-timestamp string | When the dataset was first added. |
| `modified` | ISO-timestamp string | When the dataset was last changed. |
The `rowCount`, `columnCount`, `columns`, `columnTypes`, and `size` fields are derived summaries computed when data is added or updated — including when a `url` dataset is fetched or refreshed; a fetched URL snapshot profiles exactly like inline data. They support previews and type display without re-parsing the full payload.
### Schema versioning
Both **Snippet** and **Dataset** records carry a numeric `version` recording the shape of that individual record. When a record is read from storage it is migrated up to the current shape before the app uses it; new writes always store the current version. A record written before versioning existed (no `version` field) is treated as version `1`. This is distinct from the storage container's own layout version, and from the _Import & Export_ envelope `version` (which describes the file format, not a record). Records exported via _Import & Export_ include their `version`.
The current **Dataset** version is `2`. The v1→v2 migration reflects the URL-snapshot model: a v1 `url` dataset stored its address in `data`, so migration moves that address into the new `url` field and clears `data` to `null` — the record becomes an _unfetched reference_ that renders against its live URL until the user refreshes it, at which point the fetched snapshot is stored and profiled.
## C. UserSettings
**UserSettings** holds persisted user preferences as a single structured record. The semantics and UX of each option are covered in _Settings_; the shape below is the storage contract.
| Field | Type | Meaning |
| ----------------------------- | ------- | -------------------------------------------------------------------------------------------- |
| `version` | number | Schema version of the settings record, used for migration. |
| `editor.fontSize` | number | Editor font size. |
| `editor.theme` | string | Editor color theme identifier. |
| `editor.minimap` | boolean | Whether the editor minimap is shown. |
| `editor.wordWrap` | string | `on` or `off`. |
| `editor.lineNumbers` | string | `on` or `off`. |
| `editor.tabSize` | number | Spaces per indentation level. |
| `performance.renderDebounce` | number | Delay (ms) before re-rendering the preview after edits. |
| `ui.theme` | string | App theme: `light` or `dark`. |
| `ui.previewFitMode` | string | Preview sizing: `default`, `width`, `height`, or `full`. |
| `ui.chartTheme` | string | Chart theme: `astrolabe`, `stock`, a preset id, or `custom:<id>` naming a _CustomTheme_ (G). |
| `formatting.dateFormat` | string | Date display mode: `smart`, `iso`, or `custom`. |
| `formatting.customDateFormat` | string | Pattern used when `dateFormat = custom`. |
A reference shape:
UserSettings = { version, editor: { fontSize, theme, minimap, wordWrap, lineNumbers, tabSize }, performance: { renderDebounce }, ui: { theme, previewFitMode, chartTheme }, formatting: { dateFormat, customDateFormat } }
## D. App / UI preferences (persisted separately)
Some preferences persist independently of _UserSettings_ so they can update frequently without rewriting the settings record. They are stored locally and restored on load.
- **Snippet sort preference** — how the _Snippet Library_ list is ordered. `sortBy` is one of `name`, `modified`, `created`; `sortOrder` is `asc` or `desc`. Default is `modified` / `desc` (most recently changed first).
- **Panel layout** — the resizable three-panel arrangement: per-pane widths and per-pane visibility (which panels are shown or hidden). Restored so the workspace reopens as the user left it.
## E. Persistence & limits
| Tier | What it holds | Capacity & behavior |
| ---------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| Snippet store | All _Snippet_ records | Local, with a practical budget of about 5 MB. A storage monitor tracks usage and surfaces warnings as the budget fills (see _Snippet Library_). |
| Dataset store | All _Dataset_ records | Local, in a separate, much higher-capacity store, suited to larger payloads. |
| Theme store | All _CustomTheme_ records (G) | Local, separate store; records are small (a config object plus metadata). |
| Font store | All _FontAsset_ records (H) | Local, separate store; holds raw font-file bytes, so it is sized like the dataset tier (per-face cap ~10 MB). |
| Settings & preferences | _UserSettings_ plus the app/UI preferences in (D) | Local, small. |
Everything stays in the browser — no server or account is involved. All tiers survive reload and function offline. Because capacity is finite and per-browser, _Import & Export_ is the supported path for backup and for moving data between browsers or devices.
## F. Cross-entity relationships
Snippets and datasets are linked **bidirectionally by dataset name**: `snippet.datasetRefs` holds dataset names, and each such name matches a `dataset.name`.
- From a snippet, `datasetRefs` yields its linked datasets.
- From a dataset, scanning snippets for its `name` in `datasetRefs` yields the snippets that reference it.
This name-based link is what the _Snippet Library_ and _Datasets_ surfaces use to show linkage in both directions. The actual resolution of a referenced dataset into spec data at render time is covered in _Live Preview_.
## G. CustomTheme
A **CustomTheme** is a user-named Vega-Lite config saved in the library and offered by the _Live Preview → Chart theme_ picker alongside the built-in themes and presets. It is created and edited in the _Theme Builder_ (see _Live Preview_).
| Field | Type | Meaning |
| ---------- | -------------------- | --------------------------------------------------------------------------------------------- |
| `id` | number | Unique numeric identifier. The picker/persistence selection id is the string `custom:<id>`. |
| `version` | number | Schema version of this record, used for read-time migration (see _Schema versioning_ above). |
| `name` | string | Unique, human-readable name shown in the picker (case-insensitive uniqueness, like datasets). |
| `config` | object | The Vega-Lite config injected at render time when this theme is selected. |
| `created` | ISO-timestamp string | When the theme was first created. |
| `modified` | ISO-timestamp string | When the theme was last changed. |
Selection is keyed by `id` (not name) so renaming a theme never invalidates the persisted `ui.chartTheme`. A persisted `custom:<id>` whose record no longer exists is not an error: charts render with the house style until the record appears (themes hydrate asynchronously), and deleting the actively-selected theme resets the selection to `astrolabe` explicitly. Custom themes travel in the _Import & Export_ envelope alongside snippets and datasets (spec §08).
## H. FontAsset
A **FontAsset** is a user-uploaded font face stored once and reused across themes and snippets. It is added and managed in the _Theme Builder → Type_ panel (see _Live Preview_), and referenced from a config's font slots by its `family` — exactly as a dataset is referenced by name. The raw bytes are registered as a live `FontFace` so a chart measures and renders the real face.
| Field | Type | Meaning |
| ---------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `id` | number | Unique numeric identifier (store key). |
| `version` | number | Schema version of this record, for read-time migration (see _Schema versioning_ above). |
| `family` | string | Unique CSS family name — the key configs reference (case-insensitive uniqueness, like datasets/themes). |
| `data` | bytes | The raw font-file bytes (registered as a `FontFace`). In an export envelope these are **base64-encoded** (spec §08). |
| `format` | `woff2`/`woff`/`ttf`/`otf` | Container format, from the file extension. |
| `fileName` | string | Original file name, kept for display and provenance. |
| `source` | `file`/`google` | Provenance. Only `file` (an upload) ships today; `google` is reserved for a later keyless-catalog tier. |
| `axes` | array, optional | Variation axes for a variable font (parsed from `fvar`); drives the `FontFace` weight/width ranges. Absent for a static face. |
| `size` | number | Byte length of `data`. |
| `created` | ISO-timestamp string | When the font was first added. |
| `modified` | ISO-timestamp string | When the font was last changed (e.g. renamed). |
A font is identified by its `family`. No theme or snippet stores a font field: the faces a config uses are derived by scanning its font slots (the config is the source of truth), so a snippet can use a font with no theme to carry it. Fonts travel in the _Import & Export_ envelope alongside snippets, datasets, and themes; on import a family clash **skips** the incoming face rather than renaming it (spec §08 → Name conflicts).
Reference in New Issue View Git Blame Copy Permalink