Files
astrolabe/docs/architecture/02-persistence.md
T

32 KiB
Raw Blame History

02 · Persistence Architecture

How Astrolabe stores data in the browser, and the rules that keep that storage testable, portable, and safe to evolve. This document is the implementation contract for the persistence layer. For the behavioral data model (what fields a Snippet or Dataset has, what the tiers hold), see 09 · Data Model & Persistence; this document covers how the code is structured to implement it.


1. The Infrastructure-Adapter Principle

Rule: nothing outside src/app/infrastructure/ ever touches indexedDB, localStorage, window, location, or fetch directly. Every browser interaction (storage and network) goes through a typed adapter module that exposes plain async functions returning domain objects.

src/
├── core/                    # portable engine — NO browser APIs, NO React
├── app/
│   ├── stores/              # Zustand stores; calls infrastructure, never IDB/fetch
│   ├── services/            # business logic; calls infrastructure, never IDB/fetch
│   └── infrastructure/      # the ONLY place that imports indexedDB/localStorage/fetch
│       ├── snippet-store.ts     # IndexedDB: snippets (metadata + drafts)
│       ├── dataset-store.ts     # IndexedDB: datasets (heavy payloads)
│       ├── settings-store.ts    # localStorage: UserSettings
│       ├── ux-prefs.ts          # localStorage: app/UI prefs (sort, panel layout)
│       └── remote-data.ts       # network: fetch a URL dataset's body (the ONLY fetch)

Why this boundary exists

  • Testability. Stores and services depend on a small typed surface (getSnippet(id): Promise<Snippet | null>), not on the IndexedDB request API. Tests mock the adapter, not a browser global. The adapters themselves are tested directly against fake-indexeddb / a localStorage stub in Vitest.
  • Portability. src/core/ stays free of browser APIs so the spec/parse/transform logic can run in Node (tests, future CLI, SSR). The adapters are the seam where the portable core meets the browser.
  • Single place for migrations. Schema upgrades and record migrations live in exactly one module per store. A reader looking for "how does v1 data become v2 data" has one file to open, not a scattered set of if (record.someOldField) checks across the UI.
  • Failure containment. Quota errors, corrupt JSON, and missing keys are handled at the boundary and converted into typed results (or sane fallbacks), so the rest of the app never sees a raw DOMException.

Do: import { saveSnippet } from '@/app/infrastructure/snippet-store' Don't: indexedDB.open(...), localStorage.getItem(...), or fetch(...) anywhere in a component, store, or service.

Background vs. interactive adapters

Most adapters are driven by background subscribers (arch 01 §5, Effects): a store changes, a startup subscriber writes it through to IndexedDB — the store never calls the adapter itself. The network adapter is the exception. Fetching a URL dataset is a user-initiated action with its own pending/error UI, so the component calls remote-data.ts directly (components may call adapters — cf. navigator.clipboard) and hands the fetched body to pure store actions (DatasetStore.commitUrlSnapshot / refreshDataset). The store never fetches, so it stays browser-free and unit-testable on already-fetched text.

Rule: keep fetch behind remote-data.ts; orchestrate the URL-dataset fetch in the component (busy state + the "paste data inline instead" recovery), not the store. Store commit actions only ever receive already-fetched text.

URL-dataset snapshot lifecycle (the files a change to it touches): add / Refresh → component fetches (infrastructure/remote-data.ts) → core/dataset.snapshotFromText shapes the body by sniffed format → DatasetStore.commitUrlSnapshot / refreshDataset snapshots + profiles it exactly like inline data → render resolves it cached-first in core/rendering.resolvedData (a live-URL fallback applies only while a URL dataset is still unfetched). See spec §05 for the behavior.

A fetched body is held whole in memory and stored whole in IndexedDB, so remote-data.ts caps it at a fixed ceiling (MAX_REMOTE_BYTES): it rejects on the declared Content-Length before reading, so an oversized download never lands in memory, with a body-length backstop for chunked responses that declare no length. The classified too-large reason maps to its own copy in services/remote-data-errors.ts — neither the inline-paste nor the retry recovery helps an oversized file, so that message points at using a smaller or pre-aggregated source instead.


2. IndexedDB Wrapper

IndexedDB's native API is event-based (onsuccess/onerror) and verbose. The adapter wraps it into promises and exposes a tiny CRUD surface per object store. Define one shared helper and build typed stores on top of it.

2.1 Opening the database

Open with an explicit version number and an onupgradeneeded handler that creates/upgrades object stores. The version is a monotonically increasing integer; bump it whenever the store layout changes (a new object store, a new index). It is independent of per-record schema versions (§4).

// src/app/infrastructure/db.ts
const DB_NAME = 'astrolabe';
const DB_VERSION = 1;

let dbPromise: Promise<IDBDatabase> | null = null;

export function openDB(): Promise<IDBDatabase> {
  // Memoize: opening is idempotent and cheap to share across calls.
  if (dbPromise) return dbPromise;

  dbPromise = new Promise((resolve, reject) => {
    const req = indexedDB.open(DB_NAME, DB_VERSION);

    req.onupgradeneeded = (event) => {
      const db = req.result;
      const oldVersion = event.oldVersion;

      // Create stores idempotently — guard every create.
      if (!db.objectStoreNames.contains('snippets')) {
        db.createObjectStore('snippets', { keyPath: 'id' });
      }
      if (!db.objectStoreNames.contains('datasets')) {
        db.createObjectStore('datasets', { keyPath: 'id' });
      }

      // Per-version store-layout migrations go here, gated on oldVersion.
      // if (oldVersion < 2) { /* add index, split a store, ... */ }
      void oldVersion;
    };

    req.onsuccess = () => resolve(req.result);
    req.onerror = () => reject(req.error ?? new Error('Failed to open IndexedDB'));
  });

  return dbPromise;
}

Verify the layout, don't trust the version. An interrupted upgrade can stamp the new version without creating the new stores (observed in dev: a hot reload opened a bumped DB_VERSION before the store-creation code for it existed) — after which onupgradeneeded never fires again for that version and every transaction on the missing store throws NotFoundError, permanently. The real openDB therefore checks db.objectStoreNames against the expected store list after every successful open and, if anything is missing, closes and reopens at db.version + 1 to force another (idempotent) upgrade pass. Two consequences: the database self-heals instead of being stuck until manually deleted, and the on-disk version may run ahead of DB_VERSION — so the open also catches VersionError and retries without an explicit version. Covered by db.test.ts (fake-indexeddb).

2.2 Promise-wrapped CRUD helpers

Wrap a single IDB request and a whole transaction so callers write linear async/await code.

// src/app/infrastructure/db.ts (continued)
function wrap<T>(req: IDBRequest<T>): Promise<T> {
  return new Promise((resolve, reject) => {
    req.onsuccess = () => resolve(req.result);
    req.onerror = () => reject(req.error);
  });
}

async function tx<T>(
  store: string,
  mode: IDBTransactionMode,
  run: (s: IDBObjectStore) => IDBRequest<T>,
): Promise<T> {
  const db = await openDB();
  return new Promise<T>((resolve, reject) => {
    const transaction = db.transaction(store, mode);
    const request = run(transaction.objectStore(store));
    transaction.oncomplete = () => resolve(request.result);
    transaction.onerror = () => reject(transaction.error);
    transaction.onabort = () => reject(transaction.error);
  });
}

export const get = <T>(store: string, key: IDBValidKey) =>
  tx<T | undefined>(store, 'readonly', (s) => s.get(key) as IDBRequest<T | undefined>);

export const getAll = <T>(store: string) =>
  tx<T[]>(store, 'readonly', (s) => s.getAll() as IDBRequest<T[]>);

export const put = <T>(store: string, value: T) =>
  tx<IDBValidKey>(store, 'readwrite', (s) => s.put(value as any));

export const del = (store: string, key: IDBValidKey) =>
  tx<undefined>(store, 'readwrite', (s) => s.delete(key) as IDBRequest<undefined>);

Do: resolve on transaction.oncomplete, not on the request's onsuccess — the write is only durable once the transaction commits. Don't: hold an IndexedDB transaction open across an await to non-IDB work; transactions auto-close when the microtask queue drains and you'll get TransactionInactiveError.


3. Lazy Loading: Metadata vs Heavy Payloads

A snippet library can grow large, and datasets can be megabytes each (CSV text, parsed TopoJSON). Loading every dataset payload at startup just to render a list of names is wasteful and slow. The rule:

Store record metadata separately from large payloads. Load heavy data on demand. Treat null as "exists but not loaded yet" — distinct from absent.

For Astrolabe this maps cleanly onto the two stores:

  • snippets — snippet records are small (a spec is JSON text). They load eagerly as a set when the library opens.
  • datasets — the data payload is the heavy part. The list view needs only the derived summary fields (name, format, source, rowCount, columnCount, columns, size, timestamps). Load data only when a snippet that references the dataset is actually previewed.

There are two ways to implement the split; pick per store:

  1. Two object stores (datasets for metadata, dataset-payloads keyed by the same id for data) — strongest separation; a getAll on metadata never touches payload bytes.
  2. One store, lazy field — keep data in the record but set it to null on the bulk list load and fetch it per-id on demand.

Target, not yet built. Datasets currently load in full via loadDatasets() (infrastructure/dataset-store.ts); the lazy-field approach below is the intended direction for when dataset size demands it, not a description of shipped code. The plan: keep data in the one store but set it to null on the bulk list load (data === null = "summary loaded, payload not yet") and fetch per-id on demand.

// src/app/infrastructure/dataset-store.ts
import { get, getAll, put } from './db';
import { migrateDataset, type Dataset } from './dataset-migrations';

/** List view: returns every dataset's summary, payload nulled out. */
export async function loadDatasetSummaries(): Promise<Dataset[]> {
  const records = await getAll<Dataset>('datasets');
  return records.map((r) => ({ ...migrateDataset(r), data: null }));
}

/** Detail/preview: load (or return cached) full payload for one dataset. */
export async function ensureDatasetData(dataset: Dataset): Promise<Dataset['data']> {
  if (dataset.data !== null && dataset.data !== undefined) return dataset.data; // already loaded
  const record = await get<Dataset>('datasets', dataset.id);
  dataset.data = record?.data ?? null;
  return dataset.data;
}

Do: use null for "not loaded" and a real value (including '' or []) for "loaded but empty." The distinction prevents a re-fetch loop. Don't: overwrite a stored payload with null on save. When persisting a record whose data is null (never loaded into memory), skip writing the payload field and leave the stored bytes intact — otherwise a list-load-then-save round-trip silently destroys data.


4. Per-Record Schema Versioning & Read-Time Migration

The IndexedDB database version (§2.1) governs store layout. A separate per-record version field governs the shape of an individual record. Both Snippet and Dataset records carry version (and created / modified timestamps). This lets record shapes evolve without forcing an onupgradeneeded database bump for every field rename.

Migrations are applied on read — when a record comes out of the store, run it through a migration function that upgrades it to the current shape before the app sees it. New writes always store the current version.

// src/app/infrastructure/snippet-migrations.ts
export const CURRENT_SNIPPET_VERSION = 2;

export function migrateSnippet(raw: any): Snippet {
  let r = { ...raw };
  const v = r.version ?? 1; // records written before versioning existed are v1

  if (v < 2) {
    // Example: a v1 snippet had a single `spec`; v2 splits draft from published.
    r.draftSpec = r.draftSpec ?? r.spec;
    r.tags = r.tags ?? [];
    r.datasetRefs = r.datasetRefs ?? [];
  }
  // if (v < 3) { ... }

  r.version = CURRENT_SNIPPET_VERSION;
  return r as Snippet;
}
// src/app/infrastructure/snippet-store.ts
export async function loadSnippets(): Promise<Snippet[]> {
  const records = await getAll<any>('snippets');
  return records.map(migrateSnippet); // upgrade every record at the boundary
}

export async function saveSnippet(s: Snippet): Promise<void> {
  await put('snippets', {
    ...s,
    version: CURRENT_SNIPPET_VERSION,
    modified: new Date().toISOString(),
  });
}

Rationale

  • Read-time migration is forgiving. Old records sitting untouched in the store keep working; they upgrade lazily the next time they're loaded and re-saved. There is no big-bang migration step that can fail halfway.
  • Tolerate unknown fields. A migration normalizes missing/old fields but must not strip fields it doesn't recognize — a record written by a newer build that downgraded must round-trip without data loss. Spread the original ({ ...raw }) and only fill in what's missing.
  • One function, well tested. Each migration step is a pure function over a plain object — trivial to unit-test with fixture records from each historical version.

Do: default version to the earliest shape (1) when the field is absent. Don't: branch on the presence of individual fields scattered through the app to detect "old data." Centralize that knowledge in the migration function.

Mirror shape changes in the import normalizer. Imported records are built from a file, not read from IndexedDB, so they never pass through migrate<Entity>core/import-normalize.ts upgrades them independently. A migration that changes a field's shape must be applied in both places or import produces a malformed record. (E.g. the dataset v1→v2 URL-snapshot reshaping — address moves from data into url, data cleared — lives in both migrateDataset and normalizeDataset.)


5. localStorage Preferences (Settings & App/UI Prefs)

Small, frequently-read structured records live in localStorage, not IndexedDB: UserSettings (one record) and app/UI preferences (snippet sort, panel layout). Why split them from UserSettings? UI prefs change often (drag a panel divider, toggle a sort) and shouldn't force a rewrite of the whole settings blob on every interaction.

The pattern is load-with-fallback, per-slice write-through merge.

  • Load-with-fallback: merge the parsed stored object over a complete DEFAULTS constant. Missing keys (a setting added in a later build) and malformed JSON silently fall back to defaults — the app always gets a fully-populated object and never undefined-crashes on a new field. For UserSettings, the normalization is pure and lives in @core/settings (loadSettings(raw) / defaultSettings()) — it clamps ranges and validates enums; the infra adapter is just the thin localStorage reader (loadUserSettings() = loadSettings(readRaw())).
  • Per-slice write-through merge: every update reads current, merges in just its slice, and writes back. The one astrolabe:settings record has multiple independent writers, because settings are distributed and live-applied (spec §07 — no central save, no Apply step): the header theme toggle writes ui.theme, the preview Fit control writes ui.previewFitMode, the preview Chart-theme picker writes ui.chartTheme, and the per-pane settings clusters write editor/performance/formatting. A writer that replaced the whole record would clobber the slices it doesn't own — so each must field-merge. (There is deliberately no whole-record saveSettings.)
  • Environment-guarded: localStorage is absent or throws in some test/SSR contexts; guard access and degrade to defaults rather than throwing.
// src/app/infrastructure/settings-store.ts
const KEY = 'astrolabe:settings';

export const CURRENT_SETTINGS_VERSION = 1;

export interface UserSettings {
  version: number;
  editor: {
    fontSize: number;
    theme: string;
    minimap: boolean;
    wordWrap: 'on' | 'off';
    lineNumbers: 'on' | 'off';
    tabSize: number;
  };
  performance: { renderDebounce: number };
  ui: {
    theme: 'light' | 'dark';
    previewFitMode: 'default' | 'width' | 'height' | 'full';
    chartTheme: ChartThemeId; // 'astrolabe' | 'stock' | vega-themes preset id
  };
  formatting: { dateFormat: 'smart' | 'iso' | 'custom'; customDateFormat: string };
}

// The spec §07 table records these defaults — keep the two matching; this is
// just where they're encoded.
const DEFAULTS: UserSettings = {
  version: CURRENT_SETTINGS_VERSION,
  editor: {
    fontSize: 12,
    theme: 'auto',
    minimap: false,
    wordWrap: 'on',
    lineNumbers: 'on',
    tabSize: 2,
  },
  performance: { renderDebounce: 1500 },
  ui: { theme: 'light', previewFitMode: 'default', chartTheme: 'astrolabe' },
  formatting: { dateFormat: 'smart', customDateFormat: '' },
};

// NOTE — editor.theme default is 'auto': the editor theme follows the app UI
// theme (light -> light editor theme, dark -> dark) via custom Monaco
// themes that match the app chrome, unless the user picks an explicit override.
// The explicit-override option set (custom themes; whether to include High
// Contrast or the stock Monaco themes) is still TBD — see spec §07's provisional
// editor-theme note. Resolve the `'auto'` sentinel to a concrete Monaco theme at
// editor-config time, keyed off the current UI theme.

function available(): boolean {
  try {
    return typeof localStorage !== 'undefined' && typeof localStorage.getItem === 'function';
  } catch {
    return false; // access itself can throw (e.g. blocked storage)
  }
}

export function loadSettings(): UserSettings {
  if (!available()) return structuredClone(DEFAULTS);
  try {
    const raw = localStorage.getItem(KEY);
    if (!raw) return structuredClone(DEFAULTS);
    const p = JSON.parse(raw);
    // Deep-merge each group over defaults so new keys fall back silently.
    return {
      version: CURRENT_SETTINGS_VERSION,
      editor: { ...DEFAULTS.editor, ...p.editor },
      performance: { ...DEFAULTS.performance, ...p.performance },
      ui: { ...DEFAULTS.ui, ...p.ui },
      formatting: { ...DEFAULTS.formatting, ...p.formatting },
    };
  } catch (err) {
    console.warn('[settings] failed to load, using defaults', err);
    return structuredClone(DEFAULTS);
  }
}

// No whole-record save — each live control merges only its slice into the shared
// record, so the others survive (see the per-slice rule above):
//   saveUiTheme(theme)            -> { ...current, ui: { ...current.ui, theme } }
//   savePreviewFitMode(mode)      -> { ...current, ui: { ...current.ui, previewFitMode } }
//   saveChartTheme(chartTheme)    -> { ...current, ui: { ...current.ui, chartTheme } }
//   saveManagedSettings(managed)  -> { ...current, editor, performance, formatting }

The above sketch keeps the DEFAULTS/merge shape inline for illustration, but the authoritative defaults + normalization now live in @core/settings (pure); the infra adapter calls them and owns only the localStorage IO + the slice writers. Keep the two in sync via that one core source, not a second copy here.

App/UI prefs follow the identical guard+fallback pattern, but live in one record under their own key, astrolabe:ux-prefs (the snippet sort and the panel layout together — the plan's "ux-prefs for sort + panel layout", §09D):

// src/app/infrastructure/ux-prefs.ts
const KEY = 'astrolabe:ux-prefs'; // { panelLayout: { libraryWidth, previewWidth }, sort: { … } }

// loadPanelLayout()/savePanelLayout() (and sort later) mirror §5's guard+fallback shape,
// merging the changed section so a frequent write (a drag) never clobbers the others.

One nuance vs. the settings record: the panel-layout widths are validated (positive finite numbers) and surfaced as undefined when absent/invalid; the defaults live in the PanesStore (PANE_DEFAULT), applied at hydrate, rather than merged in the adapter. Persistence of the live drag is debounced in orchestration/panes.ts (a drag emits an update per pointer move).

Do: keep a single complete DEFAULTS object as the source of truth and merge over it. Don't: read individual keys with bespoke ?? fallback at each call site; one stale default and the shapes drift.

Testing: exercise localStorage adapters against an injected stub (vi.stubGlobal('localStorage', …)), not the ambient global. Under Node + happy-dom a non-functional Node localStorage global shadows happy-dom's, so relying on the ambient one fails with localStorage.clear is not a function. Applies to every prefs/settings adapter test (settings-store today; dataset-payload/prefs stores later).


6. Storage Tiers & the Composition Monitor

Astrolabe has three tiers with different capacities and risk profiles:

Tier Backing Holds Behavior
Snippet store IndexedDB snippets All snippet records Snippets are user-authored and irreplaceable, so writes fail loudly on quota (below).
Dataset store IndexedDB datasets All dataset payloads Separate, high-capacity; suited to large payloads. Loaded in full today; lazy loading is a §3 target.
Settings & prefs localStorage UserSettings + app/UI prefs (§5) Small; effectively unbounded for this use.

Splitting snippets and datasets into separate stores means a few large datasets can't crowd out snippets, and lets the storage monitor break usage down by tier.

Composing the storage breakdown

The monitor shows what storage is made of — Snippets · Datasets · App — not a "used of quota" gauge. The browser's quota is a padded, deliberately fuzzed approximation, not a real free-space figure (web.dev → storage-for-the-web), so a budget fraction is false precision. We use only the reliable usage (bytes actually stored for the origin) and measure our own tiers, deriving the rest:

  • Snippets / Datasets — measured directly: snippets serialized (jsonByteSize), datasets summed from each record's size. Always available, no API needed.
  • App = usage snippets datasets — the precached app shell + IndexedDB overhead. Shown only when the Storage Manager API yields usage; otherwise the breakdown is just snippets + datasets.
  • Hidden below a floor. The whole monitor stays hidden until user data (snippets + datasets) reaches STORAGE_MONITOR_MIN_USER_BYTES (10 MB). Keyed off user bytes, not total: the ~precache baseline is roughly constant, so gating on total would make it always-visible and the bar App-dominated.

The pure summarizer lives in core (unit-tested without a browser); the adapter only does I/O.

// src/core/storage-estimate.ts — pure, unit-tested (no I/O)
export interface StorageComposition {
  segments: { key: 'snippets' | 'datasets' | 'app'; label: string; bytes: number }[];
  totalBytes: number; // origin usage when measured, else snippets + datasets
  originMeasured: boolean; // false when the estimate API is absent
}
export function summarizeStorage(input: {
  usageBytes?: number;
  snippetBytes: number;
  datasetBytes: number;
}): StorageComposition {
  /* app = usage  known, included only when usage ≥ known (else estimate lag) */
}
export function jsonByteSize(value: unknown): number; // UTF-8 bytes of JSON.stringify(value)

// src/app/infrastructure/storage-estimate.ts — the only browser-touching part
export async function readOriginUsage(): Promise<number | undefined> {
  const storage = typeof navigator !== 'undefined' ? navigator.storage : undefined;
  if (!storage || typeof storage.estimate !== 'function') return undefined;
  const { usage } = await storage.estimate(); // quota deliberately ignored
  return typeof usage === 'number' ? usage : undefined;
}

The bar is decorative; the legend is the data. There is no role="meter" — a meter needs a meaningful maximum, which a composition with no fixed ceiling lacks (APG → meter). The legend's text labels + sizes are the accessible source of truth, so meaning never rests on hue (WCAG 1.4.1). There are no "almost full" percentage thresholds — a fraction would key off the untrustworthy quota; the genuine out-of-room event surfaces at save time (below), where it is accurate.

Fail loudly, never silently lose data

When a write would exceed quota, IndexedDB rejects with a QuotaExceededError. Quota is whole-origin, so it's normalized once at the single write path — db.put — into a typed StorageQuotaError. Every typed adapter inherits fail-loud behavior without repeating the check, and consumers branch on the type instead of sniffing a DOMException.

// src/app/infrastructure/db.ts — the one write path
export const put = <T>(store: string, value: T): Promise<IDBValidKey> =>
  tx<IDBValidKey>(store, 'readwrite', (s) => s.put(value)).catch((err: unknown) => {
    if (err instanceof DOMException && err.name === 'QuotaExceededError') {
      throw new StorageQuotaError(); // never silently drop the write
    }
    throw err;
  });

Do: surface quota warnings before the budget is hit (the 80% threshold) and hard errors loudly when a write fails. Don't: wrap a save in a bare try/catch {} that logs and returns — that turns "your work wasn't saved" into a silent data-loss bug. The only thing that may safely swallow an error is a read failure, where falling back to defaults/empty is the correct behavior. Rule: every write goes through db.put. An adapter that opens its own tx(store, 'readwrite', …) instead bypasses quota normalization and silently loses the typed StorageQuotaError — a regression the type system won't catch.

The adapter propagating is only half — a consumer must catch and surface it. A fire-and-forget void saveSnippet(n) re-buries the very error the adapter took care to throw. Persistence write-backs are wired as store subscribers, so the surfacing path is:

orchestration/snippet-persistence.ts (write-through .catch) / orchestration/startup.ts (load .catch, then run in memory) → services/storage-errors.ts (pure error→message mapper) → notify() (stores/NotificationStore) → Toaster.

Rules this encodes (spec §10 "told when a save fails"): never void-fire a persist without a .catch that maps the error to a toast — storageErrorNotification(op, err) for snippets (bespoke "your library" / "your changes" copy), entityStorageErrorNotification(noun, op, err) for the other tiers (the same shape with the entity's own noun). The mapper splits user-fixable (storage full → next step, no diagnostic) from not (blocked storage → plain explanation + a reportable detail); and a blocked store at startup warns and runs in memory rather than rejecting into the void.

Multi-record writes (import): atomicity at the service boundary

db.ts exposes only per-store request/transaction helpers — there is no wrapper for a single transaction spanning many records across stores. So a bulk operation like import (services/transfer.ts) cannot be truly atomic at the IDB layer; it achieves atomicity at the service boundary instead: write the new records to IndexedDB first, tracking what succeeded, and only on full success commit to the Zustand stores. On any write failure (typically QuotaExceededError) it rolls back best-effort — deletes the records written so far (Promise.allSettled) and removes any datasets and custom themes already added to their stores (their persistence subscribers propagate the removals to IDB) — so the spec §08 "no partial import is committed" contract holds and the user gets an actionable "storage full, delete and retry" message.

Rule: for any operation that persists multiple records, write-then-commit and roll back on failure — never mutate the in-memory stores before the writes are known to have landed (a half-merged workspace is worse than a failed import). Limit: rollback is best-effort, not transactional; if the rollback deletes themselves fail, orphan records can remain (invisible to the app — never added to a store — and cleaned up on the next successful write). True cross-record atomicity would require exposing a raw multi-store transaction from db.ts; defer that until a second multi-record writer needs it.


7. Checklist for Adding a New Persisted Entity

  1. Define the record type with id, created, modified, and a version field.
  2. Decide the tier: small + critical → IndexedDB store with a monitored budget; large payload → separate high-capacity store with lazy loading (§3); tiny + frequently changing → localStorage pref (§5).
  3. Add the object store in openDB's onupgradeneeded, guarded by contains(...); bump DB_VERSION only if you changed store layout.
  4. Add a migrate<Entity>() function and call it on every read.
  5. Expose typed load*/save*/ensure* functions from one infrastructure module — and from only there.
  6. Add the app layer: a Zustand store whose low-level add/update/remove are the single mutation point for the collection, and a write-through subscriber in orchestration/ — a thin wrapper over the shared wireEntityWriteThrough(store, select, { save, remove, onError }) helper (entity-persistence.ts), which diffs the array against the previous snapshot, upserts changed records, deletes missing ones, and toasts on failure.
  7. Hydrate in orchestration/startup.ts and wire the subscriber after hydrate — wiring first would re-save every loaded record on each startup.
  8. Quota propagation is automatic (db.put throws StorageQuotaError) — just pass an onError that maps it via entityStorageErrorNotification(noun, …). If the tier has a budget, also hook it into the storage monitor.
  9. Test the adapter against fake-indexeddb / a localStorage stub; test the migration with fixtures from each historical version.

The stack for one entity is four files with fixed roles: infrastructure/<entity>-store.ts (typed IDB adapter) + infrastructure/<entity>-migrations.ts (read-time upgrade) + stores/<Entity>Store.ts (in-memory collection + feature state) + orchestration/<entity>-persistence.ts (write-through, a thin call to the shared wireEntityWriteThrough), joined in startup.ts. Snippets, datasets, custom themes, and user fonts each follow it.