# Naming & Relationships How Astrolabe keeps entity **names unique** within a collection, and how it tracks the **bidirectional links** between snippets and datasets so they stay consistent as entities are created, imported, and renamed. Two concerns live here, and they reinforce each other: 1. **Name uniqueness** — every dataset has a unique name. Names are the primary key users see and the key snippets reference, so duplicates would be ambiguous. We reject duplicate names on create/rename, and auto-suffix collisions during bulk import. 2. **Relationship tracking** — a snippet references datasets _by name_ through its `datasetRefs: string[]` field. This is a bidirectional, name-based link: from a snippet you read its refs; from a dataset you scan snippets to find who uses it. Renaming a dataset must propagate to every snippet that points at it, in both the spec and the `datasetRefs` array, or the links rot. The hard, testable logic is **pure** and lives in `src/core/`. The parts that read and mutate stores live in `src/app/services/`. --- ## 1. Why names, not IDs, are the link Datasets carry a numeric `id`, but snippets reference them **by name** because that is what Vega-Lite uses: a spec resolves data through a named-data reference, `{ "data": { "name": "MyDataset" } }`. The name _is_ the contract between a spec and the dataset library. Storing a numeric id in the spec would mean the spec is no longer a standalone, paste-anywhere Vega-Lite document. The consequence: names must be unique (two datasets named `Sales` would make `{ "data": { "name": "Sales" } }` ambiguous), and renaming a dataset is a **graph operation**, not a single field write — every reference to the old name must move with it. --- ## 2. Name uniqueness (pure — `src/core/naming.ts`) ### 2.1 Uniqueness check Comparisons are **case-insensitive** (`Sales` and `sales` collide), so a single display name maps to a single dataset regardless of how a user types a reference. The check takes an optional `excludeId` so a rename can ignore the record being renamed (renaming `Sales` to `Sales` is not a collision with itself). ```ts // src/core/naming.ts /** Case-insensitive set of names already in use, minus an optional excluded id. */ export function isNameTaken( desired: string, datasets: ReadonlyArray<{ id: number; name: string }>, excludeId?: number, ): boolean { const lower = desired.trim().toLowerCase(); return datasets.some((d) => d.id !== excludeId && d.name.toLowerCase() === lower); } ``` ### 2.2 Making a unique name When a desired name is taken — during import, "extract inline data", or "build chart" — we do **not** overwrite the existing dataset. We derive the next free name by appending a numeric suffix: `Name` → `Name 2` → `Name 3`. The function takes the set of existing names so it has no store dependency and is trivially unit-testable. ```ts // src/core/naming.ts /** * Returns `desired` if free, else the first available `${desired} ${n}` (n >= 2). * `existingNames` is the set of names already in the collection. * Comparison is case-insensitive; the returned name preserves `desired`'s casing. */ export function makeUniqueName(desired: string, existingNames: Iterable): string { const taken = new Set(); for (const n of existingNames) taken.add(n.toLowerCase()); const base = desired.trim(); if (!taken.has(base.toLowerCase())) return base; let n = 2; while (taken.has(`${base} ${n}`.toLowerCase())) n++; return `${base} ${n}`; } ``` > If a base name already ends in a number (`Q1 2024`), the suffix still appends > (`Q1 2024 2`). That is intentional: we never parse meaning out of the name, > we only guarantee a free slot. Keep this dumb and predictable. **Do** - Use `isNameTaken` to reject duplicate create/rename in the UI before saving, and surface an error toast. - Use `makeUniqueName` for every non-interactive path (import, extract, build) where blocking the user would be worse than a silent, reported rename. - Pass `excludeId` on rename so an unchanged or case-only edit is allowed. **Don't** - Don't compare names case-sensitively anywhere — pick `toLowerCase()` once and use it consistently. - Don't let `makeUniqueName` mutate a store or read store state; it takes plain data and returns a string. --- ## 3. The bidirectional snippet ↔ dataset link ``` datasetRefs: ["Sales", "Regions"] (forward, on the snippet) Snippet ───────────────────────────────────────────────────► Dataset "Sales" ▲ │ └──────────── scan all snippets for "Sales" in datasetRefs ◄──────┘ (reverse, derived) ``` - **Forward** (snippet → datasets): read `snippet.datasetRefs`. Cheap, stored. - **Reverse** (dataset → snippets): there is no stored back-pointer. We compute it by scanning snippets. Keeping it _derived_ means it can never disagree with the forward links — there is one source of truth. `datasetRefs` is **derived from the spec**, not hand-maintained. It mirrors the dataset names referenced by the **draft** spec — the version being edited — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft (not only the last publish) keeps a snippet's linked-datasets display and the reverse lookup in step with what the editor shows — so a hand-typed reference or an Extract links its dataset without waiting for a publish. Recomputation runs only on a _valid_ spec — auto-save is debounced and parse-gated (spec §03B) — so a transiently-invalid draft never disturbs the links. ### 3.1 What counts as a reference, and extracting them (pure — `src/core/spec-data.ts`, `spec-refs.ts`) A library reference is exactly Vega-Lite **named data**: a `data` block with a string `name` and no `values`, `url`, or generator key (`sequence`/`sphere`/ `graticule`), whose name the spec does not define for itself via a top-level `datasets` map. A `name` riding on inline `values` or a `url` is Vega-Lite's runtime-rebind label — not a dependency — and self-defined `datasets` names resolve natively; both are left untouched. This mirrors Vega-Lite's own `isNamedData`, so Astrolabe **extends** Vega-Lite rather than diverging: every native data form keeps working, and only true references are tracked and resolved. That classification lives in **`core/spec-data`** (`classifyData`, `libraryRefName`) — the single predicate that reference extraction, rename (`spec-refs`), and render-time resolution (`rendering`) all route through, so they cannot disagree on what is a dependency. ```ts // src/core/spec-data.ts — the shared classifier (mirrors vega-lite/src/data.ts) /** The library name a `data` block references, or null for native VL data / self-defined names. */ export function libraryRefName(data: unknown, selfDefined: ReadonlySet): string | null { if (classifyData(data) !== 'named') return null; // url | values | generator → not a reference const { name } = data as { name: string }; return selfDefined.has(name) ? null : name; } ``` References appear in several places — top-level `data`, per-layer `data`, `data` inside `spec`/`facet`/`hconcat`/`vconcat`, and a lookup transform's `from.data`. Rather than enumerate the grammar, each pass walks the spec recursively but **prunes two keys**: a `data` object's payload (its `values`/rows) and the top-level `datasets` map hold user data, not nested specs. Without the prune, a data _row_ carrying a field literally named `data: { name: "x" }` is misread as a reference. ```ts // src/core/spec-refs.ts — the recursive walk; classification routes through spec-data export function extractDatasetRefs(spec: Json): string[] { const root = typeof spec === 'string' ? safeParse(spec) : spec; // unparseable → no refs const selfDefined = selfDefinedNames(root); const names = new Set(); const walk = (node: Json): void => { if (Array.isArray(node)) return void node.forEach(walk); if (node && typeof node === 'object') { const obj = node as Record; const refName = libraryRefName(obj.data, selfDefined); if (refName !== null) names.add(refName); for (const key of Object.keys(obj)) { if (key === 'data' || key === 'datasets') continue; // prune user-data payloads walk(obj[key]); } } }; walk(root); return [...names]; } ``` `recomputeDatasetRefs` is the thin sorted/de-duped wrapper stored on `snippet.datasetRefs`, run on every draft change and on publish. > A `spec` may be an object or a string (see the Data Model). Normalize once, > at the boundary, so the recursive walk never has to care. **Do** - Treat `extractDatasetRefs` as the single source of truth for "what does this spec reference". The reverse-lookup and rename paths both depend on it agreeing with what the renderer actually resolves — which holds because all of them classify through the same `core/spec-data` predicate. - Recompute and store `datasetRefs` on **every draft change and on publish** — but only through the parse-gated, debounced auto-save (`commitDraft`) and the programmatic extract/revert rewrites, never on raw keystrokes. That keeps the links in step with the edited draft while never recomputing from a transiently-invalid spec. - Keep the three ref walks in lockstep — extraction here, `renameDatasetInSpec`, and the renderer's `resolveDatasetRefs` (`src/core/rendering.ts`). They agree because they share both halves: the `libraryRefName` classifier (what is a reference) and the prune of the **same two keys** (`data`, `datasets`). If one classified differently, or descended into data payloads while another didn't, a row field named `data` would get counted, rewritten, or throw `DatasetNotFoundError`. **Don't** - Don't let two code paths each have their own idea of "referenced names". Renamer, ref-recomputer, and renderer must use the same walk shape. - Don't "enumerate the grammar" (scope the walk to a fixed list of container keys) to fix the payload-descent problem — pruning the two data-bearing keys stays correct as Vega-Lite's composition grammar grows; an allow-list rots. --- ## 4. Reverse lookup: who uses this dataset? The Dataset Manager shows a **usage badge** and a **Linked Snippets** list; the Snippet Library shows a snippet's linked datasets. Both come from one scan — no stored back-pointer to drift. The scan itself is **pure** and lives in core, taking the snippets as a parameter so the _same_ implementation serves two callers: the reactive UI (which passes its live `snippets` selection straight in) and the non-reactive service wrapper (which passes a store snapshot for programmatic callers). This is what makes "who references this" have exactly one implementation (§6). ```ts // src/core/relationships.ts — pure scan (unit-tested) /** Snippets whose datasetRefs include `name` (case-insensitive). */ export function snippetsReferencingDataset( snippets: readonly T[], name: string, ): T[] { const lower = name.toLowerCase(); return snippets.filter((s) => s.datasetRefs.some((ref) => ref.toLowerCase() === lower)); } /** Usage counts for the whole library in one pass, keyed by lower-cased name. */ export function datasetUsageCounts( snippets: readonly { datasetRefs: readonly string[] }[], ): Map; ``` There is no app-layer wrapper module around the scan. The reactive UI — a component subscribed to `snippets` — calls the core helper directly, so the badge and Linked Snippets list update the moment any snippet's draft changes its refs (auto-save) or it is published. A non-reactive caller binds the same helper to a snapshot inline at its call site (`snippetsReferencingDataset(useSnippetStore.getState().snippets, name)`) — one matching implementation, no `getState()` wrapper that would go stale in reactive code. **Do** - Keep reverse lookup a pure scan, parameterized on the snippets so both the reactive and snapshot callers share it. It is O(snippets) but the collections are small (library budget ~5 MB); clarity beats an index. - Call the core helper directly from a store-subscribed component for reactivity; bind it to a `getState()` snapshot inline for non-reactive code. **Don't** - Don't add a `referencedBy` array to datasets. A stored reverse pointer is a second source of truth that _will_ fall out of sync with `datasetRefs`. --- ## 5. Import: auto-suffix collisions, then report On import we never overwrite an existing record. A record whose name collides is renamed to a unique name via `makeUniqueName`, and **every rename is collected and reported to the user** (toast / summary) so the change is never silent. Crucially, names are reserved _as we go_ — within a single import, two incoming `Sales` datasets become `Sales 2` and `Sales 3`, not two `Sales 2`. The helper is generic over `{ name: string }` because two record kinds key on a unique name: datasets and custom chart themes (only dataset renames need propagation — nothing references a theme by name). ```ts // src/core/import-normalize.ts import { makeUniqueName } from './naming'; export interface NameRename { from: string; to: string; } /** * Returns incoming records with collision-free names, plus the renames applied. * `existing` are names already in the collection; `incoming` are records to add. */ export function dedupeIncomingNames( existing: ReadonlyArray, incoming: ReadonlyArray, ): { records: T[]; renames: NameRename[] } { const reserved = new Set(existing.map((n) => n.toLowerCase())); const renames: NameRename[] = []; const records = incoming.map((r) => { const unique = makeUniqueName(r.name, reserved); reserved.add(unique.toLowerCase()); // reserve so later imports don't collide if (unique !== r.name) renames.push({ from: r.name, to: unique }); return unique === r.name ? r : { ...r, name: unique }; }); return { records, renames }; } ``` > If imported snippets reference the renamed dataset, their `datasetRefs` and > specs must be rewritten to the new name too. The import flow does this purely, > before anything is committed: `applyDatasetRenamesToSnippets` > (`core/import-normalize`) rewrites the incoming snippet set per applied rename > using the same `renameDatasetInSpec` machinery as §6. ### 5.1 Where the import/export flow lives, and its rules **Flow:** header (Import/Export buttons in `App.tsx`) → `services/transfer.ts` (the only store-touching layer) → pure core (`core/import-normalize.ts` shape detection + normalization + the dedupe/rename/id-reassign helpers; `core/export-envelope.ts`) - browser IO (`infrastructure/file-transfer.ts`). The pure helpers are unit-tested hardest; `transfer.ts` only orchestrates (read stores → call core → commit → notify). The behavioral contract is spec §08. Three rules a future change must keep: - **Datasets commit before snippets** (`DatasetStore.addDatasets` then `SnippetStore.addSnippets`) so a snippet's by-name reference resolves against the just-added (possibly suffixed) dataset. - **Imported datasets get fresh monotonic numeric ids** (`addDatasets`), not their envelope ids. Safe — and necessary — because datasets are linked **by name, not id** (§1): id reuse would collide in IndexedDB, but renaming the _id_ breaks nothing. (This is why the old `Date.now()`-collision TODO on `add` doesn't bite import.) - **Rename propagation reads the spec, not only `datasetRefs`.** `applyDatasetRenamesToSnippets` finds the referenced name via `extractDatasetRefs(spec)` ∪ `datasetRefs`, so an imported snippet whose `datasetRefs` is absent/stale (a hand-crafted or foreign file) still gets its spec rewritten — the renderer resolves by spec, so a missed rename would break it. **Do** - Reserve each chosen name immediately so collisions _within_ one import are also resolved. - Return the rename list and show it; a silent rename looks like data loss. **Don't** - Don't overwrite or merge a same-named existing dataset on import. Suffix and keep both — the user decides what to delete. --- ## 6. Rename propagation: keep the link consistent Renaming a dataset is the operation that ties §2–§5 together. A rename must, in one atomic step: 1. Update the dataset's own `name`. 2. For **every snippet referencing the old name**: rewrite the named-data references inside its spec (`{ "data": { "name": "old" } }` → `{ "data": { "name": "new" } }`) — in **both** `spec` and `draftSpec`. 3. Recompute that snippet's `datasetRefs` from the rewritten **draft** spec (the tracked surface), so the forward link mirrors reality and the reverse scan stays correct. The spec rewrite is pure; the orchestration reads and writes stores. ```ts // src/core/spec-refs.ts — pure rewrite /** Returns a copy of `spec` with every data.name === oldName replaced by newName. */ export function renameDatasetInSpec(spec: Json, oldName: string, newName: string): Json { const obj = typeof spec === 'string' ? safeParse(spec) : spec; const selfDefined = selfDefinedNames(obj); // never rename a spec's own inline dataset name const rewrite = (node: Json): Json => { if (Array.isArray(node)) return node.map(rewrite); if (node && typeof node === 'object') { const out: Record = {}; for (const [k, v] of Object.entries(node as Record)) { // A `data` object is a reference site: rename a matching name, but never // recurse into its payload. `datasets` (self-defined inline data) is left // whole. Same prune as extractDatasetRefs — see §3.1. if (k === 'data' && v && typeof v === 'object' && !Array.isArray(v)) { const dv = v as Record; out[k] = dv.name === oldName && !selfDefined.has(oldName) ? { ...dv, name: newName } : dv; } else if (k === 'data' || k === 'datasets') { out[k] = v; } else { out[k] = rewrite(v); } } return out; } return node; }; const rewritten = rewrite(obj); // Preserve the original spec's stored shape (string vs object). return typeof spec === 'string' ? JSON.stringify(rewritten, null, 2) : rewritten; } ``` The one rename implementation in the live app is the store action **`SnippetStore.renameDatasetRefs(oldName, newName, now)`** — the action that owns the snippet collection. It matches by spec+draft _content_ (not `datasetRefs`, which mirrors only the draft), rewrites `spec`/`draftSpec` via `renameDatasetInSpec`, recomputes `datasetRefs` from the rewritten draft, and returns the number of snippets changed. Keeping the loop in the store means the reactive editor buffer is refreshed in the same atomic update when the active snippet's draft is rewritten. `DatasetStore.update` calls it whenever a save changes a dataset's name, so a rename propagates everywhere as part of the one user action — there is no separate coordinator module to call. > **Collision on rename.** The UI rename form rejects a name already in use via > `isNameTaken(newName, datasets, dataset.id)`. Programmatic renames (e.g. an > import flow) instead resolve with `makeUniqueName` first. The propagation > action itself does not invent a name — it assumes `newName` is the agreed > target. **Do** - Rewrite `spec` **and** `draftSpec`. A user mid-edit must not see their draft silently break because the dataset was renamed underneath them. - Recompute `datasetRefs` from the rewritten **draft** spec rather than string-replacing the array — the draft is the source of truth, the array is its mirror. - Find affected snippets by scanning each one's **spec and draft content** for the old name (`extractDatasetRefs`), not by its `datasetRefs` array. Because `datasetRefs` mirrors the draft, a name referenced only by the still-published spec (the user removed it from the draft but hasn't published) is absent from the array; matching on content rewrites it anyway, so the published spec can't rot to a renamed-away dataset. Scan the content (don't reserialize) so a non-referencing snippet's text stays byte-for-byte intact. The import-side `applyDatasetRenamesToSnippets` already follows this spec-content rule (§5.1). **Don't** - Don't update `datasetRefs` without also rewriting the spec — the rendered named-data reference would still point at the old, now-missing name. - Don't rename the dataset and skip propagation "for now". A half-applied rename is the exact inconsistency this whole document exists to prevent. --- ## 7. Snippet name provenance — the naming hierarchy Snippet names (unlike dataset names) need no uniqueness; what they need is a rule for **who may rewrite them**. Each snippet carries `nameSource` (spec §09A): `'user'` names are frozen — set by an explicit rename (`SnippetStore.renameSnippet`) and never touched by the app again; `'auto'` names are app-picked and keep tracking the spec. On publish, an auto-named snippet is re-named from the now-published content in priority order: the spec's `title` (string, line array, or `{ text }` forms), else a mark + encodings description, else the existing name stands. The derivation dialect is deliberately the same one `generateChartName` uses for builder output, so manually authored and builder-built snippets read alike in the library. Flow: `core/snippet.ts` (`deriveSnippetName`, `isAutoNamed`, `isDefaultSnippetName`) → `SnippetStore.publish` (the only rewrite site) / `renameSnippet` (the freeze site) → `SnippetLibrary`'s metadata panel (which must adopt a publish rename — arch 01 §2, editing buffers). Records predating `nameSource` have no provenance; `isAutoNamed` treats them as user-named unless the name is **provably** app-picked — the timestamp default shape, or identical to what `deriveSnippetName` returns for the record's own published spec. The conservative default is deliberate: rewriting a chosen name is worse than failing to track an auto one. --- ## 8. Where things live | Concern | Location | Pure? | Tested | | ------------------------------------------------------------------------ | -------------------------------- | ------------------- | ----------- | | `makeUniqueName`, `isNameTaken` | `src/core/naming.ts` | yes | unit | | `extractDatasetRefs`, `recomputeDatasetRefs` | `src/core/spec-refs.ts` | yes | unit | | `renameDatasetInSpec` | `src/core/spec-refs.ts` | yes | unit | | `snippetsReferencingDataset`, `datasetUsageCounts` (reverse-lookup scan) | `src/core/relationships.ts` | yes | unit | | `renameDatasetRefs` → updated count (rename propagation) | `src/app/stores/SnippetStore.ts` | no (mutates stores) | integration | | `dedupeIncomingNames` (datasets + custom themes) | `src/core/import-normalize.ts` | yes | unit | | `deriveSnippetName`, `isAutoNamed` (snippet name provenance) | `src/core/snippet.ts` | yes | unit | The dividing line: anything that takes plain data and returns plain data is **core** and unit-tested in isolation; anything that reaches into a Zustand store is a **store action or app service**. The rule of thumb — _the **draft** spec is the source of truth, `datasetRefs` mirrors it, the reverse lookup is derived_ — is what keeps the bidirectional link from ever needing manual repair.