# Naming & Relationships How Astrolabe keeps entity **names unique** within a collection, and how it tracks the **bidirectional links** between snippets and datasets so they stay consistent as entities are created, imported, and renamed. Two concerns live here, and they reinforce each other: 1. **Name uniqueness** — every dataset has a unique name. Names are the primary key users see and the key snippets reference, so duplicates would be ambiguous. We reject duplicate names on create/rename, and auto-suffix collisions during bulk import. 2. **Relationship tracking** — a snippet references datasets _by name_ through its `datasetRefs: string[]` field. This is a bidirectional, name-based link: from a snippet you read its refs; from a dataset you scan snippets to find who uses it. Renaming a dataset must propagate to every snippet that points at it, in both the spec and the `datasetRefs` array, or the links rot. The hard, testable logic is **pure** and lives in `src/core/`. The parts that read and mutate stores live in `src/app/services/`. --- ## 1. Why names, not IDs, are the link Datasets carry a numeric `id`, but snippets reference them **by name** because that is what Vega-Lite uses: a spec resolves data through a named-data reference, `{ "data": { "name": "MyDataset" } }`. The name _is_ the contract between a spec and the dataset library. Storing a numeric id in the spec would mean the spec is no longer a standalone, paste-anywhere Vega-Lite document. The consequence: names must be unique (two datasets named `Sales` would make `{ "data": { "name": "Sales" } }` ambiguous), and renaming a dataset is a **graph operation**, not a single field write — every reference to the old name must move with it. --- ## 2. Name uniqueness (pure — `src/core/naming.ts`) ### 2.1 Uniqueness check Comparisons are **case-insensitive** (`Sales` and `sales` collide), so a single display name maps to a single dataset regardless of how a user types a reference. The check takes an optional `excludeId` so a rename can ignore the record being renamed (renaming `Sales` to `Sales` is not a collision with itself). ```ts // src/core/naming.ts /** Case-insensitive set of names already in use, minus an optional excluded id. */ export function isNameTaken( desired: string, datasets: ReadonlyArray<{ id: number; name: string }>, excludeId?: number, ): boolean { const lower = desired.trim().toLowerCase(); return datasets.some((d) => d.id !== excludeId && d.name.toLowerCase() === lower); } ``` ### 2.2 Making a unique name When a desired name is taken — during import, "extract inline data", or "build chart" — we do **not** overwrite the existing dataset. We derive the next free name by appending a numeric suffix: `Name` → `Name 2` → `Name 3`. The function takes the set of existing names so it has no store dependency and is trivially unit-testable. ```ts // src/core/naming.ts /** * Returns `desired` if free, else the first available `${desired} ${n}` (n >= 2). * `existingNames` is the set of names already in the collection. * Comparison is case-insensitive; the returned name preserves `desired`'s casing. */ export function makeUniqueName(desired: string, existingNames: Iterable): string { const taken = new Set(); for (const n of existingNames) taken.add(n.toLowerCase()); const base = desired.trim(); if (!taken.has(base.toLowerCase())) return base; let n = 2; while (taken.has(`${base} ${n}`.toLowerCase())) n++; return `${base} ${n}`; } ``` > If a base name already ends in a number (`Q1 2024`), the suffix still appends > (`Q1 2024 2`). That is intentional: we never parse meaning out of the name, > we only guarantee a free slot. Keep this dumb and predictable. **Do** - Use `isNameTaken` to reject duplicate create/rename in the UI before saving, and surface an error toast. - Use `makeUniqueName` for every non-interactive path (import, extract, build) where blocking the user would be worse than a silent, reported rename. - Pass `excludeId` on rename so an unchanged or case-only edit is allowed. **Don't** - Don't compare names case-sensitively anywhere — pick `toLowerCase()` once and use it consistently. - Don't let `makeUniqueName` mutate a store or read store state; it takes plain data and returns a string. --- ## 3. The bidirectional snippet ↔ dataset link ``` datasetRefs: ["Sales", "Regions"] (forward, on the snippet) Snippet ───────────────────────────────────────────────────► Dataset "Sales" ▲ │ └──────────── scan all snippets for "Sales" in datasetRefs ◄──────┘ (reverse, derived) ``` - **Forward** (snippet → datasets): read `snippet.datasetRefs`. Cheap, stored. - **Reverse** (dataset → snippets): there is no stored back-pointer. We compute it by scanning snippets. Keeping it _derived_ means it can never disagree with the forward links — there is one source of truth. `datasetRefs` is **derived from the spec**, not hand-maintained. It mirrors the dataset names referenced by the **draft** spec — the version being edited — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft (not only the last publish) keeps a snippet's linked-datasets display and the reverse lookup in step with what the editor shows — so a hand-typed reference or an Extract links its dataset without waiting for a publish. Recomputation runs only on a _valid_ spec — auto-save is debounced and parse-gated (spec §03B) — so a transiently-invalid draft never disturbs the links. ### 3.1 Extracting referenced names from a spec (pure — `src/core/spec-refs.ts`) A Vega-Lite spec can reference named data in several places: the top-level `data`, per-layer `data`, `data` inside `spec`/`facet`/`hconcat`/`vconcat`, and a lookup transform's `from.data`. A spec may also define its OWN inline datasets via a top-level `datasets` map — those are self-defined, not library references. Rather than enumerate Vega-Lite's grammar, we walk the spec recursively and collect every `{ data: { name } }` — but **prune two keys**: never recurse into a `data` object's payload (its `values`/rows) or the top-level `datasets` map, because those hold user data, not nested specs. Without the prune, a data _row_ carrying a field literally named `data: { name: "x" }` is misread as a reference. This is pure, deterministic, and the most heavily unit-tested function here. ```ts // src/core/spec-refs.ts type Json = unknown; /** Collects every library dataset name referenced by `{ data: { name } }`, excluding self-defined ones. */ export function extractDatasetRefs(spec: Json): string[] { const root = typeof spec === 'string' ? safeParse(spec) : spec; const selfDefined = selfDefinedNames(root); // names from the spec's own top-level `datasets` const names = new Set(); const walk = (node: Json): void => { if (Array.isArray(node)) { for (const item of node) walk(item); return; } if (node && typeof node === 'object') { const obj = node as Record; const data = obj.data as Record | undefined; if (data && typeof data === 'object' && typeof data.name === 'string') { if (!selfDefined.has(data.name)) names.add(data.name); } // Prune: a `data` payload and the `datasets` map hold user data, not refs. for (const key of Object.keys(obj)) { if (key === 'data' || key === 'datasets') continue; walk(obj[key]); } } }; walk(root); return [...names]; } function safeParse(s: string): Json { try { return JSON.parse(s); } catch { return null; // an unparseable draft simply has no resolvable refs } } ``` ```ts // src/core/spec-refs.ts — thin wrapper, run on every draft change and on publish /** The list stored on snippet.datasetRefs. Sorted + de-duped for stable diffs. */ export function recomputeDatasetRefs(spec: Json): string[] { return extractDatasetRefs(spec).sort(); } ``` > A `spec` may be an object or a string (see the Data Model). Normalize once, > at the boundary, so the recursive walk never has to care. **Do** - Treat `extractDatasetRefs` as the single source of truth for "what does this spec reference". The reverse-lookup and rename paths both depend on it agreeing with what the renderer actually resolves. - Recompute and store `datasetRefs` on **every draft change and on publish** — but only through the parse-gated, debounced auto-save (`commitDraft`) and the programmatic extract/revert rewrites, never on raw keystrokes. That keeps the links in step with the edited draft while never recomputing from a transiently-invalid spec. - Prune the **same two keys** (`data`, `datasets`) in all three ref walks — extraction here, `renameDatasetInSpec`, and the renderer's `resolveDatasetRefs` (`src/core/rendering.ts`). They must agree on what counts as a reference; if one descends into data payloads and another doesn't, extraction and rendering disagree and a row field named `data` either gets counted, rewritten, or throws `DatasetNotFoundError`. **Don't** - Don't let two code paths each have their own idea of "referenced names". Renamer, ref-recomputer, and renderer must use the same walk shape. - Don't "enumerate the grammar" (scope the walk to a fixed list of container keys) to fix the payload-descent problem — pruning the two data-bearing keys stays correct as Vega-Lite's composition grammar grows; an allow-list rots. --- ## 4. Reverse lookup: who uses this dataset? The Dataset Manager shows a **usage badge** and a **Linked Snippets** list; the Snippet Library shows a snippet's linked datasets. Both come from one scan — no stored back-pointer to drift. The scan itself is **pure** and lives in core, taking the snippets as a parameter so the _same_ implementation serves two callers: the reactive UI (which passes its live `snippets` selection straight in) and the non-reactive service wrapper (which passes a store snapshot for programmatic callers). This is what makes "who references this" have exactly one implementation (§6). ```ts // src/core/relationships.ts — pure scan (unit-tested) /** Snippets whose datasetRefs include `name` (case-insensitive). */ export function snippetsReferencingDataset( snippets: readonly T[], name: string, ): T[] { const lower = name.toLowerCase(); return snippets.filter((s) => s.datasetRefs.some((ref) => ref.toLowerCase() === lower)); } /** Usage counts for the whole library in one pass, keyed by lower-cased name. */ export function datasetUsageCounts( snippets: readonly { datasetRefs: readonly string[] }[], ): Map; ``` ```ts // src/app/services/RelationshipService.ts — snapshot wrapper for non-reactive callers import { snippetsReferencingDataset } from '../../core/relationships'; import { useSnippetStore } from '../stores/SnippetStore'; export function findSnippetsReferencingDataset(name: string): Snippet[] { return snippetsReferencingDataset(useSnippetStore.getState().snippets, name); } ``` The reactive UI does **not** go through the service: a component subscribed to `snippets` calls the core helper directly, so the badge and Linked Snippets list update the moment any snippet's draft changes its refs (auto-save) or it is published — no duplicated matching logic, no `getState()` snapshot that would miss updates. **Do** - Keep reverse lookup a pure scan, parameterized on the snippets so both the reactive and snapshot callers share it. It is O(snippets) but the collections are small (library budget ~5 MB); clarity beats an index. - Call the core helper directly from a store-subscribed component for reactivity; use the service wrapper only from non-reactive (snapshot) code. **Don't** - Don't add a `referencedBy` array to datasets. A stored reverse pointer is a second source of truth that _will_ fall out of sync with `datasetRefs`. --- ## 5. Import: auto-suffix collisions, then report On import we never overwrite an existing dataset. A dataset whose name collides is renamed to a unique name via `makeUniqueName`, and **every rename is collected and reported to the user** (toast / summary) so the change is never silent. Crucially, names are reserved _as we go_ — within a single import, two incoming `Sales` datasets become `Sales 2` and `Sales 3`, not two `Sales 2`. ```ts // src/core/import-normalize.ts import { makeUniqueName } from './naming'; import type { Dataset } from './dataset'; export interface DatasetRename { from: string; to: string; } /** * Returns incoming datasets with collision-free names, plus the renames applied. * `existing` are names already in the library; `incoming` are datasets to add. */ export function dedupeIncomingDatasetNames( existing: ReadonlyArray, incoming: ReadonlyArray, ): { datasets: Dataset[]; renames: DatasetRename[] } { const reserved = new Set(existing.map((n) => n.toLowerCase())); const renames: DatasetRename[] = []; const datasets = incoming.map((d) => { const unique = makeUniqueName(d.name, reserved); reserved.add(unique.toLowerCase()); // reserve so later imports don't collide if (unique !== d.name) renames.push({ from: d.name, to: unique }); return unique === d.name ? d : { ...d, name: unique }; }); return { datasets, renames }; } ``` > If imported snippets reference the renamed dataset, their `datasetRefs` and > specs must be rewritten to the new name too — reuse the rename machinery in > §6 over the imported snippet set, or run `renameDatasetEverywhere` per applied > rename after the import is committed. ### 5.1 Where the import/export flow lives, and its rules **Flow:** header (Import/Export buttons in `App.tsx`) → `services/transfer.ts` (the only store-touching layer) → pure core (`core/import-normalize.ts` shape detection + normalization + the dedupe/rename/id-reassign helpers; `core/export-envelope.ts`) - browser IO (`infrastructure/file-transfer.ts`). The pure helpers are unit-tested hardest; `transfer.ts` only orchestrates (read stores → call core → commit → notify). The behavioral contract is spec §08. Three rules a future change must keep: - **Datasets commit before snippets** (`DatasetStore.addDatasets` then `SnippetStore.addSnippets`) so a snippet's by-name reference resolves against the just-added (possibly suffixed) dataset. - **Imported datasets get fresh monotonic numeric ids** (`addDatasets`), not their envelope ids. Safe — and necessary — because datasets are linked **by name, not id** (§1): id reuse would collide in IndexedDB, but renaming the _id_ breaks nothing. (This is why the old `Date.now()`-collision TODO on `add` doesn't bite import.) - **Rename propagation reads the spec, not only `datasetRefs`.** `applyDatasetRenamesToSnippets` finds the referenced name via `extractDatasetRefs(spec)` ∪ `datasetRefs`, so an imported snippet whose `datasetRefs` is absent/stale (a hand-crafted or foreign file) still gets its spec rewritten — the renderer resolves by spec, so a missed rename would break it. **Do** - Reserve each chosen name immediately so collisions _within_ one import are also resolved. - Return the rename list and show it; a silent rename looks like data loss. **Don't** - Don't overwrite or merge a same-named existing dataset on import. Suffix and keep both — the user decides what to delete. --- ## 6. Rename propagation: keep the link consistent Renaming a dataset is the operation that ties §2–§5 together. A rename must, in one atomic step: 1. Update the dataset's own `name`. 2. For **every snippet referencing the old name**: rewrite the named-data references inside its spec (`{ "data": { "name": "old" } }` → `{ "data": { "name": "new" } }`) — in **both** `spec` and `draftSpec`. 3. Recompute that snippet's `datasetRefs` from the rewritten **draft** spec (the tracked surface), so the forward link mirrors reality and the reverse scan stays correct. The spec rewrite is pure; the orchestration reads and writes stores. ```ts // src/core/spec-refs.ts — pure rewrite /** Returns a copy of `spec` with every data.name === oldName replaced by newName. */ export function renameDatasetInSpec(spec: Json, oldName: string, newName: string): Json { const obj = typeof spec === 'string' ? safeParse(spec) : spec; const selfDefined = selfDefinedNames(obj); // never rename a spec's own inline dataset name const rewrite = (node: Json): Json => { if (Array.isArray(node)) return node.map(rewrite); if (node && typeof node === 'object') { const out: Record = {}; for (const [k, v] of Object.entries(node as Record)) { // A `data` object is a reference site: rename a matching name, but never // recurse into its payload. `datasets` (self-defined inline data) is left // whole. Same prune as extractDatasetRefs — see §3.1. if (k === 'data' && v && typeof v === 'object' && !Array.isArray(v)) { const dv = v as Record; out[k] = dv.name === oldName && !selfDefined.has(oldName) ? { ...dv, name: newName } : dv; } else if (k === 'data' || k === 'datasets') { out[k] = v; } else { out[k] = rewrite(v); } } return out; } return node; }; const rewritten = rewrite(obj); // Preserve the original spec's stored shape (string vs object). return typeof spec === 'string' ? JSON.stringify(rewritten, null, 2) : rewritten; } ``` ```ts // src/app/services/RelationshipService.ts — thin store coordinator import { useDatasetStore } from '../stores/DatasetStore'; import { useSnippetStore } from '../stores/SnippetStore'; /** * Renames a dataset and propagates the rename to every referencing snippet * (spec, draftSpec, and datasetRefs). Returns the number of snippets changed. * Caller is responsible for collision policy on `newName` (reject vs suffix). */ export function renameDatasetEverywhere( oldName: string, newName: string, now?: Date, ): { updated: number } { if (oldName === newName) return { updated: 0 }; // 1. Rename the dataset record itself. const dataset = useDatasetStore.getState().datasets.find((d) => d.name === oldName); if (dataset) useDatasetStore.getState().update(dataset.id, { name: newName }, now); // 2 + 3. Delegate the spec/draftSpec/refs rewrite to the ONE rename impl — the // store action that owns the snippet collection. It matches by spec+draft // *content* (not `datasetRefs`, which mirrors only the draft) and recomputes // `datasetRefs` from the rewritten draft. Keeping the loop in the store means // the reactive editor buffer is refreshed in the same atomic update when the // active snippet's draft is rewritten. return { updated: useSnippetStore.getState().renameDatasetRefs(oldName, newName, now) }; } ``` > **Collision on rename.** The UI rename form rejects a name already in use via > `isNameTaken(newName, datasets, dataset.id)`. Programmatic renames (e.g. an > import flow) instead resolve with `makeUniqueName` before calling > `renameDatasetEverywhere`. The propagation function itself does not invent a > name — it assumes `newName` is the agreed target. **Do** - Rewrite `spec` **and** `draftSpec`. A user mid-edit must not see their draft silently break because the dataset was renamed underneath them. - Recompute `datasetRefs` from the rewritten **draft** spec rather than string-replacing the array — the draft is the source of truth, the array is its mirror. - Find affected snippets by scanning each one's **spec and draft content** for the old name (`extractDatasetRefs`), not by its `datasetRefs` array. Because `datasetRefs` mirrors the draft, a name referenced only by the still-published spec (the user removed it from the draft but hasn't published) is absent from the array; matching on content rewrites it anyway, so the published spec can't rot to a renamed-away dataset. Scan the content (don't reserialize) so a non-referencing snippet's text stays byte-for-byte intact. The import-side `applyDatasetRenamesToSnippets` already follows this spec-content rule (§5.1). **Don't** - Don't update `datasetRefs` without also rewriting the spec — the rendered named-data reference would still point at the old, now-missing name. - Don't rename the dataset and skip propagation "for now". A half-applied rename is the exact inconsistency this whole document exists to prevent. --- ## 7. Where things live | Concern | Location | Pure? | Tested | | ------------------------------------------------------------------------ | ----------------------------------------- | ------------------- | ----------- | | `makeUniqueName`, `isNameTaken` | `src/core/naming.ts` | yes | unit | | `extractDatasetRefs`, `recomputeDatasetRefs` | `src/core/spec-refs.ts` | yes | unit | | `renameDatasetInSpec` | `src/core/spec-refs.ts` | yes | unit | | `snippetsReferencingDataset`, `datasetUsageCounts` (reverse-lookup scan) | `src/core/relationships.ts` | yes | unit | | `findSnippetsReferencingDataset`, usage count (snapshot wrappers) | `src/app/services/RelationshipService.ts` | no (reads store) | integration | | `renameDatasetEverywhere` → `{ updated }` | `src/app/services/RelationshipService.ts` | no (mutates stores) | integration | | `dedupeIncomingDatasetNames` | `src/core/import-normalize.ts` | yes | unit | The dividing line: anything that takes plain data and returns plain data is **core** and unit-tested in isolation; anything that reaches into a Zustand store is an **app service**. The rule of thumb — _the **draft** spec is the source of truth, `datasetRefs` mirrors it, the reverse lookup is derived_ — is what keeps the bidirectional link from ever needing manual repair.