astrolabe/docs/architecture/07-naming-and-relationships.md

Naming & Relationships

How Astrolabe keeps entity names unique within a collection, and how it tracks the bidirectional links between snippets and datasets so they stay consistent as entities are created, imported, and renamed.

Two concerns live here, and they reinforce each other:

1. Name uniqueness — every dataset has a unique name. Names are the primary key users see and the key snippets reference, so duplicates would be ambiguous. We reject duplicate names on create/rename, and auto-suffix collisions during bulk import.
2. Relationship tracking — a snippet references datasets by name through its datasetRefs: string[] field. This is a bidirectional, name-based link: from a snippet you read its refs; from a dataset you scan snippets to find who uses it. Renaming a dataset must propagate to every snippet that points at it, in both the spec and the datasetRefs array, or the links rot.

The hard, testable logic is pure and lives in src/core/. The parts that read and mutate stores live in src/app/services/.

---

1. Why names, not IDs, are the link

Datasets carry a numeric id, but snippets reference them by name because that is what Vega-Lite uses: a spec resolves data through a named-data reference, { "data": { "name": "MyDataset" } }. The name is the contract between a spec and the dataset library. Storing a numeric id in the spec would mean the spec is no longer a standalone, paste-anywhere Vega-Lite document.

The consequence: names must be unique (two datasets named Sales would make { "data": { "name": "Sales" } } ambiguous), and renaming a dataset is a graph operation, not a single field write — every reference to the old name must move with it.

---

2. Name uniqueness (pure — src/core/naming.ts)

2.1 Uniqueness check

Comparisons are case-insensitive (Sales and sales collide), so a single display name maps to a single dataset regardless of how a user types a reference. The check takes an optional excludeId so a rename can ignore the record being renamed (renaming Sales to Sales is not a collision with itself).

// src/core/naming.ts

/** Case-insensitive set of names already in use, minus an optional excluded id. */
export function isNameTaken(
  desired: string,
  datasets: ReadonlyArray<{ id: number; name: string }>,
  excludeId?: number,
): boolean {
  const lower = desired.trim().toLowerCase();
  return datasets.some((d) => d.id !== excludeId && d.name.toLowerCase() === lower);
}

2.2 Making a unique name

When a desired name is taken — during import, "extract inline data", or "build chart" — we do not overwrite the existing dataset. We derive the next free name by appending a numeric suffix: Name → Name 2 → Name 3. The function takes the set of existing names so it has no store dependency and is trivially unit-testable.

// src/core/naming.ts

/**
 * Returns `desired` if free, else the first available `${desired} ${n}` (n >= 2).
 * `existingNames` is the set of names already in the collection.
 * Comparison is case-insensitive; the returned name preserves `desired`'s casing.
 */
export function makeUniqueName(desired: string, existingNames: Iterable<string>): string {
  const taken = new Set<string>();
  for (const n of existingNames) taken.add(n.toLowerCase());

  const base = desired.trim();
  if (!taken.has(base.toLowerCase())) return base;

  let n = 2;
  while (taken.has(`${base} ${n}`.toLowerCase())) n++;
  return `${base} ${n}`;
}

If a base name already ends in a number (Q1 2024), the suffix still appends (Q1 2024 2). That is intentional: we never parse meaning out of the name, we only guarantee a free slot. Keep this dumb and predictable.

Do

• Use isNameTaken to reject duplicate create/rename in the UI before saving, and surface an error toast.
• Use makeUniqueName for every non-interactive path (import, extract, build) where blocking the user would be worse than a silent, reported rename.
• Pass excludeId on rename so an unchanged or case-only edit is allowed.

Don't

• Don't compare names case-sensitively anywhere — pick toLowerCase() once and use it consistently.
• Don't let makeUniqueName mutate a store or read store state; it takes plain data and returns a string.

---

3. The bidirectional snippet ↔ dataset link

            datasetRefs: ["Sales", "Regions"]          (forward, on the snippet)
   Snippet ───────────────────────────────────────────────────►  Dataset "Sales"
        ▲                                                                 │
        └──────────── scan all snippets for "Sales" in datasetRefs ◄──────┘
                                  (reverse, derived)

• Forward (snippet → datasets): read snippet.datasetRefs. Cheap, stored.
• Reverse (dataset → snippets): there is no stored back-pointer. We compute it by scanning snippets. Keeping it derived means it can never disagree with the forward links — there is one source of truth.

datasetRefs is derived from the spec, not hand-maintained. It mirrors the dataset names referenced by the draft spec — the version being edited — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft (not only the last publish) keeps a snippet's linked-datasets display and the reverse lookup in step with what the editor shows — so a hand-typed reference or an Extract links its dataset without waiting for a publish. Recomputation runs only on a valid spec — auto-save is debounced and parse-gated (spec §03B) — so a transiently-invalid draft never disturbs the links.

3.1 What counts as a reference, and extracting them (pure — src/core/spec-data.ts, spec-refs.ts)

A library reference is exactly Vega-Lite named data: a data block with a string name and no values, url, or generator key (sequence/sphere/ graticule), whose name the spec does not define for itself via a top-level datasets map. A name riding on inline values or a url is Vega-Lite's runtime-rebind label — not a dependency — and self-defined datasets names resolve natively; both are left untouched. This mirrors Vega-Lite's own isNamedData, so Astrolabe extends Vega-Lite rather than diverging: every native data form keeps working, and only true references are tracked and resolved.

That classification lives in core/spec-data (classifyData, libraryRefName) — the single predicate that reference extraction, rename (spec-refs), and render-time resolution (rendering) all route through, so they cannot disagree on what is a dependency.

// src/core/spec-data.ts — the shared classifier (mirrors vega-lite/src/data.ts)

/** The library name a `data` block references, or null for native VL data / self-defined names. */
export function libraryRefName(data: unknown, selfDefined: ReadonlySet<string>): string | null {
  if (classifyData(data) !== 'named') return null; // url | values | generator → not a reference
  const { name } = data as { name: string };
  return selfDefined.has(name) ? null : name;
}

References appear in several places — top-level data, per-layer data, data inside spec/facet/hconcat/vconcat, and a lookup transform's from.data. Rather than enumerate the grammar, the walk recurses the spec but prunes two keys: a data object's payload (its values/rows) and the top-level datasets map hold user data, not nested specs. Without the prune, a data row carrying a field literally named data: { name: "x" } is misread as a reference.

That walk is one shared pair in core/spec-data, not re-implemented per pass: forEachDataBinding(spec, visit) (read-only, visit returns true to stop early) and mapDataBindings(spec, mapData) (returns a copy). Both compute the spec's self-defined names once and hand them to the callback, so a caller writes only its own rule — what is a reference (libraryRefName) or what to rewrite — never the walk. extractDatasetRefs collects through forEachDataBinding; renameDatasetInSpec and promoteSelfDefinedDataset (§3.2) rewrite through mapDataBindings.

recomputeDatasetRefs is the thin sorted/de-duped wrapper stored on snippet.datasetRefs, run on every draft change and on publish.

A spec may be an object or a string (see the Data Model). Normalize once, at the boundary, so the recursive walk never has to care.

Do

• Treat extractDatasetRefs as the single source of truth for "what does this spec reference". The reverse-lookup and rename paths both depend on it agreeing with what the renderer actually resolves — which holds because all of them classify through the same core/spec-data predicate.
• Recompute and store datasetRefs on every draft change and on publish — but only through the parse-gated, debounced auto-save (commitDraft) and the programmatic extract/revert rewrites, never on raw keystrokes. That keeps the links in step with the edited draft while never recomputing from a transiently-invalid spec.
• Route every binding pass through the shared forEachDataBinding / mapDataBindings. They share both halves that must agree — the libraryRefName classifier (what is a reference) and the prune of the same two keys (data, datasets) — so extraction, rename, and the reverse extraction cannot drift. If one classified differently, or descended into data payloads while another didn't, a row field named data would get counted, rewritten, or throw DatasetNotFoundError. (The renderer's resolveDatasetRefs mutates in place and can throw mid-walk, so it keeps its own copy of the walk — the one exception, held in step by the same prune-two-keys rule.)

Don't

• Don't let two code paths each have their own idea of "referenced names". Renamer, ref-recomputer, and renderer must use the same walk shape.
• Don't add a fresh prune-walk for a new binding pass — reuse the shared pair. A hand-written copy is one classifier tweak away from disagreeing with the others.
• Don't "enumerate the grammar" (scope the walk to a fixed list of container keys) to fix the payload-descent problem — pruning the two data-bearing keys stays correct as Vega-Lite's composition grammar grows; an allow-list rots.

3.2 Extracting embedded data into a dataset — the reverse (spec-inline-data.ts, spec-refs.ts)

Extract-to-Dataset is the inverse of a reference: it lifts a view's embedded data into a stored dataset and rewrites the spec to reference it by name. It is a cursor-scoped editor action — services/extract-action resolves the focused view's binding (dataBindingAtPath), seeds the modal, then opens it; the gate specHasExtractableData hides the toolbar action when no view carries liftable data. Two embedded shapes lift, both routing through the same spec-data classifier so they never disagree with reference detection:

• Inline values — inlineValuesOf captures the payload verbatim (a CSV/TSV string is kept as-is); confirm rewrites that view's data block, at its anchor path, to { name }.
• A self-defined datasets entry the view references — selfDefinedPayloadOf reads the named rows and promoteSelfDefinedDataset drops the datasets entry (and the map when it empties). Keeping the name needs no reference rewrite — it un-shadows onto the new library dataset; renaming rewrites every matching reference. This is the reverse direction of the self-defined-vs-library precedence in §3.1.

A lookup transform's inline from.data lifts like any view binding — dataBindingAtPath finds it. A url, a generator, or an existing library reference carries nothing to lift.

---

4. Reverse lookup: who uses this dataset?

The Dataset Manager shows a usage badge and a Linked Snippets list; the Snippet Library shows a snippet's linked datasets. Both come from one scan — no stored back-pointer to drift.

The scan itself is pure and lives in core, taking the snippets as a parameter so the same implementation serves two callers: the reactive UI (which passes its live snippets selection straight in) and the non-reactive service wrapper (which passes a store snapshot for programmatic callers). This is what makes "who references this" have exactly one implementation (§6).

// src/core/relationships.ts — pure scan (unit-tested)

/** Snippets whose datasetRefs include `name` (case-insensitive). */
export function snippetsReferencingDataset<T extends { datasetRefs: readonly string[] }>(
  snippets: readonly T[],
  name: string,
): T[] {
  const lower = name.toLowerCase();
  return snippets.filter((s) => s.datasetRefs.some((ref) => ref.toLowerCase() === lower));
}

/** Usage counts for the whole library in one pass, keyed by lower-cased name. */
export function datasetUsageCounts(
  snippets: readonly { datasetRefs: readonly string[] }[],
): Map<string, number>;

There is no app-layer wrapper module around the scan. The reactive UI — a component subscribed to snippets — calls the core helper directly, so the badge and Linked Snippets list update the moment any snippet's draft changes its refs (auto-save) or it is published. A non-reactive caller binds the same helper to a snapshot inline at its call site (snippetsReferencingDataset(useSnippetStore.getState().snippets, name)) — one matching implementation, no getState() wrapper that would go stale in reactive code.

Do

• Keep reverse lookup a pure scan, parameterized on the snippets so both the reactive and snapshot callers share it. It is O(snippets) but the collections are small (library budget ~5 MB); clarity beats an index.
• Call the core helper directly from a store-subscribed component for reactivity; bind it to a getState() snapshot inline for non-reactive code.

Don't

• Don't add a referencedBy array to datasets. A stored reverse pointer is a second source of truth that will fall out of sync with datasetRefs.

---

5. Import: auto-suffix collisions, then report

On import we never overwrite an existing record. A record whose name collides is renamed to a unique name via makeUniqueName, and every rename is collected and reported to the user (toast / summary) so the change is never silent. Crucially, names are reserved as we go — within a single import, two incoming Sales datasets become Sales 2 and Sales 3, not two Sales 2. The helper is generic over { name: string } because two record kinds key on a unique name: datasets and custom chart themes (only dataset renames need propagation — nothing references a theme by name).

// src/core/import-normalize.ts

import { makeUniqueName } from './naming';

export interface NameRename {
  from: string;
  to: string;
}

/**
 * Returns incoming records with collision-free names, plus the renames applied.
 * `existing` are names already in the collection; `incoming` are records to add.
 */
export function dedupeIncomingNames<T extends { name: string }>(
  existing: ReadonlyArray<string>,
  incoming: ReadonlyArray<T>,
): { records: T[]; renames: NameRename[] } {
  const reserved = new Set(existing.map((n) => n.toLowerCase()));
  const renames: NameRename[] = [];

  const records = incoming.map((r) => {
    const unique = makeUniqueName(r.name, reserved);
    reserved.add(unique.toLowerCase()); // reserve so later imports don't collide
    if (unique !== r.name) renames.push({ from: r.name, to: unique });
    return unique === r.name ? r : { ...r, name: unique };
  });

  return { records, renames };
}

If imported snippets reference the renamed dataset, their datasetRefs and specs must be rewritten to the new name too. The import flow does this purely, before anything is committed: applyDatasetRenamesToSnippets (core/import-normalize) rewrites the incoming snippet set per applied rename using the same renameDatasetInSpec machinery as §6.

5.1 Where the import/export flow lives, and its rules

Flow: header (Import/Export buttons in App.tsx) → services/transfer.ts (the only store-touching layer) → pure core (core/import-normalize.ts shape detection + normalization + the dedupe/rename/id-reassign helpers; core/export-envelope.ts)

• browser IO (infrastructure/file-transfer.ts). The pure helpers are unit-tested hardest; transfer.ts only orchestrates (read stores → call core → commit → notify). The behavioral contract is spec §08.

Three rules a future change must keep:

• Datasets commit before snippets (DatasetStore.addDatasets then SnippetStore.addSnippets) so a snippet's by-name reference resolves against the just-added (possibly suffixed) dataset.
• Imported datasets get fresh monotonic numeric ids (addDatasets), not their envelope ids. Safe — and necessary — because datasets are linked by name, not id (§1): id reuse would collide in IndexedDB, but renaming the id breaks nothing. (This is why the old Date.now()-collision TODO on add doesn't bite import.)
• Rename propagation reads the spec, not only datasetRefs. applyDatasetRenamesToSnippets finds the referenced name via extractDatasetRefs(spec) ∪ datasetRefs, so an imported snippet whose datasetRefs is absent/stale (a hand-crafted or foreign file) still gets its spec rewritten — the renderer resolves by spec, so a missed rename would break it.

Do

• Reserve each chosen name immediately so collisions within one import are also resolved.
• Return the rename list and show it; a silent rename looks like data loss.

Don't

• Don't overwrite or merge a same-named existing dataset on import. Suffix and keep both — the user decides what to delete.

---

6. Rename propagation: keep the link consistent

Renaming a dataset is the operation that ties §2–§5 together. A rename must, in one atomic step:

1. Update the dataset's own name.
2. For every snippet referencing the old name: rewrite the named-data references inside its spec ({ "data": { "name": "old" } } → { "data": { "name": "new" } }) — in both spec and draftSpec.
3. Recompute that snippet's datasetRefs from the rewritten draft spec (the tracked surface), so the forward link mirrors reality and the reverse scan stays correct.

The spec rewrite is pure; the orchestration reads and writes stores.

// src/core/spec-refs.ts — pure rewrite

/** Returns a copy of `spec` with every data.name === oldName replaced by newName. */
export function renameDatasetInSpec(spec: Json, oldName: string, newName: string): Json {
  const obj = typeof spec === 'string' ? safeParse(spec) : spec;
  const selfDefined = selfDefinedNames(obj); // never rename a spec's own inline dataset name

  const rewrite = (node: Json): Json => {
    if (Array.isArray(node)) return node.map(rewrite);
    if (node && typeof node === 'object') {
      const out: Record<string, Json> = {};
      for (const [k, v] of Object.entries(node as Record<string, Json>)) {
        // A `data` object is a reference site: rename a matching name, but never
        // recurse into its payload. `datasets` (self-defined inline data) is left
        // whole. Same prune as extractDatasetRefs — see §3.1.
        if (k === 'data' && v && typeof v === 'object' && !Array.isArray(v)) {
          const dv = v as Record<string, Json>;
          out[k] = dv.name === oldName && !selfDefined.has(oldName) ? { ...dv, name: newName } : dv;
        } else if (k === 'data' || k === 'datasets') {
          out[k] = v;
        } else {
          out[k] = rewrite(v);
        }
      }
      return out;
    }
    return node;
  };

  const rewritten = rewrite(obj);
  // Preserve the original spec's stored shape (string vs object).
  return typeof spec === 'string' ? JSON.stringify(rewritten, null, 2) : rewritten;
}

The one rename implementation in the live app is the store action SnippetStore.renameDatasetRefs(oldName, newName, now) — the action that owns the snippet collection. It matches by spec+draft content (not datasetRefs, which mirrors only the draft), rewrites spec/draftSpec via renameDatasetInSpec, recomputes datasetRefs from the rewritten draft, and returns the number of snippets changed. Keeping the loop in the store means the reactive editor buffer is refreshed in the same atomic update when the active snippet's draft is rewritten. DatasetStore.update calls it whenever a save changes a dataset's name, so a rename propagates everywhere as part of the one user action — there is no separate coordinator module to call.

Collision on rename. The UI rename form rejects a name already in use via isNameTaken(newName, datasets, dataset.id). Programmatic renames (e.g. an import flow) instead resolve with makeUniqueName first. The propagation action itself does not invent a name — it assumes newName is the agreed target.

Do

• Rewrite spec and draftSpec. A user mid-edit must not see their draft silently break because the dataset was renamed underneath them.
• Recompute datasetRefs from the rewritten draft spec rather than string-replacing the array — the draft is the source of truth, the array is its mirror.
• Find affected snippets by scanning each one's spec and draft content for the old name (extractDatasetRefs), not by its datasetRefs array. Because datasetRefs mirrors the draft, a name referenced only by the still-published spec (the user removed it from the draft but hasn't published) is absent from the array; matching on content rewrites it anyway, so the published spec can't rot to a renamed-away dataset. Scan the content (don't reserialize) so a non-referencing snippet's text stays byte-for-byte intact. The import-side applyDatasetRenamesToSnippets already follows this spec-content rule (§5.1).

Don't

• Don't update datasetRefs without also rewriting the spec — the rendered named-data reference would still point at the old, now-missing name.
• Don't rename the dataset and skip propagation "for now". A half-applied rename is the exact inconsistency this whole document exists to prevent.

---

7. Snippet name provenance — the naming hierarchy

Snippet names (unlike dataset names) need no uniqueness; what they need is a rule for who may rewrite them. Each snippet carries nameSource (spec §09A): 'user' names are frozen — set by an explicit rename (SnippetStore.renameSnippet) and never touched by the app again; 'auto' names are app-picked and keep tracking the spec. On publish, an auto-named snippet is re-named from the now-published content in priority order: the spec's title (string, line array, or { text } forms), else a mark + encodings description, else the existing name stands. The derivation dialect is deliberately the same one generateChartName uses for builder output, so manually authored and builder-built snippets read alike in the library.

Flow: core/snippet.ts (deriveSnippetName, isAutoNamed, isDefaultSnippetName) → SnippetStore.publish (the only rewrite site) / renameSnippet (the freeze site) → SnippetLibrary's metadata panel (which must adopt a publish rename — arch 01 §2, editing buffers). Records predating nameSource have no provenance; isAutoNamed treats them as user-named unless the name is provably app-picked — the timestamp default shape, or identical to what deriveSnippetName returns for the record's own published spec. The conservative default is deliberate: rewriting a chosen name is worse than failing to track an auto one.

---

8. Where things live

Concern	Location	Pure?	Tested
makeUniqueName, isNameTaken	src/core/naming.ts	yes	unit
extractDatasetRefs, recomputeDatasetRefs	src/core/spec-refs.ts	yes	unit
renameDatasetInSpec	src/core/spec-refs.ts	yes	unit
snippetsReferencingDataset, datasetUsageCounts (reverse-lookup scan)	src/core/relationships.ts	yes	unit
renameDatasetRefs → updated count (rename propagation)	src/app/stores/SnippetStore.ts	no (mutates stores)	integration
dedupeIncomingNames (datasets + custom themes)	src/core/import-normalize.ts	yes	unit
deriveSnippetName, isAutoNamed (snippet name provenance)	src/core/snippet.ts	yes	unit

The dividing line: anything that takes plain data and returns plain data is core and unit-tested in isolation; anything that reaches into a Zustand store is a store action or app service. The rule of thumb — the draft spec is the source of truth, datasetRefs mirrors it, the reverse lookup is derived — is what keeps the bidirectional link from ever needing manual repair.