astrolabe/docs/architecture/07-naming-and-relationships.md

Naming & Relationships

How Astrolabe keeps entity names unique within a collection, and how it tracks the bidirectional links between snippets and datasets so they stay consistent as entities are created, imported, and renamed.

Two concerns live here, and they reinforce each other:

1. Name uniqueness — every dataset has a unique name. Names are the primary key users see and the key snippets reference, so duplicates would be ambiguous. We reject duplicate names on create/rename, and auto-suffix collisions during bulk import.
2. Relationship tracking — a snippet references datasets by name through its datasetRefs: string[] field. This is a bidirectional, name-based link: from a snippet you read its refs; from a dataset you scan snippets to find who uses it. Renaming a dataset must propagate to every snippet that points at it, in both the spec and the datasetRefs array, or the links rot.

The hard, testable logic is pure and lives in src/core/. The parts that read and mutate stores live in src/app/services/.

---

1. Why names, not IDs, are the link

Datasets carry a numeric id, but snippets reference them by name because that is what Vega-Lite uses: a spec resolves data through a named-data reference, { "data": { "name": "MyDataset" } }. The name is the contract between a spec and the dataset library. Storing a numeric id in the spec would mean the spec is no longer a standalone, paste-anywhere Vega-Lite document.

The consequence: names must be unique (two datasets named Sales would make { "data": { "name": "Sales" } } ambiguous), and renaming a dataset is a graph operation, not a single field write — every reference to the old name must move with it.

---

2. Name uniqueness (pure — src/core/naming.ts)

2.1 Uniqueness check

Comparisons are case-insensitive (Sales and sales collide), so a single display name maps to a single dataset regardless of how a user types a reference. The check takes an optional excludeId so a rename can ignore the record being renamed (renaming Sales to Sales is not a collision with itself).

// src/core/naming.ts

/** Case-insensitive set of names already in use, minus an optional excluded id. */
export function isNameTaken(
  desired: string,
  datasets: ReadonlyArray<{ id: number; name: string }>,
  excludeId?: number,
): boolean {
  const lower = desired.trim().toLowerCase();
  return datasets.some((d) => d.id !== excludeId && d.name.toLowerCase() === lower);
}

2.2 Making a unique name

When a desired name is taken — during import, "extract inline data", or "build chart" — we do not overwrite the existing dataset. We derive the next free name by appending a numeric suffix: Name → Name 2 → Name 3. The function takes the set of existing names so it has no store dependency and is trivially unit-testable.

// src/core/naming.ts

/**
 * Returns `desired` if free, else the first available `${desired} ${n}` (n >= 2).
 * `existingNames` is the set of names already in the collection.
 * Comparison is case-insensitive; the returned name preserves `desired`'s casing.
 */
export function makeUniqueName(desired: string, existingNames: Iterable<string>): string {
  const taken = new Set<string>();
  for (const n of existingNames) taken.add(n.toLowerCase());

  const base = desired.trim();
  if (!taken.has(base.toLowerCase())) return base;

  let n = 2;
  while (taken.has(`${base} ${n}`.toLowerCase())) n++;
  return `${base} ${n}`;
}

If a base name already ends in a number (Q1 2024), the suffix still appends (Q1 2024 2). That is intentional: we never parse meaning out of the name, we only guarantee a free slot. Keep this dumb and predictable.

Do

• Use isNameTaken to reject duplicate create/rename in the UI before saving, and surface an error toast.
• Use makeUniqueName for every non-interactive path (import, extract, build) where blocking the user would be worse than a silent, reported rename.
• Pass excludeId on rename so an unchanged or case-only edit is allowed.

Don't

• Don't compare names case-sensitively anywhere — pick toLowerCase() once and use it consistently.
• Don't let makeUniqueName mutate a store or read store state; it takes plain data and returns a string.

---

3. The bidirectional snippet ↔ dataset link

            datasetRefs: ["Sales", "Regions"]          (forward, on the snippet)
   Snippet ───────────────────────────────────────────────────►  Dataset "Sales"
        ▲                                                                 │
        └──────────── scan all snippets for "Sales" in datasetRefs ◄──────┘
                                  (reverse, derived)

• Forward (snippet → datasets): read snippet.datasetRefs. Cheap, stored.
• Reverse (dataset → snippets): there is no stored back-pointer. We compute it by scanning snippets. Keeping it derived means it can never disagree with the forward links — there is one source of truth.

datasetRefs is derived from the spec, not hand-maintained. It is recomputed whenever a snippet is published (its draft spec is promoted), so it always mirrors the dataset names actually referenced in the published spec.

3.1 Extracting referenced names from a spec (pure — src/core/spec-refs.ts)

A Vega-Lite spec can reference named data in several places: the top-level data, per-layer data, data inside spec/facet/hconcat/vconcat, and a lookup transform's from.data. A spec may also define its OWN inline datasets via a top-level datasets map — those are self-defined, not library references. Rather than enumerate Vega-Lite's grammar, we walk the spec recursively and collect every { data: { name } } — but prune two keys: never recurse into a data object's payload (its values/rows) or the top-level datasets map, because those hold user data, not nested specs. Without the prune, a data row carrying a field literally named data: { name: "x" } is misread as a reference. This is pure, deterministic, and the most heavily unit-tested function here.

// src/core/spec-refs.ts

type Json = unknown;

/** Collects every library dataset name referenced by `{ data: { name } }`, excluding self-defined ones. */
export function extractDatasetRefs(spec: Json): string[] {
  const root = typeof spec === 'string' ? safeParse(spec) : spec;
  const selfDefined = selfDefinedNames(root); // names from the spec's own top-level `datasets`
  const names = new Set<string>();

  const walk = (node: Json): void => {
    if (Array.isArray(node)) {
      for (const item of node) walk(item);
      return;
    }
    if (node && typeof node === 'object') {
      const obj = node as Record<string, Json>;
      const data = obj.data as Record<string, Json> | undefined;
      if (data && typeof data === 'object' && typeof data.name === 'string') {
        if (!selfDefined.has(data.name)) names.add(data.name);
      }
      // Prune: a `data` payload and the `datasets` map hold user data, not refs.
      for (const key of Object.keys(obj)) {
        if (key === 'data' || key === 'datasets') continue;
        walk(obj[key]);
      }
    }
  };

  walk(root);
  return [...names];
}

function safeParse(s: string): Json {
  try {
    return JSON.parse(s);
  } catch {
    return null; // an unparseable draft simply has no resolvable refs
  }
}

// src/core/spec-refs.ts — thin wrapper used at publish time

/** The list stored on snippet.datasetRefs. Sorted + de-duped for stable diffs. */
export function recomputeDatasetRefs(spec: Json): string[] {
  return extractDatasetRefs(spec).sort();
}

A spec may be an object or a string (see the Data Model). Normalize once, at the boundary, so the recursive walk never has to care.

Do

• Treat extractDatasetRefs as the single source of truth for "what does this spec reference". The reverse-lookup and rename paths both depend on it agreeing with what the renderer actually resolves.
• Recompute and store datasetRefs on publish, not on every keystroke — the draft can be transiently invalid, and only the published spec is shared.
• Prune the same two keys (data, datasets) in all three ref walks — extraction here, renameDatasetInSpec, and the renderer's resolveDatasetRefs (src/core/rendering.ts). They must agree on what counts as a reference; if one descends into data payloads and another doesn't, extraction and rendering disagree and a row field named data either gets counted, rewritten, or throws DatasetNotFoundError.

Don't

• Don't let two code paths each have their own idea of "referenced names". Renamer, ref-recomputer, and renderer must use the same walk shape.
• Don't "enumerate the grammar" (scope the walk to a fixed list of container keys) to fix the payload-descent problem — pruning the two data-bearing keys stays correct as Vega-Lite's composition grammar grows; an allow-list rots.

---

4. Reverse lookup: who uses this dataset?

The Dataset Manager shows a usage badge and a Linked Snippets list; the Snippet Library shows a snippet's linked datasets. Both come from one scan — no stored back-pointer to drift.

The scan itself is pure and lives in core, taking the snippets as a parameter so the same implementation serves two callers: the reactive UI (which passes its live snippets selection straight in) and the non-reactive service wrapper (which passes a store snapshot for programmatic callers). This is what makes "who references this" have exactly one implementation (§6).

// src/core/relationships.ts — pure scan (unit-tested)

/** Snippets whose datasetRefs include `name` (case-insensitive). */
export function snippetsReferencingDataset<T extends { datasetRefs: readonly string[] }>(
  snippets: readonly T[],
  name: string,
): T[] {
  const lower = name.toLowerCase();
  return snippets.filter((s) => s.datasetRefs.some((ref) => ref.toLowerCase() === lower));
}

/** Usage counts for the whole library in one pass, keyed by lower-cased name. */
export function datasetUsageCounts(
  snippets: readonly { datasetRefs: readonly string[] }[],
): Map<string, number>;

// src/app/services/RelationshipService.ts — snapshot wrapper for non-reactive callers

import { snippetsReferencingDataset } from '../../core/relationships';
import { useSnippetStore } from '../stores/SnippetStore';

export function findSnippetsReferencingDataset(name: string): Snippet[] {
  return snippetsReferencingDataset(useSnippetStore.getState().snippets, name);
}

The reactive UI does not go through the service: a component subscribed to snippets calls the core helper directly, so the badge and Linked Snippets list update the moment any snippet is published with changed refs — no duplicated matching logic, no getState() snapshot that would miss updates.

Do

• Keep reverse lookup a pure scan, parameterized on the snippets so both the reactive and snapshot callers share it. It is O(snippets) but the collections are small (library budget ~5 MB); clarity beats an index.
• Call the core helper directly from a store-subscribed component for reactivity; use the service wrapper only from non-reactive (snapshot) code.

Don't

• Don't add a referencedBy array to datasets. A stored reverse pointer is a second source of truth that will fall out of sync with datasetRefs.

---

5. Import: auto-suffix collisions, then report

On import we never overwrite an existing dataset. A dataset whose name collides is renamed to a unique name via makeUniqueName, and every rename is collected and reported to the user (toast / summary) so the change is never silent. Crucially, names are reserved as we go — within a single import, two incoming Sales datasets become Sales 2 and Sales 3, not two Sales 2.

// src/app/services/ImportService.ts

import { makeUniqueName } from '../../core/naming';
import type { Dataset } from '../../core/types';

export interface DatasetRename {
  from: string;
  to: string;
}

/**
 * Returns incoming datasets with collision-free names, plus the renames applied.
 * `existing` are names already in the library; `incoming` are datasets to add.
 */
export function dedupeIncomingDatasetNames(
  existing: ReadonlyArray<string>,
  incoming: ReadonlyArray<Dataset>,
): { datasets: Dataset[]; renames: DatasetRename[] } {
  const reserved = new Set(existing.map((n) => n.toLowerCase()));
  const renames: DatasetRename[] = [];

  const datasets = incoming.map((d) => {
    const unique = makeUniqueName(d.name, reserved);
    reserved.add(unique.toLowerCase()); // reserve so later imports don't collide
    if (unique !== d.name) renames.push({ from: d.name, to: unique });
    return unique === d.name ? d : { ...d, name: unique };
  });

  return { datasets, renames };
}

If imported snippets reference the renamed dataset, their datasetRefs and specs must be rewritten to the new name too — reuse the rename machinery in §6 over the imported snippet set, or run renameDatasetEverywhere per applied rename after the import is committed.

5.1 Where the import/export flow lives, and its rules

Flow: header (Import/Export buttons in App.tsx) → services/transfer.ts (the only store-touching layer) → pure core (core/import-normalize.ts shape detection + normalization + the dedupe/rename/id-reassign helpers; core/export-envelope.ts)

• browser IO (infrastructure/file-transfer.ts). The pure helpers are unit-tested hardest; transfer.ts only orchestrates (read stores → call core → commit → notify). The behavioral contract is spec §08.

Three rules a future change must keep:

• Datasets commit before snippets (DatasetStore.addDatasets then SnippetStore.addSnippets) so a snippet's by-name reference resolves against the just-added (possibly suffixed) dataset.
• Imported datasets get fresh monotonic numeric ids (addDatasets), not their envelope ids. Safe — and necessary — because datasets are linked by name, not id (§1): id reuse would collide in IndexedDB, but renaming the id breaks nothing. (This is why the old Date.now()-collision TODO on add doesn't bite import.)
• Rename propagation reads the spec, not only datasetRefs. applyDatasetRenamesToSnippets finds the referenced name via extractDatasetRefs(spec) ∪ datasetRefs, so an imported snippet whose datasetRefs is absent/stale (a hand-crafted or foreign file) still gets its spec rewritten — the renderer resolves by spec, so a missed rename would break it.

Do

• Reserve each chosen name immediately so collisions within one import are also resolved.
• Return the rename list and show it; a silent rename looks like data loss.

Don't

• Don't overwrite or merge a same-named existing dataset on import. Suffix and keep both — the user decides what to delete.

---

6. Rename propagation: keep the link consistent

Renaming a dataset is the operation that ties §2–§5 together. A rename must, in one atomic step:

1. Update the dataset's own name.
2. For every snippet referencing the old name: rewrite the named-data references inside its spec ({ "data": { "name": "old" } } → { "data": { "name": "new" } }) — in both spec and draftSpec.
3. Recompute that snippet's datasetRefs from the rewritten spec, so the forward link mirrors reality and the reverse scan stays correct.

The spec rewrite is pure; the orchestration reads and writes stores.

// src/core/spec-refs.ts — pure rewrite

/** Returns a copy of `spec` with every data.name === oldName replaced by newName. */
export function renameDatasetInSpec(spec: Json, oldName: string, newName: string): Json {
  const obj = typeof spec === 'string' ? safeParse(spec) : spec;
  const selfDefined = selfDefinedNames(obj); // never rename a spec's own inline dataset name

  const rewrite = (node: Json): Json => {
    if (Array.isArray(node)) return node.map(rewrite);
    if (node && typeof node === 'object') {
      const out: Record<string, Json> = {};
      for (const [k, v] of Object.entries(node as Record<string, Json>)) {
        // A `data` object is a reference site: rename a matching name, but never
        // recurse into its payload. `datasets` (self-defined inline data) is left
        // whole. Same prune as extractDatasetRefs — see §3.1.
        if (k === 'data' && v && typeof v === 'object' && !Array.isArray(v)) {
          const dv = v as Record<string, Json>;
          out[k] = dv.name === oldName && !selfDefined.has(oldName) ? { ...dv, name: newName } : dv;
        } else if (k === 'data' || k === 'datasets') {
          out[k] = v;
        } else {
          out[k] = rewrite(v);
        }
      }
      return out;
    }
    return node;
  };

  const rewritten = rewrite(obj);
  // Preserve the original spec's stored shape (string vs object).
  return typeof spec === 'string' ? JSON.stringify(rewritten, null, 2) : rewritten;
}

// src/app/services/RelationshipService.ts — store coordination

import { isNameTaken, makeUniqueName } from '../../core/naming';
import { renameDatasetInSpec, recomputeDatasetRefs } from '../../core/spec-refs';
import { useDatasetStore } from '../stores/DatasetStore';
import { useSnippetStore } from '../stores/SnippetStore';

/**
 * Renames a dataset and propagates the rename to every referencing snippet
 * (spec, draftSpec, and datasetRefs). Returns the snippets that changed.
 * Caller is responsible for collision policy on `newName` (reject vs suffix).
 */
export function renameDatasetEverywhere(oldName: string, newName: string): { updated: number } {
  if (oldName === newName) return { updated: 0 };

  // 1. Rename the dataset record itself.
  const dataset = useDatasetStore.getState().datasets.find((d) => d.name === oldName);
  if (!dataset) return { updated: 0 };
  useDatasetStore.getState().update(dataset.id, { name: newName });

  // 2 + 3. Rewrite every referencing snippet's specs and refs.
  let updated = 0;
  for (const snippet of useSnippetStore.getState().snippets) {
    if (!snippet.datasetRefs.some((r) => r.toLowerCase() === oldName.toLowerCase())) continue;

    const spec = renameDatasetInSpec(snippet.spec, oldName, newName);
    const draftSpec = renameDatasetInSpec(snippet.draftSpec, oldName, newName);
    useSnippetStore.getState().update(snippet.id, {
      spec,
      draftSpec,
      datasetRefs: recomputeDatasetRefs(spec),
    });
    updated++;
  }

  return { updated };
}

Collision on rename. The UI rename form rejects a name already in use via isNameTaken(newName, datasets, dataset.id). Programmatic renames (e.g. an import flow) instead resolve with makeUniqueName before calling renameDatasetEverywhere. The propagation function itself does not invent a name — it assumes newName is the agreed target.

Do

• Rewrite spec and draftSpec. A user mid-edit must not see their draft silently break because the dataset was renamed underneath them.
• Recompute datasetRefs from the rewritten spec rather than string-replacing the array — the spec is the source of truth, the array is its mirror.
• Use the §4 reverse lookup to find affected snippets, so "who references this" has exactly one implementation.

Don't

• Don't update datasetRefs without also rewriting the spec — the rendered named-data reference would still point at the old, now-missing name.
• Don't rename the dataset and skip propagation "for now". A half-applied rename is the exact inconsistency this whole document exists to prevent.

---

7. Where things live

Concern	Location	Pure?	Tested
makeUniqueName, isNameTaken	src/core/naming.ts	yes	unit
extractDatasetRefs, recomputeDatasetRefs	src/core/spec-refs.ts	yes	unit
renameDatasetInSpec	src/core/spec-refs.ts	yes	unit
snippetsReferencingDataset, datasetUsageCounts (reverse-lookup scan)	src/core/relationships.ts	yes	unit
findSnippetsReferencingDataset, usage count (snapshot wrappers)	src/app/services/RelationshipService.ts	no (reads store)	integration
renameDatasetEverywhere → { updated }	src/app/services/RelationshipService.ts	no (mutates stores)	integration
dedupeIncomingDatasetNames	src/app/services/ImportService.ts	nearly (uses makeUniqueName)	unit/integration

The dividing line: anything that takes plain data and returns plain data is core and unit-tested in isolation; anything that reaches into a Zustand store is an app service. The rename rule of thumb — the spec is the source of truth, datasetRefs mirrors it, the reverse lookup is derived — is what keeps the bidirectional link from ever needing manual repair.