Core: classify spec data blocks by Vega-Lite fidelity (spec-data) for dataset refs

This commit is contained in:
2026-06-28 19:48:24 +03:00
parent 31458114fb
commit e69430834a
9 changed files with 385 additions and 142 deletions
@@ -135,70 +135,67 @@ without waiting for a publish. Recomputation runs only on a _valid_ spec —
auto-save is debounced and parse-gated (spec §03B) — so a transiently-invalid
draft never disturbs the links.
### 3.1 Extracting referenced names from a spec (pure — `src/core/spec-refs.ts`)
### 3.1 What counts as a reference, and extracting them (pure — `src/core/spec-data.ts`, `spec-refs.ts`)
A Vega-Lite spec can reference named data in several places: the top-level
`data`, per-layer `data`, `data` inside `spec`/`facet`/`hconcat`/`vconcat`, and a
lookup transform's `from.data`. A spec may also define its OWN inline datasets via
a top-level `datasets` map — those are self-defined, not library references.
Rather than enumerate Vega-Lite's grammar, we walk the spec recursively and
collect every `{ data: { name } }` — but **prune two keys**: never recurse into a
`data` object's payload (its `values`/rows) or the top-level `datasets` map,
because those hold user data, not nested specs. Without the prune, a data _row_
carrying a field literally named `data: { name: "x" }` is misread as a reference.
This is pure, deterministic, and the most heavily unit-tested function here.
A library reference is exactly Vega-Lite **named data**: a `data` block with a
string `name` and no `values`, `url`, or generator key (`sequence`/`sphere`/
`graticule`), whose name the spec does not define for itself via a top-level
`datasets` map. A `name` riding on inline `values` or a `url` is Vega-Lite's
runtime-rebind label — not a dependency — and self-defined `datasets` names
resolve natively; both are left untouched. This mirrors Vega-Lite's own
`isNamedData`, so Astrolabe **extends** Vega-Lite rather than diverging: every
native data form keeps working, and only true references are tracked and resolved.
That classification lives in **`core/spec-data`** (`classifyData`,
`libraryRefName`) — the single predicate that reference extraction, rename
(`spec-refs`), and render-time resolution (`rendering`) all route through, so they
cannot disagree on what is a dependency.
```ts
// src/core/spec-refs.ts
// src/core/spec-data.ts — the shared classifier (mirrors vega-lite/src/data.ts)
type Json = unknown;
/** The library name a `data` block references, or null for native VL data / self-defined names. */
export function libraryRefName(data: unknown, selfDefined: ReadonlySet<string>): string | null {
if (classifyData(data) !== 'named') return null; // url | values | generator → not a reference
const { name } = data as { name: string };
return selfDefined.has(name) ? null : name;
}
```
References appear in several places — top-level `data`, per-layer `data`, `data`
inside `spec`/`facet`/`hconcat`/`vconcat`, and a lookup transform's `from.data`.
Rather than enumerate the grammar, each pass walks the spec recursively but
**prunes two keys**: a `data` object's payload (its `values`/rows) and the
top-level `datasets` map hold user data, not nested specs. Without the prune, a
data _row_ carrying a field literally named `data: { name: "x" }` is misread as a
reference.
```ts
// src/core/spec-refs.ts — the recursive walk; classification routes through spec-data
/** Collects every library dataset name referenced by `{ data: { name } }`, excluding self-defined ones. */
export function extractDatasetRefs(spec: Json): string[] {
const root = typeof spec === 'string' ? safeParse(spec) : spec;
const selfDefined = selfDefinedNames(root); // names from the spec's own top-level `datasets`
const root = typeof spec === 'string' ? safeParse(spec) : spec; // unparseable → no refs
const selfDefined = selfDefinedNames(root);
const names = new Set<string>();
const walk = (node: Json): void => {
if (Array.isArray(node)) {
for (const item of node) walk(item);
return;
}
if (Array.isArray(node)) return void node.forEach(walk);
if (node && typeof node === 'object') {
const obj = node as Record<string, Json>;
const data = obj.data as Record<string, Json> | undefined;
if (data && typeof data === 'object' && typeof data.name === 'string') {
if (!selfDefined.has(data.name)) names.add(data.name);
}
// Prune: a `data` payload and the `datasets` map hold user data, not refs.
const refName = libraryRefName(obj.data, selfDefined);
if (refName !== null) names.add(refName);
for (const key of Object.keys(obj)) {
if (key === 'data' || key === 'datasets') continue;
if (key === 'data' || key === 'datasets') continue; // prune user-data payloads
walk(obj[key]);
}
}
};
walk(root);
return [...names];
}
function safeParse(s: string): Json {
try {
return JSON.parse(s);
} catch {
return null; // an unparseable draft simply has no resolvable refs
}
}
```
```ts
// src/core/spec-refs.ts — thin wrapper, run on every draft change and on publish
/** The list stored on snippet.datasetRefs. Sorted + de-duped for stable diffs. */
export function recomputeDatasetRefs(spec: Json): string[] {
return extractDatasetRefs(spec).sort();
}
```
`recomputeDatasetRefs` is the thin sorted/de-duped wrapper stored on
`snippet.datasetRefs`, run on every draft change and on publish.
> A `spec` may be an object or a string (see the Data Model). Normalize once,
> at the boundary, so the recursive walk never has to care.
@@ -207,17 +204,19 @@ export function recomputeDatasetRefs(spec: Json): string[] {
- Treat `extractDatasetRefs` as the single source of truth for "what does this
spec reference". The reverse-lookup and rename paths both depend on it
agreeing with what the renderer actually resolves.
agreeing with what the renderer actually resolves — which holds because all of
them classify through the same `core/spec-data` predicate.
- Recompute and store `datasetRefs` on **every draft change and on publish**
but only through the parse-gated, debounced auto-save (`commitDraft`) and the
programmatic extract/revert rewrites, never on raw keystrokes. That keeps the
links in step with the edited draft while never recomputing from a
transiently-invalid spec.
- Prune the **same two keys** (`data`, `datasets`) in all three ref walks —
extraction here, `renameDatasetInSpec`, and the renderer's `resolveDatasetRefs`
(`src/core/rendering.ts`). They must agree on what counts as a reference; if one
descends into data payloads and another doesn't, extraction and rendering
disagree and a row field named `data` either gets counted, rewritten, or throws
- Keep the three ref walks in lockstep — extraction here, `renameDatasetInSpec`,
and the renderer's `resolveDatasetRefs` (`src/core/rendering.ts`). They agree
because they share both halves: the `libraryRefName` classifier (what is a
reference) and the prune of the **same two keys** (`data`, `datasets`). If one
classified differently, or descended into data payloads while another didn't, a
row field named `data` would get counted, rewritten, or throw
`DatasetNotFoundError`.
**Don't**