Editor: view-scoped Extract-to-Dataset for inline and self-defined data

This commit is contained in:
2026-06-29 02:08:59 +03:00
parent 8cad80f738
commit 6656811b8e
16 changed files with 910 additions and 180 deletions
+4 -1
View File
@@ -75,7 +75,10 @@ export interface ModalConfig {
component: ComponentType<any>; // the body rendered inside the shell component: ComponentType<any>; // the body rendered inside the shell
/** Initialize transient modal state when it opens. `arg` carries an /** Initialize transient modal state when it opens. `arg` carries an
* optional sub-target (e.g. a dataset id for chartBuilder/extract). */ * optional sub-target (e.g. a dataset id for chartBuilder/datasets). OMIT
* when a service seeds the store *before* `openModal` — Extract is seeded by
* `services/extract-action` from the editor cursor, and an `init` here would
* re-read and clobber that view-scoped capture. */
init?: (arg?: string) => void; init?: (arg?: string) => void;
/** Serializable snapshot of in-progress edits, used to detect unsaved /** Serializable snapshot of in-progress edits, used to detect unsaved
@@ -164,35 +164,18 @@ export function libraryRefName(data: unknown, selfDefined: ReadonlySet<string>):
References appear in several places — top-level `data`, per-layer `data`, `data` References appear in several places — top-level `data`, per-layer `data`, `data`
inside `spec`/`facet`/`hconcat`/`vconcat`, and a lookup transform's `from.data`. inside `spec`/`facet`/`hconcat`/`vconcat`, and a lookup transform's `from.data`.
Rather than enumerate the grammar, each pass walks the spec recursively but Rather than enumerate the grammar, the walk recurses the spec but **prunes two
**prunes two keys**: a `data` object's payload (its `values`/rows) and the keys**: a `data` object's payload (its `values`/rows) and the top-level `datasets`
top-level `datasets` map hold user data, not nested specs. Without the prune, a map hold user data, not nested specs. Without the prune, a data _row_ carrying a
data _row_ carrying a field literally named `data: { name: "x" }` is misread as a field literally named `data: { name: "x" }` is misread as a reference.
reference.
```ts That walk is **one shared pair** in `core/spec-data`, not re-implemented per pass:
// src/core/spec-refs.ts — the recursive walk; classification routes through spec-data `forEachDataBinding(spec, visit)` (read-only, `visit` returns `true` to stop early)
and `mapDataBindings(spec, mapData)` (returns a copy). Both compute the spec's
export function extractDatasetRefs(spec: Json): string[] { self-defined names once and hand them to the callback, so a caller writes only its
const root = typeof spec === 'string' ? safeParse(spec) : spec; // unparseable → no refs own rule — _what is a reference_ (`libraryRefName`) or _what to rewrite_ — never the
const selfDefined = selfDefinedNames(root); walk. `extractDatasetRefs` collects through `forEachDataBinding`; `renameDatasetInSpec`
const names = new Set<string>(); and `promoteSelfDefinedDataset` (§3.2) rewrite through `mapDataBindings`.
const walk = (node: Json): void => {
if (Array.isArray(node)) return void node.forEach(walk);
if (node && typeof node === 'object') {
const obj = node as Record<string, Json>;
const refName = libraryRefName(obj.data, selfDefined);
if (refName !== null) names.add(refName);
for (const key of Object.keys(obj)) {
if (key === 'data' || key === 'datasets') continue; // prune user-data payloads
walk(obj[key]);
}
}
};
walk(root);
return [...names];
}
```
`recomputeDatasetRefs` is the thin sorted/de-duped wrapper stored on `recomputeDatasetRefs` is the thin sorted/de-duped wrapper stored on
`snippet.datasetRefs`, run on every draft change and on publish. `snippet.datasetRefs`, run on every draft change and on publish.
@@ -211,22 +194,50 @@ export function extractDatasetRefs(spec: Json): string[] {
programmatic extract/revert rewrites, never on raw keystrokes. That keeps the programmatic extract/revert rewrites, never on raw keystrokes. That keeps the
links in step with the edited draft while never recomputing from a links in step with the edited draft while never recomputing from a
transiently-invalid spec. transiently-invalid spec.
- Keep the three ref walks in lockstep — extraction here, `renameDatasetInSpec`, - Route every binding pass through the shared `forEachDataBinding` /
and the renderer's `resolveDatasetRefs` (`src/core/rendering.ts`). They agree `mapDataBindings`. They share both halves that must agree — the `libraryRefName`
because they share both halves: the `libraryRefName` classifier (what is a classifier (what is a reference) and the prune of the **same two keys** (`data`,
reference) and the prune of the **same two keys** (`data`, `datasets`). If one `datasets`) — so extraction, rename, and the reverse extraction cannot drift. If
classified differently, or descended into data payloads while another didn't, a one classified differently, or descended into data payloads while another didn't,
row field named `data` would get counted, rewritten, or throw a row field named `data` would get counted, rewritten, or throw
`DatasetNotFoundError`. `DatasetNotFoundError`. (The renderer's `resolveDatasetRefs` mutates in place and
can throw mid-walk, so it keeps its own copy of the walk — the one exception, held
in step by the same prune-two-keys rule.)
**Don't** **Don't**
- Don't let two code paths each have their own idea of "referenced names". - Don't let two code paths each have their own idea of "referenced names".
Renamer, ref-recomputer, and renderer must use the same walk shape. Renamer, ref-recomputer, and renderer must use the same walk shape.
- Don't add a fresh prune-walk for a new binding pass — reuse the shared pair. A
hand-written copy is one classifier tweak away from disagreeing with the others.
- Don't "enumerate the grammar" (scope the walk to a fixed list of container - Don't "enumerate the grammar" (scope the walk to a fixed list of container
keys) to fix the payload-descent problem — pruning the two data-bearing keys keys) to fix the payload-descent problem — pruning the two data-bearing keys
stays correct as Vega-Lite's composition grammar grows; an allow-list rots. stays correct as Vega-Lite's composition grammar grows; an allow-list rots.
### 3.2 Extracting embedded data into a dataset — the reverse (`spec-inline-data.ts`, `spec-refs.ts`)
Extract-to-Dataset is the inverse of a reference: it lifts a view's **embedded**
data into a stored dataset and rewrites the spec to reference it by name. It is a
cursor-scoped editor action — `services/extract-action` resolves the focused view's
binding (`dataBindingAtPath`), seeds the modal, then opens it; the gate
`specHasExtractableData` hides the toolbar action when no view carries liftable
data. Two embedded shapes lift, both routing through the same `spec-data` classifier
so they never disagree with reference detection:
- **Inline `values`** — `inlineValuesOf` captures the payload verbatim (a CSV/TSV
string is kept as-is); confirm rewrites that view's `data` block, at its anchor
path, to `{ name }`.
- **A self-defined `datasets` entry** the view references — `selfDefinedPayloadOf`
reads the named rows and `promoteSelfDefinedDataset` drops the `datasets` entry
(and the map when it empties). Keeping the name needs no reference rewrite — it
un-shadows onto the new library dataset; renaming rewrites every matching
reference. This is the reverse direction of the self-defined-vs-library
precedence in §3.1.
A `lookup` transform's inline `from.data` lifts like any view binding —
`dataBindingAtPath` finds it. A `url`, a generator, or an existing library
reference carries nothing to lift.
--- ---
## 4. Reverse lookup: who uses this dataset? ## 4. Reverse lookup: who uses this dataset?
+7
View File
@@ -8,6 +8,13 @@ record the resolution into the contract (`docs/architecture/09`+`10` and the rel
## Open ## Open
- **Extract-to-Dataset has no keyboard accelerator** — its sibling editor actions (wrap /
config, in `spec-transform-actions` / `spec-config-actions`) register an F1-palette command
and a lightbulb; Extract is toolbar-only (`runExtract` in `services/extract-action.ts`),
because it opens a modal rather than making an in-place undoable edit, so the palette/lightbulb
fit awkwardly. Decide whether to add a palette command anyway for parity (a keyboard path to
open the modal at the cursor), or leave toolbar-only.
- **Storage-full copy implies a per-tier budget, but quota is whole-origin** — the messages - **Storage-full copy implies a per-tier budget, but quota is whole-origin** — the messages
say "snippet storage is full" / "dataset storage is full" and tell the user to delete that say "snippet storage is full" / "dataset storage is full" and tell the user to delete that
entity's items, yet IndexedDB quota is shared across the whole origin. Per-tier framing is entity's items, yet IndexedDB quota is shared across the whole origin. Per-tier framing is
+3 -3
View File
@@ -1,7 +1,7 @@
/** /**
* Extract-to-Dataset — the modal body (spec §03F). * Extract-to-Dataset — the modal body (spec §03F).
* *
* Shows a read-only preview of the active snippet draft's inline data and asks * Shows a read-only preview of the focused view's embedded data and asks
* for a dataset name. On confirm it saves the data as a new dataset and rewrites * for a dataset name. On confirm it saves the data as a new dataset and rewrites
* the draft to reference it by name (logic in ExtractStore), then force-closes * the draft to reference it by name (logic in ExtractStore), then force-closes
* (the commit is the user's confirmation, so no discard prompt). Cancel leaves * (the commit is the user's confirmation, so no discard prompt). Cancel leaves
@@ -38,13 +38,13 @@ export function ExtractModal() {
}; };
if (!source) { if (!source) {
return <p className={styles.muted}>This snippet has no inline data to extract.</p>; return <p className={styles.muted}>This snippet has no embedded data to extract.</p>;
} }
return ( return (
<div className={styles.extract}> <div className={styles.extract}>
<p className={styles.intro}> <p className={styles.intro}>
Save this snippet&rsquo;s inline data as a reusable dataset. The spec will be rewritten to Save this snippet&rsquo;s embedded data as a reusable dataset. The spec will be rewritten to
reference it by name. reference it by name.
</p> </p>
+17 -7
View File
@@ -26,7 +26,7 @@ import '../infrastructure/monaco-env'; // side-effect: wire workers before creat
import { configureVegaLiteJson } from '../infrastructure/monaco-schema'; import { configureVegaLiteJson } from '../infrastructure/monaco-schema';
import { configureJsonFormatter, installFormatOnPaste } from '../infrastructure/monaco-format'; import { configureJsonFormatter, installFormatOnPaste } from '../infrastructure/monaco-format';
import { parseChartSpecText } from '@core/chart-builder'; import { parseChartSpecText } from '@core/chart-builder';
import { openChartBuilderForEdit, openModal } from '../modals/ModalCoordinator'; import { openChartBuilderForEdit } from '../modals/ModalCoordinator';
import { import {
installSpecConfigActions, installSpecConfigActions,
runExtractConfig, runExtractConfig,
@@ -41,10 +41,11 @@ import {
runWrap, runWrap,
} from '../services/spec-transform-actions'; } from '../services/spec-transform-actions';
import { configureSpecDatasetHints } from '../services/spec-dataset-hints'; import { configureSpecDatasetHints } from '../services/spec-dataset-hints';
import { runExtract } from '../services/extract-action';
import { useAppStore } from '../stores/AppStore'; import { useAppStore } from '../stores/AppStore';
import { confirm } from '../stores/ConfirmStore'; import { confirm } from '../stores/ConfirmStore';
import { useDatasetStore } from '../stores/DatasetStore'; import { useDatasetStore } from '../stores/DatasetStore';
import { hasInlineData } from '../stores/ExtractStore'; import { hasExtractableData } from '../stores/ExtractStore';
import { publishActiveSnippet } from '../services/snippet-actions'; import { publishActiveSnippet } from '../services/snippet-actions';
import { notify } from '../stores/NotificationStore'; import { notify } from '../stores/NotificationStore';
import { usePreviewStore } from '../stores/PreviewStore'; import { usePreviewStore } from '../stores/PreviewStore';
@@ -222,10 +223,11 @@ function EditorToolbar({
const draft = s.editorView === 'draft' ? s.draftText : active.draftSpec; const draft = s.editorView === 'draft' ? s.draftText : active.draftSpec;
return draft !== active.spec; return draft !== active.spec;
}); });
// Offer Extract only when the live draft carries top-level inline data to lift // Offer Extract only when the live draft carries data to lift out — inline
// out (spec §03F → hidden when the spec has no inline data). // `values` in any view, or a reference to a self-defined `datasets` entry (spec
// §03F → hidden when there is nothing extractable).
const canExtract = useSnippetStore( const canExtract = useSnippetStore(
(s) => s.activeSnippetId !== null && hasInlineData(s.draftText), (s) => s.activeSnippetId !== null && hasExtractableData(s.draftText),
); );
// Offer "Open in builder" only when the active snippet's published spec is // Offer "Open in builder" only when the active snippet's published spec is
@@ -248,6 +250,14 @@ function EditorToolbar({
if (snippet) openChartBuilderForEdit(snippet); if (snippet) openChartBuilderForEdit(snippet);
}; };
// Extract is scoped to the view at the cursor (services/extract-action), so it
// goes through the editor handle like the wrap/config actions, not a bare
// openModal — the service captures the focused binding before opening the modal.
const handleExtract = () => {
const editor = editorRef.current;
if (editor) runExtract(editor);
};
// Publish + its success toast live in one place (services/snippet-actions) so // Publish + its success toast live in one place (services/snippet-actions) so
// the button and the Cmd/Ctrl+S shortcut (EventRouter) behave identically. // the button and the Cmd/Ctrl+S shortcut (EventRouter) behave identically.
const handlePublish = publishActiveSnippet; const handlePublish = publishActiveSnippet;
@@ -315,8 +325,8 @@ function EditorToolbar({
{canExtract && ( {canExtract && (
<Button <Button
className={styles.collapsible} className={styles.collapsible}
onClick={() => openModal('extract')} onClick={handleExtract}
title="Extract inline data into a reusable dataset" title="Extract embedded data into a reusable dataset"
aria-label="Extract to Dataset" aria-label="Extract to Dataset"
> >
<Icon name="dataset" className={styles.actionIcon} /> <Icon name="dataset" className={styles.actionIcon} />
+3 -2
View File
@@ -63,11 +63,12 @@ const MODAL_REGISTRY: Partial<Record<ModalName, ModalConfig>> = {
}, },
}, },
// Opened from the snippet editor with the active draft's inline data to lift out. // Opened from the snippet editor by `services/extract-action`, which seeds the
// store with the focused view's inline data (`begin`) *before* opening — so no
// `init` here, which would re-read top-level and clobber the view-scoped capture.
extract: { extract: {
name: 'extract', name: 'extract',
title: 'Extract to Dataset', title: 'Extract to Dataset',
init: () => useExtractStore.getState().init(),
getState: () => ({ name: useExtractStore.getState().name }), getState: () => ({ name: useExtractStore.getState().name }),
}, },
+82
View File
@@ -0,0 +1,82 @@
/**
* Extract-to-Dataset as an editor action (spec §03F; docs/architecture/08 →
* editor augmentation) — the data-facing counterpart to the wrap/config actions.
*
* Resolves the embedded data of the **view the cursor sits in** (the nearest
* enclosing `data` binding, honoring Vega-Lite's parent→child inheritance), seeds
* the Extract modal with it, and opens the modal. The cursor picks which view to
* extract; a single-view spec resolves to the root binding from any cursor, so the
* common case needs no thought. Two binding shapes are liftable — a view's inline
* `data.values`, and a `{ name }` reference to the spec's own top-level `datasets`
* (the data lives in the `datasets` map; we pre-fill the modal with its name).
*
* The toolbar offers the action whenever *some* view has extractable data
* (`hasExtractableData`); when the focused view itself has none — it references a
* library dataset, a url, a generator — there is nothing to lift here, so guide the
* user to a view that does rather than silently extract the wrong one.
*/
import type * as monaco from 'monaco-editor/esm/vs/editor/edcore.main';
import { pathAtOffset } from '@core/spec-cursor';
import { dataBindingAtPath } from '@core/spec-data';
import { inlineValuesOf, selfDefinedPayloadOf } from '@core/spec-inline-data';
import { openModal } from '../modals/ModalCoordinator';
import { type ExtractTarget, useExtractStore } from '../stores/ExtractStore';
import { notify } from '../stores/NotificationStore';
// TODO(ux-second-pass): toolbar-only — no F1-palette accelerator like the sibling
// wrap/config actions, since this opens a modal rather than an in-place edit.
/** Open Extract-to-Dataset scoped to the view at the cursor. */
export function runExtract(editor: monaco.editor.IStandaloneCodeEditor): void {
const model = editor.getModel();
if (!model) return;
const text = model.getValue();
let spec: unknown;
try {
spec = JSON.parse(text);
} catch {
notify({
kind: 'info',
title: 'Cant extract yet',
message: 'Fix the JSON so the spec parses, then extract.',
});
return;
}
const position = editor.getPosition();
const offset = position ? model.getOffsetAt(position) : 0;
const binding = dataBindingAtPath(spec, pathAtOffset(text, offset));
// Inline `values` → rewrite this view's data block; a self-defined `datasets`
// reference → lift the named entry and pre-fill the modal with that name.
const inline = binding && inlineValuesOf(binding.data);
const selfDefined = binding && !inline && selfDefinedPayloadOf(spec, binding.data);
let seed: {
source: ReturnType<typeof inlineValuesOf>;
target: ExtractTarget;
name?: string;
} | null = null;
if (binding && inline) {
seed = { source: inline, target: { kind: 'inline', anchorPath: binding.anchorPath } };
} else if (binding && selfDefined) {
const datasetName = (binding.data as { name: string }).name;
seed = {
source: selfDefined,
target: { kind: 'self-defined', datasetName },
name: datasetName,
};
}
if (!seed || !seed.source) {
notify({
kind: 'info',
title: 'No data to extract here',
message: 'Place the cursor in a view with inline or embedded data to extract it.',
});
return;
}
useExtractStore.getState().begin({ source: seed.source, target: seed.target, name: seed.name });
openModal('extract');
}
+199
View File
@@ -0,0 +1,199 @@
import { beforeEach, describe, expect, test } from 'vitest';
import { createSnippet } from '@core/snippet';
import { useDatasetStore } from './DatasetStore';
import { hasExtractableData, useExtractStore } from './ExtractStore';
import { useSnippetStore } from './SnippetStore';
const extract = () => useExtractStore.getState();
const snippets = () => useSnippetStore.getState();
const datasets = () => useDatasetStore.getState();
/** Load a single active snippet whose live draft buffer is `draft`. */
function activeDraft(draft: string): void {
const s = createSnippet({ id: 's', spec: draft, now: new Date('2026-01-01T00:00:00Z') });
useSnippetStore.getState().hydrate([s], 's');
useSnippetStore.getState().updateDraft(draft);
}
beforeEach(() => {
useExtractStore.getState().reset();
useSnippetStore.getState().reset();
useDatasetStore.getState().reset();
});
describe('hasExtractableData', () => {
test('true for inline data, false for a library reference or bad JSON', () => {
expect(hasExtractableData('{"data":{"values":[{"a":1}]},"mark":"bar"}')).toBe(true);
expect(hasExtractableData('{"data":{"name":"sales"}}')).toBe(false);
expect(hasExtractableData('not json')).toBe(false);
});
test('true when inline data lives only in a nested view', () => {
expect(
hasExtractableData('{"layer":[{"data":{"name":"sales"}},{"data":{"values":[{"a":1}]}}]}'),
).toBe(true);
});
test('true for a reference to a self-defined datasets entry', () => {
expect(hasExtractableData('{"datasets":{"sales":[{"a":1}]},"data":{"name":"sales"}}')).toBe(
true,
);
});
});
describe('confirm — inline rewrite', () => {
test('extracts the root binding and rewrites it to a by-name reference', () => {
activeDraft('{"data":{"values":[{"a":1},{"a":2}]},"mark":"bar"}');
extract().begin({
source: { values: [{ a: 1 }, { a: 2 }], format: 'json' },
target: { kind: 'inline', anchorPath: [] },
});
extract().setName('Sales');
expect(extract().confirm(new Date('2026-02-01T00:00:00Z'))).toBe(true);
const ds = datasets().datasets.find((d) => d.name === 'Sales');
expect(ds?.data).toEqual([{ a: 1 }, { a: 2 }]);
const draft = JSON.parse(snippets().draftText) as { data: unknown; mark: unknown };
expect(draft.data).toEqual({ name: 'Sales' });
expect(draft.mark).toBe('bar');
});
test('rewrites only the focused view in a composition, leaving siblings intact', () => {
activeDraft(
JSON.stringify({
layer: [{ data: { name: 'sales' }, mark: 'line' }, { data: { values: [{ a: 1 }] } }],
}),
);
extract().begin({
source: { values: [{ a: 1 }], format: 'json' },
target: { kind: 'inline', anchorPath: ['layer', 1] },
});
extract().setName('Overlay');
expect(extract().confirm()).toBe(true);
const draft = JSON.parse(snippets().draftText) as { layer: Array<{ data: unknown }> };
expect(draft.layer[0].data).toEqual({ name: 'sales' }); // sibling untouched
expect(draft.layer[1].data).toEqual({ name: 'Overlay' }); // focused view rewritten
});
test("rewrites a lookup transform's inline from.data (the cursor sits inside it)", () => {
activeDraft(
JSON.stringify({
data: { name: 'Library' },
transform: [
{ lookup: 'k', from: { data: { values: [{ k: 1, v: 9 }] }, key: 'k', fields: ['v'] } },
],
mark: 'bar',
}),
);
extract().begin({
source: { values: [{ k: 1, v: 9 }], format: 'json' },
target: { kind: 'inline', anchorPath: ['transform', 0, 'from'] },
});
extract().setName('Lookup');
expect(extract().confirm()).toBe(true);
const draft = JSON.parse(snippets().draftText) as {
transform: Array<{ from: { data: unknown } }>;
};
expect(draft.transform[0].from.data).toEqual({ name: 'Lookup' });
});
test('preserves a CSV string payload as a raw-text dataset', () => {
activeDraft('{"data":{"values":"a,b\\n1,2","format":{"type":"csv"}},"mark":"bar"}');
extract().begin({
source: { values: 'a,b\n1,2', format: 'csv' },
target: { kind: 'inline', anchorPath: [] },
});
extract().setName('Raw');
expect(extract().confirm()).toBe(true);
const ds = datasets().datasets.find((d) => d.name === 'Raw');
expect(ds?.format).toBe('csv');
expect(ds?.data).toBe('a,b\n1,2');
});
});
describe('confirm — self-defined datasets rewrite', () => {
test('keeping the name drops the datasets entry; the reference resolves to the library', () => {
activeDraft('{"datasets":{"sales":[{"a":1},{"a":2}]},"data":{"name":"sales"},"mark":"bar"}');
extract().begin({
source: { values: [{ a: 1 }, { a: 2 }], format: 'json' },
target: { kind: 'self-defined', datasetName: 'sales' },
name: 'sales',
});
expect(extract().confirm()).toBe(true);
const ds = datasets().datasets.find((d) => d.name === 'sales');
expect(ds?.data).toEqual([{ a: 1 }, { a: 2 }]);
const draft = JSON.parse(snippets().draftText) as { datasets?: unknown; data: unknown };
expect(draft.datasets).toBeUndefined(); // map emptied → removed
expect(draft.data).toEqual({ name: 'sales' }); // reference unchanged, now a library ref
});
test('renaming rewrites every reference and keeps other datasets entries', () => {
activeDraft(
JSON.stringify({
datasets: { sales: [{ a: 1 }], other: [{ b: 2 }] },
layer: [
{ data: { name: 'sales' } },
{ data: { name: 'sales' } },
{ data: { name: 'other' } },
],
}),
);
extract().begin({
source: { values: [{ a: 1 }], format: 'json' },
target: { kind: 'self-defined', datasetName: 'sales' },
name: 'Sales 2024',
});
expect(extract().confirm()).toBe(true);
const draft = JSON.parse(snippets().draftText) as {
datasets: Record<string, unknown>;
layer: Array<{ data: unknown }>;
};
expect(draft.layer[0].data).toEqual({ name: 'Sales 2024' }); // both sales refs rewritten
expect(draft.layer[1].data).toEqual({ name: 'Sales 2024' });
expect(draft.layer[2].data).toEqual({ name: 'other' }); // untouched
expect(draft.datasets).toEqual({ other: [{ b: 2 }] }); // sales entry dropped, other kept
});
});
describe('confirm — guards', () => {
test('rejects a duplicate name and creates nothing', () => {
activeDraft('{"data":{"values":[{"a":1}]}}');
extract().begin({
source: { values: [{ a: 1 }], format: 'json' },
target: { kind: 'inline', anchorPath: [] },
});
extract().setName('Sales');
extract().confirm();
const countAfterFirst = datasets().datasets.length;
activeDraft('{"data":{"values":[{"b":2}]}}');
extract().begin({
source: { values: [{ b: 2 }], format: 'json' },
target: { kind: 'inline', anchorPath: [] },
});
extract().setName('Sales');
expect(extract().confirm()).toBe(false);
expect(extract().error).toMatch(/already exists/);
expect(datasets().datasets.length).toBe(countAfterFirst);
});
test('refuses when the anchor path no longer resolves, creating nothing', () => {
activeDraft('{"data":{"values":[{"a":1}]}}'); // no layer array
extract().begin({
source: { values: [{ a: 1 }], format: 'json' },
target: { kind: 'inline', anchorPath: ['layer', 3] },
});
extract().setName('Ghost');
expect(extract().confirm()).toBe(false);
expect(extract().error).toMatch(/Could not locate/);
expect(datasets().datasets.length).toBe(0);
});
});
+87 -67
View File
@@ -1,92 +1,86 @@
/** /**
* Extract-inline-data → Dataset state (spec §03F). * Extract-embedded-data → Dataset state (spec §03F).
* *
* Backs the Extract modal: the reverse of a named reference. It captures the * Backs the Extract modal: the reverse of a named reference. It captures the
* active snippet draft's inline data, takes a dataset name, and on confirm saves * embedded data of the **view the cursor sits in**, takes a dataset name, and on
* the data as a new dataset and rewrites the draft so the inline data is replaced * confirm saves the data as a new library dataset and rewrites the spec so the
* by a by-name reference (`{ "data": { "name": … } }`). * embedded data is replaced by a by-name reference.
* *
* Scope (M3): the **top-level** `data` block of the draft spec — the common case * Two shapes of embedded data, captured as a `target` (spec §03F; multi-view scope
* for a single-view chart. Inline data nested inside layers/concats is left for a * doc M3):
* later pass; `hasInlineData` reflects exactly what `confirm` can lift, so the * - **inline** — a view's `data.values`. Confirm rewrites that view's `data`
* editor only offers the action when this store can act on it. * block at its anchor path to `{ name }`.
* - **self-defined** — a `{ name: X }` reference to the spec's own top-level
* `datasets.X`. Confirm removes the `datasets` entry (and the map when it
* empties); the reference resolves to the new library dataset, renamed when the
* name changes (`core/spec-refs` → `promoteSelfDefinedDataset`).
*
* The store is the editor-free side of the flow: it never reads the cursor or the
* Monaco model — `services/extract-action` resolves the focused binding and seeds
* it via `begin`, so this stays a plain, testable store.
*/ */
import { create } from 'zustand'; import { create } from 'zustand';
import type { DataFormat } from '@core/format-detection';
import { createDataset } from '@core/dataset'; import { createDataset } from '@core/dataset';
import { formatSpec } from '@core/json-format';
import { isNameTaken } from '@core/naming'; import { isNameTaken } from '@core/naming';
import { setDataBindingAtPath } from '@core/spec-data';
import { type InlinePayload, specHasExtractableData } from '@core/spec-inline-data';
import { promoteSelfDefinedDataset } from '@core/spec-refs';
import { useDatasetStore } from './DatasetStore'; import { useDatasetStore } from './DatasetStore';
import { notify } from './NotificationStore'; import { notify } from './NotificationStore';
import { useSnippetStore } from './SnippetStore'; import { useSnippetStore } from './SnippetStore';
/** The inline `data` block of a parsed spec, if it carries `values`. */ /** The path of the view whose `data` block Extract rewrites (`[]` = root). */
interface InlineData { type AnchorPath = ReadonlyArray<string | number>;
values: unknown;
format: DataFormat;
}
/** /** What confirm rewrites — an inline `data` block, or a self-defined `datasets` entry. */
* Read the top-level inline data from a draft spec's text, or null when there is export type ExtractTarget =
* none (no snippet, unparseable, or no `data.values`). The format comes from an | { kind: 'inline'; anchorPath: AnchorPath }
* explicit `data.format.type` when present (raw CSV/TSV strings), else JSON. | { kind: 'self-defined'; datasetName: string };
*/
function readInlineData(draftText: string): InlineData | null { /** True when the active snippet's draft has data Extract can lift, in any view. */
let parsed: unknown; export function hasExtractableData(draftText: string): boolean {
try { try {
parsed = JSON.parse(draftText); return specHasExtractableData(JSON.parse(draftText));
} catch { } catch {
return null; return false;
} }
if (!parsed || typeof parsed !== 'object') return null;
const data = (parsed as Record<string, unknown>).data;
if (!data || typeof data !== 'object') return null;
const values = (data as Record<string, unknown>).values;
if (values === undefined) return null;
const declared = (data as Record<string, unknown>).format;
const type =
declared && typeof declared === 'object'
? (declared as Record<string, unknown>).type
: undefined;
const format: DataFormat =
type === 'csv' || type === 'tsv' || type === 'topojson' ? type : 'json';
return { values, format };
}
/** True when the active snippet's draft has top-level inline data to extract. */
export function hasInlineData(draftText: string): boolean {
return readInlineData(draftText) !== null;
} }
export interface ExtractState { export interface ExtractState {
/** Proposed dataset name (required, unique). */ /** Proposed dataset name (required, unique). */
name: string; name: string;
/** The inline data captured at open, for the read-only preview. */ /** The focused view's embedded data captured at open, for the read-only preview. */
source: InlineData | null; source: InlinePayload | null;
/** Where `confirm` writes the by-name reference. */
target: ExtractTarget | null;
/** Inline validation message, or null. */ /** Inline validation message, or null. */
error: string | null; error: string | null;
/** Capture the active snippet's inline data and reset the form. */ /** Seed the form with the focused view's captured data (extract-action). */
init: () => void; begin: (captured: { source: InlinePayload; target: ExtractTarget; name?: string }) => void;
setName: (name: string) => void; setName: (name: string) => void;
/** /**
* Validate, create the dataset, and rewrite the active draft to reference it by * Validate, create the dataset, and rewrite the spec to reference it by name.
* name. Returns whether it committed; on failure `error` is set. `now` * Returns whether it committed; on failure `error` is set. `now` injectable for
* injectable for tests. * tests.
*/ */
confirm: (now?: Date) => boolean; confirm: (now?: Date) => boolean;
reset: () => void; reset: () => void;
} }
const INITIAL = { name: '', source: null as InlineData | null, error: null as string | null }; const INITIAL = {
name: '',
source: null as InlinePayload | null,
target: null as ExtractTarget | null,
error: null as string | null,
};
export const useExtractStore = create<ExtractState>((set, get) => ({ export const useExtractStore = create<ExtractState>((set, get) => ({
...INITIAL, ...INITIAL,
init: () => { begin: ({ source, target, name = '' }) => set({ name, source, target, error: null }),
const draft = useSnippetStore.getState().draftText;
set({ name: '', source: readInlineData(draft), error: null });
},
setName: (name) => set({ name, error: null }), setName: (name) => set({ name, error: null }),
@@ -100,13 +94,42 @@ export const useExtractStore = create<ExtractState>((set, get) => ({
set({ error: `A dataset named "${name}" already exists. Choose a different name.` }); set({ error: `A dataset named "${name}" already exists. Choose a different name.` });
return false; return false;
} }
const source = get().source; const { source, target } = get();
if (!source) { if (!source || !target) {
set({ error: 'No inline data to extract.' }); set({ error: 'No data to extract.' });
return false; return false;
} }
// JSON/TopoJSON store the parsed value; CSV/TSV keep raw text — but inline // Resolve the rewrite before any side effect: parse the live draft and apply the
// target's rewrite to a fresh copy. Both can fail (the draft is no longer valid
// JSON, or its shape changed under the modal); bail with a message and create
// nothing rather than leave a half-done extraction.
const draftText = useSnippetStore.getState().draftText;
let spec: unknown;
try {
spec = JSON.parse(draftText);
} catch {
set({ error: 'The spec is no longer valid JSON. Close and reopen Extract.' });
return false;
}
let rewritten: unknown;
if (target.kind === 'inline') {
if (!setDataBindingAtPath(spec, target.anchorPath, { name })) {
set({ error: 'Could not locate the data to replace. Close and reopen Extract.' });
return false;
}
rewritten = spec;
} else {
const next = promoteSelfDefinedDataset(spec, target.datasetName, name);
if (!next) {
set({ error: 'Could not locate the data to replace. Close and reopen Extract.' });
return false;
}
rewritten = next;
}
// JSON/TopoJSON store the parsed value; CSV/TSV keep raw text — but the captured
// `values` is already the right runtime shape for each, so store it directly. // `values` is already the right runtime shape for each, so store it directly.
const dataset = createDataset({ const dataset = createDataset({
name, name,
@@ -117,20 +140,17 @@ export const useExtractStore = create<ExtractState>((set, get) => ({
}); });
useDatasetStore.getState().add(dataset); useDatasetStore.getState().add(dataset);
// Rewrite the top-level data block to a by-name reference, preserving the rest // Re-serialize in the app's house style (json-format), preserving the spec and
// of the spec and its pretty-printed text shape. // only swapping the captured data for the reference.
const draftText = useSnippetStore.getState().draftText; useSnippetStore.getState().replaceActiveDraft(formatSpec(rewritten), now);
const spec = JSON.parse(draftText) as Record<string, unknown>;
spec.data = { name };
useSnippetStore.getState().replaceActiveDraft(JSON.stringify(spec, null, 2), now);
// Success confirmation (spec §03F). Title states the action; the message // Success confirmation (spec §03F). Title states the action; the message adds
// adds the consequence — the spec was rewritten to reference the new dataset // the consequence — the spec was rewritten to reference the new dataset by name
// by name (council toast-copy rule, docs/architecture/10 → Toast copy). // (council toast-copy rule, docs/architecture/10 → Toast copy).
notify({ notify({
kind: 'success', kind: 'success',
title: 'Dataset created', title: 'Dataset created',
message: `The spec now references "${name}" instead of its inline data.`, message: `The spec now references "${name}" instead of its embedded data.`,
}); });
set(INITIAL); set(INITIAL);
return true; return true;
+1 -1
View File
@@ -114,7 +114,7 @@ export interface SnippetState {
/** /**
* Replace the active snippet's draft spec with new text and reload the editor * Replace the active snippet's draft spec with new text and reload the editor
* on the draft view (bumps `bufferEpoch`). Used by programmatic rewrites such * on the draft view (bumps `bufferEpoch`). Used by programmatic rewrites such
* as Extract-to-Dataset (spec §03F), which substitutes inline data for a * as Extract-to-Dataset (spec §03F), which substitutes embedded data for a
* by-name reference. No-op when no snippet is active. `now` injectable. * by-name reference. No-op when no snippet is active. `now` injectable.
*/ */
replaceActiveDraft: (text: string, now?: Date) => void; replaceActiveDraft: (text: string, now?: Date) => void;
+84 -1
View File
@@ -1,5 +1,13 @@
import { describe, expect, test } from 'vitest'; import { describe, expect, test } from 'vitest';
import { classifyData, dataBindingAtPath, libraryRefName, selfDefinedNames } from './spec-data'; import {
classifyData,
dataBindingAtPath,
forEachDataBinding,
libraryRefName,
mapDataBindings,
selfDefinedNames,
setDataBindingAtPath,
} from './spec-data';
describe('classifyData', () => { describe('classifyData', () => {
test('classifies the four Vega-Lite data shapes', () => { test('classifies the four Vega-Lite data shapes', () => {
@@ -108,3 +116,78 @@ describe('dataBindingAtPath', () => {
expect(dataBindingAtPath({ mark: 'bar' }, ['mark', 'type', 'nope'])).toBeNull(); expect(dataBindingAtPath({ mark: 'bar' }, ['mark', 'type', 'nope'])).toBeNull();
}); });
}); });
describe('setDataBindingAtPath', () => {
test('sets the data block of the view at the anchor path', () => {
const spec = { layer: [{ mark: 'line', data: { values: [{ a: 1 }] } }] };
expect(setDataBindingAtPath(spec, ['layer', 0], { name: 'X' })).toBe(true);
expect(spec.layer[0].data).toEqual({ name: 'X' });
});
test('sets the root data block for the empty path', () => {
const spec: { data?: unknown; mark: string } = { mark: 'bar' };
expect(setDataBindingAtPath(spec, [], { name: 'Root' })).toBe(true);
expect(spec.data).toEqual({ name: 'Root' });
});
test('returns false when the path does not resolve to an object', () => {
const spec = { layer: [{ mark: 'line' }] };
expect(setDataBindingAtPath(spec, ['layer', 3], { name: 'X' })).toBe(false);
expect(setDataBindingAtPath({ mark: 'bar' }, ['mark'], { name: 'X' })).toBe(false);
});
});
describe('forEachDataBinding', () => {
test('visits every object node and prunes the data/datasets payloads', () => {
// A row inside `data.values` carries a field literally named `data`; the prune
// keeps the walk from visiting it as a binding.
const spec = {
datasets: { ds: [{ z: 1 }] },
layer: [{ data: { name: 'A' } }, { data: { values: [{ data: { name: 'buried' } }] } }],
};
const seen: unknown[] = [];
forEachDataBinding(spec, (node) => {
if (node.data !== undefined) seen.push(node.data);
});
expect(seen).toEqual([{ name: 'A' }, { values: [{ data: { name: 'buried' } }] }]);
});
test('passes the spec self-defined names and stops early on a true return', () => {
const spec = {
datasets: { ds: [{ a: 1 }] },
layer: [{ data: { name: 'ds' } }, { data: { name: 'other' } }],
};
let visits = 0;
let sawSelfDefined: ReadonlySet<string> | null = null;
forEachDataBinding(spec, (node, selfDefined) => {
visits++;
sawSelfDefined = selfDefined;
return node.data !== undefined; // stop at the first node carrying a data block
});
expect(sawSelfDefined && [...sawSelfDefined]).toEqual(['ds']);
// root (no data) → layer[0] (has data, stop). layer[1] never visited.
expect(visits).toBe(2);
});
});
describe('mapDataBindings', () => {
test('maps each data block, carries datasets through, and prunes payloads', () => {
const spec = {
datasets: { ds: [{ a: 1 }] },
layer: [{ data: { name: 'A' } }, { data: { values: [{ data: 'row-field' }] } }],
};
const out = mapDataBindings(spec, (data) =>
typeof (data as { name?: unknown }).name === 'string' ? { name: 'MAPPED' } : data,
) as typeof spec;
expect(out.layer[0].data).toEqual({ name: 'MAPPED' });
expect(out.layer[1].data).toEqual({ values: [{ data: 'row-field' }] }); // payload untouched
expect(out.datasets).toEqual({ ds: [{ a: 1 }] }); // carried through
});
test('does not mutate the input', () => {
const spec = { data: { name: 'A' } };
const before = structuredClone(spec);
mapDataBindings(spec, () => ({ name: 'B' }));
expect(spec).toEqual(before);
});
});
+87
View File
@@ -108,3 +108,90 @@ export function dataBindingAtPath(
} }
return binding; return binding;
} }
/**
* Set the `data` block of the view at `anchorPath` (an anchor from
* `dataBindingAtPath`), in place — the write-counterpart of the read above, used by
* Extract-to-Dataset to swap a view's inline data for a `{ name }` reference.
* Mutates `spec`, so the caller passes a copy it owns (a fresh parse). Returns false
* when the path no longer resolves to an object, so the caller can refuse rather
* than write the wrong node.
*/
export function setDataBindingAtPath(
spec: unknown,
anchorPath: ReadonlyArray<string | number>,
block: unknown,
): boolean {
let node: unknown = spec;
for (const key of anchorPath) {
if (node === null || typeof node !== 'object') return false;
node = (node as Record<string | number, unknown>)[key];
}
if (node === null || typeof node !== 'object') return false;
(node as Record<string, unknown>).data = block;
return true;
}
/**
* The one spec walk every data-binding pass shares (reference detection/resolution,
* extraction gates). Visits each object node that could carry a `data` binding,
* with the spec's self-defined `datasets` names, and **prunes two payload keys** —
* a `data` object's contents and the top-level `datasets` map hold user data, not
* nested specs, so descending into them would misread a data row carrying a field
* literally named `data` as a binding. `visit` returns `true` to stop early (for a
* predicate); any other return continues. Pruning the same two keys here is what
* keeps every pass in agreement on what counts as a reference — they cannot drift
* because they share this walk.
*/
export function forEachDataBinding(
spec: unknown,
visit: (node: Record<string, unknown>, selfDefined: ReadonlySet<string>) => boolean | void,
): void {
const selfDefined = selfDefinedNames(spec);
let stop = false;
const walk = (node: unknown): void => {
if (stop) return;
if (Array.isArray(node)) {
for (const item of node) walk(item);
return;
}
if (!isJsonObject(node)) return;
if (visit(node, selfDefined) === true) {
stop = true;
return;
}
for (const key of Object.keys(node)) {
if (key === 'data' || key === 'datasets') continue;
walk(node[key]);
}
};
walk(spec);
}
/**
* Return a deep copy of `spec` with each data binding's `data` value replaced by
* `mapData(data, selfDefined)` — the copy-rewrite counterpart of
* {@link forEachDataBinding}, for rename/promote passes. Prunes the same two keys:
* the top-level `datasets` map is carried through untouched, and a `data` payload is
* never descended into (only its block is mapped). The input is not mutated.
*/
export function mapDataBindings(
spec: unknown,
mapData: (data: unknown, selfDefined: ReadonlySet<string>) => unknown,
): unknown {
const selfDefined = selfDefinedNames(spec);
const rewrite = (node: unknown): unknown => {
if (Array.isArray(node)) return node.map(rewrite);
if (node && typeof node === 'object') {
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(node as Record<string, unknown>)) {
if (key === 'data') out[key] = mapData(value, selfDefined);
else if (key === 'datasets') out[key] = value;
else out[key] = rewrite(value);
}
return out;
}
return node;
};
return rewrite(spec);
}
+83 -1
View File
@@ -1,5 +1,10 @@
import { describe, expect, it } from 'vitest'; import { describe, expect, it } from 'vitest';
import { rowsForDataBinding } from './spec-inline-data'; import {
inlineValuesOf,
rowsForDataBinding,
selfDefinedPayloadOf,
specHasExtractableData,
} from './spec-inline-data';
describe('rowsForDataBinding', () => { describe('rowsForDataBinding', () => {
it('reads inline values from the given binding', () => { it('reads inline values from the given binding', () => {
@@ -21,3 +26,80 @@ describe('rowsForDataBinding', () => {
expect(rowsForDataBinding({ mark: 'bar' }, { values: [] })).toBeNull(); // empty rows expect(rowsForDataBinding({ mark: 'bar' }, { values: [] })).toBeNull(); // empty rows
}); });
}); });
describe('inlineValuesOf', () => {
it('returns the JSON array payload as json', () => {
expect(inlineValuesOf({ values: [{ a: 1 }] })).toEqual({ values: [{ a: 1 }], format: 'json' });
});
it('preserves a CSV/TSV string payload verbatim with its declared format', () => {
expect(inlineValuesOf({ values: 'a,b\n1,2', format: { type: 'csv' } })).toEqual({
values: 'a,b\n1,2',
format: 'csv',
});
expect(inlineValuesOf({ values: 'a\tb', format: { type: 'tsv' } })).toEqual({
values: 'a\tb',
format: 'tsv',
});
});
it('keeps a name riding on inline values (named-inline is still inline)', () => {
expect(inlineValuesOf({ name: 'x', values: [{ a: 1 }] })).toEqual({
values: [{ a: 1 }],
format: 'json',
});
});
it('returns null for a library reference, url, generator, or non-object', () => {
expect(inlineValuesOf({ name: 'sales' })).toBeNull();
expect(inlineValuesOf({ url: 'x.csv' })).toBeNull();
expect(inlineValuesOf({ sequence: { start: 0, stop: 5 } })).toBeNull();
expect(inlineValuesOf(null)).toBeNull();
});
});
describe('selfDefinedPayloadOf', () => {
it('returns the rows of a referenced self-defined datasets entry', () => {
const spec = { datasets: { sales: [{ a: 1 }, { a: 2 }] }, data: { name: 'sales' } };
expect(selfDefinedPayloadOf(spec, spec.data)).toEqual({
values: [{ a: 1 }, { a: 2 }],
format: 'json',
});
});
it('returns null for a library reference, inline values, or a non-array entry', () => {
const spec = { datasets: { sales: [{ a: 1 }] } };
expect(selfDefinedPayloadOf(spec, { name: 'LibraryRef' })).toBeNull(); // not self-defined
expect(selfDefinedPayloadOf(spec, { values: [{ a: 1 }] })).toBeNull(); // inline, not named
expect(
selfDefinedPayloadOf({ datasets: { s: 'a,b\n1,2' }, data: {} }, { name: 's' }),
).toBeNull();
});
});
describe('specHasExtractableData', () => {
it('is true for inline values and for a self-defined reference', () => {
expect(specHasExtractableData({ data: { values: [{ a: 1 }] } })).toBe(true);
expect(specHasExtractableData({ datasets: { ds: [{ a: 1 }] }, data: { name: 'ds' } })).toBe(
true,
);
});
it('is false for a library reference, url, or generator', () => {
expect(specHasExtractableData({ data: { name: 'sales' } })).toBe(false);
expect(specHasExtractableData({ data: { url: 'x.csv' } })).toBe(false);
expect(specHasExtractableData({ data: { sequence: { start: 0, stop: 5 } } })).toBe(false);
});
it('ignores a field literally named data inside a user data row (prune-the-payload)', () => {
// The walker prunes the `data`/`datasets` keys, so a row that carries a field
// named `data` is never mistaken for a binding — only the genuine top-level
// inline binding triggers, and an encoding field named `data` triggers nothing.
expect(
specHasExtractableData({ data: { values: [{ data: { name: 'x' } }] }, mark: 'bar' }),
).toBe(true);
expect(
specHasExtractableData({ layer: [{ mark: 'bar', encoding: { x: { field: 'data' } } }] }),
).toBe(false);
});
});
+84 -8
View File
@@ -1,18 +1,23 @@
/** /**
* Inline data carried by a spec (docs/architecture/08 → editor augmentation). * Inline data carried by a spec (docs/architecture/08 → editor augmentation).
* *
* Portable core. When a view binds no named library dataset, its columns still * Portable core. Two readers over a view's inline data:
* exist — inline, in `data.values` or a self-defined top-level `datasets` entry.
* This pulls those rows out so the editor can profile them (core/profile) and offer
* the same field hints a library-bound view gets — a "ghost dataset" derived from
* the spec itself, with nothing stored.
* *
* URL data has no rows to read statically, and `values` given as a CSV/TSV string * - `rowsForDataBinding` pulls **profileable JSON rows** out so the editor can
* needs format-aware parsing — both out of scope here (they yield null). * profile them (core/profile) and offer the same field hints a library-bound
* view gets — a "ghost dataset" derived from the spec, with nothing stored.
* URL data has no rows to read statically, and a CSV/TSV string `values`
* payload needs format-aware parsing — both out of scope there (they yield null).
* - `inlineValuesOf` / `selfDefinedPayloadOf` pull the **raw payload** (values +
* format) Extract-to-Dataset lifts whole into a stored dataset — from a view's
* inline `values` (kept verbatim, so a CSV/TSV string survives) or from a
* self-defined `datasets` entry the view references by name. `specHasExtractableData`
* is the gate Extract reads to decide whether any view carries data it can lift.
*/ */
import type { DataFormat } from './format-detection';
import { isJsonObject } from './spec-config'; import { isJsonObject } from './spec-config';
import { classifyData, selfDefinedNames } from './spec-data'; import { classifyData, forEachDataBinding, selfDefinedNames } from './spec-data';
/** An array of row objects, or null when the value is not tabular inline data. */ /** An array of row objects, or null when the value is not tabular inline data. */
function asRows(value: unknown): Record<string, unknown>[] | null { function asRows(value: unknown): Record<string, unknown>[] | null {
@@ -42,3 +47,74 @@ export function rowsForDataBinding(spec: unknown, data: unknown): Record<string,
} }
return null; return null;
} }
/** The raw inline payload of a `data` block, or null when it is not inline data. */
export interface InlinePayload {
/** The block's `values` verbatim — a JSON array OR a raw CSV/TSV/topojson string. */
values: unknown;
/** The format declared by `data.format.type`, defaulting to JSON. */
format: DataFormat;
}
/**
* The inline payload a `data` block carries — its `values` and declared format —
* or null when the block is not inline (`classifyData` ≠ `'inline'`). Unlike
* {@link rowsForDataBinding}, which profiles JSON rows and rejects a CSV/TSV
* string, this preserves the payload **verbatim**: it is what Extract-to-Dataset
* lifts into a stored dataset, and a raw CSV/TSV string is already the right
* runtime shape to store. The format reads an explicit `data.format.type`
* (csv/tsv/topojson), defaulting to json (a plain array of row objects).
*/
export function inlineValuesOf(data: unknown): InlinePayload | null {
if (classifyData(data) !== 'inline') return null;
const block = data as { values: unknown; format?: unknown };
const type = isJsonObject(block.format) ? block.format.type : undefined;
const format: DataFormat =
type === 'csv' || type === 'tsv' || type === 'topojson' ? type : 'json';
return { values: block.values, format };
}
/**
* The payload of a view's reference to a *self-defined* dataset — a `{ name }`
* block whose name the spec defines for itself via top-level `datasets`. This is
* the spec's own embedded data in a different shape (the rows live in the
* `datasets` map, not in a `data.values` block), so Extract can lift it too. Only a
* JSON-array `datasets` entry is liftable here; a string payload (rare) yields null
* and is left for a future pass. Returns null when `data` is not a self-defined
* reference. The library-reference case (`{ name }` not self-defined) is already a
* stored dataset — nothing to extract — and also yields null.
*/
export function selfDefinedPayloadOf(spec: unknown, data: unknown): InlinePayload | null {
if (classifyData(data) !== 'named') return null;
const { name } = data as { name: string };
if (!isJsonObject(spec) || !isJsonObject(spec.datasets) || !selfDefinedNames(spec).has(name)) {
return null;
}
const entry = (spec.datasets as Record<string, unknown>)[name];
return Array.isArray(entry) ? { values: entry, format: 'json' } : null;
}
/**
* Whether any view binds data Extract can lift — the gate that decides if
* Extract-to-Dataset is offered. That is inline `values` **or** a reference to a
* self-defined `datasets` entry (the spec's own embedded data, in either shape). A
* library reference, a url, or a generator carries nothing to lift. Runs over the
* shared `forEachDataBinding` walk, stopping at the first liftable binding.
*/
export function specHasExtractableData(spec: unknown): boolean {
let found = false;
forEachDataBinding(spec, (node, selfDefined) => {
const data = node.data;
const kind = classifyData(data);
if (
kind === 'inline' ||
(kind === 'named' &&
typeof (data as { name?: unknown }).name === 'string' &&
selfDefined.has((data as { name: string }).name))
) {
found = true;
return true; // stop at the first match
}
});
return found;
}
+58 -1
View File
@@ -1,5 +1,10 @@
import { describe, expect, test } from 'vitest'; import { describe, expect, test } from 'vitest';
import { extractDatasetRefs, recomputeDatasetRefs, renameDatasetInSpec } from './spec-refs'; import {
extractDatasetRefs,
promoteSelfDefinedDataset,
recomputeDatasetRefs,
renameDatasetInSpec,
} from './spec-refs';
describe('extractDatasetRefs', () => { describe('extractDatasetRefs', () => {
test('collects a top-level named-data reference', () => { test('collects a top-level named-data reference', () => {
@@ -161,3 +166,55 @@ describe('renameDatasetInSpec', () => {
expect(out.transform[0].from.data.name).toBe('New'); expect(out.transform[0].from.data.name).toBe('New');
}); });
}); });
describe('promoteSelfDefinedDataset', () => {
test('keeping the name drops the datasets entry and leaves references intact', () => {
const spec = { datasets: { sales: [{ a: 1 }] }, data: { name: 'sales' }, mark: 'bar' };
const out = promoteSelfDefinedDataset(spec, 'sales', 'sales')!;
expect(out.datasets).toBeUndefined(); // map emptied → removed
expect(out.data).toEqual({ name: 'sales' }); // reference un-shadows onto the library
});
test('renaming rewrites every reference and keeps sibling datasets entries', () => {
const spec = {
datasets: { sales: [{ a: 1 }], other: [{ b: 2 }] },
layer: [
{ data: { name: 'sales' } },
{ data: { name: 'sales' } },
{ data: { name: 'other' } },
],
};
const out = promoteSelfDefinedDataset(spec, 'sales', 'Sales 2024')!;
expect(out.layer).toEqual([
{ data: { name: 'Sales 2024' } },
{ data: { name: 'Sales 2024' } },
{ data: { name: 'other' } },
]);
expect(out.datasets).toEqual({ other: [{ b: 2 }] });
});
test('does not touch a named-inline block that happens to share the name', () => {
// `{ name, values }` is its own inline data, not a datasets reference.
const spec = {
datasets: { sales: [{ a: 1 }] },
layer: [{ data: { name: 'sales' } }, { data: { name: 'sales', values: [{ z: 9 }] } }],
};
const out = promoteSelfDefinedDataset(spec, 'sales', 'New')!;
expect(out.layer).toEqual([
{ data: { name: 'New' } }, // the reference
{ data: { name: 'sales', values: [{ z: 9 }] } }, // inline — left alone
]);
});
test('returns null when the name is not a self-defined dataset', () => {
expect(promoteSelfDefinedDataset({ data: { name: 'sales' } }, 'sales', 'New')).toBeNull();
expect(promoteSelfDefinedDataset({ datasets: { a: [{}] } }, 'missing', 'New')).toBeNull();
});
test('does not mutate the input', () => {
const spec = { datasets: { sales: [{ a: 1 }] }, data: { name: 'sales' } };
const before = structuredClone(spec);
promoteSelfDefinedDataset(spec, 'sales', 'New');
expect(spec).toEqual(before);
});
});
+64 -52
View File
@@ -30,7 +30,8 @@
* the recursive walk never has to care. * the recursive walk never has to care.
*/ */
import { libraryRefName, selfDefinedNames } from './spec-data'; import { isJsonObject } from './spec-config';
import { classifyData, forEachDataBinding, libraryRefName, mapDataBindings } from './spec-data';
type Json = unknown; type Json = unknown;
@@ -50,30 +51,11 @@ function safeParse(s: string): Json {
*/ */
export function extractDatasetRefs(spec: Json): string[] { export function extractDatasetRefs(spec: Json): string[] {
const root = typeof spec === 'string' ? safeParse(spec) : spec; const root = typeof spec === 'string' ? safeParse(spec) : spec;
const selfDefined = selfDefinedNames(root);
const names = new Set<string>(); const names = new Set<string>();
forEachDataBinding(root, (node, selfDefined) => {
const walk = (node: Json): void => { const refName = libraryRefName(node.data, selfDefined);
if (Array.isArray(node)) {
for (const item of node) walk(item);
return;
}
if (node && typeof node === 'object') {
const obj = node as Record<string, Json>;
const refName = libraryRefName(obj.data, selfDefined);
if (refName !== null) names.add(refName); if (refName !== null) names.add(refName);
// Recurse into every key except the two that hold data payloads (`data` — });
// classified above; `datasets` — the spec's own inline data). Pruning them
// keeps the walk out of user data rows, where a field named `data` would
// otherwise be misread as a reference. Kept in step with rendering.ts.
for (const key of Object.keys(obj)) {
if (key === 'data' || key === 'datasets') continue;
walk(obj[key]);
}
}
};
walk(root);
return [...names]; return [...names];
} }
@@ -98,34 +80,64 @@ export function renameDatasetInSpec<T>(spec: T, oldName: string, newName: string
// Unparseable text → return it unchanged (nothing resolvable to rewrite). // Unparseable text → return it unchanged (nothing resolvable to rewrite).
if (isString && root === null) return spec; if (isString && root === null) return spec;
const selfDefined = selfDefinedNames(root); // Rewrite only a `data` block that is a library reference to `oldName`; native
// data carrying the name as a label and a self-defined `datasets` key are left
const rewrite = (node: Json): Json => { // alone (the shared walk prunes the payload keys).
if (Array.isArray(node)) return node.map(rewrite); const rewritten = mapDataBindings(root, (data, selfDefined) =>
if (node && typeof node === 'object') { libraryRefName(data, selfDefined) === oldName
const out: Record<string, Json> = {}; ? { ...(data as Record<string, Json>), name: newName }
for (const [k, v] of Object.entries(node as Record<string, Json>)) { : data,
// A `data` block is a reference site, not a container: rename it when it is );
// a library reference to `oldName`, and never recurse into its payload. A
// `datasets` map is the spec's own inline data: leave it whole. Pruning
// both (mirroring extractDatasetRefs) keeps rename out of user data rows,
// where a field named `data` would otherwise be rewritten as a reference.
if (k === 'data') {
out[k] =
libraryRefName(v, selfDefined) === oldName
? { ...(v as Record<string, Json>), name: newName }
: v;
} else if (k === 'datasets') {
out[k] = v;
} else {
out[k] = rewrite(v);
}
}
return out;
}
return node;
};
const rewritten = rewrite(root);
return (isString ? JSON.stringify(rewritten, null, 2) : rewritten) as T; return (isString ? JSON.stringify(rewritten, null, 2) : rewritten) as T;
} }
/**
* Promote a self-defined dataset to a stored library reference — the rewrite behind
* Extract-to-Dataset for the `datasets.<name>` case (docs/architecture/07 §3.2).
*
* A view that binds `{ "data": { "name": X } }` where the spec defines `X` for
* itself via top-level `datasets` is referencing the spec's *own* embedded data.
* Once that data is saved to the library, the `datasets` entry must go so the same
* `{ name }` reference resolves to the stored dataset instead. This returns a copy
* of `spec` with:
* - every self-defined `{ name: oldName }` data block renamed to `newName` (a
* no-op when the name is unchanged, in which case the reference simply
* un-shadows onto the new library dataset of the same name); and
* - the `datasets[oldName]` entry removed (and the whole `datasets` map removed
* when it empties).
*
* Returns `null` when `oldName` is not actually a self-defined dataset, so the
* caller extracts nothing. The input is never mutated. (Parallel to
* `renameDatasetInSpec`, but it targets *named* references by name rather than
* *library* references via `libraryRefName` — self-defined names are exactly the
* ones the latter excludes.)
*/
export function promoteSelfDefinedDataset(
spec: unknown,
oldName: string,
newName: string,
): Record<string, Json> | null {
if (!isJsonObject(spec) || !isJsonObject(spec.datasets) || !(oldName in spec.datasets)) {
return null;
}
// Rename every named reference to oldName (a `data` block carrying `values`/`url`
// is its own data, not a `datasets` reference, so classify first). A same-name
// promotion needs no rewrite — the reference un-shadows onto the new library
// dataset once the entry below is gone.
const renamed = (
newName === oldName
? { ...spec }
: mapDataBindings(spec, (data) =>
classifyData(data) === 'named' && (data as { name: string }).name === oldName
? { ...(data as Record<string, Json>), name: newName }
: data,
)
) as Record<string, Json>;
const datasets = { ...(renamed.datasets as Record<string, Json>) };
delete datasets[oldName];
if (Object.keys(datasets).length === 0) delete renamed.datasets;
else renamed.datasets = datasets;
return renamed;
}