Editor: CodeLens to scaffold Vega-Lite data transforms

Add a discoverable '+ Add transform' / '+ filter/aggregate/...' CodeLens in the
spec editor that scaffolds field-typed transform[] steps, with a type-to-filter
completion on the same catalog as an accelerator. New pure core
(spec-data-transforms): step-slot and view-site classification, the field-typed
catalog, and comma affixing. Deliberately the one home the schema leaves bare —
inline channel transforms and the transform key stay with the JSON schema.

Rename the composition toolbar menu Transform -> Compose so it reads distinctly
from a data transform. Deferred (recorded at their sites): extract a shared
cursor-lens skeleton across the two installers; a shared completion replace-range
helper (now 3 sites); params scaffolding as the sibling feature.
This commit is contained in:
2026-07-01 15:43:48 +03:00
parent da6a675982
commit 87b1b98c12
6 changed files with 851 additions and 21 deletions
+33 -7
View File
@@ -285,7 +285,15 @@ logic is pure `src/core/`; the Monaco glue is thin app-layer services.
**Core (pure, portable):**
- `spec-transforms` — wrap a view in `layer`/`hconcat`/`vconcat`/`facet`/`repeat`; collapse
a single-child composition (`unwrapSingleton`). Object-in/object-out.
a single-child composition (`unwrapSingleton`). Object-in/object-out. (Surfaced as the
toolbar **Compose** menu — named to read distinctly from a data `transform`, below.)
- `spec-data-transforms` — the data-pipeline counterpart: `transformSiteAt` resolves the view
the cursor is in and its `transform[]` range/count (for the CodeLens), `transformPlacementAt`
classifies a step slot (for the completion) — both tagged `shared` when the pipeline sits on a
composition parent, `view` on a unit. `DATA_TRANSFORMS` is the field-typed step catalog (filter,
calculate, aggregate, bin, timeUnit, window, joinaggregate, fold, lookup); each builder takes the
columns in scope and seeds each `${n:default}` tab stop with a type-appropriate one. A test
round-trips every snippet through `snippetToPlain` to assert it's valid JSON.
- `spec-data` — the Vega-Lite data model: classify a `data` block (`classifyData`, mirroring
`isNamedData`), the library reference name (`libraryRefName`), and the data binding in scope
at a cursor path (`dataBindingAtPath`, honoring a view's data inheritance from its parent).
@@ -335,11 +343,11 @@ logic is pure `src/core/`; the Monaco glue is thin app-layer services.
curated parameter signatures for the commonly-typed functions (what the registry can't supply).
**Services (app, store-aware via `getState`):** `spec-transform-actions` (the
wrap/simplify/add-view operations and their surfaces), `spec-dataset-hints` (data-column
completion, hover, inlay providers), `spec-expression-hints` (expression completion,
signature help, hover, and the diagnostic markers), `active-dataset` (`dataInfoAt(text,
offset)` — the columns/types/stats plus derived fields the draft sees at the cursor).
`SpecEditor` does the wiring.
wrap/simplify/add-view operations and their surfaces), `spec-transform-scaffold` (the
data-transform scaffold — a CodeLens plus a completion), `spec-dataset-hints` (data-column completion, hover,
inlay providers), `spec-expression-hints` (expression completion, signature help, hover, and
the diagnostic markers), `active-dataset` (`dataInfoAt(text, offset)` — the columns/types/stats
plus derived fields the draft sees at the cursor). `SpecEditor` does the wiring.
Decision rules:
@@ -407,7 +415,25 @@ Decision rules:
`datum.` or `fn(` spans the whole `datum.`/`fn(` token; used as a completion item's range it both
mis-targets the edit and filters every suggestion out (none start with `datum.`). A provider
completing inside a string must build the replace range from the partial it parses itself — a rule
any future in-string completion (transform/param scaffolding) inherits.
the transform scaffold inherits (it parses the trailing element word off the line). This
`new Range(line, col partialLen, line, col)` construction is now at three sites
(`spec-expression-hints`, `spec-dataset-hints`, `spec-transform-scaffold`); a shared
`replaceRange(position, partialLength)` helper is earned and should be extracted on the next touch.
- **Data-transform scaffolding is a CodeLens (discoverable) plus a completion (accelerator), on
the one home the schema leaves bare.** A Vega-Lite data `transform` has three possible homes,
confirmed against the bundled schema (the `transform` array is on 16 spec types, i.e. every view
node): a `transform[]` **step**; the **inline** field props on an encoding channel
(`bin`/`timeUnit`/`aggregate`/`sort`); and a **new** pipeline on a bare view. We scaffold the step
home only, on the same "add only what the schema lacks" rule as `spec-dataset-hints`: the schema
already completes the inline channel keys and their enum values, and the `transform` key itself —
but never a ready, field-typed `{ "filter": … }`. The **CodeLens is the discoverable surface**
(a completion is invisible until provoked and competes silently with the schema's suggest items):
cursor-scoped like the composition CodeLens, it shows ` Add transform` on a view with no pipeline
and per-step ` filter`/` aggregate`/… on the array, each clicking through Monaco's snippet
engine so the field-typed tab stops survive. The completion is the type-to-filter accelerator on
the same catalog. A step's `scope` (`shared` when the array is on a composition parent, so it
feeds every child) is surfaced so the placement is not a surprise, and comma affixing keeps the
array valid whether the slot is empty, between elements, or appended after one without a trailing comma.
- **Code-action menu icons are kind-derived** (a wrench for the `refactor.*` kinds) — Monaco's
`CodeAction` carries no icon field. Custom iconography lives only where it is supported:
CodeLens titles (`$(codicon)`), completion-item kinds, and glyph-margin decorations.
+25 -13
View File
@@ -50,6 +50,10 @@ import {
configureSpecExpressionHints,
installExpressionMarkers,
} from '../services/spec-expression-hints';
import {
configureSpecTransformScaffold,
installSpecTransformScaffoldCodeLens,
} from '../services/spec-transform-scaffold';
import { runExtract } from '../services/extract-action';
import { useAppStore } from '../stores/AppStore';
import { confirm } from '../stores/ConfirmStore';
@@ -173,6 +177,8 @@ configureSpecTransformCodeActions();
configureSpecDatasetHints();
// Register the expression completion/signature-help/hover providers once (docs/architecture/08).
configureSpecExpressionHints();
// Register the data-transform step-scaffold completion once (docs/architecture/08).
configureSpecTransformScaffold();
/** The two spec↔config operations, surfaced as an overflow menu (council:
* Carbon menu-buttons — overflow for additional options under space
@@ -198,11 +204,12 @@ const CONFIG_ACTIONS = [
type ConfigActionId = (typeof CONFIG_ACTIONS)[number]['value'];
/** Structural transforms, surfaced as a sibling menu to Config (the discoverable
/** Composition restructures, surfaced as a sibling menu to Config (the discoverable
* home; the lightbulb and F1 palette are the accelerators — see
* services/spec-transform-actions). They act on the selection, else the whole
* spec. */
const TRANSFORM_ACTIONS = [
* spec. Named "Compose" so it reads distinctly from a Vega-Lite data `transform`
* (the pipeline scaffolded by the editor completion — services/spec-transform-scaffold). */
const COMPOSE_ACTIONS = [
{ value: 'layer', label: 'Wrap in layer', detail: 'Overlay marks on shared scales' },
{ value: 'hconcat', label: 'Wrap in horizontal concat', detail: 'Place views side by side' },
{ value: 'vconcat', label: 'Wrap in vertical concat', detail: 'Stack views top to bottom' },
@@ -215,7 +222,7 @@ const TRANSFORM_ACTIONS = [
},
] as const;
type TransformActionId = (typeof TRANSFORM_ACTIONS)[number]['value'];
type ComposeActionId = (typeof COMPOSE_ACTIONS)[number]['value'];
function EditorToolbar({
editorRef,
@@ -280,7 +287,7 @@ function EditorToolbar({
else runExtractConfigToTheme(editor);
};
const handleTransformAction = (action: TransformActionId) => {
const handleComposeAction = (action: ComposeActionId) => {
const editor = editorRef.current;
if (!editor) return;
if (action === 'simplify') runUnwrap(editor);
@@ -344,13 +351,13 @@ function EditorToolbar({
</Button>
)}
<SelectControl
id="editor-transform-actions"
label="Spec transform actions"
heading="Transform"
options={TRANSFORM_ACTIONS}
onSelect={handleTransformAction}
triggerContent="Transform"
triggerTitle="Structural transforms — wrap the focused view in a composition, or simplify one"
id="editor-compose-actions"
label="Spec composition actions"
heading="Compose"
options={COMPOSE_ACTIONS}
onSelect={handleComposeAction}
triggerContent="Compose"
triggerTitle="Composition restructures — wrap the focused view in a layer/concat/facet/repeat, or simplify one"
disabled={activeId === null || editorView === 'published'}
/>
<SelectControl
@@ -447,7 +454,7 @@ export function SpecEditor() {
const configActionsSub = installSpecConfigActions(editor);
// Structural-transform actions in the F1 palette (the lightbulb is registered
// once, globally, above; the toolbar Transform menu is the home). Per editor,
// once, globally, above; the toolbar Compose menu is the home). Per editor,
// disposed below like the config actions.
const transformActionsSub = installSpecTransformActions(editor);
@@ -455,6 +462,10 @@ export function SpecEditor() {
// editor, because its commands need this editor's handle to apply the edit.
const codeLensSub = installSpecTransformCodeLens(editor);
// Cursor-aware data-transform scaffold CodeLens ( Add transform / filter …)
// — per editor for the same reason: its commands drive this editor's snippet edit.
const scaffoldLensSub = installSpecTransformScaffoldCodeLens(editor);
// Validate Vega expressions in the draft and squiggle the invalid ones — per
// editor, because it writes markers to this model (docs/architecture/08).
// Debounced internally; recomputes on edit and on a draft↔published toggle.
@@ -471,6 +482,7 @@ export function SpecEditor() {
configActionsSub.dispose();
transformActionsSub.dispose();
codeLensSub.dispose();
scaffoldLensSub.dispose();
exprMarkersSub.dispose();
editor.dispose();
editorRef.current = null;
+1 -1
View File
@@ -11,7 +11,7 @@
* back to the whole document for a flat unit spec with no inner view.
*
* **Surfacing** mirrors spec-config-actions' three-tier model:
* - the editor toolbar's **Transform menu** (SpecEditor) is the discoverable
* - the editor toolbar's **Compose menu** (SpecEditor) is the discoverable
* home, calling `runWrap` / `runUnwrap`;
* - the **lightbulb** (`configureSpecTransformCodeActions`, registered once,
* global per-language) offers the same transforms contextually at the cursor;
+264
View File
@@ -0,0 +1,264 @@
/**
* Data-transform scaffolding for the editor (docs/architecture/08 → editor
* augmentation) — the "quick transform" affordance. It offers the popular Vega-Lite
* transforms (filter, calculate, aggregate, bin, timeUnit, …) as ready-to-fill
* snippets, each seeded with a type-appropriate column from the data in scope. The
* facts the schema can't supply — *where a view's pipeline is* and *what a field-typed
* skeleton looks like* — are pure core (`core/spec-data-transforms`); this is the thin
* Monaco glue.
*
* **Two surfaces, mirroring the composition transforms (`spec-transform-actions`):**
* - the **CodeLens** (`installSpecTransformScaffoldCodeLens`, per editor — its
* commands need the editor handle) is the *discoverable* home: an ` Add
* transform` lens on a view with no pipeline, and per-step ` filter`/`
* aggregate`/… lenses on the array once it exists. Clicking splices the step in
* via Monaco's snippet engine, so the field-typed placeholders are Tab-through.
* - the **completion** (`configureSpecTransformScaffold`, global-once) is the
* type-to-filter *accelerator* on the same catalog, for when the cursor is
* already in a `transform[]` slot.
*
* Both are gated to the active draft — the published view is a read-only reference.
* The scaffold is deliberately the one home the schema leaves bare: the schema already
* completes the inline channel transforms (`bin`/`timeUnit` on an encoding field) and
* the `transform` key itself, so this adds no second source there.
*/
import * as monaco from 'monaco-editor/esm/vs/editor/edcore.main';
import {
arrayAffixes,
DATA_TRANSFORMS,
neighbour,
transformPlacementAt,
transformSiteAt,
} from '@core/spec-data-transforms';
import { boundColumnsAt } from './active-dataset';
import { useSnippetStore } from '../stores/SnippetStore';
/** The transforms surfaced as CodeLens buttons — the common few; the completion has the rest. */
const COMMON_STEP_IDS = ['filter', 'aggregate', 'calculate', 'bin', 'timeUnit'] as const;
// TODO: the cursor-aware CodeLens skeleton (the `lens()` factory + `onDidChange` emitter +
// cursor-keyed refresh + dispose) is ALREADY duplicated between this installer and
// `spec-transform-actions.installSpecTransformCodeLens` — two shipped sites. A shared
// `installCursorLens(editor, resolve, keyOf, lensesOf)` (~-55 LOC) is earned; deferred so it
// lands as its own change with both lenses live-verified together, not folded into this
// feature commit and its still-unexercised click path.
//
// TODO: `params` scaffolding is the natural sibling ( Add parameter → slider / dropdown /
// radio / point / interval). It reuses the CodeLens shape but NOT the transform internals:
// `params` isn't on every view node the way `transform` is (variable params are top-level;
// selection params are unit-only), so it needs its own site resolver, not `transformSiteAt`;
// its smart defaults come from column stats (slider min/max ← numericExtent, dropdown options
// ← distinct values) and the view's encodings, not the step catalog. Build as a separate
// reviewed change — that third consumer is the right moment to shape the shared skeleton above.
/** Monaco's snippet contribution — the public entry to insert a `${n:…}` template with tab stops. */
interface SnippetInserter extends monaco.editor.IEditorContribution {
insert(template: string): void;
}
/**
* Insert a snippet template at `offset` through Monaco's snippet engine, so its
* `${n:default}` tab stops become a live, Tab-through session. Places the cursor
* first (the controller inserts at the selection). No-op if the contribution is
* absent (it ships in `edcore.main`, so this is just defensive).
*/
function insertSnippetAt(
editor: monaco.editor.IStandaloneCodeEditor,
offset: number,
template: string,
): void {
const model = editor.getModel();
if (!model) return;
editor.setPosition(model.getPositionAt(offset));
editor.focus();
editor.getContribution<SnippetInserter>('snippetController2')?.insert(template);
}
let registered = false;
/** Register the `transform[]` step-scaffold completion provider once. */
export function configureSpecTransformScaffold(): void {
if (registered) return;
registered = true;
monaco.languages.registerCompletionItemProvider('json', {
provideCompletionItems(model, position) {
// Scaffolding only edits the draft; the published view is read-only.
if (useSnippetStore.getState().editorView !== 'draft') return { suggestions: [] };
const text = model.getValue();
const offset = model.getOffsetAt(position);
const placement = transformPlacementAt(text, offset);
if (placement.kind !== 'step') return { suggestions: [] };
// Replace the bare partial the user is typing as the element (e.g. `fil`),
// parsed from the line — never `getWordUntilPosition`, whose JSON wordPattern
// would reach across punctuation (docs/architecture/08 → completion ranges).
const before = model.getLineContent(position.lineNumber).slice(0, position.column - 1);
const partial = /[A-Za-z]*$/.exec(before)?.[0] ?? '';
const range = new monaco.Range(
position.lineNumber,
position.column - partial.length,
position.lineNumber,
position.column,
);
const { lead, trail } = arrayAffixes(text, offset - partial.length, offset);
const fields = boundColumnsAt(text, offset);
const shared = placement.scope === 'shared';
const suggestions = DATA_TRANSFORMS.map((t, i) => ({
label: t.label,
kind: monaco.languages.CompletionItemKind.Snippet,
detail: t.detail,
// A shared step (on a composition parent) transforms the data feeding every
// child view, not just one — surface that so the placement isn't a surprise.
documentation: shared
? {
value:
'_Applies to all views below — this `transform` sits on a composition parent._',
}
: undefined,
insertText: `${lead}${t.step(fields)}${trail}`,
insertTextRules: monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet,
range,
// Keep catalog order and float the scaffolds above the schema's generic items.
sortText: `0${String(i).padStart(2, '0')}`,
}));
return { suggestions };
},
});
}
/**
* Splice a field-typed step into the `transform` array at `arrayOffset`, via Monaco's
* snippet engine so the placeholders are Tab-through. Re-resolves the array from the
* live text (the lens args may be a version stale) and skips silently if it's gone.
*/
function runAddStep(
editor: monaco.editor.IStandaloneCodeEditor,
id: string,
arrayOffset: number,
): void {
const model = editor.getModel();
if (!model) return;
const text = model.getValue();
const site = transformSiteAt(text, arrayOffset + 1);
const transform = DATA_TRANSFORMS.find((t) => t.id === id);
if (!site?.array || !transform) return;
const insertOffset = site.array.offset + site.array.length - 1; // just before the ']'
const { lead, trail } = arrayAffixes(text, insertOffset, insertOffset);
const fields = boundColumnsAt(text, insertOffset);
insertSnippetAt(editor, insertOffset, `${lead}${transform.step(fields)}${trail}`);
}
/**
* Insert an empty `transform: []` as the view's first property and drop the cursor
* inside it (a `$0` tab stop), so the per-step lenses appear next. Indented off the
* view line's own nesting, and separated from the next property with a comma only
* when one follows — a view with no other key (a bare `{}`) takes none, so the result
* stays valid JSON. The snippet engine precludes a reformat, so the indent is a best
* effort the next document format tidies.
*/
function runAddTransform(editor: monaco.editor.IStandaloneCodeEditor, viewOffset: number): void {
const model = editor.getModel();
if (!model) return;
const text = model.getValue();
const site = transformSiteAt(text, viewOffset + 1);
if (!site || site.array) return; // gone, or already has a pipeline
const brace = site.view.offset; // the view object's opening '{'
const tab = model.getOptions().tabSize || 2;
// Indent off the brace *line's* leading whitespace, not the brace column — a view
// the compact formatter kept inline (mid-line '{') would otherwise over-indent.
const braceLine = model.getPositionAt(brace).lineNumber;
const baseIndent = /^\s*/.exec(model.getLineContent(braceLine))?.[0].length ?? 0;
const indent = ' '.repeat(baseIndent + tab);
const inner = ' '.repeat(tab);
const trail = neighbour(text, brace + 1, 1) === '}' ? '' : ',';
insertSnippetAt(
editor,
brace + 1,
`\n${indent}"transform": [\n${indent}${inner}$0\n${indent}]${trail}`,
);
}
/**
* Install the cursor-aware transform-scaffold CodeLens (per editor — its commands
* need this editor's handle to apply the edit, exactly like the composition CodeLens
* in `spec-transform-actions`). Over the view the cursor sits in it shows ` Add
* transform` when there is no pipeline, or ` filter`/` aggregate`/… on the array
* when there is. Returns a disposable; dispose on unmount.
*/
export function installSpecTransformScaffoldCodeLens(
editor: monaco.editor.IStandaloneCodeEditor,
): monaco.IDisposable {
const addTransformCmd = editor.addCommand(0, (_a, viewOffset: number) =>
runAddTransform(editor, viewOffset),
);
const addStepCmd = editor.addCommand(0, (_a, id: string, arrayOffset: number) =>
runAddStep(editor, id, arrayOffset),
);
const lens = (
line: number,
title: string,
id: string | null,
args: unknown[],
): monaco.languages.CodeLens => ({
range: new monaco.Range(line, 1, line, 1),
command: { id: id ?? '', title, arguments: args },
});
// The lenses follow the cursor's view, so signal a refresh when that changes —
// keyed to the view and its step count so typing within one view doesn't churn them.
const onDidChange = new monaco.Emitter<monaco.languages.CodeLensProvider>();
const provider: monaco.languages.CodeLensProvider = {
onDidChange: onDidChange.event,
provideCodeLenses(model) {
if (useSnippetStore.getState().editorView !== 'draft') return { lenses: [], dispose() {} };
const pos = editor.getPosition();
if (!pos) return { lenses: [], dispose() {} };
const site = transformSiteAt(model.getValue(), model.getOffsetAt(pos));
if (!site) return { lenses: [], dispose() {} };
const lenses: monaco.languages.CodeLens[] = [];
if (!site.array) {
const line = model.getPositionAt(site.view.offset).lineNumber;
lenses.push(lens(line, '$(add) Add transform', addTransformCmd, [site.view.offset]));
} else {
const line = model.getPositionAt(site.array.offset).lineNumber;
for (const id of COMMON_STEP_IDS) {
lenses.push(lens(line, `$(add) ${id}`, addStepCmd, [id, site.array.offset]));
}
}
return { lenses, dispose() {} };
},
};
const registration = monaco.languages.registerCodeLensProvider('json', provider);
let lastKey = '';
const cursorSub = editor.onDidChangeCursorPosition(() => {
const model = editor.getModel();
const pos = editor.getPosition();
const site =
model && pos && useSnippetStore.getState().editorView === 'draft'
? transformSiteAt(model.getValue(), model.getOffsetAt(pos))
: null;
const key = site ? JSON.stringify([site.view.offset, site.array?.count ?? -1]) : '';
if (key !== lastKey) {
lastKey = key;
onDidChange.fire(provider);
}
});
return {
dispose() {
cursorSub.dispose();
onDidChange.dispose();
registration.dispose();
},
};
}
+205
View File
@@ -0,0 +1,205 @@
import { describe, expect, it } from 'vitest';
import {
arrayAffixes,
DATA_TRANSFORMS,
snippetToPlain,
transformPlacementAt,
transformSiteAt,
type TypedField,
} from './spec-data-transforms';
/** Place a `|` marker in the source, strip it, and classify at that offset. */
function placeAt(marked: string) {
const offset = marked.indexOf('|');
return transformPlacementAt(marked.replace('|', ''), offset);
}
/** Place a `|` marker in the source, strip it, and resolve the transform site. */
function siteAt(marked: string) {
const offset = marked.indexOf('|');
return transformSiteAt(marked.replace('|', ''), offset);
}
const FIELDS: TypedField[] = [
{ name: 'price', type: 'number' },
{ name: 'date', type: 'date' },
{ name: 'origin', type: 'string' },
];
describe('transformPlacementAt — step slots', () => {
it('classifies an empty transform array as a step slot', () => {
expect(placeAt('{"transform":[|],"mark":"bar"}')).toEqual({ kind: 'step', scope: 'view' });
});
it('classifies the slot after an existing step', () => {
expect(placeAt('{"transform":[{"filter":"x"},|],"mark":"bar"}')).toEqual({
kind: 'step',
scope: 'view',
});
});
it('classifies a bare partial word in the array as a step slot', () => {
// `fil` is not valid JSON, but jsonc-parser locates the slot regardless.
expect(placeAt('{"transform":[fil|],"mark":"bar"}')).toEqual({ kind: 'step', scope: 'view' });
});
it('marks a transform on a composition parent as shared (upstream of children)', () => {
const src = '{"transform":[|],"layer":[{"mark":"bar"},{"mark":"line"}]}';
expect(placeAt(src)).toEqual({ kind: 'step', scope: 'shared' });
});
it('resolves the owning view of a nested transform (layer child = local)', () => {
const src = '{"layer":[{"transform":[|],"mark":"bar"},{"mark":"line"}]}';
expect(placeAt(src)).toEqual({ kind: 'step', scope: 'view' });
});
});
describe('transformPlacementAt — not a step slot', () => {
it('stays out of a key inside an existing step object (the schema owns that)', () => {
expect(placeAt('{"transform":[{"fil|"}],"mark":"bar"}')).toEqual({ kind: 'none' });
});
it('does not fire at an encoding channel key (the schema completes those)', () => {
expect(placeAt('{"encoding":{"x":{"field":"date","ti|"}}}')).toEqual({ kind: 'none' });
});
it('does not fire at a bare view key', () => {
expect(placeAt('{"mark":"bar","tr|"}')).toEqual({ kind: 'none' });
});
it('returns none for a plain value position', () => {
expect(placeAt('{"mark":"|"}')).toEqual({ kind: 'none' });
});
});
describe('transformSiteAt', () => {
it('reports no array on a bare unit view (cursor anywhere inside it)', () => {
const site = siteAt('{"mark":"bar","encoding":{"x":{"field":"a|"}}}');
expect(site).toMatchObject({ scope: 'view', array: null });
expect(site?.view.offset).toBe(0);
});
it('reports the array and its element count when a pipeline exists', () => {
const site = siteAt('{"transform":[{"filter":"x"}],"mark":"|bar"}');
expect(site?.array).toMatchObject({ count: 1 });
});
it('reports an empty array with count 0', () => {
const site = siteAt('{"transform":[|],"mark":"bar"}');
expect(site?.array).toMatchObject({ count: 0 });
});
it('resolves the site from inside the transform array itself', () => {
const site = siteAt('{"transform":[{"filter":"x"},|],"mark":"bar"}');
expect(site?.array).toMatchObject({ count: 1 });
expect(site?.view.offset).toBe(0);
});
it('climbs to the nearest view — a layer child is local (view scope)', () => {
const site = siteAt('{"layer":[{"mark":"ba|r"},{"mark":"line"}]}');
expect(site).toMatchObject({ scope: 'view', array: null });
expect(site?.view.offset).toBeGreaterThan(0); // the child, not the root
});
it('marks a composition parents pipeline as shared', () => {
const site = siteAt('{"transform":[|],"layer":[{"mark":"bar"}]}');
expect(site).toMatchObject({ scope: 'shared' });
});
it('the view range spans the whole enclosing view object', () => {
const src = '{"mark":"bar","encodi|ng":{}}';
const site = siteAt(src);
// offset at the opening brace, length to the closing brace inclusive.
expect(site?.view.offset).toBe(0);
expect(site?.view.length).toBe(src.replace('|', '').length);
});
});
describe('catalog — every step snippet is valid JSON once tab stops are stripped', () => {
for (const t of DATA_TRANSFORMS) {
it(`${t.id} (with fields in scope)`, () => {
expect(() => {
JSON.parse(snippetToPlain(t.step(FIELDS)));
}).not.toThrow();
});
it(`${t.id} (no data in scope — generic fallbacks)`, () => {
expect(() => {
JSON.parse(snippetToPlain(t.step([])));
}).not.toThrow();
});
}
});
describe('catalog — field-typed defaults', () => {
it('seeds filter with the first available column', () => {
expect(JSON.parse(snippetToPlain(byId('filter').step(FIELDS)))).toEqual({
filter: 'datum.price > 0',
});
});
it('seeds aggregate with a numeric field and a categorical groupby', () => {
const parsed = parseAggregate(byId('aggregate').step(FIELDS));
expect(parsed.aggregate[0]).toMatchObject({ field: 'price', as: 'price_mean' });
expect(parsed.groupby).toEqual(['origin']);
});
it('seeds timeUnit with a temporal field', () => {
const parsed = JSON.parse(snippetToPlain(byId('timeUnit').step(FIELDS))) as { field: string };
expect(parsed.field).toBe('date');
});
it('falls back to generic names when no column matches the need', () => {
const parsed = parseAggregate(byId('aggregate').step([]));
expect(parsed.aggregate[0].field).toBe('value');
expect(parsed.groupby).toEqual(['category']);
});
});
describe('arrayAffixes — keeps a spliced-in element valid JSON', () => {
const at = (marked: string) => {
const i = marked.indexOf('|');
return arrayAffixes(marked.replace('|', ''), i, i);
};
it('no commas in an empty array', () => {
expect(at('[|]')).toEqual({ lead: '', trail: '' });
});
it('leading comma when appended after an element', () => {
expect(at('[{"a":1}|]')).toEqual({ lead: ', ', trail: '' });
});
it('no leading comma right after a separator comma', () => {
expect(at('[{"a":1},|]')).toEqual({ lead: '', trail: '' });
});
it('trailing comma when inserted before an element', () => {
expect(at('[|{"a":1}]')).toEqual({ lead: '', trail: ',' });
});
it('both when squeezed between two elements without commas', () => {
expect(at('[{"a":1}|{"b":2}]')).toEqual({ lead: ', ', trail: ',' });
});
});
describe('snippetToPlain', () => {
it('reduces tab stops to their default text', () => {
expect(snippetToPlain('"a": "${1:x}", "b": ${2:true}')).toBe('"a": "x", "b": true');
});
it('drops an empty final tab stop', () => {
expect(snippetToPlain('foo${0}')).toBe('foo');
});
});
function byId(id: string) {
const t = DATA_TRANSFORMS.find((x) => x.id === id);
if (!t) throw new Error(`no transform ${id}`);
return t;
}
/** Parse an aggregate step snippet into its (typed) fields, for assertions. */
function parseAggregate(snippet: string): {
aggregate: Array<{ op: string; field: string; as: string }>;
groupby: string[];
} {
return JSON.parse(snippetToPlain(snippet)) as {
aggregate: Array<{ op: string; field: string; as: string }>;
groupby: string[];
};
}
+323
View File
@@ -0,0 +1,323 @@
/**
* Data-transform scaffolding (docs/architecture/08 → editor augmentation) — the
* pure counterpart of `spec-transforms`. Where `spec-transforms` rewrites a view's
* *structure* (wrap in a composition), this scaffolds a view's *data pipeline*: the
* Vega-Lite `transform` steps (filter, calculate, aggregate, …) the editor offers as
* ready-to-fill, field-typed placeholders. Text/Monaco handling (ranges, insertion,
* the CodeLens) is the service's job (app/services/spec-transform-scaffold); this
* owns the facts the schema can't supply — *where* a view's pipeline lives
* (`transformSiteAt`, for the lens) or a step slot sits (`transformPlacementAt`, for
* the completion), and *what* a field-typed skeleton for each transform looks like.
*
* The **discoverable surface is a CodeLens** — ` Add transform` on a view with no
* pipeline, and ` filter`/` aggregate`/… on the array once it exists — mirroring
* the composition CodeLens (`spec-transform-actions`). The step-slot completion is a
* type-to-filter accelerator on the same catalog, not the primary way in.
*
* **Where a data transform can legally go (grammar-confirmed against the bundled VL
* schema — the `transform` array appears on 16 spec types, which reduce to one idea:
* every view node).** Three contexts exist; we scaffold only the one the schema
* leaves bare, on the same "add only what the schema lacks" rule as `spec-dataset-hints`:
*
* - **step (owned here)** — an element of a view's `transform: [ … ]` array. The
* array sits on *every* view node (unit, layer, facet, concat, repeat, and the
* root) — the node set `spec-cursor`/`spec-data` already call a "view" — so it
* inherits top-down: a step on a composition parent is **shared** (upstream of
* all its children); a step on a unit is **local**. The schema never offers a
* ready `{ "filter": … }` with your columns, so this is where we add value.
* - **inline (deferred to the schema)** — `bin`/`timeUnit`/`aggregate`/`sort` are
* also properties on an encoding channel's field def. The JSON schema already
* completes those keys *and* their enum values, so scaffolding them here would be
* a redundant second source.
* - **new pipeline (deferred — composition of two that already work)** — starting a
* `transform` on a bare view is the schema completing the `transform` key (→ empty
* array) followed by the step completion firing inside it. No separate surface needed.
*
* `transformPlacementAt` classifies the cursor from `jsonc-parser`'s error-tolerant
* tree, so it holds mid-edit. The catalog builders take the columns in scope (resolved
* by the service via `active-dataset`, exactly as the facet/repeat wraps do) and seed
* each placeholder with a type-appropriate default.
*/
import {
findNodeAtLocation,
findNodeAtOffset,
getLocation,
parseTree,
type Node,
} from 'jsonc-parser';
import { ARRAY_COMPOSITIONS } from './spec-transforms';
import type { ColumnType } from './type-inference';
/** A column in scope at the cursor, with its inferred data type. */
export interface TypedField {
name: string;
type: ColumnType;
}
/** A `transform` step belongs to a composition parent (shared) or a unit (local). */
type TransformScope = 'shared' | 'view';
/**
* Where the cursor sits relative to a data transform:
* - `step` — in a `transform: [ … ]` element slot (offer the field-typed catalog),
* tagged with whether that array is a composition parent's (shared) or a unit's;
* - `none` — anywhere else (including inside an existing step object, where the
* schema owns key completion).
*/
export type TransformPlacement = { kind: 'step'; scope: TransformScope } | { kind: 'none' };
/** A byte range into the spec text. */
interface ByteRange {
offset: number;
length: number;
}
/**
* The data-transform site for the view the cursor is in — what the CodeLens needs to
* anchor and act. `view` is the enclosing view object's range (where an ` Add
* transform` lens sits when there is no pipeline yet); `array` is the `transform[]`
* range and element count when one exists (where the per-step ` filter` … lenses
* sit, and where a new step is spliced in). `scope` marks a composition parent's
* pipeline as shared.
*/
export interface TransformSite {
scope: TransformScope;
view: ByteRange;
array: (ByteRange & { count: number }) | null;
}
/** Composition keys whose presence on a view makes its transform *shared* upstream —
* the array operators (shared with `spec-cursor`/`spec-insert`) plus the single-child
* `facet`/`repeat`, so this stays one spelling of the operator set. */
const COMPOSITION_KEYS = [...ARRAY_COMPOSITIONS, 'facet', 'repeat'];
/** The object node at a jsonc location path, or null if the path isn't an object. */
function objectNodeAt(tree: Node | undefined, path: ReadonlyArray<string | number>): Node | null {
if (!tree) return null;
const node = findNodeAtLocation(tree, path as (string | number)[]);
return node && node.type === 'object' ? node : null;
}
/** The property keys of an object node. */
function objectKeys(node: Node | null): string[] {
if (!node?.children) return [];
const keys: string[] = [];
for (const prop of node.children) {
const key = prop.type === 'property' ? prop.children?.[0] : undefined;
if (key?.type === 'string' && typeof key.value === 'string') keys.push(key.value);
}
return keys;
}
/** A view carrying a composition key feeds all its children, so its transform is shared. */
function scopeFromKeys(keys: readonly string[]): TransformScope {
return keys.some((k) => COMPOSITION_KEYS.includes(k)) ? 'shared' : 'view';
}
/**
* Classify the cursor's data-transform context. Read from `jsonc-parser`'s
* error-tolerant tree so it holds mid-edit — a step slot is a `transform` array
* element position (`[ …, "transform", <index> ]`) that is *not* a property key,
* which excludes the cursor being inside a step object editing its own keys (the
* schema owns that). `scope` reads the owning view's keys, defaulting to `view` when
* the tree can't resolve them.
*/
export function transformPlacementAt(text: string, offset: number): TransformPlacement {
const { path, isAtPropertyKey } = getLocation(text, offset);
if (
isAtPropertyKey ||
path.length < 2 ||
path[path.length - 2] !== 'transform' ||
typeof path[path.length - 1] !== 'number'
) {
return { kind: 'none' };
}
const owner = objectNodeAt(parseTree(text), path.slice(0, -2));
return { kind: 'step', scope: scopeFromKeys(objectKeys(owner)) };
}
/** Is this object node a view a `transform` can attach to — the root, a composition, or a unit? */
function isViewNode(node: Node): boolean {
if (!node.parent) return true; // the root spec
const keys = objectKeys(node);
if (keys.some((k) => COMPOSITION_KEYS.includes(k))) return true;
return keys.includes('mark') || keys.includes('encoding');
}
/** The `transform` array node of a view object, or null when it has none. */
function transformArrayNode(view: Node): Node | null {
for (const prop of view.children ?? []) {
if (prop.type === 'property' && prop.children?.[0]?.value === 'transform') {
const value = prop.children[1];
return value?.type === 'array' ? value : null;
}
}
return null;
}
/**
* The transform site for the nearest view enclosing `offset` — climbed from the
* error-tolerant tree (so it holds mid-edit), or null when the cursor is in no view.
* The CodeLens reads this to place its lenses and to splice a new step; the byte
* ranges are turned into Monaco positions by the service.
*/
export function transformSiteAt(text: string, offset: number): TransformSite | null {
const tree = parseTree(text);
if (!tree) return null;
let node: Node | undefined = findNodeAtOffset(tree, offset, true);
while (node) {
if (node.type === 'object' && isViewNode(node)) break;
node = node.parent;
}
if (!node) return null;
const array = transformArrayNode(node);
return {
scope: scopeFromKeys(objectKeys(node)),
view: { offset: node.offset, length: node.length },
array: array
? { offset: array.offset, length: array.length, count: array.children?.length ?? 0 }
: null,
};
}
// ─── Field-typed default picking ────────────────────────────────────────────────
/** What kind of column a placeholder wants, so its default is type-appropriate. */
type FieldNeed = 'any' | 'numeric' | 'temporal' | 'categorical';
/** The first column matching a need, or null when none is in scope. */
function pick(fields: readonly TypedField[], need: FieldNeed): string | null {
switch (need) {
case 'numeric':
return fields.find((f) => f.type === 'number')?.name ?? null;
case 'temporal':
return fields.find((f) => f.type === 'date')?.name ?? null;
case 'categorical':
return fields.find((f) => f.type === 'string' || f.type === 'boolean')?.name ?? null;
case 'any':
return fields[0]?.name ?? null;
}
}
/** A tab-stop's default text: the resolved column when one is in scope, else a name. */
function def(fields: readonly TypedField[], need: FieldNeed, fallback: string): string {
return pick(fields, need) ?? fallback;
}
// ─── The transform catalog ──────────────────────────────────────────────────────
/** One data transform the editor can scaffold as a `transform[]` step. */
export interface DataTransform {
id: string;
/** Completion label. */
label: string;
/** One-line description (completion detail). */
detail: string;
/** The step snippet: compact JSON with `${n:default}` tab stops. */
step(fields: readonly TypedField[]): string;
}
/** The scaffoldable transforms, in rough order of how often they're reached for. */
export const DATA_TRANSFORMS: readonly DataTransform[] = [
{
id: 'filter',
label: 'filter',
detail: 'Keep only rows matching a predicate',
step: (f) => `{ "filter": "datum.\${1:${def(f, 'any', 'field')}} > \${2:0}" }`,
},
{
id: 'calculate',
label: 'calculate',
detail: 'Derive a new field from an expression',
step: (f) => `{ "calculate": "datum.\${1:${def(f, 'any', 'field')}}", "as": "\${2:newField}" }`,
},
{
id: 'aggregate',
label: 'aggregate',
detail: 'Group rows and summarize (mean, sum, count…)',
step: (f) =>
`{ "aggregate": [{ "op": "\${1:mean}", "field": "\${2:${def(f, 'numeric', 'value')}}", "as": "\${3:${def(f, 'numeric', 'value')}_mean}" }], "groupby": ["\${4:${def(f, 'categorical', 'category')}}"] }`,
},
{
id: 'bin',
label: 'bin',
detail: 'Bucket a quantitative field into bins',
step: (f) =>
`{ "bin": \${1:true}, "field": "\${2:${def(f, 'numeric', 'value')}}", "as": "\${3:${def(f, 'numeric', 'value')}_binned}" }`,
},
{
id: 'timeUnit',
label: 'timeUnit',
detail: 'Derive a calendar unit (year, month…) from a date',
step: (f) =>
`{ "timeUnit": "\${1:yearmonth}", "field": "\${2:${def(f, 'temporal', 'date')}}", "as": "\${3:${def(f, 'temporal', 'date')}_unit}" }`,
},
{
id: 'window',
label: 'window',
detail: 'Running totals, ranks, row numbers over a sort',
step: (f) =>
`{ "window": [{ "op": "\${1:row_number}", "as": "\${2:rank}" }], "sort": [{ "field": "\${3:${def(f, 'any', 'field')}}", "order": "descending" }] }`,
},
{
id: 'joinaggregate',
label: 'joinaggregate',
detail: 'Group aggregate that keeps every row (share of total)',
step: (f) =>
`{ "joinaggregate": [{ "op": "\${1:sum}", "field": "\${2:${def(f, 'numeric', 'value')}}", "as": "\${3:total}" }], "groupby": ["\${4:${def(f, 'categorical', 'category')}}"] }`,
},
{
id: 'fold',
label: 'fold',
detail: 'Reshape wide columns into key/value rows',
step: (f) =>
`{ "fold": ["\${1:${f[0]?.name ?? 'field1'}}", "\${2:${f[1]?.name ?? 'field2'}}"], "as": ["\${3:key}", "\${4:value}"] }`,
},
{
id: 'lookup',
label: 'lookup',
detail: 'Join fields from another dataset by a key',
step: () =>
`{ "lookup": "\${1:key}", "from": { "data": { "name": "\${2:dataset}" }, "key": "\${3:key}", "fields": ["\${4:field}"] } }`,
},
];
/**
* The plain form of a snippet: strip tab stops down to their default text
* (`${1:price}` → `price`, `${0}` → ''). Keeps the catalog testable as valid JSON
* without a Monaco snippet engine.
*/
export function snippetToPlain(snippet: string): string {
return snippet.replace(
/\$\{(\d+)(?::([^}]*))?\}/g,
(_m: string, _n: string, defText?: string) => defText ?? '',
);
}
/** The whitespace-skipping neighbour of an offset, in the given direction (or '' at the edge). */
export function neighbour(text: string, from: number, step: -1 | 1): string {
let i = from;
while (i >= 0 && i < text.length && /\s/.test(text[i])) i += step;
return i >= 0 && i < text.length ? text[i] : '';
}
/**
* Leading/trailing commas needed to splice a new element in at `[start, end)` of a
* JSON array: a comma before it unless the preceding element already ends in one (or
* it is the first), and after it unless the following char closes the array (or a
* comma is already there). Keeps the array valid whether the slot is empty, between
* elements, or appended after one with no trailing comma. Pure text math — lives here
* (with `snippetToPlain`) so its edge cases are unit-testable off the Monaco glue.
*/
export function arrayAffixes(
text: string,
start: number,
end: number,
): { lead: string; trail: string } {
const prev = neighbour(text, start - 1, -1);
const next = neighbour(text, end, 1);
const lead = prev !== '' && prev !== '[' && prev !== ',' ? ', ' : '';
const trail = next !== '' && next !== ']' && next !== ',' ? ',' : '';
return { lead, trail };
}