Track datasetRefs against the live draft so manual edits and extract link immediately

docs/architecture/07-naming-and-relationships.md

@@ -125,9 +125,15 @@ export function makeUniqueName(desired: string, existingNames: Iterable<string>)
   it by scanning snippets. Keeping it _derived_ means it can never disagree with
   the forward links — there is one source of truth.
 
-`datasetRefs` is **derived from the spec**, not hand-maintained. It is
+`datasetRefs` is **derived from the spec**, not hand-maintained. It mirrors the
-recomputed whenever a snippet is published (its draft spec is promoted), so it
+dataset names referenced by the **draft** spec — the version being edited — and
-always mirrors the dataset names actually referenced in the published spec.
+is recomputed on every change to the draft (auto-save, the Extract-to-Dataset
+rewrite, revert) and on publish. Tracking the draft (not only the last publish)
+keeps a snippet's linked-datasets display and the reverse lookup in step with
+what the editor shows — so a hand-typed reference or an Extract links its dataset
+without waiting for a publish. Recomputation runs only on a _valid_ spec —
+auto-save is debounced and parse-gated (spec §03B) — so a transiently-invalid
+draft never disturbs the links.
 
 ### 3.1 Extracting referenced names from a spec (pure — `src/core/spec-refs.ts`)
 
@@ -186,7 +192,7 @@ function safeParse(s: string): Json {
 ```
 
 ```ts
-// src/core/spec-refs.ts — thin wrapper used at publish time
+// src/core/spec-refs.ts — thin wrapper, run on every draft change and on publish
 
 /** The list stored on snippet.datasetRefs. Sorted + de-duped for stable diffs. */
 export function recomputeDatasetRefs(spec: Json): string[] {
@@ -202,8 +208,11 @@ export function recomputeDatasetRefs(spec: Json): string[] {
 - Treat `extractDatasetRefs` as the single source of truth for "what does this
   spec reference". The reverse-lookup and rename paths both depend on it
   agreeing with what the renderer actually resolves.
-- Recompute and store `datasetRefs` on **publish**, not on every keystroke —
+- Recompute and store `datasetRefs` on **every draft change and on publish** —
-  the draft can be transiently invalid, and only the published spec is shared.
+  but only through the parse-gated, debounced auto-save (`commitDraft`) and the
+  programmatic extract/revert rewrites, never on raw keystrokes. That keeps the
+  links in step with the edited draft while never recomputing from a
+  transiently-invalid spec.
 - Prune the **same two keys** (`data`, `datasets`) in all three ref walks —
   extraction here, `renameDatasetInSpec`, and the renderer's `resolveDatasetRefs`
   (`src/core/rendering.ts`). They must agree on what counts as a reference; if one
@@ -264,8 +273,9 @@ export function findSnippetsReferencingDataset(name: string): Snippet[] {
 
 The reactive UI does **not** go through the service: a component subscribed to
 `snippets` calls the core helper directly, so the badge and Linked Snippets list
-update the moment any snippet is published with changed refs — no duplicated
+update the moment any snippet's draft changes its refs (auto-save) or it is
-matching logic, no `getState()` snapshot that would miss updates.
+published — no duplicated matching logic, no `getState()` snapshot that would
+miss updates.
 
 **Do**
 
@@ -375,8 +385,9 @@ one atomic step:
 2. For **every snippet referencing the old name**: rewrite the named-data
    references inside its spec (`{ "data": { "name": "old" } }` →
    `{ "data": { "name": "new" } }`) — in **both** `spec` and `draftSpec`.
-3. Recompute that snippet's `datasetRefs` from the rewritten spec, so the
+3. Recompute that snippet's `datasetRefs` from the rewritten **draft** spec (the
-   forward link mirrors reality and the reverse scan stays correct.
+   tracked surface), so the forward link mirrors reality and the reverse scan
+   stays correct.
 
 The spec rewrite is pure; the orchestration reads and writes stores.
 
@@ -417,42 +428,34 @@ export function renameDatasetInSpec(spec: Json, oldName: string, newName: string
 ```
 
 ```ts
-// src/app/services/RelationshipService.ts — store coordination
+// src/app/services/RelationshipService.ts — thin store coordinator
 
-import { isNameTaken, makeUniqueName } from '../../core/naming';
-import { renameDatasetInSpec, recomputeDatasetRefs } from '../../core/spec-refs';
 import { useDatasetStore } from '../stores/DatasetStore';
 import { useSnippetStore } from '../stores/SnippetStore';
 
 /**
  * Renames a dataset and propagates the rename to every referencing snippet
- * (spec, draftSpec, and datasetRefs). Returns the snippets that changed.
+ * (spec, draftSpec, and datasetRefs). Returns the number of snippets changed.
  * Caller is responsible for collision policy on `newName` (reject vs suffix).
  */
-export function renameDatasetEverywhere(oldName: string, newName: string): { updated: number } {
+export function renameDatasetEverywhere(
+  oldName: string,
+  newName: string,
+  now?: Date,
+): { updated: number } {
   if (oldName === newName) return { updated: 0 };
 
   // 1. Rename the dataset record itself.
   const dataset = useDatasetStore.getState().datasets.find((d) => d.name === oldName);
-  if (!dataset) return { updated: 0 };
+  if (dataset) useDatasetStore.getState().update(dataset.id, { name: newName }, now);
-  useDatasetStore.getState().update(dataset.id, { name: newName });
 
-  // 2 + 3. Rewrite every referencing snippet's specs and refs.
+  // 2 + 3. Delegate the spec/draftSpec/refs rewrite to the ONE rename impl — the
-  let updated = 0;
+  // store action that owns the snippet collection. It matches by spec+draft
-  for (const snippet of useSnippetStore.getState().snippets) {
+  // *content* (not `datasetRefs`, which mirrors only the draft) and recomputes
-    if (!snippet.datasetRefs.some((r) => r.toLowerCase() === oldName.toLowerCase())) continue;
+  // `datasetRefs` from the rewritten draft. Keeping the loop in the store means
-
+  // the reactive editor buffer is refreshed in the same atomic update when the
-    const spec = renameDatasetInSpec(snippet.spec, oldName, newName);
+  // active snippet's draft is rewritten.
-    const draftSpec = renameDatasetInSpec(snippet.draftSpec, oldName, newName);
+  return { updated: useSnippetStore.getState().renameDatasetRefs(oldName, newName, now) };
-    useSnippetStore.getState().update(snippet.id, {
-      spec,
-      draftSpec,
-      datasetRefs: recomputeDatasetRefs(spec),
-    });
-    updated++;
-  }
-
-  return { updated };
 }
 ```
 
@@ -466,10 +469,17 @@ export function renameDatasetEverywhere(oldName: string, newName: string): { upd
 
 - Rewrite `spec` **and** `draftSpec`. A user mid-edit must not see their draft
   silently break because the dataset was renamed underneath them.
-- Recompute `datasetRefs` from the rewritten spec rather than string-replacing
+- Recompute `datasetRefs` from the rewritten **draft** spec rather than
-  the array — the spec is the source of truth, the array is its mirror.
+  string-replacing the array — the draft is the source of truth, the array is
-- Use the §4 reverse lookup to find affected snippets, so "who references this"
+  its mirror.
-  has exactly one implementation.
+- Find affected snippets by scanning each one's **spec and draft content** for
+  the old name (`extractDatasetRefs`), not by its `datasetRefs` array. Because
+  `datasetRefs` mirrors the draft, a name referenced only by the still-published
+  spec (the user removed it from the draft but hasn't published) is absent from
+  the array; matching on content rewrites it anyway, so the published spec can't
+  rot to a renamed-away dataset. Scan the content (don't reserialize) so a
+  non-referencing snippet's text stays byte-for-byte intact. The import-side
+  `applyDatasetRenamesToSnippets` already follows this spec-content rule (§5.1).
 
 **Don't**
 
@@ -494,6 +504,6 @@ export function renameDatasetEverywhere(oldName: string, newName: string): { upd
 
 The dividing line: anything that takes plain data and returns plain data is
 **core** and unit-tested in isolation; anything that reaches into a Zustand store
-is an **app service**. The rename rule of thumb — _the spec is the source of
+is an **app service**. The rule of thumb — _the **draft** spec is the source of
 truth, `datasetRefs` mirrors it, the reverse lookup is derived_ — is what keeps
 the bidirectional link from ever needing manual repair.

docs/spec/03-editor-and-drafts.md

@@ -45,7 +45,7 @@ Every snippet carries two versions of its spec: a **published** (stable) version
 
 - A **Publish** action promotes the current draft to become the published version (the two are made identical).
 - Publish is also triggered by the keyboard shortcut Cmd/Ctrl+S.
-- On publish, the snippet's dataset references are recomputed from the now-published spec (see _Datasets_ for reference linking).
+- The snippet's dataset references track the draft continuously (recomputed on auto-save, extract, and revert), so publish needs no special reference handling — promoting the draft simply carries the already-current references onto the published version (see _Datasets_ for reference linking).
 - A success toast confirms the snippet was published.
 - Publish is unavailable when no snippet is active.
 
@@ -72,7 +72,7 @@ When a snippet's spec embeds its data inline, the user can lift that data out in
 - When the active snippet's draft spec contains inline data, an **Extract to Dataset** action is available in the pane header; it is hidden when the spec has no inline data.
 - Choosing it opens a modal that shows a read-only preview of the inline data and asks the user for a dataset name (required).
 - The user enters a name and confirms creation. Names must be non-empty and unique; if the name is blank or already in use, the modal shows an inline error and the action does not proceed.
-- On success, the system: saves the inline data as a new dataset (preserving its detected format), rewrites the snippet's draft spec so the inline data is replaced by a reference to the dataset by name, links the dataset to the snippet, and reloads the editor to show the rewritten spec.
+- On success, the system: saves the inline data as a new dataset (preserving its detected format), rewrites the snippet's draft spec so the inline data is replaced by a reference to the dataset by name, links the dataset to the snippet **immediately** (the link appears without waiting for a publish, since references track the draft), and reloads the editor to show the rewritten spec. Reverting the draft before publishing removes the link again.
 - A toast confirms the dataset was created, and the modal closes.
 - The user can cancel the modal at any time, leaving the spec unchanged.
 - Dataset-side specifics (formats, storage, the bidirectional snippet↔dataset link) are described in _Datasets_.

docs/spec/09-data-model.md

@@ -28,7 +28,7 @@ A snippet carries two specs at once. `draftSpec` is the editable working copy; `
 
 ### datasetRefs
 
-`datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see _Cross-entity relationships_). It is maintained to mirror the dataset names actually referenced in the spec.
+`datasetRefs` records the **names** of datasets the spec depends on. It is the link used to display a snippet's linked datasets and, conversely, to find which snippets use a given dataset (see _Cross-entity relationships_). It mirrors the dataset names referenced by the **draft** spec — the version the user is editing — and is recomputed on every change to the draft (auto-save, the Extract-to-Dataset rewrite, revert) and on publish. Tracking the draft means a snippet's linked datasets reflect what the editor currently shows, not only the last published version; recomputation only ever runs on a valid (parseable) spec, so a half-typed draft never disturbs the links.
 
 ## B. Dataset
 

src/app/services/RelationshipService.ts

@@ -2,8 +2,9 @@
  * Snippet ↔ dataset relationships (docs/architecture/07 §4 + §6).
  *
  * The bidirectional link is name-based and has a single source of truth: a
- * snippet's `datasetRefs` (recomputed from its published spec). The reverse
+ * snippet's `datasetRefs` (recomputed from its draft spec — the version being
- * direction — "which snippets use this dataset" — is therefore DERIVED by a scan,
+ * edited — on every draft change and on publish). The reverse direction —
+ * "which snippets use this dataset" — is therefore DERIVED by a scan,
  * never stored, so it can't drift. Rename is the one graph operation: it renames
  * the dataset record and propagates the new name into every referencing snippet's
  * spec/draftSpec/refs.

src/app/stores/SnippetStore.test.ts

@@ -254,6 +254,69 @@ describe('publish — datasetRefs recomputation', () => {
   });
 });
 
+describe('datasetRefs track the live draft (spec §03F, arch 07 §3)', () => {
+  const refSpec = (name: string) => JSON.stringify({ data: { name }, mark: 'bar' }, null, 2);
+  const inlineSpec = JSON.stringify({ data: { values: [{ a: 1 }] }, mark: 'bar' }, null, 2);
+
+  test('createSnippet links a named-data spec immediately (Chart Builder path)', () => {
+    store().createSnippet({
+      id: 'a',
+      spec: refSpec('Sales'),
+      now: new Date('2026-01-01T00:00:00Z'),
+    });
+    expect(selectActiveSnippet(store())?.datasetRefs).toEqual(['Sales']);
+  });
+
+  test('auto-save links a hand-typed reference without publishing', () => {
+    const a = createSnippet({ id: 'a', spec: '{}', now: new Date('2026-01-01T00:00:00Z') });
+    store().hydrate([a], 'a');
+
+    store().updateDraft(refSpec('Sales'));
+    store().commitDraft(new Date('2026-02-01T00:00:00Z')); // the debounced auto-save
+
+    const s = selectActiveSnippet(store())!;
+    expect(s.datasetRefs).toEqual(['Sales']); // linked before any publish
+    expect(s.spec).toBe('{}'); // published spec untouched — refs mirror the draft
+  });
+
+  test('replaceActiveDraft (Extract to Dataset) links the dataset immediately', () => {
+    const a = createSnippet({ id: 'a', spec: inlineSpec, now: new Date('2026-01-01T00:00:00Z') });
+    store().hydrate([a], 'a');
+    expect(selectActiveSnippet(store())?.datasetRefs).toEqual([]); // inline data → no refs
+
+    store().replaceActiveDraft(refSpec('Orders'), new Date('2026-02-01T00:00:00Z'));
+
+    const s = selectActiveSnippet(store())!;
+    expect(s.datasetRefs).toEqual(['Orders']); // linked the moment the draft is rewritten
+    expect(s.spec).toBe(inlineSpec); // still a draft edit — published unchanged until publish
+  });
+
+  test('revert drops a link that existed only in the discarded draft', () => {
+    const a = createSnippet({ id: 'a', spec: inlineSpec, now: new Date('2026-01-01T00:00:00Z') });
+    store().hydrate([a], 'a');
+    store().replaceActiveDraft(refSpec('Orders'), new Date('2026-02-01T00:00:00Z'));
+    expect(selectActiveSnippet(store())?.datasetRefs).toEqual(['Orders']);
+
+    store().revert(new Date('2026-03-01T00:00:00Z'));
+
+    const s = selectActiveSnippet(store())!;
+    expect(s.datasetRefs).toEqual([]); // draft restored to the inline-data published spec
+    expect(s.draftSpec).toBe(inlineSpec);
+  });
+
+  test('an unparseable draft is skipped — refs keep their last valid value (no flicker)', () => {
+    const a = createSnippet({ id: 'a', spec: '{}', now: new Date('2026-01-01T00:00:00Z') });
+    store().hydrate([a], 'a');
+    store().updateDraft(refSpec('Sales'));
+    store().commitDraft(new Date('2026-01-02T00:00:00Z'));
+    expect(selectActiveSnippet(store())?.datasetRefs).toEqual(['Sales']);
+
+    store().updateDraft('{ "data": { "name": "Sa'); // mid-keystroke, invalid JSON
+    expect(store().commitDraft(new Date('2026-01-03T00:00:00Z'))).toBe(false);
+    expect(selectActiveSnippet(store())?.datasetRefs).toEqual(['Sales']); // unchanged
+  });
+});
+
 describe('renameSnippet (spec §02 metadata panel)', () => {
   test('renames and advances modified, without touching the editor buffer', () => {
     const a = createSnippet({ id: 'a', name: 'Old', now: new Date('2026-01-01T00:00:00Z') });
@@ -409,6 +472,28 @@ describe('renameDatasetRefs', () => {
     expect(store().renameDatasetRefs('Sales', 'Sales')).toBe(0);
     expect(store().renameDatasetRefs('Unknown', 'Other')).toBe(0);
   });
+
+  test('rewrites a name referenced only by the published spec (draft removed it)', () => {
+    const a = createSnippet({
+      id: 'a',
+      spec: refSpec('Sales'),
+      now: new Date('2026-01-01T00:00:00Z'),
+    });
+    store().hydrate([a], 'a');
+    store().publish(new Date('2026-01-02T00:00:00Z')); // spec + draft reference Sales
+    // Remove the reference from the DRAFT only — do not publish. datasetRefs now
+    // mirrors the (empty) draft, but the published spec still references Sales.
+    store().updateDraft('{"mark":"bar"}');
+    store().commitDraft(new Date('2026-01-03T00:00:00Z'));
+    expect(selectActiveSnippet(store())?.datasetRefs).toEqual([]);
+
+    const updated = store().renameDatasetRefs('Sales', 'Revenue', new Date('2026-02-01T00:00:00Z'));
+
+    expect(updated).toBe(1); // matched via the published spec, not datasetRefs
+    const renamed = store().snippets.find((s) => s.id === 'a')!;
+    expect(renamed.spec).toContain('"Revenue"'); // published spec rewritten — no rot
+    expect(renamed.spec).not.toContain('"Sales"');
+  });
 });
 
 describe('library view state (spec §02 → Search / Sort)', () => {

src/app/stores/SnippetStore.ts

@@ -24,7 +24,7 @@ import {
   type CreateSnippetOptions,
   type Snippet,
 } from '@core/snippet';
-import { recomputeDatasetRefs, renameDatasetInSpec } from '@core/spec-refs';
+import { extractDatasetRefs, recomputeDatasetRefs, renameDatasetInSpec } from '@core/spec-refs';
 import {
   DEFAULT_SORT_BY,
   DEFAULT_SORT_ORDER,
@@ -204,10 +204,13 @@ export const useSnippetStore = create<SnippetState>((set, get) => ({
   createSnippet: (options) => {
     get().commitDraft(); // flush the outgoing snippet's valid edits before switching away
     const created = createSnippet(options);
-    // Mirror datasetRefs from the spec at creation, like publish does, so a snippet
+    // Mirror datasetRefs from the draft at creation, so a snippet built with a
-    // built with a named-data reference (Chart Builder, §06) is linked to its dataset
+    // named-data reference (Chart Builder, §06) is linked to its dataset immediately.
-    // immediately. Inline-data specs (the sample template) resolve to no refs.
+    // datasetRefs tracks the draft — the version being edited — at every mutation
-    const snippet = { ...created, datasetRefs: recomputeDatasetRefs(created.spec) };
+    // (create/auto-save/extract/revert/publish), so the link reflects what the user
+    // sees, not only the last publish (docs/architecture/07 §3). Inline-data specs
+    // (the sample template) resolve to no refs. spec === draftSpec at creation.
+    const snippet = { ...created, datasetRefs: recomputeDatasetRefs(created.draftSpec) };
     set((s) => ({
       snippets: [snippet, ...s.snippets],
       activeSnippetId: snippet.id,
@@ -314,9 +317,13 @@ export const useSnippetStore = create<SnippetState>((set, get) => ({
     const { activeSnippetId } = get();
     if (!activeSnippetId) return;
     const modified = (now ?? new Date()).toISOString();
+    const datasetRefs = recomputeDatasetRefs(text);
     set((s) => ({
       snippets: s.snippets.map((x) =>
-        x.id === activeSnippetId ? { ...x, draftSpec: text, modified } : x,
+        // Recompute datasetRefs from the new draft so a programmatic rewrite that
+        // introduces a by-name reference — Extract-to-Dataset (spec §03F) — links
+        // the dataset to the snippet immediately, not only on the next publish.
+        x.id === activeSnippetId ? { ...x, draftSpec: text, datasetRefs, modified } : x,
       ),
       draftText: text,
       editorView: 'draft',
@@ -341,9 +348,15 @@ export const useSnippetStore = create<SnippetState>((set, get) => ({
     }
 
     const modified = (now ?? new Date()).toISOString();
+    // Recompute datasetRefs from the just-committed draft so a hand-typed by-name
+    // reference links to its dataset as soon as auto-save fires. This runs on the
+    // debounced auto-save and only on valid JSON (the parse guard above), so it is
+    // not "every keystroke" and never sees a transiently-invalid draft — it keeps
+    // datasetRefs mirroring the draft the user is editing (docs/architecture/07 §3).
+    const datasetRefs = recomputeDatasetRefs(draftText);
     set({
       snippets: snippets.map((s) =>
-        s.id === activeSnippetId ? { ...s, draftSpec: draftText, modified } : s,
+        s.id === activeSnippetId ? { ...s, draftSpec: draftText, datasetRefs, modified } : s,
       ),
     });
     return true;
@@ -358,8 +371,18 @@ export const useSnippetStore = create<SnippetState>((set, get) => ({
     let updated = 0;
     let activeDraftAfter: string | null = null;
 
+    // Whether a spec text references the old name (case-insensitive). Reads the
+    // content rather than reserializing, so a non-referencing snippet's text is
+    // left byte-for-byte intact (no spurious reformat/modified bump).
+    const referencesOld = (text: string): boolean =>
+      extractDatasetRefs(text).some((r) => r.toLowerCase() === lower);
+
     const next = snippets.map((s) => {
-      if (!s.datasetRefs.some((r) => r.toLowerCase() === lower)) return s;
+      // Consult both spec and draft, not only datasetRefs: datasetRefs now mirrors
+      // the DRAFT, so a name still referenced only by the published spec (the user
+      // removed it from the draft but hasn't published) would otherwise be missed
+      // and the published spec would rot to a now-renamed dataset (arch 07 §6).
+      if (!referencesOld(s.spec) && !referencesOld(s.draftSpec)) return s;
       updated++;
       const spec = renameDatasetInSpec(s.spec, oldName, newName);
       const draftSpec = renameDatasetInSpec(s.draftSpec, oldName, newName);
@@ -368,7 +391,7 @@ export const useSnippetStore = create<SnippetState>((set, get) => ({
         ...s,
         spec,
         draftSpec,
-        datasetRefs: recomputeDatasetRefs(spec),
+        datasetRefs: recomputeDatasetRefs(draftSpec),
         modified: (now ?? new Date()).toISOString(),
       };
     });
@@ -422,9 +445,13 @@ export const useSnippetStore = create<SnippetState>((set, get) => ({
     if (!active) return false;
 
     const modified = (now ?? new Date()).toISOString();
+    // The draft is restored to the published spec, so datasetRefs must mirror it
+    // again — drop any link that existed only in the discarded draft (e.g. an
+    // Extract-to-Dataset rewrite the user reverted before publishing).
+    const datasetRefs = recomputeDatasetRefs(active.spec);
     set((s) => ({
       snippets: s.snippets.map((x) =>
-        x.id === activeSnippetId ? { ...x, draftSpec: x.spec, modified } : x,
+        x.id === activeSnippetId ? { ...x, draftSpec: x.spec, datasetRefs, modified } : x,
       ),
       // Reload the editor with the restored draft, on the editable view (spec §03D).
       draftText: active.spec,

src/core/import-normalize.ts

@@ -283,8 +283,8 @@ export function reassignCollidingSnippetIds(
  * Propagate dataset renames (from `dedupeIncomingDatasetNames`) into the imported
  * snippets that reference them (architecture 07 §6): for each rename, rewrite both
  * `spec` and `draftSpec` via `renameDatasetInSpec`, then recompute `datasetRefs`
- * from the rewritten spec — the spec is the source of truth, `datasetRefs` mirrors
+ * from the rewritten draft — the draft is the source of truth, `datasetRefs`
- * it. Matching is case-insensitive (consistent with the naming helpers). Only
+ * mirrors it. Matching is case-insensitive (consistent with the naming helpers). Only
  * snippets that actually change are cloned.
  *
  * Applied to the imported set ONLY — existing snippets keep their references,
@@ -323,7 +323,8 @@ export function applyDatasetRenamesToSnippets(
       draftSpec = renameDatasetInSpec(draftSpec, oldName, to);
     }
 
-    return { ...snippet, spec, draftSpec, datasetRefs: recomputeDatasetRefs(spec) };
+    // datasetRefs mirrors the draft — the version the editor shows (arch 07 §3).
+    return { ...snippet, spec, draftSpec, datasetRefs: recomputeDatasetRefs(draftSpec) };
   });
 }
 

src/core/snippet.ts

@@ -40,7 +40,9 @@ export interface Snippet {
   comment: string;
   /** User-assigned labels. */
   tags: string[];
-  /** Names of datasets referenced by this spec (maintained on publish). */
+  /** Names of datasets referenced by the draft spec; mirrors the version being
+   *  edited, recomputed on every draft change (create/auto-save/extract/revert)
+   *  and on publish (docs/architecture/07 §3). */
   datasetRefs: string[];
   /** Free-form, extensible metadata bag. */
   meta: Record<string, unknown>;