mirror of https://github.com/olehomelchenko/astrolabe.git synced 2026-08-08 02:02:33 +00:00

Files

T

Fix dataset-ref walk false positive and wire spec-mandated success toasts

- Prune the data payload and top-level datasets keys in all three ref walks
  (extractDatasetRefs, renameDatasetInSpec, resolveDatasetRefs) so a data row
  carrying a field named "data" is no longer misread as a library reference,
  spuriously rewritten, or made to throw DatasetNotFoundError. Adds tests,
  including a guard that lookup-transform refs (from.data) still resolve.
- Wire the deferred success toasts now the Toaster has landed: publish, revert,
  extract-to-dataset, and snippet/dataset delete. Copy follows the council
  title-vs-message rule (title states the action, message adds the consequence).
- Reconcile the spec's blanket toast mandate to "toast only what the user can't
  already see": no toast on visible-result creates (snippet, dataset form);
  Copy Reference stays inline and gains an aria-live announcement (new shared
  .visually-hidden utility) instead of a toast-per-copy.
- Move the toast region to bottom-right so it stops covering the header action
  cluster (Publish/Revert, theme/datasets).
- Update docs/spec 01F/02/05 and docs/architecture/07 + 10 to match.

2026-06-05 16:34:58 +03:00

7.4 KiB

Raw Blame History

05 · Datasets

The Dataset Manager is a modal for creating and managing named, reusable datasets that snippets can reference by name. It is the home of the dataset library: a place separate from snippets where data lives once and is shared across many visualizations.

Purpose & Model

Datasets are named blobs of data stored in the user's local library, independent of any single snippet. A snippet references a dataset by name rather than embedding the data inline, so the same data can power many snippets and be edited in one place.

Datasets persist locally across sessions in a high-capacity local store, far larger than the budget available to snippets — large datasets belong here, not inline in specs.
A snippet references a dataset using a Vega-Lite named-data reference, e.g. { "data": { "name": "MyDataset" } }. When the Live Preview renders a spec, it resolves any such named reference against the dataset library (see Live Preview).
See Data Model for the stored shape of a dataset.

Opened from a header control or via the keyboard shortcut Cmd/Ctrl+K.
The current view and the selected dataset are reflected in the URL, so a selected dataset produces a shareable/back-navigable location (see Application Shell & Navigation).
Closing the modal clears the current selection and any open create form.

Layout

A two-pane modal:

List pane (left): a "New Dataset" action plus the list of all datasets, sorted most-recently-modified first.
Detail pane (right): shows the selected dataset's details, the create form when creating, or an empty prompt ("Select a dataset or create a new one") when nothing is selected.

List item

Each list item shows:

The dataset name.
A meta line combining: source ("URL" prefix for URL datasets), row count when known, the format label (JSON / CSV / TSV / TOPOJSON), and size (human-readable, e.g. B / KB / MB). For URL datasets where counts are not yet known, only the source and format label are shown.
A usage badge when one or more snippets reference the dataset, indicating how many.

Clicking an item selects it and shows its detail. Per-item actions (delete, plus copy-reference and build-chart) live in the detail pane for the selected dataset.

Source Types

A dataset has one of two source types, chosen when creating it:

Inline — the data itself is pasted in and stored directly in the library.
URL — the dataset stores a remote URL (http/https). The data is not copied locally; it is fetched on demand when a referencing spec is rendered (see Live Preview).

For inline datasets the library holds the full data and can profile it. For URL datasets the library holds only the link, so row/column/size figures are typically not computed up front and show as "N/A".

Supported Formats

Four data formats are supported, named in the UI and stored on the dataset:

JSON — an array of objects (most common, profilable) or a single object.
CSV — comma-separated with a header row.
TSV — tab-separated with a header row.
TopoJSON — topology/map data (a JSON object whose type marks it as a topology).

Auto-detection

When the user pastes inline data, the app auto-detects the format and reports a confidence level (high / medium / low):

Valid JSON parses to JSON, or to TopoJSON when it is a topology object — high confidence.
Otherwise, multi-line text with a header row is detected as TSV (when tab-separated) or CSV (when comma-separated) — medium confidence.
Unrecognized input yields no format (low confidence); saving is blocked with a message asking the user to check the input.

The detected format and source are shown as badges in the create form so the user can confirm or override the source (Inline/URL) before saving. For URL datasets the format is inferred from the URL's file extension (.csv, .tsv, .json, .topojson) and shown as a hint.

Profiling

For tabular inline data (JSON array-of-objects, CSV, TSV) the app computes and stores a profile:

Row count and column count.
The list of column names.
An inferred type per column: number, text/string, date, or boolean. Type inference looks at the column's values: all-numeric becomes number, all true/false becomes boolean, otherwise string; empty cells are ignored.
Size in bytes of the stored data.

A truncated data preview of the raw data is also retained for display. URL datasets and non-tabular data are not profiled (counts show "N/A").

Detail Panel

The detail pane for a selected dataset shows:

Name.
Comment (optional free-text notes), when present.
Overview: statistics (rows, columns, size), the column list with each column's name and inferred type shown with a simple type indicator, and created/modified timestamps.
Preview: a truncated rendering of the data (raw text for CSV/TSV/URL, pretty-printed for JSON/TopoJSON).
Linked Snippets: the list of snippets that reference this dataset by name. This is the dataset side of bidirectional dataset↔snippet linking (see Snippet Library).

Actions

A destructive or off-screen outcome raises a confirming toast; an action whose result is immediately visible is confirmed by that change. Any action may raise an error toast on failure (see Application Shell & Navigation → Toasts).

Copy Reference — copies the by-name reference object to the clipboard, ready to paste into a spec: { "data": { "name": "MyDataset" } } The clipboard write is invisible, so it is confirmed inline on the control ("Copied"), announced politely to assistive technology — not a toast.
New / Create New — opens the create form in the detail pane with fields: name (required, unique), source toggle (Inline / URL), the data (a paste area for inline, a URL field for URL source), and an optional comment. Save is disabled until a name and valid data/URL are present. On success the new dataset is shown selected in the detail pane — that visible result is the confirmation, so no toast is raised (a dataset created off-screen via Extract does toast; see Spec Editor).
Edit — rename, edit the comment, and update the data (re-paste inline data or refresh the URL). Updating inline data re-profiles it; the modified timestamp advances.
Delete — asks for confirmation ("Delete "Name"? This cannot be undone."), then removes the dataset and clears the selection.

Build Chart From Dataset

From a selected dataset the user can launch the visual Chart Builder (see Chart Builder) pre-targeted at that dataset, producing a new snippet whose spec references the dataset by name.

Extract Inline Data → Dataset

The reverse flow starts in the editor: a user can extract inline data.values out of a spec into a new named dataset (see Spec Editor & Draft/Published Workflow). The result appears here as a new dataset, and the originating snippet's spec is rewritten to reference it by name.

Naming & Uniqueness

Dataset names must be unique. Attempting to create a dataset with a name already in use is rejected with an error toast.
During bulk operations such as import, conflicting names are automatically suffixed to remain unique rather than overwriting existing datasets (see Import & Export).
Renaming a dataset that is referenced by snippets keeps references consistent by updating the matching named-data references in affected specs.

7.4 KiB Raw Blame History