mirror of
https://github.com/olehomelchenko/astrolabe.git
synced 2026-08-08 02:02:33 +00:00
Snapshot URL datasets locally on add; preview tabular data as a table
This commit is contained in:
+12
-10
@@ -28,7 +28,7 @@ A two-pane modal:
|
||||
Each list item shows:
|
||||
|
||||
- The dataset **name**.
|
||||
- A **meta line** combining: source ("URL" prefix for URL datasets), row count when known, the **format label** (JSON / CSV / TSV / TOPOJSON), and **size** (human-readable, e.g. B / KB / MB). For URL datasets where counts are not yet known, only the source and format label are shown.
|
||||
- A **meta line** combining: source ("URL" prefix for URL datasets), row count when known, the **format label** (JSON / CSV / TSV / TOPOJSON), and **size** (human-readable, e.g. B / KB / MB). A URL dataset that has not yet been fetched shows "not fetched" in place of the figures it does not have.
|
||||
- A **usage badge** when one or more snippets reference the dataset, indicating how many.
|
||||
|
||||
Clicking an item selects it and shows its detail. Per-item actions (delete, plus copy-reference and build-chart) live in the detail pane for the selected dataset.
|
||||
@@ -38,9 +38,9 @@ Clicking an item selects it and shows its detail. Per-item actions (delete, plus
|
||||
A dataset has one of two source types, chosen when creating it:
|
||||
|
||||
- **Inline** — the data itself is pasted in and stored directly in the library.
|
||||
- **URL** — the dataset stores a remote URL (http/https). The data is not copied locally; it is fetched on demand when a referencing spec is rendered (see _Live Preview_).
|
||||
- **URL** — the dataset is fetched once from a remote http/https address when it is created, and the fetched data is **snapshotted** into the library, along with the address (kept so the snapshot can be re-fetched). A URL dataset then works offline and renders from its local copy; it is never re-fetched at render time, only on demand (see _Refresh_, below).
|
||||
|
||||
For inline datasets the library holds the full data and can profile it. For URL datasets the library holds only the link, so row/column/size figures are typically not computed up front and show as "N/A".
|
||||
Either way the library holds the full data and profiles it. A URL dataset additionally records the address it was fetched from and when it was last fetched. Because the fetch happens in the browser, an address the browser cannot reach — offline, or one that blocks cross-origin requests — cannot be snapshotted; see _Actions → New / Create New_ for how that failure is handled.
|
||||
|
||||
## Supported Formats
|
||||
|
||||
@@ -59,18 +59,18 @@ When the user pastes inline data, the app auto-detects the format and reports a
|
||||
- Otherwise, multi-line text with a header row is detected as TSV (when tab-separated) or CSV (when comma-separated) — medium confidence.
|
||||
- Unrecognized input yields no format (low confidence); saving is blocked with a message asking the user to check the input.
|
||||
|
||||
The detected format and source are shown as badges in the create form so the user can confirm or override the source (Inline/URL) before saving. For URL datasets the format is inferred from the URL's file extension (`.csv`, `.tsv`, `.json`, `.topojson`) and shown as a hint.
|
||||
The detected format and source are shown as badges in the create form so the user can confirm or override the source (Inline/URL) before saving. For URL datasets the create form shows a format hint inferred from the URL's file extension (`.csv`, `.tsv`, `.json`, `.topojson`); the dataset's actual format is determined from the **fetched content** when it is created, falling back to the extension when the content is ambiguous.
|
||||
|
||||
## Profiling
|
||||
|
||||
For tabular inline data (JSON array-of-objects, CSV, TSV) the app computes and stores a profile:
|
||||
For tabular data (JSON array-of-objects, CSV, TSV) — whether pasted inline or fetched from a URL — the app computes and stores a profile:
|
||||
|
||||
- **Row count** and **column count**.
|
||||
- The list of **column names**.
|
||||
- An **inferred type per column**: number, text/string, date, or boolean. Type inference looks at the column's values: all-numeric becomes number, all `true`/`false` becomes boolean, otherwise string; empty cells are ignored.
|
||||
- **Size** in bytes of the stored data.
|
||||
|
||||
A **truncated data preview** of the raw data is also retained for display. URL datasets and non-tabular data are not profiled (counts show "N/A").
|
||||
A **truncated data preview** of the raw data is also retained for display. A fetched URL dataset is profiled exactly like inline data; only non-tabular data and a URL dataset that has **not yet been fetched** are unprofiled (counts show "N/A").
|
||||
|
||||
## Detail Panel
|
||||
|
||||
@@ -78,8 +78,9 @@ The detail pane for a selected dataset shows:
|
||||
|
||||
- **Name**.
|
||||
- **Comment** (optional free-text notes), when present.
|
||||
- **Source** (URL datasets only): the address the snapshot was fetched from, and when it was last fetched (or "Not fetched yet"), alongside the **Refresh** action.
|
||||
- **Overview**: statistics (rows, columns, size), the **column list** with each column's name and inferred type shown with a simple type indicator, and created/modified timestamps.
|
||||
- **Preview**: a truncated rendering of the data (raw text for CSV/TSV/URL, pretty-printed for JSON/TopoJSON).
|
||||
- **Preview**: a sample of the data. Tabular datasets (CSV, TSV, or a JSON array-of-objects — inline or fetched) render as a **table** of the first rows under the profiled column names, with a note when more rows exist than are shown. Non-tabular payloads (a single JSON object, TopoJSON) render as pretty-printed JSON, and a URL dataset that has not yet been fetched shows a short placeholder.
|
||||
- **Linked Snippets**: the list of snippets that reference this dataset by name. This is the dataset side of bidirectional dataset↔snippet linking (see _Snippet Library_).
|
||||
|
||||
## Actions
|
||||
@@ -89,13 +90,14 @@ A destructive or off-screen outcome raises a confirming toast; an action whose r
|
||||
- **Copy Reference** — copies the by-name reference object to the clipboard, ready to paste into a spec:
|
||||
`{ "data": { "name": "MyDataset" } }`
|
||||
The clipboard write is invisible, so it is confirmed _inline on the control_ ("Copied"), announced politely to assistive technology — not a toast.
|
||||
- **New / Create New** — opens the create form in the detail pane with fields: **name** (required, unique), **source** toggle (Inline / URL), the **data** (a paste area for inline, a URL field for URL source), and an optional **comment**. Save is disabled until a name and valid data/URL are present. On success the new dataset is shown selected in the detail pane — that visible result is the confirmation, so no toast is raised (a dataset created _off-screen_ via Extract does toast; see _Spec Editor_).
|
||||
- **Edit** — rename, edit the comment, and update the data (re-paste inline data or refresh the URL). Updating inline data re-profiles it; the modified timestamp advances.
|
||||
- **New / Create New** — opens the create form in the detail pane with fields: **name** (required, unique), **source** toggle (Inline / URL), the **data** (a paste area for inline, a URL field for URL source), and an optional **comment**. Save is disabled until a name and valid data/URL are present. For a **URL** dataset, saving fetches and snapshots the address; the Save control shows a busy state while fetching. If the fetch fails — offline, blocked by the host (cross-origin), not found, empty, or timed out — the form shows a readable, cause-specific error and offers a one-click **"Paste data inline instead"** that switches the form to an inline paste (keeping the name and comment), rather than saving a broken record. On success the new dataset is shown selected in the detail pane — that visible result is the confirmation, so no toast is raised (a dataset created _off-screen_ via Extract does toast; see _Spec Editor_).
|
||||
- **Refresh** (URL datasets) — re-fetches the dataset's address and re-snapshots it, re-profiling the data and advancing the last-fetched time. The updated figures are the visible confirmation, so success raises no toast; a failed refresh raises an error toast. Refresh is also how a URL dataset that has not yet been fetched (e.g. one migrated from an older version) acquires its snapshot.
|
||||
- **Edit** — rename, edit the comment, change the source data, and (for URL datasets) change the address. Updating inline data re-profiles it; changing a URL dataset's address re-fetches and re-snapshots it; editing only a URL dataset's name or comment does **not** re-fetch. The modified timestamp advances.
|
||||
- **Delete** — asks for confirmation ("Delete \"Name\"? This cannot be undone."), then removes the dataset and clears the selection.
|
||||
|
||||
## Build Chart From Dataset
|
||||
|
||||
From a selected dataset the user can launch the visual _Chart Builder_ (see _Chart Builder_) pre-targeted at that dataset, producing a new snippet whose spec references the dataset by name.
|
||||
From a selected dataset the user can launch the visual _Chart Builder_ (see _Chart Builder_) pre-targeted at that dataset, producing a new snippet whose spec references the dataset by name. This relies on the dataset's profiled columns, so a URL dataset must have been fetched first — an unfetched URL reference has no schema to map (use _Refresh_ to fetch it).
|
||||
|
||||
## Extract Inline Data → Dataset
|
||||
|
||||
|
||||
Reference in New Issue
Block a user