Remote datasets: cap fetch body size

This commit is contained in:
2026-06-11 19:21:51 +03:00
parent 983c052f3b
commit fe2b039698
6 changed files with 83 additions and 8 deletions
+2
View File
@@ -40,6 +40,8 @@ Most adapters are driven by **background subscribers** (arch 01 §5, _Effects_):
**URL-dataset snapshot lifecycle** (the files a change to it touches): add / Refresh → component fetches (`infrastructure/remote-data.ts`) → `core/dataset.snapshotFromText` shapes the body by sniffed format → `DatasetStore.commitUrlSnapshot` / `refreshDataset` snapshots + profiles it _exactly like inline data_ → render resolves it cached-first in `core/rendering.resolvedData` (a live-URL fallback applies only while a URL dataset is still unfetched). See spec §05 for the behavior.
A fetched body is held whole in memory and stored whole in IndexedDB, so `remote-data.ts` caps it at a fixed ceiling (`MAX_REMOTE_BYTES`): it rejects on the declared `Content-Length` _before_ reading, so an oversized download never lands in memory, with a body-length backstop for chunked responses that declare no length. The classified `too-large` reason maps to its own copy in `services/remote-data-errors.ts` — neither the inline-paste nor the retry recovery helps an oversized file, so that message points at using a smaller or pre-aggregated source instead.
---
## 2. IndexedDB Wrapper
+1 -1
View File
@@ -90,7 +90,7 @@ A destructive or off-screen outcome raises a confirming toast; an action whose r
- **Copy Reference** — copies the by-name reference object to the clipboard, ready to paste into a spec:
`{ "data": { "name": "MyDataset" } }`
The clipboard write is invisible, so it is confirmed _inline on the control_ ("Copied"), announced politely to assistive technology — not a toast.
- **New / Create New** — opens the create form in the detail pane with fields: **name** (required, unique), **source** toggle (Inline / URL), the **data** (a paste area for inline, a URL field for URL source), and an optional **comment**. Save is disabled until a name and valid data/URL are present. For a **URL** dataset, saving fetches and snapshots the address; the Save control shows a busy state while fetching. If the fetch fails — offline, blocked by the host (cross-origin), not found, empty, or timed out — the form shows a readable, cause-specific error and offers a one-click **"Paste data inline instead"** that switches the form to an inline paste (keeping the name and comment), rather than saving a broken record. On success the new dataset is shown selected in the detail pane — that visible result is the confirmation, so no toast is raised (a dataset created _off-screen_ via Extract does toast; see _Spec Editor_).
- **New / Create New** — opens the create form in the detail pane with fields: **name** (required, unique), **source** toggle (Inline / URL), the **data** (a paste area for inline, a URL field for URL source), and an optional **comment**. Save is disabled until a name and valid data/URL are present. For a **URL** dataset, saving fetches and snapshots the address; the Save control shows a busy state while fetching. If the fetch fails — offline, blocked by the host (cross-origin), not found, empty, timed out, or larger than the fetch size limit — the form shows a readable, cause-specific error and offers a one-click **"Paste data inline instead"** that switches the form to an inline paste (keeping the name and comment), rather than saving a broken record. The size-limit case is the exception: because pasting an oversized file inline would not help, its message points at using a smaller or pre-aggregated source rather than the inline fallback. On success the new dataset is shown selected in the detail pane — that visible result is the confirmation, so no toast is raised (a dataset created _off-screen_ via Extract does toast; see _Spec Editor_).
- **Refresh** (URL datasets) — re-fetches the dataset's address and re-snapshots it, re-profiling the data and advancing the last-fetched time. The updated figures are the visible confirmation, so success raises no toast; a failed refresh raises an error toast. Refresh is also how a URL dataset that has not yet been fetched (e.g. one migrated from an older version) acquires its snapshot.
- **Edit** — rename, edit the comment, change the source data, and (for URL datasets) change the address. Updating inline data re-profiles it; changing a URL dataset's address re-fetches and re-snapshots it; editing only a URL dataset's name or comment does **not** re-fetch. The modified timestamp advances.
- **Delete** — asks for confirmation ("Delete \"Name\"? This cannot be undone."), then removes the dataset and clears the selection.