Remote datasets: cap fetch body size

2026-08-08 02:02:33 +00:00 · 2026-06-11 19:21:51 +03:00
parent 983c052f3b
commit fe2b039698
6 changed files with 83 additions and 8 deletions
@@ -40,6 +40,8 @@ Most adapters are driven by **background subscribers** (arch 01 §5, _Effects_):

 **URL-dataset snapshot lifecycle** (the files a change to it touches): add / Refresh → component fetches (`infrastructure/remote-data.ts`) → `core/dataset.snapshotFromText` shapes the body by sniffed format → `DatasetStore.commitUrlSnapshot` / `refreshDataset` snapshots + profiles it _exactly like inline data_ → render resolves it cached-first in `core/rendering.resolvedData` (a live-URL fallback applies only while a URL dataset is still unfetched). See spec §05 for the behavior.

+A fetched body is held whole in memory and stored whole in IndexedDB, so `remote-data.ts` caps it at a fixed ceiling (`MAX_REMOTE_BYTES`): it rejects on the declared `Content-Length` _before_ reading, so an oversized download never lands in memory, with a body-length backstop for chunked responses that declare no length. The classified `too-large` reason maps to its own copy in `services/remote-data-errors.ts` — neither the inline-paste nor the retry recovery helps an oversized file, so that message points at using a smaller or pre-aggregated source instead.
+
 ---

 ## 2. IndexedDB Wrapper