Snapshot URL datasets locally on add; preview tabular data as a table

This commit is contained in:
2026-06-10 10:24:42 +03:00
parent eb5e7ac53a
commit 2410c6e965
23 changed files with 1239 additions and 148 deletions
+17 -6
View File
@@ -6,19 +6,20 @@ How Astrolabe stores data in the browser, and the rules that keep that storage t
## 1. The Infrastructure-Adapter Principle
**Rule: nothing outside `src/app/infrastructure/` ever touches `indexedDB`, `localStorage`, `window`, or `location` directly.** Every browser-storage interaction goes through a typed adapter module that exposes plain async functions returning domain objects.
**Rule: nothing outside `src/app/infrastructure/` ever touches `indexedDB`, `localStorage`, `window`, `location`, or `fetch` directly.** Every browser interaction (storage _and_ network) goes through a typed adapter module that exposes plain async functions returning domain objects.
```
src/
├── core/ # portable engine — NO browser APIs, NO React
├── app/
│ ├── stores/ # Zustand stores; calls infrastructure, never IDB
│ ├── services/ # business logic; calls infrastructure, never IDB
│ └── infrastructure/ # the ONLY place that imports indexedDB/localStorage
│ ├── stores/ # Zustand stores; calls infrastructure, never IDB/fetch
│ ├── services/ # business logic; calls infrastructure, never IDB/fetch
│ └── infrastructure/ # the ONLY place that imports indexedDB/localStorage/fetch
│ ├── snippet-store.ts # IndexedDB: snippets (metadata + drafts)
│ ├── dataset-store.ts # IndexedDB: datasets (heavy payloads)
│ ├── settings-store.ts # localStorage: UserSettings
── ux-prefs.ts # localStorage: app/UI prefs (sort, panel layout)
── ux-prefs.ts # localStorage: app/UI prefs (sort, panel layout)
│ └── remote-data.ts # network: fetch a URL dataset's body (the ONLY fetch)
```
### Why this boundary exists
@@ -29,7 +30,15 @@ src/
- **Failure containment.** Quota errors, corrupt JSON, and missing keys are handled at the boundary and converted into typed results (or sane fallbacks), so the rest of the app never sees a raw `DOMException`.
> **Do:** `import { saveSnippet } from '@/app/infrastructure/snippet-store'`
> **Don't:** `indexedDB.open(...)` or `localStorage.getItem(...)` anywhere in a component, store, or service.
> **Don't:** `indexedDB.open(...)`, `localStorage.getItem(...)`, or `fetch(...)` anywhere in a component, store, or service.
### Background vs. interactive adapters
Most adapters are driven by **background subscribers** (arch 01 §5, _Effects_): a store changes, a startup subscriber writes it through to IndexedDB — the store never calls the adapter itself. The **network** adapter is the exception. Fetching a URL dataset is a user-initiated action with its own pending/error UI, so the **component** calls `remote-data.ts` directly (components may call adapters — cf. `navigator.clipboard`) and hands the fetched body to **pure** store actions (`DatasetStore.commitUrlSnapshot` / `refreshDataset`). The store never fetches, so it stays browser-free and unit-testable on already-fetched text.
> **Rule:** keep `fetch` behind `remote-data.ts`; orchestrate the URL-dataset fetch in the _component_ (busy state + the "paste data inline instead" recovery), not the store. Store commit actions only ever receive already-fetched text.
**URL-dataset snapshot lifecycle** (the files a change to it touches): add / Refresh → component fetches (`infrastructure/remote-data.ts`) → `core/dataset.snapshotFromText` shapes the body by sniffed format → `DatasetStore.commitUrlSnapshot` / `refreshDataset` snapshots + profiles it _exactly like inline data_ → render resolves it cached-first in `core/rendering.resolvedData` (a live-URL fallback applies only while a URL dataset is still unfetched). See spec §05 for the behavior.
---
@@ -221,6 +230,8 @@ export async function saveSnippet(s: Snippet): Promise<void> {
> **Do:** default `version` to the earliest shape (`1`) when the field is absent.
> **Don't:** branch on the presence of individual fields scattered through the app to detect "old data." Centralize that knowledge in the migration function.
> **Mirror shape changes in the import normalizer.** Imported records are built from a file, not read from IndexedDB, so they **never pass through `migrate<Entity>`** — `core/import-normalize.ts` upgrades them independently. A migration that changes a field's _shape_ must be applied in both places or import produces a malformed record. (E.g. the dataset v1→v2 URL-snapshot reshaping — address moves from `data` into `url`, `data` cleared — lives in **both** `migrateDataset` and `normalizeDataset`.)
---
## 5. localStorage Preferences (Settings & App/UI Prefs)
+7 -3
View File
@@ -183,7 +183,10 @@ how the UI shows **"N/A"** — see §3.2.
### 3.1 What gets profiled
Profiling applies only to **tabular inline data**:
Profiling applies to any **tabular payload**, whether pasted inline or fetched
from a URL (the snapshot model stores a URL dataset's data locally, so it profiles
through the same path as inline data — `snapshotFromText` shapes the fetched body,
then `computeDatasetProfile` runs):
- **JSON** that is an array of objects.
- **CSV** (comma-separated, header row).
@@ -191,8 +194,9 @@ Profiling applies only to **tabular inline data**:
Everything else is **not profiled**:
- **URL datasets** — the library holds only the link, not the data, so there is
nothing to scan. Counts are `null` / N/A.
- **Unfetched URL datasets** — a URL reference with no snapshot yet (e.g. one
migrated from an older record), so there is nothing to scan. Counts are `null` /
N/A until it is refreshed.
- **Non-tabular data** — a single JSON object, TopoJSON, or anything we can't
read as rows-of-columns. Counts are `null` / N/A.