Profile per-column cardinality and extent; add data-aware chart warnings (M4, A3/A4)

docs/architecture/06-type-inference.md

@@ -169,16 +169,17 @@ A **profile** is the set of derived summary fields stored on a dataset record so
 the UI can describe it without re-parsing the payload. Per the data model, a
 profiled dataset carries:
 
-| Field         | Type                    | Meaning                            |
-| ------------- | ----------------------- | ---------------------------------- |
-| `rowCount`    | `number \| null`        | Data rows, or `null` when N/A.     |
-| `columnCount` | `number \| null`        | Columns, or `null` when N/A.       |
-| `columns`     | `string[]`              | Column names, in order.            |
-| `columnTypes` | `Array<{ name; type }>` | Per-column inferred type (see §2). |
-| `size`        | `number`                | Approximate payload size in bytes. |
+| Field         | Type                                                       | Meaning                                                            |
+| ------------- | ---------------------------------------------------------- | ------------------------------------------------------------------ |
+| `rowCount`    | `number \| null`                                           | Data rows, or `null` when N/A.                                     |
+| `columnCount` | `number \| null`                                           | Columns, or `null` when N/A.                                       |
+| `columns`     | `string[]`                                                 | Column names, in order.                                            |
+| `columnTypes` | `Array<{ name; type }>`                                    | Per-column inferred type (see §2).                                 |
+| `columnStats` | `Array<{ name; distinct; distinctCapped; numericExtent }>` | Per-column cardinality + numeric range (see §3.3). Empty when N/A. |
+| `size`        | `number`                                                   | Approximate payload size in bytes.                                 |
 
-`null` row/column counts and an empty `columns`/`columnTypes` are how the UI
-shows **"N/A"** — see §3.2.
+`null` row/column counts and an empty `columns`/`columnTypes`/`columnStats` are
+how the UI shows **"N/A"** — see §3.2.
 
 ### 3.1 What gets profiled
 
@@ -208,12 +209,14 @@ of the stored payload), but `rowCount` and `columnCount` are `null`, and
    - `json` that is a non-empty **array of objects** → use it directly.
    - anything else (`topojson`, a lone JSON object, an empty array) → not
      tabular; return the N/A profile (`rowCount: null`, `columnCount: null`,
-     `columns: []`, `columnTypes: []`, plus `size`).
+     `columns: []`, `columnTypes: []`, `columnStats: []`, plus `size`).
 3. **Derive columns** from the union of keys across the rows (or the CSV/TSV
    header), preserving first-seen order.
-4. **Infer each column's type** by collecting that column's values across the
-   rows and calling `inferColumnType` (§2), sampling per §4.
-5. **Assemble** `rowCount`, `columnCount`, `columns`, `columnTypes`, `size`.
+4. **Infer each column's type and stats** by collecting that column's values across
+   the rows (sampling per §4) and calling `inferColumnType` (§2) plus deriving its
+   cardinality + numeric extent (§3.3) — one pass per column over the same sample.
+5. **Assemble** `rowCount`, `columnCount`, `columns`, `columnTypes`, `columnStats`,
+   `size`.
 
 ### Sketch
 
@@ -226,6 +229,12 @@ export interface DatasetProfile {
   columnCount: number | null;
   columns: string[];
   columnTypes: Array<{ name: string; type: ColumnType }>;
+  columnStats: Array<{
+    name: string;
+    distinct: number; // capped at DISTINCT_CAP
+    distinctCapped: boolean; // true ⇒ real cardinality ≥ DISTINCT_CAP
+    numericExtent: { min: number; max: number } | null; // numeric columns only
+  }>;
   size: number;
 }
 
@@ -234,6 +243,7 @@ const NA = (size: number): DatasetProfile => ({
   columnCount: null,
   columns: [],
   columnTypes: [],
+  columnStats: [],
   size,
 });
 
@@ -259,16 +269,22 @@ export function profileData(
   if (columns.length === 0) return NA(size);
 
   const sample = sampleRows(rows);
-  const columnTypes = columns.map((name) => ({
-    name,
-    type: inferColumnType(sample.map((r) => r[name])),
-  }));
+  // One pass per column over the sample: type + stats (cardinality, numeric extent).
+  const columnTypes = [];
+  const columnStats = [];
+  for (const name of columns) {
+    const values = sample.map((r) => r[name]);
+    const type = inferColumnType(values);
+    columnTypes.push({ name, type });
+    columnStats.push(columnStatsFor(name, values, type)); // see §3.3
+  }
 
   return {
     rowCount: rows.length,
     columnCount: columns.length,
     columns,
     columnTypes,
+    columnStats,
     size,
   };
 }
@@ -278,17 +294,41 @@ Parsing CSV/TSV text and detecting the payload shape happen _upstream_ of
 `profileData`; this function takes already-parsed rows so it stays pure and
 trivially testable. The caller passes `null` for URL and non-tabular datasets.
 
+### 3.3 Column stats: cardinality + numeric extent
+
+Alongside the display type, each column carries the two data-shape signals the
+**Chart Builder** needs for its data-aware Tier-B hints (spec §06; see
+`chart-builder.ts` `builderWarnings`):
+
+- **`distinct`** — the count of distinct non-empty values **in the sample**,
+  counted only up to `DISTINCT_CAP` (50). Past the cap the exact number stops
+  mattering — a legend or category axis with 50 entries is already unreadable — so
+  counting stops and **`distinctCapped`** flags that the real cardinality is at
+  least the cap. Powers the crowded-legend (discrete colour) and crowded-category
+  warnings.
+- **`numericExtent`** — `{ min, max }` over the numeric values, for `number`
+  columns only (`null` otherwise), using the **same numeric test** as
+  `inferColumnType` so the two never disagree. Powers the negative-value **Size**
+  guard (size encodes magnitude → negatives mislead; Draco `hard.lp:56`).
+
+Both are derived in the same sample pass as type inference, so they are nearly
+free, and both are **estimates** (sampled from the head, like the type) — fine for
+soft advisories, not guarantees. URL / non-tabular data and datasets stored before
+this field carry empty `columnStats`, and the dependent hints simply skip.
+
 ---
 
 ## 4. Sampling vs. full scan
 
 `rowCount`/`columnCount`/`size` always reflect the **whole** dataset — they're
-cheap (a length and a byte count). Only **type inference** has a per-value cost,
-and it's the one place a huge dataset could hurt.
+cheap (a length and a byte count). Only the **per-value** work — type inference
+and the §3.3 column stats — has a per-row cost, and it's the one place a huge
+dataset could hurt.
 
-So: infer types from a **bounded sample** of rows, not the full column. A fixed
-cap (e.g. the first ~200 rows) keeps profiling fast and predictable on large
-pasted datasets while still being more than enough signal to classify a column.
+So: derive types **and** stats from a **bounded sample** of rows, not the full
+column. A fixed cap (e.g. the first ~200 rows) keeps profiling fast and
+predictable on large pasted datasets while still being more than enough signal to
+classify a column and estimate its cardinality / range.
 
 ```ts
 const SAMPLE_SIZE = 200;
@@ -306,7 +346,8 @@ the same paste profiles the same way twice.
 ### Do / Don't
 
 - **Do** count rows/columns and size over the full payload.
-- **Do** cap type-inference sampling at a fixed head slice for determinism.
+- **Do** cap type-inference **and column-stats** sampling at a fixed head slice for
+  determinism (so `distinct` / `numericExtent` are estimates, like the type).
 - **Don't** randomly sample — non-deterministic profiles break tests and confuse
   users.
 - **Don't** scan every value of a million-row paste to guess a type.