mirror of
https://github.com/olehomelchenko/astrolabe.git
synced 2026-08-08 02:02:33 +00:00
Profile per-column cardinality and extent; add data-aware chart warnings (M4, A3/A4)
This commit is contained in:
@@ -214,30 +214,29 @@ as of 2026-06-06.
|
||||
un-aggregated case. A general "long labels → go horizontal" suggestion on _any_ vertical
|
||||
bar is still deferred (needs a label-length / cardinality signal); a blanket warning was
|
||||
rejected — it would fire on every ordinary vertical bar.
|
||||
- **A3 · Crowded-axis & high-cardinality warnings** _(partly done)_ —
|
||||
- **A3 · Crowded-axis & high-cardinality warnings** _(done)_ —
|
||||
- _Done:_ the **un-aggregated crowded axis** — a bar/line/area with a category axis and a
|
||||
**raw** measure draws one mark (and one label) per row, so over `CROWDED_CATEGORY_ROWS`
|
||||
(30) rows it warns and points to aggregating, or a horizontal bar. Row-count-based:
|
||||
`builderWarnings(config, rowCount)`, with `rowCount` from the loaded dataset; detects
|
||||
exactly the mark-count == row-count case (URL/non-tabular → `rowCount` null → skipped).
|
||||
- _Remaining (needs the profiling extension below):_ an **aggregated** axis that still has
|
||||
many distinct **categories**, an unreadable **Color legend** (>10/>20 categories), and
|
||||
**number-typed-Nominal**. All need per-column **cardinality**, which the profile lacks.
|
||||
- **A4 · Data-aware Size guard** _(deferred — needs the profiling extension)_ — exclude
|
||||
**negative**-valued columns from Size (Draco `hard.lp:56`; size implies positive
|
||||
magnitude). Today only the type-level Size discipline is enforced.
|
||||
detects exactly the mark-count == row-count case (URL/non-tabular → `rowCount` null → skipped).
|
||||
- _Done (via the profiling extension below):_ an **aggregated** axis that still has many
|
||||
distinct **categories** (`CROWDED_CATEGORY_DISTINCT` = 30) and an unreadable **Color
|
||||
legend** (`CROWDED_LEGEND_DISTINCT` = 12, discrete colour only) now warn from per-column
|
||||
cardinality. (Number-typed-Nominal remains a possible future nudge; not yet flagged.)
|
||||
- **A4 · Data-aware Size guard** _(done)_ — Size mapped to a field whose profiled numeric
|
||||
**extent** goes negative warns (Draco `hard.lp:56`; size implies positive magnitude),
|
||||
on top of the type-level Size discipline.
|
||||
|
||||
> **TODO — profiling extension (the A3-remaining + A4 enabler).** `profile.ts` computes only
|
||||
> `rowCount` / `columnCount` / `columnTypes`. Extend it, in the **same sample pass** that
|
||||
> already feeds `inferColumnType` (so it's nearly free), to also derive per column: a
|
||||
> **capped distinct count** (cardinality — cap at ~50; a sampled count is enough for a
|
||||
> ">N categories" threshold, no full scan) and a **numeric extent** (min/max → sign).
|
||||
> Surface them on `DatasetProfile` (alongside `columnTypes`). Then: `builderWarnings` consumes
|
||||
> cardinality (legend/axis crowding) and the Size gate consumes sign (A4). **Caveats:**
|
||||
> URL / non-tabular datasets have no rows at profile time → these fields are null and the
|
||||
> dependent warnings simply skip; and stored datasets predate the field, so this needs a
|
||||
> recompute-on-read or a `dataset-migrations` bump (see architecture 02 / 06). Keep
|
||||
> thresholds in `chart-builder.ts` constants like `CROWDED_CATEGORY_ROWS`.
|
||||
> **Done — profiling extension (the A3 / A4 enabler).** `profile.ts` now derives, in the
|
||||
> **same sample pass** that feeds `inferColumnType`, a per-column **capped distinct count**
|
||||
> (`DISTINCT_CAP` = 50, with a `distinctCapped` overflow flag) and a **numeric extent**
|
||||
> (min/max, numeric columns only), surfaced on `DatasetProfile.columnStats` and the `Dataset`
|
||||
> record. `builderWarnings(config, rowCount, columns)` consumes them for the legend/axis
|
||||
> crowding hints and the negative-value Size guard. **Caveats handled:** URL / non-tabular
|
||||
> data has no rows → `columnStats` is `[]` and the dependent hints skip; datasets stored
|
||||
> before the field default to `[]` via `dataset-migrations` / import normalization (no forced
|
||||
> re-profile on read — they pick up stats on next save). Thresholds live in `chart-builder.ts`
|
||||
> constants alongside `CROWDED_CATEGORY_ROWS`.
|
||||
|
||||
**B · Transform-enabled coverage (new core capability + §06 extension)** _(done)_
|
||||
|
||||
|
||||
Reference in New Issue
Block a user