feat: implement auto-detection for data formats and enhance dataset input UI

This commit is contained in:
2025-10-15 12:37:00 +03:00
parent a8f5ba44ea
commit acdc619e4e
5 changed files with 374 additions and 144 deletions

View File

@@ -199,6 +199,12 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
- Support for multiple data sources:
- **Inline data**: JSON, CSV, TSV, TopoJSON stored directly
- **URL data**: Remote data sources with format specification
- **Intelligent auto-detection system**:
- Single input field for data or URL
- Automatic URL detection and content fetching
- Format detection from content (JSON, CSV, TSV, TopoJSON)
- Confidence scoring (high/medium/low)
- Visual confirmation UI with badges and preview
- Automatic metadata calculation:
- Row count, column count, column names
- Data size in bytes
@@ -217,7 +223,6 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
- CSV/TSV: `{ values: data, format: { type: 'csv'/'tsv' } }`
- TopoJSON: `{ values: data, format: { type: 'topojson' } }`
- URL: `{ url: "...", format: { type: '...' } }`
- Button-group UI for source/format selection (matches editor style)
- Graceful error handling for CORS and network failures
- Modal scrolling support for small viewports
@@ -246,6 +251,12 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
- Vega-Lite's native format parsers used for rendering
- Metadata refresh fetches live data and updates statistics
- Modal resizes with viewport (max-height: 90vh)
- **Auto-detection algorithms**:
- URL validation with protocol check (http/https)
- JSON parsing with TopoJSON identification
- CSV/TSV detection via delimiter counting and consistency checks
- Format inference from URL file extensions
- Debounced input handling for real-time feedback
---
@@ -388,10 +399,11 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
- Multi-format support: JSON, CSV, TSV, TopoJSON
- Multi-source support: Inline data and URL references
- Modal-based Dataset Manager with full CRUD
- Intelligent auto-detection (URL/format/confidence)
- Visual confirmation UI with badges and preview
- Automatic metadata calculation and display
- URL metadata fetching and refresh
- Dataset reference resolution in Vega-Lite specs
- Button-group UI for source/format selection
- Retro Windows 2000 aesthetic throughout
### Technical Implementation
@@ -410,4 +422,5 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
- **Dataset Storage**: IndexedDB with async/Promise-based API, unique name constraint
- **Dataset Resolution**: Async spec transformation before rendering, format-aware data injection
- **URL Metadata**: Fetch on creation with graceful CORS error handling
- **Modal UI**: Flexbox with overflow:auto, max-height responsive to viewport
- **Modal UI**: Flexbox with overflow:auto, max-height responsive to viewport
- **Auto-detection**: URL validation, JSON/CSV/TSV parsing, confidence scoring, real-time feedback

View File

@@ -129,9 +129,13 @@
- Copy reference button (generates `"data": {"name": "..."}`)
- Delete dataset with confirmation
- Refresh metadata button for URL datasets (🔄)
- Automatic metadata calculation on creation
- **Auto-detection system**:
- Single input field for data or URL
- Automatic URL detection and content fetching
- Format detection (JSON, CSV, TSV, TopoJSON)
- Confidence scoring (high/medium/low)
- Visual confirmation with badges and preview
- URL fetching with CORS error handling
- Button-group UI for source/format selection
- Unique dataset name constraint (IndexedDB index)
- Empty state message for no datasets
@@ -197,14 +201,14 @@ src/
├── js/
│ ├── config.js # Global variables, settings, sample data
│ ├── snippet-manager.js # Snippet CRUD, storage, search, sort (977 lines)
│ ├── dataset-manager.js # Dataset CRUD, IndexedDB, formats (637 lines)
│ ├── dataset-manager.js # Dataset CRUD, IndexedDB, auto-detection (714 lines)
│ ├── panel-manager.js # Layout resizing, toggling, persistence (200 lines)
│ ├── editor.js # Monaco setup, Vega rendering, dataset resolution (150 lines)
│ └── app.js # Event handlers, initialization (197 lines)
└── styles.css # Retro Windows 2000 aesthetic
```
**Total JS Lines**: ~2,161 lines (excluding comments and blank lines)
**Total JS Lines**: ~2,238 lines (excluding comments and blank lines)
---