mirror of
https://github.com/olehomelchenko/astrolabe-nvc.git
synced 2025-12-21 21:22:23 +00:00
Add Dataset Manager functionality with IndexedDB support
- Introduced a new modal for managing datasets, including options to create, view, and delete datasets. - Implemented IndexedDB for persistent storage of datasets, allowing for efficient data retrieval and management. - Added UI components for dataset details, including statistics and preview. - Enhanced the app's JavaScript to handle dataset operations such as saving, updating, and deleting datasets. - Integrated dataset reference resolution in the visualization editor to support dynamic data sources. - Updated styles for the new modal and dataset management UI for improved user experience.
This commit is contained in:
23
CLAUDE.md
23
CLAUDE.md
@@ -9,13 +9,26 @@ Instructions for Claude Code when working on this project.
|
||||
## Architecture
|
||||
|
||||
- **Frontend-only**: HTML/CSS/JavaScript with CDN dependencies (Monaco Editor, Vega-Embed)
|
||||
- **Storage**: localStorage with Phase 0 schema (id, name, created, modified, spec, draftSpec, comment, tags, datasetRefs, meta)
|
||||
- **Structure**: Three resizable panels (snippet library, Monaco editor, live preview)
|
||||
- **No build tools**: Open `index.html` directly in browser
|
||||
- **Storage**:
|
||||
- **Snippets**: localStorage with Phase 0 schema (id, name, created, modified, spec, draftSpec, comment, tags, datasetRefs, meta)
|
||||
- **Datasets**: IndexedDB (unlimited size, multi-format: JSON/CSV/TSV/TopoJSON, inline & URL sources)
|
||||
- **Structure**: Three resizable panels (snippet library, Monaco editor, live preview) + Dataset Manager modal
|
||||
- **No build tools**: Open `index.html` directly in browser (needs local server for IndexedDB)
|
||||
|
||||
## Current Status
|
||||
|
||||
**Completed**: Phases 0-9 (All core functionality including import/export)
|
||||
**Next**: Phase 10 - Dataset Management
|
||||
**Completed**: Phases 0-10 (Core functionality + Dataset Management)
|
||||
**Next**: Phase 11 - Advanced Dataset Features (optional enhancements)
|
||||
|
||||
### Key Features Implemented
|
||||
- ✅ Snippet management with draft/published workflow
|
||||
- ✅ Multi-field sorting and real-time search
|
||||
- ✅ Storage monitoring and import/export
|
||||
- ✅ **Dataset management with IndexedDB**
|
||||
- Multi-format support (JSON, CSV, TSV, TopoJSON)
|
||||
- Multi-source support (inline data, URL references)
|
||||
- Automatic metadata calculation and URL fetching
|
||||
- Dataset reference resolution in Vega-Lite specs
|
||||
- Modal UI with button-group selectors
|
||||
|
||||
See `docs/dev-plan.md` for complete roadmap and technical details.
|
||||
|
||||
102
docs/dev-plan.md
102
docs/dev-plan.md
@@ -189,31 +189,79 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
|
||||
---
|
||||
|
||||
### **Phase 10: Dataset Management - Part 1**
|
||||
**Goal**: Separate dataset storage infrastructure
|
||||
### **Phase 10: Dataset Management** ✅ **COMPLETE**
|
||||
**Goal**: Separate dataset storage with multiple data sources and formats
|
||||
|
||||
- [ ] Implement dataset storage schema from Phase 0
|
||||
- [ ] Create dataset CRUD operations
|
||||
- [ ] Add dataset library panel/modal
|
||||
- [ ] List all stored datasets with metadata
|
||||
- [ ] Add/delete/rename datasets
|
||||
- [ ] Display dataset size and row counts
|
||||
**Deliverables**:
|
||||
- IndexedDB-based dataset storage (separate from snippets, no 5MB localStorage limit)
|
||||
- Full CRUD operations for datasets (create, read, update, delete)
|
||||
- Modal-based Dataset Manager UI (accessible via header link and 📁 toggle button)
|
||||
- Support for multiple data sources:
|
||||
- **Inline data**: JSON, CSV, TSV, TopoJSON stored directly
|
||||
- **URL data**: Remote data sources with format specification
|
||||
- Automatic metadata calculation:
|
||||
- Row count, column count, column names
|
||||
- Data size in bytes
|
||||
- Automatic URL fetching on dataset creation
|
||||
- Refresh metadata button for URL datasets (🔄)
|
||||
- Dataset list with informative metadata display
|
||||
- Dataset details panel with:
|
||||
- Editable name and comment
|
||||
- Statistics display (rows, columns, size)
|
||||
- Data preview (first 5 rows or URL info)
|
||||
- Copy reference button (copies `"data": {"name": "dataset-name"}`)
|
||||
- Delete button with confirmation
|
||||
- Auto-resolve dataset references in Vega-Lite specs during rendering
|
||||
- Format-aware rendering:
|
||||
- JSON: `{ values: data }`
|
||||
- CSV/TSV: `{ values: data, format: { type: 'csv'/'tsv' } }`
|
||||
- TopoJSON: `{ values: data, format: { type: 'topojson' } }`
|
||||
- URL: `{ url: "...", format: { type: '...' } }`
|
||||
- Button-group UI for source/format selection (matches editor style)
|
||||
- Graceful error handling for CORS and network failures
|
||||
- Modal scrolling support for small viewports
|
||||
|
||||
**Deliverable**: Basic dataset storage separate from snippets
|
||||
**Dataset Schema**:
|
||||
```javascript
|
||||
{
|
||||
id: number,
|
||||
name: string,
|
||||
created: ISO string,
|
||||
modified: ISO string,
|
||||
data: array/string/object, // Inline data or URL string
|
||||
format: 'json'|'csv'|'tsv'|'topojson',
|
||||
source: 'inline'|'url',
|
||||
comment: string,
|
||||
rowCount: number|null,
|
||||
columnCount: number|null,
|
||||
columns: string[],
|
||||
size: number|null // bytes
|
||||
}
|
||||
```
|
||||
|
||||
**Technical Implementation**:
|
||||
- IndexedDB with keyPath 'id', indexes on 'name' (unique) and 'modified'
|
||||
- Async/Promise-based DatasetStorage API
|
||||
- Format-specific parsing for metadata calculation
|
||||
- Vega-Lite's native format parsers used for rendering
|
||||
- Metadata refresh fetches live data and updates statistics
|
||||
- Modal resizes with viewport (max-height: 90vh)
|
||||
|
||||
---
|
||||
|
||||
### **Phase 11: Dataset Management - Part 2**
|
||||
**Goal**: Reference datasets from specs
|
||||
### **Phase 11: Advanced Dataset Features** _(Future)_
|
||||
**Goal**: Enhanced dataset workflows
|
||||
|
||||
- [ ] Detect inline data in Vega-Lite specs
|
||||
- [ ] "Extract to dataset" feature for inline data
|
||||
- [ ] Replace inline data with dataset references
|
||||
- [ ] Auto-resolve dataset references when rendering
|
||||
- [ ] Update snippet UI to show linked datasets
|
||||
- [ ] Handle missing dataset references gracefully
|
||||
- [ ] Dataset usage tracking (which snippets reference which datasets)
|
||||
- [ ] Import datasets from file upload
|
||||
- [ ] Export individual datasets
|
||||
- [ ] Dataset preview with table view
|
||||
- [ ] Column type detection and display
|
||||
|
||||
**Deliverable**: Specs can reference shared datasets
|
||||
**Deliverable**: Advanced dataset management and discovery
|
||||
|
||||
---
|
||||
|
||||
@@ -280,8 +328,9 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
│ └── js/ # Modular JavaScript organization
|
||||
│ ├── config.js # Global variables, settings, & sample data
|
||||
│ ├── snippet-manager.js # Snippet storage, CRUD operations & localStorage wrapper
|
||||
│ ├── dataset-manager.js # Dataset storage, CRUD operations & IndexedDB wrapper
|
||||
│ ├── panel-manager.js # Panel resize, toggle & memory system
|
||||
│ ├── editor.js # Monaco Editor initialization & Vega-Lite rendering
|
||||
│ ├── editor.js # Monaco Editor initialization, dataset resolution & Vega-Lite rendering
|
||||
│ └── app.js # Application initialization & event handlers
|
||||
└── docs/
|
||||
├── dev-plan.md # This development roadmap
|
||||
@@ -311,15 +360,15 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
|
||||
## Current Status
|
||||
|
||||
**Completed**: Phases 0-9 (Storage, UI, editor, rendering, persistence, CRUD, organization, draft/published workflow, storage monitoring, import/export)
|
||||
**Next**: Phase 10 - Dataset Management
|
||||
**Completed**: Phases 0-10 (Storage, UI, editor, rendering, persistence, CRUD, organization, draft/published workflow, storage monitoring, import/export, dataset management)
|
||||
**Next**: Phase 11 - Advanced Dataset Features (optional enhancements)
|
||||
**See**: `CLAUDE.md` for concise current state summary
|
||||
|
||||
---
|
||||
|
||||
## Implemented Features
|
||||
|
||||
### Core Capabilities (Phases 0-9)
|
||||
### Core Capabilities (Phases 0-10)
|
||||
- Three-panel resizable layout with memory and persistence
|
||||
- Monaco Editor v0.47.0 with Vega-Lite v5 schema validation
|
||||
- Live Vega-Lite rendering with debounced updates and error display
|
||||
@@ -334,6 +383,15 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
- Storage monitoring with visual progress bar and warning states
|
||||
- Export/Import functionality with format auto-detection
|
||||
- Snippet size display (right-aligned, shown for ≥ 1 KB)
|
||||
- **Dataset Management (Phase 10)**:
|
||||
- IndexedDB storage for datasets (unlimited size)
|
||||
- Multi-format support: JSON, CSV, TSV, TopoJSON
|
||||
- Multi-source support: Inline data and URL references
|
||||
- Modal-based Dataset Manager with full CRUD
|
||||
- Automatic metadata calculation and display
|
||||
- URL metadata fetching and refresh
|
||||
- Dataset reference resolution in Vega-Lite specs
|
||||
- Button-group UI for source/format selection
|
||||
- Retro Windows 2000 aesthetic throughout
|
||||
|
||||
### Technical Implementation
|
||||
@@ -348,4 +406,8 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
- **Storage Calculation**: Blob API for accurate byte counting of snippet data
|
||||
- **Flexbox Layout**: Scrollable snippet list with fixed metadata and storage monitor at bottom
|
||||
- **Import/Export**: Format detection, field normalization, ID conflict resolution, additive merging
|
||||
- **Size Display**: Per-snippet size calculation with conditional rendering (≥ 1 KB threshold)
|
||||
- **Size Display**: Per-snippet size calculation with conditional rendering (≥ 1 KB threshold)
|
||||
- **Dataset Storage**: IndexedDB with async/Promise-based API, unique name constraint
|
||||
- **Dataset Resolution**: Async spec transformation before rendering, format-aware data injection
|
||||
- **URL Metadata**: Fetch on creation with graceful CORS error handling
|
||||
- **Modal UI**: Flexbox with overflow:auto, max-height responsive to viewport
|
||||
134
index.html
134
index.html
@@ -21,6 +21,7 @@
|
||||
<div class="header-links">
|
||||
<span class="header-link" id="import-link">Import</span>
|
||||
<span class="header-link" id="export-link">Export</span>
|
||||
<span class="header-link" id="datasets-link">Datasets</span>
|
||||
<span class="header-link" id="help-link">Help</span>
|
||||
<input type="file" id="import-file-input" accept=".json" style="display: none;" />
|
||||
</div>
|
||||
@@ -38,6 +39,9 @@
|
||||
<button class="toggle-btn active" id="toggle-preview-panel" title="Toggle Preview Panel">
|
||||
👁️
|
||||
</button>
|
||||
<button class="toggle-btn" id="toggle-datasets" title="Datasets">
|
||||
📁
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div class="main-panels">
|
||||
@@ -144,8 +148,138 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Dataset Manager Modal -->
|
||||
<div id="dataset-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<span class="modal-title">Dataset Manager</span>
|
||||
<button class="modal-close" id="dataset-modal-close">×</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<!-- List View (default) -->
|
||||
<div id="dataset-list-view" class="dataset-view">
|
||||
<div class="dataset-list-header">
|
||||
<button class="modal-btn primary" id="new-dataset-btn">New Dataset</button>
|
||||
</div>
|
||||
<div class="dataset-container">
|
||||
<div class="dataset-list" id="dataset-list">
|
||||
<!-- Dynamically populated by renderDatasetList() -->
|
||||
</div>
|
||||
<div class="dataset-details" id="dataset-details" style="display: none;">
|
||||
<div class="dataset-detail-section">
|
||||
<div class="dataset-detail-header">Name</div>
|
||||
<input type="text" id="dataset-detail-name" class="dataset-input" placeholder="Dataset name..." />
|
||||
|
||||
<div class="dataset-detail-header">Comment</div>
|
||||
<textarea id="dataset-detail-comment" class="dataset-textarea" placeholder="Add a comment..." rows="3"></textarea>
|
||||
|
||||
<div class="dataset-detail-header-row">
|
||||
<span class="dataset-detail-header">Statistics</span>
|
||||
<button class="dataset-refresh-btn" id="refresh-metadata-btn" style="display: none;" title="Refresh metadata from URL">🔄</button>
|
||||
</div>
|
||||
<div class="dataset-stats">
|
||||
<div class="dataset-stat-item">
|
||||
<span class="dataset-stat-label">Rows:</span>
|
||||
<span id="dataset-detail-rows">0</span>
|
||||
</div>
|
||||
<div class="dataset-stat-item">
|
||||
<span class="dataset-stat-label">Columns:</span>
|
||||
<span id="dataset-detail-columns">0</span>
|
||||
</div>
|
||||
<div class="dataset-stat-item">
|
||||
<span class="dataset-stat-label">Size:</span>
|
||||
<span id="dataset-detail-size">0 B</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="dataset-detail-header">Timestamps</div>
|
||||
<div class="dataset-stats">
|
||||
<div class="dataset-stat-item">
|
||||
<span class="dataset-stat-label">Created:</span>
|
||||
<span id="dataset-detail-created">-</span>
|
||||
</div>
|
||||
<div class="dataset-stat-item">
|
||||
<span class="dataset-stat-label">Modified:</span>
|
||||
<span id="dataset-detail-modified">-</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="dataset-detail-header">Preview</div>
|
||||
<pre id="dataset-preview" class="dataset-preview-box"></pre>
|
||||
|
||||
<div class="dataset-actions">
|
||||
<button class="modal-btn" id="copy-reference-btn">Copy Reference</button>
|
||||
<button class="modal-btn delete-btn" id="delete-dataset-btn">Delete</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Form View (for creating new datasets) -->
|
||||
<div id="dataset-form-view" class="dataset-view" style="display: none;">
|
||||
<div class="dataset-form">
|
||||
<div class="dataset-form-header">Create New Dataset</div>
|
||||
|
||||
<div class="dataset-form-group">
|
||||
<label class="dataset-form-label">Name *</label>
|
||||
<input type="text" id="dataset-form-name" class="dataset-input" placeholder="Enter dataset name..." />
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-group">
|
||||
<div class="dataset-toggle-row">
|
||||
<div class="dataset-toggle-section">
|
||||
<span class="dataset-toggle-label">Source:</span>
|
||||
<div class="dataset-toggle-group">
|
||||
<button class="dataset-toggle-btn active" data-source="inline" type="button">Inline</button>
|
||||
<button class="dataset-toggle-btn" data-source="url" type="button">URL</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dataset-toggle-section">
|
||||
<span class="dataset-toggle-label">Format:</span>
|
||||
<div class="dataset-toggle-group">
|
||||
<button class="dataset-toggle-btn active" data-format="json" type="button">JSON</button>
|
||||
<button class="dataset-toggle-btn" data-format="csv" type="button">CSV</button>
|
||||
<button class="dataset-toggle-btn" data-format="tsv" type="button">TSV</button>
|
||||
<button class="dataset-toggle-btn" data-format="topojson" type="button">TopoJSON</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-group" id="dataset-url-group" style="display: none;">
|
||||
<label class="dataset-form-label">URL *</label>
|
||||
<input type="text" id="dataset-form-url" class="dataset-input" placeholder="https://example.com/data.csv" />
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-group" id="dataset-data-group">
|
||||
<label class="dataset-form-label">Data *</label>
|
||||
<div class="dataset-format-hint" id="dataset-format-hint">
|
||||
JSON array of objects: [{"col1": "value", "col2": 123}, ...]
|
||||
</div>
|
||||
<textarea id="dataset-form-data" class="dataset-textarea" placeholder='[{"col1": "value", "col2": 123}, ...]' rows="12"></textarea>
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-group">
|
||||
<label class="dataset-form-label">Comment</label>
|
||||
<textarea id="dataset-form-comment" class="dataset-textarea" placeholder="Optional description..." rows="3"></textarea>
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-error" id="dataset-form-error"></div>
|
||||
|
||||
<div class="dataset-form-actions">
|
||||
<button class="modal-btn primary" id="save-dataset-btn">Save Dataset</button>
|
||||
<button class="modal-btn" id="cancel-dataset-btn">Cancel</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="src/js/config.js"></script>
|
||||
<script src="src/js/snippet-manager.js"></script>
|
||||
<script src="src/js/dataset-manager.js"></script>
|
||||
<script src="src/js/panel-manager.js"></script>
|
||||
<script src="src/js/editor.js"></script>
|
||||
<script src="src/js/app.js"></script>
|
||||
|
||||
@@ -117,6 +117,70 @@ document.addEventListener('DOMContentLoaded', function () {
|
||||
});
|
||||
}
|
||||
|
||||
// Dataset Manager
|
||||
const datasetsLink = document.getElementById('datasets-link');
|
||||
const toggleDatasetsBtn = document.getElementById('toggle-datasets');
|
||||
const datasetModal = document.getElementById('dataset-modal');
|
||||
const datasetModalClose = document.getElementById('dataset-modal-close');
|
||||
const newDatasetBtn = document.getElementById('new-dataset-btn');
|
||||
const cancelDatasetBtn = document.getElementById('cancel-dataset-btn');
|
||||
const saveDatasetBtn = document.getElementById('save-dataset-btn');
|
||||
const deleteDatasetBtn = document.getElementById('delete-dataset-btn');
|
||||
const copyReferenceBtn = document.getElementById('copy-reference-btn');
|
||||
|
||||
// Open dataset manager
|
||||
if (datasetsLink) {
|
||||
datasetsLink.addEventListener('click', openDatasetManager);
|
||||
}
|
||||
if (toggleDatasetsBtn) {
|
||||
toggleDatasetsBtn.addEventListener('click', openDatasetManager);
|
||||
}
|
||||
|
||||
// Close dataset manager
|
||||
if (datasetModalClose) {
|
||||
datasetModalClose.addEventListener('click', closeDatasetManager);
|
||||
}
|
||||
|
||||
// Close on overlay click
|
||||
if (datasetModal) {
|
||||
datasetModal.addEventListener('click', function (e) {
|
||||
if (e.target === datasetModal) {
|
||||
closeDatasetManager();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// New dataset button
|
||||
if (newDatasetBtn) {
|
||||
newDatasetBtn.addEventListener('click', showNewDatasetForm);
|
||||
}
|
||||
|
||||
// Cancel dataset button
|
||||
if (cancelDatasetBtn) {
|
||||
cancelDatasetBtn.addEventListener('click', hideNewDatasetForm);
|
||||
}
|
||||
|
||||
// Save dataset button
|
||||
if (saveDatasetBtn) {
|
||||
saveDatasetBtn.addEventListener('click', saveNewDataset);
|
||||
}
|
||||
|
||||
// Delete dataset button
|
||||
if (deleteDatasetBtn) {
|
||||
deleteDatasetBtn.addEventListener('click', deleteCurrentDataset);
|
||||
}
|
||||
|
||||
// Copy reference button
|
||||
if (copyReferenceBtn) {
|
||||
copyReferenceBtn.addEventListener('click', copyDatasetReference);
|
||||
}
|
||||
|
||||
// Refresh metadata button
|
||||
const refreshMetadataBtn = document.getElementById('refresh-metadata-btn');
|
||||
if (refreshMetadataBtn) {
|
||||
refreshMetadataBtn.addEventListener('click', refreshDatasetMetadata);
|
||||
}
|
||||
|
||||
// View mode toggle buttons
|
||||
document.getElementById('view-draft').addEventListener('click', () => {
|
||||
switchViewMode('draft');
|
||||
|
||||
636
src/js/dataset-manager.js
Normal file
636
src/js/dataset-manager.js
Normal file
@@ -0,0 +1,636 @@
|
||||
// Dataset management with IndexedDB
|
||||
|
||||
const DB_NAME = 'astrolabe-datasets';
|
||||
const DB_VERSION = 1;
|
||||
const STORE_NAME = 'datasets';
|
||||
|
||||
let db = null;
|
||||
|
||||
// Initialize IndexedDB
|
||||
function initializeDatasetDB() {
|
||||
return new Promise((resolve, reject) => {
|
||||
const request = indexedDB.open(DB_NAME, DB_VERSION);
|
||||
|
||||
request.onerror = () => reject(request.error);
|
||||
request.onsuccess = () => {
|
||||
db = request.result;
|
||||
resolve(db);
|
||||
};
|
||||
|
||||
request.onupgradeneeded = (event) => {
|
||||
const db = event.target.result;
|
||||
|
||||
// Create object store if it doesn't exist
|
||||
if (!db.objectStoreNames.contains(STORE_NAME)) {
|
||||
const objectStore = db.createObjectStore(STORE_NAME, { keyPath: 'id' });
|
||||
objectStore.createIndex('name', 'name', { unique: true });
|
||||
objectStore.createIndex('modified', 'modified', { unique: false });
|
||||
}
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
// Generate unique ID
|
||||
function generateDatasetId() {
|
||||
return Date.now() + Math.random() * 1000;
|
||||
}
|
||||
|
||||
// Calculate dataset statistics
|
||||
function calculateDatasetStats(data, format, source) {
|
||||
let rowCount = 0;
|
||||
let columnCount = 0;
|
||||
let columns = [];
|
||||
let size = 0;
|
||||
|
||||
// For URL sources, we can't calculate stats without fetching
|
||||
if (source === 'url') {
|
||||
return { rowCount: null, columnCount: null, columns: [], size: null };
|
||||
}
|
||||
|
||||
if (format === 'json' || format === 'topojson') {
|
||||
if (!Array.isArray(data) || data.length === 0) {
|
||||
return { rowCount: 0, columnCount: 0, columns: [], size: 0 };
|
||||
}
|
||||
rowCount = data.length;
|
||||
const firstRow = data[0];
|
||||
columns = typeof firstRow === 'object' ? Object.keys(firstRow) : [];
|
||||
columnCount = columns.length;
|
||||
size = new Blob([JSON.stringify(data)]).size;
|
||||
} else if (format === 'csv' || format === 'tsv') {
|
||||
// For CSV/TSV, data is stored as raw text
|
||||
const lines = data.trim().split('\n');
|
||||
rowCount = Math.max(0, lines.length - 1); // Subtract header row
|
||||
if (lines.length > 0) {
|
||||
const separator = format === 'csv' ? ',' : '\t';
|
||||
columns = lines[0].split(separator).map(h => h.trim().replace(/^"|"$/g, ''));
|
||||
columnCount = columns.length;
|
||||
}
|
||||
size = new Blob([data]).size;
|
||||
}
|
||||
|
||||
return { rowCount, columnCount, columns, size };
|
||||
}
|
||||
|
||||
// Dataset Storage API
|
||||
const DatasetStorage = {
|
||||
// Initialize database
|
||||
async init() {
|
||||
if (!db) {
|
||||
await initializeDatasetDB();
|
||||
}
|
||||
return db;
|
||||
},
|
||||
|
||||
// Create new dataset
|
||||
async createDataset(name, data, format, source, comment = '') {
|
||||
await this.init();
|
||||
|
||||
const now = new Date().toISOString();
|
||||
const stats = calculateDatasetStats(data, format, source);
|
||||
|
||||
const dataset = {
|
||||
id: generateDatasetId(),
|
||||
name: name.trim(),
|
||||
created: now,
|
||||
modified: now,
|
||||
data: data, // For inline: actual data, for URL: the URL string
|
||||
format: format, // 'json', 'csv', 'tsv', or 'topojson'
|
||||
source: source, // 'inline' or 'url'
|
||||
comment: comment.trim(),
|
||||
...stats
|
||||
};
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction([STORE_NAME], 'readwrite');
|
||||
const store = transaction.objectStore(STORE_NAME);
|
||||
const request = store.add(dataset);
|
||||
|
||||
request.onsuccess = () => resolve(dataset);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
},
|
||||
|
||||
// Get all datasets
|
||||
async listDatasets() {
|
||||
await this.init();
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction([STORE_NAME], 'readonly');
|
||||
const store = transaction.objectStore(STORE_NAME);
|
||||
const request = store.getAll();
|
||||
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
},
|
||||
|
||||
// Get single dataset by ID
|
||||
async getDataset(id) {
|
||||
await this.init();
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction([STORE_NAME], 'readonly');
|
||||
const store = transaction.objectStore(STORE_NAME);
|
||||
const request = store.get(id);
|
||||
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
},
|
||||
|
||||
// Get dataset by name
|
||||
async getDatasetByName(name) {
|
||||
await this.init();
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction([STORE_NAME], 'readonly');
|
||||
const store = transaction.objectStore(STORE_NAME);
|
||||
const index = store.index('name');
|
||||
const request = index.get(name);
|
||||
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
},
|
||||
|
||||
// Update dataset
|
||||
async updateDataset(id, updates) {
|
||||
await this.init();
|
||||
|
||||
const dataset = await this.getDataset(id);
|
||||
if (!dataset) {
|
||||
throw new Error('Dataset not found');
|
||||
}
|
||||
|
||||
// Update fields
|
||||
if (updates.name !== undefined) dataset.name = updates.name.trim();
|
||||
if (updates.data !== undefined) {
|
||||
dataset.data = updates.data;
|
||||
if (updates.format !== undefined) dataset.format = updates.format;
|
||||
if (updates.source !== undefined) dataset.source = updates.source;
|
||||
|
||||
// If metadata fields are explicitly provided, use them directly
|
||||
if (updates.rowCount !== undefined) dataset.rowCount = updates.rowCount;
|
||||
if (updates.columnCount !== undefined) dataset.columnCount = updates.columnCount;
|
||||
if (updates.columns !== undefined) dataset.columns = updates.columns;
|
||||
if (updates.size !== undefined) dataset.size = updates.size;
|
||||
|
||||
// Otherwise, calculate stats (for inline data)
|
||||
if (updates.rowCount === undefined && updates.columnCount === undefined) {
|
||||
const stats = calculateDatasetStats(updates.data, dataset.format, dataset.source);
|
||||
Object.assign(dataset, stats);
|
||||
}
|
||||
}
|
||||
if (updates.comment !== undefined) dataset.comment = updates.comment.trim();
|
||||
|
||||
dataset.modified = new Date().toISOString();
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction([STORE_NAME], 'readwrite');
|
||||
const store = transaction.objectStore(STORE_NAME);
|
||||
const request = store.put(dataset);
|
||||
|
||||
request.onsuccess = () => resolve(dataset);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
},
|
||||
|
||||
// Delete dataset
|
||||
async deleteDataset(id) {
|
||||
await this.init();
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction([STORE_NAME], 'readwrite');
|
||||
const store = transaction.objectStore(STORE_NAME);
|
||||
const request = store.delete(id);
|
||||
|
||||
request.onsuccess = () => resolve(true);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
},
|
||||
|
||||
// Check if dataset name exists
|
||||
async nameExists(name, excludeId = null) {
|
||||
const datasets = await this.listDatasets();
|
||||
return datasets.some(d => d.name === name && d.id !== excludeId);
|
||||
}
|
||||
};
|
||||
|
||||
// Format bytes for display
|
||||
function formatDatasetSize(bytes) {
|
||||
if (bytes < 1024) return `${bytes} B`;
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||
return `${(bytes / (1024 * 1024)).toFixed(2)} MB`;
|
||||
}
|
||||
|
||||
// Fetch URL data and calculate metadata
|
||||
async function fetchURLMetadata(url, format) {
|
||||
try {
|
||||
const response = await fetch(url);
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const contentLength = response.headers.get('content-length');
|
||||
const text = await response.text();
|
||||
|
||||
let rowCount = 0;
|
||||
let columnCount = 0;
|
||||
let columns = [];
|
||||
let size = contentLength ? parseInt(contentLength) : new Blob([text]).size;
|
||||
|
||||
// Parse based on format
|
||||
if (format === 'json') {
|
||||
const data = JSON.parse(text);
|
||||
if (Array.isArray(data)) {
|
||||
rowCount = data.length;
|
||||
if (data.length > 0 && typeof data[0] === 'object') {
|
||||
columns = Object.keys(data[0]);
|
||||
columnCount = columns.length;
|
||||
}
|
||||
}
|
||||
} else if (format === 'csv' || format === 'tsv') {
|
||||
const lines = text.trim().split('\n');
|
||||
rowCount = Math.max(0, lines.length - 1); // Subtract header
|
||||
if (lines.length > 0) {
|
||||
const separator = format === 'csv' ? ',' : '\t';
|
||||
columns = lines[0].split(separator).map(h => h.trim().replace(/^"|"$/g, ''));
|
||||
columnCount = columns.length;
|
||||
}
|
||||
} else if (format === 'topojson') {
|
||||
// TopoJSON structure is complex, just note it exists
|
||||
rowCount = null;
|
||||
columnCount = null;
|
||||
}
|
||||
|
||||
return { rowCount, columnCount, columns, size };
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to fetch URL metadata: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Render dataset list in modal
|
||||
async function renderDatasetList() {
|
||||
const datasets = await DatasetStorage.listDatasets();
|
||||
const listContainer = document.getElementById('dataset-list');
|
||||
|
||||
if (datasets.length === 0) {
|
||||
listContainer.innerHTML = '<div class="dataset-empty">No datasets yet. Click "New Dataset" to create one.</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
// Sort by modified date (most recent first)
|
||||
datasets.sort((a, b) => new Date(b.modified) - new Date(a.modified));
|
||||
|
||||
const html = datasets.map(dataset => {
|
||||
let metaText;
|
||||
if (dataset.source === 'url') {
|
||||
// Show metadata if available, otherwise just URL and format
|
||||
if (dataset.rowCount !== null && dataset.size !== null) {
|
||||
metaText = `URL • ${dataset.rowCount} rows • ${dataset.format.toUpperCase()} • ${formatDatasetSize(dataset.size)}`;
|
||||
} else {
|
||||
metaText = `URL • ${dataset.format.toUpperCase()}`;
|
||||
}
|
||||
} else {
|
||||
metaText = `${dataset.rowCount} rows • ${dataset.format.toUpperCase()} • ${formatDatasetSize(dataset.size)}`;
|
||||
}
|
||||
|
||||
return `
|
||||
<div class="dataset-item" data-dataset-id="${dataset.id}">
|
||||
<div class="dataset-info">
|
||||
<div class="dataset-name">${dataset.name}</div>
|
||||
<div class="dataset-meta">${metaText}</div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
|
||||
listContainer.innerHTML = html;
|
||||
|
||||
// Attach click handlers
|
||||
document.querySelectorAll('.dataset-item').forEach(item => {
|
||||
item.addEventListener('click', function() {
|
||||
const datasetId = parseFloat(this.dataset.datasetId);
|
||||
selectDataset(datasetId);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Select a dataset and show details
|
||||
async function selectDataset(datasetId) {
|
||||
const dataset = await DatasetStorage.getDataset(datasetId);
|
||||
if (!dataset) return;
|
||||
|
||||
// Update selection state
|
||||
document.querySelectorAll('.dataset-item').forEach(item => {
|
||||
item.classList.remove('selected');
|
||||
});
|
||||
document.querySelector(`[data-dataset-id="${datasetId}"]`).classList.add('selected');
|
||||
|
||||
// Show details panel
|
||||
const detailsPanel = document.getElementById('dataset-details');
|
||||
detailsPanel.style.display = 'block';
|
||||
|
||||
// Show/hide refresh button for URL datasets
|
||||
const refreshBtn = document.getElementById('refresh-metadata-btn');
|
||||
if (dataset.source === 'url') {
|
||||
refreshBtn.style.display = 'flex';
|
||||
} else {
|
||||
refreshBtn.style.display = 'none';
|
||||
}
|
||||
|
||||
// Populate details
|
||||
document.getElementById('dataset-detail-name').value = dataset.name;
|
||||
document.getElementById('dataset-detail-comment').value = dataset.comment;
|
||||
document.getElementById('dataset-detail-rows').textContent = dataset.rowCount !== null ? dataset.rowCount : 'N/A';
|
||||
document.getElementById('dataset-detail-columns').textContent = dataset.columnCount !== null ? dataset.columnCount : 'N/A';
|
||||
document.getElementById('dataset-detail-size').textContent = dataset.size !== null ? formatDatasetSize(dataset.size) : 'N/A';
|
||||
document.getElementById('dataset-detail-created').textContent = new Date(dataset.created).toLocaleString();
|
||||
document.getElementById('dataset-detail-modified').textContent = new Date(dataset.modified).toLocaleString();
|
||||
|
||||
// Show preview
|
||||
let previewText;
|
||||
if (dataset.source === 'url') {
|
||||
previewText = `URL: ${dataset.data}\nFormat: ${dataset.format.toUpperCase()}`;
|
||||
} else if (dataset.format === 'json' || dataset.format === 'topojson') {
|
||||
const previewData = Array.isArray(dataset.data) ? dataset.data.slice(0, 5) : dataset.data;
|
||||
previewText = JSON.stringify(previewData, null, 2);
|
||||
} else if (dataset.format === 'csv' || dataset.format === 'tsv') {
|
||||
const lines = dataset.data.split('\n');
|
||||
previewText = lines.slice(0, 6).join('\n'); // Header + 5 rows
|
||||
}
|
||||
document.getElementById('dataset-preview').textContent = previewText;
|
||||
|
||||
// Store current dataset ID
|
||||
window.currentDatasetId = datasetId;
|
||||
}
|
||||
|
||||
// Open dataset manager modal
|
||||
function openDatasetManager() {
|
||||
const modal = document.getElementById('dataset-modal');
|
||||
modal.style.display = 'flex';
|
||||
renderDatasetList();
|
||||
}
|
||||
|
||||
// Close dataset manager modal
|
||||
function closeDatasetManager() {
|
||||
const modal = document.getElementById('dataset-modal');
|
||||
modal.style.display = 'none';
|
||||
window.currentDatasetId = null;
|
||||
}
|
||||
|
||||
// Update format hint and placeholder
|
||||
function updateFormatHint(format) {
|
||||
const hintEl = document.getElementById('dataset-format-hint');
|
||||
const dataEl = document.getElementById('dataset-form-data');
|
||||
|
||||
if (format === 'json') {
|
||||
hintEl.textContent = 'JSON array of objects: [{"col1": "value", "col2": 123}, ...]';
|
||||
dataEl.placeholder = '[{"col1": "value", "col2": 123}, ...]';
|
||||
} else if (format === 'csv') {
|
||||
hintEl.textContent = 'CSV with header row: col1,col2\\nvalue1,123\\nvalue2,456';
|
||||
dataEl.placeholder = 'col1,col2\nvalue1,123\nvalue2,456';
|
||||
} else if (format === 'tsv') {
|
||||
hintEl.textContent = 'TSV with header row: col1\\tcol2\\nvalue1\\t123\\nvalue2\\t456';
|
||||
dataEl.placeholder = 'col1\tcol2\nvalue1\t123\nvalue2\t456';
|
||||
} else if (format === 'topojson') {
|
||||
hintEl.textContent = 'TopoJSON object: {"type": "Topology", "objects": {...}, "arcs": [...]}';
|
||||
dataEl.placeholder = '{"type": "Topology", "objects": {...}}';
|
||||
}
|
||||
}
|
||||
|
||||
// Toggle between URL and inline data inputs
|
||||
function toggleDataSource(source) {
|
||||
const urlGroup = document.getElementById('dataset-url-group');
|
||||
const dataGroup = document.getElementById('dataset-data-group');
|
||||
|
||||
if (source === 'url') {
|
||||
urlGroup.style.display = 'block';
|
||||
dataGroup.style.display = 'none';
|
||||
} else {
|
||||
urlGroup.style.display = 'none';
|
||||
dataGroup.style.display = 'block';
|
||||
}
|
||||
}
|
||||
|
||||
// Show new dataset form
|
||||
function showNewDatasetForm() {
|
||||
document.getElementById('dataset-list-view').style.display = 'none';
|
||||
document.getElementById('dataset-form-view').style.display = 'block';
|
||||
document.getElementById('dataset-form-name').value = '';
|
||||
document.getElementById('dataset-form-data').value = '';
|
||||
document.getElementById('dataset-form-url').value = '';
|
||||
document.getElementById('dataset-form-comment').value = '';
|
||||
document.getElementById('dataset-form-error').textContent = '';
|
||||
|
||||
// Reset to inline data source and JSON format
|
||||
document.querySelectorAll('[data-source]').forEach(btn => {
|
||||
btn.classList.toggle('active', btn.dataset.source === 'inline');
|
||||
});
|
||||
document.querySelectorAll('[data-format]').forEach(btn => {
|
||||
btn.classList.toggle('active', btn.dataset.format === 'json');
|
||||
});
|
||||
toggleDataSource('inline');
|
||||
updateFormatHint('json');
|
||||
|
||||
// Add listeners if not already added
|
||||
if (!window.datasetListenersAdded) {
|
||||
// Source toggle button listeners
|
||||
document.querySelectorAll('[data-source]').forEach(btn => {
|
||||
btn.addEventListener('click', function () {
|
||||
// Update active state
|
||||
document.querySelectorAll('[data-source]').forEach(b => b.classList.remove('active'));
|
||||
this.classList.add('active');
|
||||
toggleDataSource(this.dataset.source);
|
||||
});
|
||||
});
|
||||
|
||||
// Format toggle button listeners
|
||||
document.querySelectorAll('[data-format]').forEach(btn => {
|
||||
btn.addEventListener('click', function () {
|
||||
// Update active state
|
||||
document.querySelectorAll('[data-format]').forEach(b => b.classList.remove('active'));
|
||||
this.classList.add('active');
|
||||
updateFormatHint(this.dataset.format);
|
||||
});
|
||||
});
|
||||
|
||||
window.datasetListenersAdded = true;
|
||||
}
|
||||
}
|
||||
|
||||
// Hide new dataset form
|
||||
function hideNewDatasetForm() {
|
||||
document.getElementById('dataset-list-view').style.display = 'block';
|
||||
document.getElementById('dataset-form-view').style.display = 'none';
|
||||
}
|
||||
|
||||
// Save new dataset
|
||||
async function saveNewDataset() {
|
||||
const name = document.getElementById('dataset-form-name').value.trim();
|
||||
const source = document.querySelector('[data-source].active').dataset.source;
|
||||
const format = document.querySelector('[data-format].active').dataset.format;
|
||||
const comment = document.getElementById('dataset-form-comment').value.trim();
|
||||
const errorEl = document.getElementById('dataset-form-error');
|
||||
|
||||
errorEl.textContent = '';
|
||||
|
||||
// Validation
|
||||
if (!name) {
|
||||
errorEl.textContent = 'Dataset name is required';
|
||||
return;
|
||||
}
|
||||
|
||||
let data;
|
||||
let metadata = null;
|
||||
|
||||
if (source === 'url') {
|
||||
const url = document.getElementById('dataset-form-url').value.trim();
|
||||
if (!url) {
|
||||
errorEl.textContent = 'URL is required';
|
||||
return;
|
||||
}
|
||||
// Basic URL validation
|
||||
try {
|
||||
new URL(url);
|
||||
} catch (error) {
|
||||
errorEl.textContent = 'Invalid URL format';
|
||||
return;
|
||||
}
|
||||
|
||||
// Fetch metadata from URL
|
||||
errorEl.textContent = 'Fetching data from URL...';
|
||||
try {
|
||||
metadata = await fetchURLMetadata(url, format);
|
||||
errorEl.textContent = '';
|
||||
} catch (error) {
|
||||
errorEl.textContent = `Warning: ${error.message}. Dataset will be created without metadata.`;
|
||||
// Continue anyway - URL might require CORS or auth
|
||||
await new Promise(resolve => setTimeout(resolve, 2000)); // Show warning briefly
|
||||
errorEl.textContent = '';
|
||||
}
|
||||
|
||||
data = url; // Store the URL string
|
||||
} else {
|
||||
// Inline data
|
||||
const dataText = document.getElementById('dataset-form-data').value.trim();
|
||||
if (!dataText) {
|
||||
errorEl.textContent = 'Data is required';
|
||||
return;
|
||||
}
|
||||
|
||||
// Basic validation of data format
|
||||
try {
|
||||
if (format === 'json' || format === 'topojson') {
|
||||
const parsed = JSON.parse(dataText);
|
||||
if (format === 'json' && !Array.isArray(parsed)) {
|
||||
errorEl.textContent = 'JSON data must be an array of objects';
|
||||
return;
|
||||
}
|
||||
if (format === 'json' && parsed.length === 0) {
|
||||
errorEl.textContent = 'Data array cannot be empty';
|
||||
return;
|
||||
}
|
||||
data = parsed; // Store as parsed JSON
|
||||
} else if (format === 'csv' || format === 'tsv') {
|
||||
const lines = dataText.trim().split('\n');
|
||||
if (lines.length < 2) {
|
||||
errorEl.textContent = `${format.toUpperCase()} must have at least a header row and one data row`;
|
||||
return;
|
||||
}
|
||||
data = dataText; // Store as raw CSV/TSV string
|
||||
}
|
||||
} catch (error) {
|
||||
errorEl.textContent = `Validation error: ${error.message}`;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Check if name already exists
|
||||
if (await DatasetStorage.nameExists(name)) {
|
||||
errorEl.textContent = 'A dataset with this name already exists';
|
||||
return;
|
||||
}
|
||||
|
||||
// Create dataset
|
||||
try {
|
||||
const dataset = await DatasetStorage.createDataset(name, data, format, source, comment);
|
||||
|
||||
// If we have metadata from URL fetch, update the dataset
|
||||
if (metadata) {
|
||||
await DatasetStorage.updateDataset(dataset.id, {
|
||||
data: data,
|
||||
...metadata
|
||||
});
|
||||
}
|
||||
|
||||
hideNewDatasetForm();
|
||||
await renderDatasetList();
|
||||
} catch (error) {
|
||||
errorEl.textContent = `Failed to save dataset: ${error.message}`;
|
||||
}
|
||||
}
|
||||
|
||||
// Delete current dataset
|
||||
async function deleteCurrentDataset() {
|
||||
if (!window.currentDatasetId) return;
|
||||
|
||||
const dataset = await DatasetStorage.getDataset(window.currentDatasetId);
|
||||
if (!dataset) return;
|
||||
|
||||
if (confirm(`Delete dataset "${dataset.name}"? This action cannot be undone.`)) {
|
||||
await DatasetStorage.deleteDataset(window.currentDatasetId);
|
||||
document.getElementById('dataset-details').style.display = 'none';
|
||||
window.currentDatasetId = null;
|
||||
await renderDatasetList();
|
||||
}
|
||||
}
|
||||
|
||||
// Copy dataset reference to clipboard
|
||||
function copyDatasetReference() {
|
||||
if (!window.currentDatasetId) return;
|
||||
|
||||
DatasetStorage.getDataset(window.currentDatasetId).then(dataset => {
|
||||
const reference = `"data": {"name": "${dataset.name}"}`;
|
||||
navigator.clipboard.writeText(reference).then(() => {
|
||||
alert('Dataset reference copied to clipboard!');
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Refresh metadata for URL dataset
|
||||
async function refreshDatasetMetadata() {
|
||||
if (!window.currentDatasetId) return;
|
||||
|
||||
const dataset = await DatasetStorage.getDataset(window.currentDatasetId);
|
||||
if (!dataset || dataset.source !== 'url') return;
|
||||
|
||||
const refreshBtn = document.getElementById('refresh-metadata-btn');
|
||||
refreshBtn.disabled = true;
|
||||
refreshBtn.textContent = '⏳';
|
||||
|
||||
try {
|
||||
const metadata = await fetchURLMetadata(dataset.data, dataset.format);
|
||||
|
||||
// Update dataset with new metadata
|
||||
await DatasetStorage.updateDataset(dataset.id, {
|
||||
data: dataset.data,
|
||||
...metadata
|
||||
});
|
||||
|
||||
// Refresh the display
|
||||
await selectDataset(dataset.id);
|
||||
await renderDatasetList();
|
||||
|
||||
// Brief success indicator
|
||||
refreshBtn.textContent = '✓';
|
||||
setTimeout(() => {
|
||||
refreshBtn.textContent = '🔄';
|
||||
refreshBtn.disabled = false;
|
||||
}, 1000);
|
||||
} catch (error) {
|
||||
alert(`Failed to refresh metadata: ${error.message}`);
|
||||
refreshBtn.textContent = '🔄';
|
||||
refreshBtn.disabled = false;
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,76 @@
|
||||
// Resolve dataset references in a spec
|
||||
async function resolveDatasetReferences(spec) {
|
||||
// If spec has data.name, look it up
|
||||
if (spec.data && spec.data.name && typeof spec.data.name === 'string') {
|
||||
const datasetName = spec.data.name;
|
||||
const dataset = await DatasetStorage.getDatasetByName(datasetName);
|
||||
|
||||
if (dataset) {
|
||||
// Replace data reference with actual data in the format Vega-Lite expects
|
||||
if (dataset.source === 'url') {
|
||||
// For URL sources, pass the URL and format
|
||||
spec.data = {
|
||||
url: dataset.data,
|
||||
format: { type: dataset.format }
|
||||
};
|
||||
} else {
|
||||
// For inline sources
|
||||
if (dataset.format === 'json') {
|
||||
spec.data = { values: dataset.data };
|
||||
} else if (dataset.format === 'csv') {
|
||||
spec.data = {
|
||||
values: dataset.data,
|
||||
format: { type: 'csv' }
|
||||
};
|
||||
} else if (dataset.format === 'tsv') {
|
||||
spec.data = {
|
||||
values: dataset.data,
|
||||
format: { type: 'tsv' }
|
||||
};
|
||||
} else if (dataset.format === 'topojson') {
|
||||
spec.data = {
|
||||
values: dataset.data,
|
||||
format: { type: 'topojson' }
|
||||
};
|
||||
}
|
||||
}
|
||||
} else {
|
||||
throw new Error(`Dataset "${datasetName}" not found`);
|
||||
}
|
||||
}
|
||||
|
||||
// Recursively resolve in layers (for layered specs)
|
||||
if (spec.layer && Array.isArray(spec.layer)) {
|
||||
for (let layer of spec.layer) {
|
||||
await resolveDatasetReferences(layer);
|
||||
}
|
||||
}
|
||||
|
||||
// Recursively resolve in concat/hconcat/vconcat
|
||||
if (spec.concat && Array.isArray(spec.concat)) {
|
||||
for (let view of spec.concat) {
|
||||
await resolveDatasetReferences(view);
|
||||
}
|
||||
}
|
||||
if (spec.hconcat && Array.isArray(spec.hconcat)) {
|
||||
for (let view of spec.hconcat) {
|
||||
await resolveDatasetReferences(view);
|
||||
}
|
||||
}
|
||||
if (spec.vconcat && Array.isArray(spec.vconcat)) {
|
||||
for (let view of spec.vconcat) {
|
||||
await resolveDatasetReferences(view);
|
||||
}
|
||||
}
|
||||
|
||||
// Recursively resolve in facet
|
||||
if (spec.spec) {
|
||||
await resolveDatasetReferences(spec.spec);
|
||||
}
|
||||
|
||||
return spec;
|
||||
}
|
||||
|
||||
// Render function that takes spec from editor
|
||||
async function renderVisualization() {
|
||||
const previewContainer = document.getElementById('vega-preview');
|
||||
@@ -5,7 +78,10 @@ async function renderVisualization() {
|
||||
try {
|
||||
// Get current content from editor
|
||||
const specText = editor.getValue();
|
||||
const spec = JSON.parse(specText);
|
||||
let spec = JSON.parse(specText);
|
||||
|
||||
// Resolve dataset references
|
||||
spec = await resolveDatasetReferences(spec);
|
||||
|
||||
// Render with Vega-Embed (use global variable)
|
||||
await window.vegaEmbed('#vega-preview', spec, {
|
||||
|
||||
404
src/styles.css
404
src/styles.css
@@ -661,4 +661,408 @@ body {
|
||||
|
||||
.storage-fill.critical {
|
||||
background: #ff0000;
|
||||
}
|
||||
|
||||
/* Modal Styles */
|
||||
.modal {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100vw;
|
||||
height: 100vh;
|
||||
background: rgba(0, 0, 0, 0.5);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
z-index: 1000;
|
||||
}
|
||||
|
||||
.modal-content {
|
||||
background: #c0c0c0;
|
||||
border: 2px outset #c0c0c0;
|
||||
width: 90%;
|
||||
max-width: 900px;
|
||||
max-height: 90vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
box-shadow: 4px 4px 8px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
.modal-header {
|
||||
background: #000080;
|
||||
color: #ffffff;
|
||||
padding: 4px 8px;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
height: 24px;
|
||||
border-bottom: 2px solid #808080;
|
||||
}
|
||||
|
||||
.modal-title {
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.modal-close {
|
||||
background: #c0c0c0;
|
||||
border: 2px outset #c0c0c0;
|
||||
color: #000000;
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
cursor: pointer;
|
||||
font-size: 14px;
|
||||
font-family: 'MS Sans Serif', Tahoma, sans-serif;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.modal-close:hover {
|
||||
background: #d4d0c8;
|
||||
}
|
||||
|
||||
.modal-close:active {
|
||||
border: 2px inset #c0c0c0;
|
||||
}
|
||||
|
||||
.modal-body {
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
background: #ffffff;
|
||||
border: 2px inset #c0c0c0;
|
||||
margin: 8px;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* Dataset Views */
|
||||
.dataset-view {
|
||||
min-height: 100%;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* List View */
|
||||
.dataset-list-header {
|
||||
padding: 8px;
|
||||
background: #d4d0c8;
|
||||
border-bottom: 2px solid #808080;
|
||||
}
|
||||
|
||||
.dataset-container {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.dataset-list {
|
||||
width: 300px;
|
||||
overflow-y: auto;
|
||||
border-right: 2px solid #808080;
|
||||
background: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-item {
|
||||
padding: 8px;
|
||||
border-bottom: 1px solid #d0d0d0;
|
||||
cursor: pointer;
|
||||
background: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-item:hover {
|
||||
background: #6a9ad5;
|
||||
color: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-item.selected {
|
||||
background: #316ac5;
|
||||
color: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-name {
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
margin-bottom: 2px;
|
||||
}
|
||||
|
||||
.dataset-meta {
|
||||
font-size: 10px;
|
||||
color: #606060;
|
||||
}
|
||||
|
||||
.dataset-item.selected .dataset-meta,
|
||||
.dataset-item:hover .dataset-meta {
|
||||
color: inherit;
|
||||
opacity: 0.9;
|
||||
}
|
||||
|
||||
.dataset-empty {
|
||||
padding: 32px;
|
||||
text-align: center;
|
||||
color: #808080;
|
||||
font-style: italic;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
/* Dataset Details */
|
||||
.dataset-details {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
background: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-detail-section {
|
||||
padding: 16px;
|
||||
}
|
||||
|
||||
.dataset-detail-header {
|
||||
font-size: 11px;
|
||||
font-weight: bold;
|
||||
margin-bottom: 6px;
|
||||
margin-top: 12px;
|
||||
color: #000000;
|
||||
}
|
||||
|
||||
.dataset-detail-header:first-child {
|
||||
margin-top: 0;
|
||||
}
|
||||
|
||||
.dataset-detail-header-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 6px;
|
||||
margin-top: 12px;
|
||||
}
|
||||
|
||||
.dataset-refresh-btn {
|
||||
background: #c0c0c0;
|
||||
border: 2px outset #c0c0c0;
|
||||
color: #000000;
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.dataset-refresh-btn:hover {
|
||||
background: #d4d0c8;
|
||||
}
|
||||
|
||||
.dataset-refresh-btn:active {
|
||||
border: 2px inset #c0c0c0;
|
||||
}
|
||||
|
||||
.dataset-refresh-btn:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.dataset-input,
|
||||
.dataset-textarea {
|
||||
width: 100%;
|
||||
font-family: 'MS Sans Serif', Tahoma, sans-serif;
|
||||
font-size: 11px;
|
||||
border: 2px inset #c0c0c0;
|
||||
padding: 4px;
|
||||
background: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-textarea {
|
||||
resize: vertical;
|
||||
}
|
||||
|
||||
.dataset-stats {
|
||||
background: #f0f0f0;
|
||||
border: 1px inset #c0c0c0;
|
||||
padding: 8px;
|
||||
font-size: 10px;
|
||||
}
|
||||
|
||||
.dataset-stat-item {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
|
||||
.dataset-stat-item:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.dataset-stat-label {
|
||||
font-weight: bold;
|
||||
color: #000000;
|
||||
}
|
||||
|
||||
.dataset-preview-box {
|
||||
background: #f0f0f0;
|
||||
border: 2px inset #c0c0c0;
|
||||
padding: 8px;
|
||||
font-family: 'Courier New', monospace;
|
||||
font-size: 10px;
|
||||
overflow: auto;
|
||||
max-height: 200px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.dataset-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-top: 16px;
|
||||
}
|
||||
|
||||
/* Dataset Form */
|
||||
.dataset-form {
|
||||
padding: 16px;
|
||||
height: 100%;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.dataset-form-header {
|
||||
font-size: 14px;
|
||||
font-weight: bold;
|
||||
margin-bottom: 16px;
|
||||
color: #000000;
|
||||
}
|
||||
|
||||
.dataset-form-group {
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.dataset-form-label {
|
||||
display: block;
|
||||
font-size: 11px;
|
||||
font-weight: bold;
|
||||
margin-bottom: 4px;
|
||||
color: #000000;
|
||||
}
|
||||
|
||||
.dataset-toggle-row {
|
||||
display: flex;
|
||||
gap: 24px;
|
||||
padding: 8px;
|
||||
background: #f0f0f0;
|
||||
border: 1px inset #c0c0c0;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.dataset-toggle-section {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.dataset-toggle-label {
|
||||
font-size: 10px;
|
||||
color: #000000;
|
||||
}
|
||||
|
||||
.dataset-toggle-group {
|
||||
display: flex;
|
||||
}
|
||||
|
||||
.dataset-toggle-btn {
|
||||
background: #c0c0c0;
|
||||
border: 1px solid #808080;
|
||||
color: #000000;
|
||||
padding: 2px 8px;
|
||||
cursor: pointer;
|
||||
font-size: 10px;
|
||||
font-family: 'MS Sans Serif', Tahoma, sans-serif;
|
||||
height: 20px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.dataset-toggle-btn:not(:first-child) {
|
||||
border-left: none;
|
||||
}
|
||||
|
||||
.dataset-toggle-btn:hover:not(.active) {
|
||||
background: #d4d0c8;
|
||||
}
|
||||
|
||||
.dataset-toggle-btn:active {
|
||||
background: #316ac5;
|
||||
color: #ffffff;
|
||||
}
|
||||
|
||||
.dataset-toggle-btn.active {
|
||||
background: #316ac5;
|
||||
color: #ffffff;
|
||||
border-top: 1px solid #0a246a;
|
||||
border-left: 1px solid #0a246a;
|
||||
border-bottom: 1px solid #4a7ac5;
|
||||
border-right: 1px solid #4a7ac5;
|
||||
}
|
||||
|
||||
.dataset-toggle-btn.active:not(:first-child) {
|
||||
border-left: 1px solid #0a246a;
|
||||
}
|
||||
|
||||
.dataset-format-hint {
|
||||
font-size: 10px;
|
||||
color: #606060;
|
||||
font-style: italic;
|
||||
margin-bottom: 4px;
|
||||
padding: 4px;
|
||||
background: #fffacd;
|
||||
border: 1px solid #e0e0a0;
|
||||
}
|
||||
|
||||
.dataset-form-error {
|
||||
color: #ff0000;
|
||||
font-size: 11px;
|
||||
margin-bottom: 12px;
|
||||
min-height: 16px;
|
||||
}
|
||||
|
||||
.dataset-form-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-top: 16px;
|
||||
}
|
||||
|
||||
/* Modal Buttons */
|
||||
.modal-btn {
|
||||
background: #c0c0c0;
|
||||
border: 2px outset #c0c0c0;
|
||||
color: #000000;
|
||||
padding: 6px 12px;
|
||||
cursor: pointer;
|
||||
font-size: 11px;
|
||||
font-family: 'MS Sans Serif', Tahoma, sans-serif;
|
||||
}
|
||||
|
||||
.modal-btn:hover {
|
||||
background: #d4d0c8;
|
||||
}
|
||||
|
||||
.modal-btn:active {
|
||||
border: 2px inset #c0c0c0;
|
||||
}
|
||||
|
||||
.modal-btn.primary {
|
||||
background: #90ee90;
|
||||
}
|
||||
|
||||
.modal-btn.primary:hover {
|
||||
background: #a0ffa0;
|
||||
}
|
||||
|
||||
.modal-btn.delete-btn {
|
||||
background: #ff8080;
|
||||
border: 2px outset #ff8080;
|
||||
}
|
||||
|
||||
.modal-btn.delete-btn:hover {
|
||||
background: #ff9999;
|
||||
}
|
||||
|
||||
.modal-btn.delete-btn:active {
|
||||
border: 2px inset #ff8080;
|
||||
}
|
||||
Reference in New Issue
Block a user