mirror of
https://github.com/olehomelchenko/astrolabe-nvc.git
synced 2025-12-21 21:22:23 +00:00
feat: dataset preview and interconnection (phase 12)
This commit is contained in:
9
.claude/settings.local.json
Normal file
9
.claude/settings.local.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(awk:*)"
|
||||
],
|
||||
"deny": [],
|
||||
"ask": []
|
||||
}
|
||||
}
|
||||
10
CLAUDE.md
10
CLAUDE.md
@@ -17,8 +17,8 @@ Instructions for Claude Code when working on this project.
|
||||
|
||||
## Current Status
|
||||
|
||||
**Completed**: Phases 0-10 (Core functionality + Dataset Management)
|
||||
**Next**: Phase 11 - Advanced Dataset Features (optional enhancements)
|
||||
**Completed**: Phases 0-12 (Core functionality + Dataset Management + Advanced Dataset Features)
|
||||
**Next**: Phase 13 - Polish & UX Refinements or Phase 14 - Advanced Snippet Features
|
||||
|
||||
### Key Features Implemented
|
||||
- ✅ Snippet management with draft/published workflow
|
||||
@@ -30,5 +30,11 @@ Instructions for Claude Code when working on this project.
|
||||
- Automatic metadata calculation and URL fetching
|
||||
- Dataset reference resolution in Vega-Lite specs
|
||||
- Modal UI with button-group selectors
|
||||
- ✅ **Advanced Dataset Features (Phase 12)**
|
||||
- Bidirectional snippet ↔ dataset linking with usage tracking
|
||||
- Extract inline data to datasets
|
||||
- Import/Export datasets with auto-format detection
|
||||
- Table preview with type detection (🔢📅🔤✓)
|
||||
- On-demand URL preview loading with caching
|
||||
|
||||
See `docs/dev-plan.md` for complete roadmap and technical details.
|
||||
|
||||
@@ -300,19 +300,39 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
|
||||
---
|
||||
|
||||
### **Phase 12: Advanced Dataset Features** _(Future)_
|
||||
### **Phase 12: Advanced Dataset Features** ✅ **COMPLETE**
|
||||
**Goal**: Enhanced dataset workflows
|
||||
|
||||
- [ ] Detect inline data in Vega-Lite specs
|
||||
- [ ] "Extract to dataset" feature for inline data
|
||||
- [ ] Update snippet UI to show linked datasets
|
||||
- [ ] Dataset usage tracking (which snippets reference which datasets)
|
||||
- [ ] Import datasets from file upload
|
||||
- [ ] Export individual datasets
|
||||
- [ ] Dataset preview with table view
|
||||
- [ ] Column type detection and display
|
||||
- [x] Detect inline data in Vega-Lite specs
|
||||
- [x] "Extract to dataset" feature for inline data
|
||||
- [x] Update snippet UI to show linked datasets
|
||||
- [x] Dataset usage tracking (which snippets reference which datasets)
|
||||
- [x] Bidirectional linking between snippets and datasets
|
||||
- [x] Usage count badges on dataset list items
|
||||
- [x] "New Snippet" button to create snippet from dataset
|
||||
- [x] Import datasets from file upload
|
||||
- [x] Export individual datasets
|
||||
- [x] Dataset preview with table view
|
||||
- [x] Column type detection and display
|
||||
|
||||
**Deliverable**: Advanced dataset management and discovery
|
||||
**Deliverables**:
|
||||
- Recursive dataset reference extraction from Vega-Lite specs
|
||||
- Extract to Dataset modal with automatic spec transformation
|
||||
- Bidirectional linking between snippets and datasets
|
||||
- Usage tracking with visual indicators (📁 icon, 📄 count badges)
|
||||
- Dataset import/export with auto-format detection
|
||||
- Table preview with type detection (🔢📅✓🔤)
|
||||
- On-demand URL preview loading with session cache
|
||||
- New Snippet creation from datasets
|
||||
|
||||
**Technical Implementation**:
|
||||
- Type detection: 80% threshold for number/date/boolean/text inference
|
||||
- Table rendering: Vanilla JS with sticky headers, 20-row preview
|
||||
- Cell formatting: Type-specific styling and alignment
|
||||
- Import: Automatic naming with timestamp collision handling
|
||||
- Export: Format-specific extensions with URL content fetching
|
||||
- Preview cache: In-memory storage, not persisted to database
|
||||
- Modal: 95% width (max 1200px), 85% height for improved data visibility
|
||||
|
||||
---
|
||||
|
||||
@@ -411,8 +431,8 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
|
||||
## Current Status
|
||||
|
||||
**Completed**: Phases 0-11 (Storage, UI, editor, rendering, persistence, CRUD, organization, draft/published workflow, storage monitoring, import/export, dataset management, URL state management)
|
||||
**Next**: Phase 12 - Advanced Dataset Features (optional enhancements)
|
||||
**Completed**: Phases 0-12 (Storage, UI, editor, rendering, persistence, CRUD, organization, draft/published workflow, storage monitoring, import/export, dataset management, URL state management, advanced dataset features)
|
||||
**Next**: Phase 13 - Polish & UX Refinements or Phase 14 - Advanced Snippet Features
|
||||
**See**: `CLAUDE.md` for concise current state summary
|
||||
|
||||
---
|
||||
@@ -450,6 +470,13 @@ Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visu
|
||||
- Page reload preserves state
|
||||
- Shareable URLs for specific snippets/datasets
|
||||
- Restores snippet URL when closing dataset modal
|
||||
- **Advanced Dataset Features (Phase 12)**:
|
||||
- Bidirectional linking with usage tracking and navigation
|
||||
- Extract to Dataset with automatic spec transformation
|
||||
- Import/Export with format detection and URL content fetching
|
||||
- Table preview with type detection and formatting (🔢📅✓🔤)
|
||||
- On-demand URL preview loading with session cache
|
||||
- New Snippet creation from datasets
|
||||
- Retro Windows 2000 aesthetic throughout
|
||||
- Component-based CSS architecture with base classes
|
||||
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
|
||||
> **Purpose**: Comprehensive inventory of all implemented features for code review and optimization
|
||||
> **Created**: 2025-10-15
|
||||
> **Status**: Phases 0-10 Complete
|
||||
> **Updated**: 2025-10-16
|
||||
> **Status**: Phases 0-12 Complete
|
||||
|
||||
---
|
||||
|
||||
@@ -200,15 +201,67 @@
|
||||
|
||||
---
|
||||
|
||||
### **13. Advanced Dataset Features (Phase 12)**
|
||||
- **Dataset Dependencies & Linking**:
|
||||
- Recursive dataset reference extraction from Vega-Lite specs
|
||||
- Bidirectional snippet ↔ dataset linking with clickable links
|
||||
- Dataset usage tracking and count badges (📄 count)
|
||||
- Visual indicators (📁) for snippets using datasets
|
||||
- Linked datasets section in snippet metadata
|
||||
- Linked snippets section in dataset details
|
||||
- Navigation between snippets and datasets via links
|
||||
- **Extract to Dataset**:
|
||||
- Inline data detection in draft specs
|
||||
- Extract modal with name generation and preview
|
||||
- Automatic spec transformation (inline → reference)
|
||||
- Auto-update snippet with dataset reference
|
||||
- Conditional Extract button (shows when inline data detected)
|
||||
- **Import/Export Datasets**:
|
||||
- Import from file (.json, .csv, .tsv, .txt)
|
||||
- Auto-format detection from content and filename
|
||||
- Automatic naming from filename with duplicate handling
|
||||
- Timestamp suffix for duplicate names (e.g., "data_123456")
|
||||
- Export to format-specific files (.json, .csv, .tsv, .topojson)
|
||||
- URL dataset export fetches and downloads live content
|
||||
- **Table Preview with Type Detection**:
|
||||
- Raw/Table toggle for tabular data (JSON arrays, CSV, TSV)
|
||||
- Vanilla JS table rendering (first 20 rows)
|
||||
- Sticky headers with scrollable content
|
||||
- Column type detection (80% threshold): number, date, boolean, text
|
||||
- Type icons in headers: 🔢 📅 ✓ 🔤
|
||||
- Type-specific formatting: italic numbers (right-aligned), italic dates, bold booleans
|
||||
- Color coding: blue numbers, green dates, orange booleans, gray nulls
|
||||
- Monospace font (Consolas/Monaco/Courier New)
|
||||
- **URL Preview Loading**:
|
||||
- On-demand fetch with "Load Preview" button
|
||||
- In-memory cache for fetched data (session-only)
|
||||
- Full table view and type detection for fetched data
|
||||
- Error handling with retry option
|
||||
- Data not saved to database (preview only)
|
||||
- **New Snippet from Dataset**:
|
||||
- Button in dataset details to create snippet
|
||||
- Auto-generated minimal Vega-Lite spec with dataset reference
|
||||
- Pre-populated comment and datasetRefs
|
||||
- **UI Enhancements**:
|
||||
- Larger modal (95% width, max 1200px, 85% height)
|
||||
- Actions moved to top of dataset details
|
||||
- Dataset list with usage badges
|
||||
|
||||
**Files**: `dataset-manager.js` (lines 19-1165), `snippet-manager.js` (dataset tracking), `app.js` (event handlers), `index.html` (UI), `styles.css` (table styles)
|
||||
|
||||
---
|
||||
|
||||
## 📊 **Feature Statistics**
|
||||
|
||||
- **Core Feature Groups**: 13
|
||||
- **Total Individual Capabilities**: ~70+
|
||||
- **Core Feature Groups**: 14
|
||||
- **Total Individual Capabilities**: ~100+
|
||||
- **Storage Systems**: 2 (localStorage for snippets, IndexedDB for datasets)
|
||||
- **UI Panels**: 3 main + 1 modal
|
||||
- **Auto-save Points**: 3 (draft spec, name, comment)
|
||||
- **Data Formats**: 4 (JSON, CSV, TSV, TopoJSON)
|
||||
- **Data Sources**: 2 (inline, URL)
|
||||
- **Type Detection**: 4 types (number, date, boolean, text)
|
||||
- **Import/Export**: Snippets + Datasets
|
||||
|
||||
---
|
||||
|
||||
@@ -218,15 +271,15 @@
|
||||
src/
|
||||
├── js/
|
||||
│ ├── config.js # Global variables, settings, sample data
|
||||
│ ├── snippet-manager.js # Snippet CRUD, storage, search, sort (977 lines)
|
||||
│ ├── dataset-manager.js # Dataset CRUD, IndexedDB, auto-detection (714 lines)
|
||||
│ ├── snippet-manager.js # Snippet CRUD, storage, search, sort, extract (1,100+ lines)
|
||||
│ ├── dataset-manager.js # Dataset CRUD, IndexedDB, preview, types (1,200+ lines)
|
||||
│ ├── panel-manager.js # Layout resizing, toggling, persistence (200 lines)
|
||||
│ ├── editor.js # Monaco setup, Vega rendering, dataset resolution (150 lines)
|
||||
│ └── app.js # Event handlers, initialization (197 lines)
|
||||
└── styles.css # Retro Windows 2000 aesthetic
|
||||
│ └── app.js # Event handlers, initialization (250+ lines)
|
||||
└── styles.css # Retro Windows 2000 aesthetic (280+ lines)
|
||||
```
|
||||
|
||||
**Total JS Lines**: ~2,238 lines (excluding comments and blank lines)
|
||||
**Total JS Lines**: ~2,900+ lines (excluding comments and blank lines)
|
||||
|
||||
---
|
||||
|
||||
@@ -242,7 +295,11 @@ src/
|
||||
|
||||
## 📝 **Next Steps**
|
||||
|
||||
- Choose a feature number (1-12) to review in detail
|
||||
- Analyze code for that feature to ensure all parts are necessary
|
||||
- Remove dead code or consolidate redundancies
|
||||
- Document any optimizations made
|
||||
### Phase 12 Complete! All dataset features implemented.
|
||||
|
||||
**Potential Next Phases**:
|
||||
- **Phase 13**: Polish & UX Refinements (keyboard shortcuts, tooltips, loading states)
|
||||
- **Phase 14**: Advanced Snippet Features (tagging system, templates, bulk operations)
|
||||
- **Phase 15**: Authentication & Backend (cloud sync, sharing)
|
||||
|
||||
**Current Priority**: Code review and optimization of Phase 12 features if needed.
|
||||
|
||||
64
index.html
64
index.html
@@ -94,6 +94,13 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="snippet-datasets-section" style="display: none;">
|
||||
<div class="meta-header">Linked Datasets</div>
|
||||
<div class="meta-info" id="snippet-datasets">
|
||||
<!-- Dynamically populated by updateLinkedDatasets() -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="meta-actions">
|
||||
<button class="btn btn-standard flex" id="duplicate-btn">Duplicate</button>
|
||||
<button class="btn btn-standard flex danger" id="delete-btn">Delete</button>
|
||||
@@ -119,6 +126,7 @@
|
||||
<div class="panel-header">
|
||||
<span>Editor</span>
|
||||
<div class="editor-controls">
|
||||
<button class="btn btn-action" id="extract-btn" style="display: none; background: #87CEEB;">Extract to Dataset</button>
|
||||
<button class="btn btn-action publish" id="publish-btn">Publish</button>
|
||||
<button class="btn btn-action revert" id="revert-btn">Revert</button>
|
||||
<span class="view-label">View:</span>
|
||||
@@ -160,6 +168,8 @@
|
||||
<div id="dataset-list-view" class="dataset-view">
|
||||
<div class="dataset-list-header">
|
||||
<button class="btn btn-modal primary" id="new-dataset-btn">New Dataset</button>
|
||||
<button class="btn btn-modal" id="import-dataset-btn">Import</button>
|
||||
<input type="file" id="import-dataset-file" accept=".json,.csv,.tsv,.txt" style="display: none;" />
|
||||
</div>
|
||||
<div class="dataset-container">
|
||||
<div class="dataset-list" id="dataset-list">
|
||||
@@ -167,6 +177,13 @@
|
||||
</div>
|
||||
<div class="dataset-details" id="dataset-details" style="display: none;">
|
||||
<div class="dataset-detail-section">
|
||||
<div class="dataset-actions">
|
||||
<button class="btn btn-modal primary" id="new-snippet-btn">New Snippet</button>
|
||||
<button class="btn btn-modal" id="export-dataset-btn">Export</button>
|
||||
<button class="btn btn-modal" id="copy-reference-btn">Copy Reference</button>
|
||||
<button class="btn btn-modal danger" id="delete-dataset-btn">Delete</button>
|
||||
</div>
|
||||
|
||||
<div class="dataset-detail-header">Name</div>
|
||||
<input type="text" id="dataset-detail-name" class="input" placeholder="Dataset name..." />
|
||||
|
||||
@@ -204,12 +221,21 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="dataset-detail-header">Preview</div>
|
||||
<div class="dataset-detail-header-row">
|
||||
<span class="dataset-detail-header">Preview</span>
|
||||
<div class="preview-toggle-group" id="preview-toggle-group" style="display: none;">
|
||||
<button class="btn btn-toggle small active" id="preview-raw-btn">Raw</button>
|
||||
<button class="btn btn-toggle small" id="preview-table-btn">Table</button>
|
||||
</div>
|
||||
</div>
|
||||
<pre id="dataset-preview" class="preview-box large"></pre>
|
||||
<div id="dataset-preview-table" class="preview-table-container" style="display: none;"></div>
|
||||
|
||||
<div class="dataset-actions">
|
||||
<button class="btn btn-modal" id="copy-reference-btn">Copy Reference</button>
|
||||
<button class="btn btn-modal danger" id="delete-dataset-btn">Delete</button>
|
||||
<div id="dataset-snippets-section" style="display: none;">
|
||||
<div class="dataset-detail-header">Linked Snippets</div>
|
||||
<div class="stats-box" id="dataset-snippets">
|
||||
<!-- Dynamically populated by updateLinkedSnippets() -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -265,6 +291,36 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Extract to Dataset Modal -->
|
||||
<div id="extract-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 600px; height: auto; max-height: 80vh;">
|
||||
<div class="modal-header">
|
||||
<span class="modal-title">Extract to Dataset</span>
|
||||
<button class="btn btn-icon" id="extract-modal-close">×</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<div style="padding: 16px;">
|
||||
<div class="dataset-form-group">
|
||||
<label class="dataset-form-label">Dataset Name *</label>
|
||||
<input type="text" id="extract-dataset-name" class="input" placeholder="Enter dataset name..." />
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-group">
|
||||
<label class="dataset-form-label">Data Preview</label>
|
||||
<pre id="extract-data-preview" class="preview-box large" style="max-height: 250px;"></pre>
|
||||
</div>
|
||||
|
||||
<div class="dataset-form-error" id="extract-form-error"></div>
|
||||
|
||||
<div class="dataset-form-actions">
|
||||
<button class="btn btn-modal primary" id="extract-create-btn">Create Dataset</button>
|
||||
<button class="btn btn-modal" id="extract-cancel-btn">Cancel</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="src/js/config.js"></script>
|
||||
<script src="src/js/snippet-manager.js"></script>
|
||||
<script src="src/js/dataset-manager.js"></script>
|
||||
|
||||
@@ -160,6 +160,19 @@ document.addEventListener('DOMContentLoaded', function () {
|
||||
newDatasetBtn.addEventListener('click', showNewDatasetForm);
|
||||
}
|
||||
|
||||
// Import dataset button and file input
|
||||
const importDatasetBtn = document.getElementById('import-dataset-btn');
|
||||
const importDatasetFile = document.getElementById('import-dataset-file');
|
||||
if (importDatasetBtn && importDatasetFile) {
|
||||
importDatasetBtn.addEventListener('click', function () {
|
||||
importDatasetFile.click();
|
||||
});
|
||||
|
||||
importDatasetFile.addEventListener('change', function () {
|
||||
importDatasetFromFile(this);
|
||||
});
|
||||
}
|
||||
|
||||
// Cancel dataset button
|
||||
if (cancelDatasetBtn) {
|
||||
cancelDatasetBtn.addEventListener('click', hideNewDatasetForm);
|
||||
@@ -186,6 +199,36 @@ document.addEventListener('DOMContentLoaded', function () {
|
||||
refreshMetadataBtn.addEventListener('click', refreshDatasetMetadata);
|
||||
}
|
||||
|
||||
// New snippet from dataset button
|
||||
const newSnippetBtn = document.getElementById('new-snippet-btn');
|
||||
if (newSnippetBtn) {
|
||||
newSnippetBtn.addEventListener('click', createNewSnippetFromDataset);
|
||||
}
|
||||
|
||||
// Export dataset button
|
||||
const exportDatasetBtn = document.getElementById('export-dataset-btn');
|
||||
if (exportDatasetBtn) {
|
||||
exportDatasetBtn.addEventListener('click', exportCurrentDataset);
|
||||
}
|
||||
|
||||
// Preview toggle buttons
|
||||
const previewRawBtn = document.getElementById('preview-raw-btn');
|
||||
const previewTableBtn = document.getElementById('preview-table-btn');
|
||||
if (previewRawBtn) {
|
||||
previewRawBtn.addEventListener('click', function() {
|
||||
if (window.currentDatasetData) {
|
||||
showRawPreview(window.currentDatasetData);
|
||||
}
|
||||
});
|
||||
}
|
||||
if (previewTableBtn) {
|
||||
previewTableBtn.addEventListener('click', function() {
|
||||
if (window.currentDatasetData) {
|
||||
showTablePreview(window.currentDatasetData);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// View mode toggle buttons
|
||||
document.getElementById('view-draft').addEventListener('click', () => {
|
||||
switchViewMode('draft');
|
||||
@@ -198,6 +241,39 @@ document.addEventListener('DOMContentLoaded', function () {
|
||||
// Publish and Revert buttons
|
||||
document.getElementById('publish-btn').addEventListener('click', publishDraft);
|
||||
document.getElementById('revert-btn').addEventListener('click', revertDraft);
|
||||
|
||||
// Extract to Dataset button
|
||||
const extractBtn = document.getElementById('extract-btn');
|
||||
if (extractBtn) {
|
||||
extractBtn.addEventListener('click', showExtractModal);
|
||||
}
|
||||
|
||||
// Extract modal buttons
|
||||
const extractModalClose = document.getElementById('extract-modal-close');
|
||||
const extractCancelBtn = document.getElementById('extract-cancel-btn');
|
||||
const extractCreateBtn = document.getElementById('extract-create-btn');
|
||||
const extractModal = document.getElementById('extract-modal');
|
||||
|
||||
if (extractModalClose) {
|
||||
extractModalClose.addEventListener('click', hideExtractModal);
|
||||
}
|
||||
|
||||
if (extractCancelBtn) {
|
||||
extractCancelBtn.addEventListener('click', hideExtractModal);
|
||||
}
|
||||
|
||||
if (extractCreateBtn) {
|
||||
extractCreateBtn.addEventListener('click', extractToDataset);
|
||||
}
|
||||
|
||||
// Close modal on overlay click
|
||||
if (extractModal) {
|
||||
extractModal.addEventListener('click', function (e) {
|
||||
if (e.target === extractModal) {
|
||||
hideExtractModal();
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// Handle URL hash changes (browser back/forward)
|
||||
|
||||
@@ -221,6 +221,14 @@ async function getCurrentDataset() {
|
||||
return window.currentDatasetId ? await DatasetStorage.getDataset(window.currentDatasetId) : null;
|
||||
}
|
||||
|
||||
// Count how many snippets use a specific dataset
|
||||
function countSnippetUsage(datasetName) {
|
||||
const snippets = SnippetStorage.loadSnippets();
|
||||
return snippets.filter(snippet =>
|
||||
snippet.datasetRefs && snippet.datasetRefs.includes(datasetName)
|
||||
).length;
|
||||
}
|
||||
|
||||
// Fetch URL data and calculate metadata
|
||||
async function fetchURLMetadata(url, format) {
|
||||
try {
|
||||
@@ -293,12 +301,19 @@ async function renderDatasetList() {
|
||||
metaText = `${dataset.rowCount} rows • ${dataset.format.toUpperCase()} • ${formatBytes(dataset.size)}`;
|
||||
}
|
||||
|
||||
// Count snippet usage and create badge
|
||||
const usageCount = countSnippetUsage(dataset.name);
|
||||
const usageBadge = usageCount > 0
|
||||
? `<div class="dataset-usage-badge" title="${usageCount} snippet${usageCount !== 1 ? 's' : ''} using this dataset">📄 ${usageCount}</div>`
|
||||
: '';
|
||||
|
||||
return `
|
||||
<div class="dataset-item" data-dataset-id="${dataset.id}">
|
||||
<div class="dataset-info">
|
||||
<div class="dataset-name">${dataset.name}</div>
|
||||
<div class="dataset-meta">${metaText}</div>
|
||||
</div>
|
||||
${usageBadge}
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
@@ -346,7 +361,165 @@ async function selectDataset(datasetId, updateURL = true) {
|
||||
document.getElementById('dataset-detail-created').textContent = new Date(dataset.created).toLocaleString();
|
||||
document.getElementById('dataset-detail-modified').textContent = new Date(dataset.modified).toLocaleString();
|
||||
|
||||
// Show preview
|
||||
// Show/hide preview toggle based on data type
|
||||
const toggleGroup = document.getElementById('preview-toggle-group');
|
||||
const canShowTable = (dataset.format === 'json' || dataset.format === 'csv' || dataset.format === 'tsv');
|
||||
|
||||
if (dataset.source === 'url') {
|
||||
// For URL datasets, check if we have cached preview data
|
||||
if (window.urlPreviewCache && window.urlPreviewCache[dataset.id]) {
|
||||
if (canShowTable) {
|
||||
toggleGroup.style.display = 'flex';
|
||||
} else {
|
||||
toggleGroup.style.display = 'none';
|
||||
}
|
||||
showRawPreview(dataset);
|
||||
} else {
|
||||
// Show load preview option
|
||||
toggleGroup.style.display = 'none';
|
||||
showURLPreviewPrompt(dataset);
|
||||
}
|
||||
} else {
|
||||
// For inline datasets
|
||||
if (canShowTable) {
|
||||
toggleGroup.style.display = 'flex';
|
||||
} else {
|
||||
toggleGroup.style.display = 'none';
|
||||
}
|
||||
showRawPreview(dataset);
|
||||
}
|
||||
|
||||
// Store current dataset ID and data
|
||||
window.currentDatasetId = datasetId;
|
||||
window.currentDatasetData = dataset;
|
||||
|
||||
// Update linked snippets display
|
||||
updateLinkedSnippets(dataset);
|
||||
|
||||
// Update URL state (URLState.update will add 'dataset-' prefix)
|
||||
if (updateURL) {
|
||||
URLState.update({ view: 'datasets', snippetId: null, datasetId: datasetId });
|
||||
}
|
||||
}
|
||||
|
||||
// Show URL preview prompt (button to load data)
|
||||
function showURLPreviewPrompt(dataset) {
|
||||
const previewBox = document.getElementById('dataset-preview');
|
||||
const tableContainer = document.getElementById('dataset-preview-table');
|
||||
|
||||
previewBox.style.display = 'block';
|
||||
tableContainer.style.display = 'none';
|
||||
|
||||
const promptHTML = `
|
||||
<div style="text-align: center; padding: 20px;">
|
||||
<div style="margin-bottom: 12px; font-size: 11px; color: #606060;">
|
||||
URL: ${dataset.data}<br/>
|
||||
Format: ${dataset.format.toUpperCase()}
|
||||
</div>
|
||||
<button class="btn btn-standard primary" id="load-preview-btn">Load Preview</button>
|
||||
<div style="margin-top: 8px; font-size: 10px; color: #808080; font-style: italic;">
|
||||
Data will be fetched but not saved
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
previewBox.innerHTML = promptHTML;
|
||||
|
||||
// Add click handler
|
||||
const loadBtn = document.getElementById('load-preview-btn');
|
||||
if (loadBtn) {
|
||||
loadBtn.addEventListener('click', async () => {
|
||||
await loadURLPreview(dataset);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Load and cache URL preview data
|
||||
async function loadURLPreview(dataset) {
|
||||
const previewBox = document.getElementById('dataset-preview');
|
||||
previewBox.innerHTML = '<div style="text-align: center; padding: 20px; font-size: 11px;">Loading data from URL...</div>';
|
||||
|
||||
try {
|
||||
const response = await fetch(dataset.data);
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const text = await response.text();
|
||||
|
||||
// Parse data based on format
|
||||
let parsedData;
|
||||
if (dataset.format === 'json' || dataset.format === 'topojson') {
|
||||
parsedData = JSON.parse(text);
|
||||
} else if (dataset.format === 'csv' || dataset.format === 'tsv') {
|
||||
parsedData = text;
|
||||
}
|
||||
|
||||
// Cache the preview data (don't save to DB)
|
||||
if (!window.urlPreviewCache) {
|
||||
window.urlPreviewCache = {};
|
||||
}
|
||||
window.urlPreviewCache[dataset.id] = {
|
||||
data: parsedData,
|
||||
fetchedAt: Date.now()
|
||||
};
|
||||
|
||||
// Create a temporary dataset object with the fetched data
|
||||
const previewDataset = {
|
||||
...dataset,
|
||||
data: parsedData,
|
||||
source: 'inline' // Treat as inline for preview purposes
|
||||
};
|
||||
|
||||
// Update current dataset data for preview
|
||||
window.currentDatasetData = previewDataset;
|
||||
|
||||
// Show toggle buttons now that we have data
|
||||
const toggleGroup = document.getElementById('preview-toggle-group');
|
||||
const canShowTable = (dataset.format === 'json' || dataset.format === 'csv' || dataset.format === 'tsv');
|
||||
if (canShowTable) {
|
||||
toggleGroup.style.display = 'flex';
|
||||
}
|
||||
|
||||
// Show the preview
|
||||
showRawPreview(previewDataset);
|
||||
|
||||
} catch (error) {
|
||||
previewBox.innerHTML = `
|
||||
<div style="text-align: center; padding: 20px; color: #f00; font-size: 11px;">
|
||||
<div style="margin-bottom: 8px;">Failed to load URL data:</div>
|
||||
<div>${error.message}</div>
|
||||
<button class="btn btn-standard" id="retry-preview-btn" style="margin-top: 12px;">Retry</button>
|
||||
</div>
|
||||
`;
|
||||
|
||||
const retryBtn = document.getElementById('retry-preview-btn');
|
||||
if (retryBtn) {
|
||||
retryBtn.addEventListener('click', async () => {
|
||||
await loadURLPreview(dataset);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Show raw preview
|
||||
function showRawPreview(dataset) {
|
||||
const rawBtn = document.getElementById('preview-raw-btn');
|
||||
const tableBtn = document.getElementById('preview-table-btn');
|
||||
const previewBox = document.getElementById('dataset-preview');
|
||||
const tableContainer = document.getElementById('dataset-preview-table');
|
||||
|
||||
// Update button states
|
||||
if (rawBtn && tableBtn) {
|
||||
rawBtn.classList.add('active');
|
||||
tableBtn.classList.remove('active');
|
||||
}
|
||||
|
||||
// Show raw, hide table
|
||||
previewBox.style.display = 'block';
|
||||
tableContainer.style.display = 'none';
|
||||
|
||||
// Generate preview text
|
||||
let previewText;
|
||||
if (dataset.source === 'url') {
|
||||
previewText = `URL: ${dataset.data}\nFormat: ${dataset.format.toUpperCase()}`;
|
||||
@@ -357,15 +530,249 @@ async function selectDataset(datasetId, updateURL = true) {
|
||||
const lines = dataset.data.split('\n');
|
||||
previewText = lines.slice(0, 6).join('\n'); // Header + 5 rows
|
||||
}
|
||||
document.getElementById('dataset-preview').textContent = previewText;
|
||||
|
||||
// Store current dataset ID
|
||||
window.currentDatasetId = datasetId;
|
||||
|
||||
// Update URL state (URLState.update will add 'dataset-' prefix)
|
||||
if (updateURL) {
|
||||
URLState.update({ view: 'datasets', snippetId: null, datasetId: datasetId });
|
||||
previewBox.textContent = previewText;
|
||||
}
|
||||
|
||||
// Detect column type from sample values
|
||||
function detectColumnType(values) {
|
||||
// Filter out null/undefined values
|
||||
const validValues = values.filter(v => v !== null && v !== undefined && v !== '');
|
||||
if (validValues.length === 0) return 'text';
|
||||
|
||||
let numberCount = 0;
|
||||
let booleanCount = 0;
|
||||
let dateCount = 0;
|
||||
|
||||
for (const val of validValues) {
|
||||
const str = String(val).trim();
|
||||
|
||||
// Check boolean
|
||||
if (str === 'true' || str === 'false' || str === '0' || str === '1' ||
|
||||
str === 'True' || str === 'False' || str === 'TRUE' || str === 'FALSE') {
|
||||
booleanCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check number
|
||||
const num = parseFloat(str);
|
||||
if (!isNaN(num) && isFinite(num) && str === String(num)) {
|
||||
numberCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check date (ISO format or common patterns)
|
||||
// ISO: 2024-01-15, 2024-01-15T10:30:00, etc.
|
||||
const isoDatePattern = /^\d{4}-\d{2}-\d{2}(T\d{2}:\d{2}:\d{2})?/;
|
||||
// Common: 01/15/2024, 15-01-2024, etc.
|
||||
const commonDatePattern = /^\d{1,2}[-/]\d{1,2}[-/]\d{2,4}$/;
|
||||
|
||||
if (isoDatePattern.test(str) || commonDatePattern.test(str)) {
|
||||
const parsed = new Date(str);
|
||||
if (!isNaN(parsed.getTime())) {
|
||||
dateCount++;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const total = validValues.length;
|
||||
const threshold = 0.8; // 80% of values must match type
|
||||
|
||||
if (booleanCount / total >= threshold) return 'boolean';
|
||||
if (numberCount / total >= threshold) return 'number';
|
||||
if (dateCount / total >= threshold) return 'date';
|
||||
return 'text';
|
||||
}
|
||||
|
||||
// Get type icon
|
||||
function getTypeIcon(type) {
|
||||
switch (type) {
|
||||
case 'number': return '🔢';
|
||||
case 'date': return '📅';
|
||||
case 'boolean': return '✓';
|
||||
case 'text':
|
||||
default: return '🔤';
|
||||
}
|
||||
}
|
||||
|
||||
// Show table preview
|
||||
function showTablePreview(dataset) {
|
||||
const rawBtn = document.getElementById('preview-raw-btn');
|
||||
const tableBtn = document.getElementById('preview-table-btn');
|
||||
const previewBox = document.getElementById('dataset-preview');
|
||||
const tableContainer = document.getElementById('dataset-preview-table');
|
||||
|
||||
// Update button states
|
||||
rawBtn.classList.remove('active');
|
||||
tableBtn.classList.add('active');
|
||||
|
||||
// Hide raw, show table
|
||||
previewBox.style.display = 'none';
|
||||
tableContainer.style.display = 'block';
|
||||
|
||||
// Generate table HTML
|
||||
let tableHTML = '';
|
||||
const maxRows = 20; // Show first 20 rows
|
||||
|
||||
if (dataset.format === 'json') {
|
||||
if (!Array.isArray(dataset.data) || dataset.data.length === 0) {
|
||||
tableHTML = '<div class="preview-table-info">Cannot display non-array JSON data in table format</div>';
|
||||
} else {
|
||||
const rows = dataset.data.slice(0, maxRows);
|
||||
const columns = Object.keys(rows[0] || {});
|
||||
|
||||
// Detect column types
|
||||
const columnTypes = {};
|
||||
columns.forEach(col => {
|
||||
const values = dataset.data.map(row => row[col]);
|
||||
columnTypes[col] = detectColumnType(values);
|
||||
});
|
||||
|
||||
tableHTML = '<table class="preview-table">';
|
||||
tableHTML += '<thead><tr>';
|
||||
columns.forEach(col => {
|
||||
const typeIcon = getTypeIcon(columnTypes[col]);
|
||||
tableHTML += `<th><span class="type-icon">${typeIcon}</span> ${col}</th>`;
|
||||
});
|
||||
tableHTML += '</tr></thead>';
|
||||
tableHTML += '<tbody>';
|
||||
rows.forEach(row => {
|
||||
tableHTML += '<tr>';
|
||||
columns.forEach(col => {
|
||||
const value = row[col];
|
||||
const type = columnTypes[col];
|
||||
let displayValue = '';
|
||||
let cssClass = '';
|
||||
|
||||
if (value === null || value === undefined) {
|
||||
displayValue = '';
|
||||
cssClass = 'cell-null';
|
||||
} else if (typeof value === 'object') {
|
||||
displayValue = JSON.stringify(value);
|
||||
cssClass = 'cell-text';
|
||||
} else {
|
||||
displayValue = String(value);
|
||||
cssClass = `cell-${type}`;
|
||||
}
|
||||
|
||||
tableHTML += `<td class="${cssClass}">${displayValue}</td>`;
|
||||
});
|
||||
tableHTML += '</tr>';
|
||||
});
|
||||
tableHTML += '</tbody></table>';
|
||||
|
||||
if (dataset.data.length > maxRows) {
|
||||
tableHTML += `<div class="preview-table-info">Showing first ${maxRows} of ${dataset.data.length} rows</div>`;
|
||||
}
|
||||
}
|
||||
} else if (dataset.format === 'csv' || dataset.format === 'tsv') {
|
||||
const separator = dataset.format === 'csv' ? ',' : '\t';
|
||||
const lines = dataset.data.split('\n').filter(line => line.trim());
|
||||
|
||||
if (lines.length < 2) {
|
||||
tableHTML = '<div class="preview-table-info">No data to display</div>';
|
||||
} else {
|
||||
const headerLine = lines[0];
|
||||
const headers = headerLine.split(separator).map(h => h.trim().replace(/^"|"$/g, ''));
|
||||
const dataLines = lines.slice(1, maxRows + 1);
|
||||
const allDataLines = lines.slice(1); // All lines for type detection
|
||||
|
||||
// Parse all data for type detection
|
||||
const columnData = headers.map(() => []);
|
||||
allDataLines.forEach(line => {
|
||||
const cells = line.split(separator).map(c => c.trim().replace(/^"|"$/g, ''));
|
||||
cells.forEach((cell, idx) => {
|
||||
if (columnData[idx]) {
|
||||
columnData[idx].push(cell);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Detect column types
|
||||
const columnTypes = columnData.map(colValues => detectColumnType(colValues));
|
||||
|
||||
tableHTML = '<table class="preview-table">';
|
||||
tableHTML += '<thead><tr>';
|
||||
headers.forEach((header, idx) => {
|
||||
const typeIcon = getTypeIcon(columnTypes[idx]);
|
||||
tableHTML += `<th><span class="type-icon">${typeIcon}</span> ${header}</th>`;
|
||||
});
|
||||
tableHTML += '</tr></thead>';
|
||||
tableHTML += '<tbody>';
|
||||
dataLines.forEach(line => {
|
||||
const cells = line.split(separator).map(c => c.trim().replace(/^"|"$/g, ''));
|
||||
tableHTML += '<tr>';
|
||||
cells.forEach((cell, idx) => {
|
||||
const type = columnTypes[idx] || 'text';
|
||||
const cssClass = cell === '' ? 'cell-null' : `cell-${type}`;
|
||||
tableHTML += `<td class="${cssClass}">${cell}</td>`;
|
||||
});
|
||||
tableHTML += '</tr>';
|
||||
});
|
||||
tableHTML += '</tbody></table>';
|
||||
|
||||
if (lines.length > maxRows + 1) {
|
||||
tableHTML += `<div class="preview-table-info">Showing first ${maxRows} of ${lines.length - 1} rows</div>`;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tableContainer.innerHTML = tableHTML;
|
||||
}
|
||||
|
||||
// Update linked snippets display in dataset details panel
|
||||
function updateLinkedSnippets(dataset) {
|
||||
const snippetsSection = document.getElementById('dataset-snippets-section');
|
||||
const snippetsContainer = document.getElementById('dataset-snippets');
|
||||
|
||||
if (!snippetsSection || !snippetsContainer) return;
|
||||
|
||||
// Find all snippets that reference this dataset
|
||||
const snippets = SnippetStorage.loadSnippets();
|
||||
const linkedSnippets = snippets.filter(snippet =>
|
||||
snippet.datasetRefs && snippet.datasetRefs.includes(dataset.name)
|
||||
);
|
||||
|
||||
if (linkedSnippets.length === 0) {
|
||||
snippetsSection.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
|
||||
// Show section and populate with snippet links
|
||||
snippetsSection.style.display = 'block';
|
||||
|
||||
const snippetItems = linkedSnippets.map(snippet => {
|
||||
return `
|
||||
<div class="stat-item">
|
||||
<span class="stat-label">📄</span>
|
||||
<span>
|
||||
<a href="#" class="snippet-link" data-snippet-id="${snippet.id}">${snippet.name}</a>
|
||||
</span>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
|
||||
snippetsContainer.innerHTML = snippetItems;
|
||||
|
||||
// Attach click handlers to snippet links
|
||||
snippetsContainer.querySelectorAll('.snippet-link').forEach(link => {
|
||||
link.addEventListener('click', function(e) {
|
||||
e.preventDefault();
|
||||
const snippetId = parseFloat(this.dataset.snippetId);
|
||||
openSnippetFromDataset(snippetId);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Close dataset manager and open snippet
|
||||
function openSnippetFromDataset(snippetId) {
|
||||
// Close dataset manager
|
||||
closeDatasetManager();
|
||||
|
||||
// Small delay to ensure UI is ready
|
||||
setTimeout(() => {
|
||||
selectSnippet(snippetId);
|
||||
}, 100);
|
||||
}
|
||||
|
||||
// Open dataset manager modal
|
||||
@@ -742,7 +1149,17 @@ async function deleteCurrentDataset() {
|
||||
const dataset = await getCurrentDataset();
|
||||
if (!dataset) return;
|
||||
|
||||
if (confirm(`Delete dataset "${dataset.name}"? This action cannot be undone.`)) {
|
||||
// Check if dataset is in use
|
||||
const usageCount = countSnippetUsage(dataset.name);
|
||||
let confirmMessage = `Delete dataset "${dataset.name}"?`;
|
||||
|
||||
if (usageCount > 0) {
|
||||
confirmMessage = `⚠️ Warning: Dataset "${dataset.name}" is currently used by ${usageCount} snippet${usageCount !== 1 ? 's' : ''}.\n\nDeleting this dataset will break those visualizations. Are you sure you want to delete it?`;
|
||||
} else {
|
||||
confirmMessage += ' This action cannot be undone.';
|
||||
}
|
||||
|
||||
if (confirm(confirmMessage)) {
|
||||
await DatasetStorage.deleteDataset(dataset.id);
|
||||
document.getElementById('dataset-details').style.display = 'none';
|
||||
window.currentDatasetId = null;
|
||||
@@ -794,3 +1211,187 @@ async function refreshDatasetMetadata() {
|
||||
refreshBtn.disabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Create new snippet from current dataset
|
||||
async function createNewSnippetFromDataset() {
|
||||
const dataset = await getCurrentDataset();
|
||||
if (!dataset) return;
|
||||
|
||||
// Close dataset manager
|
||||
closeDatasetManager();
|
||||
|
||||
// Small delay to ensure UI is ready
|
||||
setTimeout(() => {
|
||||
// Call the function from snippet-manager.js
|
||||
createSnippetFromDataset(dataset.name);
|
||||
}, 100);
|
||||
}
|
||||
|
||||
// Export dataset to file
|
||||
async function exportCurrentDataset() {
|
||||
const dataset = await getCurrentDataset();
|
||||
if (!dataset) return;
|
||||
|
||||
let dataToExport;
|
||||
let filename;
|
||||
let mimeType;
|
||||
|
||||
try {
|
||||
if (dataset.source === 'url') {
|
||||
// For URL datasets, fetch the content and export it
|
||||
const response = await fetch(dataset.data);
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const content = await response.text();
|
||||
dataToExport = content;
|
||||
} else {
|
||||
// For inline datasets, export the stored data
|
||||
if (dataset.format === 'json' || dataset.format === 'topojson') {
|
||||
dataToExport = JSON.stringify(dataset.data, null, 2);
|
||||
} else if (dataset.format === 'csv' || dataset.format === 'tsv') {
|
||||
dataToExport = dataset.data;
|
||||
}
|
||||
}
|
||||
|
||||
// Determine file extension and MIME type
|
||||
switch (dataset.format) {
|
||||
case 'json':
|
||||
filename = `${dataset.name}.json`;
|
||||
mimeType = 'application/json';
|
||||
break;
|
||||
case 'csv':
|
||||
filename = `${dataset.name}.csv`;
|
||||
mimeType = 'text/csv';
|
||||
break;
|
||||
case 'tsv':
|
||||
filename = `${dataset.name}.tsv`;
|
||||
mimeType = 'text/tab-separated-values';
|
||||
break;
|
||||
case 'topojson':
|
||||
filename = `${dataset.name}.topojson`;
|
||||
mimeType = 'application/json';
|
||||
break;
|
||||
default:
|
||||
filename = `${dataset.name}.txt`;
|
||||
mimeType = 'text/plain';
|
||||
}
|
||||
|
||||
// Create blob and download
|
||||
const blob = new Blob([dataToExport], { type: mimeType });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
link.download = filename;
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
document.body.removeChild(link);
|
||||
URL.revokeObjectURL(url);
|
||||
|
||||
} catch (error) {
|
||||
alert(`Failed to export dataset: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Detect format from file extension
|
||||
function detectFormatFromFilename(filename) {
|
||||
const lower = filename.toLowerCase();
|
||||
if (lower.endsWith('.json')) return 'json';
|
||||
if (lower.endsWith('.csv')) return 'csv';
|
||||
if (lower.endsWith('.tsv') || lower.endsWith('.tab') || lower.endsWith('.txt')) return 'tsv';
|
||||
if (lower.endsWith('.topojson')) return 'topojson';
|
||||
return null;
|
||||
}
|
||||
|
||||
// Import dataset from file
|
||||
async function importDatasetFromFile(fileInput) {
|
||||
const file = fileInput.files[0];
|
||||
if (!file) return;
|
||||
|
||||
try {
|
||||
// Read file content
|
||||
const text = await file.text();
|
||||
|
||||
// Try to detect format from filename first
|
||||
let formatHint = detectFormatFromFilename(file.name);
|
||||
|
||||
// Auto-detect format from content
|
||||
const detection = detectDataFormat(text);
|
||||
|
||||
// Use filename hint if content detection is uncertain
|
||||
let format = detection.format;
|
||||
if (!format && formatHint) {
|
||||
format = formatHint;
|
||||
}
|
||||
|
||||
if (!format) {
|
||||
alert('Could not detect data format from file. Please ensure the file contains valid JSON, CSV, or TSV data.');
|
||||
return;
|
||||
}
|
||||
|
||||
// Generate default name from filename (remove extension)
|
||||
let baseName = file.name.replace(/\.(json|csv|tsv|txt|topojson)$/i, '');
|
||||
|
||||
// Check if name already exists and make it unique
|
||||
let datasetName = baseName;
|
||||
let wasRenamed = false;
|
||||
let counter = 1;
|
||||
while (await DatasetStorage.nameExists(datasetName)) {
|
||||
wasRenamed = true;
|
||||
// Add timestamp-based suffix for uniqueness
|
||||
const timestamp = Date.now().toString().slice(-6); // Last 6 digits of timestamp
|
||||
datasetName = `${baseName}_${timestamp}`;
|
||||
|
||||
// If still exists (unlikely), add a counter
|
||||
if (await DatasetStorage.nameExists(datasetName)) {
|
||||
datasetName = `${baseName}_${timestamp}_${counter}`;
|
||||
counter++;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Prepare data based on format
|
||||
let data;
|
||||
if (format === 'json' || format === 'topojson') {
|
||||
if (!detection.parsed) {
|
||||
alert('Invalid JSON data in file.');
|
||||
return;
|
||||
}
|
||||
data = detection.parsed;
|
||||
} else if (format === 'csv' || format === 'tsv') {
|
||||
const lines = text.trim().split('\n');
|
||||
if (lines.length < 2) {
|
||||
alert(`${format.toUpperCase()} file must have at least a header row and one data row.`);
|
||||
return;
|
||||
}
|
||||
data = text.trim();
|
||||
}
|
||||
|
||||
// Create dataset
|
||||
await DatasetStorage.createDataset(
|
||||
datasetName,
|
||||
data,
|
||||
format,
|
||||
'inline',
|
||||
`Imported from file: ${file.name}`
|
||||
);
|
||||
|
||||
// Refresh the list
|
||||
await renderDatasetList();
|
||||
|
||||
// Show success message with rename notification if applicable
|
||||
if (wasRenamed) {
|
||||
alert(`Dataset name "${baseName}" was already taken, so your dataset was automatically renamed to "${datasetName}".`);
|
||||
} else {
|
||||
alert(`Dataset "${datasetName}" imported successfully!`);
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
alert(`Failed to import dataset: ${error.message}`);
|
||||
} finally {
|
||||
// Reset file input
|
||||
fileInput.value = '';
|
||||
}
|
||||
}
|
||||
|
||||
@@ -16,6 +16,117 @@ function generateSnippetName() {
|
||||
return `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}_${pad(now.getHours())}-${pad(now.getMinutes())}-${pad(now.getSeconds())}`;
|
||||
}
|
||||
|
||||
// Extract dataset references from Vega-Lite spec
|
||||
function extractDatasetRefs(spec) {
|
||||
const datasetNames = new Set();
|
||||
|
||||
function traverse(obj) {
|
||||
if (!obj || typeof obj !== 'object') return;
|
||||
|
||||
// Check if this is a data object with a name property
|
||||
if (obj.data && typeof obj.data === 'object' && obj.data.name) {
|
||||
datasetNames.add(obj.data.name);
|
||||
}
|
||||
|
||||
// Recursively check all properties
|
||||
for (const key in obj) {
|
||||
if (obj.hasOwnProperty(key)) {
|
||||
traverse(obj[key]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
traverse(spec);
|
||||
return Array.from(datasetNames);
|
||||
}
|
||||
|
||||
// Detect if spec has inline data (data.values)
|
||||
function hasInlineData(spec) {
|
||||
if (!spec || typeof spec !== 'object') return false;
|
||||
|
||||
// Check top-level data.values
|
||||
if (spec.data && Array.isArray(spec.data.values)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// Check common nested locations (layer, concat, hconcat, vconcat, facet)
|
||||
const nestedKeys = ['layer', 'concat', 'hconcat', 'vconcat', 'spec'];
|
||||
for (const key of nestedKeys) {
|
||||
if (Array.isArray(spec[key])) {
|
||||
for (const item of spec[key]) {
|
||||
if (hasInlineData(item)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
} else if (spec[key] && typeof spec[key] === 'object') {
|
||||
if (hasInlineData(spec[key])) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
// Extract inline data from spec (finds first occurrence)
|
||||
function extractInlineDataFromSpec(spec) {
|
||||
if (!spec || typeof spec !== 'object') return null;
|
||||
|
||||
// Check top-level data.values
|
||||
if (spec.data && Array.isArray(spec.data.values)) {
|
||||
return spec.data.values;
|
||||
}
|
||||
|
||||
// Check nested locations
|
||||
const nestedKeys = ['layer', 'concat', 'hconcat', 'vconcat', 'spec'];
|
||||
for (const key of nestedKeys) {
|
||||
if (Array.isArray(spec[key])) {
|
||||
for (const item of spec[key]) {
|
||||
const data = extractInlineDataFromSpec(item);
|
||||
if (data) return data;
|
||||
}
|
||||
} else if (spec[key] && typeof spec[key] === 'object') {
|
||||
const data = extractInlineDataFromSpec(spec[key]);
|
||||
if (data) return data;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// Replace inline data with dataset reference
|
||||
function replaceInlineDataWithReference(spec, datasetName) {
|
||||
if (!spec || typeof spec !== 'object') return spec;
|
||||
|
||||
// Clone the spec to avoid mutation
|
||||
const newSpec = JSON.parse(JSON.stringify(spec));
|
||||
|
||||
function replaceData(obj) {
|
||||
if (!obj || typeof obj !== 'object') return;
|
||||
|
||||
// Replace top-level data.values with data.name
|
||||
if (obj.data && Array.isArray(obj.data.values)) {
|
||||
obj.data = { name: datasetName };
|
||||
return; // Stop after first replacement
|
||||
}
|
||||
|
||||
// Check nested locations
|
||||
const nestedKeys = ['layer', 'concat', 'hconcat', 'vconcat', 'spec'];
|
||||
for (const key of nestedKeys) {
|
||||
if (Array.isArray(obj[key])) {
|
||||
for (const item of obj[key]) {
|
||||
replaceData(item);
|
||||
}
|
||||
} else if (obj[key] && typeof obj[key] === 'object') {
|
||||
replaceData(obj[key]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
replaceData(newSpec);
|
||||
return newSpec;
|
||||
}
|
||||
|
||||
// Create a new snippet using Phase 0 schema
|
||||
function createSnippet(spec, name = null) {
|
||||
const now = new Date().toISOString();
|
||||
@@ -247,10 +358,14 @@ function renderSnippetList(searchQuery = null) {
|
||||
const hasDraft = JSON.stringify(snippet.spec) !== JSON.stringify(snippet.draftSpec);
|
||||
const statusClass = hasDraft ? 'draft' : 'published';
|
||||
|
||||
// Check if snippet uses external datasets
|
||||
const usesDatasets = snippet.datasetRefs && snippet.datasetRefs.length > 0;
|
||||
const datasetIconHTML = usesDatasets ? '<span class="snippet-dataset-icon" title="Uses external dataset">📁</span>' : '';
|
||||
|
||||
return `
|
||||
<li class="snippet-item" data-snippet-id="${snippet.id}">
|
||||
<div class="snippet-info">
|
||||
<div class="snippet-name">${snippet.name}</div>
|
||||
<div class="snippet-name">${snippet.name}${datasetIconHTML}</div>
|
||||
<div class="snippet-date">${dateText}</div>
|
||||
</div>
|
||||
${sizeHTML}
|
||||
@@ -501,12 +616,82 @@ function selectSnippet(snippetId, updateURL = true) {
|
||||
// Store currently selected snippet ID globally
|
||||
window.currentSnippetId = snippetId;
|
||||
|
||||
// Update linked datasets display
|
||||
updateLinkedDatasets(snippet);
|
||||
|
||||
// Update Extract to Dataset button visibility
|
||||
updateExtractButton();
|
||||
|
||||
// Update URL state (URLState.update will add 'snippet-' prefix)
|
||||
if (updateURL) {
|
||||
URLState.update({ view: 'snippets', snippetId: snippetId, datasetId: null });
|
||||
}
|
||||
}
|
||||
|
||||
// Update linked datasets display in metadata panel
|
||||
function updateLinkedDatasets(snippet) {
|
||||
const datasetsSection = document.getElementById('snippet-datasets-section');
|
||||
const datasetsContainer = document.getElementById('snippet-datasets');
|
||||
|
||||
if (!datasetsSection || !datasetsContainer) return;
|
||||
|
||||
// Get dataset references from snippet
|
||||
const datasetRefs = snippet.datasetRefs || [];
|
||||
|
||||
if (datasetRefs.length === 0) {
|
||||
datasetsSection.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
|
||||
// Show section and populate with dataset references
|
||||
datasetsSection.style.display = 'block';
|
||||
|
||||
const datasetItems = datasetRefs.map(datasetName => {
|
||||
return `
|
||||
<div class="meta-info-item">
|
||||
<span class="meta-info-label">📁</span>
|
||||
<span class="meta-info-value">
|
||||
<a href="#" class="dataset-link" data-dataset-name="${datasetName}">${datasetName}</a>
|
||||
</span>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
|
||||
datasetsContainer.innerHTML = datasetItems;
|
||||
|
||||
// Attach click handlers to dataset links
|
||||
datasetsContainer.querySelectorAll('.dataset-link').forEach(link => {
|
||||
link.addEventListener('click', async function(e) {
|
||||
e.preventDefault();
|
||||
const datasetName = this.dataset.datasetName;
|
||||
await openDatasetByName(datasetName);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Open dataset manager and select dataset by name
|
||||
async function openDatasetByName(datasetName) {
|
||||
// Open dataset manager modal
|
||||
openDatasetManager();
|
||||
|
||||
// Wait for datasets to load and find the one with matching name
|
||||
// We need to use DatasetStorage which is defined in dataset-manager.js
|
||||
try {
|
||||
const dataset = await DatasetStorage.getDatasetByName(datasetName);
|
||||
if (dataset) {
|
||||
// Small delay to ensure UI is ready
|
||||
setTimeout(() => {
|
||||
selectDataset(dataset.id);
|
||||
}, 100);
|
||||
} else {
|
||||
alert(`Dataset "${datasetName}" not found. It may have been deleted.`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error opening dataset:', error);
|
||||
alert(`Could not open dataset "${datasetName}".`);
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-save functionality
|
||||
let autoSaveTimeout;
|
||||
window.isUpdatingEditor = false; // Global flag to prevent auto-save/debounce during programmatic updates
|
||||
@@ -524,9 +709,13 @@ function autoSaveDraft() {
|
||||
|
||||
if (snippet) {
|
||||
snippet.draftSpec = currentSpec;
|
||||
|
||||
// Extract and update dataset references
|
||||
snippet.datasetRefs = extractDatasetRefs(currentSpec);
|
||||
|
||||
SnippetStorage.saveSnippet(snippet);
|
||||
|
||||
// Refresh snippet list to update status light
|
||||
// Refresh snippet list to update status light and dataset indicator
|
||||
renderSnippetList();
|
||||
// Restore selection
|
||||
restoreSnippetSelection();
|
||||
@@ -670,6 +859,157 @@ function duplicateSnippet(snippetId) {
|
||||
return newSnippet;
|
||||
}
|
||||
|
||||
// Create new snippet from dataset with minimal spec
|
||||
function createSnippetFromDataset(datasetName) {
|
||||
const minimalSpec = {
|
||||
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
|
||||
"data": {"name": datasetName},
|
||||
"mark": "point",
|
||||
"encoding": {}
|
||||
};
|
||||
|
||||
const newSnippet = createSnippet(minimalSpec);
|
||||
newSnippet.comment = `Visualization using dataset: ${datasetName}`;
|
||||
newSnippet.datasetRefs = [datasetName];
|
||||
|
||||
SnippetStorage.saveSnippet(newSnippet);
|
||||
|
||||
// Refresh the list and select the new snippet
|
||||
renderSnippetList();
|
||||
selectSnippet(newSnippet.id);
|
||||
|
||||
return newSnippet;
|
||||
}
|
||||
|
||||
// Show extract to dataset modal
|
||||
function showExtractModal() {
|
||||
const snippet = getCurrentSnippet();
|
||||
if (!snippet) return;
|
||||
|
||||
// Get the draft spec (most recent version)
|
||||
const spec = snippet.draftSpec;
|
||||
|
||||
// Check if spec has inline data
|
||||
if (!hasInlineData(spec)) {
|
||||
alert('No inline data found in this snippet.');
|
||||
return;
|
||||
}
|
||||
|
||||
// Extract the inline data
|
||||
const inlineData = extractInlineDataFromSpec(spec);
|
||||
if (!inlineData || inlineData.length === 0) {
|
||||
alert('No inline data could be extracted.');
|
||||
return;
|
||||
}
|
||||
|
||||
// Generate default dataset name from snippet name
|
||||
const defaultName = `${snippet.name}_data`.replace(/[^a-zA-Z0-9_-]/g, '_');
|
||||
|
||||
// Show modal
|
||||
const modal = document.getElementById('extract-modal');
|
||||
const nameInput = document.getElementById('extract-dataset-name');
|
||||
const previewEl = document.getElementById('extract-data-preview');
|
||||
const errorEl = document.getElementById('extract-form-error');
|
||||
|
||||
nameInput.value = defaultName;
|
||||
previewEl.textContent = JSON.stringify(inlineData.slice(0, 10), null, 2);
|
||||
if (inlineData.length > 10) {
|
||||
previewEl.textContent += `\n\n... (${inlineData.length - 10} more rows)`;
|
||||
}
|
||||
errorEl.textContent = '';
|
||||
|
||||
modal.style.display = 'flex';
|
||||
}
|
||||
|
||||
// Hide extract to dataset modal
|
||||
function hideExtractModal() {
|
||||
const modal = document.getElementById('extract-modal');
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
|
||||
// Extract to dataset - create dataset and update snippet
|
||||
async function extractToDataset() {
|
||||
const snippet = getCurrentSnippet();
|
||||
if (!snippet) return;
|
||||
|
||||
const nameInput = document.getElementById('extract-dataset-name');
|
||||
const errorEl = document.getElementById('extract-form-error');
|
||||
const datasetName = nameInput.value.trim();
|
||||
|
||||
errorEl.textContent = '';
|
||||
|
||||
// Validation
|
||||
if (!datasetName) {
|
||||
errorEl.textContent = 'Dataset name is required';
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if dataset name already exists
|
||||
if (await DatasetStorage.nameExists(datasetName)) {
|
||||
errorEl.textContent = 'A dataset with this name already exists';
|
||||
return;
|
||||
}
|
||||
|
||||
// Extract inline data from draft spec
|
||||
const inlineData = extractInlineDataFromSpec(snippet.draftSpec);
|
||||
if (!inlineData) {
|
||||
errorEl.textContent = 'Could not extract inline data';
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// Create dataset in IndexedDB
|
||||
await DatasetStorage.createDataset(datasetName, inlineData, 'json', 'inline', `Extracted from snippet: ${snippet.name}`);
|
||||
|
||||
// Replace inline data with dataset reference in draft spec
|
||||
snippet.draftSpec = replaceInlineDataWithReference(snippet.draftSpec, datasetName);
|
||||
|
||||
// Update dataset references
|
||||
snippet.datasetRefs = extractDatasetRefs(snippet.draftSpec);
|
||||
|
||||
// Save snippet
|
||||
SnippetStorage.saveSnippet(snippet);
|
||||
|
||||
// Update editor with new spec
|
||||
if (editor && currentViewMode === 'draft') {
|
||||
window.isUpdatingEditor = true;
|
||||
editor.setValue(JSON.stringify(snippet.draftSpec, null, 2));
|
||||
window.isUpdatingEditor = false;
|
||||
}
|
||||
|
||||
// Refresh UI
|
||||
renderSnippetList();
|
||||
restoreSnippetSelection();
|
||||
updateLinkedDatasets(snippet);
|
||||
updateViewModeUI(snippet);
|
||||
updateExtractButton();
|
||||
|
||||
// Close modal
|
||||
hideExtractModal();
|
||||
|
||||
// Show success message
|
||||
alert(`Dataset "${datasetName}" created successfully!`);
|
||||
} catch (error) {
|
||||
errorEl.textContent = `Failed to create dataset: ${error.message}`;
|
||||
}
|
||||
}
|
||||
|
||||
// Update visibility of Extract to Dataset button
|
||||
function updateExtractButton() {
|
||||
const extractBtn = document.getElementById('extract-btn');
|
||||
if (!extractBtn) return;
|
||||
|
||||
const snippet = getCurrentSnippet();
|
||||
if (!snippet) {
|
||||
extractBtn.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if draft spec has inline data
|
||||
const hasInline = hasInlineData(snippet.draftSpec);
|
||||
extractBtn.style.display = hasInline ? 'block' : 'none';
|
||||
}
|
||||
|
||||
// Delete snippet with confirmation
|
||||
function deleteSnippet(snippetId) {
|
||||
const snippet = SnippetStorage.getSnippet(snippetId);
|
||||
@@ -760,6 +1100,10 @@ function publishDraft() {
|
||||
|
||||
// Copy draftSpec to spec
|
||||
snippet.spec = JSON.parse(JSON.stringify(snippet.draftSpec));
|
||||
|
||||
// Update dataset references for published spec
|
||||
snippet.datasetRefs = extractDatasetRefs(snippet.spec);
|
||||
|
||||
SnippetStorage.saveSnippet(snippet);
|
||||
|
||||
// Refresh UI
|
||||
|
||||
@@ -156,6 +156,9 @@ body { font-family: var(--font-main); height: 100vh; overflow: hidden; backgroun
|
||||
.snippet-name { font-size: 12px; }
|
||||
.snippet-date { font-size: 11px; color: inherit; margin-top: 1px; }
|
||||
.snippet-size { font-size: 10px; color: var(--win-gray-dark); margin-left: auto; margin-right: 8px; flex-shrink: 0; }
|
||||
.snippet-dataset-icon { margin-left: 4px; font-size: 10px; opacity: 0.7; }
|
||||
.snippet-item.selected .snippet-dataset-icon,
|
||||
.snippet-item:hover .snippet-dataset-icon { opacity: 1; }
|
||||
|
||||
/* Placeholders */
|
||||
.editor-placeholder,
|
||||
@@ -184,6 +187,10 @@ body { font-family: var(--font-main); height: 100vh; overflow: hidden; backgroun
|
||||
.meta-info-item:last-child { margin-bottom: 0; }
|
||||
.meta-info-label { font-weight: bold; }
|
||||
.meta-info-value { color: var(--win-gray-darker); }
|
||||
.dataset-link { color: var(--win-blue); text-decoration: underline; cursor: pointer; }
|
||||
.dataset-link:hover { color: var(--win-blue-dark); background: #e0e8f0; }
|
||||
.snippet-link { color: var(--win-blue); text-decoration: underline; cursor: pointer; font-size: 10px; }
|
||||
.snippet-link:hover { color: var(--win-blue-dark); background: #e0e8f0; }
|
||||
|
||||
/* Meta Actions */
|
||||
.meta-actions { display: flex; gap: 6px; margin-top: 8px; }
|
||||
@@ -206,7 +213,7 @@ body { font-family: var(--font-main); height: 100vh; overflow: hidden; backgroun
|
||||
|
||||
/* Modal */
|
||||
.modal { position: fixed; top: 0; left: 0; width: 100vw; height: 100vh; background: rgba(0,0,0,0.5); display: flex; align-items: center; justify-content: center; z-index: 1000; }
|
||||
.modal-content { background: var(--win-gray); border: 2px outset var(--win-gray); width: 90%; max-width: 900px; height: 80vh; max-height: 90vh; display: flex; flex-direction: column; box-shadow: 4px 4px 8px rgba(0,0,0,0.3); }
|
||||
.modal-content { background: var(--win-gray); border: 2px outset var(--win-gray); width: 95%; max-width: 1200px; height: 85vh; max-height: 90vh; display: flex; flex-direction: column; box-shadow: 4px 4px 8px rgba(0,0,0,0.3); }
|
||||
.modal-header { background: #008; color: var(--bg-white); padding: 4px 8px; display: flex; justify-content: space-between; align-items: center; height: 24px; border-bottom: 2px solid var(--win-gray-dark); }
|
||||
.modal-title { font-size: 12px; font-weight: bold; }
|
||||
.modal-body { flex: 1; overflow: auto; background: var(--bg-white); border: 2px inset var(--win-gray); margin: 8px; min-height: 0; }
|
||||
@@ -216,13 +223,17 @@ body { font-family: var(--font-main); height: 100vh; overflow: hidden; backgroun
|
||||
.dataset-list-header { padding: 8px; background: var(--win-gray-light); border-bottom: 2px solid var(--win-gray-dark); }
|
||||
.dataset-container { display: flex; flex: 1; overflow: hidden; }
|
||||
.dataset-list { width: 300px; overflow-y: auto; border-right: 2px solid var(--win-gray-dark); background: var(--bg-white); }
|
||||
.dataset-item { padding: 8px; border-bottom: 1px solid #d0d0d0; cursor: pointer; background: var(--bg-white); }
|
||||
.dataset-item { padding: 8px; border-bottom: 1px solid #d0d0d0; cursor: pointer; background: var(--bg-white); display: flex; justify-content: space-between; align-items: center; gap: 8px; }
|
||||
.dataset-item:hover { background: var(--win-blue-lighter); color: var(--bg-white); }
|
||||
.dataset-item.selected { background: var(--win-blue); color: var(--bg-white); }
|
||||
.dataset-info { flex: 1; min-width: 0; }
|
||||
.dataset-name { font-size: 12px; font-weight: bold; margin-bottom: 2px; }
|
||||
.dataset-meta { font-size: 10px; color: var(--win-gray-darker); }
|
||||
.dataset-item.selected .dataset-meta,
|
||||
.dataset-item:hover .dataset-meta { color: inherit; opacity: 0.9; }
|
||||
.dataset-usage-badge { background: var(--win-blue); color: var(--bg-white); padding: 2px 6px; font-size: 10px; font-weight: bold; border-radius: 3px; white-space: nowrap; flex-shrink: 0; }
|
||||
.dataset-item.selected .dataset-usage-badge { background: var(--win-blue-dark); }
|
||||
.dataset-item:hover .dataset-usage-badge { background: var(--win-blue-dark); }
|
||||
.dataset-empty { padding: 32px; text-align: center; color: var(--win-gray-dark); font-style: italic; font-size: 12px; }
|
||||
|
||||
/* Dataset Details */
|
||||
@@ -242,6 +253,26 @@ body { font-family: var(--font-main); height: 100vh; overflow: hidden; backgroun
|
||||
.preview-box.medium { max-height: 150px; }
|
||||
.preview-box.large { max-height: 200px; }
|
||||
|
||||
/* Preview Toggle */
|
||||
.preview-toggle-group { display: flex; }
|
||||
.btn-toggle.small { padding: 2px 6px; font-size: 9px; height: 18px; }
|
||||
|
||||
/* Preview Table */
|
||||
.preview-table-container { background: var(--bg-light); border: 2px inset var(--win-gray); overflow: auto; max-height: 200px; }
|
||||
.preview-table { width: 100%; border-collapse: collapse; font-size: 10px; font-family: 'Consolas', 'Monaco', 'Courier New', monospace; }
|
||||
.preview-table th { background: var(--win-gray-light); border: 1px solid var(--win-gray-dark); padding: 4px 6px; text-align: left; font-weight: bold; position: sticky; top: 0; }
|
||||
.preview-table td { border: 1px solid #d0d0d0; padding: 3px 6px; background: var(--bg-white); }
|
||||
.preview-table tr:hover td { background: var(--bg-lighter); }
|
||||
.preview-table-info { padding: 8px; font-size: 10px; color: var(--win-gray-darker); font-style: italic; text-align: center; }
|
||||
|
||||
/* Type-specific cell formatting */
|
||||
.type-icon { font-size: 11px; margin-right: 3px; }
|
||||
.cell-number { font-style: italic; text-align: right; color: #0066cc; }
|
||||
.cell-date { font-style: italic; color: #228b22; }
|
||||
.cell-boolean { font-weight: bold; text-align: center; color: #ff6600; }
|
||||
.cell-text { color: #000; }
|
||||
.cell-null { color: var(--win-gray-dark); font-style: italic; text-align: center; }
|
||||
|
||||
.dataset-actions { display: flex; gap: 8px; margin-top: 16px; }
|
||||
|
||||
/* Dataset Form */
|
||||
|
||||
Reference in New Issue
Block a user