mirror of
https://github.com/olehomelchenko/astrolabe-nvc.git
synced 2025-12-21 21:22:23 +00:00
chore: reorganize file structure
This commit is contained in:
532
project-docs/dev-plan.md
Normal file
532
project-docs/dev-plan.md
Normal file
@@ -0,0 +1,532 @@
|
||||
# Astrolabe Development Plan
|
||||
|
||||
**A lightweight, browser-based snippet manager for Vega-Lite visualizations**
|
||||
|
||||
> **For AI Context**: See `CLAUDE.md` for current status and key patterns.
|
||||
> **This Document**: Complete development roadmap and technical decisions.
|
||||
|
||||
## Project Vision
|
||||
|
||||
Astrolabe is a focused tool for managing, editing, and previewing Vega-Lite visualization specifications. It emphasizes efficient snippet management with a clean, resizable three-panel interface: snippet library, Monaco editor with schema-aware autocomplete, and live preview.
|
||||
|
||||
## Design Principles
|
||||
|
||||
- **Local-first**: All data stored in browser (localStorage, future IndexedDB migration)
|
||||
- **Minimal dependencies**: Vanilla JavaScript, no build tools, direct CDN imports
|
||||
- **Developer-friendly**: Full JSON schema support, syntax validation, and intellisense
|
||||
- **Version-aware**: Draft/published workflow for safe experimentation
|
||||
- **Dataset management**: Separate storage for datasets with reference system
|
||||
- **Extensible**: Clean architecture for future cloud sync and authentication
|
||||
|
||||
## Planned Feature Set
|
||||
|
||||
- Resizable three-panel layout with show/hide toggles
|
||||
- Auto-saving draft system with explicit publishing
|
||||
- Snippet organization with sorting and search
|
||||
- Storage monitoring and size tracking
|
||||
- Export/import functionality
|
||||
- Dataset library and management
|
||||
- (Future) Authentication and cloud synchronization
|
||||
|
||||
---
|
||||
|
||||
## Development Roadmap
|
||||
|
||||
### **Phase 0: Storage Architecture Design** ✅ **COMPLETE**
|
||||
**Goal**: Define data structures that accommodate full feature set
|
||||
|
||||
- [x] Design snippet schema: `{id, name, created, modified, spec, draftSpec, comment, tags[], datasetRefs[], meta}`
|
||||
- [x] Design dataset schema: `{id, name, created, modified, data, format, rowCount, columnCount, size, columns, comment, tags, meta}`
|
||||
- [x] Design settings schema: `{panelWidths[], panelVisibility[], autoSaveDelay, meta}`
|
||||
- [x] Plan localStorage key structure and namespacing
|
||||
- [x] Define upgrade/migration path for schema changes
|
||||
- [x] Document storage architecture decisions (see storage-examples.md)
|
||||
|
||||
**Deliverable**: Documented data model that supports all planned features
|
||||
|
||||
**Key Decisions Made**:
|
||||
- ID generation: `Date.now() + random numbers` for uniqueness
|
||||
- Auto-naming: ISO datetime format for default snippet names
|
||||
- Draft/published workflow: separate `spec` and `draftSpec` fields
|
||||
- Computed metadata: automatic calculation of size, rowCount, columns for datasets
|
||||
- Extensibility: `meta` field in all schemas for future enhancements
|
||||
- Settings persistence: panel state and UI preferences stored locally
|
||||
|
||||
---
|
||||
|
||||
### **Phase 1: Static HTML Structure** ✅ **COMPLETE**
|
||||
**Goal**: Basic three-panel layout with placeholder content
|
||||
|
||||
**Deliverables**:
|
||||
- Three-panel flexbox layout (snippet library, editor, preview)
|
||||
- Header with site branding (🔭 Astrolabe) and action links
|
||||
- Toggle button strip with emoji icons (📄 ✏️ 👁️)
|
||||
- Retro Windows 2000 aesthetic with classic colors and square edges
|
||||
- Sort and search controls in snippet panel header
|
||||
- Metadata panel for snippet details
|
||||
|
||||
---
|
||||
|
||||
### **Phase 2: Resizable Panels** ✅ **COMPLETE**
|
||||
**Goal**: Make panels draggable to resize
|
||||
|
||||
**Deliverables**:
|
||||
- Drag handles between panels with hover/active states
|
||||
- Mouse-based horizontal resizing with 200px minimum widths
|
||||
- Panel memory system preserving preferred sizes when toggled
|
||||
- Proportional redistribution when panels are hidden/shown
|
||||
- localStorage persistence for panel widths, visibility, and memory
|
||||
- Toggle buttons with active state indicators
|
||||
|
||||
---
|
||||
|
||||
### **Phase 3: Monaco Editor Integration** ✅ **COMPLETE**
|
||||
**Goal**: Working code editor with Vega-Lite schema
|
||||
|
||||
**Deliverables**:
|
||||
- Monaco Editor v0.47.0 loaded from CDN
|
||||
- JSON language mode with Vega-Lite v5 schema validation
|
||||
- Autocomplete and intellisense for Vega-Lite specifications
|
||||
- Format on paste/type enabled
|
||||
- Error handling for schema loading failures
|
||||
|
||||
---
|
||||
|
||||
### **Phase 4: Vega-Lite Rendering** ✅ **COMPLETE**
|
||||
**Goal**: Live preview of visualizations
|
||||
|
||||
**Deliverables**:
|
||||
- Vega, Vega-Lite, and Vega-Embed libraries loaded via CDN
|
||||
- AMD loader conflict resolution between Monaco and Vega
|
||||
- 1.5s debounced rendering on editor changes
|
||||
- Immediate rendering on snippet selection
|
||||
- Error display in preview panel for invalid specs
|
||||
- SVG rendering for high-quality output
|
||||
|
||||
---
|
||||
|
||||
### **Phase 5: Data Model + LocalStorage** ✅ **COMPLETE**
|
||||
**Goal**: Persist snippets and load them on page refresh
|
||||
|
||||
**Deliverables**:
|
||||
- SnippetStorage wrapper with error handling
|
||||
- Full Phase 0 schema implementation (id, name, created, modified, spec, draftSpec, comment, tags, datasetRefs, meta)
|
||||
- ID generation using timestamp + random numbers
|
||||
- Auto-naming with ISO datetime format (YYYY-MM-DD_HH-MM-SS)
|
||||
- Dynamic snippet list rendering with relative date formatting
|
||||
- Default snippet initialization on first load
|
||||
- Snippet selection loading specs into editor
|
||||
|
||||
---
|
||||
|
||||
### **Phase 6: Snippet Selection & Basic CRUD** ✅ **COMPLETE**
|
||||
**Goal**: Core snippet management with advanced organization
|
||||
|
||||
**Deliverables**:
|
||||
- Snippet selection loading into editor with live preview
|
||||
- Ghost card "Create New Snippet" interface at top of list
|
||||
- Duplicate button creating copies with "_copy" suffix
|
||||
- Delete button with confirmation dialog
|
||||
- Inline name editing in metadata panel with auto-save
|
||||
- Comment field with auto-save
|
||||
- 1-second debounced auto-save for spec and metadata changes
|
||||
- Multi-field sorting (Modified/Created/Name) with ascending/descending toggle
|
||||
- Visual sort indicators with arrow icons (⬇⬆)
|
||||
- Real-time search across name, comment, and spec content (300ms debounce)
|
||||
- Search clear button
|
||||
- Settings persistence for sort preferences
|
||||
- Smart state management maintaining selection during operations
|
||||
- Synchronous flag-based prevention of auto-save during programmatic updates
|
||||
|
||||
---
|
||||
|
||||
### **Phase 7: Draft/Published Workflow** ✅ **COMPLETE**
|
||||
**Goal**: Safe experimentation without losing working versions
|
||||
|
||||
**Deliverables**:
|
||||
- Draft/Published toggle buttons in editor header (merged visual design)
|
||||
- Status indicator lights on snippets (🟢 green = no changes, 🟡 yellow = has draft)
|
||||
- Publish button (green, copies draftSpec → spec)
|
||||
- Revert button (orange, copies spec → draftSpec with confirmation)
|
||||
- Context-aware button visibility (only shown in draft mode)
|
||||
- Read-only published view when draft exists
|
||||
- Auto-draft creation when editing published view without draft
|
||||
- Auto-select first snippet on page load
|
||||
- Instant status light updates after auto-save
|
||||
|
||||
---
|
||||
|
||||
### **Phase 8: Storage Monitoring** ✅ **COMPLETE**
|
||||
**Goal**: Show storage usage and limits
|
||||
|
||||
**Deliverables**:
|
||||
- Storage usage calculation using Blob API for accurate byte counting
|
||||
- Progress bar display showing usage vs 5MB limit (e.g., "2.3 MB / 5 MB")
|
||||
- Visual warning states: green (normal), orange (90%+ warning), red (95%+ critical)
|
||||
- Storage monitor positioned at bottom of snippet panel below metadata
|
||||
- Automatic updates after every save operation
|
||||
- Alert dialog when localStorage quota is exceeded
|
||||
- Flexbox layout ensuring monitor stays at panel bottom with scrollable snippet list
|
||||
|
||||
---
|
||||
|
||||
### **Phase 9: Export/Import** ✅ **COMPLETE**
|
||||
**Goal**: Portability and backup
|
||||
|
||||
**Deliverables**:
|
||||
- Export all snippets to JSON file with auto-generated filename (astrolabe-snippets-YYYY-MM-DD.json)
|
||||
- Import snippets from JSON with automatic format detection
|
||||
- Support for both Astrolabe native format and external formats
|
||||
- Automatic field mapping and normalization (createdAt → created, content → spec, draft → draftSpec)
|
||||
- "imported" tag automatically added to externally imported snippets
|
||||
- ID conflict resolution with automatic regeneration
|
||||
- Additive import (merges with existing snippets, no overwrites)
|
||||
- Success/error feedback with import count
|
||||
- File picker integration with header Import/Export links
|
||||
- Snippet size display in list (shows KB for snippets ≥ 1 KB, right-aligned)
|
||||
|
||||
**Note**: Drag-and-drop import and selective export options deferred to future phases
|
||||
|
||||
---
|
||||
|
||||
### **Phase 10: Dataset Management** ✅ **COMPLETE**
|
||||
**Goal**: Separate dataset storage with multiple data sources and formats
|
||||
|
||||
**Deliverables**:
|
||||
- IndexedDB-based dataset storage (separate from snippets, no 5MB localStorage limit)
|
||||
- Full CRUD operations for datasets (create, read, update, delete)
|
||||
- Modal-based Dataset Manager UI (accessible via header link and 📁 toggle button)
|
||||
- Support for multiple data sources:
|
||||
- **Inline data**: JSON, CSV, TSV, TopoJSON stored directly
|
||||
- **URL data**: Remote data sources with format specification
|
||||
- **Intelligent auto-detection system**:
|
||||
- Single input field for data or URL
|
||||
- Automatic URL detection and content fetching
|
||||
- Format detection from content (JSON, CSV, TSV, TopoJSON)
|
||||
- Confidence scoring (high/medium/low)
|
||||
- Visual confirmation UI with badges and preview
|
||||
- Automatic metadata calculation:
|
||||
- Row count, column count, column names
|
||||
- Data size in bytes
|
||||
- Automatic URL fetching on dataset creation
|
||||
- Refresh metadata button for URL datasets (🔄)
|
||||
- Dataset list with informative metadata display
|
||||
- Dataset details panel with:
|
||||
- Editable name and comment
|
||||
- Statistics display (rows, columns, size)
|
||||
- Data preview (first 5 rows or URL info)
|
||||
- Copy reference button (copies `"data": {"name": "dataset-name"}`)
|
||||
- Delete button with confirmation
|
||||
- Auto-resolve dataset references in Vega-Lite specs during rendering
|
||||
- Format-aware rendering:
|
||||
- JSON: `{ values: data }`
|
||||
- CSV/TSV: `{ values: data, format: { type: 'csv'/'tsv' } }`
|
||||
- TopoJSON: `{ values: data, format: { type: 'topojson' } }`
|
||||
- URL: `{ url: "...", format: { type: '...' } }`
|
||||
- Graceful error handling for CORS and network failures
|
||||
- Modal scrolling support for small viewports
|
||||
|
||||
**Dataset Schema**:
|
||||
```javascript
|
||||
{
|
||||
id: number,
|
||||
name: string,
|
||||
created: ISO string,
|
||||
modified: ISO string,
|
||||
data: array/string/object, // Inline data or URL string
|
||||
format: 'json'|'csv'|'tsv'|'topojson',
|
||||
source: 'inline'|'url',
|
||||
comment: string,
|
||||
rowCount: number|null,
|
||||
columnCount: number|null,
|
||||
columns: string[],
|
||||
size: number|null // bytes
|
||||
}
|
||||
```
|
||||
|
||||
**Technical Implementation**:
|
||||
- IndexedDB with keyPath 'id', indexes on 'name' (unique) and 'modified'
|
||||
- Async/Promise-based DatasetStorage API
|
||||
- Format-specific parsing for metadata calculation
|
||||
- Vega-Lite's native format parsers used for rendering
|
||||
- Metadata refresh fetches live data and updates statistics
|
||||
- Modal resizes with viewport (max-height: 90vh)
|
||||
- **Auto-detection algorithms**:
|
||||
- URL validation with protocol check (http/https)
|
||||
- JSON parsing with TopoJSON identification
|
||||
- CSV/TSV detection via delimiter counting and consistency checks
|
||||
- Format inference from URL file extensions
|
||||
- Debounced input handling for real-time feedback
|
||||
|
||||
---
|
||||
|
||||
### **Phase 11: URL State Management** ✅ **COMPLETE**
|
||||
**Goal**: Shareable URLs and browser navigation support
|
||||
|
||||
**Deliverables**:
|
||||
- Hash-based URL routing for snippets and datasets
|
||||
- URL patterns:
|
||||
- Snippet selection: `#snippet-123456`
|
||||
- Dataset modal open: `#datasets`
|
||||
- Dataset selected: `#datasets/dataset-123456`
|
||||
- New dataset form: `#datasets/new`
|
||||
- URLState utility in config.js with parse/update/clear methods
|
||||
- Browser back/forward navigation (hashchange event listener)
|
||||
- Page reload preserves selected snippet or dataset state
|
||||
- Automatic state restoration after editor initialization
|
||||
- Restores snippet URL when closing dataset modal
|
||||
- No external dependencies (native Hash API and History API)
|
||||
|
||||
**Technical Implementation**:
|
||||
- `URLState.parse()`: Parses hash into state object
|
||||
- `URLState.update(state, replaceState)`: Updates URL with optional history.replaceState
|
||||
- `URLState.clear()`: Removes hash from URL
|
||||
- Integration points:
|
||||
- `selectSnippet()`: Updates URL on snippet selection
|
||||
- `selectDataset()`: Updates URL on dataset selection
|
||||
- `openDatasetManager()`: Sets `#datasets` hash
|
||||
- `closeDatasetManager()`: Restores snippet hash or clears
|
||||
- `showNewDatasetForm()`: Sets `#datasets/new` hash
|
||||
- `handleURLStateChange()`: Responds to hashchange events
|
||||
- `initializeURLStateManagement()`: Called after Monaco editor ready
|
||||
- Prevents double-prefix issues by handling prefix addition in URLState.update()
|
||||
- Optional `updateURL` parameter on all functions to prevent infinite loops
|
||||
|
||||
**Benefits**:
|
||||
- Shareable links to specific snippets or datasets
|
||||
- Browser navigation works intuitively
|
||||
- Page refresh preserves user context
|
||||
- Better UX for multi-tab workflows
|
||||
|
||||
---
|
||||
|
||||
### **Phase 12: Advanced Dataset Features** ✅ **COMPLETE**
|
||||
**Goal**: Enhanced dataset workflows
|
||||
|
||||
**Deliverables**:
|
||||
- Recursive dataset reference extraction from Vega-Lite specs
|
||||
- Extract to Dataset modal with automatic spec transformation
|
||||
- Bidirectional linking between snippets and datasets
|
||||
- Usage tracking with visual indicators (📁 icon, 📄 count badges)
|
||||
- Dataset import/export with auto-format detection
|
||||
- Table preview with type detection (🔢📅✓🔤)
|
||||
- On-demand URL preview loading with session cache
|
||||
- New Snippet creation from datasets
|
||||
|
||||
**Technical Implementation**:
|
||||
- Type detection: 80% threshold for number/date/boolean/text inference
|
||||
- Table rendering: Vanilla JS with sticky headers, 20-row preview
|
||||
- Cell formatting: Type-specific styling and alignment
|
||||
- Import: Automatic naming with timestamp collision handling
|
||||
- Export: Format-specific extensions with URL content fetching
|
||||
- Preview cache: In-memory storage, not persisted to database
|
||||
- Modal: 95% width (max 1200px), 85% height for improved data visibility
|
||||
|
||||
---
|
||||
|
||||
### **Phase 13: Polish & UX Refinements** ✅ **COMPLETE**
|
||||
**Goal**: Professional feel and usability
|
||||
|
||||
- [x] **Keyboard shortcuts** - Cross-platform support (Cmd/Ctrl+Shift+N, Cmd/Ctrl+K, Cmd/Ctrl+S, Escape)
|
||||
- [x] **Tooltips for buttons and features** - Added to all interactive elements throughout the UI
|
||||
- [x] **Better error messages and validation feedback** - Toast notification system with 4 types (error, success, warning, info)
|
||||
- [x] **Enhanced Help documentation** - Comprehensive help modal with About, Features, Getting Started, Shortcuts, Storage, and Privacy sections
|
||||
- [x] **User Settings System** - Configurable editor options, performance tuning, date formatting, and theme selection
|
||||
- [x] **Theme System** - Light theme (Windows 2000) and experimental dark theme with CSS variables
|
||||
|
||||
**Deliverable**: Polished, production-ready MVP
|
||||
|
||||
**Completed Features**:
|
||||
- Keyboard shortcut system with dual handlers (document-level + Monaco integration)
|
||||
- Toast notification system (bottom-right, 4s auto-dismiss, slide animations)
|
||||
- Replaced 19 `alert()` calls with contextual toasts
|
||||
- Added success feedback for silent operations (publish, duplicate, delete, export, etc.)
|
||||
- Comprehensive tooltips with keyboard shortcut hints
|
||||
- Data persistence warning in Help section
|
||||
- Help modal with 6 sections covering full app functionality (About, Features, Getting Started, Keyboard Shortcuts, Storage, Privacy & Data)
|
||||
- User settings with localStorage persistence and validation
|
||||
- Light theme (Windows 2000 aesthetic) and experimental dark theme
|
||||
- Automatic editor theme synchronization with UI theme
|
||||
- Custom date formatting with smart/relative times, locale, ISO, and token-based formats
|
||||
- Size sorting for snippets (blob-based calculation)
|
||||
- Accessibility improvements with proper ARIA labels and color contrast
|
||||
|
||||
---
|
||||
|
||||
### **Phase 14: Advanced Snippet Features**
|
||||
**Goal**: Power user functionality
|
||||
|
||||
- [ ] Basic tagging system with tag filtering
|
||||
- [ ] Snippet templates/starter library
|
||||
- [ ] Bulk operations (delete multiple, export selected)
|
||||
- [ ] Size calculation and display for snippets
|
||||
|
||||
**Deliverable**: Enhanced snippet discovery and organization
|
||||
|
||||
**Note**: Search/filter and sorting already implemented in Phase 6
|
||||
|
||||
---
|
||||
|
||||
### **Phase 15: Authentication & Backend** _(Future)_
|
||||
**Goal**: Multi-device sync and sharing
|
||||
|
||||
- [ ] Design minimal backend API
|
||||
- [ ] User authentication (email/password or OAuth)
|
||||
- [ ] Sync localStorage ↔ cloud storage
|
||||
- [ ] Conflict resolution strategy
|
||||
- [ ] Share snippets via URL
|
||||
- [ ] Public snippet gallery (optional)
|
||||
|
||||
**Deliverable**: Cloud-backed, multi-device Astrolabe
|
||||
|
||||
---
|
||||
|
||||
## Technical Stack
|
||||
|
||||
- **Frontend**: Vanilla JavaScript (ES6+), HTML5, CSS3
|
||||
- **Editor**: Monaco Editor v0.47.0 (via CDN)
|
||||
- **Visualization**: Vega-Embed v6 (includes Vega v5 & Vega-Lite v5)
|
||||
- **Storage**: LocalStorage → IndexedDB (when needed)
|
||||
- **Architecture**: Modular script organization with simple file separation
|
||||
- **Backend** _(future)_: TBD (minimal REST API)
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
/
|
||||
├── index.html # Main HTML structure and markup
|
||||
├── CLAUDE.md # Project instructions for Claude Code
|
||||
├── src/
|
||||
│ ├── styles.css # Retro Windows 2000 aesthetic styling
|
||||
│ └── js/ # Modular JavaScript organization
|
||||
│ ├── config.js # Global variables, settings, & sample data
|
||||
│ ├── snippet-manager.js # Snippet storage, CRUD operations & localStorage wrapper
|
||||
│ ├── dataset-manager.js # Dataset storage, CRUD operations & IndexedDB wrapper
|
||||
│ ├── panel-manager.js # Panel resize, toggle & memory system
|
||||
│ ├── editor.js # Monaco Editor initialization, dataset resolution & Vega-Lite rendering
|
||||
│ └── app.js # Application initialization & event handlers
|
||||
└── docs/
|
||||
├── dev-plan.md # This development roadmap
|
||||
└── storage-examples.md # Data model specifications
|
||||
```
|
||||
|
||||
## Development Principles
|
||||
|
||||
- **Iterative**: Each phase produces working, testable functionality
|
||||
- **Lean**: No frameworks, no build step, minimal dependencies
|
||||
- **Data-first**: Storage schema designed upfront for extensibility
|
||||
- **User-focused**: Auto-save, clear state, forgiving UX
|
||||
- **Maintainable**: Clean code organization with logical separation of concerns
|
||||
- **Simple**: Favor code removal over addition; avoid over-engineering
|
||||
|
||||
## Code Organization
|
||||
|
||||
**Web Application** (`/web` directory):
|
||||
- **index.html**: Main HTML structure with semantic markup
|
||||
- **src/app.js**: Application initialization and event handler registration
|
||||
- **src/config.js**: Global variables, settings management, and sample data
|
||||
- **src/snippet-manager.js**: Storage wrapper and all snippet CRUD operations
|
||||
- **src/panel-manager.js**: Layout resizing, toggling, and persistence
|
||||
- **src/editor.js**: Monaco and Vega library loading and rendering logic
|
||||
- **src/dataset-manager.js**: Dataset CRUD and IndexedDB operations
|
||||
- **src/user-settings.js**: Settings management with localStorage persistence and validation
|
||||
- **src/styles.css**: Retro Windows 2000 aesthetic with CSS variables for theming
|
||||
|
||||
**Documentation** (`/docs` directory - not served on GitHub Pages):
|
||||
- **dev-plan.md**: Complete development roadmap and technical decisions
|
||||
- **features-list.md**: Feature matrix and implementation status
|
||||
- **storage-examples.md**: Data structure examples and schema documentation
|
||||
- **CLAUDE.md**: Context and status summary for AI assistance
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
**Completed**: Phases 0-13 (All core features, dataset management, advanced features, and UX refinements)
|
||||
**In Progress**: GitHub Pages deployment preparation and project restructuring for public distribution
|
||||
**Next**: Phase 14 - Advanced Snippet Features (tagging system, templates, bulk operations) or public launch
|
||||
**See**: `CLAUDE.md` for concise current state summary
|
||||
|
||||
---
|
||||
|
||||
## Implemented Features
|
||||
|
||||
### Core Capabilities (Phases 0-11)
|
||||
- Three-panel resizable layout with memory and persistence
|
||||
- Monaco Editor v0.47.0 with Vega-Lite v5 schema validation
|
||||
- Live Vega-Lite rendering with debounced updates and error display
|
||||
- localStorage-based snippet management with full CRUD
|
||||
- Multi-field sorting (Modified/Created/Name) with direction toggle
|
||||
- Real-time search across snippet name, comment, and spec content
|
||||
- Auto-save system (1s debounce) for specs and metadata
|
||||
- Ghost card interface for snippet creation
|
||||
- Draft/Published workflow with version control
|
||||
- Status indicator lights (green/yellow) showing draft state
|
||||
- Context-aware Publish/Revert buttons with color coding
|
||||
- Storage monitoring with visual progress bar and warning states
|
||||
- Export/Import functionality with format auto-detection
|
||||
- Snippet size display (right-aligned, shown for ≥ 1 KB)
|
||||
- **Dataset Management (Phase 10)**:
|
||||
- IndexedDB storage for datasets (unlimited size)
|
||||
- Multi-format support: JSON, CSV, TSV, TopoJSON
|
||||
- Multi-source support: Inline data and URL references
|
||||
- Modal-based Dataset Manager with full CRUD
|
||||
- Intelligent auto-detection (URL/format/confidence)
|
||||
- Visual confirmation UI with badges and preview
|
||||
- Automatic metadata calculation and display
|
||||
- URL metadata fetching and refresh
|
||||
- Dataset reference resolution in Vega-Lite specs
|
||||
- **URL State Management (Phase 11)**:
|
||||
- Hash-based routing for snippets and datasets
|
||||
- Browser back/forward navigation support
|
||||
- Page reload preserves state
|
||||
- Shareable URLs for specific snippets/datasets
|
||||
- Restores snippet URL when closing dataset modal
|
||||
- **Advanced Dataset Features (Phase 12)**:
|
||||
- Bidirectional linking with usage tracking and navigation
|
||||
- Extract to Dataset with automatic spec transformation
|
||||
- Import/Export with format detection and URL content fetching
|
||||
- Table preview with type detection and formatting (🔢📅✓🔤)
|
||||
- On-demand URL preview loading with session cache
|
||||
- New Snippet creation from datasets
|
||||
- **Polish & UX Refinements (Phase 13)**:
|
||||
- Cross-platform keyboard shortcuts (Cmd/Ctrl+Shift+N, Cmd/Ctrl+K, Cmd/Ctrl+S, Escape)
|
||||
- Dual keyboard handlers (document-level + Monaco integration)
|
||||
- Toast notification system (error, success, warning, info)
|
||||
- Contextual feedback for all major operations
|
||||
- Comprehensive tooltips with keyboard hints
|
||||
- Enhanced Help modal with 6 sections (About, Features, Getting Started, Shortcuts, Storage, Privacy & Data)
|
||||
- Data persistence warning in Help section
|
||||
- User Settings System with localStorage persistence
|
||||
- Configurable editor options (font size, tab size, minimap, word wrap, line numbers)
|
||||
- Performance tuning (render debounce delay)
|
||||
- Date formatting options (smart/relative, locale, ISO, custom tokens)
|
||||
- Theme system (Light and experimental Dark themes)
|
||||
- Automatic editor theme synchronization
|
||||
- Size sorting for snippets
|
||||
- Retro Windows 2000 aesthetic throughout
|
||||
- Component-based CSS architecture with base classes
|
||||
|
||||
### Technical Implementation
|
||||
- **State Management**: Synchronous `isUpdatingEditor` flag prevents unwanted auto-saves
|
||||
- **View Modes**: `currentViewMode` tracks draft vs published state
|
||||
- **Read-only Logic**: Monaco editor locked in published view when draft exists
|
||||
- **Auto-draft Creation**: Editing published without draft auto-switches to draft mode
|
||||
- **Debouncing**: 1.5s render, 1s auto-save, 300ms search
|
||||
- **AMD Resolution**: Temporary `window.define` disabling for Vega library loading
|
||||
- **Panel Memory**: localStorage persistence for sizes and visibility across sessions
|
||||
- **Data Model**: Phase 0 schema with `spec` (published) and `draftSpec` (working) fields
|
||||
- **Storage Calculation**: Blob API for accurate byte counting of snippet data
|
||||
- **Flexbox Layout**: Scrollable snippet list with fixed metadata and storage monitor at bottom
|
||||
- **Import/Export**: Format detection, field normalization, ID conflict resolution, additive merging
|
||||
- **Size Display**: Per-snippet size calculation with conditional rendering (≥ 1 KB threshold)
|
||||
- **Dataset Storage**: IndexedDB with async/Promise-based API, unique name constraint
|
||||
- **Dataset Resolution**: Async spec transformation before rendering, format-aware data injection
|
||||
- **URL Metadata**: Fetch on creation with graceful CORS error handling
|
||||
- **Modal UI**: Flexbox with overflow:auto, max-height responsive to viewport (80vh fixed height)
|
||||
- **Auto-detection**: URL validation, JSON/CSV/TSV parsing, confidence scoring, real-time feedback
|
||||
- **URL State Management**: Native Hash API with hashchange listener, initialized after editor ready
|
||||
- **CSS Architecture**: Component-based with base classes (.btn, .input, .preview-box) and modifiers
|
||||
- **Keyboard Shortcuts**: Dual system with document listeners + Monaco commands for full coverage
|
||||
- **Toast Notifications**: Bottom-right positioned, 4s auto-dismiss, slide animations, 4 types (error/success/warning/info)
|
||||
- **User Feedback**: Replaced 19 alert() calls with contextual toasts, kept confirm() for destructive actions
|
||||
344
project-docs/features-list.md
Normal file
344
project-docs/features-list.md
Normal file
@@ -0,0 +1,344 @@
|
||||
# Astrolabe - Current Features List
|
||||
|
||||
> **Purpose**: Comprehensive inventory of all implemented features for code review and optimization
|
||||
> **Created**: 2025-10-15
|
||||
> **Updated**: 2025-10-16
|
||||
> **Status**: Phases 0-12 Complete, Phase 13 In Progress (Keyboard Shortcuts, Notifications, Tooltips, Help Documentation)
|
||||
|
||||
---
|
||||
|
||||
## 🔭 **ASTROLABE - Current Features Inventory**
|
||||
|
||||
### **1. Layout & UI Structure**
|
||||
- Three-panel resizable layout (Snippet Library | Editor | Preview)
|
||||
- Manual drag-to-resize with 200px minimum widths
|
||||
- Panel show/hide toggle buttons (📄 ✏️ 👁️ 📁)
|
||||
- Panel memory system (remembers sizes when toggled)
|
||||
- Proportional width redistribution when panels are hidden/shown
|
||||
- Layout persistence to localStorage across sessions
|
||||
- Retro Windows 2000 aesthetic styling
|
||||
|
||||
**Files**: `panel-manager.js`, `styles.css`, `index.html`
|
||||
|
||||
---
|
||||
|
||||
### **2. Monaco Code Editor**
|
||||
- Monaco Editor v0.47.0 integration via CDN
|
||||
- JSON syntax highlighting and validation
|
||||
- Vega-Lite v5 schema autocomplete and intellisense
|
||||
- Format-on-paste and format-on-type
|
||||
- Read-only mode (when viewing published with draft)
|
||||
- Automatic layout resizing
|
||||
|
||||
**Files**: `app.js` (lines 31-82), `index.html` (line 11)
|
||||
|
||||
---
|
||||
|
||||
### **3. Vega-Lite Rendering**
|
||||
- Live preview panel with Vega-Embed v6
|
||||
- SVG rendering for high quality
|
||||
- 1.5-second debounced rendering on editor changes
|
||||
- Immediate rendering on snippet selection
|
||||
- Error display with helpful messages
|
||||
- Dataset reference resolution (inline & URL)
|
||||
- Recursive resolution for layered/concat/faceted specs
|
||||
|
||||
**Files**: `editor.js`, `app.js` (rendering calls)
|
||||
|
||||
---
|
||||
|
||||
### **4. Snippet Management (CRUD)**
|
||||
- Create new snippet (ghost card interface)
|
||||
- Duplicate snippet (with "_copy" suffix)
|
||||
- Delete snippet (with confirmation dialog)
|
||||
- Inline name editing with auto-save
|
||||
- Comment field with auto-save
|
||||
- Auto-generated ISO datetime names (YYYY-MM-DD_HH-MM-SS)
|
||||
- Unique ID generation (timestamp + random)
|
||||
- Created/Modified timestamps with formatted display
|
||||
|
||||
**Files**: `snippet-manager.js` (lines 7-36, 621-701), `app.js` (event handlers)
|
||||
|
||||
---
|
||||
|
||||
### **5. Snippet Organization**
|
||||
- Multi-field sorting (Modified/Created/Name)
|
||||
- Ascending/descending toggle with arrow indicators (⬇⬆)
|
||||
- Real-time search across name, comment, and spec content
|
||||
- 300ms debounced search
|
||||
- Search clear button with enabled/disabled state
|
||||
- Visual sort indicators on active button
|
||||
- Sort preferences persisted to localStorage
|
||||
- Snippet size display (shows KB for ≥1KB snippets, right-aligned)
|
||||
|
||||
**Files**: `snippet-manager.js` (lines 96-405), `app.js` (initialization)
|
||||
|
||||
---
|
||||
|
||||
### **6. Draft/Published Workflow**
|
||||
- Dual-spec storage (spec = published, draftSpec = working)
|
||||
- View mode toggle (Draft/Published buttons)
|
||||
- Status indicator lights (🟢 green = clean, 🟡 yellow = has draft)
|
||||
- Publish button (copies draftSpec → spec)
|
||||
- Revert button (copies spec → draftSpec with confirmation)
|
||||
- Context-aware button visibility (only in draft mode)
|
||||
- Auto-draft creation when editing published view
|
||||
- Read-only published view when draft exists
|
||||
- 1-second debounced auto-save for draft changes
|
||||
|
||||
**Files**: `snippet-manager.js` (lines 494-810), `config.js` (currentViewMode)
|
||||
|
||||
---
|
||||
|
||||
### **7. Storage Management**
|
||||
- localStorage for snippets (5MB limit awareness)
|
||||
- Storage usage monitor with progress bar
|
||||
- Visual warning states (green/orange/red at 90%/95%)
|
||||
- Accurate byte counting using Blob API
|
||||
- Storage monitor positioned below metadata panel
|
||||
- Real-time updates after save operations
|
||||
- Quota exceeded alerts
|
||||
|
||||
**Files**: `snippet-manager.js` (lines 3-5, 812-853), `index.html` (storage monitor UI)
|
||||
|
||||
---
|
||||
|
||||
### **8. Import/Export**
|
||||
- Export all snippets to JSON with auto-generated filename
|
||||
- Import from JSON with format auto-detection
|
||||
- Support for Astrolabe native format
|
||||
- Support for external formats with field mapping
|
||||
- Automatic "imported" tag for external snippets
|
||||
- ID conflict resolution with regeneration
|
||||
- Additive import (no overwrites)
|
||||
- Success/error feedback with count
|
||||
|
||||
**Files**: `snippet-manager.js` (lines 855-976), `app.js` (lines 92-112)
|
||||
|
||||
---
|
||||
|
||||
### **9. Dataset Management**
|
||||
- IndexedDB storage (unlimited size, separate from snippets)
|
||||
- Multi-format support: JSON, CSV, TSV, TopoJSON
|
||||
- Multi-source support: Inline data & URL references
|
||||
- Full CRUD operations for datasets
|
||||
- Modal-based Dataset Manager UI
|
||||
- Dataset list with metadata display
|
||||
- Dataset details panel with editable name/comment
|
||||
- Statistics display (rows, columns, size)
|
||||
- Data preview (first 5 rows or URL info)
|
||||
- Copy reference button (generates `"data": {"name": "..."}`)
|
||||
- Delete dataset with confirmation
|
||||
- Refresh metadata button for URL datasets (🔄)
|
||||
- **Auto-detection system**:
|
||||
- Single input field for data or URL
|
||||
- Automatic URL detection and content fetching
|
||||
- Format detection (JSON, CSV, TSV, TopoJSON)
|
||||
- Confidence scoring (high/medium/low)
|
||||
- Visual confirmation with badges and preview
|
||||
- URL fetching with CORS error handling
|
||||
- Unique dataset name constraint (IndexedDB index)
|
||||
- Empty state message for no datasets
|
||||
|
||||
**Files**: `dataset-manager.js`, `editor.js` (resolution), `app.js` (event handlers), `index.html` (modal UI)
|
||||
|
||||
---
|
||||
|
||||
### **10. Settings & Persistence**
|
||||
- Settings storage in localStorage (separate from snippets)
|
||||
- Default settings (sortBy: modified, sortOrder: desc)
|
||||
- Settings API (load, save, get, set)
|
||||
- Panel layout persistence
|
||||
- Panel memory persistence
|
||||
- Sort preferences persistence
|
||||
- Auto-restore on page load
|
||||
|
||||
**Files**: `config.js` (lines 19-65), `panel-manager.js` (lines 76-131)
|
||||
|
||||
---
|
||||
|
||||
### **11. Auto-Save System**
|
||||
- 1-second debounced auto-save for editor changes
|
||||
- 1-second debounced auto-save for name/comment changes
|
||||
- Synchronous flag (`isUpdatingEditor`) to prevent unwanted saves
|
||||
- Auto-save only in draft mode
|
||||
- Instant status light updates after save
|
||||
- Storage monitor updates after save
|
||||
|
||||
**Files**: `snippet-manager.js` (lines 494-619), `app.js` (lines 73-75)
|
||||
|
||||
---
|
||||
|
||||
### **12. URL State Management**
|
||||
- Hash-based URL routing for snippets and datasets
|
||||
- Snippet selection persists in URL (`#snippet-123456`)
|
||||
- Dataset modal state persists in URL:
|
||||
- Modal open: `#datasets`
|
||||
- Dataset selected: `#datasets/dataset-123456`
|
||||
- New dataset form: `#datasets/new`
|
||||
- Browser back/forward navigation support
|
||||
- Page reload preserves state (selected snippet/dataset)
|
||||
- URL sharing for specific snippets or datasets
|
||||
- Automatic state restoration on page load
|
||||
- Restores snippet URL when closing dataset modal
|
||||
- No external libraries (native Hash API)
|
||||
|
||||
**Files**: `config.js` (URLState utility), `snippet-manager.js` (snippet URLs), `dataset-manager.js` (dataset URLs), `app.js` (hashchange listener)
|
||||
|
||||
---
|
||||
|
||||
### **13. User Experience Enhancements**
|
||||
- Auto-select first snippet on page load
|
||||
- Relative date formatting (Today/Yesterday/X days ago)
|
||||
- Full datetime display in metadata panel
|
||||
- Empty state placeholders
|
||||
- "No snippets match your search" for search results
|
||||
- Confirmation dialogs for destructive actions
|
||||
- Flexbox layout with scrollable snippet list
|
||||
- Fixed metadata and storage monitor at bottom
|
||||
|
||||
**Files**: `snippet-manager.js` (formatting functions), `app.js` (auto-select), `styles.css` (layout)
|
||||
|
||||
---
|
||||
|
||||
### **13. Advanced Dataset Features (Phase 12)**
|
||||
- **Dataset Dependencies & Linking**:
|
||||
- Recursive dataset reference extraction from Vega-Lite specs
|
||||
- Bidirectional snippet ↔ dataset linking with clickable links
|
||||
- Dataset usage tracking and count badges (📄 count)
|
||||
- Visual indicators (📁) for snippets using datasets
|
||||
- Linked datasets section in snippet metadata
|
||||
- Linked snippets section in dataset details
|
||||
- Navigation between snippets and datasets via links
|
||||
- **Extract to Dataset**:
|
||||
- Inline data detection in draft specs
|
||||
- Extract modal with name generation and preview
|
||||
- Automatic spec transformation (inline → reference)
|
||||
- Auto-update snippet with dataset reference
|
||||
- Conditional Extract button (shows when inline data detected)
|
||||
- **Import/Export Datasets**:
|
||||
- Import from file (.json, .csv, .tsv, .txt)
|
||||
- Auto-format detection from content and filename
|
||||
- Automatic naming from filename with duplicate handling
|
||||
- Timestamp suffix for duplicate names (e.g., "data_123456")
|
||||
- Export to format-specific files (.json, .csv, .tsv, .topojson)
|
||||
- URL dataset export fetches and downloads live content
|
||||
- **Table Preview with Type Detection**:
|
||||
- Raw/Table toggle for tabular data (JSON arrays, CSV, TSV)
|
||||
- Vanilla JS table rendering (first 20 rows)
|
||||
- Sticky headers with scrollable content
|
||||
- Column type detection (80% threshold): number, date, boolean, text
|
||||
- Type icons in headers: 🔢 📅 ✓ 🔤
|
||||
- Type-specific formatting: italic numbers (right-aligned), italic dates, bold booleans
|
||||
- Color coding: blue numbers, green dates, orange booleans, gray nulls
|
||||
- Monospace font (Consolas/Monaco/Courier New)
|
||||
- **URL Preview Loading**:
|
||||
- On-demand fetch with "Load Preview" button
|
||||
- In-memory cache for fetched data (session-only)
|
||||
- Full table view and type detection for fetched data
|
||||
- Error handling with retry option
|
||||
- Data not saved to database (preview only)
|
||||
- **New Snippet from Dataset**:
|
||||
- Button in dataset details to create snippet
|
||||
- Auto-generated minimal Vega-Lite spec with dataset reference
|
||||
- Pre-populated comment and datasetRefs
|
||||
- **UI Enhancements**:
|
||||
- Larger modal (95% width, max 1200px, 85% height)
|
||||
- Actions moved to top of dataset details
|
||||
- Dataset list with usage badges
|
||||
|
||||
**Files**: `dataset-manager.js` (lines 19-1165), `snippet-manager.js` (dataset tracking), `app.js` (event handlers), `index.html` (UI), `styles.css` (table styles)
|
||||
|
||||
---
|
||||
|
||||
## 📊 **Feature Statistics**
|
||||
|
||||
- **Core Feature Groups**: 14
|
||||
- **Total Individual Capabilities**: ~100+
|
||||
- **Storage Systems**: 2 (localStorage for snippets, IndexedDB for datasets)
|
||||
- **UI Panels**: 3 main + 1 modal
|
||||
- **Auto-save Points**: 3 (draft spec, name, comment)
|
||||
- **Data Formats**: 4 (JSON, CSV, TSV, TopoJSON)
|
||||
- **Data Sources**: 2 (inline, URL)
|
||||
- **Type Detection**: 4 types (number, date, boolean, text)
|
||||
- **Import/Export**: Snippets + Datasets
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ **Code Organization**
|
||||
|
||||
```
|
||||
src/
|
||||
├── js/
|
||||
│ ├── config.js # Global variables, settings, sample data
|
||||
│ ├── snippet-manager.js # Snippet CRUD, storage, search, sort, extract (1,100+ lines)
|
||||
│ ├── dataset-manager.js # Dataset CRUD, IndexedDB, preview, types (1,200+ lines)
|
||||
│ ├── panel-manager.js # Layout resizing, toggling, persistence (200 lines)
|
||||
│ ├── editor.js # Monaco setup, Vega rendering, dataset resolution (150 lines)
|
||||
│ └── app.js # Event handlers, initialization (250+ lines)
|
||||
└── styles.css # Retro Windows 2000 aesthetic (280+ lines)
|
||||
```
|
||||
|
||||
**Total JS Lines**: ~2,900+ lines (excluding comments and blank lines)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 **Use This Document For**
|
||||
|
||||
1. **Feature Review**: Select a feature to audit code relevance
|
||||
2. **Code Cleanup**: Identify potential leftovers or redundancies
|
||||
3. **Optimization**: Find opportunities to reduce codebase size
|
||||
4. **Documentation**: Reference for what's actually implemented
|
||||
5. **Planning**: Determine what to polish vs what to add
|
||||
|
||||
---
|
||||
|
||||
### **13. Polish & UX Refinements (Phase 13)** ⏳ _In Progress_
|
||||
|
||||
**Keyboard Shortcuts**:
|
||||
- Cmd/Ctrl+Shift+N: Create new snippet
|
||||
- Cmd/Ctrl+K: Toggle dataset manager
|
||||
- Cmd/Ctrl+S: Publish draft
|
||||
- Escape: Close any open modal
|
||||
- Dual handler system (document-level + Monaco integration)
|
||||
- Cross-platform support (Cmd on Mac, Ctrl on Windows/Linux)
|
||||
|
||||
**Toast Notification System**:
|
||||
- Four types: error, success, warning, info
|
||||
- Bottom-right positioning, 4-second auto-dismiss
|
||||
- Slide-in/slide-out animations
|
||||
- Manual close button
|
||||
- Replaced 19 `alert()` calls with contextual toasts
|
||||
- Kept `confirm()` dialogs for destructive actions
|
||||
|
||||
**Tooltips**:
|
||||
- Added to all interactive elements (buttons, links, controls)
|
||||
- Keyboard shortcut hints included where applicable
|
||||
- Consistent tooltip text across the application
|
||||
|
||||
**Enhanced Help Documentation**:
|
||||
- Comprehensive Help modal with 6 sections
|
||||
- About Astrolabe (project vision and approach)
|
||||
- Key Features (scannable highlights)
|
||||
- Getting Started (4-step workflow guide)
|
||||
- Keyboard Shortcuts (reference table)
|
||||
- Storage & Limits (with data persistence warning)
|
||||
- Tips & Tricks (power user features)
|
||||
|
||||
**Files**: `app.js` (keyboard handlers), `config.js` (Toast utility), `styles.css` (toast/help styles), `index.html` (help modal)
|
||||
|
||||
---
|
||||
|
||||
## 📝 **Next Steps**
|
||||
|
||||
### Phase 13 In Progress! Keyboard shortcuts, notifications, tooltips, and help documentation complete.
|
||||
|
||||
**Remaining Phase 13 Tasks**:
|
||||
- [ ] Loading states for rendering
|
||||
- [ ] Empty states (no snippets, no datasets)
|
||||
- [ ] Improved visual design polish
|
||||
- [ ] Cross-browser testing
|
||||
|
||||
**Future Phases**:
|
||||
- **Phase 14**: Advanced Snippet Features (tagging system, templates, bulk operations)
|
||||
- **Phase 15**: Authentication & Backend (cloud sync, sharing)
|
||||
92
project-docs/storage-examples.md
Normal file
92
project-docs/storage-examples.md
Normal file
@@ -0,0 +1,92 @@
|
||||
|
||||
|
||||
Snippet example:
|
||||
```json
|
||||
|
||||
{
|
||||
id: 1760200058.123123, // Date.now() + random numbers
|
||||
name: "2025-10-13_14-23-45", // auto-populated
|
||||
created: "2025-10-13T14:23:45.123Z",
|
||||
modified: "2025-10-13T14:25:30.456Z",
|
||||
|
||||
spec: {
|
||||
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
|
||||
"data": {"values": [...]},
|
||||
"mark": "bar",
|
||||
"encoding": {
|
||||
"x": {"field": "category", "type": "nominal"},
|
||||
"y": {"field": "value", "type": "quantitative"}
|
||||
}
|
||||
},
|
||||
|
||||
draftSpec: {
|
||||
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
|
||||
"data": {"values": [...]},
|
||||
"mark": "bar",
|
||||
"encoding": {
|
||||
"x": {"field": "category", "type": "nominal"},
|
||||
"y": {"field": "value", "type": "quantitative"},
|
||||
"color": {"field": "category", "type": "nominal"} // draft addition
|
||||
}
|
||||
},
|
||||
|
||||
comment: "SELECT category, SUM(value) FROM sales GROUP BY category",
|
||||
tags: [],
|
||||
datasetRefs: [],
|
||||
meta: {}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Dataset example:
|
||||
|
||||
```json
|
||||
{
|
||||
// Core identity
|
||||
id: 1760200058.123123, // Date.now() + random numbers
|
||||
name: "sales_2024_q1", // editable
|
||||
|
||||
// Timestamps
|
||||
created: 1760200058, // timestamp
|
||||
modified: 1760304958 //timestamp,
|
||||
|
||||
// Data (stored as-is, format-agnostic)
|
||||
data: [
|
||||
{"category": "A", "value": 100},
|
||||
{"category": "B", "value": 200}
|
||||
],
|
||||
format: "json", // "json" | "csv" | "tsv" (for future import/export)
|
||||
|
||||
// Computed on save
|
||||
rowCount: 2,
|
||||
columnCount: 2,
|
||||
size: 78, // bytes: JSON.stringify(data).length
|
||||
columns: ["category", "value"], // inferred from first row
|
||||
|
||||
// Metadata
|
||||
comment: "",
|
||||
tags: [],
|
||||
|
||||
// Extensibility
|
||||
meta: {}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
Settings example:
|
||||
|
||||
```json
|
||||
|
||||
|
||||
{
|
||||
// UI State
|
||||
panelWidths: [25, 25, 50], // % for [list, editor, preview]
|
||||
panelVisibility: [true, true, true],
|
||||
|
||||
// App behavior (empty for MVP, but structure exists)
|
||||
autoSaveDelay: 1000,
|
||||
|
||||
// Extensibility
|
||||
meta: {}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user