Spaces:

Sid-WC121
/

visual-notebook

Sleeping

Sid-WC121

bug fix1

018ab56 6 months ago

13 kB

# Frontend Architecture

This document describes the frontend architecture of the Visual Notebook, covering the React + TypeScript + Vite setup, Tailwind CSS configuration, Zustand state management, the API client layer, component hierarchy, the cell system, chart rendering, and map visualization.

## React + TypeScript + Vite Setup

The frontend is built as a single-page application using React 18 with TypeScript and Vite 5. The project lives under the `frontend/` directory.

### Vite Configuration

The Vite configuration in `vite.config.ts` defines two key features:

**Development proxy**: During local development, API requests to `/api` and `/health` are proxied to the backend at `http://localhost:8000`, avoiding CORS issues:

```typescript
server: {
    proxy: {
        '/api': { target: 'http://localhost:8000', changeOrigin: true },
        '/health': { target: 'http://localhost:8000', changeOrigin: true },
    }
}
```

**Production build optimization**: The Rollup output is manually chunked to separate large dependencies:

```typescript
manualChunks: {
    echarts: ['echarts', 'echarts-for-react'],
    react: ['react', 'react-dom'],
    vendor: ['axios', 'zustand'],
}
```

This separation ensures that ECharts (the largest dependency) and React are loaded as separate cacheable chunks, improving initial page load performance and enabling better browser caching on subsequent visits.

### TypeScript Configuration

TypeScript is configured with strict mode. The `api/types.ts` file mirrors the Pydantic response schemas from the backend, ensuring compile-time type safety for API responses. This contract between frontend and backend types catches breaking changes at build time.

## Tailwind CSS Configuration

The project uses Tailwind CSS 3 with a custom brand color theme defined in `tailwind.config.js`:

```javascript
colors: {
    brand: {
        50: '#eef2ff',
        100: '#e0e7ff',
        200: '#c7d2fe',
        300: '#a5b4fc',
        400: '#818cf8',
        500: '#6366f1',
        600: '#4f46e5',
        700: '#4338ca',
        800: '#3730a3',
        900: '#312e81',
    }
}
```

This indigo-based palette is used throughout the application for primary buttons, active states, focus rings, and brand accents. The 10-shade scale provides sufficient contrast for accessibility while maintaining visual consistency.

PostCSS with Autoprefixer processes the CSS for cross-browser compatibility. The global stylesheet in `src/styles/index.css` imports the Tailwind base, components, and utilities layers.

## Zustand Stores

The frontend uses four independent Zustand stores, each managing a distinct domain of application state.

### useSessionStore

Manages session and dataset metadata:

| Field | Type | Description |
|-------|------|-------------|
| `sessionId` | `string \| null` | Backend session ID |
| `datasetId` | `string \| null` | Uploaded dataset ID |
| `tableName` | `string \| null` | DuckDB table name |
| `rowCount` | `number` | Total row count |
| `columns` | `ColumnInfo[]` | Column schema with statistics |
| `isConnected` | `boolean` | Backend connectivity flag |
| `isLoaded` | `boolean` | Whether a dataset is loaded |

Actions: `setSession`, `setSchema`, `clearSession`, `setConnected`. The `isLoaded` flag is the primary gate for conditional rendering -- components that require a dataset check this flag before rendering.

### useNotebookStore

Manages the notebook cell layout:

| Field | Type | Description |
|-------|------|-------------|
| `cells` | `Cell[]` | Ordered list of notebook cells |
| `activeCellId` | `string \| null` | Currently selected cell |

Actions: `addCell(type, title?, config?)`, `removeCell(id)`, `updateCell(id, updates)`, `setActiveCell(id)`, `moveCell(fromIndex, toIndex)`, `toggleCollapse(id)`.

Each cell has a unique ID generated from a counter and timestamp, ensuring stability across re-renders. The `addCell` method returns the new cell ID for programmatic selection.

### useUIStore

Manages UI panel toggles and error state:

| Field | Type | Description |
|-------|------|-------------|
| `selectedSlot` | `'table' \| 'chart' \| 'schema' \| 'map' \| null` | Active cell type slot |
| `showUploadDialog` | `boolean` | Upload dialog visibility |
| `showManipulationPanel` | `boolean` | Transform panel visibility |
| `showVisualizationPanel` | `boolean` | Visualization panel visibility |
| `errors` | `string[]` | Queue of error messages |

Actions: `setSelectedSlot`, `toggleUploadDialog`, `toggleManipulationPanel`, `toggleVisualizationPanel`, `addError`, `clearErrors`, `removeError`.

The error queue allows multiple errors to be displayed simultaneously, with individual dismissal.

### useHistoryStore

Manages the undo/redo state synchronized with the backend:

| Field | Type | Description |
|-------|------|-------------|
| `nodes` | `HistoryNode[]` | Full history tree nodes |
| `currentId` | `string \| null` | Current state ID |
| `order` | `string[]` | Linear state order |
| `canUndo` | `boolean` | Whether undo is available |
| `canRedo` | `boolean` | Whether redo is available |

Actions: `setHistory`, `updateCurrent`, `setUndoRedo`, `reset`. The `canUndo` and `canRedo` flags are computed from the position of `currentId` in the `order` array.

## API Client Layer

### Axios Instance

The API client in `api/client.ts` creates a configured Axios instance:

```typescript
const api = axios.create({
    baseURL: import.meta.env.VITE_API_URL || '',
    withCredentials: true,
    headers: { 'Content-Type': 'application/json' },
});
```

Key configuration:

- **baseURL**: Reads from `VITE_API_URL` environment variable, defaulting to the same origin (for production where the backend serves the frontend). In development, the Vite proxy handles routing.
- **withCredentials**: Enables sending cookies (specifically the `session_id` cookie) with every request.
- **Response interceptor**: Extracts the `detail` field from error responses, providing a consistent error message format.

### API Helper Functions

The client module exports typed async functions for each API endpoint:

| Function | Method | Endpoint |
|----------|--------|----------|
| `uploadCSV(file)` | POST | `/api/upload/csv` |
| `getSchema()` | GET | `/api/schema` |
| `getRows(offset, limit)` | GET | `/api/rows` |
| `listOperations()` | GET | `/api/ops` |
| `applyOperation(opKey, params)` | POST | `/api/ops/{opKey}` |
| `undo()` | POST | `/api/history/undo` |
| `redo()` | POST | `/api/history/redo` |
| `getHistory()` | GET | `/api/history` |
| `createChart(req)` | POST | `/api/viz/chart` |
| `exportCSV()` | GET | `/api/export/csv` |
| `createCheckpoint()` | POST | `/api/checkpoint` |
| `checkHealth()` | GET | `/health` |

The CSV upload uses `multipart/form-data` content type. The CSV export uses `responseType: 'blob'` for binary file download.

### React Hooks

The `api/hooks.ts` module wraps the API functions in React hooks with loading and error state management. Each hook follows the same pattern:

```typescript
export function useUpload() {
    const [loading, setLoading] = useState(false);
    const [error, setError] = useState<string | null>(null);
    const upload = useCallback(async (file: File): Promise<UploadResponse | null> => {
        setLoading(true);
        setError(null);
        try {
            return await uploadCSV(file);
        } catch (err: any) {
            setError(err.message);
            return null;
        } finally {
            setLoading(false);
        }
    }, []);
    return { upload, loading, error };
}
```

Available hooks: `useUpload`, `useSchema`, `useRows`, `useApplyOp`, `useHistory`, `useChart`, `useExport`.

The `useExport` hook handles file download by creating a temporary anchor element and triggering a click event.

## Component Hierarchy

The component tree is structured as follows:

```
App (ErrorBoundary)
  +-- Header
  +-- NotebookPage
  |     +-- UploadPanel (modal, shown when no dataset or "New File" clicked)
  |     +-- TableCellView (for each cell of type "table")
  |     +-- ChartCellView (for each cell of type "chart")
  |     +-- SchemaView (for each cell of type "schema")
  |     +-- MapCanvas (for each cell of type "map")
  |     +-- ManipulationPanel (side panel, toggled from header)
  |     +-- VisualizationPanel (side panel, toggled from header)
  +-- ErrorToast
```

### Header

The header component displays the application title, dataset metadata (table name and row count), undo/redo buttons, cell creation buttons, transform/visualize panel toggles, and the CSV export button. It reads from `useSessionStore`, `useHistoryStore`, and `useUIStore`.

### NotebookPage

The main content area that conditionally renders the landing screen (when no dataset is loaded) or the notebook cell list. The landing screen displays a centered call-to-action with the upload button. The notebook area renders each cell with a collapsible header showing the cell index, title, type badge, and collapse/remove controls.

### UploadPanel

A modal dialog that wraps an HTML file input for CSV upload. Uses the `useUpload` hook to manage the upload lifecycle and update the session store on success.

### TableCellView

Renders paginated table data using the `useRows` hook. Displays column headers and row data in a scrollable grid.

### ChartCellView

Renders ECharts visualizations using the `echarts-for-react` library. When a chart configuration is provided (chartType and params), it calls the `useChart` hook to compute the chart spec and renders it with `ReactEChartsCore`. The component supports dynamic chart type changes and displays a loading spinner during computation.

### SchemaView

Displays the column schema as a table with column name, type, statistics (null count, unique count, min, max, mean, std, quartiles). Uses the `useSchema` hook.

### ManipulationPanel

A side panel that provides forms for applying data operations (filter, sort, select, rename, groupby, fill_nulls, clip). Uses the `useApplyOp` hook and reads available operations from `listOperations()`.
### VisualizationPanel
A side panel for configuring chart visualizations. Allows the user to select a chart type, choose columns, and set parameters. Creates chart cells in the notebook when a visualization is configured.
### MapCanvas
Renders geographic point data. Currently uses an SVG-based renderer with coordinate projection. Each point is rendered as a circle with latitude mapped to the Y axis and longitude to the X axis. The component displays point count and center coordinates.
### ErrorToast
A notification component that displays errors from the `useUIStore` error queue. Each error can be individually dismissed. Errors are rendered at the bottom of the viewport.
## Cell System
The notebook is organized as a collection of cells, each with a type, title, and configuration:
```typescript
interface Cell {
    id: string;
    type: CellType;  // 'table' | 'chart' | 'schema' | 'map'
    title: string;
    config: Record<string, any>;
    state_id?: string;
    collapsed?: boolean;
}
```
Cell operations:

- **Add**: Header buttons create new cells with default configuration.
- **Remove**: Each cell has a close button that removes it from the notebook.
- **Collapse/Expand**: Each cell can be collapsed to its header, hiding the content.
- **Active selection**: Clicking a cell sets it as active (highlighted with a brand ring).

## Drag and Drop

The project includes `@dnd-kit/core` and `@dnd-kit/sortable` as dependencies for future drag-and-drop cell reordering. The `moveCell` action in the notebook store is prepared for this functionality, accepting source and destination indices to reorder the cell array.

## ECharts Rendering

Chart rendering uses `echarts-for-react`, a React wrapper for the Apache ECharts library. The `ChartCellView` component receives the chart specification from the backend and passes it directly to `ReactEChartsCore`:

```typescript
<ReactEChartsCore
    echarts={echarts}
    option={chartSpec}
    style={{ height: 400, width: '100%' }}
    notMerge={true}
    lazyUpdate={true}
/>
```

- **notMerge**: Ensures each update replaces the entire chart rather than merging with previous options.
- **lazyUpdate**: Defers rendering to the next animation frame for better performance.
- The chart height is fixed at 400px, with full-width responsive behavior.

## Map Rendering

The `MapCanvas` component currently uses an SVG-based renderer rather than Leaflet. This decision was made to reduce the initial bundle size and avoid Leaflet's CSS dependency. The SVG renderer:

1. Computes the projection by mapping latitude/longitude bounds to a 400x400 viewport.
2. Renders grid lines at regular intervals.
3. Plots each point as an SVG circle with brand-colored fill and stroke.
4. Displays coordinate bounds as text labels.

A future migration to `react-leaflet` is planned for interactive map features (zoom, pan, marker popups). The `leaflet` and `react-leaflet` packages are already included as dependencies.

	# Frontend Architecture

	This document describes the frontend architecture of the Visual Notebook, covering the React + TypeScript + Vite setup, Tailwind CSS configuration, Zustand state management, the API client layer, component hierarchy, the cell system, chart rendering, and map visualization.

	## React + TypeScript + Vite Setup

	The frontend is built as a single-page application using React 18 with TypeScript and Vite 5. The project lives under the `frontend/` directory.

	### Vite Configuration

	The Vite configuration in `vite.config.ts` defines two key features:

	Development proxy: During local development, API requests to `/api` and `/health` are proxied to the backend at `http://localhost:8000`, avoiding CORS issues:

	```typescript
	server: {
	proxy: {
	'/api': { target: 'http://localhost:8000', changeOrigin: true },
	'/health': { target: 'http://localhost:8000', changeOrigin: true },
	}
	}
	```

	Production build optimization: The Rollup output is manually chunked to separate large dependencies:

	```typescript
	manualChunks: {
	echarts: ['echarts', 'echarts-for-react'],
	react: ['react', 'react-dom'],
	vendor: ['axios', 'zustand'],
	}
	```

	This separation ensures that ECharts (the largest dependency) and React are loaded as separate cacheable chunks, improving initial page load performance and enabling better browser caching on subsequent visits.

	### TypeScript Configuration

	TypeScript is configured with strict mode. The `api/types.ts` file mirrors the Pydantic response schemas from the backend, ensuring compile-time type safety for API responses. This contract between frontend and backend types catches breaking changes at build time.

	## Tailwind CSS Configuration

	The project uses Tailwind CSS 3 with a custom brand color theme defined in `tailwind.config.js`:

	```javascript
	colors: {
	brand: {
	50: '#eef2ff',
	100: '#e0e7ff',
	200: '#c7d2fe',
	300: '#a5b4fc',
	400: '#818cf8',
	500: '#6366f1',
	600: '#4f46e5',
	700: '#4338ca',
	800: '#3730a3',
	900: '#312e81',
	}
	}
	```

	This indigo-based palette is used throughout the application for primary buttons, active states, focus rings, and brand accents. The 10-shade scale provides sufficient contrast for accessibility while maintaining visual consistency.

	PostCSS with Autoprefixer processes the CSS for cross-browser compatibility. The global stylesheet in `src/styles/index.css` imports the Tailwind base, components, and utilities layers.

	## Zustand Stores

	The frontend uses four independent Zustand stores, each managing a distinct domain of application state.

	### useSessionStore

	Manages session and dataset metadata:

	\| Field \| Type \| Description \|
	\|-------\|------\|-------------\|
	\| `sessionId` \| `string \\| null` \| Backend session ID \|
	\| `datasetId` \| `string \\| null` \| Uploaded dataset ID \|
	\| `tableName` \| `string \\| null` \| DuckDB table name \|
	\| `rowCount` \| `number` \| Total row count \|
	\| `columns` \| `ColumnInfo[]` \| Column schema with statistics \|
	\| `isConnected` \| `boolean` \| Backend connectivity flag \|
	\| `isLoaded` \| `boolean` \| Whether a dataset is loaded \|

	Actions: `setSession`, `setSchema`, `clearSession`, `setConnected`. The `isLoaded` flag is the primary gate for conditional rendering -- components that require a dataset check this flag before rendering.

	### useNotebookStore

	Manages the notebook cell layout:

	\| Field \| Type \| Description \|
	\|-------\|------\|-------------\|
	\| `cells` \| `Cell[]` \| Ordered list of notebook cells \|
	\| `activeCellId` \| `string \\| null` \| Currently selected cell \|

	Actions: `addCell(type, title?, config?)`, `removeCell(id)`, `updateCell(id, updates)`, `setActiveCell(id)`, `moveCell(fromIndex, toIndex)`, `toggleCollapse(id)`.

	Each cell has a unique ID generated from a counter and timestamp, ensuring stability across re-renders. The `addCell` method returns the new cell ID for programmatic selection.

	### useUIStore

	Manages UI panel toggles and error state:

	\| Field \| Type \| Description \|
	\|-------\|------\|-------------\|
	\| `selectedSlot` \| `'table' \\| 'chart' \\| 'schema' \\| 'map' \\| null` \| Active cell type slot \|
	\| `showUploadDialog` \| `boolean` \| Upload dialog visibility \|
	\| `showManipulationPanel` \| `boolean` \| Transform panel visibility \|
	\| `showVisualizationPanel` \| `boolean` \| Visualization panel visibility \|
	\| `errors` \| `string[]` \| Queue of error messages \|

	Actions: `setSelectedSlot`, `toggleUploadDialog`, `toggleManipulationPanel`, `toggleVisualizationPanel`, `addError`, `clearErrors`, `removeError`.

	The error queue allows multiple errors to be displayed simultaneously, with individual dismissal.

	### useHistoryStore

	Manages the undo/redo state synchronized with the backend:

	\| Field \| Type \| Description \|
	\|-------\|------\|-------------\|
	\| `nodes` \| `HistoryNode[]` \| Full history tree nodes \|
	\| `currentId` \| `string \\| null` \| Current state ID \|
	\| `order` \| `string[]` \| Linear state order \|
	\| `canUndo` \| `boolean` \| Whether undo is available \|
	\| `canRedo` \| `boolean` \| Whether redo is available \|

	Actions: `setHistory`, `updateCurrent`, `setUndoRedo`, `reset`. The `canUndo` and `canRedo` flags are computed from the position of `currentId` in the `order` array.

	## API Client Layer

	### Axios Instance

	The API client in `api/client.ts` creates a configured Axios instance:

	```typescript
	const api = axios.create({
	baseURL: import.meta.env.VITE_API_URL \|\| '',
	withCredentials: true,
	headers: { 'Content-Type': 'application/json' },
	});
	```

	Key configuration:

	- baseURL: Reads from `VITE_API_URL` environment variable, defaulting to the same origin (for production where the backend serves the frontend). In development, the Vite proxy handles routing.
	- withCredentials: Enables sending cookies (specifically the `session_id` cookie) with every request.
	- Response interceptor: Extracts the `detail` field from error responses, providing a consistent error message format.

	### API Helper Functions

	The client module exports typed async functions for each API endpoint:

	\| Function \| Method \| Endpoint \|
	\|----------\|--------\|----------\|
	\| `uploadCSV(file)` \| POST \| `/api/upload/csv` \|
	\| `getSchema()` \| GET \| `/api/schema` \|
	\| `getRows(offset, limit)` \| GET \| `/api/rows` \|
	\| `listOperations()` \| GET \| `/api/ops` \|
	\| `applyOperation(opKey, params)` \| POST \| `/api/ops/{opKey}` \|
	\| `undo()` \| POST \| `/api/history/undo` \|
	\| `redo()` \| POST \| `/api/history/redo` \|
	\| `getHistory()` \| GET \| `/api/history` \|
	\| `createChart(req)` \| POST \| `/api/viz/chart` \|
	\| `exportCSV()` \| GET \| `/api/export/csv` \|
	\| `createCheckpoint()` \| POST \| `/api/checkpoint` \|
	\| `checkHealth()` \| GET \| `/health` \|

	The CSV upload uses `multipart/form-data` content type. The CSV export uses `responseType: 'blob'` for binary file download.

	### React Hooks

	The `api/hooks.ts` module wraps the API functions in React hooks with loading and error state management. Each hook follows the same pattern:

	```typescript
	export function useUpload() {
	const [loading, setLoading] = useState(false);
	const [error, setError] = useState<string \| null>(null);
	const upload = useCallback(async (file: File): Promise<UploadResponse \| null> => {
	setLoading(true);
	setError(null);
	try {
	return await uploadCSV(file);
	} catch (err: any) {
	setError(err.message);
	return null;
	} finally {
	setLoading(false);
	}
	}, []);
	return { upload, loading, error };
	}
	```

	Available hooks: `useUpload`, `useSchema`, `useRows`, `useApplyOp`, `useHistory`, `useChart`, `useExport`.

	The `useExport` hook handles file download by creating a temporary anchor element and triggering a click event.

	## Component Hierarchy

	The component tree is structured as follows:

	```
	App (ErrorBoundary)
	+-- Header
	+-- NotebookPage
	\| +-- UploadPanel (modal, shown when no dataset or "New File" clicked)
	\| +-- TableCellView (for each cell of type "table")
	\| +-- ChartCellView (for each cell of type "chart")
	\| +-- SchemaView (for each cell of type "schema")
	\| +-- MapCanvas (for each cell of type "map")
	\| +-- ManipulationPanel (side panel, toggled from header)
	\| +-- VisualizationPanel (side panel, toggled from header)
	+-- ErrorToast
	```

	### Header

	The header component displays the application title, dataset metadata (table name and row count), undo/redo buttons, cell creation buttons, transform/visualize panel toggles, and the CSV export button. It reads from `useSessionStore`, `useHistoryStore`, and `useUIStore`.

	### NotebookPage

	The main content area that conditionally renders the landing screen (when no dataset is loaded) or the notebook cell list. The landing screen displays a centered call-to-action with the upload button. The notebook area renders each cell with a collapsible header showing the cell index, title, type badge, and collapse/remove controls.

	### UploadPanel

	A modal dialog that wraps an HTML file input for CSV upload. Uses the `useUpload` hook to manage the upload lifecycle and update the session store on success.

	### TableCellView

	Renders paginated table data using the `useRows` hook. Displays column headers and row data in a scrollable grid.

	### ChartCellView

	Renders ECharts visualizations using the `echarts-for-react` library. When a chart configuration is provided (chartType and params), it calls the `useChart` hook to compute the chart spec and renders it with `ReactEChartsCore`. The component supports dynamic chart type changes and displays a loading spinner during computation.

	### SchemaView

	Displays the column schema as a table with column name, type, statistics (null count, unique count, min, max, mean, std, quartiles). Uses the `useSchema` hook.

	### ManipulationPanel

	A side panel that provides forms for applying data operations (filter, sort, select, rename, groupby, fill_nulls, clip). Uses the `useApplyOp` hook and reads available operations from `listOperations()`.

	### VisualizationPanel

	A side panel for configuring chart visualizations. Allows the user to select a chart type, choose columns, and set parameters. Creates chart cells in the notebook when a visualization is configured.

	### MapCanvas

	Renders geographic point data. Currently uses an SVG-based renderer with coordinate projection. Each point is rendered as a circle with latitude mapped to the Y axis and longitude to the X axis. The component displays point count and center coordinates.

	### ErrorToast

	A notification component that displays errors from the `useUIStore` error queue. Each error can be individually dismissed. Errors are rendered at the bottom of the viewport.

	## Cell System

	The notebook is organized as a collection of cells, each with a type, title, and configuration:

	```typescript
	interface Cell {
	id: string;
	type: CellType; // 'table' \| 'chart' \| 'schema' \| 'map'
	title: string;
	config: Record<string, any>;
	state_id?: string;
	collapsed?: boolean;
	}
	```

	Cell operations:

	- Add: Header buttons create new cells with default configuration.
	- Remove: Each cell has a close button that removes it from the notebook.
	- Collapse/Expand: Each cell can be collapsed to its header, hiding the content.
	- Active selection: Clicking a cell sets it as active (highlighted with a brand ring).

	## Drag and Drop

	The project includes `@dnd-kit/core` and `@dnd-kit/sortable` as dependencies for future drag-and-drop cell reordering. The `moveCell` action in the notebook store is prepared for this functionality, accepting source and destination indices to reorder the cell array.

	## ECharts Rendering

	Chart rendering uses `echarts-for-react`, a React wrapper for the Apache ECharts library. The `ChartCellView` component receives the chart specification from the backend and passes it directly to `ReactEChartsCore`:

	```typescript
	<ReactEChartsCore
	echarts={echarts}
	option={chartSpec}
	style={{ height: 400, width: '100%' }}
	notMerge={true}
	lazyUpdate={true}
	/>
	```

	- notMerge: Ensures each update replaces the entire chart rather than merging with previous options.
	- lazyUpdate: Defers rendering to the next animation frame for better performance.
	- The chart height is fixed at 400px, with full-width responsive behavior.

	## Map Rendering

	The `MapCanvas` component currently uses an SVG-based renderer rather than Leaflet. This decision was made to reduce the initial bundle size and avoid Leaflet's CSS dependency. The SVG renderer:

	1. Computes the projection by mapping latitude/longitude bounds to a 400x400 viewport.
	2. Renders grid lines at regular intervals.
	3. Plots each point as an SVG circle with brand-colored fill and stroke.
	4. Displays coordinate bounds as text labels.

	A future migration to `react-leaflet` is planned for interactive map features (zoom, pan, marker popups). The `leaflet` and `react-leaflet` packages are already included as dependencies.