Spaces:

Sid-WC121
/

visual-notebook

Sleeping

App Files Files Community

visual-notebook / docs /development.md

Sid-WC121

bug fix1

018ab56 6 months ago

preview code

Raw History Blame Contribute Delete

17.5 kB

# Development Guide

This document provides a comprehensive guide for setting up a local development environment, understanding the project structure, and extending the Visual Notebook with new operations, chart types, and frontend components.

## Prerequisites

Before setting up the development environment, ensure you have the following tools installed:

| Tool | Version | Purpose |
|------|---------|---------|
| Python | 3.11+ | Backend runtime |
| Node.js | 20+ | Frontend build and development |
| npm | 10+ | Frontend package management |
| uv | Latest | Python package management (replaces pip/venv) |
| Git | 2.40+ | Version control |

### Installing uv

```bash
# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
```

## Local Development Setup

### Backend Setup

1. **Navigate to the project root**:

```bash
cd /path/to/va_project
```

2. **Create a virtual environment and install dependencies**:

```bash
uv sync
```

This command reads `pyproject.toml`, resolves dependencies, creates a virtual environment in `.venv/`, and installs all packages including development dependencies (pytest, httpx).

3. **Run the backend server**:

```bash
uv run uvicorn visual_notebook.main:app --reload --host 0.0.0.0 --port 8000
```

The `--reload` flag enables automatic server restart when source files change.

4. **Verify the server is running**:

```bash
curl http://localhost:8000/health
# Expected: {"status":"ok","version":"0.1.0"}
```

5. **Access the API documentation**:

Open `http://localhost:8000/docs` in a browser to view the Swagger UI generated by FastAPI.

### Frontend Setup

1. **Navigate to the frontend directory**:

```bash
cd frontend
```

2. **Install dependencies**:

```bash
npm install
```

3. **Start the development server**:

```bash
npm run dev
```

The Vite dev server starts at `http://localhost:5173` with hot module replacement (HMR). API requests to `/api` and `/health` are automatically proxied to the backend at `http://localhost:8000` as configured in `vite.config.ts`.

4. **Build for production**:

```bash
npm run build
```

The production build outputs to `frontend/dist/` with code-split chunks for ECharts, React, and vendor libraries.

### Running Both Together

For the best development experience, run the backend and frontend in separate terminal sessions:

- Terminal 1: `uv run uvicorn visual_notebook.main:app --reload --port 8000`
- Terminal 2: `cd frontend && npm run dev`

Visit `http://localhost:5173` for the full application with live reloading on both frontend and backend changes.

## Project Structure Overview

```
va_project/
    pyproject.toml                    # Root workspace config
    Dockerfile                        # Combined build (frontend + backend)
    Dockerfile.backend                # Backend-only build
    docker-compose.yml                # Development Docker Compose
    datasets/                         # Uploaded CSV files
    checkpoints/                      # Session checkpoint files
    static/                           # Built frontend (production)
    backend/
        pyproject.toml                # Backend dependencies
        visual_notebook/
            main.py                   # FastAPI application entry
            config.py                 # Settings from environment
            session.py                # Session and SessionStore
            api/
                routes.py             # API endpoint definitions
                schemas.py            # Pydantic request/response models
                deps.py               # Dependency injection
            domain/
                dataframe.py          # DataFrameRef wrapper
                history.py            # History tree (Memento)
                operations.py         # Operation registry
                checkpoints.py        # Checkpoint persistence
                validators.py         # Validation helpers
            data/
                csv_loader.py         # CSV loading and export
                schema.py             # Schema inference
                types.py              # Column type classification
            viz/
                _common.py            # Shared chart utilities
                histogram.py          # Histogram computation
                scatter.py            # Scatter plot computation
                bar_topn.py           # Top-N bar chart
                timeline.py           # Timeline/line chart
                heatmap.py            # Heatmap computation
                map.py                # Map computation
            tests/
                test_history.py       # History tests
                test_operations.py    # Operations tests
    frontend/
        package.json                  # Frontend dependencies
        vite.config.ts                # Vite configuration
        tailwind.config.js            # Tailwind CSS theme
        tsconfig.json                 # TypeScript configuration
        index.html                    # HTML entry point
        src/
            main.tsx                  # React entry point
            App.tsx                   # Root component with ErrorBoundary
            api/
                client.ts             # Axios instance and API functions
                hooks.ts              # React hooks for data fetching
                types.ts              # TypeScript type definitions
            store/
                session.ts            # Session state store
                notebook.ts           # Notebook cell store
                ui.ts                 # UI toggle store
                history.ts            # Undo/redo state store
            components/
                Header.tsx            # Top navigation bar
                NotebookPage.tsx      # Main notebook layout
                UploadPanel.tsx       # CSV upload dialog
                TableCellView.tsx     # Table data display
                ChartCellView.tsx     # ECharts rendering
                SchemaView.tsx        # Column schema display
                ManipulationPanel.tsx  # Operation controls
                VisualizationPanel.tsx # Chart configuration
                TablePreview.tsx      # Table row preview
                MapCanvas.tsx         # Map point visualization
                ErrorToast.tsx        # Error notification
            lib/
                chartTypes.ts         # Chart type definitions
                csv.ts                # CSV utility functions
                format.ts             # Number/formatting helpers
            styles/
                index.css             # Global styles and Tailwind
    docs/                            # Documentation (this directory)
```

## Adding a New Operation

Follow these steps to add a new data operation (e.g., `deduplicate`) to the system:

### Step 1: Define the OperationDef

Add a new entry to the `OPERATIONS` dictionary in `backend/visual_notebook/domain/operations.py`:

```python
"deduplicate": OperationDef(
    key="deduplicate",
    label="Remove Duplicates",
    description="Remove duplicate rows based on specified columns",
    param_schema={
        "type": "object",
        "required": ["columns"],
        "properties": {
            "columns": {"type": "array", "items": {"type": "string"}},
            "keep": {"type": "string", "enum": ["first", "last", "none"], "default": "first"},
        },
    },
    validate=_validate_deduplicate,
    apply=_apply_deduplicate,
),
```

### Step 2: Implement the Validate Function

```python
def _validate_deduplicate(params: dict, schema: list[dict]) -> dict:
    columns = params.get("columns", [])
    if not columns:
        raise ValueError("'columns' list is required")
    params.setdefault("keep", "first")
    return params
```

The validate function should check that all required parameters are present and have valid values. It receives the raw user params and the current table schema.

### Step 3: Implement the Apply Function

Provide both DuckDB and Polars implementations:

```python
def _apply_deduplicate(ref: DataFrameRef, params: dict) -> DataFrameRef:
    columns = params["columns"]
    keep = params.get("keep", "first")
    if ref.backend == "duckdb" and ref.name and ref.con:
        new_name = f"_dedup_{uuid.uuid4().hex[:6]}"
        cols = ", ".join(f'"{c}"' for c in columns)
        if keep == "first":
            sql = f'''
                SELECT DISTINCT ON ({cols}) *
                FROM "{ref.name}"
                ORDER BY {cols}
            '''
        elif keep == "last":
            sql = f'''
                SELECT DISTINCT ON ({cols}) *
                FROM "{ref.name}"
                ORDER BY {cols} DESC
            '''
        else:
            sql = f'''
                SELECT * FROM "{ref.name}"
                GROUP BY {cols}
                HAVING COUNT(*) = 1
            '''
        ref.con.execute(f'CREATE OR REPLACE TABLE "{new_name}" AS {sql}')
        return DataFrameRef(backend="duckdb", name=new_name, con=ref.con, source_path=ref.source_path)
    elif ref.backend == "polars" and ref.lf is not None:
        lf = ref.lf.unique(subset=columns, keep=keep)
        return DataFrameRef(backend="polars", lf=lf, source_path=ref.source_path)
    raise ValueError("Invalid DataFrameRef")
```

### Step 4: Add Tests

Add test cases to `backend/visual_notebook/tests/test_operations.py`:

```python
def test_deduplicate_validation():
    op = OPERATIONS["deduplicate"]
    with pytest.raises(ValueError):
        op.validate({}, [])
    result = op.validate({"columns": ["a", "b"]}, [])
    assert result["keep"] == "first"
def test_deduplicate_in_registry():
    assert "deduplicate" in OPERATIONS
```

### Step 5: Verify

1. Run the tests: `uv run pytest backend/visual_notebook/tests/ -v`
2. Start the server and verify the operation appears in `GET /api/ops`.
3. Test the operation via the Swagger UI at `http://localhost:8000/docs`.

## Adding a New Chart Type

Follow these steps to add a new chart type (e.g., `pie`) to the system:

### Step 1: Create the Computation Module

Create a new file `backend/visual_notebook/viz/pie.py`:

```python
"""Pie chart generation."""
from __future__ import annotations
from typing import Any
import duckdb
from visual_notebook.viz._common import PALETTE, base_chart, title, tooltip_style, palette_colors
def compute_pie(
    con: duckdb.DuckDBPyConnection,
    table_name: str,
    category_col: str,
    value_col: str | None = None,
    n: int = 10,
) -> dict[str, Any]:
    """Compute pie chart data and return ECharts spec."""
    if value_col:
        result = con.execute(f'''
            SELECT "{category_col}" as cat, SUM("{value_col}") as val
            FROM "{table_name}"
            WHERE "{category_col}" IS NOT NULL
            GROUP BY cat
            ORDER BY val DESC
            LIMIT {n}
        ''').fetchall()
    else:
        result = con.execute(f'''
            SELECT "{category_col}" as cat, COUNT(*) as val
            FROM "{table_name}"
            WHERE "{category_col}" IS NOT NULL
            GROUP BY cat
            ORDER BY val DESC
            LIMIT {n}
        ''').fetchall()
    data = [{"name": str(r[0]), "value": float(r[1])} for r in result]
    colors = palette_colors(len(data))
    chart = base_chart()
    chart["title"] = title(f"Distribution of {category_col}")
    chart["tooltip"] = {**tooltip_style(), "trigger": "item", "formatter": "{b}: {c} ({d}%)"}
    chart["legend"] = {"orient": "vertical", "left": "left", "top": "middle"}
    chart["series"] = [{
        "type": "pie",
        "radius": ["40%", "70%"],
        "data": data,
        "emphasis": {"itemStyle": {"shadowBlur": 10, "shadowOffsetX": 0, "shadowColor": "rgba(0,0,0,0.5)"}},
    }]
    return chart
```

### Step 2: Register in the Route Handler

Add the import and handler in `backend/visual_notebook/api/routes.py`:

```python
from visual_notebook.viz.pie import compute_pie
# In the create_chart endpoint:
elif req.chart_type == "pie":
    spec = compute_pie(con, table, params["category_col"], params.get("value_col"), params.get("n", 10))
```

### Step 3: Add Frontend Support

Update `frontend/src/lib/chartTypes.ts` to include the new chart type, and add a configuration form in the `VisualizationPanel` component.

## Adding a New Frontend Component

### Step 1: Create the Component File

Create a new file in `frontend/src/components/`:

```typescript
// StatsPanel.tsx
import React from 'react';
import { useSessionStore } from '../store/session';
export function StatsPanel() {
    const { columns, rowCount } = useSessionStore();
    return (
        <div className="bg-white rounded-lg shadow p-4">
            <h3 className="text-sm font-semibold text-gray-700 mb-3">Dataset Statistics</h3>
            <div className="space-y-2">
                <div className="flex justify-between text-sm">
                    <span className="text-gray-500">Total Rows</span>
                    <span className="font-mono">{rowCount.toLocaleString()}</span>
                </div>
                <div className="flex justify-between text-sm">
                    <span className="text-gray-500">Total Columns</span>
                    <span className="font-mono">{columns.length}</span>
                </div>
            </div>
        </div>
    );
}
```

### Step 2: Use Tailwind Classes

Follow the project's Tailwind conventions:

- Use `brand-*` colors for primary actions and accents.
- Use `gray-*` colors for neutral text, borders, and backgrounds.
- Use `text-sm` for body text, `text-xs` for secondary labels.
- Use `rounded-lg` for card corners, `shadow` for elevation.
- Use `flex` and `grid` for layout, with `gap-*` for spacing.

### Step 3: Integrate into the Application

Import and render the component in the appropriate parent:

```typescript
// In NotebookPage.tsx
import { StatsPanel } from './StatsPanel';
// Render conditionally
{showStatsPanel && <StatsPanel />}
```

### Step 4: Add a UI Toggle

If the component should be togglable, add a flag to `useUIStore`:

```typescript
showStatsPanel: boolean;
toggleStatsPanel: () => void;
```

## Testing

### Backend Tests

The backend uses pytest with test files in `backend/visual_notebook/tests/`.

**Run all tests**:

```bash
uv run pytest backend/visual_notebook/tests/ -v
```

**Run a specific test file**:

```bash
uv run pytest backend/visual_notebook/tests/test_operations.py -v
```

**Run with coverage** (if pytest-cov is installed):

```bash
uv run pytest backend/visual_notebook/tests/ -v --cov=visual_notebook
```

**Test structure**:

- `test_history.py`: Tests for commit, undo, redo, branch, and tree serialization.
- `test_operations.py`: Tests for the operations registry, parameter validation, and default values.

### Frontend Testing

The frontend currently does not have automated tests configured. For future implementation, consider:

- **Vitest**: Fast unit testing framework compatible with Vite.
- **React Testing Library**: Component testing with DOM assertions.
- **Playwright**: End-to-end testing for API integration.

## Code Style and Conventions

### Python (Backend)

- **Line length**: 100 characters maximum.
- **Type hints**: Use type hints on all function signatures. Import `from __future__ import annotations` for modern type syntax.
- **Docstrings**: Use triple-quoted docstrings for all public functions and classes.
- **Naming**: `snake_case` for functions and variables, `PascalCase` for classes.
- **Imports**: Standard library first, then third-party, then local. Use absolute imports within the package.
- **f-strings**: Use f-strings for SQL query construction (the SQL injection risk is mitigated by server-generated table names and column names from the schema).
- **Error handling**: Raise `ValueError` in validators, `HTTPException` in route handlers.

### TypeScript (Frontend)

- **Strict mode**: TypeScript strict mode is enabled. All variables must have explicit or inferred types.
- **Functional components**: Use function declarations with explicit return types.
- **Naming**: `camelCase` for variables and functions, `PascalCase` for components and types.
- **Imports**: Use relative paths for local modules, package names for third-party libraries.
- **Hooks**: Custom hooks in `api/hooks.ts` follow the `use[X]` naming pattern and return `{ action, loading, error }`.
- **CSS**: Use Tailwind utility classes exclusively. Avoid inline styles except for dynamic values (chart dimensions, SVG coordinates).

## Git Workflow

### Branch Naming

- `feature/[description]`: New features
- `fix/[description]`: Bug fixes
- `docs/[description]`: Documentation changes
- `refactor/[description]`: Code refactoring

### Commit Messages

Follow conventional commit format:

```
type(scope): description
[optional body]
```

Types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`

Examples:
- `feat(operations): add deduplicate operation`
- `fix(history): correct undo pointer after branch`
- `docs(api): update endpoint descriptions`

### Pull Request Process

1. Create a feature branch from `main`.
2. Implement changes with tests.
3. Run the full test suite: `uv run pytest`.
4. Ensure the frontend builds: `cd frontend && npm run build`.
5. Open a pull request with a description of the changes and any manual testing performed.
6. Request review from a maintainer.
7. Address review feedback and re-run tests.
8. Merge when approved and all checks pass.

	# Development Guide

	This document provides a comprehensive guide for setting up a local development environment, understanding the project structure, and extending the Visual Notebook with new operations, chart types, and frontend components.

	## Prerequisites

	Before setting up the development environment, ensure you have the following tools installed:

	\| Tool \| Version \| Purpose \|
	\|------\|---------\|---------\|
	\| Python \| 3.11+ \| Backend runtime \|
	\| Node.js \| 20+ \| Frontend build and development \|
	\| npm \| 10+ \| Frontend package management \|
	\| uv \| Latest \| Python package management (replaces pip/venv) \|
	\| Git \| 2.40+ \| Version control \|

	### Installing uv

	```bash
	# macOS / Linux
	curl -LsSf https://astral.sh/uv/install.sh \| sh

	# Windows
	powershell -c "irm https://astral.sh/uv/install.ps1 \| iex"
	```

	## Local Development Setup

	### Backend Setup

	1. Navigate to the project root:

	```bash
	cd /path/to/va_project
	```

	2. Create a virtual environment and install dependencies:

	```bash
	uv sync
	```

	This command reads `pyproject.toml`, resolves dependencies, creates a virtual environment in `.venv/`, and installs all packages including development dependencies (pytest, httpx).

	3. Run the backend server:

	```bash
	uv run uvicorn visual_notebook.main:app --reload --host 0.0.0.0 --port 8000
	```

	The `--reload` flag enables automatic server restart when source files change.

	4. Verify the server is running:

	```bash
	curl http://localhost:8000/health
	# Expected: {"status":"ok","version":"0.1.0"}
	```

	5. Access the API documentation:

	Open `http://localhost:8000/docs` in a browser to view the Swagger UI generated by FastAPI.

	### Frontend Setup

	1. Navigate to the frontend directory:

	```bash
	cd frontend
	```

	2. Install dependencies:

	```bash
	npm install
	```

	3. Start the development server:

	```bash
	npm run dev
	```

	The Vite dev server starts at `http://localhost:5173` with hot module replacement (HMR). API requests to `/api` and `/health` are automatically proxied to the backend at `http://localhost:8000` as configured in `vite.config.ts`.

	4. Build for production:

	```bash
	npm run build
	```

	The production build outputs to `frontend/dist/` with code-split chunks for ECharts, React, and vendor libraries.

	### Running Both Together

	For the best development experience, run the backend and frontend in separate terminal sessions:

	- Terminal 1: `uv run uvicorn visual_notebook.main:app --reload --port 8000`
	- Terminal 2: `cd frontend && npm run dev`

	Visit `http://localhost:5173` for the full application with live reloading on both frontend and backend changes.

	## Project Structure Overview

	```
	va_project/
	pyproject.toml # Root workspace config
	Dockerfile # Combined build (frontend + backend)
	Dockerfile.backend # Backend-only build
	docker-compose.yml # Development Docker Compose
	datasets/ # Uploaded CSV files
	checkpoints/ # Session checkpoint files
	static/ # Built frontend (production)
	backend/
	pyproject.toml # Backend dependencies
	visual_notebook/
	main.py # FastAPI application entry
	config.py # Settings from environment
	session.py # Session and SessionStore
	api/
	routes.py # API endpoint definitions
	schemas.py # Pydantic request/response models
	deps.py # Dependency injection
	domain/
	dataframe.py # DataFrameRef wrapper
	history.py # History tree (Memento)
	operations.py # Operation registry
	checkpoints.py # Checkpoint persistence
	validators.py # Validation helpers
	data/
	csv_loader.py # CSV loading and export
	schema.py # Schema inference
	types.py # Column type classification
	viz/
	_common.py # Shared chart utilities
	histogram.py # Histogram computation
	scatter.py # Scatter plot computation
	bar_topn.py # Top-N bar chart
	timeline.py # Timeline/line chart
	heatmap.py # Heatmap computation
	map.py # Map computation
	tests/
	test_history.py # History tests
	test_operations.py # Operations tests
	frontend/
	package.json # Frontend dependencies
	vite.config.ts # Vite configuration
	tailwind.config.js # Tailwind CSS theme
	tsconfig.json # TypeScript configuration
	index.html # HTML entry point
	src/
	main.tsx # React entry point
	App.tsx # Root component with ErrorBoundary
	api/
	client.ts # Axios instance and API functions
	hooks.ts # React hooks for data fetching
	types.ts # TypeScript type definitions
	store/
	session.ts # Session state store
	notebook.ts # Notebook cell store
	ui.ts # UI toggle store
	history.ts # Undo/redo state store
	components/
	Header.tsx # Top navigation bar
	NotebookPage.tsx # Main notebook layout
	UploadPanel.tsx # CSV upload dialog
	TableCellView.tsx # Table data display
	ChartCellView.tsx # ECharts rendering
	SchemaView.tsx # Column schema display
	ManipulationPanel.tsx # Operation controls
	VisualizationPanel.tsx # Chart configuration
	TablePreview.tsx # Table row preview
	MapCanvas.tsx # Map point visualization
	ErrorToast.tsx # Error notification
	lib/
	chartTypes.ts # Chart type definitions
	csv.ts # CSV utility functions
	format.ts # Number/formatting helpers
	styles/
	index.css # Global styles and Tailwind
	docs/ # Documentation (this directory)
	```

	## Adding a New Operation

	Follow these steps to add a new data operation (e.g., `deduplicate`) to the system:

	### Step 1: Define the OperationDef

	Add a new entry to the `OPERATIONS` dictionary in `backend/visual_notebook/domain/operations.py`:

	```python
	"deduplicate": OperationDef(
	key="deduplicate",
	label="Remove Duplicates",
	description="Remove duplicate rows based on specified columns",
	param_schema={
	"type": "object",
	"required": ["columns"],
	"properties": {
	"columns": {"type": "array", "items": {"type": "string"}},
	"keep": {"type": "string", "enum": ["first", "last", "none"], "default": "first"},
	},
	},
	validate=_validate_deduplicate,
	apply=_apply_deduplicate,
	),
	```

	### Step 2: Implement the Validate Function

	```python
	def _validate_deduplicate(params: dict, schema: list[dict]) -> dict:
	columns = params.get("columns", [])
	if not columns:
	raise ValueError("'columns' list is required")
	params.setdefault("keep", "first")
	return params
	```

	The validate function should check that all required parameters are present and have valid values. It receives the raw user params and the current table schema.

	### Step 3: Implement the Apply Function

	Provide both DuckDB and Polars implementations:

	```python
	def _apply_deduplicate(ref: DataFrameRef, params: dict) -> DataFrameRef:
	columns = params["columns"]
	keep = params.get("keep", "first")

	if ref.backend == "duckdb" and ref.name and ref.con:
	new_name = f"_dedup_{uuid.uuid4().hex[:6]}"
	cols = ", ".join(f'"{c}"' for c in columns)

	if keep == "first":
	sql = f'''
	SELECT DISTINCT ON ({cols}) *
	FROM "{ref.name}"
	ORDER BY {cols}
	'''
	elif keep == "last":
	sql = f'''
	SELECT DISTINCT ON ({cols}) *
	FROM "{ref.name}"
	ORDER BY {cols} DESC
	'''
	else:
	sql = f'''
	SELECT * FROM "{ref.name}"
	GROUP BY {cols}
	HAVING COUNT(*) = 1
	'''

	ref.con.execute(f'CREATE OR REPLACE TABLE "{new_name}" AS {sql}')
	return DataFrameRef(backend="duckdb", name=new_name, con=ref.con, source_path=ref.source_path)

	elif ref.backend == "polars" and ref.lf is not None:
	lf = ref.lf.unique(subset=columns, keep=keep)
	return DataFrameRef(backend="polars", lf=lf, source_path=ref.source_path)

	raise ValueError("Invalid DataFrameRef")
	```

	### Step 4: Add Tests

	Add test cases to `backend/visual_notebook/tests/test_operations.py`:

	```python
	def test_deduplicate_validation():
	op = OPERATIONS["deduplicate"]
	with pytest.raises(ValueError):
	op.validate({}, [])

	result = op.validate({"columns": ["a", "b"]}, [])
	assert result["keep"] == "first"

	def test_deduplicate_in_registry():
	assert "deduplicate" in OPERATIONS
	```

	### Step 5: Verify

	1. Run the tests: `uv run pytest backend/visual_notebook/tests/ -v`
	2. Start the server and verify the operation appears in `GET /api/ops`.
	3. Test the operation via the Swagger UI at `http://localhost:8000/docs`.

	## Adding a New Chart Type

	Follow these steps to add a new chart type (e.g., `pie`) to the system:

	### Step 1: Create the Computation Module

	Create a new file `backend/visual_notebook/viz/pie.py`:

	```python
	"""Pie chart generation."""
	from __future__ import annotations
	from typing import Any
	import duckdb
	from visual_notebook.viz._common import PALETTE, base_chart, title, tooltip_style, palette_colors


	def compute_pie(
	con: duckdb.DuckDBPyConnection,
	table_name: str,
	category_col: str,
	value_col: str \| None = None,
	n: int = 10,
	) -> dict[str, Any]:
	"""Compute pie chart data and return ECharts spec."""
	if value_col:
	result = con.execute(f'''
	SELECT "{category_col}" as cat, SUM("{value_col}") as val
	FROM "{table_name}"
	WHERE "{category_col}" IS NOT NULL
	GROUP BY cat
	ORDER BY val DESC
	LIMIT {n}
	''').fetchall()
	else:
	result = con.execute(f'''
	SELECT "{category_col}" as cat, COUNT(*) as val
	FROM "{table_name}"
	WHERE "{category_col}" IS NOT NULL
	GROUP BY cat
	ORDER BY val DESC
	LIMIT {n}
	''').fetchall()

	data = [{"name": str(r[0]), "value": float(r[1])} for r in result]
	colors = palette_colors(len(data))

	chart = base_chart()
	chart["title"] = title(f"Distribution of {category_col}")
	chart["tooltip"] = {**tooltip_style(), "trigger": "item", "formatter": "{b}: {c} ({d}%)"}
	chart["legend"] = {"orient": "vertical", "left": "left", "top": "middle"}
	chart["series"] = [{
	"type": "pie",
	"radius": ["40%", "70%"],
	"data": data,
	"emphasis": {"itemStyle": {"shadowBlur": 10, "shadowOffsetX": 0, "shadowColor": "rgba(0,0,0,0.5)"}},
	}]

	return chart
	```

	### Step 2: Register in the Route Handler

	Add the import and handler in `backend/visual_notebook/api/routes.py`:

	```python
	from visual_notebook.viz.pie import compute_pie

	# In the create_chart endpoint:
	elif req.chart_type == "pie":
	spec = compute_pie(con, table, params["category_col"], params.get("value_col"), params.get("n", 10))
	```

	### Step 3: Add Frontend Support

	Update `frontend/src/lib/chartTypes.ts` to include the new chart type, and add a configuration form in the `VisualizationPanel` component.

	## Adding a New Frontend Component

	### Step 1: Create the Component File

	Create a new file in `frontend/src/components/`:

	```typescript
	// StatsPanel.tsx
	import React from 'react';
	import { useSessionStore } from '../store/session';

	export function StatsPanel() {
	const { columns, rowCount } = useSessionStore();

	return (
	<div className="bg-white rounded-lg shadow p-4">
	<h3 className="text-sm font-semibold text-gray-700 mb-3">Dataset Statistics</h3>
	<div className="space-y-2">
	<div className="flex justify-between text-sm">
	<span className="text-gray-500">Total Rows</span>
	<span className="font-mono">{rowCount.toLocaleString()}</span>
	</div>
	<div className="flex justify-between text-sm">
	<span className="text-gray-500">Total Columns</span>
	<span className="font-mono">{columns.length}</span>
	</div>
	</div>
	</div>
	);
	}
	```

	### Step 2: Use Tailwind Classes

	Follow the project's Tailwind conventions:

	- Use `brand-*` colors for primary actions and accents.
	- Use `gray-*` colors for neutral text, borders, and backgrounds.
	- Use `text-sm` for body text, `text-xs` for secondary labels.
	- Use `rounded-lg` for card corners, `shadow` for elevation.
	- Use `flex` and `grid` for layout, with `gap-*` for spacing.

	### Step 3: Integrate into the Application

	Import and render the component in the appropriate parent:

	```typescript
	// In NotebookPage.tsx
	import { StatsPanel } from './StatsPanel';

	// Render conditionally
	{showStatsPanel && <StatsPanel />}
	```

	### Step 4: Add a UI Toggle

	If the component should be togglable, add a flag to `useUIStore`:

	```typescript
	showStatsPanel: boolean;
	toggleStatsPanel: () => void;
	```

	## Testing

	### Backend Tests

	The backend uses pytest with test files in `backend/visual_notebook/tests/`.

	Run all tests:

	```bash
	uv run pytest backend/visual_notebook/tests/ -v
	```

	Run a specific test file:

	```bash
	uv run pytest backend/visual_notebook/tests/test_operations.py -v
	```

	Run with coverage (if pytest-cov is installed):

	```bash
	uv run pytest backend/visual_notebook/tests/ -v --cov=visual_notebook
	```

	Test structure:

	- `test_history.py`: Tests for commit, undo, redo, branch, and tree serialization.
	- `test_operations.py`: Tests for the operations registry, parameter validation, and default values.

	### Frontend Testing

	The frontend currently does not have automated tests configured. For future implementation, consider:

	- Vitest: Fast unit testing framework compatible with Vite.
	- React Testing Library: Component testing with DOM assertions.
	- Playwright: End-to-end testing for API integration.

	## Code Style and Conventions

	### Python (Backend)

	- Line length: 100 characters maximum.
	- Type hints: Use type hints on all function signatures. Import `from __future__ import annotations` for modern type syntax.
	- Docstrings: Use triple-quoted docstrings for all public functions and classes.
	- Naming: `snake_case` for functions and variables, `PascalCase` for classes.
	- Imports: Standard library first, then third-party, then local. Use absolute imports within the package.
	- f-strings: Use f-strings for SQL query construction (the SQL injection risk is mitigated by server-generated table names and column names from the schema).
	- Error handling: Raise `ValueError` in validators, `HTTPException` in route handlers.

	### TypeScript (Frontend)

	- Strict mode: TypeScript strict mode is enabled. All variables must have explicit or inferred types.
	- Functional components: Use function declarations with explicit return types.
	- Naming: `camelCase` for variables and functions, `PascalCase` for components and types.
	- Imports: Use relative paths for local modules, package names for third-party libraries.
	- Hooks: Custom hooks in `api/hooks.ts` follow the `use[X]` naming pattern and return `{ action, loading, error }`.
	- CSS: Use Tailwind utility classes exclusively. Avoid inline styles except for dynamic values (chart dimensions, SVG coordinates).

	## Git Workflow

	### Branch Naming

	- `feature/[description]`: New features
	- `fix/[description]`: Bug fixes
	- `docs/[description]`: Documentation changes
	- `refactor/[description]`: Code refactoring

	### Commit Messages

	Follow conventional commit format:

	```
	type(scope): description

	[optional body]
	```

	Types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`

	Examples:
	- `feat(operations): add deduplicate operation`
	- `fix(history): correct undo pointer after branch`
	- `docs(api): update endpoint descriptions`

	### Pull Request Process

	1. Create a feature branch from `main`.
	2. Implement changes with tests.
	3. Run the full test suite: `uv run pytest`.
	4. Ensure the frontend builds: `cd frontend && npm run build`.
	5. Open a pull request with a description of the changes and any manual testing performed.
	6. Request review from a maintainer.
	7. Address review feedback and re-run tests.
	8. Merge when approved and all checks pass.