PLAN.md

BookmarkBrain — Local-First Semantic Bookmark Manager

Vision

591+ Bookmarks in chaotischen Ordnern (15 Ebenen tief!) werden durchsuchbar per natuerlicher Sprache: "der Bash-Cheatsheet von neulich", "AWS Tutorials", "was hatte ich zu Vertical Farming gespeichert?"

Local-first. Keine Cloud. Keine Accounts. Deine Daten bleiben bei dir.

---

Architecture Overview

+------------------+     HTTP (localhost:7471)     +------------------+
|  Browser Extension| <--------------------------> |  Python Backend  |
|  (Chrome + Firefox)|                             |  (FastAPI)       |
|                    |                             |                  |
|  - Popup UI        |                             |  - REST API      |
|  - Quick Search    |                             |  - Import/Export |
|  - Save Bookmark   |                             |  - Auto-Tagger   |
|  - Content Extract |                             |  - Embedding Gen |
+------------------+                               +------------------+
                                                          |
                                                   +------+------+
                                                   |   SQLite    |
                                                   | + Embeddings|
                                                   | (numpy .npy)|
                                                   +-------------+

---

Tech Stack

Component	Technology	Why
Backend	Python 3.11+ / FastAPI	Async, schnell, Samuels Hauptsprache
Database	SQLite (via sqlite3)	Local-first, zero config, single file
Embeddings	paraphrase-multilingual-MiniLM-L12-v2	384 dims, DE+EN, CPU-fast
Vector Search	sqlite-vec (SQLite Extension)	Vektor-Suche + SQL in einer DB
Frontend	htmx + Jinja2 (Web UI)	Wie StudyAgent, bewaehrter Stack
Browser Extension	WXT Framework	Ein Codebase fuer Chrome + Firefox
Build	uv (Python), pnpm (Extension)	Schnell, modern

Warum sqlite-vec statt ChromaDB/FAISS/NumPy?

• Single File: Bookmarks + Vektoren in einer SQLite-DB, kein separates .npy
• SQL + Vektor kombinierbar: WHERE domain = 'github.com' AND embedding MATCH ?
• Zero Dependencies: Pure C Extension, pip install sqlite-vec
• ChromaDB hat 100+ transitive Dependencies, FAISS ist overkill bei 2000 Bookmarks

Warum paraphrase-multilingual-MiniLM-L12-v2?

• 384 Dimensionen, 118M Parameter, ~450 MB
• 50+ Sprachen inkl. Deutsch und Englisch
• ~2000-4000 Sätze/Sekunde auf CPU
• ONNX-Beschleunigung moeglich (2-3x Speedup)
• Deine 591 Bookmarks: Embedding in <1 Sekunde

---

Data Model

CREATE TABLE bookmarks (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    url TEXT NOT NULL UNIQUE,
    title TEXT NOT NULL,
    description TEXT DEFAULT '',         -- User-supplied or auto-extracted
    context TEXT DEFAULT '',             -- "Was wollte ich damit?" - User note
    folder_path TEXT DEFAULT '',         -- Original browser folder path
    domain TEXT DEFAULT '',              -- Extracted from URL
    favicon_url TEXT DEFAULT '',

    -- Auto-generated
    auto_tags TEXT DEFAULT '[]',         -- JSON array of AI-generated tags
    user_tags TEXT DEFAULT '[]',         -- JSON array of user-set tags
    content_snippet TEXT DEFAULT '',     -- First ~500 chars of page content

    -- Metadata
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    last_visited_at TIMESTAMP,
    visit_count INTEGER DEFAULT 0,
    is_dead BOOLEAN DEFAULT FALSE,      -- Link check: 404?

    -- Search
    embedding_text TEXT DEFAULT ''       -- Combined text used for embedding
);

CREATE VIRTUAL TABLE bookmarks_fts USING fts5(
    title, description, context, auto_tags, user_tags, content_snippet,
    content='bookmarks', content_rowid='id'
);

CREATE INDEX idx_bookmarks_domain ON bookmarks(domain);
CREATE INDEX idx_bookmarks_created ON bookmarks(created_at);

Embeddings als sqlite-vec Virtual Table:

CREATE VIRTUAL TABLE vec_bookmarks USING vec0(
    id INTEGER PRIMARY KEY,
    embedding FLOAT[384]
);

Alles in einer einzigen SQLite-Datei: data/bookmarks.db

---

Features (nach Prioritaet)

Phase 1: Core (Woche 1-2)

1. Bookmark Import — Netscape HTML Parser (deine Datei), Chrome/Firefox JSON
2. SQLite Storage — Schema wie oben, FTS5 fuer Volltextsuche
3. Embedding Generation — Alle Bookmarks embedden bei Import
4. Semantic Search API — GET /api/search?q=bash+cheatsheet → Top-10 Ergebnisse
5. Web UI — FastAPI + htmx, Suchfeld + Ergebnisliste + Tag-Filter
6. Auto-Tagging — Clustering + Keyword-Extraction aus Titel/URL/Domain

Phase 2: Browser Extension (Woche 3)

7. Extension Popup — Suchfeld, zeigt Ergebnisse aus Backend
8. Quick Save — Aktuelle Seite speichern mit einem Klick
9. Content Extraction — Content Script extrahiert Meta-Description, OG-Tags, ersten Absatz
10. Context Note — "Was wollte ich damit?" Textfeld beim Speichern

Phase 3: Intelligence (Woche 4)

11. Dead Link Checker — Async HEAD requests, markiert tote Links
12. Duplicate Detection — Embedding-Similarity > 0.95 → wahrscheinlich Duplikat
13. Smart Collections — Auto-gruppierung via Embedding-Cluster (K-Means)
14. "Related Bookmarks" — Klicke einen Bookmark → zeige aehnliche
15. Bookmark Health Dashboard — Stats, tote Links, Duplikate, ungenutztes

Phase 4: Polish (Woche 5+)

16. Export — Netscape HTML, JSON, Markdown
17. Keyboard Shortcuts — Cmd+K Style Quick-Search
18. Dark Mode
19. Backup/Restore
20. Browser Bookmark Sync — Periodic import from browser

---

Project Structure

~/Projekte/BookmarkBrain/
├── PLAN.md                    # Dieser Plan
├── FOR_SMLFLG.md              # Architektur-Doku
├── pyproject.toml             # Python project config (uv)
├── backend/
│   ├── __init__.py
│   ├── main.py                # FastAPI app, routes
│   ├── database.py            # SQLite connection, schema, migrations
│   ├── models.py              # Pydantic models
│   ├── search.py              # Semantic + FTS search engine
│   ├── embeddings.py          # Model loading, embedding generation
│   ├── importer.py            # Netscape HTML / JSON import
│   ├── tagger.py              # Auto-tag generation
│   ├── health.py              # Dead link checker, duplicate detection
│   └── templates/
│       ├── base.html
│       ├── index.html          # Main search UI
│       ├── bookmark.html       # Single bookmark detail/edit
│       └── components/
│           ├── search_results.html
│           ├── tag_cloud.html
│           └── stats.html
├── extension/
│   ├── wxt.config.ts
│   ├── package.json
│   ├── entrypoints/
│   │   ├── popup/              # Search popup
│   │   │   ├── App.tsx
│   │   │   └── main.tsx
│   │   ├── background.ts      # Service worker
│   │   └── content.ts         # Page content extraction
│   └── utils/
│       └── api.ts              # Backend API client
├── data/                       # Gitignored
│   └── bookmarks.db            # SQLite + sqlite-vec (alles in einer Datei)
├── scripts/
│   ├── import_bookmarks.py    # CLI import tool
│   └── reindex.py             # Rebuild embeddings
└── tests/
    ├── test_importer.py
    ├── test_search.py
    └── test_tagger.py

---

API Endpoints

GET  /                          # Web UI (htmx)
GET  /api/search?q=...&limit=10 # Semantic + FTS hybrid search
GET  /api/bookmarks             # List all (paginated)
GET  /api/bookmarks/:id         # Single bookmark
POST /api/bookmarks             # Create bookmark
PUT  /api/bookmarks/:id         # Update (tags, context, etc.)
DELETE /api/bookmarks/:id       # Delete

POST /api/import                # Import from file (Netscape HTML / JSON)
GET  /api/export?format=html    # Export

GET  /api/tags                  # All tags with counts
GET  /api/stats                 # Dashboard stats
GET  /api/health                # Dead link report
GET  /api/duplicates            # Suspected duplicates

POST /api/reindex               # Rebuild all embeddings

---

Search Algorithm (Hybrid)

import struct

def search(query: str, limit: int = 10) -> list[SearchResult]:
    query_vec = model.encode(query)
    blob = struct.pack(f"{len(query_vec)}f", *query_vec)

    # 1. Semantic Search (sqlite-vec)
    semantic = db.execute("""
        SELECT b.*, v.distance
        FROM vec_bookmarks v
        JOIN bookmarks b ON b.id = v.id
        WHERE v.embedding MATCH ? AND k = ?
        ORDER BY v.distance
    """, (blob, limit * 2)).fetchall()

    # 2. Full-Text Search (SQLite FTS5)
    fts = db.execute(
        "SELECT rowid, rank FROM bookmarks_fts WHERE bookmarks_fts MATCH ?",
        (query,)
    ).fetchall()

    # 3. Hybrid Score: 0.7 * semantic + 0.3 * fts_normalized
    return merge_and_rerank(semantic, fts)[:limit]

---

Auto-Tagging Strategy

Kein LLM noetig fuer 500-2000 Bookmarks. Stattdessen:

1. Domain-basierte Tags — github.com → "github", youtube.com → "video"
2. URL-Path Keywords — /docs/python/ → "python", "docs"
3. TF-IDF auf Titel — Haeufige Woerter ueber alle Bookmarks = schlechte Tags, seltene aber wiederkehrende Woerter = gute Tags
4. Folder-Path Tags — Software Arc > Linux > Code → "software-architecture", "linux", "code"
5. Embedding Cluster Labels — K-Means Cluster, repraesentativstes Wort pro Cluster

---

Browser Extension ↔ Backend Communication

Localhost HTTP (nicht Native Messaging):

• Einfacher zu implementieren
• Funktioniert identisch in Chrome + Firefox
• CORS Headers im Backend setzen
• Backend muss laufen (Systemd Service oder manuell)

// extension/utils/api.ts
const API_BASE = "http://localhost:7471/api";

export async function search(query: string) {
  const res = await fetch(`${API_BASE}/search?q=${encodeURIComponent(query)}`);
  return res.json();
}

export async function saveBookmark(data: BookmarkData) {
  const res = await fetch(`${API_BASE}/bookmarks`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(data),
  });
  return res.json();
}

---

Dependencies

Python (backend)

fastapi>=0.115
uvicorn[standard]>=0.30
sentence-transformers>=3.0
sqlite-vec>=0.1
numpy>=1.26
jinja2>=3.1
python-multipart>=0.0.9
httpx>=0.27            # For dead link checking
beautifulsoup4>=4.12   # For HTML bookmark parsing
scikit-learn>=1.4       # For K-Means clustering, TF-IDF

Node (extension)

wxt                    # Browser extension framework
typescript

---

Differenzierung vs. Prior Art

Feature	Raindrop	Memex	Linkding	Hoarder	BookmarkBrain
Local-first	No	Yes	Self-host	Self-host	Yes, zero infra
Semantic Search	No	No	No	No (Meilisearch)	Yes, local embeddings
AI Tags	Pro ($3/mo)	No	No	LLM-based	Local, free
"Was wollte ich?"	No	Annotations	No	Notes	First-class feature
Browser Extension	Yes	Yes	Yes	Yes	Yes (Chrome+FF)
Setup Complexity	None (cloud)	Extension only	Docker	Docker+4 containers	pip install + start
Multilingual	N/A	N/A	N/A	N/A	DE+EN nativ

Kern-Differenzierung:

1. Semantic Search LOKAL ohne Cloud/LLM-API — unique
2. "Was wollte ich damit?" als First-Class-Feature
3. Zero-Setup: kein Docker, kein Cloud-Account, pip install && run
4. Chaotische Ordner → saubere AI-Tags automatisch

---

Run Commands

# Setup
cd ~/Projekte/BookmarkBrain
uv venv && source .venv/bin/activate
uv pip install -e .

# Import existing bookmarks
python scripts/import_bookmarks.py ~/Downloads/bookmarks_05.03.26.html

# Start backend
uvicorn backend.main:app --reload --port 7471

# Extension dev (separate terminal)
cd extension && pnpm install && pnpm dev

---

Open Decisions (fuer Samuel)

1. Port: 7471 OK? (Mnemonic: "BRAIN" auf dem Telefon)
2. Extension zuerst oder Web UI zuerst? Plan sagt Web UI first.
3. ONNX-Optimierung sofort oder spaeter? Plan sagt spaeter (Phase 4).
4. Soll der Backend-Server als Systemd-Service laufen? Oder manuell starten?
5. Projektname: BookmarkBrain? Oder was anderes?