Initial ZeroGPU deployment with spaces shim

.github/workflows/ci.yml

@@ -0,0 +1,27 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test-quality:
+    runs-on: windows-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip setuptools
+          python -m pip install -r requirements.txt -r requirements-dev.txt
+
+      - name: Tests, coverage, lint, type, security
+        shell: pwsh
+        run: .\scripts\run_quality.ps1

.gitignore

@@ -0,0 +1,37 @@
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+.coverage
+htmlcov/
+.mypy_cache/
+.ruff_cache/
+.gradio/
+node_modules/
+#test-results/
+#playwright-report/
+blob-report/
+#assets/e2e/playwright-report/
+#assets/e2e/plant-playwright-report/
+gradio.out.log
+gradio.err.log
+data/*
+!data/.gitkeep
+exports/*
+!exports/.gitkeep
+models_cache/
+hf_cache/
+wandb/
+*.gguf
+*.safetensors
+*.bin
+*.ckpt
+*.onnx
+*.tflite
+*.pt
+*.pth
+*.model
+.env
+.env.*
+*.pem
+*.key

AGENTS.md

@@ -0,0 +1,62 @@
+# AGENTS.md
+
+## Project Goal
+Build a Gradio-based OpenBMB Local AI Workbench for the Build Small Hackathon.
+The app should run locally first, deploy as a Hugging Face Space, and document the
+small-model choices clearly.
+
+## Project Memory
+- Start with `docs/README.md`.
+- Use `docs/TASKS.md` as the task checklist.
+- Use `docs/IMPLEMENTATION_STATUS.md` to decide whether something is really done.
+- Use `docs/PRD_IMPLEMENTATION_MATRIX.md` before claiming PRD or extension PRD completion.
+- Use `docs/ACCEPTANCE_CRITERIA.md` before marking work complete.
+- Use `docs/ROADMAP.md` to prioritize hackathon work over stretch work.
+- Use `docs/ROADMAP_V2_CRITICAL_IMPROVEMENT_PLAN.md` for hard judge-oriented prioritization.
+- Use `docs/TEMPLATE_HOWTO.md` before creating a new domain app from the template.
+- Use `docs/PLANT_DISCOVERY_APP_PLAN.md` before changing the Plant Discovery reference app.
+- Use `docs/PLANT_MODEL_AND_TRAINING_HOWTO.md` before changing Plant Discovery model or training behavior.
+- Use `docs/ARCHITECTURE.md` before changing app structure.
+- Use `docs/EXTENDING.md` before adding models, tabs, services, or training.
+- Main PRD: `HF_PRD_v1.md`.
+- Extension PRD: `HF_PRD_ext.md`.
+
+## Hackathon Constraints
+- Use Gradio as the app surface.
+- Keep all model choices at or below 32B parameters.
+- Prefer local/offline inference paths when practical.
+- Avoid cloud APIs unless the user explicitly approves them.
+- Make the README useful for judges: goal, setup, model choice, local run, Space deployment, demo flow.
+
+## Current MVP Scope
+- Start with a working Gradio app and deterministic placeholder service.
+- Add real inference incrementally through Ollama, llama.cpp, Transformers, or SGLang.
+- Keep training, GGUF export, Trackio, MCP, and agent functionality as documented extension points until implemented.
+- Never claim the full PRD is done while placeholders or unchecked PRD tasks remain.
+
+## Commands
+- Create venv: `python -m venv .venv`
+- Activate venv: `.venv\Scripts\Activate.ps1`
+- Install: `python -m pip install -r requirements.txt`
+- Install dev tools: `python -m pip install -r requirements-dev.txt`
+- Run locally: `python app.py`
+- Run tests: `.\scripts\run_tests.ps1`
+- Run performance tests: `.\scripts\run_performance.ps1`
+- Run quality checks: `.\scripts\run_quality.ps1`
+
+## Code Style
+- Prefer small modules with clear responsibilities.
+- Keep model IDs in `config/models.yaml`; avoid hardcoding them in UI logic.
+- Keep user-facing text concise and demo-oriented.
+- Do not download model weights automatically on app startup.
+
+## Verification
+- Before claiming the app runs, verify `python app.py` starts locally.
+- If Python or dependencies are missing, say exactly what is missing.
+- Keep generated caches, model weights, exports, and virtualenvs out of git.
+- After implementing a task, update `docs/TASKS.md` and `docs/IMPLEMENTATION_STATUS.md`.
+- Every feature needs tests. Add or update unit tests for logic and user-story tests for workflows.
+- Whenever a bug, failed check, or regression is found, create or update a test that would catch it before fixing the code.
+- Do not mark a feature done until its tests pass or the test blocker is documented.
+- Avoid blanket coverage escapes. Do not add `pragma: no cover` without a specific documented reason.
+- For PRD work, update `docs/PRD_IMPLEMENTATION_MATRIX.md` along with task/status docs.

HF_PRD_ext.md

@@ -0,0 +1,1631 @@
+# OpenBMB Workbench — Planned Extensions: Detailed Spec
+> 2026-06-05 | Companion to PRD v2.0
+
+---
+
+## Table of Contents
+
+1. [vLLM Serving Tab](#1-vllm-serving-tab)
+2. [Ollama Quick-Start](#2-ollama-quick-start)
+3. [Reward Model Eval](#3-reward-model-eval)
+4. [Synthetic Data Gen](#4-synthetic-data-gen)
+5. [Paper-to-Code Agent](#5-paper-to-code-agent)
+6. [HF Spaces Deploy](#6-hf-spaces-deploy)
+7. [VINDEX Integration](#7-vindex-integration)
+8. [OCR Pipeline Hook](#8-ocr-pipeline-hook)
+9. [MiniCPM Desk-Pet](#9-minicpm-desk-pet)
+10. [MiniCPM-o Audio Tab](#10-minicpm-o-audio-tab)
+11. [Cross-Extension Wiring](#11-cross-extension-wiring)
+
+---
+
+## 1. vLLM Serving Tab
+
+### What it is
+
+vLLM is a production-grade inference engine built around *PagedAttention* — a KV-cache management
+algorithm that treats GPU memory like virtual memory pages. The result is dramatically higher
+throughput when multiple requests run concurrently, compared to naive Transformers inference.
+
+In the workbench context, vLLM adds a fourth inference mode alongside llama.cpp, SGLang, and
+Ollama. You use it when you want OpenAI-compatible HTTP endpoints, continuous batching, or when
+benchmarking production serving latency.
+
+### Why it matters
+
+| Scenario | Benefit |
+|----------|---------|
+| Benchmarking fine-tuned LoRA | Compare throughput before/after fine-tune |
+| Multi-user demo | Queue and batch concurrent requests |
+| Production deployment | OpenAI-compatible API, drop-in for existing tooling |
+| MiniCPM4.1-8B long context | PagedAttention shines on 128K context — avoids OOM |
+
+### Architecture
+
+```
+models/vllm_runner.py
+  VLLMRunner
+    .start(model_id, cfg)      → subprocess: vllm serve ...
+    .stop()                    → terminate subprocess
+    .chat(messages) → str      → POST /v1/chat/completions
+    .batch(prompts) → list[str]→ concurrent POST via asyncio
+    .stats() → dict            → GET /metrics (Prometheus)
+```
+
+```python
+# models/vllm_runner.py
+import subprocess, asyncio, requests
+from openai import AsyncOpenAI
+
+class VLLMRunner:
+    def __init__(self, cfg: dict):
+        self.model_id   = cfg["hf_id"]
+        self.port       = cfg.get("port", 8000)
+        self.gpu_memory = cfg.get("gpu_memory_utilization", 0.85)
+        self.trust_rc   = cfg.get("trust_remote_code", False)
+        self._proc      = None
+        self._client    = AsyncOpenAI(
+            base_url=f"http://localhost:{self.port}/v1",
+            api_key="vllm-local"
+        )
+
+    def start(self):
+        cmd = [
+            "vllm", "serve", self.model_id,
+            "--port",                    str(self.port),
+            "--gpu-memory-utilization",  str(self.gpu_memory),
+        ]
+        if self.trust_rc:
+            cmd.append("--trust-remote-code")
+        self._proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
+        self._wait_ready()
+
+    def _wait_ready(self, timeout: int = 60):
+        import time
+        for _ in range(timeout):
+            try:
+                r = requests.get(f"http://localhost:{self.port}/health")
+                if r.status_code == 200:
+                    return
+            except Exception:
+                pass
+            time.sleep(1)
+        raise RuntimeError("vLLM server did not become healthy")
+
+    async def chat(self, messages: list[dict], **kwargs) -> str:
+        resp = await self._client.chat.completions.create(
+            model=self.model_id,
+            messages=messages,
+            max_tokens=kwargs.get("max_tokens", 512),
+        )
+        return resp.choices[0].message.content
+
+    async def batch(self, prompts: list[str], system: str = "") -> list[str]:
+        tasks = [
+            self.chat([
+                {"role": "system",  "content": system},
+                {"role": "user",    "content": p},
+            ])
+            for p in prompts
+        ]
+        return await asyncio.gather(*tasks)
+
+    def stats(self) -> dict:
+        """Prometheus metrics parsed to dict. Returns throughput + latency."""
+        r = requests.get(f"http://localhost:{self.port}/metrics")
+        lines = r.text.splitlines()
+        metrics = {}
+        for line in lines:
+            if line.startswith("vllm:") and not line.startswith("#"):
+                key, val = line.rsplit(" ", 1)
+                metrics[key] = float(val)
+        return metrics
+
+    def stop(self):
+        if self._proc:
+            self._proc.terminate()
+            self._proc = None
+```
+
+### UI tab (models_tab.py → extend)
+
+Add a "vLLM" section to the models tab:
+
+```python
+with gr.Tab("⚡ vLLM"):
+    model_dd      = gr.Dropdown(label="Model", choices=list_vllm_compatible())
+    gpu_mem_sl    = gr.Slider(0.5, 0.95, value=0.85, label="GPU memory utilization")
+    start_btn     = gr.Button("Start server")
+    stop_btn      = gr.Button("Stop server")
+    status_lbl    = gr.Label(label="Status")
+    metrics_json  = gr.JSON(label="Live metrics")
+
+    def start_vllm(model_id, gpu_mem):
+        cfg = {**model_registry.get(model_id).cfg,
+               "gpu_memory_utilization": gpu_mem}
+        runner = VLLMRunner(cfg)
+        runner.start()
+        model_registry.register(f"{model_id}_vllm", runner)
+        return "Running", runner.stats()
+
+    start_btn.click(start_vllm, [model_dd, gpu_mem_sl], [status_lbl, metrics_json])
+```
+
+### Trackio events fired
+
+```python
+trackio.init(project="workbench", run_name="vllm_benchmark")
+trackio.log({"throughput_tok_per_s": ..., "p50_latency_ms": ..., "gpu_mem_used": ...})
+trackio.finish()
+```
+
+---
+
+## 2. Ollama Quick-Start
+
+### What it is
+
+Ollama is zero-configuration local model serving. One `ollama pull` downloads a quantized model
+and one `ollama serve` runs it. No CUDA setup, no Python environment issues. The REST API is
+OpenAI-compatible on port 11434.
+
+MiniCPM-V-4.6 and MiniCPM5-1B are both in the Ollama registry:
+```bash
+ollama pull openbmb/minicpm-v4.6
+ollama pull openbmb/minicpm5-1b
+```
+
+### Why it matters
+
+Ollama is the fastest path from "nothing" to "running model" — ideal for demos, non-GPU machines
+(Apple Silicon is well-optimized), and users who shouldn't need to understand quantization.
+
+### Architecture
+
+```
+models/ollama_runner.py
+  OllamaRunner
+    .pull(model_id)            → subprocess: ollama pull ...
+    .chat(messages) → str      → POST http://localhost:11434/api/chat
+    .generate(prompt) → str    → POST http://localhost:11434/api/generate (streaming)
+    .list() → list[str]        → GET /api/tags
+```
+
+```python
+# models/ollama_runner.py
+import requests, subprocess, json
+from typing import Generator
+
+class OllamaRunner:
+    BASE = "http://localhost:11434"
+
+    def __init__(self, model_id: str):
+        # Ollama uses "openbmb/minicpm-v4.6" style IDs directly
+        self.model_id = model_id
+
+    @staticmethod
+    def pull(model_id: str):
+        subprocess.run(["ollama", "pull", model_id], check=True)
+
+    @staticmethod
+    def list_local() -> list[str]:
+        r = requests.get(f"{OllamaRunner.BASE}/api/tags")
+        return [m["name"] for m in r.json().get("models", [])]
+
+    def chat(self, messages: list[dict], stream: bool = False) -> str | Generator:
+        payload = {"model": self.model_id, "messages": messages, "stream": stream}
+        r = requests.post(f"{self.BASE}/api/chat", json=payload, stream=stream)
+        if not stream:
+            return r.json()["message"]["content"]
+        # Generator for Gradio streaming
+        def _stream():
+            for line in r.iter_lines():
+                if line:
+                    chunk = json.loads(line)
+                    yield chunk["message"]["content"]
+                    if chunk.get("done"):
+                        break
+        return _stream()
+
+    def vision_chat(self, image_b64: str, prompt: str) -> str:
+        """Multimodal chat — Ollama passes images as base64 in the message."""
+        messages = [{
+            "role":    "user",
+            "content": prompt,
+            "images":  [image_b64],
+        }]
+        return self.chat(messages)
+```
+
+### UI integration
+
+The models tab gets an "Ollama" subtab with a model browser, pull button, and instant chat
+that requires zero setup — the simplest entry point to the whole workbench.
+
+```python
+with gr.Tab("🦙 Ollama"):
+    available = gr.Dropdown(label="Pull model",
+        choices=["openbmb/minicpm-v4.6", "openbmb/minicpm5-1b"],
+        allow_custom_value=True)
+    pull_btn   = gr.Button("Pull")
+    local_list = gr.JSON(label="Locally available")
+    pull_status= gr.Textbox(label="Status")
+
+    pull_btn.click(
+        lambda m: (OllamaRunner.pull(m), OllamaRunner.list_local()),
+        [available],
+        [pull_status, local_list]
+    )
+```
+
+---
+
+## 3. Reward Model Eval
+
+### What it is
+
+A reward model is a model trained to score (prompt, response) pairs — answering "how good is this
+output?" It's the missing piece between fine-tuning and verified alignment improvement. Without it
+you can train a LoRA and only know quantitatively that loss went down, not whether outputs actually
+got better by human-relevant criteria.
+
+### Why it matters
+
+- Validates that LoRA fine-tuning improved quality (not just minimized loss)
+- Enables best-of-N sampling: generate N responses, keep highest-scored
+- Enables DPO data creation: generate response pairs, reward model labels preferences
+- Closes the RLHF loop within the workbench itself
+
+### Reward model options
+
+| Model | Size | Focus |
+|-------|------|-------|
+| `OpenAssistant/reward-model-deberta-v3-large-v2` | 450M | General helpfulness |
+| `Salesforce/SFR-Reward-FsfairX-LLaMA3-RM-v0.1` | 8B | Instruction following |
+| MiniCPM5-1B itself (self-eval) | 1B | Domain-specific, via prompt |
+
+For the workbench, using MiniCPM5-1B as a judge (LLM-as-judge pattern) is the lowest-friction
+option since the model is already loaded.
+
+### Architecture
+
+```
+training/reward_eval.py
+  RewardEvaluator
+    .score(prompt, response) → float
+    .best_of_n(prompt, n, generator) → str
+    .create_dpo_pairs(dataset, generator, n=4) → Dataset
+    .eval_lora_vs_base(base_svc, lora_svc, eval_ds) → dict
+```
+
+```python
+# training/reward_eval.py
+from transformers import pipeline, AutoModelForSequenceClassification, AutoTokenizer
+import torch
+
+class RewardEvaluator:
+    def __init__(self, reward_model_id: str = "OpenAssistant/reward-model-deberta-v3-large-v2"):
+        self.pipe = pipeline(
+            "text-classification",
+            model=reward_model_id,
+            device=0 if torch.cuda.is_available() else -1,
+        )
+
+    def score(self, prompt: str, response: str) -> float:
+        """Returns a scalar reward score (higher = better)."""
+        text = f"Human: {prompt}\n\nAssistant: {response}"
+        result = self.pipe(text, truncation=True, max_length=512)
+        return result[0]["score"]
+
+    def best_of_n(
+        self,
+        prompt: str,
+        n: int,
+        generator_fn,    # callable: prompt -> str
+    ) -> tuple[str, float]:
+        """Generate N responses, return the one with highest reward."""
+        candidates = [(generator_fn(prompt), ) for _ in range(n)]
+        scored = [(r[0], self.score(prompt, r[0])) for r in candidates]
+        return max(scored, key=lambda x: x[1])
+
+    def create_dpo_pairs(
+        self,
+        prompts: list[str],
+        generator_fn,
+        n: int = 4,
+    ):
+        """
+        For each prompt: generate n responses, rank by reward,
+        return (prompt, chosen, rejected) triplets for DPO training.
+        """
+        from datasets import Dataset
+        rows = []
+        for prompt in prompts:
+            responses = [(generator_fn(prompt), ) for _ in range(n)]
+            scored = sorted(
+                [(r[0], self.score(prompt, r[0])) for r in responses],
+                key=lambda x: x[1], reverse=True
+            )
+            if len(scored) >= 2:
+                rows.append({
+                    "prompt":   prompt,
+                    "chosen":   scored[0][0],
+                    "rejected": scored[-1][0],
+                    "reward_gap": scored[0][1] - scored[-1][1],
+                })
+        return Dataset.from_list(rows)
+
+    def eval_lora_vs_base(
+        self,
+        base_svc,
+        lora_svc,
+        eval_prompts: list[str],
+    ) -> dict:
+        """
+        Compare base vs LoRA checkpoint by average reward score.
+        Returns win rate and per-prompt scores.
+        """
+        base_scores = [self.score(p, base_svc.generate(p)) for p in eval_prompts]
+        lora_scores = [self.score(p, lora_svc.generate(p)) for p in eval_prompts]
+        wins = sum(l > b for l, b in zip(lora_scores, base_scores))
+        return {
+            "base_mean":   sum(base_scores) / len(base_scores),
+            "lora_mean":   sum(lora_scores) / len(lora_scores),
+            "lora_win_rate": wins / len(eval_prompts),
+            "per_prompt":  list(zip(eval_prompts, base_scores, lora_scores)),
+        }
+```
+
+### Trackio logging
+
+```python
+results = evaluator.eval_lora_vs_base(base_svc, lora_svc, eval_prompts)
+trackio.init(project="workbench", run_name="reward_eval")
+trackio.log({
+    "base_reward_mean":     results["base_mean"],
+    "lora_reward_mean":     results["lora_mean"],
+    "lora_win_rate":        results["lora_win_rate"],
+})
+trackio.finish()
+```
+
+---
+
+## 4. Synthetic Data Gen
+
+### What it is
+
+The ml-intern finding: *when real data is insufficient, have an LLM generate training data*.
+This module does exactly that — it uses a capable model (MiniCPM4.1-8B or a cloud model via
+HF Router) to generate diverse, high-quality (prompt, response) pairs on demand.
+
+### Why it matters
+
+Real-world fine-tuning is often blocked not by compute but by data. You have 50 good examples
+but need 5000. Synthetic gen + quality filtering bridges that gap, especially for specialized
+domains (plant species, historical OCR corrections, industrial inspection defect labels).
+
+### Architecture
+
+```
+datasets/synthetic.py
+  SyntheticGenerator
+    .generate(topic, n, schema) → Dataset
+    .augment(existing_ds, n) → Dataset
+    .filter_quality(ds, min_score) → Dataset
+    .generate_dpo_pairs(topic, n) → Dataset
+```
+
+```python
+# datasets/synthetic.py
+import json
+from datasets import Dataset
+
+GENERATION_PROMPT = """You are a training data generator. Generate {n} diverse, high-quality 
+training examples for the topic: {topic}.
+
+Output ONLY a valid JSON array. Each item must have these fields: {schema}
+No explanation, no markdown, no preamble. Raw JSON array only."""
+
+class SyntheticGenerator:
+    def __init__(self, generator_svc):
+        """generator_svc: any loaded ModelService with a .generate(prompt) method."""
+        self.gen = generator_svc
+
+    def generate(
+        self,
+        topic: str,
+        n: int = 100,
+        schema: dict | None = None,
+    ) -> Dataset:
+        """
+        Generate n training examples on a topic.
+        schema: dict of field_name → description, e.g.
+          {"instruction": "task to perform", "response": "ideal answer"}
+        """
+        schema = schema or {"instruction": "user task", "response": "ideal answer"}
+        schema_str = ", ".join(f'"{k}": "{v}"' for k, v in schema.items())
+
+        # Generate in batches of 20 to stay within context
+        rows = []
+        for batch_start in range(0, n, 20):
+            batch_n = min(20, n - batch_start)
+            prompt = GENERATION_PROMPT.format(
+                n=batch_n, topic=topic, schema="{" + schema_str + "}"
+            )
+            raw = self.gen.generate(prompt)
+            try:
+                # Strip any accidental markdown fences
+                clean = raw.strip().lstrip("```json").lstrip("```").rstrip("```")
+                batch = json.loads(clean)
+                rows.extend(batch)
+            except json.JSONDecodeError:
+                # Skip malformed batches; log the failure
+                continue
+
+        return Dataset.from_list(rows)
+
+    def augment(self, existing_ds: Dataset, n: int) -> Dataset:
+        """
+        Use existing examples as few-shot demonstrations to generate n more.
+        Samples up to 5 examples from existing_ds as context.
+        """
+        import random
+        samples = existing_ds.shuffle().select(range(min(5, len(existing_ds))))
+        examples_str = json.dumps(samples.to_list(), indent=2)
+
+        prompt = f"""Here are {len(samples)} example training items:
+{examples_str}
+
+Generate {n} MORE diverse examples in the exact same JSON format.
+Output only the JSON array, no explanation."""
+
+        raw = self.gen.generate(prompt)
+        try:
+            clean = raw.strip().lstrip("```json").rstrip("```")
+            new_rows = json.loads(clean)
+        except json.JSONDecodeError:
+            return existing_ds
+        return Dataset.from_list(existing_ds.to_list() + new_rows)
+
+    def filter_quality(
+        self,
+        ds: Dataset,
+        reward_evaluator=None,
+        min_score: float = 0.6,
+    ) -> Dataset:
+        """
+        Filter with reward model if available, else heuristic filters.
+        Heuristics: min length, no repeated n-grams, valid JSON fields.
+        """
+        if reward_evaluator:
+            def _score(row):
+                return reward_evaluator.score(
+                    row.get("instruction", ""),
+                    row.get("response", "")
+                ) >= min_score
+            return ds.filter(_score)
+        else:
+            # Basic heuristics
+            def _heuristic(row):
+                resp = row.get("response", "")
+                return (
+                    len(resp) >= 20 and       # not too short
+                    len(resp) <= 4096 and     # not too long
+                    resp.count(resp[:20]) < 3  # not repetitive
+                )
+            return ds.filter(_heuristic)
+
+    def generate_for_domain(
+        self,
+        domain: str,
+        output_path: str,
+        n: int = 500,
+    ):
+        """
+        Convenience method: generate, augment, filter, save to disk.
+        Use for plant ID: domain="Plant species identification from photo descriptions"
+        """
+        ds = self.generate(topic=domain, n=n // 2)
+        ds = self.augment(ds, n=n // 2)
+        ds = self.filter_quality(ds)
+        ds.save_to_disk(output_path)
+        return ds
+```
+
+### Domain-specific example: plant ID
+
+```python
+gen = SyntheticGenerator(model_registry.get("minicpm41_8b"))
+plant_ds = gen.generate_for_domain(
+    domain="Identifying plant species from visual descriptions. "
+           "Include common name, latin name, family, key visual features, and care tips.",
+    output_path="data/synthetic_plants",
+    n=2000,
+)
+# → 2000 synthetic (description → species JSON) training pairs
+```
+
+---
+
+## 5. Paper-to-Code Agent
+
+### What it is
+
+An autonomous agent that takes an arXiv paper URL or title, reads the methodology section,
+and implements the described technique within the workbench codebase. Directly inspired by
+the ml-intern architecture (Research → Plan → Implement → Trace).
+
+### Why it matters
+
+The gap between reading a paper and running an experiment is usually days of engineering. This
+agent compresses that to minutes for techniques that fit the workbench's model family.
+Practical use cases: implement a new PEFT variant, add a new evaluation metric, adapt a new
+data augmentation from a recent VLM paper.
+
+### Architecture
+
+```
+agent/paper_agent.py
+  PaperAgent
+    .run(paper_ref) → AgentResult
+      → Phase 1: Research (fetch + parse paper)
+      → Phase 2: Plan (identify workbench integration points)
+      → Phase 3: Implement (generate + write code)
+      → Phase 4: Test (run + log to Trackio)
+      → Phase 5: Trace (upload session to HF Dataset)
+```
+
+```python
+# agent/paper_agent.py
+import re
+from dataclasses import dataclass, field
+from huggingface_hub import HfApi
+from smolagents import CodeAgent, HfApiModel
+import trackio
+
+@dataclass
+class AgentResult:
+    paper_title: str = ""
+    summary:     str = ""
+    files_modified: list[str] = field(default_factory=list)
+    test_results:   dict      = field(default_factory=dict)
+    trace_url:      str       = ""
+
+class PaperAgent:
+    SYSTEM_PROMPT = """You are an ML engineer working inside the OpenBMB Workbench codebase.
+Given a research paper, your job is to:
+1. Understand the core algorithm or technique.
+2. Identify which module in the workbench it extends (training/, models/, datasets/, tools/).
+3. Implement it as a new class or function, following the existing patterns.
+4. Write a simple test that runs within the workbench and logs results to Trackio.
+
+The workbench uses: transformers, peft, trl, trackio, mcp, gradio.
+All new code must: fire events via the EventBus, log to Trackio, register in the Registry."""
+
+    def __init__(self, orchestrator_model: str = "openbmb/MiniCPM4.1-8B"):
+        self.model   = HfApiModel(orchestrator_model)
+        self.api     = HfApi()
+        self._log    = []
+
+    def run(self, paper_ref: str) -> AgentResult:
+        """
+        paper_ref: arXiv URL like "https://arxiv.org/abs/2106.09685"
+                   or paper title like "LoRA: Low-Rank Adaptation of Large Language Models"
+        """
+        result = AgentResult()
+
+        # Phase 1: Research
+        paper_text = self._fetch_paper(paper_ref)
+        result.paper_title = self._extract_title(paper_text)
+
+        # Phase 2: Plan
+        plan = self._plan(paper_text)
+
+        # Phase 3: Implement
+        code_files = self._implement(plan, paper_text)
+        result.files_modified = list(code_files.keys())
+        for path, code in code_files.items():
+            self._write_file(path, code)
+
+        # Phase 4: Test
+        trackio.init(project="workbench", run_name=f"paper_agent_{result.paper_title[:30]}")
+        test_result = self._test(code_files)
+        result.test_results = test_result
+        trackio.log({"test_passed": test_result.get("passed", False), **test_result})
+        trackio.finish()
+
+        # Phase 5: Trace (ml-intern pattern)
+        result.trace_url = self._upload_trace(result)
+
+        return result
+
+    def _fetch_paper(self, paper_ref: str) -> str:
+        """Fetch paper text via HF Papers API or arXiv."""
+        import requests
+        if "arxiv.org" in paper_ref:
+            arxiv_id = paper_ref.split("/abs/")[-1]
+            r = requests.get(f"https://export.arxiv.org/abs/{arxiv_id}")
+            return r.text
+        # Fall back to HF Papers search
+        from huggingface_hub import list_papers
+        results = list(list_papers(query=paper_ref, limit=1))
+        return str(results[0]) if results else ""
+
+    def _plan(self, paper_text: str) -> str:
+        """Ask the LLM to analyze the paper and produce an integration plan."""
+        agent = CodeAgent(tools=[], model=self.model, max_steps=5)
+        return agent.run(
+            f"Read this paper excerpt and produce a 5-step integration plan "
+            f"for the OpenBMB Workbench:\n\n{paper_text[:8000]}"
+        )
+
+    def _implement(self, plan: str, paper_text: str) -> dict[str, str]:
+        """Generate code files from the plan."""
+        agent = CodeAgent(tools=[], model=self.model, max_steps=15)
+        code = agent.run(
+            f"Implementation plan:\n{plan}\n\n"
+            f"Paper details:\n{paper_text[:4000]}\n\n"
+            f"Generate the Python file(s). Return a JSON dict: "
+            f"{{\"path/to/file.py\": \"file_content\", ...}}"
+        )
+        import json
+        try:
+            return json.loads(code)
+        except Exception:
+            return {}
+
+    def _write_file(self, path: str, content: str):
+        import os
+        os.makedirs(os.path.dirname(path), exist_ok=True)
+        with open(path, "w") as f:
+            f.write(content)
+
+    def _test(self, code_files: dict) -> dict:
+        """Run a quick import + instantiation test on generated files."""
+        results = {}
+        for path in code_files:
+            try:
+                module_name = path.replace("/", ".").replace(".py", "")
+                import importlib.util, sys
+                spec = importlib.util.spec_from_file_location(module_name, path)
+                mod = importlib.util.module_from_spec(spec)
+                spec.loader.exec_module(mod)
+                results[path] = "imported_ok"
+            except Exception as e:
+                results[path] = f"error: {e}"
+        results["passed"] = all("ok" in v for v in results.values())
+        return results
+
+    def _upload_trace(self, result: AgentResult) -> str:
+        """Upload session trace to private HF Dataset (ml-intern pattern)."""
+        import json, datetime
+        session = {
+            "timestamp":      datetime.datetime.utcnow().isoformat(),
+            "paper":          result.paper_title,
+            "files_modified": result.files_modified,
+            "test_results":   result.test_results,
+        }
+        user = self.api.whoami()["name"]
+        dataset_id = f"{user}/workbench-paper-sessions"
+        # Upload as JSONL
+        # ... (HF Dataset API)
+        return f"https://huggingface.co/datasets/{dataset_id}"
+```
+
+---
+
+## 6. HF Spaces Deploy
+
+### What it is
+
+One-click packaging and deployment of the current workbench state as a Hugging Face Space.
+The Space runs the same Gradio app on HF infrastructure, shareable via URL, with ZeroGPU
+support for serverless GPU access.
+
+### Why it matters
+
+Sharing a workbench state with a collaborator currently requires: git push, environment setup,
+model download, config sync. With one-click deploy: copy URL → colleague sees the live app.
+For hackathons this is especially valuable — deploy a domain-specific variant as a demo Space
+in under 2 minutes.
+
+### Architecture
+
+```
+deploy/spaces.py
+  SpacesDeployer
+    .prepare_repo() → creates/updates HF Space repo
+    .upload_code()  → pushes app code (not model weights)
+    .set_hardware(tier) → sets GPU tier in README
+    .configure_secrets(env_vars) → sets HF Space secrets
+    .deploy() → trigger Space rebuild
+    .get_url() → returns live Space URL
+```
+
+```python
+# deploy/spaces.py
+import os, shutil, tempfile
+from pathlib import Path
+from huggingface_hub import HfApi, SpaceHardware
+
+class SpacesDeployer:
+    HARDWARE_MAP = {
+        "cpu":    SpaceHardware.CPU_BASIC,
+        "t4":     SpaceHardware.T4_SMALL,
+        "t4_lg":  SpaceHardware.T4_MEDIUM,
+        "a10":    SpaceHardware.A10G_SMALL,
+        "a100":   SpaceHardware.A100_LARGE,
+        "zero":   SpaceHardware.CPU_BASIC,   # ZeroGPU: uses CPU_BASIC + @spaces.GPU
+    }
+    EXCLUDE = {".git", "__pycache__", "exports", "data", "checkpoints",
+               "*.gguf", "*.bin", "*.safetensors", ".env"}
+
+    def __init__(self, space_id: str, hardware: str = "zero"):
+        self.api      = HfApi()
+        self.space_id = space_id   # "username/my-workbench"
+        self.hardware = hardware
+
+    def deploy(
+        self,
+        src_dir: str = ".",
+        env_vars: dict | None = None,
+    ) -> str:
+        """Full deploy pipeline. Returns live Space URL."""
+        self._create_or_update_repo()
+        self._upload_code(src_dir)
+        self._configure_secrets(env_vars or {})
+        self._patch_app_for_zerogpu()
+        return f"https://huggingface.co/spaces/{self.space_id}"
+
+    def _create_or_update_repo(self):
+        try:
+            self.api.create_repo(
+                repo_id=self.space_id,
+                repo_type="space",
+                space_sdk="gradio",
+                private=False,
+                exist_ok=True,
+            )
+        except Exception as e:
+            print(f"Repo create/update: {e}")
+
+    def _upload_code(self, src_dir: str):
+        with tempfile.TemporaryDirectory() as tmp:
+            src  = Path(src_dir)
+            dest = Path(tmp)
+            # Copy only non-excluded files
+            for item in src.rglob("*"):
+                if any(item.match(pat) for pat in self.EXCLUDE):
+                    continue
+                rel = item.relative_to(src)
+                target = dest / rel
+                target.parent.mkdir(parents=True, exist_ok=True)
+                if item.is_file():
+                    shutil.copy2(item, target)
+            self.api.upload_folder(
+                folder_path=str(dest),
+                repo_id=self.space_id,
+                repo_type="space",
+                commit_message="workbench deploy",
+            )
+
+    def _configure_secrets(self, env_vars: dict):
+        for key, val in env_vars.items():
+            self.api.add_space_secret(
+                repo_id=self.space_id,
+                key=key,
+                value=val,
+            )
+        # Always set hardware
+        self.api.request_space_hardware(
+            repo_id=self.space_id,
+            hardware=self.HARDWARE_MAP[self.hardware],
+        )
+
+    def _patch_app_for_zerogpu(self):
+        """
+        If hardware=zero, wrap inference functions with @spaces.GPU decorator.
+        Uploads a patched app.py.
+        """
+        if self.hardware != "zero":
+            return
+        # Read existing app.py from Space, inject @spaces.GPU, re-upload
+        app_content = self.api.hf_hub_download(
+            repo_id=self.space_id, filename="app.py", repo_type="space"
+        )
+        with open(app_content) as f:
+            code = f.read()
+        # Simple injection: add import and decorator before inference functions
+        patched = "import spaces\n" + code.replace(
+            "def run_inference(",
+            "@spaces.GPU\ndef run_inference("
+        )
+        self.api.upload_file(
+            path_or_fileobj=patched.encode(),
+            path_in_repo="app.py",
+            repo_id=self.space_id,
+            repo_type="space",
+        )
+```
+
+### UI: deploy button in any tab header
+
+```python
+with gr.Row():
+    space_id_box = gr.Textbox(placeholder="username/my-workbench", label="Space ID")
+    hardware_dd  = gr.Dropdown(["cpu","t4","a10","zero"], value="zero", label="Hardware")
+    deploy_btn   = gr.Button("🚀 Deploy to HF Spaces")
+    deploy_url   = gr.Textbox(label="Live URL", interactive=False)
+
+    def do_deploy(space_id, hw):
+        d = SpacesDeployer(space_id, hw)
+        url = d.deploy(env_vars={"HF_TOKEN": os.environ.get("HF_TOKEN","")})
+        return url
+
+    deploy_btn.click(do_deploy, [space_id_box, hardware_dd], deploy_url)
+```
+
+---
+
+## 7. VINDEX Integration
+
+### What it is
+
+VINDEX is your own knowledge-editing engine (ki-fusion-labs.de). It exposes eight methods for
+mechanistic interpretability and targeted weight editing on transformer models. Integrating it
+into the workbench closes the loop between training a LoRA and verifying (or surgically fixing)
+what the model actually knows.
+
+VINDEX endpoints (from your PRD v1):
+- `logit_lens` — per-layer prediction visualization
+- `slot_neighbors` — embedding space neighbors
+- `layer_contribution` — per-layer influence on final prediction
+- `transition_spectrum` — attention head transition matrix
+- `calibrated_edit` — targeted fact edit in weight space
+- `derive_scale` — scaling factor derivation for safe edits
+- `star_spread` — spread edit across semantically related slots
+- `protect_relations` — guard against side effects
+
+### Why it matters for the workbench
+
+After LoRA fine-tuning a model on plant data:
+1. `logit_lens` — does the model correctly predict "Rosa" at the right layer for a rose image?
+2. `calibrated_edit` — if it consistently misidentifies Acer as Quercus, edit that slot
+3. `star_spread` — propagate the Acer correction to closely related maple species
+4. `protect_relations` — verify the edit didn't break "plant → living thing → organism"
+
+This is not possible with LoRA alone — LoRA changes weight statistics globally. VINDEX does
+surgical point edits, making it a *complement* to LoRA, not a replacement.
+
+### Architecture
+
+```
+tools/vindex_tool.py
+  VINDEXClient
+    .logit_lens(model, tokenizer, text) → dict[layer, prediction]
+    .calibrated_edit(model, subject, relation, old_obj, new_obj) → model
+    .star_spread(model, anchor_subject, n=5) → list[affected_slots]
+    .protect_relations(model, protected_triplets) → model
+    .layer_contribution(model, text) → dict[layer, score]
+    .slot_neighbors(model, token_id, n=10) → list[str]
+```
+
+```python
+# tools/vindex_tool.py
+"""
+VINDEX integration.
+Assumes VINDEX FastAPI server is running locally on port 8765,
+OR VINDEX modules are importable from your local install.
+"""
+import requests
+from mcp.server.fastmcp import FastMCP
+
+VINDEX_BASE = "http://localhost:8765"   # your local VINDEX FastAPI
+
+mcp = FastMCP("VINDEXTools")
+
+@mcp.tool()
+async def logit_lens(
+    model_id:  str,
+    text:      str,
+    layer_range: tuple[int, int] = (0, -1),
+) -> dict:
+    """
+    Run logit lens on a loaded model for the given text.
+    Returns per-layer top-5 token predictions and probabilities.
+    Useful for finding the 'phase layer' where the model commits to an answer.
+    """
+    r = requests.post(f"{VINDEX_BASE}/logit_lens", json={
+        "model_id":    model_id,
+        "text":        text,
+        "layer_range": list(layer_range),
+    })
+    return r.json()
+
+@mcp.tool()
+async def calibrated_edit(
+    model_id:  str,
+    subject:   str,
+    relation:  str,
+    old_obj:   str,
+    new_obj:   str,
+    causal_window: int = 3,   # ±3 layers around logit lens phase layer
+) -> dict:
+    """
+    Perform a targeted knowledge edit: change the model's belief about
+    (subject, relation) from old_obj to new_obj.
+    causal_window: restrict causal search to ±N layers around phase layer.
+    Returns edit_success, layers_modified, side_effect_score.
+    """
+    r = requests.post(f"{VINDEX_BASE}/calibrated_edit", json={
+        "model_id":      model_id,
+        "subject":       subject,
+        "relation":      relation,
+        "old_obj":       old_obj,
+        "new_obj":       new_obj,
+        "causal_window": causal_window,
+    })
+    return r.json()
+
+@mcp.tool()
+async def star_spread(
+    model_id:       str,
+    anchor_subject: str,
+    n_neighbors:    int = 5,
+) -> dict:
+    """
+    Find semantically related slots and spread a recent edit across them.
+    Example: after editing "Acer palmatum → maple", also update
+    "Acer japonicum", "Acer shirasawanum" etc.
+    Returns list of affected subjects and their edit scores.
+    """
+    r = requests.post(f"{VINDEX_BASE}/star_spread", json={
+        "model_id":       model_id,
+        "anchor_subject": anchor_subject,
+        "n_neighbors":    n_neighbors,
+    })
+    return r.json()
+
+@mcp.tool()
+async def protect_relations(
+    model_id:          str,
+    protected_triplets: list[dict],   # [{"s": ..., "r": ..., "o": ...}]
+) -> dict:
+    """
+    After a knowledge edit, verify that listed subject-relation-object triplets
+    remain intact. Returns a pass/fail table and a side_effect_score.
+    """
+    r = requests.post(f"{VINDEX_BASE}/protect_relations", json={
+        "model_id":          model_id,
+        "protected_triplets": protected_triplets,
+    })
+    return r.json()
+
+@mcp.tool()
+async def layer_contribution(
+    model_id: str,
+    text:     str,
+) -> dict:
+    """
+    Per-layer contribution score to the final prediction.
+    Use to find which layers drive the target behavior before editing.
+    """
+    r = requests.post(f"{VINDEX_BASE}/layer_contribution", json={
+        "model_id": model_id,
+        "text":     text,
+    })
+    return r.json()
+```
+
+### Known bugs to fix before integration (from VINDEX PRD v1)
+
+1. **GPU memory leak** — after repeated edits, VRAM grows unbounded. Fix: explicitly call
+   `torch.cuda.empty_cache()` after each `calibrated_edit` call and detach gradient graphs.
+
+2. **Dead-code blocks** — several helper functions in the weight-surgery path are unreachable
+   after a recent refactor. Before integrating: `grep -n "def " vindex/core.py | xargs` and
+   verify each function has at least one call site.
+
+3. **Weight imbalance** — `star_spread` can over-edit related slots if `n_neighbors > 5`.
+   Hard-cap at 5 in the MCP tool until the scaling formula is validated.
+
+4. **Forward optimization** — restrict causal search to ±3 layers around the logit lens
+   phase layer (already implemented as `causal_window` param above).
+
+### UI tab: "🧠 Knowledge Editor"
+
+```
+Inputs:
+  Model selector (loaded models)
+  Subject text (e.g. "Acer palmatum")
+  Relation (e.g. "is a type of")
+  Old object (e.g. "oak")
+  New object (e.g. "maple")
+  [Run Logit Lens] button → displays per-layer heatmap via gr.Plot
+  [Apply Edit] button → runs calibrated_edit
+  [Spread] button → runs star_spread
+  [Verify] button → runs protect_relations on a default triplet set
+  
+Outputs:
+  Per-layer prediction table
+  Edit success / layers modified
+  Side effect score (0 = safe, 1 = dangerous)
+```
+
+---
+
+## 8. OCR Pipeline Hook
+
+### What it is
+
+Your self-improving multilingual OCR pipeline (Latin, Arabic, Cyrillic) already exists and
+produces output files: image + predicted_text + confidence scores. This extension hooks those
+outputs directly into the workbench Field Notes system, creating a tight correction loop:
+
+```
+OCR pipeline outputs (uncertain predictions)
+        ↓
+Auto-created Field Notes (image + OCR text + empty correction field)
+        ↓
+Human reviews in UI → fills in correction
+        ↓
+Accepted corrections auto-tagged "use_for_training=True"
+        ↓
+LoRA training run on correction pairs
+        ↓
+Better OCR model → fewer uncertain predictions
+```
+
+This is the active learning loop your OCR pipeline was designed for but didn't yet have
+a clean UI for corrections and retraining.
+
+### Architecture
+
+```
+datasets/ocr_loader.py
+  OCRPipelineLoader
+    .watch(output_dir, threshold) → poll for new low-confidence outputs
+    .ingest(output_dir) → batch import all outputs
+    .to_field_notes(threshold) → FieldNote[] (uncertain ones only)
+    .to_training_dataset() → Dataset (corrected ones only)
+```
+
+```python
+# datasets/ocr_loader.py
+import json, os
+from pathlib import Path
+from datasets.field_notes import FieldNote, FieldNoteStore
+from core.events import bus, EventType, Event
+
+class OCRPipelineLoader:
+    """
+    Watches a directory written by the OCR pipeline.
+    Expected format per document:
+      <doc_id>.json → {"image_path": ..., "predicted_text": ...,
+                        "confidence": float, "script": "latin"|"arabic"|"cyrillic"}
+    """
+    def __init__(
+        self,
+        output_dir: str,
+        store: FieldNoteStore,
+        confidence_threshold: float = 0.85,
+    ):
+        self.output_dir  = Path(output_dir)
+        self.store       = store
+        self.threshold   = confidence_threshold
+
+    def ingest(self, limit: int | None = None) -> int:
+        """
+        Read all pipeline outputs. Create Field Notes for uncertain predictions
+        (confidence < threshold). Skip already-ingested docs.
+        Returns number of new Field Notes created.
+        """
+        count = 0
+        json_files = sorted(self.output_dir.glob("*.json"))
+        if limit:
+            json_files = json_files[:limit]
+
+        for jf in json_files:
+            try:
+                data = json.loads(jf.read_text())
+            except json.JSONDecodeError:
+                continue
+
+            # Skip high-confidence outputs
+            if data.get("confidence", 1.0) >= self.threshold:
+                continue
+
+            note = FieldNote(
+                id=f"ocr_{jf.stem}",
+                model_id="ocr_pipeline",
+                modality="image",
+                image_path=data["image_path"],
+                prompt=(
+                    f"Transcribe this {data.get('script','latin')} text accurately. "
+                    f"OCR predicted: '{data['predicted_text']}'"
+                ),
+                response=data["predicted_text"],
+                correction="",      # human fills this in
+                tags=[
+                    f"script:{data.get('script','unknown')}",
+                    f"conf:{data.get('confidence',0.0):.2f}",
+                    "source:ocr_pipeline",
+                ],
+            )
+            self.store.save(note)
+            count += 1
+
+        # Fire event
+        import asyncio
+        asyncio.run(bus.emit(Event(
+            type=EventType.DATASET_LOADED,
+            payload={"source": "ocr_pipeline", "new_notes": count}
+        )))
+        return count
+
+    def watch(self, poll_interval: int = 30):
+        """
+        Background thread: poll output_dir every N seconds, ingest new files.
+        Use in production when OCR pipeline runs continuously.
+        """
+        import threading, time
+        seen = set()
+        def _poll():
+            while True:
+                for jf in self.output_dir.glob("*.json"):
+                    if jf.stem not in seen:
+                        seen.add(jf.stem)
+                        self.ingest.__wrapped__([jf])  # single-file ingest
+                time.sleep(poll_interval)
+        t = threading.Thread(target=_poll, daemon=True)
+        t.start()
+
+    def to_training_dataset(self, script_filter: str | None = None):
+        """
+        Export corrected field notes as a training dataset.
+        schema: {"image_path": ..., "instruction": ..., "response": ...}
+        Ready to pass to LoRATextTrainer or a vision LoRA config.
+        """
+        from datasets import Dataset
+        query = "SELECT data FROM notes WHERE json_extract(data,'$.correction') != ''"
+        if script_filter:
+            query += f" AND json_extract(data,'$.tags') LIKE '%script:{script_filter}%'"
+
+        rows = [
+            json.loads(r[0])
+            for r in self.store.conn.execute(query)
+        ]
+        training_rows = [
+            {
+                "image_path":  r["image_path"],
+                "instruction": r["prompt"],
+                "response":    r["correction"],  # human-corrected
+                "script":      next(
+                    (t.split(":")[1] for t in r["tags"] if t.startswith("script:")),
+                    "unknown"
+                ),
+            }
+            for r in rows
+        ]
+        return Dataset.from_list(training_rows)
+```
+
+### UI: OCR correction view (Field Notes tab, new subtab)
+
+```
+[OCR Pipeline Output dir: ____]  [Confidence threshold: 0.85]  [Ingest]
+  
+Table of uncertain predictions:
+  | Image | OCR text          | Confidence | Your correction | Save |
+  | [img] | "Rechung 18. Ap"  | 0.73       | [____________]  | [✓]  |
+  | [img] | "Beschlußprotoko" | 0.69       | [____________]  | [✓]  |
+
+[Export corrections as training dataset]  [Start LoRA retrain]
+```
+
+### Connection to the active learning loop
+
+Your OCR pipeline already has:
+- `abstention logic` (the "council abstains" on uncertain predictions)
+- `acceptance-gated fine-tuning`
+- `RAG-based post-correction`
+
+The workbench hook provides the missing UI layer: human-in-the-loop corrections that feed
+the acceptance gate. The `FieldNoteStore.to_hf_dataset()` output plugs directly into the
+pipeline's `acceptance-gated fine-tuning` step.
+
+---
+
+## 9. MiniCPM Desk-Pet
+
+### What it is
+
+OpenBMB ships `MiniCPM-Desk-Pet`, a desktop companion app powered by MiniCPM5-1B, alongside
+the model release (2026-05-19). Key features:
+- Runs locally on Apple Silicon, NVIDIA GPU, or CPU
+- LoRA persona switching — different personalities loaded as adapters
+- Integrates with coding agents (Cursor, Claude Code, Codex)
+- Tiny footprint (~2GB VRAM with Q4_K_M)
+
+The workbench extension lets you train LoRA personas directly and export them to the Desk-Pet
+format.
+
+### What "persona" means here
+
+A LoRA persona is a small adapter (rank 8–16) trained on ~100–500 conversation examples in a
+specific voice or style. Examples:
+- "Botanist assistant" — answers in scientific plant terminology
+- "Friendly field guide" — casual, encouraging tone for beginners
+- "Historical document expert" — formal, precise, citation-aware (connects to OCR pipeline)
+
+Training data is small enough that synthetic gen (Extension 4) can produce it in minutes.
+
+### Architecture
+
+```
+agent/desk_pet.py
+  DeskPetExporter
+    .train_persona(name, style_desc, n_examples) → LoRA checkpoint
+    .export_to_deskpet(checkpoint_path) → deskpet_compatible.gguf
+    .list_personas() → [PersonaMeta]
+    .load_persona(name) → activates adapter in current session
+```
+
+```python
+# agent/desk_pet.py
+from dataclasses import dataclass
+from pathlib import Path
+import json, shutil
+
+@dataclass
+class PersonaMeta:
+    name:        str
+    description: str
+    checkpoint:  str
+    gguf_path:   str | None = None
+    n_examples:  int = 0
+
+class DeskPetExporter:
+    PERSONA_DIR = Path("data/personas")
+
+    def __init__(self, base_model_id: str = "openbmb/MiniCPM5-1B"):
+        self.base_model_id = base_model_id
+        self.PERSONA_DIR.mkdir(parents=True, exist_ok=True)
+
+    def train_persona(
+        self,
+        name:        str,
+        style_desc:  str,
+        n_examples:  int = 200,
+        lora_rank:   int = 8,      # small rank: personas need only ~50-100 examples
+    ) -> str:
+        """
+        1. Use SyntheticGenerator to create conversation examples in the persona style.
+        2. Fine-tune MiniCPM5-1B LoRA.
+        3. Save checkpoint.
+        Returns checkpoint path.
+        """
+        from datasets.synthetic import SyntheticGenerator
+        from training.lora import LoRATextTrainer
+        from models.minicpm_text import MiniCPMTextService
+        import torch
+
+        # Generate persona training data
+        from transformers import AutoModelForCausalLM, AutoTokenizer
+        base_model = AutoModelForCausalLM.from_pretrained(
+            self.base_model_id, torch_dtype=torch.bfloat16, device_map="auto"
+        )
+        base_tok = AutoTokenizer.from_pretrained(self.base_model_id)
+
+        gen_svc = MiniCPMTextService.__new__(MiniCPMTextService)
+        gen_svc.model, gen_svc.tokenizer = base_model, base_tok
+
+        synth = SyntheticGenerator(gen_svc)
+        ds = synth.generate(
+            topic=f"Conversation examples in the style of: {style_desc}. "
+                  f"Each example: a user message and a response in that persona's voice.",
+            n=n_examples,
+            schema={"instruction": "user message", "response": "persona reply"},
+        )
+
+        # LoRA fine-tune
+        trainer = LoRATextTrainer(cfg={
+            "lora_rank":      lora_rank,
+            "lora_alpha":     lora_rank * 2,
+            "epochs":         2,
+            "batch_size":     8,
+            "grad_accum":     2,
+        })
+        output_dir = str(self.PERSONA_DIR / name / "checkpoint")
+        trainer.train(base_model, base_tok, ds, run_name=f"persona_{name}")
+
+        # Save metadata
+        meta = PersonaMeta(
+            name=name, description=style_desc,
+            checkpoint=output_dir, n_examples=n_examples
+        )
+        (self.PERSONA_DIR / name / "meta.json").write_text(
+            json.dumps(meta.__dict__, indent=2)
+        )
+        return output_dir
+
+    def export_to_deskpet(self, persona_name: str) -> str:
+        """
+        Merge LoRA into base weights, then export as GGUF for Desk-Pet.
+        Returns path to merged GGUF.
+        """
+        from peft import PeftModel
+        from transformers import AutoModelForCausalLM, AutoTokenizer
+        import torch
+        from training.export import GGUFExporter
+
+        meta_path = self.PERSONA_DIR / persona_name / "meta.json"
+        meta      = PersonaMeta(**json.loads(meta_path.read_text()))
+
+        # Merge LoRA into base
+        base  = AutoModelForCausalLM.from_pretrained(
+            self.base_model_id, torch_dtype=torch.bfloat16
+        )
+        peft_model = PeftModel.from_pretrained(base, meta.checkpoint)
+        merged = peft_model.merge_and_unload()
+
+        merged_path = str(self.PERSONA_DIR / persona_name / "merged")
+        merged.save_pretrained(merged_path)
+        AutoTokenizer.from_pretrained(self.base_model_id).save_pretrained(merged_path)
+
+        # Export GGUF
+        exporter  = GGUFExporter()
+        gguf_path = exporter.export(
+            model_path=merged_path,
+            output_dir=str(self.PERSONA_DIR / persona_name / "gguf"),
+            quant="Q4_K_M",
+            model_type="text",
+        )[1]   # [0] = F16, [1] = quantized
+
+        # Update metadata
+        meta.gguf_path = gguf_path
+        meta_path.write_text(json.dumps(meta.__dict__, indent=2))
+        return gguf_path
+```
+
+### Usage flow
+
+```
+1. UI: "New persona" → enter name + style description
+2. Synthetic gen: 200 examples of that voice → fine-tune LoRA (rank 8, ~10 min on RTX)
+3. Export → Q4_K_M GGUF
+4. Copy to Desk-Pet personas/ dir
+5. Desk-Pet: switch persona → instant personality change
+```
+
+---
+
+## 10. MiniCPM-o Audio Tab
+
+### What it is
+
+MiniCPM-o-4.5 (released 2026-05-17) is a true omnimodal model — it sees, listens, and speaks
+simultaneously in real-time. It supports proactive interactions (proactive reminding) and
+real-time conversation with both visual and audio input.
+
+This extension adds a new Gradio tab with a microphone + camera (or image) interface, streaming
+audio output, and real-time MiniCPM-o inference.
+
+### Architecture
+
+```
+ui/audio_tab.py
+  OmnimodalTab
+    .build() → gr.Column with audio+image inputs and streaming output
+    
+models/minicpm_omni.py
+  MiniCPMOmniService
+    .stream_chat(audio_bytes, image=None, text=None) → Generator[str]
+    .speak(text) → bytes  (TTS for audio output)
+```
+
+```python
+# models/minicpm_omni.py
+import torch
+import numpy as np
+from transformers import AutoProcessor, AutoModel
+
+class MiniCPMOmniService:
+    """
+    MiniCPM-o-4.5: omnimodal service.
+    Handles text + image + audio simultaneously.
+    """
+    MODEL_ID = "openbmb/MiniCPM-o-4.5"
+
+    def __init__(self, cfg: dict):
+        self.processor = AutoProcessor.from_pretrained(
+            self.MODEL_ID, trust_remote_code=True
+        )
+        self.model = AutoModel.from_pretrained(
+            self.MODEL_ID,
+            trust_remote_code=True,
+            torch_dtype=torch.bfloat16,
+            device_map="auto",
+        )
+        self.thinking = cfg.get("thinking_mode", False)
+
+    def chat(
+        self,
+        text:        str | None = None,
+        image=None,             # PIL Image
+        audio:       np.ndarray | None = None,
+        sample_rate: int = 16000,
+    ) -> str:
+        """
+        Full omnimodal chat: pass any combination of text, image, audio.
+        MiniCPM-o-4.5 handles them natively.
+        """
+        content = []
+        if image is not None:
+            content.append({"type": "image", "image": image})
+        if audio is not None:
+            content.append({
+                "type":        "audio",
+                "audio":       audio,
+                "sample_rate": sample_rate,
+            })
+        if text:
+            content.append({"type": "text", "text": text})
+
+        messages = [{"role": "user", "content": content}]
+        inputs   = self.processor.apply_chat_template(
+            messages,
+            add_generation_prompt=True,
+            tokenize=True,
+            return_tensors="pt",
+            return_dict=True,
+            chat_template_kwargs={"enable_thinking": self.thinking},
+        ).to(self.model.device)
+
+        with torch.no_grad():
+            outputs = self.model.generate(**inputs, max_new_tokens=512)
+        return self.processor.decode(
+            outputs[0][inputs["input_ids"].shape[1]:],
+            skip_special_tokens=True
+        )
+```
+
+```python
+# ui/audio_tab.py
+import gradio as gr
+import numpy as np
+
+def build_audio_tab(model_registry):
+    with gr.Column():
+        gr.Markdown("### 🎙️ Omnimodal — MiniCPM-o-4.5")
+        gr.Markdown("Speak, show an image, or type — all at once.")
+
+        with gr.Row():
+            audio_in  = gr.Audio(
+                sources=["microphone"],
+                type="numpy",
+                label="Microphone input",
+                streaming=True,
+            )
+            image_in  = gr.Image(
+                type="pil",
+                label="Optional image",
+            )
+
+        text_in   = gr.Textbox(label="Optional text", placeholder="Or type here...")
+        submit    = gr.Button("Send")
+        output    = gr.Textbox(label="Response", lines=8)
+        audio_out = gr.Audio(label="Spoken response (TTS)", autoplay=True)
+
+        def respond(audio_data, image, text):
+            svc = model_registry.get("minicpm_o45")
+            audio_arr = None
+            if audio_data is not None:
+                sr, arr = audio_data
+                audio_arr = arr.astype(np.float32) / 32768.0
+            response = svc.chat(text=text, image=image, audio=audio_arr)
+            # Optional: TTS for spoken output
+            # spoken = tts(response)
+            return response, None   # None = no audio out yet
+
+        submit.click(respond, [audio_in, image_in, text_in], [output, audio_out])
+```
+
+### Real-time streaming version (advanced)
+
+For true real-time conversation (proactive reminding, interrupt detection):
+
+```python
+# streaming audio inference using gradio's streaming audio + SSE
+demo = gr.Interface(
+    fn=stream_omni_response,
+    inputs=[
+        gr.Audio(streaming=True, sources=["microphone"]),
+        gr.Image(type="pil"),
+    ],
+    outputs=gr.Textbox(),
+    live=True,
+)
+```
+
+### TTS for audio output
+
+MiniCPM-o-4.5 has its own audio generation capability — check the model card for the
+`generate_audio` method. Fallback: use `kokoro-82M` (Apache 2.0, 82M, runs locally) for TTS.
+
+```python
+# Kokoro TTS fallback (lightweight, local)
+from kokoro import KPipeline
+tts_pipe = KPipeline(lang_code="en-us")
+audio, sr = tts_pipe(response_text, voice="af_heart")
+```
+
+---
+
+## 11. Cross-Extension Wiring
+
+Most extensions are independent, but several combinations unlock powerful compound workflows:
+
+### OCR → VINDEX
+
+Low-confidence OCR outputs → Field Notes → human corrections → LoRA retrain. But additionally:
+if the OCR model consistently misreads a specific character class, use VINDEX's `logit_lens` to
+identify which layer is responsible, then `calibrated_edit` to target that slot directly — a
+faster fix than a full retraining cycle.
+
+### Synthetic Gen → Reward Model → DPO
+
+```
+SyntheticGenerator.generate(topic, n=1000)
+    → RewardEvaluator.create_dpo_pairs(prompts, generator, n=4)
+    → DPO training via TRL DPOTrainer
+    → Trackio logs win rate
+    → VINDEX verify alignment not broken
+```
+
+### Paper Agent → Desk-Pet Persona
+
+```
+PaperAgent reads: "Persona-based dialogue systems for domain experts"
+    → Implements: persona training data format
+    → DeskPetExporter.train_persona("expert_botanist", "...")
+    → Export GGUF → load in Desk-Pet
+```
+
+### HF Spaces + vLLM + Trackio
+
+```
+SpacesDeployer.deploy(hardware="a10")   # production GPU
+    → app.py runs VLLMRunner on A10G
+    → All requests logged via Trackio with space_id=deployed_space
+    → Dashboard visible to collaborators at trackio Space URL
+```
+
+### Full active-learning loop (all extensions combined)
+
+```
+MiniCPM-o Audio Tab: user speaks + shows image of plant
+        ↓
+OmnimodalService: identify species (low confidence)
+        ↓
+OCRPipelineLoader: auto-create Field Note (uncertain prediction)
+        ↓
+Human: corrects species name in UI
+        ↓
+SyntheticGenerator: augment with 50 similar examples
+        ↓
+RewardEvaluator: filter synthetic examples
+        ↓
+LoRATextTrainer + TRL + Trackio: fine-tune
+        ↓
+VINDEX: verify the target species slot was corrected
+        ↓
+DeskPetExporter: export updated persona
+        ↓
+SpacesDeployer: push updated app to HF Spaces
+```
+
+---
+
+*Extensions spec v1.0 — Christof Kaller / ki-fusion-labs.de — 2026-06-05*

HF_PRD_v1.md

@@ -0,0 +1,1361 @@
+# OpenBMB Local AI Workbench — Technical PRD v2.0
+> Revised: 2026-06-05 | Corrections from online research embedded inline
+
+---
+
+## Table of Contents
+
+1. [Purpose & Design Philosophy](#1-purpose--design-philosophy)
+2. [Design Principles (Template Architecture)](#2-design-principles-template-architecture)
+3. [System Architecture](#3-system-architecture)
+4. [Model Registry](#4-model-registry)
+5. [Inference Stack — Five Modes](#5-inference-stack--five-modes)
+6. [Tracking: Trackio (Corrected API)](#6-tracking-trackio-corrected-api)
+7. [MCP Layer (Three Integration Paths)](#7-mcp-layer-three-integration-paths)
+8. [Training Pipeline](#8-training-pipeline)
+9. [Export & Quantization](#9-export--quantization)
+10. [Agent Mode (ml-intern Inspired)](#10-agent-mode-ml-intern-inspired)
+11. [UI — Tab Specification](#11-ui--tab-specification)
+12. [Field Notes & Correction Loop](#12-field-notes--correction-loop)
+13. [Directory Structure](#13-directory-structure)
+14. [Configuration Schema](#14-configuration-schema)
+15. [Dependencies](#15-dependencies)
+16. [Hackathon Demo Flow](#16-hackathon-demo-flow)
+17. [Corrections from PRD v1](#17-corrections-from-prd-v1)
+18. [Roadmap & Extension Points](#18-roadmap--extension-points)
+
+---
+
+## 1. Purpose & Design Philosophy
+
+**One-line:** A modular AI experimentation platform for the OpenBMB model family that covers
+dataset ingestion → LoRA training → evaluation → GGUF export → llama.cpp/SGLang deployment →
+multimodal inference → MCP tool exposure → trace sharing — in a single Gradio app.
+
+**Broader framing:** The platform is deliberately designed as a *template*. The `config/models.yaml`
+and `config/training.yaml` files drive almost all model-specific behaviour. Replacing the OpenBMB
+section with any other model family (Qwen, Phi, Gemma …) should require only config changes and
+a matching service class, never core rewrites.
+
+**Why OpenBMB?**
+- MiniCPM5-1B (2026-05-19): SOTA 1B on-device, 128K context, LlamaForCausalLM — no custom kernels
+- MiniCPM4.1-8B (2025-09): sparse InfLLM-v2 attention, ~7× long-context speedup vs Qwen3-8B
+- MiniCPM-V-4.6 (2026-05-11): 1.3B VLM, SigLIP2-400M + Qwen3.5-0.8B, 262K ctx, mixed 4×/16× token compression
+- MiniCPM-V-4.6-Thinking: same architecture + long-CoT for multimodal reasoning
+- MiniCPM-o 4.5 (2026-05-17): omnimodal (vision + audio), real-time conversation
+- Apache 2.0 across the board; vLLM / SGLang / llama.cpp / Ollama native support
+
+---
+
+## 2. Design Principles (Template Architecture)
+
+| Principle | Implementation |
+|-----------|----------------|
+| Config-driven | `models.yaml` + `training.yaml` define all models; no model names hardcoded in logic |
+| Event-driven | `core/events.py` — typed event bus; all cross-module comms via events |
+| Registry pattern | `core/registry.py` — all services register by name; swap without restart |
+| Separation of concerns | `models/` = loading; `training/` = train/eval; `ui/` = Gradio; `mcp/` = protocol |
+| Plugin tools | Any Python function decorated `@mcp.tool()` is auto-exposed; no wiring needed |
+| Async-first | FastAPI backend, Gradio queuing, non-blocking inference |
+| Observability | Every significant action fires a Trackio log + event |
+| Template portability | Replace `config/models.yaml` → different model family; keep all else |
+
+---
+
+## 3. System Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                        OpenBMB Local AI Workbench                           │
+├───────────────────────────┬─────────────────────────────────────────────────┤
+│        Gradio UI Layer    │          Backend Service Layer                  │
+│  ┌─────────────────────┐  │  ┌─────────────────────────────────────────┐   │
+│  │  gr.Blocks / Tabs   │  │  │  gradio.Server (FastAPI-based)          │   │
+│  │  chat_tab.py        │  │  │  @app.api()   — Gradio-queued endpoints │   │
+│  │  vision_tab.py      │  │  │  @app.mcp.tool() — MCP registration     │   │
+│  │  train_tab.py       │  │  │  @app.get("/") — static/custom frontend │   │
+│  │  dataset_tab.py     │  │  └─────────────────────────────────────────┘   │
+│  │  export_tab.py      │  │                                                 │
+│  │  traces_tab.py      │  │  ┌─────────────────────────────────────────┐   │
+│  │  agent_tab.py       │  │  │  FastMCP server (standalone / bridged)  │   │
+│  └─────────────────────┘  │  │  mcp = FastMCP("OpenBMBWorkbench")      │   │
+│                           │  │  Registered tools (see §7)              │   │
+│  SSE MCP endpoint:        │  └─────────────────────────────────────────┘   │
+│  /gradio_api/mcp/sse      │                                                 │
+└───────────────────────────┴─────────────────────────────────────────────────┘
+          │                                │
+          ▼                                ▼
+┌─────────────────────┐        ┌────────────────────────┐
+│   Model Services    │        │   Tracking & Storage   │
+│  minicpm_text.py    │        │  trackio (local/Space) │
+│  minicpm_vision.py  │        │  field_notes/ (SQLite) │
+│  llama_cpp_runner.py│        │  exports/ (GGUF files) │
+│  sglang_runner.py   │        │  data/ (HF datasets)   │
+│  ollama_runner.py   │        └────────────────────────┘
+└─────────────────────┘
+          │
+          ▼
+┌───────────────────────────────────────────────────────────┐
+│                   Inference Backends                      │
+│  Transformers  │  llama.cpp  │  SGLang  │  vLLM  │ Ollama│
+└───────────────────────────────────────────────────────────┘
+```
+
+### 3.1 Event Bus (core/events.py)
+
+```python
+from dataclasses import dataclass, field
+from enum import Enum
+from typing import Any, Callable
+import asyncio
+
+class EventType(str, Enum):
+    DATASET_LOADED      = "dataset_loaded"
+    TRAINING_STARTED    = "training_started"
+    TRAINING_STEP       = "training_step"
+    TRAINING_FINISHED   = "training_finished"
+    EVAL_FINISHED       = "eval_finished"
+    INFERENCE_REQUEST   = "inference_request"
+    INFERENCE_RESPONSE  = "inference_response"
+    TOOL_CALL           = "tool_call"
+    MODEL_LOADED        = "model_loaded"
+    MODEL_SWITCHED      = "model_switched"
+    EXPORT_STARTED      = "export_started"
+    EXPORT_FINISHED     = "export_finished"
+    FIELD_NOTE_SAVED    = "field_note_saved"
+    AGENT_STEP          = "agent_step"
+
+@dataclass
+class Event:
+    type: EventType
+    payload: dict[str, Any] = field(default_factory=dict)
+    run_id: str = ""
+
+class EventBus:
+    _handlers: dict[EventType, list[Callable]] = {}
+
+    def on(self, event_type: EventType):
+        """Decorator to register an event handler."""
+        def decorator(fn: Callable):
+            self._handlers.setdefault(event_type, []).append(fn)
+            return fn
+        return decorator
+
+    async def emit(self, event: Event):
+        for handler in self._handlers.get(event.type, []):
+            await handler(event)
+
+bus = EventBus()
+```
+
+### 3.2 Service Registry (core/registry.py)
+
+```python
+from typing import TypeVar, Generic, Type
+
+T = TypeVar("T")
+
+class Registry(Generic[T]):
+    def __init__(self):
+        self._services: dict[str, T] = {}
+
+    def register(self, name: str, service: T):
+        self._services[name] = service
+
+    def get(self, name: str) -> T:
+        if name not in self._services:
+            raise KeyError(f"Service '{name}' not registered. Available: {list(self._services)}")
+        return self._services[name]
+
+    def list(self) -> list[str]:
+        return list(self._services.keys())
+
+# Singleton registries
+model_registry   = Registry()   # model services
+dataset_registry = Registry()   # dataset loaders
+tool_registry    = Registry()   # MCP tools
+```
+
+---
+
+## 4. Model Registry
+
+### 4.1 Text Models
+
+| Config ID | HF ID | Architecture | Context | Notes |
+|-----------|-------|-------------|---------|-------|
+| `minicpm5_1b` | `openbmb/MiniCPM5-1B` | LlamaForCausalLM | 128K | Standard; no custom kernels; XML tool calls; SGLang `minicpm5` parser |
+| `minicpm5_1b_thinking` | `openbmb/MiniCPM5-1B-Thinking` | LlamaForCausalLM | 128K | CoT mode; `chat_template_kwargs={"enable_thinking": True}` |
+| `minicpm41_8b` | `openbmb/MiniCPM4.1-8B` | Sparse InfLLM-v2 | 128K (YaRN) | `--trust-remote-code` required; 7× faster long-ctx vs Qwen3-8B |
+
+### 4.2 Vision Models
+
+| Config ID | HF ID | Backbone | Context | Notes |
+|-----------|-------|---------|---------|-------|
+| `minicpm_v46` | `openbmb/MiniCPM-V-4.6` | SigLIP2-400M + Qwen3.5-0.8B | 262K | `transformers>=5.7.0` + `torchcodec`; mixed 4×/16× compression |
+| `minicpm_v46_thinking` | `openbmb/MiniCPM-V-4.6-Thinking` | same + CoT head | 262K | `enable_thinking=True` in chat_template_kwargs |
+| `minicpm_o45` | `openbmb/MiniCPM-o-4.5` | Omnimodal | - | Vision + Audio; real-time conversation; see §4.4 |
+
+### 4.3 Model Architecture Notes (critical for correct code)
+
+**MiniCPM5-1B** — standard Llama, loads with:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-1B", trust_remote_code=True)
+```
+
+**MiniCPM-V-4.6** — vision-language, loads with:
+```python
+# ⚠️ CORRECTION from v1: NOT AutoModelForCausalLM — use AutoModelForImageTextToText
+from transformers import AutoProcessor, AutoModelForImageTextToText
+
+processor = AutoProcessor.from_pretrained("openbmb/MiniCPM-V-4.6", trust_remote_code=True)
+model = AutoModelForImageTextToText.from_pretrained(
+    "openbmb/MiniCPM-V-4.6",
+    trust_remote_code=True,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+```
+
+Chat format (vision):
+```python
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "image": pil_image},   # PIL Image
+            {"type": "text",  "text": prompt}
+        ]
+    }
+]
+inputs = processor.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=True,
+    return_tensors="pt",
+    return_dict=True,
+    chat_template_kwargs={"enable_thinking": False}   # True for Thinking variant
+).to(model.device)
+```
+
+**MiniCPM4.1-8B** — requires `trust_remote_code=True` for sparse attention kernel.
+
+### 4.4 models.yaml Schema
+
+```yaml
+# config/models.yaml
+# ─── Template: add any HF model here ──────────────────────────────────────
+models:
+  minicpm5_1b:
+    hf_id:          openbmb/MiniCPM5-1B
+    type:           text                        # text | vision | omnimodal
+    architecture:   llama                       # llama | minicpm41 | minicpm_v
+    context_length: 131072
+    dtype:          bfloat16
+    trust_remote_code: false                    # LlamaForCausalLM — no fork
+    thinking_mode:  false                       # set true for Thinking checkpoints
+    tool_call_parser: minicpm5                  # for SGLang serving
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM5-1B-GGUF
+      default_quant: Q4_K_M
+
+  minicpm41_8b:
+    hf_id:          openbmb/MiniCPM4.1-8B
+    type:           text
+    architecture:   minicpm41                   # sparse InfLLM-v2
+    context_length: 131072
+    dtype:          bfloat16
+    trust_remote_code: true
+    thinking_mode:  false
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM4.1-8B-GGUF
+      default_quant: Q4_K_M
+
+  minicpm_v46:
+    hf_id:          openbmb/MiniCPM-V-4.6
+    type:           vision
+    architecture:   minicpm_v
+    context_length: 262144
+    dtype:          bfloat16
+    trust_remote_code: true
+    thinking_mode:  false
+    vision_encoder: siglip2_400m
+    token_compression: "4x_16x"               # mixed compression
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM-V-4.6-gguf        # official repo — no surgery script needed
+      main_file:  MiniCPM-V-4.6-Q4_K_M.gguf
+      mmproj:     mmproj-MiniCPM-V-4.6-F16.gguf
+      default_quant: Q4_K_M
+
+  minicpm_v46_thinking:
+    hf_id:          openbmb/MiniCPM-V-4.6-Thinking
+    type:           vision
+    architecture:   minicpm_v
+    context_length: 262144
+    dtype:          bfloat16
+    trust_remote_code: true
+    thinking_mode:  true
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM-V-4.6-Thinking-gguf
+      default_quant: Q4_K_M
+
+  minicpm_o45:
+    hf_id:          openbmb/MiniCPM-o-4.5
+    type:           omnimodal
+    architecture:   minicpm_o
+    context_length: 131072
+    dtype:          bfloat16
+    trust_remote_code: true
+    thinking_mode:  false
+    note: "Vision + Audio; real-time conversation mode"
+```
+
+---
+
+## 5. Inference Stack — Five Modes
+
+### Mode A: Transformers (train + eval + fine-tuning)
+
+Used for: LoRA training, evaluation, PEFT
+Load text: `AutoModelForCausalLM`
+Load vision: `AutoModelForImageTextToText`  ← corrected from v1
+
+```python
+# models/minicpm_text.py
+class MiniCPMTextService:
+    def __init__(self, model_id: str, cfg: dict):
+        self.model = AutoModelForCausalLM.from_pretrained(
+            model_id,
+            torch_dtype=torch.bfloat16,
+            device_map="auto",
+            trust_remote_code=cfg.get("trust_remote_code", False)
+        )
+        self.tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+```
+
+```python
+# models/minicpm_vision.py  ← new dedicated service
+class MiniCPMVisionService:
+    def __init__(self, model_id: str, cfg: dict):
+        self.processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
+        self.model = AutoModelForImageTextToText.from_pretrained(
+            model_id,
+            torch_dtype=torch.bfloat16,
+            device_map="auto",
+            trust_remote_code=cfg.get("trust_remote_code", True)
+        )
+        self.thinking = cfg.get("thinking_mode", False)
+
+    def chat(self, messages: list[dict], max_new_tokens: int = 512) -> str:
+        inputs = self.processor.apply_chat_template(
+            messages,
+            add_generation_prompt=True,
+            tokenize=True,
+            return_tensors="pt",
+            return_dict=True,
+            chat_template_kwargs={"enable_thinking": self.thinking}
+        ).to(self.model.device)
+        with torch.no_grad():
+            outputs = self.model.generate(**inputs, max_new_tokens=max_new_tokens)
+        response = self.processor.decode(
+            outputs[0][inputs["input_ids"].shape[1]:],
+            skip_special_tokens=True
+        )
+        return response
+```
+
+### Mode B: llama.cpp (local deployment, CPU/GPU offload)
+
+> ⚠️ CORRECTION from v1: For MiniCPM-V 4.6, official GGUFs are already published.
+> Download from `openbmb/MiniCPM-V-4.6-gguf` — no need to run `convert_hf_to_gguf.py` yourself.
+> The `minicpmv-surgery.py` script was for the older MiniCPM-o-2.6; **do not use it for V4.6**.
+
+**Text models** → `llama-server` (standard OpenAI-compatible endpoint)
+**Vision models** → `llama-server --mmproj <mmproj.gguf>` (multimodal server)
+
+```python
+# models/llama_cpp_runner.py
+import subprocess, requests, time
+
+class LlamaCppDeployment:
+    def __init__(self, cfg: dict):
+        self.model_path  = cfg["model_path"]
+        self.mmproj_path = cfg.get("mmproj_path")      # None for text models
+        self.port        = cfg.get("port", 8080)
+        self._proc       = None
+
+    def start(self):
+        cmd = [
+            "llama-server",
+            "--model", self.model_path,
+            "--port", str(self.port),
+            "--ctx-size", "8192",
+            "--n-gpu-layers", "-1",       # full GPU offload if available
+        ]
+        if self.mmproj_path:
+            cmd += ["--mmproj", self.mmproj_path]   # enables vision
+        self._proc = subprocess.Popen(cmd)
+        self._wait_ready()
+
+    def _wait_ready(self, timeout: int = 30):
+        for _ in range(timeout):
+            try:
+                r = requests.get(f"http://127.0.0.1:{self.port}/health")
+                if r.status_code == 200:
+                    return
+            except Exception:
+                pass
+            time.sleep(1)
+        raise RuntimeError("llama-server did not start in time")
+
+    def chat(self, messages: list[dict]) -> str:
+        r = requests.post(
+            f"http://127.0.0.1:{self.port}/v1/chat/completions",
+            json={"messages": messages, "max_tokens": 512}
+        )
+        return r.json()["choices"][0]["message"]["content"]
+
+    def stop(self):
+        if self._proc:
+            self._proc.terminate()
+```
+
+**GGUF Export** (only needed if building custom quants):
+```bash
+# For MiniCPM-V 4.6 — convert_hf_to_gguf.py (NOT legacy surgery scripts)
+python convert_hf_to_gguf.py openbmb/MiniCPM-V-4.6 --outtype f16
+# Output: MiniCPM-V-4.6-F16.gguf  +  mmproj-MiniCPM-V-4.6-F16.gguf
+
+# Quantize
+./llama-quantize MiniCPM-V-4.6-F16.gguf MiniCPM-V-4.6-Q4_K_M.gguf Q4_K_M
+```
+
+Supported quants: `Q4_K_M`, `Q5_K_M`, `Q8_0`, `Q2_K`, `F16`
+
+### Mode C: SGLang (tool-use, function calling, recommended for MiniCPM5)
+
+MiniCPM5-1B emits XML-style tool calls. SGLang has a native `minicpm5` parser:
+
+```bash
+python -m sglang.launch_server \
+  --model-path openbmb/MiniCPM5-1B \
+  --port 30000 \
+  --tool-call-parser minicpm5     # or: --tool-call-parser auto
+```
+
+For MiniCPM-V-4.6 vision:
+```bash
+python -m sglang.launch_server \
+  --model-path openbmb/MiniCPM-V-4.6-Thinking \
+  --port 30000 \
+  --enable-auto-tool-choice \
+  --tool-call-parser qwen3_coder \
+  --default-chat-template-kwargs '{"enable_thinking": true}'
+```
+
+### Mode D: vLLM (batch serving, production)
+
+```bash
+vllm serve openbmb/MiniCPM4.1-8B --trust-remote-code
+vllm serve openbmb/MiniCPM-V-4.6 --trust-remote-code
+```
+
+### Mode E: Ollama (zero-config, consumer hardware)
+
+```bash
+ollama run openbmb/minicpm-v4.6    # vision + text
+ollama run openbmb/minicpm5-1b     # text only
+```
+
+### Mode Selection Matrix
+
+| Use case | Recommended mode |
+|----------|-----------------|
+| LoRA training | A (Transformers) |
+| Evaluation | A (Transformers) |
+| Local chat (fast) | B (llama.cpp) |
+| Tool/function calling | C (SGLang) |
+| Production batch | D (vLLM) |
+| Consumer demo | E (Ollama) |
+| Vision inference local | B (llama-server --mmproj) |
+| Vision fine-tuning | A (SWIFT / LLaMA-Factory) |
+
+---
+
+## 6. Tracking: Trackio (Corrected API)
+
+> ⚠️ CRITICAL CORRECTION from v1: `trackio.trace()` **does not exist**.
+> There is no context-manager trace API in Trackio. The v1 PRD example was wrong.
+> Correct API: `init()` / `log()` / `finish()` / `alert()` / `Table()` / `Markdown()`
+
+Trackio is a wandb drop-in replacement, local-first, free, Gradio dashboard, HF Spaces sync.
+
+```python
+# tracking/trackio_service.py
+import trackio
+import time
+from core.events import bus, EventType, Event
+
+class TrackioService:
+    def __init__(self, project: str, space_id: str | None = None):
+        self.project  = project
+        self.space_id = space_id
+        self._active  = False
+        self._register_handlers()
+
+    def _register_handlers(self):
+        @bus.on(EventType.TRAINING_STARTED)
+        async def on_training_started(event: Event):
+            trackio.init(
+                project=self.project,
+                space_id=self.space_id,
+                run_name=event.payload.get("run_name", "training"),
+            )
+            self._active = True
+
+        @bus.on(EventType.TRAINING_STEP)
+        async def on_step(event: Event):
+            if self._active:
+                trackio.log(event.payload)
+
+        @bus.on(EventType.TRAINING_FINISHED)
+        async def on_done(event: Event):
+            if self._active:
+                trackio.log({"status": "finished", **event.payload})
+                trackio.finish()
+                self._active = False
+
+        @bus.on(EventType.EVAL_FINISHED)
+        async def on_eval(event: Event):
+            trackio.init(
+                project=self.project,
+                space_id=self.space_id,
+                run_name=event.payload.get("eval_run_name", "eval"),
+            )
+            trackio.log(event.payload)
+            trackio.finish()
+
+        @bus.on(EventType.INFERENCE_RESPONSE)
+        async def on_inference(event: Event):
+            # Log inference latency and token count
+            trackio.init(
+                project=self.project,
+                space_id=self.space_id,
+                run_name="inference",
+            )
+            trackio.log({
+                "model":          event.payload.get("model"),
+                "latency_ms":     event.payload.get("latency_ms"),
+                "tokens_out":     event.payload.get("tokens_out"),
+                "mode":           event.payload.get("mode", "text"),
+            })
+            trackio.finish()
+
+        @bus.on(EventType.TOOL_CALL)
+        async def on_tool(event: Event):
+            trackio.alert(
+                title=f"Tool call: {event.payload.get('tool_name')}",
+                level=trackio.AlertLevel.INFO,
+            )
+```
+
+**TRL LoRA integration** (recommended for training):
+```python
+from trl import SFTConfig
+training_args = SFTConfig(
+    output_dir="./checkpoints",
+    report_to="trackio",   # ← one-liner Trackio integration via TRL
+    run_name="lora_minicpm5_1b",
+)
+```
+
+**SQL query via CLI** (LLM-friendly):
+```bash
+trackio query project --project workbench --sql "SELECT run_name, loss FROM metrics ORDER BY loss LIMIT 5"
+```
+
+---
+
+## 7. MCP Layer (Three Integration Paths)
+
+### Path 1: Gradio Blocks + `launch(mcp_server=True)` ← simplest
+
+```python
+# app.py (simple path)
+import gradio as gr
+from mcp.server.fastmcp import FastMCP
+
+demo = gr.Blocks()
+# ... tabs ...
+demo.launch(
+    mcp_server=True,    # exposes /gradio_api/mcp/sse
+    server_port=7860,
+)
+```
+
+Every `gr.Interface` function and `gr.api()` endpoint becomes an MCP tool automatically.
+Docstrings become tool descriptions. Type hints become parameter schemas.
+
+### Path 2: `gradio.Server` (custom frontend + MCP)
+
+> New API published 2026-04-01. Extends FastAPI. Better for custom dashboards.
+
+```python
+# app.py (Server path)
+from gradio import Server
+from fastapi.responses import HTMLResponse
+
+app = Server()
+
+@app.api(name="run_inference")             # Gradio-queued + MCP-compatible
+def run_inference(prompt: str, model_id: str) -> str:
+    """Run text inference on a MiniCPM text model."""
+    svc = model_registry.get(model_id)
+    return svc.generate(prompt)
+
+@app.mcp.tool()                            # explicit MCP tool registration
+async def export_gguf(model_id: str, quant: str = "Q4_K_M") -> str:
+    """Export a Transformers model to GGUF format with specified quantization."""
+    return await export_service.run(model_id, quant)
+
+@app.get("/", response_class=HTMLResponse) # custom frontend
+async def homepage():
+    return open("ui/index.html").read()
+
+app.launch(show_error=True)
+```
+
+### Path 3: FastMCP standalone + Gradio client bridge
+
+Use when you need stateful MCP tools, or when running multiple Gradio apps and want to save memory:
+
+```python
+# mcp/server.py
+from mcp.server.fastmcp import FastMCP
+from gradio_client import Client
+
+mcp = FastMCP("OpenBMBWorkbench")
+_clients: dict[str, Client] = {}
+
+def _gradio_client(space_id: str) -> Client:
+    if space_id not in _clients:
+        _clients[space_id] = Client(space_id)
+    return _clients[space_id]
+
+@mcp.tool()
+async def calculate(expression: str) -> str:
+    """Evaluate a mathematical expression."""
+    import math
+    return str(eval(expression, {"__builtins__": {}}, vars(math)))
+
+@mcp.tool()
+async def dataset_stats(dataset_name: str, split: str = "train") -> dict:
+    """Return basic statistics for a HuggingFace dataset."""
+    from datasets import load_dataset
+    ds = load_dataset(dataset_name, split=split)
+    return {
+        "rows":    len(ds),
+        "columns": ds.column_names,
+        "features": str(ds.features),
+    }
+
+@mcp.tool()
+async def search_dataset(query: str, max_results: int = 5) -> list[dict]:
+    """Search HuggingFace Hub for datasets matching a query."""
+    from huggingface_hub import list_datasets
+    results = list(list_datasets(search=query, limit=max_results))
+    return [{"id": d.id, "downloads": d.downloads, "tags": d.tags} for d in results]
+
+@mcp.tool()
+async def start_training(
+    model_id: str,
+    dataset_name: str,
+    lora_rank: int = 16,
+    epochs: int = 3,
+) -> dict:
+    """Start a LoRA fine-tuning run on the specified model and dataset."""
+    # Dispatch to training service via event bus
+    await bus.emit(Event(
+        type=EventType.TRAINING_STARTED,
+        payload={"model_id": model_id, "dataset": dataset_name,
+                 "lora_rank": lora_rank, "epochs": epochs}
+    ))
+    return {"status": "started", "model_id": model_id}
+
+@mcp.tool()
+async def evaluate_model(model_id: str, dataset_name: str) -> dict:
+    """Evaluate a model on a dataset and return metrics."""
+    svc = model_registry.get(model_id)
+    return await eval_service.run(svc, dataset_name)
+
+@mcp.tool()
+async def export_gguf(model_id: str, quant: str = "Q4_K_M") -> str:
+    """Convert a fine-tuned model to GGUF format."""
+    return await export_service.run(model_id, quant)
+
+@mcp.tool()
+async def vision_chat(image_path: str, prompt: str, model_id: str = "minicpm_v46") -> str:
+    """Run multimodal inference on an image with a prompt."""
+    svc: MiniCPMVisionService = model_registry.get(model_id)
+    from PIL import Image
+    img = Image.open(image_path).convert("RGB")
+    messages = [{"role": "user", "content": [
+        {"type": "image", "image": img},
+        {"type": "text",  "text": prompt},
+    ]}]
+    return svc.chat(messages)
+
+if __name__ == "__main__":
+    mcp.run(transport="sse", port=8081)
+```
+
+MCP SSE endpoint: `http://localhost:8081/sse`
+Gradio MCP SSE: `http://localhost:7860/gradio_api/mcp/sse`
+
+---
+
+## 8. Training Pipeline
+
+### 8.1 LoRA — Text Models (PEFT)
+
+```python
+# training/lora.py
+from peft import LoraConfig, get_peft_model, TaskType
+from trl import SFTTrainer, SFTConfig
+
+class LoRATextTrainer:
+    DEFAULT_TARGET_MODULES = ["q_proj", "k_proj", "v_proj", "o_proj",
+                               "gate_proj", "up_proj", "down_proj"]
+
+    def __init__(self, cfg: dict):
+        self.cfg = cfg
+
+    def train(self, model, tokenizer, dataset, run_name: str):
+        lora_cfg = LoraConfig(
+            r=self.cfg.get("lora_rank", 16),
+            lora_alpha=self.cfg.get("lora_alpha", 32),
+            target_modules=self.cfg.get("target_modules", self.DEFAULT_TARGET_MODULES),
+            lora_dropout=self.cfg.get("lora_dropout", 0.05),
+            bias="none",
+            task_type=TaskType.CAUSAL_LM,
+        )
+        model = get_peft_model(model, lora_cfg)
+        model.print_trainable_parameters()
+
+        training_args = SFTConfig(
+            output_dir=f"./checkpoints/{run_name}",
+            num_train_epochs=self.cfg.get("epochs", 3),
+            per_device_train_batch_size=self.cfg.get("batch_size", 4),
+            gradient_accumulation_steps=self.cfg.get("grad_accum", 4),
+            learning_rate=self.cfg.get("lr", 2e-4),
+            fp16=False,
+            bf16=True,
+            report_to="trackio",         # ← Trackio via TRL
+            run_name=run_name,
+            save_steps=100,
+            logging_steps=10,
+        )
+        trainer = SFTTrainer(
+            model=model,
+            args=training_args,
+            train_dataset=dataset,
+            tokenizer=tokenizer,
+        )
+        trainer.train()
+        return model
+```
+
+### 8.2 LoRA — Vision Models (SWIFT / LLaMA-Factory recommended)
+
+> OpenBMB recommends SWIFT or LLaMA-Factory for MiniCPM-V fine-tuning.
+> Vision tower is frozen; only language backbone layers are adapted.
+
+```python
+# training/lora_vision.py
+class LoRAVisionConfig:
+    """
+    For MiniCPM-V 4.6 LoRA:
+    - Freeze vision encoder (SigLIP2-400M) + projector
+    - Apply LoRA only to Qwen3.5-0.8B language layers
+    - Use SWIFT CLI or LLaMA-Factory for full pipeline
+    """
+    LANGUAGE_TARGET_MODULES = ["q_proj", "k_proj", "v_proj", "o_proj"]
+    FROZEN_PREFIXES = ["vpm", "resampler", "vision_model"]  # freeze vision
+
+    @staticmethod
+    def freeze_vision(model):
+        for name, param in model.named_parameters():
+            if any(name.startswith(pfx) for pfx in LoRAVisionConfig.FROZEN_PREFIXES):
+                param.requires_grad = False
+        return model
+```
+
+**SWIFT CLI approach** (recommended for MiniCPM-V 4.6):
+```bash
+swift sft \
+  --model openbmb/MiniCPM-V-4.6 \
+  --dataset <your-dataset> \
+  --lora_rank 16 \
+  --freeze_vit true \
+  --output_dir ./checkpoints/minicpm_v46_lora
+```
+
+### 8.3 HF Jobs Offload (ml-intern pattern)
+
+For long training runs, offload to HF Jobs (free `cpu-basic`, paid `gpu-a100`):
+
+```python
+# training/hf_jobs.py
+from huggingface_hub import HfApi
+
+class HFJobsTrainer:
+    def submit(self, script_path: str, hardware: str = "gpu-a100") -> str:
+        api = HfApi()
+        job = api.create_job(
+            repo_id=f"{api.whoami()['name']}/ml-workbench-jobs",
+            script=script_path,
+            hardware=hardware,
+        )
+        return job.job_id
+```
+
+### 8.4 Evaluation
+
+```python
+# training/evaluation.py
+class Evaluator:
+    def run(self, model, tokenizer, dataset, metrics=("perplexity", "accuracy")) -> dict:
+        results = {}
+        if "perplexity" in metrics:
+            results["perplexity"] = self._perplexity(model, tokenizer, dataset)
+        if "accuracy" in metrics:
+            results["accuracy"] = self._accuracy(model, tokenizer, dataset)
+        return results
+```
+
+---
+
+## 9. Export & Quantization
+
+```python
+# training/export.py
+import subprocess
+from pathlib import Path
+
+class GGUFExporter:
+    SUPPORTED_QUANTS = ["F16", "Q4_K_M", "Q5_K_M", "Q8_0", "Q2_K"]
+
+    def export(
+        self,
+        model_path: str,
+        output_dir: str,
+        quant: str = "Q4_K_M",
+        model_type: str = "text",    # "text" | "vision"
+    ) -> list[str]:
+        output_dir = Path(output_dir)
+        output_dir.mkdir(parents=True, exist_ok=True)
+
+        # Step 1: Convert HF → GGUF F16
+        # For MiniCPM-V 4.6: convert_hf_to_gguf.py produces main gguf + mmproj
+        # For text models: same script, no mmproj
+        f16_path = output_dir / "model-f16.gguf"
+        subprocess.run([
+            "python", "convert_hf_to_gguf.py",
+            model_path,
+            "--outtype", "f16",
+            "--outfile", str(f16_path),
+        ], check=True)
+
+        outputs = [str(f16_path)]
+
+        # Step 2: Quantize
+        if quant != "F16":
+            quant_path = output_dir / f"model-{quant}.gguf"
+            subprocess.run([
+                "llama-quantize",
+                str(f16_path),
+                str(quant_path),
+                quant,
+            ], check=True)
+            outputs.append(str(quant_path))
+
+        return outputs
+```
+
+> Note: For MiniCPM-V 4.6, official GGUF files are already published at
+> `openbmb/MiniCPM-V-4.6-gguf`. Download them directly unless you need custom quants.
+
+---
+
+## 10. Agent Mode (ml-intern Inspired)
+
+The `agent/` module implements a lightweight research-plan-implement loop inspired by HF's
+ml-intern (April 2026). This gives the workbench autonomous ML experimentation capabilities.
+
+```
+ml-intern architecture (reference):
+  Research  → browse arXiv/HF Papers, read methodology, traverse citations
+  Plan      → break task down, verify resources (models, datasets, hardware)
+  Implement → execute scripts on HF Jobs / local GPU
+  Trace     → auto-upload sessions as JSONL to HF Dataset
+```
+
+```python
+# agent/loop.py
+from smolagents import CodeAgent, HfApiModel
+from huggingface_hub import HfApi
+
+class WorkbenchAgent:
+    """
+    Lightweight agent loop for autonomous experimentation.
+    Uses smolagents + HF ecosystem + registered MCP tools.
+    """
+    def __init__(self, model_id: str = "claude-sonnet-4-6"):
+        self.model = HfApiModel(model_id)
+        self.api   = HfApi()
+        self._session_log = []
+
+    def run(self, prompt: str, max_steps: int = 20) -> str:
+        agent = CodeAgent(
+            tools=self._get_tools(),
+            model=self.model,
+            max_steps=max_steps,
+        )
+        result = agent.run(prompt)
+        self._save_session(prompt, result)
+        return result
+
+    def _save_session(self, prompt: str, result: str):
+        import json, datetime
+        session = {
+            "timestamp": datetime.datetime.utcnow().isoformat(),
+            "prompt": prompt,
+            "result": result,
+            "log": self._session_log,
+        }
+        # Auto-upload to private HF dataset (ml-intern pattern)
+        # Dataset: {username}/workbench-sessions
+        pass
+
+    def _get_tools(self):
+        # Expose MCP tools as smolagents tools
+        from mcp.server.fastmcp import FastMCP
+        # ...
+        return []
+```
+
+---
+
+## 11. UI — Tab Specification
+
+Using `gr.Blocks` with tabs. Optionally swap to `gradio.Server` for custom frontend.
+
+```python
+# app.py
+import gradio as gr
+from ui.chat_tab    import build_chat_tab
+from ui.vision_tab  import build_vision_tab
+from ui.train_tab   import build_train_tab
+from ui.dataset_tab import build_dataset_tab
+from ui.export_tab  import build_export_tab
+from ui.traces_tab  import build_traces_tab
+from ui.agent_tab   import build_agent_tab
+from ui.notes_tab   import build_notes_tab
+
+with gr.Blocks(title="OpenBMB Workbench", theme=gr.themes.Ocean()) as demo:
+    gr.Markdown("# 🔬 OpenBMB Local AI Workbench")
+
+    with gr.Tabs():
+        with gr.Tab("💬 Chat"):       build_chat_tab()
+        with gr.Tab("👁️ Vision"):     build_vision_tab()
+        with gr.Tab("📊 Dataset"):    build_dataset_tab()
+        with gr.Tab("🏋️ Train"):      build_train_tab()
+        with gr.Tab("📦 Export"):     build_export_tab()
+        with gr.Tab("📈 Traces"):     build_traces_tab()
+        with gr.Tab("🤖 Agent"):      build_agent_tab()
+        with gr.Tab("📝 Field Notes"):build_notes_tab()
+
+demo.launch(
+    mcp_server=True,    # SSE at /gradio_api/mcp/sse
+    server_port=7860,
+    share=False,
+)
+```
+
+### Tab Summary
+
+| Tab | Purpose | Key components |
+|-----|---------|----------------|
+| Chat | Text inference on all text models | Model selector, streaming output, system prompt |
+| Vision | Image + text inference on V-4.6/Thinking | Image upload, thinking toggle, bounding box overlay |
+| Dataset | HF Hub browser + local CSV/JSON loader | Search bar, split preview, schema inspector |
+| Train | LoRA training launcher | Config form, live loss chart (Trackio), checkpoint browser |
+| Export | GGUF export + quantization | Model selector, quant dropdown, file download |
+| Traces | Trackio run browser + comparison | Run table, metric plots, SQL query box |
+| Agent | ml-intern style agent loop | Prompt, step log, paper browser, HF Jobs monitor |
+| Field Notes | Correction capture → retrain trigger | Image+prompt+response+correction form, JSONL export |
+
+### Vision Tab — key detail
+
+```python
+# ui/vision_tab.py
+def build_vision_tab():
+    with gr.Column():
+        model_dd = gr.Dropdown(
+            choices=["minicpm_v46", "minicpm_v46_thinking", "minicpm_o45"],
+            value="minicpm_v46",
+            label="Vision Model"
+        )
+        thinking_cb = gr.Checkbox(label="Enable Thinking Mode", value=False)
+        image_in    = gr.Image(type="pil", label="Upload Image")
+        video_in    = gr.Video(label="Upload Video (MiniCPM-V-4.6 supports video)")
+        prompt_in   = gr.Textbox(label="Prompt")
+        submit_btn  = gr.Button("Run")
+        output_txt  = gr.Textbox(label="Response", lines=10)
+
+        def infer(model_id, thinking, image, video, prompt):
+            svc = model_registry.get(model_id)
+            content = []
+            if image:
+                content.append({"type": "image", "image": image})
+            if video:
+                content.append({"type": "video", "video": video})
+            content.append({"type": "text", "text": prompt})
+            msgs = [{"role": "user", "content": content}]
+            svc.thinking = thinking
+            return svc.chat(msgs)
+
+        submit_btn.click(infer, [model_dd, thinking_cb, image_in, video_in, prompt_in], output_txt)
+```
+
+---
+
+## 12. Field Notes & Correction Loop
+
+```python
+# datasets/field_notes.py
+import sqlite3, json
+from dataclasses import dataclass, field, asdict
+from datetime import datetime
+
+@dataclass
+class FieldNote:
+    id:          str = field(default_factory=lambda: datetime.utcnow().isoformat())
+    model_id:    str = ""
+    modality:    str = "text"      # "text" | "image" | "video" | "multimodal"
+    image_path:  str | None = None
+    video_path:  str | None = None
+    prompt:      str = ""
+    response:    str = ""
+    correction:  str = ""          # human-corrected output
+    tags:        list = field(default_factory=list)
+    used_in_train: bool = False
+
+class FieldNoteStore:
+    def __init__(self, db_path: str = "data/field_notes.db"):
+        self.conn = sqlite3.connect(db_path, check_same_thread=False)
+        self._init_db()
+
+    def _init_db(self):
+        self.conn.execute("""
+            CREATE TABLE IF NOT EXISTS notes (
+                id TEXT PRIMARY KEY,
+                data JSON NOT NULL,
+                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+            )
+        """)
+        self.conn.commit()
+
+    def save(self, note: FieldNote):
+        self.conn.execute(
+            "INSERT OR REPLACE INTO notes (id, data) VALUES (?, ?)",
+            (note.id, json.dumps(asdict(note)))
+        )
+        self.conn.commit()
+
+    def to_hf_dataset(self, output_path: str):
+        """Export uncorrected notes as HF Dataset for retraining."""
+        from datasets import Dataset
+        rows = [json.loads(r[0]) for r in
+                self.conn.execute("SELECT data FROM notes WHERE json_extract(data,'$.correction') != ''")]
+        ds = Dataset.from_list(rows)
+        ds.save_to_disk(output_path)
+        return ds
+```
+
+---
+
+## 13. Directory Structure
+
+```
+ml-workbench/
+├── app.py                         # Entry point: gr.Blocks or gradio.Server
+├── requirements.txt
+│
+├── config/
+│   ├── models.yaml                # ← TEMPLATE AXIS: change this for any model family
+│   └── training.yaml              # LoRA hyperparams, export settings
+│
+├── core/
+│   ├── events.py                  # EventBus, EventType enum
+│   ├── registry.py                # Registry[T] generic
+│   └── state.py                   # Global AppState dataclass
+│
+├── models/
+│   ├── base.py                    # Abstract ModelService
+│   ├── hf_loader.py               # Generic HF Hub downloader
+│   ├── minicpm_text.py            # MiniCPM5-1B, MiniCPM4.1-8B service
+│   ├── minicpm_vision.py          # MiniCPM-V-4.6 / Thinking service
+│   ├── minicpm_omni.py            # MiniCPM-o-4.5 service
+│   ├── llama_cpp_runner.py        # llama-server wrapper
+│   ├── sglang_runner.py           # SGLang server wrapper (tool-use)
+│   └── ollama_runner.py           # Ollama client
+│
+├── datasets/
+│   ├── loader.py                  # Abstract DatasetLoader
+│   ├── hf_datasets.py             # HF Hub datasets.load_dataset()
+│   ├── field_notes.py             # SQLite-backed correction store
+│   └── synthetic.py               # ml-intern style synthetic data gen
+│
+├── training/
+│   ├── lora.py                    # LoRA text trainer (PEFT + TRL)
+│   ├── lora_vision.py             # LoRA vision config (SWIFT/LLaMA-Factory)
+│   ├── evaluation.py              # Perplexity, accuracy, custom metrics
+│   ├── export.py                  # GGUF exporter (convert_hf_to_gguf.py)
+│   └── hf_jobs.py                 # HF Jobs offload (ml-intern pattern)
+│
+├── tools/
+│   ├── calculator.py              # Safe math eval tool
+│   ├── dataset_stats.py           # Dataset statistics tool
+│   ├── hf_search.py               # HF Hub search tool
+│   └── paper_search.py            # arXiv / HF Papers search (ml-intern)
+│
+├── tracking/
+│   └── trackio_service.py         # Trackio integration (init/log/finish/alert)
+│
+├── mcp/
+│   ├── server.py                  # FastMCP standalone server
+│   └── tools.py                   # All @mcp.tool() decorators
+│
+├── agent/
+│   ├── loop.py                    # Research → Plan → Implement agent
+│   └── prompts/
+│       └── system_prompt.yaml     # Agent system prompt (ml-intern style)
+│
+├── ui/
+│   ├── chat_tab.py
+│   ├── vision_tab.py
+│   ├── train_tab.py
+│   ├── dataset_tab.py
+│   ├── export_tab.py
+│   ├── traces_tab.py
+│   ├── agent_tab.py
+│   └── notes_tab.py
+│
+├── exports/                       # GGUF + quantized model outputs
+└── data/                          # Cached datasets, field_notes.db
+```
+
+---
+
+## 14. Configuration Schema
+
+### training.yaml
+
+```yaml
+# config/training.yaml
+lora:
+  rank:          16
+  alpha:         32
+  dropout:       0.05
+  target_modules_text:
+    - q_proj
+    - k_proj
+    - v_proj
+    - o_proj
+    - gate_proj
+    - up_proj
+    - down_proj
+  target_modules_vision:
+    - q_proj
+    - k_proj
+    - v_proj
+    - o_proj                       # language layers only; vision tower frozen
+
+training:
+  epochs:        3
+  batch_size:    4
+  grad_accum:    4
+  lr:            2.0e-4
+  bf16:          true
+  report_to:     trackio           # or: wandb, tensorboard, none
+
+export:
+  default_quant: Q4_K_M
+  supported_quants:
+    - F16
+    - Q4_K_M
+    - Q5_K_M
+    - Q8_0
+    - Q2_K
+
+trackio:
+  project:   ml-workbench
+  space_id:  null                  # set to "username/trackio-space" for HF sync
+
+hf_jobs:
+  enabled:   false
+  hardware:  gpu-a100
+```
+
+---
+
+## 15. Dependencies
+
+```text
+# requirements.txt
+# Core
+gradio[mcp]>=5.38.0
+mcp[cli]>=1.0.0
+fastapi
+uvicorn[standard]
+
+# Models
+transformers>=5.7.0               # required for MiniCPM-V-4.6
+torch>=2.1.0
+torchvision
+torchcodec                        # video decoding for MiniCPM-V-4.6
+accelerate
+bitsandbytes
+
+# Training
+peft>=0.10.0
+trl>=0.9.0
+datasets>=2.18.0
+
+# Tracking
+trackio>=0.1.0
+
+# MCP + Agents
+mcp>=1.0.0
+smolagents>=1.0.0
+
+# Serving backends (optional, install as needed)
+# sglang                          # pip install sglang
+# vllm                            # pip install vllm
+# ollama                          # install separately
+
+# Hub
+huggingface_hub>=0.23.0
+Pillow
+requests
+pyyaml
+```
+
+---
+
+## 16. Hackathon Demo Flow
+
+```
+1. DATASET
+   Load HF dataset via Dataset tab (e.g. "tatsu-lab/alpaca", 10K rows)
+   → EventType.DATASET_LOADED
+   → Trackio: trackio.init("demo-run") + trackio.log({"dataset": ..., "rows": ...})
+
+2. TRAIN
+   Start LoRA on MiniCPM5-1B (rank=16, 1 epoch demo)
+   → EventType.TRAINING_STARTED
+   → TRL SFTTrainer + report_to="trackio" → live loss in Traces tab
+
+3. EXPORT
+   Click "Export GGUF" → Q4_K_M
+   → convert_hf_to_gguf.py → llama-quantize
+   → EventType.EXPORT_FINISHED → file download
+
+4. LLAMA.CPP CHAT
+   Load exported GGUF in llama-server
+   → Chat tab, backend: llama.cpp
+   → Verify text generation quality
+
+5. SWITCH TO VISION
+   Select MiniCPM-V-4.6 in Vision tab
+   Upload image → enter prompt → Run
+
+6. THINKING MODE
+   Toggle "Enable Thinking Mode" → switch to MiniCPM-V-4.6-Thinking
+   → Observe explicit reasoning trace in response
+
+7. FIELD NOTE
+   Response is wrong → open Field Notes tab
+   → Enter correction → Save
+   → EventType.FIELD_NOTE_SAVED
+
+8. AGENT MODE
+   Agent tab: "Improve MiniCPM-V-4.6 on [domain] using field notes"
+   → Research phase: browse HF Papers
+   → Plan: dataset selection, LoRA config
+   → Implement: trigger SWIFT fine-tune
+
+9. SHARE TRACES
+   Traces tab → set space_id → Sync to HF Space
+   → Shareable dashboard URL
+```
+
+---
+
+## 17. Corrections from PRD v1
+
+| v1 Claim | Correct v2 |
+|----------|-----------|
+| `trackio.trace("vision_inference")` context manager | **Does not exist.** Use `trackio.init()` + `trackio.log()` + `trackio.finish()` |
+| `AutoModelForCausalLM` for MiniCPM-V-4.6 | **Wrong.** Use `AutoModelForImageTextToText` with `transformers>=5.7.0` |
+| `llama-mtmd-cli` for vision deployment | Use `llama-server --mmproj <mmproj.gguf>` for server; `llama-mtmd-cli` for CLI-only |
+| Run `convert_hf_to_gguf.py` for V4.6 | Official GGUFs at `openbmb/MiniCPM-V-4.6-gguf` — download directly; run script only for custom quants |
+| Surgery scripts for V4.6 GGUF | Surgery scripts are for MiniCPM-o-2.6 only; **do not use for V4.6** |
+| `future MiniCPM-o` (vague) | **MiniCPM-o-4.5** released 2026-05-17; specific HF ID available |
+| `app.launch(mcp_server=True)` only option | Two paths: `gr.Blocks` + `mcp_server=True`, or `gradio.Server` with `@app.mcp.tool()` |
+| FastMCP as only MCP approach | Three paths: Gradio native, gradio.Server, standalone FastMCP (see §7) |
+| MiniCPM4.1-8B: standard architecture | Has sparse InfLLM-v2 attention; requires `--trust-remote-code` |
+| LoRA for vision: PEFT only | OpenBMB recommends **SWIFT** or **LLaMA-Factory** for MiniCPM-V fine-tuning |
+| No HF Jobs mention | HF Jobs integration (ml-intern pattern) enables GPU cloud offload |
+| No SGLang mention | SGLang is the **recommended** backend for MiniCPM5-1B tool calling |
+| No video support | MiniCPM-V-4.6 supports **video understanding** (`torchcodec` required) |
+
+---
+
+## 18. Roadmap & Extension Points
+
+### Swap the model family (template portability)
+
+To build a "Qwen3 Workbench":
+1. Edit `config/models.yaml` — add Qwen3 model IDs
+2. Create `models/qwen3_service.py` extending `ModelService`
+3. Register in `app.py` — done
+
+No changes to training, export, MCP, Trackio, or UI layers.
+
+### Planned extensions
+
+| Feature | Module | Notes |
+|---------|--------|-------|
+| vLLM serving tab | `models/vllm_runner.py` | Batch inference, production |
+| Ollama quick-start | `models/ollama_runner.py` | Zero-config consumer demo |
+| Reward model eval | `training/reward_eval.py` | For RLHF experiments |
+| Synthetic data gen | `datasets/synthetic.py` | ml-intern pattern: LLM writes training data |
+| Paper-to-code agent | `agent/paper_agent.py` | Read arXiv → implement in workbench |
+| HF Spaces deploy | `deploy/spaces.py` | One-click push as HF Space |
+| VINDEX integration | `tools/vindex_tool.py` | Knowledge editing via your VINDEX engine |
+| OCR pipeline hook | `datasets/ocr_loader.py` | Feed your OCR pipeline output as field notes |
+| MiniCPM Desk-Pet | `agent/desk_pet.py` | LoRA persona switching via MiniCPM5-1B |
+| MiniCPM-o audio | `ui/audio_tab.py` | Real-time omnimodal via MiniCPM-o-4.5 |
+
+---
+
+*PRD v2.0 — Christof Kaller / ki-fusion-labs.de — 2026-06-05*
+*Research sources: openbmb/MiniCPM-V, openbmb/MiniCPM, gradio-app/trackio,*
+*huggingface/ml-intern, huggingface.co/blog/introducing-gradio-server*

README.md

@@ -0,0 +1,227 @@
+---
+title: OpenBMB Local AI Workbench
+colorFrom: green
+colorTo: gray
+sdk: gradio
+app_file: app.py
+pinned: false
+---
+
+# OpenBMB Local AI Workbench
+
+A Gradio workbench for the Build Small Hackathon: small local models, practical experimentation,
+and a clear path from local demo to Hugging Face Space.
+
+## What This Is
+
+The project turns the PRD in `HF_PRD_v1.md` into a staged implementation:
+
+1. A working Gradio app shell.
+2. Config-driven OpenBMB model registry.
+3. Local-first inference path through real backends: Transformers, Ollama, llama.cpp,
+   LM Studio/OpenAI-compatible, SGLang, and vLLM.
+4. Field notes for collecting corrections.
+5. Extension points for training, GGUF export, Trackio traces, MCP tools, and agent workflows.
+
+## Hackathon Fit
+
+- **Track:** Backyard AI or Thousand Token Wood, depending on the final user story.
+- **Canvas:** Gradio app, deployable to Hugging Face Spaces.
+- **Small model rule:** target models stay at or below 32B parameters.
+- **Bonus quests:** local-first, field notes/report, possible llama.cpp and trace sharing.
+
+## Quick Start
+
+Python is not currently available on PATH in this workspace shell. Install Python first, then:
+
+```powershell
+.\scripts\verify_structure.ps1
+& "$env:LOCALAPPDATA\Microsoft\WindowsApps\python3.11.exe" -m venv .venv
+.venv\Scripts\Activate.ps1
+python -m pip install -r requirements.txt
+python -m pip install -r requirements-dev.txt
+.\scripts\run_tests.ps1
+python app.py
+```
+
+## Browser E2E Screenshots
+
+After installing Node.js:
+
+```powershell
+npm install
+npm run e2e:install
+npm run e2e
+```
+
+The Playwright user-story tests run separate Workbench and Plant flows and save documentation
+screenshots under `assets/e2e/workbench/` and `assets/e2e/plant/`. The Workbench screenshot now runs
+local GGUF chat through `llama-cpp-python`; the Plant screenshot can run OpenBMB MiniCPM-V on
+`assets/plant_sample.jpg` with `RUN_REAL_MODEL_E2E=1`.
+only when `RUN_REAL_MODEL_E2E=1` is set; otherwise the browser tests verify real-backend setup
+surfaces without using mock responses. To record or edit the browser flow manually, run
+`npm run e2e:record`.
+
+Generated screenshot sets:
+
+- [Workbench home](assets/e2e/workbench/01-workbench-home.png)
+- [Workbench backend status](assets/e2e/workbench/05-backend-status.png)
+- [Plant tool home](assets/e2e/plant/01-plant-home.png)
+- [Plant corrections export](assets/e2e/plant/03-corrections-export.png)
+
+## Template And Reference Apps
+
+This repo is also a template for focused local-first Gradio apps. The first reference app is
+Plant Discovery under `plant/`.
+
+```powershell
+.venv\Scripts\python.exe -m plant.app --no-model --port 7861
+```
+
+Use the real OpenBMB VLM path after installing optional plant dependencies:
+
+```powershell
+.venv\Scripts\python.exe -m plant.app --model-mode openbmb --port 7861
+```
+
+The detailed build guide is [docs/TEMPLATE_HOWTO.md](docs/TEMPLATE_HOWTO.md), and the Plant
+Discovery checklist is [docs/PLANT_DISCOVERY_APP_PLAN.md](docs/PLANT_DISCOVERY_APP_PLAN.md).
+Model and adapter training steps are in
+[docs/PLANT_MODEL_AND_TRAINING_HOWTO.md](docs/PLANT_MODEL_AND_TRAINING_HOWTO.md).
+
+If `python` is not recognized, install Python 3.11+ from python.org or the Windows Store,
+then reopen the terminal so PATH is refreshed.
+
+Open the local URL shown by Gradio, usually `http://127.0.0.1:7860`.
+
+## Project Structure
+
+```text
+.
+|-- app.py
+|-- AGENTS.md
+|-- README.md
+|-- requirements.txt
+|-- config/
+|   |-- models.yaml
+|   `-- training.yaml
+|-- core/
+|   |-- events.py
+|   `-- registry.py
+|-- datasets/
+|   `-- field_notes.py
+|-- models/
+|   |-- placeholder_service.py
+|   `-- model_catalog.py
+|-- ui/
+|   |-- chat_tab.py
+|   |-- dataset_tab.py
+|   |-- export_tab.py
+|   |-- notes_tab.py
+|   |-- traces_tab.py
+|   |-- agent_tab.py
+|   |-- status_tab.py
+|   |-- train_tab.py
+|   `-- vision_tab.py
+|-- data/
+|   `-- .gitkeep
+|-- exports/
+|   `-- .gitkeep
+|-- HF_PRD_v1.md
+`-- HF_PRD_ext.md
+```
+
+## Project Docs
+
+The working docs live in [docs/README.md](docs/README.md).
+
+- [Task checklist](docs/TASKS.md)
+- [Implementation status](docs/IMPLEMENTATION_STATUS.md)
+- [PRD implementation matrix](docs/PRD_IMPLEMENTATION_MATRIX.md)
+- [Acceptance criteria](docs/ACCEPTANCE_CRITERIA.md)
+- [Roadmap](docs/ROADMAP.md)
+- [Template how-to](docs/TEMPLATE_HOWTO.md)
+- [Plant Discovery plan](docs/PLANT_DISCOVERY_APP_PLAN.md)
+- [Usage guide](docs/USAGE.md)
+- [Architecture guide](docs/ARCHITECTURE.md)
+- [Extension guide](docs/EXTENDING.md)
+- [Hackathon submission package](docs/HACKATHON_SUBMISSION.md)
+- [Test and quality policy](docs/ACCEPTANCE_CRITERIA.md)
+
+## Current Truth
+
+The full PRD is not implemented yet. The current app is a tested, quality-gated scaffold moving
+from placeholder-first local verification to real-backend Workbench deployment. GitHub push is
+complete at `https://github.com/Ckal/codex`. LM Studio/OpenAI-compatible text inference has been
+verified previously; OpenBMB Transformers, Ollama OpenBMB, llama.cpp, MiniCPM-V, Space builds,
+training execution, served MCP, and most extension PRD items still need proof before being claimed
+done.
+
+## Model Plan
+
+Initial candidates from the PRD:
+
+| Config ID | Model | Purpose |
+| --- | --- | --- |
+| `minicpm5_1b` | `openbmb/MiniCPM5-1B` | text chat, LoRA, local-first baseline |
+| `minicpm_v46` | `openbmb/MiniCPM-V-4.6` | image/video understanding |
+| `minicpm_o45` | `openbmb/MiniCPM-o-4.5` | omnimodal stretch goal |
+
+The app does not download large model files automatically. In deployed Space mode, placeholder
+backend choices are hidden and model calls require real backend configuration. llama.cpp,
+llama-cpp-python, Ollama, LM Studio/OpenAI-compatible, SGLang, Nemotron Nano 9B v2, and Transformers text can be selected as
+backends, but the backend tool/package/server must be installed and populated with the selected
+model explicitly by the user.
+
+## Deployment Target
+
+For Hugging Face Spaces, keep these files at repo root:
+
+- `app.py`
+- `requirements.txt`
+- `README.md`
+- `config/`
+- `core/`
+- `models/`
+- `ui/`
+
+Workbench Space target:
+
+```text
+https://huggingface.co/spaces/build-small-hackathon/workbench
+```
+
+## Spaces
+
+- Workbench Space: https://huggingface.co/spaces/build-small-hackathon/workbench
+- Plant Identification Tool Space: https://huggingface.co/spaces/build-small-hackathon/plant_identification_tool
+
+Both Spaces have been pushed. At the latest local poll they were still in Hugging Face `BUILDING`
+state on `zero-a10g`, so final build/run smoke verification is still open.
+
+Plant Identification Tool Space target:
+
+```text
+https://huggingface.co/spaces/build-small-hackathon/plant_identification_tool
+```
+
+Use a freshly generated token through `hf auth login`; do not paste tokens into files or commit
+them.
+
+Later deployment commands:
+
+```powershell
+hf auth login
+git remote add space-workbench https://huggingface.co/spaces/build-small-hackathon/workbench
+git push space-workbench main
+git remote add space-plant https://huggingface.co/spaces/build-small-hackathon/plant_identification_tool
+git push space-plant main
+```
+
+## Next Implementation Steps
+
+1. Decide the exact hackathon story and user.
+2. Add screenshot/demo media and Space submission URLs.
+3. Push and verify the two Hugging Face Spaces, then finish llama.cpp MiniCPM-V mmproj vision verification.
+4. Add field-note export to JSONL/HF Dataset.
+5. Polish README with screenshots, demo video script, and submission links.

agent/__init__.py

@@ -0,0 +1 @@
+"""Local deterministic agent helpers."""

agent/runner.py

@@ -0,0 +1,203 @@
+from __future__ import annotations
+
+import json
+from dataclasses import asdict, dataclass
+from pathlib import Path
+from typing import Any
+
+from core.file_exports import copy_text_file_or_empty
+from mcp_tools.tools import safe_calculator_tool, tool_registry
+
+AGENT_SYSTEM_PROMPT = (
+    "You are a local workbench agent. Research the request, draft a small plan, "
+    "name tools you would use, and require verification before marking work done."
+)
+
+
+@dataclass(frozen=True)
+class AgentStep:
+    """One deterministic agent trace step."""
+
+    phase: str
+    content: str
+
+
+@dataclass(frozen=True)
+class AgentSession:
+    """Agent session trace."""
+
+    task: str
+    steps: list[AgentStep]
+    tools: list[str]
+    limitations: list[str]
+    safety_gates: list[str]
+
+    def as_dict(self) -> dict[str, Any]:
+        return {
+            "task": self.task,
+            "steps": [asdict(step) for step in self.steps],
+            "tools": self.tools,
+            "limitations": self.limitations,
+            "safety_gates": self.safety_gates,
+            "system_prompt": AGENT_SYSTEM_PROMPT,
+        }
+
+    def as_markdown(self) -> str:
+        lines = [f"Task: {self.task or '(none)'}", ""]
+        for step in self.steps:
+            lines.append(f"{step.phase}: {step.content}")
+        lines.append("")
+        lines.append(f"Tools: {', '.join(self.tools)}")
+        lines.append(f"Limitations: {'; '.join(self.limitations)}")
+        lines.append(f"Safety gates: {'; '.join(self.safety_gates)}")
+        return "\n".join(lines)
+
+
+def run_agent_loop(task: str) -> AgentSession:
+    tools = sorted(tool_registry())
+    steps = [
+        AgentStep("research", _research_summary(task)),
+        AgentStep("plan", _plan_summary(task)),
+        AgentStep("implement", _implementation_summary(task)),
+        AgentStep("verify", "Run unit tests, smoke checks, quality gates, and update docs/tasks."),
+    ]
+
+    calculator_result = _maybe_calculate(task)
+    if calculator_result is not None:
+        steps.insert(
+            1,
+            AgentStep(
+                "tool:safe_calculator",
+                json.dumps(calculator_result.payload, ensure_ascii=False),
+            ),
+        )
+
+    return AgentSession(
+        task=task,
+        steps=steps,
+        tools=tools,
+        limitations=[
+            "Does not execute shell commands.",
+            "Does not commit, push, deploy, download models, or call external services.",
+            "Requires Codex or a human to apply and verify implementation changes.",
+        ],
+        safety_gates=default_safety_gates(),
+    )
+
+
+def run_paper_to_code_loop(
+    paper_title: str,
+    paper_notes: str,
+    implementation_goal: str,
+) -> AgentSession:
+    task = f"Paper-to-code: {paper_title.strip() or 'untitled paper'}"
+    steps = [
+        AgentStep("research", _paper_research_summary(paper_title, paper_notes)),
+        AgentStep("plan", _paper_plan_summary(implementation_goal)),
+        AgentStep("implement", _paper_implementation_trace(implementation_goal)),
+        AgentStep("verify", "Map claims to tests, run quality gates, and document gaps."),
+    ]
+    return AgentSession(
+        task=task,
+        steps=steps,
+        tools=sorted(tool_registry()),
+        limitations=[
+            "Does not read remote papers automatically.",
+            "Does not execute code changes autonomously.",
+            "Requires human/Codex review before implementation claims are marked done.",
+        ],
+        safety_gates=default_safety_gates(),
+    )
+
+
+def default_safety_gates() -> list[str]:
+    return [
+        "No shell commands are executed by the agent trace.",
+        "No model weights, datasets, or papers are downloaded automatically.",
+        "Every implementation claim needs a matching test or documented blocker.",
+        "External services require explicit user credentials and approval.",
+    ]
+
+
+def save_agent_trace(
+    session: AgentSession,
+    path: str | Path = "data/agent_traces.jsonl",
+) -> Path:
+    output = Path(path)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    with output.open("a", encoding="utf-8") as f:
+        f.write(json.dumps(session.as_dict(), ensure_ascii=False) + "\n")
+    return output
+
+
+def export_agent_traces(
+    source_path: str | Path = "data/agent_traces.jsonl",
+    output_path: str | Path = "exports/agent_traces.jsonl",
+) -> Path:
+    return copy_text_file_or_empty(source_path, output_path)
+
+
+def export_agent_traces_hf_dataset(
+    source_path: str | Path = "data/agent_traces.jsonl",
+    output_dir: str | Path = "exports/agent_traces_dataset",
+) -> Path:
+    target = Path(output_dir)
+    target.mkdir(parents=True, exist_ok=True)
+    data_file = target / "data.jsonl"
+    if Path(source_path).exists():
+        data_file.write_text(Path(source_path).read_text(encoding="utf-8"), encoding="utf-8")
+    else:
+        data_file.write_text("", encoding="utf-8")
+    (target / "README.md").write_text(
+        "# Agent Traces Dataset\n\n"
+        "Local Hugging Face Dataset-style export for OpenBMB Local AI Workbench traces.\n",
+        encoding="utf-8",
+    )
+    return target
+
+
+def _research_summary(task: str) -> str:
+    if not task.strip():
+        return "No task provided. Ask for a concrete task before implementation."
+    return "Inspect PRD/tasks/docs, identify affected modules, and check existing tests."
+
+
+def _plan_summary(task: str) -> str:
+    if any(word in task.casefold() for word in ["deploy", "push", "github", "huggingface"]):
+        return "Prepare repo/deploy steps, verify auth/remotes, then push only after tests pass."
+    return "Make a focused implementation slice, add or update tests, then update docs."
+
+
+def _implementation_summary(task: str) -> str:
+    if "model" in task.casefold():
+        return "Use configured backend services and avoid startup downloads."
+    return "Apply changes in the smallest relevant modules and keep unrelated files untouched."
+
+
+def _paper_research_summary(paper_title: str, paper_notes: str) -> str:
+    title = paper_title.strip() or "untitled paper"
+    notes = paper_notes.strip()
+    if not notes:
+        return f"Summarize the claims, assumptions, and reproducibility risks for {title}."
+    return f"Extract implementation claims from {title}: {notes[:240]}"
+
+
+def _paper_plan_summary(implementation_goal: str) -> str:
+    goal = implementation_goal.strip() or "create a minimal local reproduction plan"
+    return f"Break the goal into local modules, tests, data assumptions, and blockers: {goal}."
+
+
+def _paper_implementation_trace(implementation_goal: str) -> str:
+    goal = implementation_goal.strip() or "minimal reproducible scaffold"
+    return (
+        "Draft a non-executing implementation trace for "
+        f"{goal}; keep dependencies explicit and update docs before claiming completion."
+    )
+
+
+def _maybe_calculate(task: str):
+    prefix = "calculate:"
+    if task.casefold().strip().startswith(prefix):
+        expression = task.split(":", 1)[1].strip()
+        return safe_calculator_tool(expression)
+    return None

app.py

@@ -0,0 +1,110 @@
+from __future__ import annotations
+
+from typing import cast
+
+import gradio as gr
+
+from core.app_logging import configure_app_logging
+from core.deployment import current_policy
+from models.model_catalog import load_model_catalog
+from ui.agent_tab import build_agent_tab
+from ui.chat_tab import build_chat_tab
+from ui.dataset_tab import build_dataset_tab
+from ui.export_tab import build_export_tab
+from ui.notes_tab import build_notes_tab
+from ui.status_tab import build_status_tab
+from ui.traces_tab import build_traces_tab
+from ui.train_tab import build_train_tab
+from ui.vision_tab import build_vision_tab
+from ui.vllm_tab import build_vllm_tab
+
+APP_THEME = gr.themes.Soft(primary_hue="teal", neutral_hue="slate")
+APP_CSS = """
+.app-title {
+  margin-bottom: 0.25rem;
+}
+
+.gradio-container {
+  max-width: 1180px !important;
+}
+
+.tabs {
+  overflow-x: auto;
+}
+
+button {
+  min-height: 2.5rem;
+}
+
+textarea,
+input,
+.wrap {
+  max-width: 100%;
+}
+
+@media (max-width: 720px) {
+  .gradio-container {
+    padding-left: 0.75rem !important;
+    padding-right: 0.75rem !important;
+  }
+
+  .app-title h1 {
+    font-size: 1.55rem;
+    line-height: 1.2;
+  }
+}
+"""
+
+
+def build_app() -> gr.Blocks:
+    configure_app_logging()
+    policy = current_policy()
+    catalog = load_model_catalog("config/models.yaml")
+
+    with gr.Blocks(title="OpenBMB Local AI Workbench", analytics_enabled=False) as demo:
+        gr.Markdown(
+            """
+            # OpenBMB Local AI Workbench
+            Real small-model experimentation for Gradio + Hugging Face Spaces.
+            """,
+            elem_classes=["app-title"],
+        )
+        if policy.is_space:
+            gr.Markdown(
+                "Deployment mode: Space. Placeholder and demo inference are disabled; "
+                "select a configured real backend before running model calls."
+            )
+
+        with gr.Tabs():
+            with gr.Tab("Chat"):
+                build_chat_tab(catalog, policy)
+            with gr.Tab("Vision"):
+                build_vision_tab(catalog, policy)
+            with gr.Tab("Dataset"):
+                build_dataset_tab()
+            with gr.Tab("Train"):
+                build_train_tab()
+            with gr.Tab("vLLM"):
+                build_vllm_tab(catalog)
+            with gr.Tab("Export"):
+                build_export_tab(catalog)
+            with gr.Tab("Field Notes"):
+                build_notes_tab(catalog)
+            with gr.Tab("Traces"):
+                build_traces_tab()
+            with gr.Tab("Agent"):
+                build_agent_tab()
+            with gr.Tab("Status"):
+                build_status_tab(catalog, policy)
+
+    return cast(gr.Blocks, demo)
+
+
+if __name__ == "__main__":
+    build_app().launch(
+        server_port=7860,
+        share=False,
+        theme=APP_THEME,
+        css=APP_CSS,
+        mcp_server=True,
+    )

assets/e2e/README.md

@@ -0,0 +1,21 @@
+# E2E Documentation Assets
+
+Run the Playwright user-story test to generate screenshots for the README and hackathon docs:
+
+```powershell
+npm install
+npm run e2e:install
+npm run e2e
+```
+
+The test starts or reuses the local Gradio app at `http://127.0.0.1:7860`, walks all major tabs,
+executes safe local actions, and writes screenshots plus `user-story.md` into this folder.
+
+To record or edit the flow manually:
+
+```powershell
+npm run e2e:record
+```
+
+Generated images are intentionally stored under `assets/e2e/` so they can be linked from
+`README.md`, `docs/HACKATHON_SUBMISSION.md`, and demo notes.

assets/samples/README.md

@@ -0,0 +1,7 @@
+# Public Sample Assets
+
+These files are intentionally small and public-safe. They are used for local verification,
+Playwright screenshots, and documentation examples.
+
+- `plant_leaf.ppm` and `plant_leaf.png` are generated bitmap-style plant samples for upload workflow testing.
+  It is not a botanical benchmark image and should not be used to claim model accuracy.

config/models.yaml

@@ -0,0 +1,155 @@
+models:
+  minicpm5_1b:
+    hf_id: openbmb/MiniCPM5-1B
+    display_name: MiniCPM5 1B
+    type: text
+    parameters_b: 1
+    backend: transformers
+    backend_capabilities:
+      placeholder: [text]
+      ollama: [text, streaming]
+      llama.cpp: [text, gguf]
+      llama-cpp-python: [text, gguf]
+      transformers: [text, lora]
+      openai-compatible: [text, openai-compatible]
+      sglang: [text, tools]
+      vllm: [text, openai-compatible]
+    context_length: 131072
+    local_first: true
+    thinking_mode: false
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM5-1B-GGUF
+      default_quant: Q4_K_M
+    notes: Text baseline for local chat, LoRA experiments, and GGUF export.
+
+  minicpm5_1b_thinking:
+    hf_id: openbmb/MiniCPM5-1B-Thinking
+    display_name: MiniCPM5 1B Thinking
+    type: text
+    parameters_b: 1
+    backend: transformers
+    backend_capabilities:
+      placeholder: [text]
+      ollama: [text, thinking, streaming]
+      llama.cpp: [text, gguf]
+      llama-cpp-python: [text, gguf]
+      transformers: [text, thinking, lora]
+      openai-compatible: [text, thinking, openai-compatible]
+      sglang: [text, thinking, tools]
+      vllm: [text, openai-compatible]
+    context_length: 131072
+    local_first: true
+    thinking_mode: true
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM5-1B-Thinking-GGUF
+      default_quant: Q4_K_M
+    notes: Thinking variant for explicit reasoning experiments.
+
+  minicpm41_8b:
+    hf_id: openbmb/MiniCPM4.1-8B
+    display_name: MiniCPM4.1 8B
+    type: text
+    parameters_b: 8
+    backend: transformers
+    backend_capabilities:
+      placeholder: [text]
+      ollama: [text, streaming]
+      llama.cpp: [text, gguf]
+      llama-cpp-python: [text, gguf]
+      transformers: [text, long-context]
+      openai-compatible: [text, long-context, openai-compatible]
+      sglang: [text, long-context, tools]
+      vllm: [text, long-context, openai-compatible]
+    context_length: 131072
+    local_first: true
+    thinking_mode: false
+    trust_remote_code: true
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM4.1-8B-GGUF
+      default_quant: Q4_K_M
+    notes: Long-context sparse-attention text model; real backend will require trust_remote_code.
+
+  nemotron_nano_9b_v2:
+    hf_id: nvidia/NVIDIA-Nemotron-Nano-9B-v2
+    display_name: NVIDIA Nemotron Nano 9B v2
+    type: text
+    parameters_b: 9
+    backend: transformers
+    backend_capabilities:
+      placeholder: [text]
+      llama.cpp: [text, gguf]
+      llama-cpp-python: [text, gguf]
+      transformers: [text, custom-code]
+      openai-compatible: [text, openai-compatible]
+      sglang: [text]
+      vllm: [text, openai-compatible]
+    context_length: 131072
+    local_first: true
+    thinking_mode: false
+    trust_remote_code: true
+    gguf:
+      available: true
+      repo: bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF
+      default_quant: Q4_K_M
+    notes: Small Nemotron text option under the 32B hackathon limit; verify license and hardware before using in a public Space.
+
+  minicpm_v46:
+    hf_id: openbmb/MiniCPM-V-4.6
+    display_name: MiniCPM-V 4.6
+    type: vision
+    parameters_b: 1.3
+    backend: transformers
+    backend_capabilities:
+      placeholder: [vision]
+      ollama: [vision]
+      llama.cpp: [vision, gguf, mmproj]
+      transformers: [vision, image-text-to-text]
+      sglang: [vision, openai-compatible]
+    context_length: 262144
+    local_first: true
+    thinking_mode: false
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM-V-4.6-gguf
+      main_file: MiniCPM-V-4.6-Q4_K_M.gguf
+      mmproj: mmproj-MiniCPM-V-4.6-F16.gguf
+      default_quant: Q4_K_M
+    notes: Vision-language model candidate for image and video understanding.
+
+  minicpm_v46_thinking:
+    hf_id: openbmb/MiniCPM-V-4.6-Thinking
+    display_name: MiniCPM-V 4.6 Thinking
+    type: vision
+    parameters_b: 1.3
+    backend: transformers
+    backend_capabilities:
+      placeholder: [vision, thinking]
+      ollama: [vision, thinking]
+      llama.cpp: [vision, gguf, mmproj]
+      transformers: [vision, thinking, image-text-to-text]
+      sglang: [vision, thinking, openai-compatible]
+    context_length: 262144
+    local_first: true
+    thinking_mode: true
+    gguf:
+      available: true
+      repo: openbmb/MiniCPM-V-4.6-Thinking-gguf
+      default_quant: Q4_K_M
+    notes: Vision-language thinking variant for multimodal reasoning demos.
+
+  minicpm_o45:
+    hf_id: openbmb/MiniCPM-o-4.5
+    display_name: MiniCPM-o 4.5
+    type: omnimodal
+    parameters_b: 8
+    backend: transformers
+    backend_capabilities:
+      placeholder: [text, vision, audio]
+      transformers: [text, vision, audio]
+    context_length: 131072
+    local_first: true
+    thinking_mode: false
+    notes: Omnimodal stretch goal for audio, image, and text.

config/training.yaml

@@ -0,0 +1,24 @@
+lora:
+  rank: 16
+  alpha: 32
+  dropout: 0.05
+
+training:
+  epochs: 1
+  batch_size: 2
+  grad_accum: 4
+  lr: 0.0002
+  report_to: none
+
+export:
+  default_quant: Q4_K_M
+  supported_quants:
+    - F16
+    - Q4_K_M
+    - Q5_K_M
+    - Q8_0
+
+trackio:
+  enabled: false
+  project: openbmb-local-ai-workbench
+  local_path: data/traces.jsonl

core/__init__.py

@@ -0,0 +1 @@
+"""Core shared helpers for events, state, and registries."""

core/app_logging.py

@@ -0,0 +1,15 @@
+from __future__ import annotations
+
+import logging
+
+
+def configure_app_logging() -> None:
+    """Configure compact application logging once."""
+
+    if logging.getLogger().handlers:
+        return
+
+    logging.basicConfig(
+        level=logging.INFO,
+        format="%(asctime)s %(levelname)s %(name)s %(message)s",
+    )

core/app_state.py

@@ -0,0 +1,56 @@
+from __future__ import annotations
+
+import logging
+from dataclasses import asdict
+
+from core.events import Event, EventBus, EventType
+from tracking.trackio_client import TrackingClient
+
+
+class AppState:
+    """Shared in-process state for UI events and lightweight traces."""
+
+    def __init__(
+        self,
+        event_bus: EventBus | None = None,
+        tracking_client: TrackingClient | None = None,
+    ) -> None:
+        self.event_bus = event_bus or EventBus()
+        self.events: list[Event] = []
+        self.logger = logging.getLogger("openbmb_workbench")
+        self.tracking_client = tracking_client or TrackingClient()
+        self.tracking_client.init()
+
+    def emit(self, event: Event) -> None:
+        self.events.append(event)
+        self.logger.info("event=%s payload=%s", event.type.value, event.payload)
+        self.tracking_client.log(event.type.value, event.payload)
+        self.event_bus.emit(event)
+
+    def recent_events(self, limit: int = 20) -> list[dict]:
+        recent = self.events[-limit:]
+        return [asdict(event) for event in recent]
+
+
+def emit_inference_response(
+    mode: str,
+    model_id: str,
+    backend: str,
+    response: str,
+    state: AppState | None = None,
+) -> None:
+    target_state = state or APP_STATE
+    target_state.emit(
+        Event(
+            EventType.INFERENCE_RESPONSE,
+            {
+                "mode": mode,
+                "model_id": model_id,
+                "backend": backend,
+                "response_chars": len(response),
+            },
+        )
+    )
+
+
+APP_STATE = AppState()

core/deployment.py

@@ -0,0 +1,77 @@
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+
+LOCAL_DEPLOYMENT = "local"
+SPACE_DEPLOYMENT = "space"
+VALID_DEPLOYMENTS = {LOCAL_DEPLOYMENT, SPACE_DEPLOYMENT}
+PLACEHOLDER_BACKEND = "placeholder"
+
+
+@dataclass(frozen=True)
+class DeploymentPolicy:
+    mode: str = LOCAL_DEPLOYMENT
+
+    @property
+    def is_space(self) -> bool:
+        return self.mode == SPACE_DEPLOYMENT
+
+    @property
+    def allow_placeholder_backend(self) -> bool:
+        return not self.is_space
+
+    @property
+    def allow_demo_mode(self) -> bool:
+        return not self.is_space
+
+
+def deployment_mode(env_value: str | None = None) -> str:
+    raw_value = env_value if env_value is not None else os.getenv("WORKBENCH_DEPLOYMENT", "")
+    value = raw_value.strip().lower() or LOCAL_DEPLOYMENT
+    if value not in VALID_DEPLOYMENTS:
+        raise ValueError(
+            "WORKBENCH_DEPLOYMENT must be 'local' or 'space'; "
+            f"got {raw_value!r}."
+        )
+    return value
+
+
+def current_policy() -> DeploymentPolicy:
+    return DeploymentPolicy(deployment_mode())
+
+
+def filter_backends_for_policy(backends: list[str], policy: DeploymentPolicy) -> list[str]:
+    if policy.allow_placeholder_backend:
+        return backends
+    return [backend for backend in backends if backend != PLACEHOLDER_BACKEND]
+
+
+def default_backend_for_policy(
+    backends: list[str],
+    preferred: str,
+    policy: DeploymentPolicy,
+) -> str:
+    allowed = filter_backends_for_policy(backends, policy)
+    if preferred in allowed:
+        return preferred
+    if allowed:
+        return allowed[0]
+    raise ValueError("No real backend is available for this deployment policy.")
+
+
+def ensure_backend_allowed(backend: str, policy: DeploymentPolicy) -> None:
+    if backend == PLACEHOLDER_BACKEND and not policy.allow_placeholder_backend:
+        raise ValueError(
+            "Placeholder inference is disabled in WORKBENCH_DEPLOYMENT=space. "
+            "Select a real backend such as transformers, Ollama, llama.cpp, "
+            "SGLang, or an OpenAI-compatible local server."
+        )
+
+
+def ensure_demo_mode_allowed(policy: DeploymentPolicy) -> None:
+    if not policy.allow_demo_mode:
+        raise ValueError(
+            "Demo/no-model mode is disabled in WORKBENCH_DEPLOYMENT=space. "
+            "Use a real OpenBMB model configuration for the deployed app."
+        )

core/events.py

@@ -0,0 +1,54 @@
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass, field
+from enum import Enum
+from typing import Any
+
+
+class StrEnum(str, Enum):
+    """Python 3.10-compatible subset of enum.StrEnum."""
+
+    def __str__(self) -> str:
+        return str(self.value)
+
+
+class EventType(StrEnum):
+    """Event names shared across future services."""
+
+    DATASET_LOADED = "dataset_loaded"
+    INFERENCE_REQUEST = "inference_request"
+    INFERENCE_RESPONSE = "inference_response"
+    UI_ERROR = "ui_error"
+    FIELD_NOTE_SAVED = "field_note_saved"
+    TRAINING_STARTED = "training_started"
+    EXPORT_STARTED = "export_started"
+
+
+@dataclass
+class Event:
+    """A lightweight event payload."""
+
+    type: EventType
+    payload: dict[str, Any] = field(default_factory=dict)
+
+
+class EventBus:
+    """In-process event dispatcher for low-volume app events."""
+
+    def __init__(self) -> None:
+        self._handlers: dict[EventType, list[Callable[[Event], None]]] = {}
+
+    def on(self, event_type: EventType) -> Callable:
+        def decorator(fn: Callable[[Event], None]) -> Callable[[Event], None]:
+            self._handlers.setdefault(event_type, []).append(fn)
+            return fn
+
+        return decorator
+
+    def emit(self, event: Event) -> None:
+        for handler in self._handlers.get(event.type, []):
+            handler(event)
+
+
+bus = EventBus()

core/file_exports.py

@@ -0,0 +1,16 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+
+def copy_text_file_or_empty(source_path: str | Path, output_path: str | Path) -> Path:
+    """Copy a text file to an export path, or create an empty export."""
+
+    source = Path(source_path)
+    output = Path(output_path)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    if source.exists():
+        output.write_text(source.read_text(encoding="utf-8"), encoding="utf-8")
+    else:
+        output.write_text("", encoding="utf-8")
+    return output

core/registry.py

@@ -0,0 +1,24 @@
+from __future__ import annotations
+
+from typing import Generic, TypeVar
+
+T = TypeVar("T")
+
+
+class Registry(Generic[T]):
+    """Small name-to-service registry."""
+
+    def __init__(self) -> None:
+        self._items: dict[str, T] = {}
+
+    def register(self, name: str, item: T) -> None:
+        self._items[name] = item
+
+    def get(self, name: str) -> T:
+        if name not in self._items:
+            available = ", ".join(self._items)
+            raise KeyError(f"{name!r} is not registered. Available: {available}")
+        return self._items[name]
+
+    def list(self) -> list[str]:
+        return list(self._items)

core/spaces_runtime.py

@@ -0,0 +1,21 @@
+from __future__ import annotations
+
+from collections.abc import Callable
+from importlib import import_module
+from types import SimpleNamespace
+from typing import Any, TypeVar
+
+F = TypeVar("F", bound=Callable[..., Any])
+spaces: Any
+
+try:
+    spaces = import_module("spaces")
+except ImportError:
+
+    def _gpu(*_args: Any, **_kwargs: Any) -> Callable[[F], F]:
+        def decorator(func: F) -> F:
+            return func
+
+        return decorator
+
+    spaces = SimpleNamespace(GPU=_gpu)

core/tab_feedback.py

@@ -0,0 +1,24 @@
+from __future__ import annotations
+
+from typing import Any
+
+from core.app_state import APP_STATE, AppState
+from core.events import Event, EventType
+
+
+def status_ok(message: str) -> str:
+    return f"Status: {message}"
+
+
+def emit_tab_error(
+    tab: str,
+    message: str,
+    context: dict[str, Any] | None = None,
+    state: AppState | None = None,
+) -> str:
+    target_state = state or APP_STATE
+    payload = {"tab": tab, "message": message}
+    if context:
+        payload.update(context)
+    target_state.emit(Event(EventType.UI_ERROR, payload))
+    return f"Error: {message}"

data/.gitkeep

@@ -0,0 +1 @@
+

datasets/__init__.py

@@ -0,0 +1 @@
+"""Dataset and correction-loop helpers."""

datasets/field_notes.py

@@ -0,0 +1,252 @@
+from __future__ import annotations
+
+import csv
+import datetime as dt
+import json
+import sqlite3
+from contextlib import closing
+from dataclasses import asdict, dataclass
+from pathlib import Path
+
+UTC = getattr(dt, "UTC", dt.timezone.utc)  # noqa: UP017
+
+
+@dataclass
+class FieldNote:
+    """One human correction record."""
+
+    created_at: str
+    model_id: str
+    prompt: str
+    response: str
+    correction: str
+    tags: str
+    image_path: str = ""
+    video_path: str = ""
+    use_for_training: bool = True
+
+    @classmethod
+    def create(
+        cls,
+        model_id: str,
+        prompt: str,
+        response: str,
+        correction: str,
+        tags: str,
+        image_path: str = "",
+        video_path: str = "",
+        use_for_training: bool = True,
+    ) -> FieldNote:
+        return cls(
+            created_at=dt.datetime.now(UTC).isoformat(),
+            model_id=model_id,
+            prompt=prompt,
+            response=response,
+            correction=correction,
+            tags=tags,
+            image_path=image_path,
+            video_path=video_path,
+            use_for_training=use_for_training,
+        )
+
+    @classmethod
+    def from_row(cls, row: dict[str, str]) -> FieldNote:
+        use_for_training = str(row.get("use_for_training", "true")).lower() in {
+            "1",
+            "true",
+            "yes",
+        }
+        return cls(
+            created_at=row["created_at"],
+            model_id=row["model_id"],
+            prompt=row["prompt"],
+            response=row["response"],
+            correction=row["correction"],
+            tags=row["tags"],
+            image_path=row.get("image_path", ""),
+            video_path=row.get("video_path", ""),
+            use_for_training=use_for_training,
+        )
+
+    def to_dict(self) -> dict[str, object]:
+        return asdict(self)
+
+
+class FieldNoteStore:
+    """CSV-backed field note storage."""
+
+    def __init__(self, path: str | Path = "data/field_notes.csv") -> None:
+        self.path = Path(path)
+
+    def save(self, note: FieldNote) -> Path:
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        is_new = not self.path.exists()
+
+        with self.path.open("a", newline="", encoding="utf-8") as f:
+            writer = csv.DictWriter(f, fieldnames=list(note.to_dict()))
+            if is_new:
+                writer.writeheader()
+            writer.writerow(note.to_dict())
+
+        return self.path
+
+    def list_notes(
+        self,
+        corrected_only: bool = False,
+        tag: str = "",
+        training_only: bool = False,
+    ) -> list[FieldNote]:
+        if not self.path.exists():
+            return []
+
+        with self.path.open(newline="", encoding="utf-8") as f:
+            rows = list(csv.DictReader(f))
+
+        notes = [FieldNote.from_row(row) for row in rows]
+        if corrected_only:
+            notes = [note for note in notes if note.correction.strip()]
+        if tag:
+            notes = [note for note in notes if tag in _split_tags(note.tags)]
+        if training_only:
+            notes = [note for note in notes if note.use_for_training]
+        return notes
+
+    def export_jsonl(
+        self,
+        output_path: str | Path = "data/field_notes.jsonl",
+        corrected_only: bool = True,
+        training_only: bool = True,
+    ) -> Path:
+        output = Path(output_path)
+        output.parent.mkdir(parents=True, exist_ok=True)
+
+        notes = self.list_notes(
+            corrected_only=corrected_only,
+            training_only=training_only,
+        )
+        with output.open("w", encoding="utf-8") as f:
+            for note in notes:
+                f.write(json.dumps(note.to_dict(), ensure_ascii=False) + "\n")
+
+        return output
+
+    def export_hf_dataset(
+        self,
+        output_dir: str | Path = "data/hf_field_notes",
+        corrected_only: bool = True,
+        training_only: bool = True,
+    ) -> Path:
+        target = Path(output_dir)
+        target.mkdir(parents=True, exist_ok=True)
+        data_file = self.export_jsonl(
+            target / "data.jsonl",
+            corrected_only=corrected_only,
+            training_only=training_only,
+        )
+        (target / "README.md").write_text(
+            "# Field Notes Dataset\n\n"
+            "Local export generated by OpenBMB Local AI Workbench.\n\n"
+            f"- Data file: `{data_file.name}`\n"
+            "- Intended split: `train`\n",
+            encoding="utf-8",
+        )
+        return target
+
+
+class SQLiteFieldNoteStore:
+    """SQLite-backed field note storage for larger correction loops."""
+
+    def __init__(self, path: str | Path = "data/field_notes.sqlite") -> None:
+        self.path = Path(path)
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        self._init_schema()
+
+    def _connect(self) -> sqlite3.Connection:
+        return sqlite3.connect(self.path)
+
+    def _init_schema(self) -> None:
+        with closing(self._connect()) as conn:
+            conn.execute(
+                """
+                CREATE TABLE IF NOT EXISTS field_notes (
+                    created_at TEXT NOT NULL,
+                    model_id TEXT NOT NULL,
+                    prompt TEXT NOT NULL,
+                    response TEXT NOT NULL,
+                    correction TEXT NOT NULL,
+                    tags TEXT NOT NULL,
+                    image_path TEXT NOT NULL,
+                    video_path TEXT NOT NULL,
+                    use_for_training INTEGER NOT NULL
+                )
+                """
+            )
+            conn.commit()
+
+    def save(self, note: FieldNote) -> Path:
+        with closing(self._connect()) as conn:
+            conn.execute(
+                """
+                INSERT INTO field_notes (
+                    created_at, model_id, prompt, response, correction, tags,
+                    image_path, video_path, use_for_training
+                )
+                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+                """,
+                (
+                    note.created_at,
+                    note.model_id,
+                    note.prompt,
+                    note.response,
+                    note.correction,
+                    note.tags,
+                    note.image_path,
+                    note.video_path,
+                    int(note.use_for_training),
+                ),
+            )
+            conn.commit()
+        return self.path
+
+    def list_notes(
+        self,
+        corrected_only: bool = False,
+        tag: str = "",
+        training_only: bool = False,
+    ) -> list[FieldNote]:
+        with closing(self._connect()) as conn:
+            conn.row_factory = sqlite3.Row
+            rows = conn.execute(
+                """
+                SELECT created_at, model_id, prompt, response, correction, tags,
+                       image_path, video_path, use_for_training
+                FROM field_notes
+                ORDER BY created_at
+                """
+            ).fetchall()
+
+        notes = [
+            FieldNote(
+                created_at=str(row["created_at"]),
+                model_id=str(row["model_id"]),
+                prompt=str(row["prompt"]),
+                response=str(row["response"]),
+                correction=str(row["correction"]),
+                tags=str(row["tags"]),
+                image_path=str(row["image_path"]),
+                video_path=str(row["video_path"]),
+                use_for_training=bool(row["use_for_training"]),
+            )
+            for row in rows
+        ]
+        if corrected_only:
+            notes = [note for note in notes if note.correction.strip()]
+        if tag:
+            notes = [note for note in notes if tag in _split_tags(note.tags)]
+        if training_only:
+            notes = [note for note in notes if note.use_for_training]
+        return notes
+
+
+def _split_tags(tags: str) -> set[str]:
+    return {tag.strip() for tag in tags.split(",") if tag.strip()}

datasets/loader.py

@@ -0,0 +1,173 @@
+from __future__ import annotations
+
+import csv
+import json
+from dataclasses import dataclass
+from importlib import import_module
+from pathlib import Path
+from typing import Any
+
+
+@dataclass(frozen=True)
+class DatasetPreview:
+    """Small local dataset preview result."""
+
+    source: str
+    rows: int
+    columns: list[str]
+    samples: list[dict[str, Any]]
+
+    def as_table(self) -> list[list[str]]:
+        table = [
+            ["source", self.source],
+            ["rows", str(self.rows)],
+            ["columns", ", ".join(self.columns)],
+        ]
+        for index, sample in enumerate(self.samples, start=1):
+            table.append([f"sample_{index}", json.dumps(sample, ensure_ascii=False)])
+        return table
+
+
+@dataclass(frozen=True)
+class DatasetStats:
+    """Basic dataset statistics for local preview and tools."""
+
+    source: str
+    rows: int
+    columns: int
+    column_names: list[str]
+    non_empty_by_column: dict[str, int]
+
+    def as_dict(self) -> dict[str, Any]:
+        return {
+            "source": self.source,
+            "rows": self.rows,
+            "columns": self.columns,
+            "column_names": self.column_names,
+            "non_empty_by_column": self.non_empty_by_column,
+        }
+
+
+@dataclass(frozen=True)
+class HuggingFaceDatasetPreview:
+    """Small preview result for optional Hugging Face datasets integration."""
+
+    dataset_id: str
+    split: str
+    rows: int
+    columns: list[str]
+    samples: list[dict[str, Any]]
+    status: str = "loaded"
+
+    def as_table(self) -> list[list[str]]:
+        table = [
+            ["dataset_id", self.dataset_id],
+            ["split", self.split],
+            ["rows", str(self.rows)],
+            ["columns", ", ".join(self.columns)],
+            ["status", self.status],
+        ]
+        for index, sample in enumerate(self.samples, start=1):
+            table.append([f"sample_{index}", json.dumps(sample, ensure_ascii=False)])
+        return table
+
+
+def preview_local_dataset(path: str | Path, limit: int = 5) -> DatasetPreview:
+    dataset_path = Path(path)
+    if not dataset_path.exists():
+        raise FileNotFoundError(f"Dataset path not found: {dataset_path}")
+    if dataset_path.suffix.lower() == ".csv":
+        return _preview_csv(dataset_path, limit)
+    if dataset_path.suffix.lower() in {".jsonl", ".ndjson"}:
+        return _preview_jsonl(dataset_path, limit)
+    raise ValueError(f"Unsupported dataset format: {dataset_path.suffix}")
+
+
+def dataset_statistics(path: str | Path) -> DatasetStats:
+    preview = preview_local_dataset(path, limit=0)
+    non_empty_by_column = {column: 0 for column in preview.columns}
+
+    dataset_path = Path(path)
+    if dataset_path.suffix.lower() == ".csv":
+        with dataset_path.open(newline="", encoding="utf-8") as f:
+            for row in csv.DictReader(f):
+                for column in preview.columns:
+                    if str(row.get(column, "")).strip():
+                        non_empty_by_column[column] += 1
+    else:
+        with dataset_path.open(encoding="utf-8") as f:
+            for line in f:
+                if not line.strip():
+                    continue
+                row = json.loads(line)
+                for column in preview.columns:
+                    if str(row.get(column, "")).strip():
+                        non_empty_by_column[column] += 1
+
+    return DatasetStats(
+        source=preview.source,
+        rows=preview.rows,
+        columns=len(preview.columns),
+        column_names=preview.columns,
+        non_empty_by_column=non_empty_by_column,
+    )
+
+
+def preview_huggingface_dataset(
+    dataset_id: str,
+    split: str = "train",
+    limit: int = 5,
+) -> HuggingFaceDatasetPreview:
+    datasets_module = import_module("datasets")
+    load_dataset = getattr(datasets_module, "load_dataset", None)
+    if load_dataset is None:
+        return HuggingFaceDatasetPreview(
+            dataset_id=dataset_id,
+            split=split,
+            rows=0,
+            columns=[],
+            samples=[],
+            status="Hugging Face package 'datasets' is not installed.",
+        )
+
+    dataset = load_dataset(dataset_id, split=split)
+    rows = len(dataset)
+    columns = list(dataset.column_names)
+    samples = [dict(dataset[index]) for index in range(min(limit, rows))]
+    return HuggingFaceDatasetPreview(dataset_id, split, rows, columns, samples)
+
+
+def _preview_csv(path: Path, limit: int) -> DatasetPreview:
+    samples: list[dict[str, Any]] = []
+    rows = 0
+    columns: list[str] = []
+
+    with path.open(newline="", encoding="utf-8") as f:
+        reader = csv.DictReader(f)
+        columns = list(reader.fieldnames or [])
+        for row in reader:
+            rows += 1
+            if len(samples) < limit:
+                samples.append(dict(row))
+
+    return DatasetPreview(str(path), rows, columns, samples)
+
+
+def _preview_jsonl(path: Path, limit: int) -> DatasetPreview:
+    samples: list[dict[str, Any]] = []
+    rows = 0
+    columns: set[str] = set()
+
+    with path.open(encoding="utf-8") as f:
+        for line in f:
+            if not line.strip():
+                continue
+            rows += 1
+            item = json.loads(line)
+            if not isinstance(item, dict):
+                raise ValueError("JSONL rows must be objects")
+            columns.update(item)
+            if len(samples) < limit:
+                samples.append(item)
+
+    return DatasetPreview(str(path), rows, sorted(columns), samples)

datasets/ocr.py

@@ -0,0 +1,137 @@
+from __future__ import annotations
+
+import csv
+import json
+from dataclasses import asdict, dataclass
+from pathlib import Path
+from typing import Any
+
+from datasets.field_notes import FieldNote, FieldNoteStore
+
+
+@dataclass(frozen=True)
+class OCRPrediction:
+    """One local OCR prediction that may need human correction."""
+
+    source_path: str
+    text: str
+    confidence: float
+    page: str = ""
+
+    @classmethod
+    def from_row(cls, row: dict[str, Any]) -> OCRPrediction:
+        source_path = _first_present(row, ["source_path", "image_path", "file_path", "path"])
+        text = _first_present(row, ["text", "prediction", "ocr_text", "response"])
+        confidence_raw = _first_present(row, ["confidence", "score", "probability"], "0")
+        return cls(
+            source_path=source_path,
+            text=text,
+            confidence=_parse_confidence(confidence_raw),
+            page=str(row.get("page", "")),
+        )
+
+    def to_dict(self) -> dict[str, object]:
+        return asdict(self)
+
+
+def load_ocr_predictions(path: str | Path) -> list[OCRPrediction]:
+    """Load OCR predictions from local CSV, JSONL, or NDJSON files."""
+
+    source = Path(path)
+    if not source.exists():
+        raise FileNotFoundError(f"OCR prediction file not found: {source}")
+
+    suffix = source.suffix.lower()
+    if suffix == ".csv":
+        with source.open(newline="", encoding="utf-8") as f:
+            return [OCRPrediction.from_row(row) for row in csv.DictReader(f)]
+    if suffix in {".jsonl", ".ndjson"}:
+        return [
+            OCRPrediction.from_row(json.loads(line))
+            for line in source.read_text(encoding="utf-8").splitlines()
+            if line.strip()
+        ]
+    raise ValueError("OCR predictions must be a .csv, .jsonl, or .ndjson file.")
+
+
+def uncertain_predictions(
+    predictions: list[OCRPrediction],
+    confidence_threshold: float = 0.8,
+) -> list[OCRPrediction]:
+    return [
+        prediction
+        for prediction in predictions
+        if prediction.confidence <= confidence_threshold or not prediction.text.strip()
+    ]
+
+
+def import_uncertain_predictions(
+    store: FieldNoteStore,
+    predictions: list[OCRPrediction],
+    model_id: str,
+    confidence_threshold: float = 0.8,
+    tags: str = "ocr,uncertain",
+) -> int:
+    imported = 0
+    for prediction in uncertain_predictions(predictions, confidence_threshold):
+        page_note = f" page {prediction.page}" if prediction.page else ""
+        note = FieldNote.create(
+            model_id=model_id,
+            prompt=f"Review OCR text for {prediction.source_path}{page_note}.",
+            response=prediction.text,
+            correction="",
+            tags=tags,
+            image_path=prediction.source_path,
+            use_for_training=False,
+        )
+        store.save(note)
+        imported += 1
+    return imported
+
+
+def export_corrected_ocr_notes(
+    store: FieldNoteStore,
+    output_path: str | Path = "data/ocr_corrections.jsonl",
+) -> Path:
+    output = Path(output_path)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    notes = store.list_notes(corrected_only=True, tag="ocr")
+    with output.open("w", encoding="utf-8") as f:
+        for note in notes:
+            row = {
+                "source_path": note.image_path,
+                "predicted_text": note.response,
+                "corrected_text": note.correction,
+                "model_id": note.model_id,
+                "created_at": note.created_at,
+                "tags": note.tags,
+            }
+            f.write(json.dumps(row, ensure_ascii=False) + "\n")
+    return output
+
+
+def ocr_import_summary(path: str | Path, confidence_threshold: float = 0.8) -> dict[str, object]:
+    predictions = load_ocr_predictions(path)
+    uncertain = uncertain_predictions(predictions, confidence_threshold)
+    return {
+        "source": str(path),
+        "rows": len(predictions),
+        "uncertain_rows": len(uncertain),
+        "confidence_threshold": confidence_threshold,
+        "sample": [prediction.to_dict() for prediction in uncertain[:5]],
+    }
+
+
+def _first_present(row: dict[str, Any], names: list[str], default: str = "") -> str:
+    for name in names:
+        value = row.get(name)
+        if value is not None:
+            return str(value)
+    return default
+
+
+def _parse_confidence(value: str) -> float:
+    try:
+        return float(value)
+    except ValueError:
+        return 0.0

datasets/synthetic.py

@@ -0,0 +1,90 @@
+from __future__ import annotations
+
+import json
+from dataclasses import asdict, dataclass
+from pathlib import Path
+
+
+@dataclass(frozen=True)
+class SyntheticExample:
+    prompt: str
+    response: str
+    correction: str
+    tags: list[str]
+    source: str = "synthetic"
+
+    def as_dict(self) -> dict:
+        return asdict(self)
+
+
+def generate_synthetic_examples(
+    topic: str,
+    count: int,
+    tag: str = "synthetic",
+) -> list[SyntheticExample]:
+    if count < 0:
+        raise ValueError("count must be non-negative")
+    cleaned_topic = topic.strip() or "local AI workflow"
+    return [
+        SyntheticExample(
+            prompt=f"Write a concise field note about {cleaned_topic} #{index}.",
+            response=f"Draft note {index}: {cleaned_topic}.",
+            correction=f"Corrected note {index}: {cleaned_topic} with concrete evidence.",
+            tags=[tag, cleaned_topic.replace(" ", "-").lower()],
+        )
+        for index in range(1, count + 1)
+    ]
+
+
+def validate_synthetic_example(example: SyntheticExample) -> list[str]:
+    errors = []
+    if not example.prompt.strip():
+        errors.append("prompt is required")
+    if not example.response.strip():
+        errors.append("response is required")
+    if not example.correction.strip():
+        errors.append("correction is required")
+    if not example.tags:
+        errors.append("at least one tag is required")
+    return errors
+
+
+def quality_filter_examples(examples: list[SyntheticExample]) -> list[SyntheticExample]:
+    return [
+        example
+        for example in examples
+        if not validate_synthetic_example(example)
+        and example.prompt != example.response
+        and example.response != example.correction
+    ]
+
+
+def augment_examples(
+    examples: list[SyntheticExample],
+    extra_tag: str = "augmented",
+) -> list[SyntheticExample]:
+    augmented = list(examples)
+    for example in examples:
+        tags = list(dict.fromkeys([*example.tags, extra_tag]))
+        augmented.append(
+            SyntheticExample(
+                prompt=f"{example.prompt} Include one verification detail.",
+                response=example.response,
+                correction=f"{example.correction} Verification detail recorded.",
+                tags=tags,
+                source=example.source,
+            )
+        )
+    return augmented
+
+
+def export_synthetic_jsonl(
+    examples: list[SyntheticExample],
+    output_path: str | Path = "data/synthetic_examples.jsonl",
+) -> Path:
+    output = Path(output_path)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    with output.open("w", encoding="utf-8") as f:
+        for example in examples:
+            f.write(json.dumps(example.as_dict(), ensure_ascii=False) + "\n")
+    return output

deployment/hf_space.py

@@ -0,0 +1,98 @@
+from __future__ import annotations
+
+from dataclasses import asdict, dataclass
+from pathlib import Path
+
+SPACE_REQUIRED_FILES = [
+    "app.py",
+    "plant_space_app.py",
+    "requirements.txt",
+    "README.md",
+    "config/models.yaml",
+    "config/training.yaml",
+]
+
+
+@dataclass(frozen=True)
+class SpaceBuildStatus:
+    required_files_present: bool
+    missing_files: list[str]
+    readme_has_space_metadata: bool
+    has_space_remote: bool
+
+    def as_dict(self) -> dict:
+        return asdict(self)
+
+
+def missing_required_files(root: str | Path = ".") -> list[str]:
+    repo_root = Path(root)
+    return [path for path in SPACE_REQUIRED_FILES if not (repo_root / path).exists()]
+
+
+def readme_has_space_metadata(root: str | Path = ".") -> bool:
+    readme_path = Path(root) / "README.md"
+    if not readme_path.exists():
+        return False
+    text = readme_path.read_text(encoding="utf-8")
+    return text.startswith("---\n") and "\nsdk: gradio\n" in text and "\napp_file: app.py\n" in text
+
+
+def has_space_remote(remote_output: str, remote_name: str = "space") -> bool:
+    return any(line.startswith(f"{remote_name}\t") for line in remote_output.splitlines())
+
+
+def space_build_status(root: str | Path = ".", remote_output: str = "") -> SpaceBuildStatus:
+    missing = missing_required_files(root)
+    return SpaceBuildStatus(
+        required_files_present=not missing,
+        missing_files=missing,
+        readme_has_space_metadata=readme_has_space_metadata(root),
+        has_space_remote=(
+            has_space_remote(remote_output, "space")
+            or has_space_remote(remote_output, "space-workbench")
+            or has_space_remote(remote_output, "space-plant")
+        ),
+    )
+
+
+def create_space_commands(
+    hf_user: str,
+    space_name: str = "openbmb-local-ai-workbench",
+    branch: str = "main",
+    remote_name: str = "space",
+    app_file: str = "app.py",
+) -> list[str]:
+    if not hf_user.strip():
+        raise ValueError("hf_user is required")
+    if not space_name.strip():
+        raise ValueError("space_name is required")
+    if not remote_name.strip():
+        raise ValueError("remote_name is required")
+    if not app_file.strip():
+        raise ValueError("app_file is required")
+    return [
+        "hf auth login",
+        f"hf repo create {hf_user}/{space_name} --type space --space-sdk gradio",
+        f"git remote add {remote_name} https://huggingface.co/spaces/{hf_user}/{space_name}",
+        f"git push {remote_name} {branch}",
+        f"Space metadata app_file must be {app_file} for this deployment.",
+    ]
+
+
+def hackathon_space_commands(branch: str = "main") -> dict[str, list[str]]:
+    return {
+        "workbench": create_space_commands(
+            "build-small-hackathon",
+            "workbench",
+            branch,
+            remote_name="space-workbench",
+            app_file="app.py",
+        ),
+        "plant": create_space_commands(
+            "build-small-hackathon",
+            "plant_identification_tool",
+            branch,
+            remote_name="space-plant",
+            app_file="plant_space_app.py",
+        ),
+    }

docs/ACCEPTANCE_CRITERIA.md

@@ -0,0 +1,96 @@
+# Acceptance Criteria
+
+This file defines how we decide whether something is really done.
+
+## Global Rule
+
+Do not mark a feature done unless:
+
+1. The files exist.
+2. The feature is linked from docs where relevant.
+3. The behavior was tested locally, or the blocker is written in `IMPLEMENTATION_STATUS.md`.
+4. The checklist in `TASKS.md` is updated.
+5. Unit and user-story tests are added or updated.
+
+If a bug or failed check is discovered, add or update a test that reproduces it before marking
+the fix done.
+
+## MVP App
+
+Done when:
+
+- `python app.py` launches without errors.
+- The Gradio UI opens locally.
+- Chat tab returns a response.
+- Vision tab accepts an image and returns a response.
+- Field Notes tab saves a row to `data/field_notes.csv`.
+- README explains that current inference is placeholder/local-safe.
+- Unit and user-story tests pass.
+
+## Testing And Quality
+
+Done when:
+
+- Unit tests cover new logic.
+- User-story tests cover new workflows.
+- `scripts/run_tests.ps1` passes.
+- `ruff`, `mypy`, `pylint`, `bandit`, and `pip-audit` are installed or blockers are documented.
+- `scripts/run_quality.ps1` passes before release or documents exact blockers.
+
+## Real Local Inference
+
+Done when:
+
+- A user can choose a backend.
+- Model loading happens only after an explicit click.
+- The app does not download weights on startup.
+- At least one small model returns a real response.
+- Setup steps are documented.
+
+## Hugging Face Space
+
+Done when:
+
+- Space exists.
+- Repo is pushed.
+- Build succeeds.
+- Public/private Space URL is added to README.
+- Any hardware or model limitations are documented.
+
+## GitHub
+
+Done when:
+
+- GitHub repo exists.
+- Initial commit is pushed.
+- README includes GitHub URL.
+- Large files and secrets are excluded.
+
+## Training
+
+Done when:
+
+- Training input format is documented.
+- A dry-run or actual LoRA run completes.
+- Output checkpoint path is shown.
+- Metrics or logs are captured.
+- Hardware assumptions are documented.
+
+## Field Notes
+
+Done when:
+
+- Corrections save reliably.
+- Corrected records can be exported.
+- Export schema is documented.
+- Export can feed the training pipeline.
+
+## Hackathon Submission
+
+Done when:
+
+- Space URL is live.
+- Demo video is recorded.
+- Social post is published or drafted.
+- Submission form fields are complete.
+- Model-size compliance is documented.

docs/ARCHITECTURE.md

@@ -0,0 +1,575 @@
+# Architecture
+
+The project is intentionally small at first. The PRD describes a large workbench; this repo starts
+with the smallest version that can grow into it.
+
+## High-Level Flow
+
+```text
+app.py
+  loads config/models.yaml
+  configures lightweight logging
+  builds Gradio tabs
+  passes model catalog to UI modules
+
+ui/*
+  defines each Gradio tab
+  calls service classes
+  emits local app events for inference, datasets, and field notes
+  uses shared progress settings for callback loading indicators
+
+agent/*
+  holds deterministic local agent planning and trace export helpers
+
+models/*
+  holds model catalog, local backend config, and inference services
+
+datasets/*
+  stores dataset, synthetic data, and correction-loop helpers
+
+mcp_tools/*
+  holds local tool functions, VINDEX call planning, and Gradio-native MCP bridge metadata
+
+config/*
+  holds model and training settings
+
+training/*
+  holds non-executing training, LoRA request, evaluation, and export planning helpers
+
+tracking/*
+  holds local JSONL tracing and optional Trackio integration
+
+deployment/*
+  holds Hugging Face Space deployment planning and validation helpers
+
+plant/*
+  holds the first reference domain app built from the template
+  can run standalone with python -m plant.app --no-model
+  keeps heavy model dependencies optional
+
+core/*
+  shared app state, event, logging, and registry helpers
+```
+
+## Files And Classes
+
+### `app.py`
+
+Builds and launches the Gradio app.
+
+- `build_app()` creates the Gradio `Blocks` app.
+- Loads the model catalog from `config/models.yaml`.
+- Registers the current UI tabs.
+- `APP_CSS` defines compact responsive layout rules for app width, mobile padding, scrollable tabs,
+  and button touch targets.
+
+### `plant/app.py`
+
+Standalone Plant Discovery reference app built from the template.
+
+- `build_app(no_model=True)` creates a Gradio app without loading model weights.
+- Loads `plant/models.yaml`.
+- Builds a local species index.
+- Reuses `datasets.field_notes.FieldNoteStore` for corrections.
+- Uses `DemoPlantVisionService` for screenshots/tests or `PlantVisionService` for OpenBMB
+  MiniCPM-V zero-shot and fine-tuned adapter inference.
+
+### `plant/plant_service.py`
+
+Domain service and schema for Plant Discovery.
+
+- `PlantID` is the structured output schema.
+- `DemoPlantVisionService` provides deterministic no-model results.
+- `PlantVisionService` lazy-loads optional MiniCPM-V dependencies only during identification.
+- `PlantVisionService.from_config(..., "plant_vlm_finetuned")` can load a PEFT adapter after a real
+  adapter repo is configured.
+- `extract_json_object()` and `parse_plant_response()` make model JSON output testable.
+
+### `plant/training.py`
+
+Non-executing training planner for Plant Discovery.
+
+- `build_plant_training_plan()` returns SWIFT and LLaMA-Factory command previews.
+- `plant_training_dependency_report()` reports optional training dependency availability.
+- `write_llamafactory_dataset_info()` writes a dataset-info preview for LLaMA-Factory workflows.
+- Training is never started by the Gradio UI or script.
+
+### `plant/plant_loader.py`
+
+Domain data and export helpers for Plant Discovery.
+
+- `PlantRecord` normalizes plant examples into training rows.
+- `LocalFolderLoader` maps species folders to image metadata.
+- `SpeciesIndexBuilder` builds a no-network species index with demo fallback.
+- `FieldNotesPlantExporter` exports corrected field notes to plant training JSONL.
+
+### `plant/plant_tab.py`
+
+Focused Gradio UI for Plant Discovery.
+
+- Identify tab uploads images and renders a safe escaped result card.
+- Field Guide tab searches the species index.
+- Corrections tab saves and exports training-ready corrections.
+- Stats tab summarizes species and correction counts.
+- Training is represented as a non-executing plan, not a subprocess.
+
+### `plant/plant_tools.py`
+
+Optional local/MCP tools for Plant Discovery.
+
+- Pure functions can be tested without an MCP server.
+- `build_mcp_server()` imports `mcp` only when explicitly requested.
+- Tools expose identify, species search, correction save/export, stats, and training plan.
+
+### `models/model_catalog.py`
+
+Reads model configuration and turns it into typed Python objects.
+
+- `ModelInfo` describes one configured model.
+- `load_model_catalog(path)` reads YAML and returns all configured models.
+- `model_choices(catalog, model_type)` filters models for a UI dropdown.
+- `model_summary(model)` returns display metadata for the Gradio JSON panel.
+- `backend_capabilities` maps each model to supported local backend capabilities.
+
+### `models/placeholder_service.py`
+
+Deterministic placeholder model service used before real inference is wired.
+
+- `PlaceholderModelService.chat()` returns a deterministic text response.
+- `PlaceholderModelService.vision_chat()` returns a deterministic image/prompt response.
+
+This file should be replaced or complemented by real services such as:
+
+- `ollama_service.py`
+- `llama_cpp_service.py`
+- `openai_compatible_service.py`
+- `sglang_runner.py`
+- `minicpm_vision.py`
+- `transformers_text.py`
+- `sglang_service.py`
+
+### `models/base.py`
+
+Defines service contracts and backend status records.
+
+- `BackendStatus` describes whether a backend is available.
+- `TextModelService` is the text chat protocol.
+- `VisionModelService` is the vision chat protocol.
+
+### `models/ollama_service.py`
+
+Ollama-backed local inference client.
+
+- Checks whether `ollama` is installed and reachable.
+- Sends text and vision chat requests to `http://127.0.0.1:11434/api/chat`.
+- Lists locally available Ollama models through `/api/tags`.
+- Builds explicit `ollama pull <model>` commands for the Status tab.
+- Does not pull or download models automatically.
+
+### `models/llama_cpp_service.py`
+
+llama.cpp HTTP client for local GGUF inference.
+
+- Checks whether `llama-server` is installed and reachable.
+- Builds explicit `llama-server -m <model.gguf>` commands.
+- Supports `--mmproj <mmproj.gguf>` command metadata for multimodal models.
+- Sends text chat requests to `/v1/chat/completions`.
+- Does not download GGUF files or start background servers automatically.
+
+### `models/local_backend_config.py`
+
+User-local backend settings stored under ignored `data/local_backends.yaml`.
+
+- `LocalBackendConfig` stores llama.cpp server URL, OpenAI-compatible base URL, optional served
+  model name, GGUF path, mmproj path, context length, and GPU layers.
+- `save_local_backend_config()` writes local-only settings without touching tracked model config.
+- `build_llama_server_command()` returns the explicit command the user can run.
+- `local_backend_summary()` reports file status and confirms no startup downloads or automatic model loads.
+
+### `models/openai_compatible_service.py`
+
+Local OpenAI-compatible chat client for LM Studio, vLLM-style servers, or similar local endpoints.
+
+- Checks `/v1/models` for reachability.
+- Sends text chat requests to `/v1/chat/completions`.
+- Supports an optional served-model-name override for tools such as LM Studio.
+- Returns visible unavailable/request-failed messages instead of crashing the Gradio callback.
+- Does not call cloud APIs or download model weights.
+
+### `models/llama_cpp_python_service.py`
+
+Optional direct Python binding backend for GGUF inference.
+
+- Checks whether `llama_cpp` is importable.
+- Requires an explicit local GGUF path.
+- Does not download model files.
+- Provides text chat through `Llama.create_chat_completion()`.
+- Vision support remains routed through llama-server until mmproj/image serialization is wired.
+
+### `models/minicpm_vision.py`
+
+Optional MiniCPM vision backend.
+
+- Checks whether the `transformers` package is available.
+- Lazy-loads `AutoProcessor` and `AutoModelForImageTextToText` only when selected.
+- Formats image/text messages for image-text-to-text generation.
+- Maps thinking mode into the prompt template.
+- Provides a video support plan for future local frame sampling.
+
+### `models/sglang_runner.py`
+
+SGLang local server planner and OpenAI-compatible chat client.
+
+- Builds an explicit `python -m sglang.launch_server` command.
+- Includes MiniCPM tool parser configuration.
+- Checks `/health`, sends chat requests to `/v1/chat/completions`, and can request `/shutdown`.
+- Does not install SGLang, start a process, download model weights, or load a model on app startup.
+
+### `models/vllm_runner.py`
+
+vLLM local server planner and OpenAI-compatible chat client.
+
+- Builds explicit `vllm serve <model>` command plans.
+- Checks `/health`, parses Prometheus-style `/metrics`, and sends chat requests to
+  `/v1/chat/completions`.
+- Logs parsed benchmark metrics through `TrackingClient`.
+- Does not install vLLM, start a process, download model weights, or load a model on app startup.
+
+### `models/transformers_text.py`
+
+Optional Transformers text backend.
+
+- Checks whether the `transformers` package is installed.
+- Lazy-loads `AutoTokenizer` and `AutoModelForCausalLM` only when the backend is selected.
+- Reads `trust_remote_code`, device map, dtype, max token, and temperature settings from explicit config.
+- Provides a simple token-list streaming helper for future Gradio streaming wiring.
+- Does not download model weights on startup.
+
+### `models/service_factory.py`
+
+Creates the selected backend service for the UI.
+
+- `TEXT_SERVICE_REGISTRY` registers available text backend factories.
+- `VISION_SERVICE_REGISTRY` registers available vision backend factories.
+- `create_text_service()` chooses placeholder, llama.cpp, llama-cpp-python, Ollama,
+  OpenAI-compatible, SGLang, or Transformers text service.
+- `create_vision_service()` chooses placeholder, llama.cpp, llama-cpp-python, Ollama, or
+  Transformers MiniCPM vision service.
+- `backend_statuses()` reports current backend availability.
+- llama.cpp, llama-cpp-python, and OpenAI-compatible services read ignored local backend settings
+  when selected.
+
+### `ui/chat_tab.py`
+
+Builds the text chat tab.
+
+- Shows text models from the catalog.
+- Displays selected model metadata.
+- Calls the selected backend service.
+- Emits inference request and response events.
+
+### `ui/vision_tab.py`
+
+Builds the vision tab.
+
+- Shows vision models from the catalog.
+- Accepts an image and prompt.
+- Calls the selected backend service.
+- Emits inference request and response events.
+
+### `ui/dataset_tab.py`
+
+Local dataset preview surface.
+
+- Previews local CSV, JSONL, and NDJSON files.
+- Previews Hugging Face datasets when the optional external `datasets` package is installed.
+- Shows source, row count, columns, and sample rows.
+- Calculates basic local dataset statistics.
+- Emits dataset loaded events.
+
+Future behavior:
+
+- Serve dataset tools through the selected MCP path.
+
+### `ui/train_tab.py`
+
+Training planning and local evaluation surface.
+
+- Builds a LoRA dry-run training plan without launching training.
+- Builds a non-executing LoRA trainer request with dependency status.
+- Shows SWIFT/LLaMA-Factory vision fine-tuning plan.
+- Shows checkpoint output path, validation status, and hardware notes.
+- Runs local base-vs-tuned evaluation from newline-separated response text.
+- Shows exact-match summary and a qualitative eval table.
+- Logs tuned evaluation reports to `data/eval_results.jsonl`.
+
+Future behavior:
+
+- Start LoRA training.
+- Show loss and metrics.
+- Write Trackio traces.
+
+### `ui/vllm_tab.py`
+
+vLLM local serving planner.
+
+- Builds explicit `vllm serve` command plans.
+- Checks local vLLM `/health`.
+- Fetches and parses `/metrics`.
+- Logs vLLM benchmark metrics through local JSONL/Trackio fallback tracking.
+- Does not install vLLM, start a process, download models, or load weights on startup.
+
+### `ui/export_tab.py`
+
+GGUF export planning surface.
+
+- Selects a configured model and quantization.
+- Shows official GGUF download command plans when the model has GGUF metadata.
+- Shows local HF-to-GGUF conversion and llama.cpp quantization command plans.
+- Lists files already present under the selected export directory.
+- Exposes existing exported files through a Gradio download output.
+- Does not execute downloads, conversion, or quantization.
+
+Future behavior:
+
+- Execute downloads and conversions after explicit user action.
+
+### `ui/notes_tab.py`
+
+Field notes implementation.
+
+- Saves prompt, model response, correction, and tags to `data/field_notes.csv`.
+- Captures optional image path, video path, and a use-for-training flag.
+- Exports corrected notes to JSONL.
+- Exports local Hugging Face Dataset-style files under `data/hf_field_notes`.
+- Imports uncertain OCR predictions for human correction.
+- Exports corrected OCR rows to JSONL.
+- Emits field note saved events.
+
+Future behavior:
+
+- Push corrected notes to a remote Hugging Face Dataset after login.
+- Feed notes into fine-tuning.
+
+### `ui/traces_tab.py`
+
+Local trace and tracking preview.
+
+- Shows manual trace event previews.
+- Shows recent local app events.
+- Shows JSONL trace rows and tracking status.
+- Exports local traces to `exports/traces.jsonl`.
+- Calls Trackio only when the optional package is installed and enabled.
+
+### `ui/agent_tab.py`
+
+Local non-autonomous agent mode.
+
+- Drafts a research-plan-implement-verify trace.
+- Saves agent traces to `data/agent_traces.jsonl`.
+- Exports trace JSONL and local HF Dataset-style trace files.
+- Does not execute shell commands, commit, push, deploy, download models, or call external services.
+
+### `ui/status_tab.py`
+
+Shows configured models and backend metadata.
+
+- Helps verify model-size compliance and backend status.
+- Provides local llama.cpp settings, GGUF/mmproj file pickers, and command generation.
+- Provides LM Studio/OpenAI-compatible base URL, optional model-name storage, and reachability check.
+- Provides SGLang command planning, health check, and shutdown request controls.
+
+### `datasets/field_notes.py`
+
+Field note data model and CSV store.
+
+- `FieldNote` captures prompt, response, correction, tags, and timestamp.
+- `FieldNote` also captures optional image/video paths and a training inclusion flag.
+- `FieldNoteStore.save()` persists notes to `data/field_notes.csv`.
+- `FieldNoteStore.list_notes()` filters by correction, tag, and training inclusion.
+- `FieldNoteStore.export_jsonl()` writes training-ready JSONL.
+- `FieldNoteStore.export_hf_dataset()` writes local HF Dataset-style files.
+- `SQLiteFieldNoteStore` stores and lists notes in SQLite for larger correction loops.
+
+### `datasets/loader.py`
+
+Dataset preview and statistics helpers.
+
+- `preview_local_dataset()` previews CSV, JSONL, and NDJSON files.
+- `dataset_statistics()` reports row count, column count, names, and non-empty counts.
+- `preview_huggingface_dataset()` optionally uses the external Hugging Face `datasets` package.
+
+### `datasets/synthetic.py`
+
+Deterministic local synthetic data helpers.
+
+- `generate_synthetic_examples()` creates local prompt/response/correction examples.
+- `validate_synthetic_example()` checks schema requirements.
+- `quality_filter_examples()` removes incomplete or low-value examples.
+- `augment_examples()` creates deterministic variants for workflow testing.
+- `export_synthetic_jsonl()` writes JSONL without external services.
+
+### `datasets/ocr.py`
+
+Local OCR correction helpers.
+
+- `OCRPrediction` stores source path, predicted text, confidence, and optional page.
+- `load_ocr_predictions()` loads local `.csv`, `.jsonl`, and `.ndjson` prediction files.
+- `uncertain_predictions()` filters rows at or below a confidence threshold or with empty text.
+- `import_uncertain_predictions()` creates Field Notes correction tasks for uncertain rows.
+- `export_corrected_ocr_notes()` writes corrected OCR examples to JSONL for evaluation or training.
+- `ocr_import_summary()` previews uncertain rows for the Field Notes tab.
+
+### `mcp_tools/tools.py`
+
+Local MCP-style tools.
+
+- `dataset_stats_tool()` returns local dataset statistics.
+- `hf_dataset_preview_tool()` previews Hugging Face datasets when optional dependencies exist.
+- `safe_calculator_tool()` evaluates numeric arithmetic only.
+- `model_inference_tool()` routes text prompts through the selected model service.
+- `tool_registry()` returns the local tool map for a future MCP endpoint.
+
+### `mcp_tools/vindex_tool.py`
+
+Non-executing VINDEX integration boundary.
+
+- Defines the eight VINDEX PRD methods and their local FastAPI paths.
+- `build_vindex_call_plan()` validates method names and builds endpoint/payload plans.
+- Caps `star_spread.n_neighbors` at 5 and `calibrated_edit.causal_window` at 3 based on the PRD
+  safety notes.
+- `vindex_dependency_report()` checks whether the optional `vindex` package or local health
+  endpoint is available.
+- `vindex_verification_report()` combines dependency status with a safe call plan and keeps
+  execution disabled until the local VINDEX install is verified.
+
+### `mcp_tools/bridge.py`
+
+Gradio-native MCP bridge metadata and local invocation helper.
+
+- `MCP_PATH` documents `/gradio_api/mcp/sse`.
+- `mcp_manifest()` returns the selected mode, path, and tool definitions.
+- `invoke_mcp_tool()` verifies local tool invocation by name.
+
+### `agent/runner.py`
+
+Deterministic local agent trace runner.
+
+- `AGENT_SYSTEM_PROMPT` defines the agent behavior contract.
+- `run_agent_loop()` produces research, plan, implement, and verify trace steps.
+- `run_paper_to_code_loop()` produces paper-to-code research, plan, implement, and verify trace steps.
+- `default_safety_gates()` lists the non-autonomous safety requirements.
+- `save_agent_trace()` appends traces to JSONL.
+- `export_agent_traces()` exports trace JSONL.
+- `export_agent_traces_hf_dataset()` writes local HF Dataset-style trace files.
+- The runner can call safe local tools, but it is not autonomous.
+
+### `core/file_exports.py`
+
+Shared export helper.
+
+- `copy_text_file_or_empty()` copies a text artifact to an export path or creates an empty one.
+
+### `training/export.py`
+
+Non-executing GGUF export planning.
+
+- `detect_llama_cpp_tools()` checks `llama-server`, `llama-cli`, and `llama-quantize`.
+- `build_export_plan()` creates explicit download, conversion, and quantization command plans.
+- `list_exported_files()` lists generated/local export files.
+- `ExportPlan.as_dict()` marks that commands are not executed and no startup downloads occur.
+
+### `training/evaluation.py`
+
+Local deterministic evaluation helpers.
+
+- `default_prompt_cases()` returns a small built-in prompt test set.
+- `load_prompt_cases()` loads prompt/expected pairs from JSONL.
+- `evaluate_responses()` computes exact-match rows and a qualitative table.
+- `perplexity_from_losses()` computes perplexity from explicit negative log likelihood values.
+- `compare_base_vs_tuned()` reports exact-match delta.
+- `log_eval_report()` appends JSONL evaluation results.
+
+### `training/lora_trainer.py`
+
+Non-executing LoRA trainer request builder.
+
+- `lora_dependency_report()` reports PEFT, TRL, Transformers, and Torch availability.
+- `build_lora_training_request()` combines the training plan with dependency status and a command
+  preview.
+- `vision_finetuning_plan()` documents SWIFT/LLaMA-Factory as the future MiniCPM-V fine-tuning path.
+- Keeps `execute_training` false until dependencies, hardware, and dataset schema are approved.
+
+### `training/reward_eval.py`
+
+Deterministic local reward-style evaluation helpers.
+
+- `RewardEvaluator.evaluate()` scores supplied responses with transparent lexical heuristics.
+- `best_of_n()` selects the highest-scoring candidate without model calls.
+- `create_dpo_pairs()` creates chosen/rejected pairs for DPO-style datasets.
+- `eval_lora_vs_base()` compares base and LoRA response rewards.
+
+### `training/planner.py`
+
+Non-executing LoRA training planner.
+
+- `load_training_config()` reads LoRA and training settings from `config/training.yaml`.
+- `build_training_plan()` creates a dry-run plan with checkpoint output path.
+- `validate_training_plan()` checks dataset existence and numeric training settings.
+- `training_hardware_notes()` documents practical local hardware expectations.
+
+### `tracking/trackio_client.py`
+
+Tracking client with JSONL fallback.
+
+- `load_tracking_config()` reads Trackio settings from `config/training.yaml`.
+- `TrackingClient.init()` starts Trackio only when enabled and installed.
+- `TrackingClient.log()` always writes local JSONL and optionally forwards to Trackio.
+- `TrackingClient.finish()` closes optional Trackio state.
+- `export_traces()` copies local traces to `exports/traces.jsonl`.
+- `read_trace_rows()` returns recent local trace rows for the UI.
+
+### `core/events.py`
+
+Small event bus reserved for future cross-module events.
+
+- `EventType` names app events.
+- `UI_ERROR` records visible tab-level failures.
+- `Event` carries event data.
+- `EventBus` registers handlers and emits events.
+
+### `core/app_state.py`
+
+Shared local app state.
+
+- `AppState.emit()` records events, logs them, and dispatches them through `EventBus`.
+- `AppState.emit()` also writes trace events through `TrackingClient`.
+- `AppState.recent_events()` returns local trace previews for the Traces tab.
+- `emit_inference_response()` records shared response metadata.
+
+### `core/tab_feedback.py`
+
+Formats tab status text and emits `ui_error` events for visible tab-level failures.
+
+### `ui/progress.py`
+
+Defines the shared Gradio progress mode used by tab button callbacks.
+
+### `core/app_logging.py`
+
+Lightweight logging setup.
+
+- `configure_app_logging()` configures compact process logging once.
+
+### `core/registry.py`
+
+Generic registry helper.
+
+- `Registry.register(name, item)` stores a service.
+- `Registry.get(name)` retrieves a service.
+- `Registry.list()` lists registered services.
+
+## Current Design Rule
+
+The app must not download model weights on startup. Model loading should happen only after the
+user chooses a backend/model and clicks an explicit action.

docs/EXTENDING.md

@@ -0,0 +1,125 @@
+# How To Extend The Workbench
+
+## Add A New Model
+
+1. Open `config/models.yaml`.
+2. Add a new model entry.
+3. Set `type` to `text`, `vision`, or `omnimodal`.
+4. Keep `parameters_b` at or below 32 for hackathon eligibility.
+5. Keep `backend: placeholder` until a real service supports it.
+
+Example:
+
+```yaml
+models:
+  my_model:
+    hf_id: org/model-name
+    display_name: My Model
+    type: text
+    parameters_b: 7
+    backend: placeholder
+    context_length: 32768
+    local_first: true
+    notes: Why this model is useful.
+```
+
+## Add A Real Backend
+
+Create a service in `models/`, for example:
+
+```text
+models/ollama_service.py
+```
+
+The service should expose a small interface:
+
+```python
+class OllamaService:
+    def chat(self, system_prompt: str, user_prompt: str) -> str:
+        ...
+```
+
+Then update the service factory or relevant UI tab to choose between placeholder and real services.
+
+## Add A New Gradio Tab
+
+1. Create `ui/new_tab.py`.
+2. Add a `build_new_tab(...)` function.
+3. Import it in `app.py`.
+4. Add it inside the `gr.Tabs()` block.
+5. Update `docs/ARCHITECTURE.md`.
+6. Add a checklist item in `docs/TASKS.md`.
+7. Update `docs/IMPLEMENTATION_STATUS.md`.
+
+## Add Field Notes Export
+
+Next useful field notes upgrade:
+
+- Add a button to export `data/field_notes.csv` to JSONL.
+- Add a button to upload that JSONL as a Hugging Face Dataset.
+- Document the dataset schema in `README.md`.
+
+Suggested JSONL schema:
+
+```json
+{"model_id":"minicpm5_1b","prompt":"...","response":"...","correction":"...","tags":["demo"]}
+```
+
+## Add OCR Corrections
+
+The local OCR extension starts from prediction files rather than running an OCR engine directly.
+Use `.csv`, `.jsonl`, or `.ndjson` rows with fields like:
+
+```json
+{"source_path":"receipt.png","text":"Tota1 12.30","confidence":0.54}
+```
+
+The Field Notes tab can preview uncertain rows, import them as correction tasks, and export
+corrected OCR rows to `data/ocr_corrections.jsonl`. The intended wiring is:
+
+```text
+OCR predictions -> uncertain Field Notes -> corrected JSONL/HF Dataset -> training/evaluation
+```
+
+## Add VINDEX Execution
+
+The current VINDEX integration is a safety boundary, not an edit runner. It validates the eight PRD
+methods, builds non-executing local FastAPI call plans, and reports whether a local VINDEX package
+or `http://127.0.0.1:8765/health` server is available.
+
+Before allowing execution:
+
+1. Verify the local VINDEX package or FastAPI server.
+2. Re-check the PRD bug list: GPU cache cleanup, dead-code paths, star-spread over-editing, and
+   causal-window limits.
+3. Keep `star_spread.n_neighbors <= 5` and `calibrated_edit.causal_window <= 3` until the scaling
+   formula is validated.
+4. Add protected-relation tests for every edit workflow.
+5. Only then add an explicit user-triggered execute button or MCP tool.
+
+## Add Training
+
+Training should be added only after local inference works.
+
+Recommended order:
+
+1. Export field notes to JSONL.
+2. Load JSONL as a dataset.
+3. Add PEFT/TRL LoRA for text model.
+4. Add Trackio logging.
+5. Add checkpoint output folder.
+6. Add README instructions.
+
+## Add Hugging Face Space Deployment
+
+After the local app runs:
+
+```powershell
+.venv\Scripts\python.exe scripts\plan_hf_space.py --user <hf-user-or-org>
+huggingface-cli login
+huggingface-cli repo create openbmb-local-ai-workbench --type space --space-sdk gradio
+git remote add space https://huggingface.co/spaces/<user>/openbmb-local-ai-workbench
+git push space main
+```
+
+Never commit Hugging Face tokens.

docs/HACKATHON_SUBMISSION.md

@@ -0,0 +1,103 @@
+# Hackathon Submission Package
+
+This document collects the judge-facing story, demo flow, and submission assets for the Gradio
+Hugging Face Build Small Hackathon.
+
+## Track
+
+Recommended track: Backyard AI.
+
+Reason: the app is local-first, small-model focused, and designed to help a solo builder inspect,
+test, correct, and document OpenBMB model workflows on their own machine before moving to a Space.
+
+## Project Story
+
+OpenBMB Local AI Workbench is a Gradio app for trying small OpenBMB models locally, capturing
+human corrections as field notes, and turning those notes into training or evaluation artifacts.
+
+## Target User
+
+The target user is a hackathon builder or small-team AI tinkerer who wants a practical local
+workflow before committing to cloud deployment, GPU rentals, or a larger training stack.
+
+## Measurable Benefit
+
+The app reduces setup uncertainty by keeping model choices, backend availability, field-note
+exports, traces, and deployment next steps visible in one Gradio surface.
+
+## Final Model Family
+
+Primary family: OpenBMB MiniCPM.
+
+Current configured models:
+
+| Config ID | Model | Parameters | Role |
+| --- | --- | ---: | --- |
+| `minicpm5_1b` | `openbmb/MiniCPM5-1B` | 1B | local text baseline |
+| `minicpm5_1b_thinking` | `openbmb/MiniCPM5-1B-Thinking` | 1B | reasoning/text variant |
+| `minicpm41_8b` | `openbmb/MiniCPM4.1-8B` | 8B | long-context text candidate |
+| `minicpm_v46` | `openbmb/MiniCPM-V-4.6` | 1.3B | vision candidate |
+| `minicpm_v46_thinking` | `openbmb/MiniCPM-V-4.6-Thinking` | 1.3B | vision reasoning candidate |
+| `minicpm_o45` | `openbmb/MiniCPM-o-4.5` | 8B | omnimodal stretch candidate |
+
+All configured models are at or below the 32B hackathon limit.
+
+## Badge Targets
+
+- Local-first: yes, through verified llama.cpp CLI, llama-cpp-python GGUF, LM Studio/OpenAI-compatible, and OpenBMB MiniCPM-V Plant image paths; Ollama and SGLang remain setup paths until generation is verified.
+- llama.cpp: target badge path; requires local llama.cpp install and GGUF model verification.
+- Open trace: yes, through local JSONL tracking and trace export.
+- Field notes/report: yes, through corrected field notes, JSONL export, and local HF Dataset-style export.
+
+## Demo Flow
+
+1. Open the Gradio app locally.
+2. Show the Status tab and explain model-size compliance plus backend availability.
+3. Use Chat with the local `llama-cpp-python` GGUF backend to show a visible real response.
+4. Use the Dataset tab to preview a local JSONL/CSV training candidate.
+5. Save a correction in Field Notes and export corrected rows to JSONL.
+6. Open Traces to show local event history and optional Trackio status.
+7. Open Export to show GGUF conversion and quantization planning.
+8. Explain the remaining llama.cpp mmproj, Ollama, SGLang, and Space build verification tasks.
+9. Show the GitHub repo and, when available, the Hugging Face Space URL.
+
+## Screenshot Assets
+
+- Workbench home: `assets/e2e/workbench/01-workbench-home.png`
+- Workbench backend status: `assets/e2e/workbench/05-backend-status.png`
+- Plant tool home: `assets/e2e/plant/01-plant-home.png`
+- Plant corrections export: `assets/e2e/plant/03-corrections-export.png`
+
+## Demo Video Script
+
+1. "This is OpenBMB Local AI Workbench, a Gradio app for small-model local experimentation."
+2. "The model registry keeps every configured model below 32B parameters."
+3. "The app starts safely in placeholder mode, so it never downloads model weights on startup."
+4. "The Status tab shows which local backends are configured or missing."
+5. "A user can try a prompt, capture a correction, and export those corrections as training data."
+6. "The Traces tab records local workflow events for reproducibility."
+7. "The Export tab prepares explicit GGUF conversion and quantization commands."
+8. "The next deployment step is pushing this same Gradio app to a Hugging Face Space."
+
+## Social Post Draft
+
+Built OpenBMB Local AI Workbench for the Gradio Hugging Face Build Small Hackathon: a local-first
+Gradio app for testing MiniCPM models, collecting field-note corrections, exporting training data,
+planning GGUF/llama.cpp workflows, and keeping traceable evidence of small-model experiments.
+
+GitHub: https://github.com/Ckal/codex
+Workbench Space: https://huggingface.co/spaces/build-small-hackathon/workbench
+Plant Space: https://huggingface.co/spaces/build-small-hackathon/plant_identification_tool
+
+## Submission Checklist
+
+- GitHub URL: https://github.com/Ckal/codex
+- Workbench Space URL: https://huggingface.co/spaces/build-small-hackathon/workbench
+- Plant Identification Tool Space URL: https://huggingface.co/spaces/build-small-hackathon/plant_identification_tool
+- Space build verification: blocked until `hf auth login --force` is run with a fresh token
+- Demo video URL: pending
+- Social post URL: pending
+- Field notes/report URL: pending
+- Final track: Backyard AI
+- App name: OpenBMB Local AI Workbench
+- Deadline: June 15, 2026

docs/IMPLEMENTATION_STATUS.md

@@ -0,0 +1,186 @@
+# Implementation Status
+
+This file answers: "How do we know what is done?"
+
+An item is done only when:
+
+1. The code or document exists.
+2. It is linked from the relevant docs.
+3. It has been manually reviewed.
+4. If it is executable, it has been run locally or the blocker is documented.
+
+## Current Status
+
+| Area | Status | Evidence |
+| --- | --- | --- |
+| PRDs | Done | `HF_PRD_v1.md`, `HF_PRD_ext.md` exist at repo root |
+| Codex guidance | Done | `AGENTS.md` exists |
+| Docs folder | Done | `docs/` exists with task, usage, architecture, extension docs |
+| README | Done | `README.md` exists |
+| Hackathon submission package | Drafted | `docs/HACKATHON_SUBMISSION.md` contains track, story, user, benefit, model compliance, demo flow, video script, social post draft, and submission checklist |
+| Critical improvement roadmap | Done | `docs/ROADMAP_V2_CRITICAL_IMPROVEMENT_PLAN.md` gives a hard judge-oriented rating, architecture/security critique, and second roadmap |
+| Template how-to | Done | `docs/TEMPLATE_HOWTO.md` documents how to build new domain apps from the template |
+| Plant Discovery plan | Done | `docs/PLANT_DISCOVERY_APP_PLAN.md` tracks the first reference app and remaining work |
+| Plant model/training how-to | Done | `docs/PLANT_MODEL_AND_TRAINING_HOWTO.md` documents demo, OpenBMB zero-shot, fine-tuned adapter mode, correction export, training plan, and adapter configuration |
+| Gradio app shell | Implemented and launch-verified | `app.py` builds and foreground launch holds the server open; not currently running |
+| Model config | Implemented | `config/models.yaml` exists |
+| Expanded model config | Implemented | MiniCPM text, thinking, 4.1, V 4.6, V thinking, omnimodal entries, NVIDIA Nemotron Nano 9B v2, GGUF metadata, and backend capability metadata including OpenAI-compatible text serving |
+| Catalog validation | Implemented | `validate_catalog()` and Status tab warnings |
+| Training config | Implemented | `config/training.yaml` exists |
+| Placeholder chat | Implemented | `ui/chat_tab.py`, `models/placeholder_service.py` |
+| Placeholder vision | Implemented | `ui/vision_tab.py`, `models/placeholder_service.py` |
+| Service abstraction | Implemented | `models/base.py`, `models/service_factory.py` |
+| Local backend config | Implemented | `models/local_backend_config.py` saves ignored local settings in `data/local_backends.yaml` |
+| llama.cpp backend | Implemented and locally verified for text CLI | `models/llama_cpp_service.py`; Status tab can pick GGUF/mmproj paths and build a command using `C:\llama-b9587-bin-win-cuda-13.3-x64\llama-server.exe`; `llama-cli.exe` generated a real response from a local GGUF |
+| llama-cpp-python backend | Implemented and locally verified | `models/llama_cpp_python_service.py`; `llama_cpp 0.3.8` generated a real local response from `Llama-3.2-1B-Instruct-Q4_K_M.gguf`; Workbench Playwright captures this response |
+| Ollama backend | Implemented, not locally verified | `models/ollama_service.py`; Status tab lists local Ollama models and prepares explicit `ollama pull` commands; Ollama executable not found on PATH |
+| OpenAI-compatible backend | Implemented and live-verified | `models/openai_compatible_service.py`; Status tab stores LM Studio/vLLM-style base URL and optional served model name, checks `/v1/models`, and posts to `/v1/chat/completions` only when selected; verified `http://192.168.188.37:1234` with `llama-3.2-1b-instruct` |
+| Transformers text backend | Implemented, package installed | `models/transformers_text.py`; lazy-loads tokenizer/model only when selected; `transformers 5.10.2` is installed for MiniCPM-V support |
+| MiniCPM vision backend | Implemented and Plant-verified | `models/minicpm_vision.py` and `plant/plant_service.py` use `AutoProcessor` and `AutoModelForImageTextToText` lazily; `assets/plant_sample.jpg` produced a structured OpenBMB MiniCPM-V result |
+| SGLang backend | Implemented, optional local backend | `models/sglang_runner.py`; builds explicit local start commands, reports health, sends OpenAI-compatible chat requests, and provides a shutdown request; SGLang server launch remains unverified and is not a root Space dependency |
+| App state | Implemented | `core/app_state.py` records local events and dispatches through `EventBus`; `core/tab_feedback.py` emits tab-level UI errors |
+| Service registry | Implemented | `models/service_factory.py` registers text and vision backend factories |
+| Dataset tab | Partial | Local CSV/JSONL preview, optional HF dataset preview, schema, split selector, row count, samples, stats, dataset event emission, and tab-level error status |
+| Local dataset preview | Implemented | CSV, JSONL, NDJSON preview and statistics via `datasets/loader.py` |
+| MCP tools | Implemented locally, Gradio MCP path selected | `mcp_tools/tools.py` provides dataset stats, HF dataset preview/search-style helper, safe calculator, and model inference tool functions; `mcp_tools/bridge.py` documents `/gradio_api/mcp/sse` and verifies local invocation |
+| VINDEX boundary | Implemented locally, execution disabled | `mcp_tools/vindex_tool.py` validates the eight PRD methods, builds non-executing call plans, caps risky edit parameters, and reports local package/server availability |
+| Training tab | Partial | `ui/train_tab.py` builds a LoRA dry-run plan, checkpoint output path, hardware notes, local deterministic evaluation, and optional loss-based perplexity summary |
+| Training planner | Implemented, non-executing | `training/planner.py` parses LoRA/training config, validates dry runs, and never starts training |
+| LoRA trainer planner | Implemented locally, execution disabled | `training/lora_trainer.py` reports PEFT/TRL/Transformers/Torch availability, builds a non-executing LoRA request, and documents SWIFT/LLaMA-Factory vision fine-tuning path |
+| Evaluation | Implemented, local-only | `training/evaluation.py` provides prompt cases, exact-match scoring, optional loss-based perplexity, qualitative table, base-vs-tuned comparison, and JSONL logging |
+| Export tab | Partial | `ui/export_tab.py` builds GGUF download/conversion/quantization plans, lists exported files, and exposes existing exported files through a download output |
+| Export planner | Implemented, non-executing | `training/export.py` detects llama.cpp tools, builds explicit commands, and does not run downloads/conversions |
+| Reward evaluation | Implemented locally | `training/reward_eval.py` provides deterministic reward scoring, best-of-N selection, DPO pair generation, and LoRA-vs-base reward reports |
+| Synthetic data generation | Implemented locally | `datasets/synthetic.py` provides deterministic generation, validation, quality filtering, augmentation, and JSONL export |
+| Field notes | Partial | `ui/notes_tab.py` saves CSV, supports media paths/training flag, imports uncertain OCR predictions, emits field-note events, and exports JSONL/local HF Dataset files |
+| Field note module | Implemented | CSV save, SQLite store, corrected/tag/training filters, JSONL export, local HF Dataset export via `datasets/field_notes.py` |
+| OCR correction loop | Implemented locally | `datasets/ocr.py` loads local CSV/JSONL OCR predictions, filters by confidence threshold, imports uncertain rows to Field Notes, and exports corrected OCR JSONL |
+| Tracking | Implemented with local fallback | `tracking/trackio_client.py` loads Trackio config, writes local JSONL traces, and calls Trackio when installed/enabled |
+| Traces tab | Partial | `ui/traces_tab.py` previews app events, reads local trace rows, shows tracking status, and exports traces |
+| Agent mode | Implemented locally, non-autonomous | `agent/runner.py` provides system prompt, deterministic research-plan-implement-verify trace, paper-to-code trace mode, safety gates, tool registry integration, JSONL trace save/export, and local HF Dataset-style export |
+| Agent tab | Implemented locally | `ui/agent_tab.py` drafts task traces, paper-to-code traces, and exports trace files/datasets |
+| Status tab | Implemented | `ui/status_tab.py` lists model config, backend status, local llama.cpp setup, LM Studio/OpenAI-compatible setup, SGLang setup, and Ollama list/pull planning |
+| Tab-level error messages | Implemented | Chat, Vision, and Dataset tabs show status/error messages and emit `ui_error` trace events |
+| Loading/progress states | Implemented | `ui/progress.py` applies full Gradio progress indicators to tab actions |
+| Compact responsive layout | Implemented | `APP_CSS` constrains app width, keeps tabs scrollable, sizes touch targets, and adds mobile padding/type rules |
+| Structure verification | Done | `scripts/verify_structure.ps1` passed |
+| Unit tests | Passing | 187 unit/user-story tests pass |
+| User-story tests | Passing | Included in the 187-test suite |
+| Coverage | Passing | 68% line/branch coverage at current configured threshold |
+| Performance tests | Passing | 2 lightweight performance tests pass |
+| Playwright E2E | Passing with real response screenshots | Workbench Playwright captures a real local GGUF `llama-cpp-python` chat response; Plant Playwright with `RUN_REAL_MODEL_E2E=1` captures a real OpenBMB MiniCPM-V image result from `assets/plant_sample.jpg` |
+| CI pipeline | Added, not run remotely | `.github/workflows/ci.yml` |
+| Quality tooling | Passing | Tests, coverage, performance, ruff, mypy, pylint, bandit, and pip-audit pass through `scripts/run_quality.ps1` |
+| Secrets and model-weight git policy | Implemented | `.gitignore` excludes env files, keys, caches, generated data/exports, and common model weight formats; policy has a unit test |
+| Real model inference | Partial but materially verified | Verified paths: LM Studio/OpenAI-compatible text, llama.cpp CLI text, llama-cpp-python GGUF text, and OpenBMB MiniCPM-V Plant image inference. Remaining unverified paths: Ollama generation, SGLang server generation, llama.cpp MiniCPM-V mmproj vision, full Transformers text generation |
+| Hugging Face Space deploy | Pushed, startup/build pending | Workbench pushed to `build-small-hackathon/workbench` at `6aafdc2083a9b82e9dca2cca5b87c3a1be05121b`; Plant pushed to `build-small-hackathon/plant_identification_tool` at `50897b3167a844b6a66ca0552b73a1791cdff926`; both include Python 3.10 compatibility fixes for HF Spaces |
+| HF Space deployment helper | Implemented locally | `deployment/hf_space.py` and `scripts/plan_hf_space.py` validate required files, README Space metadata, Workbench/Plant remote status, and manual `hf` deployment commands |
+| vLLM serving tab | Implemented locally, not locally verified | `models/vllm_runner.py` and `ui/vllm_tab.py` build explicit vLLM commands, check health, parse metrics, log benchmark metrics through local tracking, and use OpenAI-compatible chat when a server is running |
+| Plant Discovery reference app | Implemented locally, no-model verified; Space wrapper added | `plant/` is a standalone template-built app with demo/no-model service for local tests, default OpenBMB MiniCPM-V mode, optional fine-tuned adapter mode, local species index, correction export, non-executing training plan, optional MCP tools, unit tests, HTTP smoke verification on port 7861, and `plant_space_app.py` for real-model Space launch |
+| GitHub push | Done | GitHub remote `https://github.com/Ckal/codex.git`; commits pushed to `origin/main` |
+
+## Known Blockers
+
+- `python` is available on PATH as Python 3.13 in the current shell. The documented `.venv` was not
+  visible during the latest verification run, so tests ran against the global Python environment
+  after reinstalling/updating `requirements.txt`.
+- App launch was verified by Playwright through `python app.py`; no long-running server is currently active.
+- llama.cpp tools are installed at `C:\llama-b9587-bin-win-cuda-13.3-x64`; they are not on PATH,
+  so the app stores/uses the explicit `llama-server.exe` path.
+- Export planning works, but actual conversion/quantization remains blocked until a specific export
+  run is requested and verified.
+- A GGUF path is configured locally for `Llama-3.2-1B-Instruct-Q4_K_M.gguf`; direct
+  `llama-cli.exe` and `llama-cpp-python` generation both produced real text.
+- Ollama is installed on PATH (`ollama version is 0.24.0`). `ollama pull openbmb/minicpm-v4.6`
+  succeeded and `ollama list` shows `openbmb/minicpm-v4.6:latest`; a tiny `ollama run` prompt
+  currently fails with a 500 model-load error, so real Ollama generation remains unverified.
+- LM Studio/OpenAI-compatible setup is implemented and verified for `http://192.168.188.37:1234`
+  with served model override `llama-3.2-1b-instruct`. This local override is stored in ignored
+  `data/local_backends.yaml`.
+- `transformers 5.10.2`, `torch`, and MiniCPM-V dependencies are installed. This conflicts with
+  the currently installed `sentence-transformers 3.4.1` requirement of `transformers<5.0.0`, so
+  future quality runs should watch for that dependency edge.
+- MiniCPM-V Plant inference is verified with `assets/plant_sample.jpg`; the app also supports
+  bounded E2E inference through `PLANT_MAX_NEW_TOKENS=320` and `PLANT_AUTO_THINKING=0`.
+- SGLang command planning, health, stop, and chat client code is implemented. SGLang remains an
+  optional local backend, not a root Space dependency, because the local Python 3.13 package index
+  cannot install the newer non-vulnerable SGLang server stack.
+- vLLM command planning, health, metrics, benchmark logging, and chat client code is implemented,
+  but the `vllm` package/server is not installed or running in this workspace.
+- LoRA training request planning is implemented, but real execution remains blocked until PEFT/TRL,
+  Transformers, Torch, and final hardware are approved and installed.
+- Trackio is optional; local JSONL tracing works without the `trackio` package, but remote Trackio/HF
+  sync still needs package availability and credentials.
+- LoRA dry-run planning works locally, but real training remains blocked until a final backend,
+  PEFT/TRL or SWIFT/LLaMA-Factory path, and hardware are chosen.
+- Hugging Face dataset preview is optional and requires the external `datasets` package; the app
+  reports a clear status when it is not installed.
+- Hugging Face Space deployment is pushed with Python 3.10 compatibility fixes for `StrEnum` and
+  UTC timestamps. Final run verification and smoke workflows remain open until rebuilt Spaces
+  report `RUNNING`.
+- Node.js and npm are installed (`node v23.11.0`, `npm 10.9.2`). `npm install`,
+  `npm run e2e:install`, and `npm run e2e` pass.
+- Plant Discovery OpenBMB MiniCPM-V inference is implemented as the default real mode, and
+  `plant_space_app.py` launches that mode for Space deployment. `RUN_REAL_MODEL_E2E=1`
+  Playwright passed and captured a real response screenshot.
+- Plant Discovery fine-tuned adapter mode is implemented, but no trained plant adapter exists in
+  this workspace yet.
+- Plant Discovery public Space mode still needs path/url hardening and screenshots.
+- Full PRD implementation is not complete. There are still unchecked tasks in `docs/TASKS.md`.
+- Current unchecked task count needs recounting after the latest Workbench/Space changes. Several PRD/ext PRD items still
+  need real local setup, credentials, hardware, product decisions, or hackathon submission artifacts.
+
+## Latest Local Verification
+
+- `.\scripts\run_tests.ps1` passed: 200 tests and 70% coverage.
+- `.\scripts\run_quality.ps1` passed: tests, app smoke, 70% coverage, performance, Ruff, mypy,
+  Pylint, Bandit, and project-scoped pip-audit.
+- Plant Discovery no-model HTTP smoke passed on `http://127.0.0.1:7861`; the process was stopped
+  after verification.
+- `python scripts\plan_plant_training.py --corrected-examples 30` prints a non-executing SWIFT /
+  LLaMA-Factory adapter training plan; current environment is missing torch, transformers, PEFT,
+  TRL, and SWIFT.
+- `pytest tests/unit/test_local_backend_config.py tests/unit/test_llama_cpp_service.py tests/unit/test_model_catalog.py tests/unit/test_service_factory.py -q` passed: 30 tests.
+- `npm run e2e:workbench` passed and captured a visible local GGUF response through `llama-cpp-python`.
+- `RUN_REAL_MODEL_E2E=1 PLANT_MAX_NEW_TOKENS=320 PLANT_AUTO_THINKING=0 npx playwright test tests/e2e/plant_real_model.spec.ts --config playwright.plant.config.ts --reporter=list` passed and captured a real MiniCPM-V response.
+- `hf upload build-small-hackathon/workbench C:\tmp\workbench_space_payload . --repo-type space` pushed commit `6aafdc2083a9b82e9dca2cca5b87c3a1be05121b`.
+- `hf upload build-small-hackathon/plant_identification_tool C:\tmp\plant_space_payload . --repo-type space` pushed commit `50897b3167a844b6a66ca0552b73a1791cdff926`.
+- `hf spaces variables add` set `WORKBENCH_DEPLOYMENT=space` for Workbench and Plant; Plant also has `PLANT_MAX_NEW_TOKENS=320` and `PLANT_AUTO_THINKING=0`.
+- `hf spaces info` showed Workbench in `APP_STARTING` and Plant in `BUILDING` on `zero-a10g`; final build/run verification remains pending.
+- `pytest tests/unit tests/user_stories -q` passed earlier in this workstream: 197 tests.
+- `node --version` returned `v23.11.0`; `npm --version` returned `10.9.2`.
+- `ollama --version` returned `ollama version is 0.24.0`; `ollama list` returned `gemma4:latest`.
+- `ollama pull openbmb/minicpm5-1b` failed because the registry manifest does not exist.
+- `ollama pull openbmb/minicpm-v4.6` succeeded; `ollama run openbmb/minicpm-v4.6` failed with
+  a local 500 model-load error.
+- `hf --version` returned `1.17.0`; `huggingface-cli --version` reports the legacy command is
+  deprecated and recommends `hf`.
+- `hf auth whoami` failed with "Invalid user token"; Space push/build verification is blocked
+  until `hf auth login --force` is run with a fresh token.
+- `llama-server --version` failed because `llama-server` is not on PATH.
+- `git remote -v` shows `space-workbench` and `space-plant` remotes configured.
+- Direct `ruff check .` passed; cache-write warnings were caused by OneDrive permissions.
+- Direct `mypy . --no-incremental` passed when `MYPY_CACHE_DIR` was moved to `%TEMP%`.
+- LM Studio `/v1/models` at `http://192.168.188.37:1234` returned
+  `text-embedding-nomic-embed-text-v1.5`, `qwen2.5-coder-3b-instruct`, and
+  `llama-3.2-1b-instruct`.
+- LM Studio `/v1/chat/completions` returned a text response from `llama-3.2-1b-instruct`.
+- Focused OCR callback and pipeline tests pass: 8 tests.
+- Focused VINDEX/MCP tests pass: 13 tests.
+
+## Verification Commands
+
+Run these after installing Python:
+
+```powershell
+.\scripts\verify_structure.ps1
+.venv\Scripts\python.exe --version
+.venv\Scripts\Activate.ps1
+python -m pip install -r requirements.txt
+python -m pip install -r requirements-dev.txt
+.\scripts\run_tests.ps1
+.\scripts\run_quality.ps1
+python app.py
+```
+
+When the app starts, update this file and `docs/TASKS.md`.

docs/PLANT_DISCOVERY_APP_PLAN.md

@@ -0,0 +1,119 @@
+# Plant Discovery Reference App Plan
+
+This checklist turns the rough `plant/` sample into the first concrete app built around the
+template. It is intentionally detailed so future apps can copy the pattern.
+
+## Product Story
+
+Plant Discovery helps a gardener, teacher, or field-note collector:
+
+1. Upload one or more plant images.
+2. Get a structured plant identification.
+3. Correct mistakes.
+4. Browse a small field guide.
+5. Export corrections as local training data.
+6. Later fine-tune or evaluate a small vision model.
+
+This is stronger than a generic workbench for hackathon judging because it has a real user outcome.
+
+## Current Implementation Status
+
+- [x] Add `plant/` package.
+- [x] Add standalone `plant/app.py`.
+- [x] Add clean `plant/models.yaml`.
+- [x] Add deterministic no-model demo service.
+- [x] Add optional MiniCPM-V service adapter.
+- [x] Add explicit `demo`, `openbmb`, and `finetuned` model modes.
+- [x] Make OpenBMB MiniCPM-V the default real model mode.
+- [x] Add fine-tuned adapter loading path through PEFT when configured.
+- [x] Avoid hard `torch`/`transformers` imports at module import time.
+- [x] Add plant schema/parser through `PlantID`.
+- [x] Add local species index builder.
+- [x] Add local folder loader.
+- [x] Add field-note export to plant training JSONL.
+- [x] Add focused Gradio UI with Identify, Field Guide, Corrections, and Stats.
+- [x] Replace direct training execution with non-executing training plan.
+- [x] Add optional pure tool functions and lazy MCP server builder.
+- [x] Add non-executing plant training planner.
+- [x] Add Plant model/training how-to.
+- [x] Add plant unit tests.
+- [x] Add no-model app build verification.
+
+## Remaining Work
+
+### P0 - Make The Demo Judgeable
+
+- [x] Run `python -m plant.app --no-model --port 7861`.
+- [ ] Install Node.js/npm.
+- [ ] Generate Playwright screenshots for the plant app.
+- [ ] Add Plant Discovery screenshots to README.
+- [ ] Add a 60-90 second demo script focused on plant correction.
+- [x] Decide whether the hackathon Space should launch `app.py` or `plant/app.py`.
+- [x] If using `plant/app.py` for Space, add a Space-specific entrypoint or README note.
+
+### P1 - Real Model Path
+
+- [ ] Install optional plant dependencies from `plant/requirements.txt`.
+- [ ] Verify `PlantVisionService.dependency_report()` is fully available.
+- [ ] Run one MiniCPM-V identification on a local public-safe plant image.
+- [ ] Capture model ID, hardware, latency, and output JSON.
+- [ ] Add a real-backend integration test profile that is skipped unless dependencies are present.
+- [ ] Document that model weights are not downloaded on startup.
+
+### P2 - Data And Correction Loop
+
+- [ ] Add a tiny public-safe sample plant image folder under ignored local data.
+- [ ] Add a sample `plant/data/plantnet_labels.json` cache or document how to create it.
+- [ ] Add path allowlist before public Space deployment.
+- [ ] Add JSONL schema documentation for exported corrections.
+- [ ] Add validation for minimum correction count before recommending training.
+
+### P3 - Template Extraction
+
+- [ ] Identify reusable parts that should move from `plant/` to generic modules.
+- [ ] Keep domain-specific prompt/schema/UI inside `plant/`.
+- [ ] Consider `domains/<name>/` convention if more than one reference app is added.
+- [ ] Add a cookiecutter-style checklist or scaffold script only after the second app proves the pattern.
+
+### P4 - Security And Public Mode
+
+- [ ] Add public/local mode config.
+- [ ] In public mode, disable arbitrary local file paths.
+- [ ] In public mode, disable arbitrary backend URLs.
+- [ ] Add max upload size and allowed image extension checks.
+- [ ] Add tests for path traversal and malformed image/data inputs.
+- [ ] Add a plant-specific disclaimer: identification can be wrong; do not use for medical, edible,
+  or toxicity-critical decisions without expert verification.
+
+### P5 - Training And Evaluation
+
+- [ ] Collect at least 30 corrected examples before any LoRA experiment.
+- [ ] Split exported corrections into train/eval.
+- [ ] Add exact species-name evaluation.
+- [ ] Add before/after model comparison.
+- [ ] Run SWIFT or LLaMA-Factory locally only after dependencies and hardware are approved.
+- [ ] Publish adapter only if it improves the evaluation set.
+- [ ] Update `plant/models.yaml` with the real adapter repo.
+- [ ] Verify `python -m plant.app --model-mode finetuned --port 7861`.
+
+## Recommended Next Command Sequence
+
+```powershell
+.venv\Scripts\python.exe -m pytest tests/unit/test_plant_reference_app.py -q
+.venv\Scripts\ruff.exe check plant tests/unit/test_plant_reference_app.py --no-cache
+.venv\Scripts\python.exe -m mypy plant tests/unit/test_plant_reference_app.py --cache-dir "$env:TEMP\openbmb-workbench-mypy-cache"
+.venv\Scripts\python.exe -m plant.app --no-model --port 7861
+.venv\Scripts\python.exe scripts\plan_plant_training.py --corrected-examples 30
+```
+
+For the deployed Plant Identification Tool Space, use `plant_space_app.py`. It launches
+OpenBMB MiniCPM-V mode and does not enable `--no-model`/demo mode.
+
+## Template Lessons From Plant Discovery
+
+- Domain app first screen should be the product, not the infrastructure.
+- Demo/no-model mode is essential for tests and screenshots.
+- Heavy model dependencies should be optional.
+- Correction loops are more valuable than promising immediate fine-tuning.
+- Tool/MCP functions should be pure and importable without a running server.
+- Training should be a plan until data, hardware, and dependencies are verified.

docs/PLANT_MODEL_AND_TRAINING_HOWTO.md

@@ -0,0 +1,145 @@
+# Plant Model And Training How-To
+
+This answers three common questions:
+
+1. Does Plant Discovery use an LLM?
+2. Does it use a specially trained plant model?
+3. How do we train and then use one?
+
+## Current Model Modes
+
+Plant Discovery has three explicit modes.
+
+| Mode | Command | Uses an LLM/VLM? | Trained specifically for this app? |
+| --- | --- | --- | --- |
+| Demo | `python -m plant.app --model-mode demo --port 7861` | No | No |
+| OpenBMB zero-shot | `python -m plant.app --model-mode openbmb --port 7861` | Yes, MiniCPM-V | No, uses base OpenBMB model |
+| Fine-tuned adapter | `python -m plant.app --model-mode finetuned --port 7861` | Yes, MiniCPM-V plus adapter | Yes, after you train/publish an adapter |
+
+The app defaults to `openbmb` mode. The `--no-model` flag is an alias for demo mode and exists for
+tests, screenshots, and machines without GPU/model dependencies.
+
+## What Runs Today
+
+- Demo mode is fully verified locally.
+- OpenBMB mode is implemented but requires optional dependencies and model weights.
+- Fine-tuned mode is implemented as an adapter-loading path, but there is no real adapter yet.
+
+This means the app can use a real OpenBMB VLM, but this workspace has not installed the heavy
+`plant/requirements.txt` dependencies or downloaded model weights.
+
+## Step 1 - Use OpenBMB MiniCPM-V Zero-Shot
+
+Install optional plant dependencies:
+
+```powershell
+.venv\Scripts\python.exe -m pip install -r plant\requirements.txt
+```
+
+Run the app with the real OpenBMB model path:
+
+```powershell
+.venv\Scripts\python.exe -m plant.app --model-mode openbmb --port 7861
+```
+
+Open `http://127.0.0.1:7861`, upload a public-safe plant image, and click `Identify plant`.
+
+Expected behavior:
+
+- The UI shows model status with `uses_llm: true`.
+- The first identify action loads `openbmb/MiniCPM-V-4.6`.
+- If confidence is low and auto-thinking is enabled, the app can try
+  `openbmb/MiniCPM-V-4.6-Thinking`.
+- Output is parsed into the `PlantID` JSON schema.
+
+## Step 2 - Collect Corrections
+
+Use the app normally:
+
+1. Upload plant image.
+2. Identify.
+3. Add a human correction when the model is wrong.
+4. Save correction.
+5. Open `Corrections`.
+6. Click `Export training JSONL`.
+
+The export path is:
+
+```text
+data/plant_training.jsonl
+```
+
+Do not train on one or two examples. The current minimum recommendation is 30 corrected examples,
+and more is better.
+
+## Step 3 - Plan Training
+
+Generate a non-executing training plan:
+
+```powershell
+.venv\Scripts\python.exe scripts\plan_plant_training.py --corrected-examples 30
+```
+
+This prints:
+
+- dependency availability,
+- SWIFT command preview,
+- LLaMA-Factory command preview,
+- publish commands,
+- command to run the trained adapter.
+
+The Gradio UI also exposes the same non-executing plan from the `Corrections` tab.
+
+## Step 4 - Train Locally
+
+Preferred path for MiniCPM-V vision LoRA is SWIFT or another tool that supports multimodal LoRA.
+The generated SWIFT command looks like this:
+
+```powershell
+swift sft --model openbmb/MiniCPM-V-4.6 --dataset data/plant_training.jsonl --lora_rank 16 --num_train_epochs 3 --per_device_train_batch_size 4 --gradient_accumulation_steps 4 --learning_rate 2.0e-4 --freeze_vit true --output_dir checkpoints/plant_lora
+```
+
+Run it only after:
+
+- optional dependencies are installed,
+- GPU memory is sufficient,
+- exported JSONL has been reviewed,
+- private images/notes are removed,
+- you accept that training may take time and disk space.
+
+## Step 5 - Publish Or Configure The Adapter
+
+After training, publish the adapter or keep it local. If publishing:
+
+```powershell
+huggingface-cli repo create your-username/minicpm-v46-plant-lora --type model
+huggingface-cli upload your-username/minicpm-v46-plant-lora checkpoints/plant_lora .
+```
+
+Then edit `plant/models.yaml`:
+
+```yaml
+plant_vlm_finetuned:
+  base_model: openbmb/MiniCPM-V-4.6
+  adapter_id: your-username/minicpm-v46-plant-lora
+```
+
+Run:
+
+```powershell
+.venv\Scripts\python.exe -m plant.app --model-mode finetuned --port 7861
+```
+
+## Important Honesty Rule
+
+Do not claim Plant Discovery uses a specially trained model until:
+
+- a real adapter exists,
+- it is configured in `plant/models.yaml`,
+- `--model-mode finetuned` has been run,
+- evaluation shows it improves over the base OpenBMB model.
+
+Until then, the honest claim is:
+
+> Plant Discovery uses OpenBMB MiniCPM-V zero-shot, with a correction loop and documented path to
+> fine-tune a plant adapter.

docs/PRD_IMPLEMENTATION_MATRIX.md

@@ -0,0 +1,95 @@
+# PRD Implementation Matrix
+
+This file maps the main PRD and extension PRD to current implementation status.
+
+## Summary
+
+The full PRD and extension PRD are not fully implemented yet.
+
+Current state:
+
+- Foundation, docs, test policy, quality gates, CI, placeholder Gradio surfaces, and a Plant
+  Discovery reference app exist.
+- Shared app state, service registries, local event logging, tab-level error events, and local
+  trace preview exist.
+- Local llama.cpp settings, GGUF/mmproj pickers, and command generation exist without startup downloads.
+- GGUF export planning exists with tool detection and explicit non-executing command plans.
+- Local JSONL tracing and optional Trackio wrapper exist.
+- Dataset statistics and local MCP tool functions exist.
+- OCR correction loop exists locally for CSV/JSONL prediction imports into Field Notes.
+- VINDEX integration boundary exists locally as non-executing MCP-style planning tools.
+- Local non-autonomous agent mode exists with trace export.
+- Real local model inference is partially implemented through llama.cpp, llama-cpp-python, Ollama,
+  OpenAI-compatible/LM Studio, SGLang, Transformers text, and MiniCPM vision services. Verified
+  local paths now include llama.cpp CLI text, llama-cpp-python GGUF text, LM Studio text, and
+  OpenBMB MiniCPM-V Plant image inference. The Status
+  tab includes llama.cpp setup, LM Studio/OpenAI-compatible setup, SGLang command/check/stop setup,
+  and Ollama local model listing plus explicit pull-command planning. LM Studio text generation is
+  live-verified with `llama-3.2-1b-instruct`; the other real backends still need local verification.
+- `WORKBENCH_DEPLOYMENT=space` now hides placeholder backend choices and refuses placeholder/demo
+  service creation for deployed app paths.
+- LoRA training execution, served MCP endpoint, deployment, and most extensions are not implemented.
+- Placeholder services remain intentionally visible so the app never pretends to be real inference.
+
+## Main PRD
+
+| PRD Area | Status | Evidence / Next Step |
+| --- | --- | --- |
+| Purpose and design philosophy | Documented | `README.md`, `docs/ROADMAP.md` |
+| Template architecture | Partial | Config-driven model catalog exists; `docs/TEMPLATE_HOWTO.md` and `plant/` show the first domain-app pattern |
+| System architecture | Partial | `app.py`, `core/`, `models/`, `ui/`, `datasets/`, local app state/events |
+| Model registry | Partial | `config/models.yaml`, `models/model_catalog.py`; includes GGUF and backend capability metadata |
+| Five inference modes | Partial | llama.cpp, llama-cpp-python, OpenAI-compatible/LM Studio, and MiniCPM-V Plant image inference are locally verified; Ollama generation, SGLang server generation, vLLM server generation, and llama.cpp mmproj vision remain unverified |
+| Trackio | Partial | Local traces, optional Trackio wrapper, and HF Space sync docs exist; credentials/package setup still missing |
+| MCP layer | Partial | Local tool functions, Gradio-native MCP path metadata, `mcp_server=True` launch flag, and local invocation tests exist; full external client verification still missing |
+| Training pipeline | Partial | `training/` package supports dry-run planning, non-executing LoRA request planning, export planning, exact-match/perplexity evaluation, and local logging; real PEFT/TRL execution missing |
+| Export and quantization | Partial | `training/export.py` and Export tab plan downloads/conversion/quantization and expose existing exported files for download; execution still missing |
+| Agent mode | Partial | Local deterministic task and paper-to-code trace loops exist with safety gates; autonomous execution and remote uploads missing |
+| UI tabs | Partial | Tabs exist; Chat/Vision/Dataset/Field Notes/Status have behavior; Status includes SGLang setup; tab actions have Gradio progress indicators; Chat/Vision/Dataset have tab-level status/error messages; compact responsive CSS exists; several tabs are still placeholders |
+| Field notes | Partial | CSV save, SQLite store, corrected/tag/training filters, media paths, OCR uncertain import, JSONL export, and local HF Dataset export exist; remote HF upload missing |
+| Directory structure | Partial | Foundation exists; many PRD packages missing |
+| Configuration schema | Partial | Model/training config plus ignored local backend config exists; validation is lightweight |
+| Dependencies | Partial | Runtime/dev deps exist for scaffold; full model/training deps not added |
+| Hackathon demo flow | Partial | `docs/HACKATHON_SUBMISSION.md` drafts story, user, demo flow, script, social post, and URLs; real backend and Space URL still missing |
+| Corrections from PRD v1 | Documented in PRD | Not all implemented |
+| Roadmap and extension points | Documented | `docs/ROADMAP.md`, `docs/TASKS.md` |
+
+## Extension PRD
+
+| Extension | Status | Evidence / Next Step |
+| --- | --- | --- |
+| vLLM serving tab | Implemented locally, not locally verified | `models/vllm_runner.py` and vLLM tab provide command planning, health checks, metrics parsing, benchmark logging, and OpenAI-compatible chat client; needs installed/running vLLM for real serving |
+| Ollama quick-start | Partial | Service, UI backend selector, local model listing, explicit pull-command planning, and setup docs exist; local Ollama install/real model verification missing |
+| Reward model eval | Implemented locally | `training/reward_eval.py` provides deterministic reward scoring, best-of-N, DPO pairs, and LoRA-vs-base reward reports |
+| Synthetic data generation | Implemented locally | `datasets/synthetic.py` provides deterministic generation, validation, filtering, augmentation, and JSONL export |
+| Paper-to-code agent | Implemented locally | Agent tab and `agent/runner.py` support paper input, research/plan/implementation/verify trace, and safety gates without autonomous execution |
+| HF Spaces deploy | Partial | README metadata, deployment helper, command plan, required-file validation, Workbench/Plant target URLs, and remote/build status checks exist; HF auth/remote/push/build verification still missing |
+| VINDEX integration | Implemented locally, execution disabled | `mcp_tools/vindex_tool.py` validates VINDEX methods, builds safe call plans, reports dependency/server status, and documents that actual edits require a verified local VINDEX install |
+| OCR pipeline hook | Implemented locally | `datasets/ocr.py` and Field Notes tab support local OCR prediction loading, confidence thresholds, uncertain import, human correction, and corrected JSONL export |
+| MiniCPM Desk-Pet | Not implemented | Needs persona schema/export |
+| MiniCPM-o audio tab | Not implemented | Needs audio tab and omnimodal backend |
+| Cross-extension wiring | Partial | OCR -> Field Notes -> Training, Synthetic Gen -> Reward Eval -> DPO, Agent -> Desk-Pet Persona, and HF Spaces -> Trackio are documented; remaining wiring depends on unimplemented runtime modules |
+
+## Quality Coverage
+
+Current verified gates:
+
+- Structure check passes.
+- 187 unit/user-story tests pass.
+- Coverage report passes at 68%, above the current 60% configured threshold.
+- 2 lightweight performance tests pass.
+- Ruff passes.
+- Mypy passes.
+- Pylint passes at 10/10.
+- Bandit reports no issues.
+- Pip-audit reports no known vulnerabilities in `.venv`.
+- LM Studio `/v1/models` and `/v1/chat/completions` are verified locally for
+  `llama-3.2-1b-instruct`.
+- Workbench and Plant Playwright screenshot flows pass through `npm run e2e`.
+- CI workflow exists but has not run remotely.
+- App launch has been verified locally through Playwright, but the server is not currently left running.
+
+## No Pretend-Done Rule
+
+Any row marked `Partial`, `Placeholder`, or `Not implemented` must not be described as complete.
+When a row is implemented, update this file, `docs/TASKS.md`, and `docs/IMPLEMENTATION_STATUS.md`.

docs/README.md

@@ -0,0 +1,32 @@
+# Project Docs
+
+This folder is the project memory. Use it before asking Codex to make larger changes.
+
+## Start Here
+
+- [TASKS.md](TASKS.md) - checklist of hackathon tasks and subtasks.
+- [IMPLEMENTATION_STATUS.md](IMPLEMENTATION_STATUS.md) - what is done, partial, or not started.
+- [PRD_IMPLEMENTATION_MATRIX.md](PRD_IMPLEMENTATION_MATRIX.md) - exact PRD/ext implementation coverage.
+- [ACCEPTANCE_CRITERIA.md](ACCEPTANCE_CRITERIA.md) - proof required before marking work done.
+- [ROADMAP.md](ROADMAP.md) - hackathon path versus long-term PRD vision.
+- [ROADMAP_V2_CRITICAL_IMPROVEMENT_PLAN.md](ROADMAP_V2_CRITICAL_IMPROVEMENT_PLAN.md) - hard
+  judge-oriented critique and second roadmap.
+- [TEMPLATE_HOWTO.md](TEMPLATE_HOWTO.md) - how to build new domain apps from this template.
+- [PLANT_DISCOVERY_APP_PLAN.md](PLANT_DISCOVERY_APP_PLAN.md) - detailed checklist for the first
+  reference app.
+- [PLANT_MODEL_AND_TRAINING_HOWTO.md](PLANT_MODEL_AND_TRAINING_HOWTO.md) - model modes and
+  adapter training path for Plant Discovery.
+- [PLANT_MODEL_AND_TRAINING_HOWTO.md](PLANT_MODEL_AND_TRAINING_HOWTO.md) - how Plant Discovery
+  uses OpenBMB MiniCPM-V and how to train an adapter.
+- [USAGE.md](USAGE.md) - how to install, run, and use the app.
+- [ARCHITECTURE.md](ARCHITECTURE.md) - how the code is organized and how pieces connect.
+- [EXTENDING.md](EXTENDING.md) - how to add models, tabs, services, and features later.
+- [HACKATHON_SUBMISSION.md](HACKATHON_SUBMISSION.md) - story, demo flow, video script, and submission checklist.
+
+## Source PRDs
+
+- [HF_PRD_v1.md](../HF_PRD_v1.md) - main technical PRD.
+- [HF_PRD_ext.md](../HF_PRD_ext.md) - planned extensions.
+
+The PRDs describe the full vision. The task and status docs describe what is actually implemented
+or planned for the hackathon MVP.

docs/ROADMAP.md

@@ -0,0 +1,49 @@
+# Roadmap
+
+The PRD is intentionally broad. This roadmap keeps the hackathon path separate from the larger
+Workbench vision.
+
+## Must Ship For Hackathon
+
+1. Local app runs.
+2. One real small model backend works.
+3. User story is clear.
+4. Field notes or demo evidence shows practical value.
+5. Hugging Face Space is live.
+6. README is clear enough for judges.
+7. Demo video and social post are ready.
+
+## Should Ship If Time Allows
+
+1. Ollama quick-start.
+2. llama.cpp GGUF mode.
+3. Field notes JSONL export.
+4. Trackio trace logging.
+5. Better visual polish.
+6. Architecture/report write-up.
+
+## Stretch After Hackathon
+
+1. LoRA training.
+2. GGUF export and quantization.
+3. MCP tools.
+4. Agent mode.
+5. Reward model evaluation.
+6. Synthetic data generation.
+7. OCR pipeline hook.
+8. MiniCPM-o audio tab.
+9. Desk-Pet persona export.
+
+## Current Recommended Path
+
+For a new Codex user, the best path is:
+
+```text
+docs/TASKS.md
+  -> install Python
+  -> run placeholder app
+  -> add Ollama backend
+  -> push GitHub
+  -> push Hugging Face Space
+  -> polish submission
+```

docs/ROADMAP_V2_CRITICAL_IMPROVEMENT_PLAN.md

@@ -0,0 +1,361 @@
+# Roadmap V2 - Critical Hackathon Improvement Plan
+
+This document is intentionally hard on the project. It answers: "If a strong automated judge or
+human judge reviews this repo, what will cost points, and what should be fixed first?"
+
+Sources checked on 2026-06-08:
+
+- Official Build Small Hackathon page: <https://huggingface.co/build-small-hackathon>
+- Current repo status: `docs/IMPLEMENTATION_STATUS.md`
+- PRD coverage: `docs/PRD_IMPLEMENTATION_MATRIX.md`
+- Task checklist: `docs/TASKS.md`
+
+## Executive Verdict
+
+This is not yet a SOTA hackathon entry. It is a serious engineering scaffold with good local
+quality discipline, but it is not yet a sharp, shipped product.
+
+Current estimated rating:
+
+| Dimension | Score | Hard reason |
+| --- | ---: | --- |
+| Hackathon fit | 5/10 | It uses Gradio and small-model framing, but the user story is diluted by too many tabs. |
+| Shipped product | 3/10 | No verified Hugging Face Space URL, no generated screenshots, no submitted demo media. |
+| Real usefulness | 5/10 | Field Notes plus correction loop can be useful, but it needs one real user, one real workflow, and evidence. |
+| Technical architecture | 6/10 | Modules are clean enough, but UI callbacks still orchestrate too much and global state is too central. |
+| Security posture | 5/10 | Basic secret/model-weight policy exists, but URL/path inputs and future command execution need hardening. |
+| Test discipline | 7/10 | Good Python quality gates and unit tests; missing executed browser E2E, visual checks, and real backend integration. |
+| Design and UX | 4/10 | Functional Gradio layout, but not memorable, not guided, and not custom enough for an "Off-Brand" award. |
+| SOTA small-model angle | 4/10 | Many backend planners exist, but there is no benchmarked small-model capability story yet. |
+
+Overall: **5/10 as a repo, 3.5/10 as a winning submission today.**
+
+The strongest path is to stop trying to show everything and make one narrow workflow excellent:
+
+> A local-first Field Notes Correction Workbench for collecting messy human observations, correcting
+> small-model/OCR outputs, exporting training data, and showing how small models improve with human
+> feedback.
+
+That can plausibly compete in Backyard AI, Field Notes, Sharing is Caring, Llama Champion, Tiny Titan,
+OpenBMB, and Best Demo if executed cleanly.
+
+## What A GPT-Style Judge Will Likely Measure
+
+The official page does not publish a numeric rubric, but it does state the visible judging signals.
+Assume an automated GPT judge will inspect the Space, README, demo, screenshots, repository, and
+submission text for the following:
+
+1. **Track alignment**
+   - Backyard AI: specific real problem, real person used it, honest small-model fit, Gradio polish.
+   - Thousand Token Wood: delight, AI as the core experience, originality, Gradio polish.
+
+2. **Hard constraints**
+   - Gradio app.
+   - Hosted as a Hugging Face Space.
+   - Models at or below 32B parameters.
+   - Short demo video and social/social-style post.
+
+3. **Bonus quest evidence**
+   - Off the Grid: no cloud APIs.
+   - Well-Tuned: published fine-tuned model.
+   - Off-Brand: custom UI beyond default Gradio.
+   - Llama Champion: llama.cpp runtime.
+   - Sharing is Caring: public trace on the Hub.
+   - Field Notes: report or blog post about build and lessons.
+
+4. **Repo truthfulness**
+   - README claims match implementation.
+   - No hidden placeholders presented as working model features.
+   - Tests and status docs do not contradict the demo.
+
+5. **User outcome**
+   - A judge can answer: who is this for, what did they do before, what do they do now, and what got
+     measurably better?
+
+6. **Demo reliability**
+   - App loads quickly.
+   - One golden path works without local-only secrets.
+   - Screenshots and demo video show actual outputs, not empty controls.
+
+7. **Novelty and "small model" honesty**
+   - The app should show why a small model is enough or better.
+   - A tiny/small model should do something concrete, not merely be listed in config.
+
+## Current Critical Weaknesses
+
+### Product Weaknesses
+
+- The app reads like a toolbox, not a product. A judge will not spend long discovering value across
+  Chat, Vision, Dataset, Train, vLLM, Export, Field Notes, Traces, Agent, and Status.
+- The project story is not yet visceral. "OpenBMB Local AI Workbench" sounds useful to builders,
+  but the hackathon asks for either a real person or a delightful experience.
+- The strongest workflow is Field Notes/OCR/correction, but it is not yet the first screen or the
+  main README story.
+- No demo screenshots have been generated yet because Node/npm is missing.
+- No public Space URL means the current entry cannot satisfy the central shipping requirement.
+
+### Architecture Weaknesses
+
+- UI callbacks currently act as controllers. They validate inputs, call services, emit events, and
+  format output. This is acceptable for an MVP, but it will become brittle.
+- `APP_STATE` is a global singleton. That is simple locally, but it is risky for multi-user Spaces,
+  test isolation, and future per-session traces.
+- Service creation is centralized in `models/service_factory.py`, but there is no application-layer
+  facade for workflows such as "correct note -> export dataset -> evaluate -> log trace".
+- There are many non-executing planners. That is honest, but it creates a product risk: the app can
+  look more complete than it is.
+- There is no domain model for a complete demo run. The app records events, but not a first-class
+  `DemoSession`, `CorrectionSession`, or `EvaluationRun`.
+
+### Security Weaknesses
+
+- User-controlled URLs are accepted for local backends. This can become SSRF if deployed on a Space
+  and allowed to reach internal services.
+- User-controlled local paths are accepted for datasets, GGUF files, OCR files, exports, and media.
+  This is useful locally, but dangerous on a public Space without path allowlists.
+- Future command execution is planned. Command builders are currently non-executing, which is good,
+  but the moment execution is added, strict argument arrays, allowlists, and confirmation gates are
+  mandatory.
+- MCP exposure is attractive for judges, but it can widen the attack surface. Tool inputs need
+  explicit schemas, rate limits, and audit logs before public use.
+- Agent mode is non-autonomous today. If autonomy is added, shell/git/deploy/model-download actions
+  need hard policy gates.
+
+### Testing Weaknesses
+
+- Playwright E2E exists but is not run locally because Node/npm is missing.
+- No browser screenshots have been generated yet for docs.
+- No visual regression baseline.
+- No real backend integration test for LM Studio, llama.cpp, Ollama, SGLang, vLLM, or MiniCPM Vision.
+- Coverage is enough for the current threshold, but important UI code remains lightly covered.
+- Security tests are thin: path allowlist, URL allowlist, SSRF prevention, file-size limits, and
+  malformed JSONL/CSV cases should be tested.
+
+### Design Weaknesses
+
+- The UI is still mostly default Gradio. That likely misses the Off-Brand award.
+- The first screen does not guide a judge into a single demo path.
+- There is no "Judge Mode" that preloads sample data, explains what to click, and proves the core
+  value in under two minutes.
+- The README does not yet lead with screenshots, output examples, and the exact demo path.
+- Status/setup surfaces are useful, but they should not dominate the judged experience.
+
+## Architecture Recommendation
+
+Do not add heavy enterprise architecture. Add a thin application layer.
+
+### Current Shape
+
+```text
+ui tab callback -> model/dataset/training module -> event logging
+```
+
+This keeps files small, but the UI owns too much orchestration.
+
+### Recommended Shape
+
+```text
+ui tab callback
+  -> controller
+    -> workflow facade
+      -> domain service / backend adapter
+      -> event/tracking/security policy
+```
+
+Add these modules:
+
+- `core/context.py`
+  - `AppContext`
+  - owns catalog, event bus, tracking client, local config, and security policy.
+  - replaces direct use of the global `APP_STATE` in new code.
+
+- `controllers/chat_controller.py`
+  - validates chat input.
+  - selects backend through an inference facade.
+  - emits trace events.
+  - returns UI-safe results.
+
+- `controllers/field_notes_controller.py`
+  - owns save/import/export correction workflows.
+  - produces documentation-ready demo summaries.
+
+- `controllers/dataset_controller.py`
+  - owns local/HF dataset preview with path policy checks.
+
+- `controllers/serving_controller.py`
+  - owns llama.cpp, SGLang, vLLM, Ollama command/status planning.
+
+- `workflows/correction_loop.py`
+  - `CorrectionLoopFacade`
+  - one golden path: import sample -> correct -> export JSONL -> evaluate -> trace.
+
+- `security/policies.py`
+  - URL allowlist for local backends.
+  - path allowlist for public Space mode.
+  - file size/type limits.
+  - execution policy for future command-running features.
+
+Keep the existing model services as adapters. Do not replace everything at once.
+
+### Singleton Decision
+
+The current `APP_STATE` singleton is tolerable for a local prototype. It is not ideal for a public
+Space. Move toward:
+
+- `AppContext` created in `build_app()`.
+- Per-session UI state through Gradio `gr.State` where user-specific data matters.
+- Global event log only for public-safe aggregate events.
+- Tests that create isolated contexts.
+
+## Roadmap V2
+
+### P0 - Make It Shippable And Judgeable
+
+- [ ] Choose one final judged track: Backyard AI is recommended.
+- [ ] Rename the public product around the user outcome, not the infrastructure.
+- [ ] Make Field Notes Correction the first and primary demo path.
+- [ ] Add a "Judge Mode" tab or first-screen panel with sample data and exact steps.
+- [ ] Generate Playwright screenshots into `assets/e2e/`.
+- [ ] Add screenshots to README and `docs/HACKATHON_SUBMISSION.md`.
+- [ ] Deploy a working Hugging Face Space.
+- [ ] Add the Space URL to README and submission docs.
+- [ ] Record a 60-90 second demo video.
+- [ ] Publish or draft the social post with Space, GitHub, and demo video links.
+
+Definition of done: a judge can open the Space and understand the product in under 30 seconds.
+
+### P1 - Prove One Real Small-Model Backend
+
+- [ ] Pick the final backend for the demo: LM Studio today, llama.cpp if badge is achievable.
+- [ ] Run one verified local generation path end-to-end.
+- [ ] Capture model ID, parameter count, runtime, hardware, and latency.
+- [ ] Add a "Model Evidence" section to README.
+- [ ] Add a tiny benchmark: response latency, tokens/sec if available, and sample output quality.
+- [ ] If using OpenBMB, show exactly where the OpenBMB model is used.
+- [ ] If claiming Llama Champion, install llama.cpp, select GGUF, and verify real text generation.
+
+Definition of done: the app has one undeniable real-model path, not just planners.
+
+### P2 - Make The Field Notes Workflow Real
+
+- [ ] Create a small public-safe example dataset.
+- [ ] Add sample OCR predictions or sample model mistakes.
+- [ ] Build "import -> correct -> export -> evaluate" as one guided workflow.
+- [ ] Add before/after examples to README.
+- [ ] Export corrected JSONL and trace data.
+- [ ] Publish public-safe traces or dataset to Hugging Face.
+- [ ] Write the Field Notes report/blog.
+
+Definition of done: the app demonstrates measurable improvement from human corrections.
+
+### P3 - Simplify The UI For Judges
+
+- [ ] Add a first-screen guided demo panel.
+- [ ] Move advanced backend tabs behind an "Advanced setup" accordion or tab group.
+- [ ] Add concise status badges: local-first, model size, backend, Space-ready, trace-ready.
+- [ ] Add visual hierarchy and screenshots.
+- [ ] Use a small custom CSS theme that fits the product story.
+- [ ] Keep default Gradio controls where useful, but avoid an unstyled toolbox feel.
+
+Definition of done: the app feels intentional, not like a pile of tabs.
+
+### P4 - Add Controller And Facade Layer
+
+- [ ] Add `core/context.py` with injectable `AppContext`.
+- [ ] Add `security/policies.py` with path and URL checks.
+- [ ] Add `controllers/field_notes_controller.py`.
+- [ ] Add `controllers/dataset_controller.py`.
+- [ ] Add `controllers/chat_controller.py`.
+- [ ] Add `controllers/serving_controller.py`.
+- [ ] Add `workflows/correction_loop.py`.
+- [ ] Refactor one tab at a time to call controllers instead of raw service modules.
+- [ ] Keep old behavior covered by tests during each refactor.
+
+Definition of done: UI callbacks become thin and domain workflows are testable without Gradio.
+
+### P5 - Security Hardening
+
+- [ ] Add public/local mode setting.
+- [ ] In public Space mode, restrict backend URLs to an allowlist or disable arbitrary URL checks.
+- [ ] Restrict file paths to repo-local safe directories.
+- [ ] Add max file size and supported extension checks.
+- [ ] Add tests for path traversal attempts.
+- [ ] Add tests for SSRF-style URLs.
+- [ ] Add tests for malformed CSV/JSONL/OCR inputs.
+- [ ] Add clear privacy labels for traces and field notes.
+- [ ] Ensure demo data contains no secrets, private prompts, or hidden reasoning.
+
+Definition of done: the public Space cannot be used to probe internal URLs or arbitrary files.
+
+### P6 - Real Integration And E2E Tests
+
+- [ ] Install Node.js/npm.
+- [ ] Run `npm install`, `npm run e2e:install`, and `npm run e2e`.
+- [ ] Commit generated screenshot assets that are safe for docs.
+- [ ] Add Playwright to CI if Node is available.
+- [ ] Add one live-backend integration profile for LM Studio/OpenAI-compatible.
+- [ ] Add optional integration profiles for llama.cpp, Ollama, SGLang, and vLLM.
+- [ ] Add visual smoke checks for mobile and desktop.
+
+Definition of done: the demo path is proven in a browser and documented with images.
+
+### P7 - SOTA Stretch
+
+- [ ] Fine-tune or publish a tiny adapter if time allows.
+- [ ] Publish a public-safe correction dataset or trace dataset.
+- [ ] Add a simple reward/evaluation leaderboard for base vs corrected/fine-tuned outputs.
+- [ ] Add a model card or dataset card with limitations.
+- [ ] Show why the small model is good enough: privacy, latency, cost, offline use, or local control.
+- [ ] If using llama.cpp, publish exact GGUF/runtime instructions.
+
+Definition of done: the project has a defensible small-model contribution, not just a UI.
+
+## What To Cut
+
+Cut or hide these from the judged first impression unless they support the demo:
+
+- vLLM tab.
+- SGLang controls.
+- Training execution promises.
+- VINDEX boundary.
+- Desk-Pet backlog.
+- MiniCPM-o audio backlog.
+- Generic agent mode, unless it directly produces the public trace badge.
+
+Keep them in docs as extension points, but do not make the judge click through them first.
+
+## Better Tool Vision
+
+A better tool is not "all PRD features implemented." A better tool is:
+
+1. **Specific**
+   - Helps one person correct and turn messy field observations into useful training/eval data.
+
+2. **Local-first**
+   - Runs with a small local model and explains why that matters.
+
+3. **Evidence-driven**
+   - Shows before/after outputs, corrected examples, traces, and screenshots.
+
+4. **Safe**
+   - Does not leak private notes, probe arbitrary URLs, or execute commands unexpectedly.
+
+5. **Small but complete**
+   - One polished path beats ten partial tabs.
+
+## Brutal Final Advice
+
+If the deadline is close, do not chase full PRD completion. The PRD is too broad for a winning
+hackathon submission. Win by making one real, visual, end-to-end story undeniable.
+
+Priority order:
+
+1. Space URL.
+2. One real model path.
+3. Field Notes golden workflow.
+4. Screenshots and demo video.
+5. README story with evidence.
+6. Security hardening for public mode.
+7. Controller/facade cleanup.
+
+Everything else is secondary.

docs/TASKS.md

@@ -0,0 +1,530 @@
+# Full Task Checklist
+
+This is the shared task list for you and Codex. It covers the hackathon MVP, the main PRD, and
+the extension PRD. A task is complete only when the matching acceptance criteria are met and
+`docs/IMPLEMENTATION_STATUS.md` is updated.
+
+## Legend
+
+- `[x]` done and documented
+- `[~]` partially implemented or placeholder exists
+- `[ ]` not started
+- `[blocked]` blocked by missing local setup, credentials, hardware, or external decision
+
+## Phase 0 - Project Memory And Setup
+
+- [x] Add root `README.md`.
+- [x] Add root `AGENTS.md`.
+- [x] Add `.gitignore`.
+- [x] Add `requirements.txt`.
+- [x] Add `docs/` folder.
+- [x] Add docs index.
+- [x] Add full task checklist.
+- [x] Add implementation status doc.
+- [x] Add usage guide.
+- [x] Add architecture guide.
+- [x] Add extension guide.
+- [x] Add acceptance criteria.
+- [x] Add roadmap.
+- [x] Add critical judge-oriented improvement roadmap.
+- [x] Add template how-to for building new domain apps.
+- [x] Add Plant Discovery reference app checklist.
+- [x] Add Plant Discovery model and training how-to.
+- [x] Add PRD implementation matrix.
+- [x] Add test folder.
+- [x] Add user-story test folder.
+- [x] Add dev requirements.
+- [x] Add Python quality config.
+- [x] Add test runner script.
+- [x] Add quality runner script.
+- [x] Add CI workflow.
+- [x] Add coverage gate.
+- [x] Add performance test script.
+- [x] Install Python 3.11+.
+- [x] Verify `python --version`.
+- [x] Create `.venv`.
+- [x] Install dependencies.
+- [x] Run `python app.py`.
+- [x] Capture screenshot or note local URL.
+
+## Phase 1 - Hackathon Definition
+
+- [x] Choose track: Backyard AI or Thousand Token Wood.
+- [x] Write one-sentence project story.
+- [x] Define target user.
+- [x] Define measurable user benefit.
+- [x] Decide final model family and model IDs.
+- [x] Confirm every model is <= 32B parameters.
+- [x] Decide local-first badge target.
+- [x] Decide llama.cpp badge target.
+- [x] Decide open trace badge target.
+- [x] Decide field notes/report badge target.
+- [x] Write final demo flow.
+- [x] Write demo video script.
+- [x] Write social post draft.
+- [ ] Add final submission checklist with exact URLs.
+
+## Phase 2 - MVP Gradio App
+
+- [x] Add `app.py`.
+- [x] Add Gradio `Blocks` shell.
+- [x] Add model config loader.
+- [x] Add model metadata display.
+- [x] Add Chat tab.
+- [x] Add Vision tab.
+- [x] Add Dataset tab placeholder.
+- [x] Add Train tab placeholder.
+- [x] Add Export tab placeholder.
+- [x] Add Field Notes tab.
+- [x] Add placeholder text service.
+- [x] Add placeholder vision service.
+- [x] Add Traces tab placeholder.
+- [x] Add Agent tab placeholder.
+- [x] Add Status tab placeholder.
+- [x] Add PowerShell structure verification script.
+- [x] Run structure verification script.
+- [x] Run app locally.
+- [x] Fix local launch errors found so far.
+- [x] Add screenshot capture path to docs or README.
+- [x] Add first demo GIF/video plan.
+
+## Phase 3 - Config-Driven Model Registry
+
+- [x] Add `config/models.yaml`.
+- [x] Add text model entry for MiniCPM5-1B.
+- [x] Add vision model entry for MiniCPM-V-4.6.
+- [x] Add omnimodal model entry for MiniCPM-o-4.5.
+- [x] Add typed `ModelInfo`.
+- [x] Add `load_model_catalog()`.
+- [x] Add `model_choices()`.
+- [x] Add `model_summary()`.
+- [x] Add MiniCPM5-1B-Thinking config.
+- [x] Add MiniCPM4.1-8B config.
+- [x] Add MiniCPM-V-4.6-Thinking config.
+- [x] Add GGUF metadata in config.
+- [x] Add backend capability metadata.
+- [x] Add lightweight catalog validation helper.
+- [x] Show warnings for models over 32B parameters.
+
+## Phase 4 - Core Architecture
+
+- [x] Add `core/events.py`.
+- [x] Add `EventType`.
+- [x] Add `Event`.
+- [x] Add `EventBus`.
+- [x] Add `core/registry.py`.
+- [x] Add generic `Registry`.
+- [x] Add global app state.
+- [x] Register model services in a service registry.
+- [x] Emit inference events from UI.
+- [x] Emit field note events.
+- [x] Add lightweight logging.
+- [x] Add unit tests for config and registry.
+
+## Phase 5 - Testing And Quality
+
+- [x] Add `tests/unit/`.
+- [x] Add `tests/user_stories/`.
+- [x] Add model catalog unit tests.
+- [x] Add field notes unit tests.
+- [x] Add new-user user-story test.
+- [x] Add `requirements-dev.txt`.
+- [x] Add `pyproject.toml`.
+- [x] Add `scripts/run_tests.ps1`.
+- [x] Add `scripts/run_quality.ps1`.
+- [x] Run unit and user-story tests.
+- [x] Install dev quality tools.
+- [x] Run `ruff`.
+- [x] Run `mypy`.
+- [x] Run `pylint`.
+- [x] Run `bandit`.
+- [x] Run `pip-audit`.
+- [x] Add rule: failing bug/check requires a new or updated test.
+- [x] Add coverage report.
+- [x] Add lightweight performance tests.
+- [x] Add CI pipeline.
+- [x] Add Playwright or equivalent browser e2e test after Gradio runs.
+- [ ] Add tests for each real backend as it is implemented.
+- [x] Add tests for backend service selection.
+- [x] Add tests for Ollama unavailable path.
+- [x] Add tests for llama.cpp unavailable path and command building.
+- [x] Add tests for llama-cpp-python unavailable path.
+- [x] Add tests for OpenAI-compatible/LM Studio unavailable and request paths.
+
+## Phase 6 - Local Inference Backends
+
+- [x] Choose first real backend.
+- [x] Add backend selector in UI.
+- [x] Add model status panel.
+- [x] Add explicit model load button.
+- [x] Ensure no model weights download on startup.
+
+### Ollama Backend
+
+- [x] Confirm Ollama is installed.
+- [x] Add `models/ollama_service.py`.
+- [x] Add local model list.
+- [x] Add pull model command with explicit user action.
+- [x] Add text chat through Ollama.
+- [x] Add vision chat through Ollama when supported.
+- [x] Document Ollama setup.
+
+### llama.cpp Backend
+
+- [x] Confirm llama.cpp tools are installed.
+- [x] Add `models/llama_cpp_service.py`.
+- [x] Add `models/llama_cpp_python_service.py`.
+- [x] Add GGUF file picker.
+- [x] Add `llama-server` launch command builder.
+- [x] Add health check.
+- [x] Add text generation through server.
+- [x] Add vision `mmproj` support metadata.
+- [x] Document llama.cpp setup.
+
+### llama-cpp-python Backend
+
+- [x] Add optional Python binding service.
+- [x] Add backend selector support.
+- [x] Install `llama-cpp-python` locally.
+- [x] Configure local GGUF path.
+- [x] Verify real text generation through Python binding.
+- [x] Decide whether to keep Python binding as fallback or primary local path.
+
+### Transformers Backend
+
+- [x] Add `models/transformers_text.py`.
+- [x] Add `AutoModelForCausalLM` loading for text models.
+- [x] Add tokenizer loading.
+- [x] Add explicit trust-remote-code handling.
+- [x] Add device/dtype settings.
+- [x] Add streaming generation.
+- [x] Document hardware expectations.
+
+### OpenAI-Compatible / LM Studio Backend
+
+- [x] Add `models/openai_compatible_service.py`.
+- [x] Add backend selector support.
+- [x] Add local base URL and served-model-name config.
+- [x] Add Status tab setup and reachability check.
+- [x] Add text chat through OpenAI-compatible `/v1/chat/completions`.
+- [x] Document LM Studio setup.
+- [x] Verify real text generation through LM Studio.
+
+### MiniCPM Vision Backend
+
+- [x] Add `models/minicpm_vision.py`.
+- [x] Use `AutoModelForImageTextToText`.
+- [x] Use `AutoProcessor`.
+- [x] Add image prompt formatting.
+- [x] Add thinking-mode toggle mapping.
+- [x] Add video support plan.
+
+### SGLang Backend
+
+- [x] Add `models/sglang_runner.py`.
+- [x] Add server start/stop.
+- [x] Add MiniCPM5 tool parser config.
+- [x] Add health check.
+- [x] Add chat endpoint client.
+- [x] Install `sglang` locally.
+
+## Phase 7 - UI Tabs From Main PRD
+
+- [x] Chat tab placeholder.
+- [x] Vision tab placeholder.
+- [x] Dataset tab placeholder.
+- [x] Train tab placeholder.
+- [x] Export tab placeholder.
+- [x] Field Notes tab minimal save.
+- [x] Add Traces tab with local event preview.
+- [x] Add Agent tab placeholder.
+- [x] Add model/backend status tab or panel.
+- [x] Add settings panel.
+- [x] Add tab-level error messages.
+- [x] Add loading/progress states.
+- [x] Add compact responsive layout review.
+
+## Phase 8 - Dataset Layer
+
+- [x] Add `datasets/` package.
+- [x] Add local CSV loader.
+- [x] Add local JSONL loader.
+- [x] Add Hugging Face dataset loader.
+- [x] Add dataset schema preview.
+- [x] Add split selector.
+- [x] Add row count and sample preview.
+- [x] Add dataset statistics tool.
+- [x] Emit `DATASET_LOADED` event.
+- [x] Document dataset formats.
+
+## Phase 9 - Field Notes And Correction Loop
+
+- [x] Save field notes to CSV.
+- [x] Move field note logic out of UI into `datasets/field_notes.py`.
+- [x] Add `FieldNote` dataclass.
+- [x] Add SQLite-backed store.
+- [x] Add JSONL export.
+- [x] Add local HF Dataset export.
+- [x] Add corrected-only filter.
+- [x] Add tags filter.
+- [x] Add image path support.
+- [x] Add video path support.
+- [x] Add use-for-training flag.
+- [x] Add docs for correction loop.
+
+## Phase 10 - Training Pipeline
+
+- [x] Add training config placeholder.
+- [x] Add training UI placeholder.
+- [x] Add `training/` package.
+- [x] Add LoRA text trainer.
+- [x] Add LoRA config parser.
+- [ ] Add PEFT/TRL dependencies when ready.
+- [x] Add training dry-run validation.
+- [x] Add local checkpoint output.
+- [x] Add Trackio integration.
+- [x] Add evaluation after training.
+- [x] Add LoRA vs base comparison.
+- [x] Add vision fine-tuning plan using SWIFT or LLaMA-Factory.
+- [x] Document training hardware requirements.
+
+## Phase 11 - Evaluation
+
+- [x] Add `training/evaluation.py`.
+- [x] Add simple prompt test set.
+- [x] Add exact-match metric.
+- [x] Add qualitative eval table.
+- [x] Add perplexity metric where appropriate.
+- [x] Add base vs tuned comparison.
+- [x] Log eval results.
+- [x] Document evaluation method.
+
+## Phase 12 - Export And Quantization
+
+- [x] Add export UI placeholder.
+- [x] Add `training/export.py`.
+- [x] Add official GGUF download path.
+- [x] Add local HF-to-GGUF conversion path.
+- [x] Add quantization selector.
+- [x] Add llama.cpp tool detection.
+- [x] Add exported file listing.
+- [x] Add download link in UI.
+- [x] Document GGUF export.
+
+## Phase 13 - Trackio Tracing
+
+- [x] Add `tracking/` package.
+- [x] Add Trackio config.
+- [x] Add `trackio.init()`.
+- [x] Add `trackio.log()`.
+- [x] Add `trackio.finish()`.
+- [x] Log inference events locally.
+- [x] Log dataset events locally.
+- [x] Log training metrics.
+- [x] Add Traces tab.
+- [x] Add HF Space sync docs.
+
+## Phase 13 - MCP Layer
+
+- [x] Decide MCP path: Gradio native, `gradio.Server` 
+- [x] Add MCP tools module.
+- [x] Add dataset stats tool.
+- [x] Add HF search tool.
+- [x] Add safe calculator tool.
+- [x] Add model inference tool.
+- [x] Expose tools through selected MCP path.
+- [x] Document MCP endpoint.
+- [x] Verify endpoint locally.
+
+## Phase 14 - Agent Mode
+
+- [x] Add `agent/` package.
+- [x] Add agent system prompt.
+- [x] Add research-plan-implement loop placeholder.
+- [x] Add tool registry integration.
+- [x] Add session trace logging.
+- [x] Add Agent tab.
+- [x] Add trace export to JSONL.
+- [x] Add local HF Dataset export for traces.
+- [x] Document limitations.
+
+## Phase 15 - Hugging Face Space Deployment
+
+- [x] Install/verify `huggingface_hub`.
+- [x] Login with `hf auth login`.
+- [ ] Create Space.
+- [x] Add Space README metadata if needed.
+- [x] Add Space remote.
+- [x] Push to Space.
+- [ ] Verify Space builds.
+- [ ] Add Space URL to README.
+- [x] Document hardware choice.
+- [x] Document model download behavior.
+
+## Phase 16 - GitHub
+
+- [x] Create GitHub repo.
+- [x] Add GitHub remote.
+- [x] Commit initial project.
+- [x] Push to GitHub.
+- [x] Add GitHub URL to README.
+- [ ] Add issue checklist or project board if desired.
+
+## Phase 17 - Hackathon Submission Package
+
+- [x] Finalize app name.
+- [x] Finalize track.
+- [x] Verify Gradio app polish.
+- [x] Verify model-size compliance.
+- [ ] Verify Space URLs.
+- [x] Verify GitHub URL.
+- [ ] Record demo video.
+- [ ] Publish social post.
+- [ ] Add field notes/report link.
+- [ ] Submit before June 15, 2026.
+
+## Extension PRD Backlog
+
+### vLLM Serving Tab
+
+- [x] Add vLLM runner.
+- [x] Add vLLM start/stop UI.
+- [x] Add OpenAI-compatible client.
+- [x] Add metrics parsing.
+- [x] Add Trackio benchmark logging.
+
+### Ollama Quick-Start
+
+- [x] Add Ollama pull/list UI.
+- [x] Add Ollama chat service.
+- [x] Add Ollama vision service.
+- [x] Add setup docs.
+
+### Llama.cpp Champion Path
+
+- [x] Add llama.cpp backend selection.
+- [x] Add llama.cpp service.
+- [x] Add llama-cpp-python service.
+- [x] Add llama.cpp status check.
+- [x] Install llama.cpp locally.
+- [x] Download/pick GGUF model.
+- [x] Verify real text generation.
+- [ ] Verify MiniCPM-V mmproj flow.
+
+### Reward Model Eval
+
+- [x] Add reward evaluator.
+- [x] Add best-of-N generation.
+- [x] Add DPO pair generation.
+- [x] Add LoRA vs base reward report.
+
+### Synthetic Data Generation
+
+- [x] Add synthetic generator.
+- [x] Add JSON validation.
+- [x] Add quality filters.
+- [x] Add augmentation flow.
+- [x] Add dataset save/export.
+
+### Paper-To-Code Agent
+
+- [x] Add paper input UI.
+- [x] Add research phase.
+- [x] Add plan phase.
+- [x] Add implementation trace.
+- [x] Add safety gates.
+
+### HF Spaces Deploy Tool
+
+- [x] Add deployment helper script.
+- [x] Add Space creation docs.
+- [x] Add remote validation.
+- [x] Add build status checks.
+
+### VINDEX Integration
+
+- [x] Define integration boundary.
+- [x] Add tool stub.
+- [x] Add verification report.
+- [x] Document dependency.
+
+### OCR Pipeline Hook
+
+- [x] Add OCR loader.
+- [x] Add confidence threshold.
+- [x] Add uncertain prediction import.
+- [x] Add correction UI.
+- [x] Add corrected export.
+
+### MiniCPM Desk-Pet
+
+- [ ] Add persona data schema.
+- [ ] Add persona training plan.
+- [ ] Add Desk-Pet export plan.
+- [ ] Add docs.
+
+### MiniCPM-o Audio Tab
+
+- [ ] Add audio tab.
+- [ ] Add microphone input.
+- [ ] Add omnimodal service.
+- [ ] Add TTS plan.
+- [ ] Add streaming plan.
+
+### Cross-Extension Wiring
+
+- [x] Document OCR -> Field Notes -> Training.
+- [x] Document Synthetic Gen -> Reward Eval -> DPO.
+- [x] Document Agent -> Desk-Pet Persona.
+- [x] Document HF Spaces -> Trackio.
+
+## Phase 18 - Template And Reference Apps
+
+### Template How-To
+
+- [x] Document branch strategy for new domain apps.
+- [x] Document required domain app file contract.
+- [x] Document schema, service, loader, UI, tools, tests, and docs pattern.
+- [x] Document no-model/demo-mode requirement.
+- [x] Document correction-loop-first workflow.
+- [x] Document optional training and real-model verification steps.
+- [x] Document security requirements for public Space mode.
+
+### Plant Discovery Reference App
+
+- [x] Add `plant/` package.
+- [x] Add standalone Plant Discovery Gradio entrypoint.
+- [x] Add clean plant model/domain config.
+- [x] Add deterministic no-model plant service.
+- [x] Add optional MiniCPM-V plant service adapter.
+- [x] Make OpenBMB MiniCPM-V the default real model mode.
+- [x] Add explicit demo/openbmb/finetuned runtime modes.
+- [x] Add optional fine-tuned adapter loading path.
+- [x] Keep optional model dependencies lazy.
+- [x] Add plant structured result schema and parser.
+- [x] Add species index builder.
+- [x] Add local image-folder loader.
+- [x] Add field-note correction export to plant training JSONL.
+- [x] Add focused Identify, Field Guide, Corrections, and Stats UI.
+- [x] Replace direct training execution with non-executing training plan.
+- [x] Add optional plant tool functions with lazy MCP server construction.
+- [x] Add non-executing plant training planner.
+- [x] Add `scripts/plan_plant_training.py`.
+- [x] Add Plant Discovery unit tests.
+- [x] Verify no-model app shell builds.
+- [x] Run Plant Discovery as a long-running local app.
+- [x] Generate Plant Discovery screenshots.
+- [x] Add Plant Discovery screenshots to README/docs.
+- [x] Decide whether hackathon Space launches root workbench or Plant Discovery app.
+- [x] Verify real MiniCPM-V plant identification with optional dependencies.
+- [ ] Train or configure a real Plant Discovery adapter.
+- [ ] Verify `--model-mode finetuned` with the real adapter.
+- [ ] Add public-mode file/path/url hardening before Space deployment.
+
+## Ongoing Maintenance
+
+- [x] Update docs after every implemented feature.
+- [x] Keep `IMPLEMENTATION_STATUS.md` current.
+- [x] Keep unchecked tasks visible.
+- [x] Keep secrets and model weights out of git.
+- [x] Re-run local app after code changes.

docs/TEMPLATE_HOWTO.md

@@ -0,0 +1,207 @@
+# Template How-To: Build A New Domain App
+
+This repository is a local-first Gradio AI app template. The base workbench provides shared
+patterns for model configuration, field notes, tracking, export planning, tests, docs, and
+deployment. A domain app is a focused product built around those patterns.
+
+Use `plant/` as the first reference domain app.
+
+## Core Principle
+
+Do not start by training a model. Start by shipping a useful zero-shot or demo-mode workflow:
+
+```text
+domain idea
+  -> user story
+  -> schema
+  -> model choice
+  -> focused UI
+  -> correction loop
+  -> export data
+  -> optional fine-tune
+  -> deploy and document
+```
+
+Training is a later optimization after you have corrected examples and a reason to tune.
+
+## Recommended Branch Flow
+
+1. Keep `main` as the reusable template.
+2. Create a branch for each app:
+
+   ```powershell
+   git checkout -b plant-discovery-app
+   ```
+
+3. Build the app under a domain folder such as `plant/`, `invoice/`, `recipe/`, or `field_notes/`.
+4. Keep domain-specific heavy requirements in `<domain>/requirements.txt`.
+5. Merge reusable improvements back into `main` only after they are generic.
+
+## Domain App File Contract
+
+Each generated app should have these files:
+
+```text
+<domain>/
+  __init__.py
+  app.py              # standalone Gradio entrypoint
+  models.yaml         # domain config, model IDs, data sources, training defaults
+  <domain>_service.py # optional real model adapter plus demo/no-model fallback
+  <domain>_loader.py  # data loading, schema normalization, export rows
+  <domain>_tab.py     # focused Gradio UI
+  <domain>_tools.py   # optional MCP/local tools with no hard optional imports
+  requirements.txt    # optional heavy dependencies for this app only
+```
+
+Add tests under:
+
+```text
+tests/unit/test_<domain>_reference_app.py
+```
+
+Add docs under:
+
+```text
+docs/<DOMAIN>_APP_PLAN.md
+```
+
+## Step-By-Step Build Process
+
+### 1. Define The Product
+
+- [ ] Pick one user.
+- [ ] Pick one job they need done.
+- [ ] Write one sentence: "This app helps X do Y without Z."
+- [ ] Choose one golden path that works in under two minutes.
+- [ ] Decide whether the app is a standalone product or a tab inside the workbench.
+- [ ] Decide whether it must run on a public Hugging Face Space.
+
+Example:
+
+> Plant Discovery helps gardeners identify a plant from a photo, correct mistakes, and export
+> local training examples without sending private field notes to a cloud API.
+
+### 2. Define The Domain Schema
+
+- [ ] Create a dataclass for the structured output.
+- [ ] Include confidence and model metadata.
+- [ ] Include a `to_dict()` method for Gradio JSON.
+- [ ] Add a robust parser for model responses.
+- [ ] Add tests for valid JSON, fenced JSON, trailing commas, and unparseable text.
+
+Plant example: `PlantID` in `plant/plant_service.py`.
+
+### 3. Pick The Model
+
+- [ ] Pick a small model at or below 32B parameters.
+- [ ] Document the exact model ID.
+- [ ] Add model metadata to `<domain>/models.yaml`.
+- [ ] Avoid loading weights on startup.
+- [ ] Add a deterministic demo/no-model service for screenshots and tests.
+- [ ] Add an unavailable-path response when optional packages are missing.
+- [ ] Add explicit runtime modes such as `demo`, `base-model`, and `finetuned`.
+- [ ] Do not claim a fine-tuned model until a real adapter/checkpoint is configured and verified.
+
+For vision apps, start with a VLM such as MiniCPM-V. For text apps, start with a small instruct
+model through LM Studio, Ollama, llama.cpp, or Transformers.
+
+### 4. Build The Focused UI
+
+- [ ] Make the first screen the golden path, not a generic dashboard.
+- [ ] Add only the controls needed for the user story.
+- [ ] Keep advanced setup behind a secondary tab or accordion.
+- [ ] Add visible status messages.
+- [ ] Add structured JSON output for debugging and reproducibility.
+- [ ] Add correction capture if model output can be wrong.
+- [ ] Add screenshots through Playwright after the UI is stable.
+
+### 5. Add The Correction Loop
+
+- [ ] Save user corrections locally.
+- [ ] Reuse `datasets.field_notes.FieldNoteStore` where possible.
+- [ ] Mark training-ready rows explicitly.
+- [ ] Export JSONL without starting training.
+- [ ] Add tests for save, filter, and export.
+
+### 6. Add Data Loaders
+
+- [ ] Support a small local demo dataset.
+- [ ] Support domain data from local folders or CSV/JSONL.
+- [ ] Keep Hugging Face dataset loading optional and explicit.
+- [ ] Do not download large datasets on startup.
+- [ ] Normalize every source into one training row schema.
+- [ ] Add loader tests with temporary local files.
+
+### 7. Add Optional Tools
+
+- [ ] Keep MCP/tool imports optional.
+- [ ] Tool functions should work locally without starting a server.
+- [ ] Add `build_mcp_server()` only if `mcp` is installed.
+- [ ] Avoid direct shell execution from tools.
+- [ ] Return command plans rather than running commands.
+- [ ] Add tests for pure tool functions.
+
+### 8. Add Training Plans
+
+- [ ] Start with a non-executing training plan.
+- [ ] Include required dependencies, hardware notes, and command preview.
+- [ ] Require enough corrected examples before recommending training.
+- [ ] Keep real training as a separate local command or approved action.
+- [ ] Add evaluation before/after tuning.
+- [ ] Add a small script that prints the training plan as JSON.
+
+### 9. Add Security Guardrails
+
+- [ ] Escape model text rendered as HTML.
+- [ ] Restrict file paths in public Space mode.
+- [ ] Disable arbitrary backend URL checks in public Space mode.
+- [ ] Do not execute subprocesses from Gradio callbacks.
+- [ ] Keep tokens, private data, model weights, and exports out of git.
+- [ ] Add tests for path traversal and malformed inputs when public deployment is planned.
+
+### 10. Verify The App
+
+Minimum local verification:
+
+```powershell
+.venv\Scripts\python.exe -m pytest tests/unit/test_<domain>_reference_app.py -q
+.venv\Scripts\ruff.exe check <domain> tests/unit/test_<domain>_reference_app.py --no-cache
+.venv\Scripts\python.exe -m mypy <domain> tests/unit/test_<domain>_reference_app.py --cache-dir "$env:TEMP\openbmb-workbench-mypy-cache"
+.venv\Scripts\python.exe -c "from <domain>.app import build_app; app=build_app(no_model=True); print(type(app).__name__)"
+```
+
+Before claiming it works:
+
+- [ ] Run the standalone app.
+- [ ] Generate screenshots.
+- [ ] Add screenshot links to README/docs.
+- [ ] Run full quality checks.
+- [ ] Commit and push.
+
+## When To Integrate Into The Main Workbench
+
+Keep the domain app standalone if:
+
+- it has its own brand/story,
+- it needs a focused judging experience,
+- it has domain-specific dependencies,
+- it should become a Hugging Face Space.
+
+Add it to the main workbench only if:
+
+- it is a generic reusable tab,
+- it does not add heavy dependencies,
+- it strengthens the template for all future apps.
+
+For the hackathon, standalone `plant/` is the better route because judges need one clear product.
+
+## What "Done" Means For A Domain App
+
+- [ ] Standalone no-model app builds.
+- [ ] Optional real model adapter is documented and lazy-loaded.
+- [ ] Golden path has tests.
+- [ ] Corrections export to training data.
+- [ ] Training is planned, not accidentally executed.
+- [ ] Screenshots are generated.
+- [ ] README explains setup, model choice, demo flow, and limitations.
+- [ ] Space deployment is verified or blocker is documented.

docs/USAGE.md

@@ -0,0 +1,245 @@
+# How To Use The Project
+
+## 1. Install Python
+
+Install Python 3.11 or newer. On Windows, make sure "Add Python to PATH" is enabled.
+
+Verify:
+
+```powershell
+python --version
+```
+
+Before Python is available, you can still verify the repository structure:
+
+```powershell
+.\scripts\verify_structure.ps1
+```
+
+In this workspace, the direct WindowsApps interpreter worked even though `python` was not on PATH:
+
+```powershell
+& "$env:LOCALAPPDATA\Microsoft\WindowsApps\python3.11.exe" --version
+```
+
+## 2. Create A Virtual Environment
+
+```powershell
+python -m venv .venv
+.venv\Scripts\Activate.ps1
+```
+
+If `python` is not on PATH, use the WindowsApps interpreter:
+
+```powershell
+& "$env:LOCALAPPDATA\Microsoft\WindowsApps\python3.11.exe" -m venv .venv
+```
+
+## 3. Install Dependencies
+
+```powershell
+python -m pip install -r requirements.txt
+```
+
+For tests and quality checks:
+
+```powershell
+python -m pip install -r requirements-dev.txt
+```
+
+## 4. Run The App
+
+```powershell
+python app.py
+```
+
+## 5. Run Tests
+
+```powershell
+.\scripts\run_tests.ps1
+```
+
+## 6. Run Quality Checks
+
+```powershell
+.\scripts\run_quality.ps1
+```
+
+Open the local Gradio URL printed in the terminal, usually:
+
+```text
+http://127.0.0.1:7860
+```
+
+## 7. Current App Behavior
+
+Local development can still use deterministic helper services for tests, but deployed Space mode
+uses `WORKBENCH_DEPLOYMENT=space`, hides placeholder backends, and requires real backend
+configuration for model calls. The app still does not download model weights on startup.
+
+Available tabs. User-triggered tab actions show Gradio progress while callbacks run, and the app
+uses compact responsive styling for narrow screens:
+
+- Chat - llama.cpp, llama-cpp-python, Ollama, OpenAI-compatible/LM Studio, SGLang, or Transformers text inference with tab status/errors.
+- Vision - llama.cpp, llama-cpp-python, Ollama, or Transformers MiniCPM image + prompt inference with tab status/errors.
+- Dataset - local CSV/JSONL/NDJSON preview, optional Hugging Face dataset preview, stats, and tab status/errors.
+- Train - LoRA dry-run training plan plus local base-vs-tuned exact-match evaluation.
+- Export - GGUF download/conversion/quantization planning, exported-file listing, and existing-file downloads.
+- Field Notes - saves human corrections to CSV, imports uncertain OCR predictions, captures media paths/training flags, exports corrected JSONL, corrected OCR JSONL, and local HF Dataset files.
+- Traces - local event preview, JSONL trace rows, tracking status, and trace export.
+- Agent - local non-autonomous research-plan-implement-verify trace mode.
+- Status - shows configured models, backend metadata, local llama.cpp setup, LM Studio/OpenAI-compatible setup, SGLang command/check/stop planning, and Ollama list/pull planning.
+
+Ollama is optional and is not installed automatically. Install and start Ollama yourself, then
+pull a compatible model explicitly before selecting the Ollama backend in the app. The Status tab
+can list models from a running local Ollama server and prepare an explicit
+`ollama pull <model>` command. It shows the command only; it does not run downloads for you.
+
+llama.cpp is the preferred hackathon backend path. Install llama.cpp separately, open the Status
+tab, pick or type an explicit GGUF path, optionally pick an mmproj GGUF for vision, then click
+`Prepare local model config`. The app writes `data/local_backends.yaml`, shows the
+`llama-server` command, and still does not download or load model weights on startup.
+Start `llama-server` yourself with that command, then select the `llama.cpp` backend in the app.
+`llama-cpp-python` is also available as an optional backend when the package is installed and a
+local GGUF path is configured.
+
+LM Studio or another local OpenAI-compatible server is optional and is not a cloud API path. Start
+the local server yourself, open the Status tab, set the server URL such as
+`http://127.0.0.1:1234`, optionally enter the exact served model name shown by LM Studio, and
+click `Save OpenAI-compatible config`. `Check server` calls `/v1/models`; selecting
+`openai-compatible` in the Chat tab posts to `/v1/chat/completions` only after you submit a prompt.
+No model weights are downloaded or loaded on app startup.
+
+The Transformers text backend is optional. It requires installing `transformers` and a compatible
+PyTorch build for your CPU/GPU, and it may download model weights when you explicitly select it and
+run a prompt. It is not installed automatically and is not used on startup.
+
+The Transformers vision backend is optional for MiniCPM-V models. It uses `AutoProcessor` and
+`AutoModelForImageTextToText` only after you select `transformers` in the Vision tab and submit an
+image. The current app maps the thinking toggle into the prompt template and documents video as a
+future frame-sampling path; it does not download or load vision weights on startup.
+
+SGLang is optional. The Status tab can prepare a local `python -m sglang.launch_server` command,
+check `/health`, request `/shutdown`, and the Chat tab can use the `sglang` backend against
+`/v1/chat/completions` after you start the server yourself. The app does not start SGLang on
+startup.
+
+Dataset preview supports local `.csv`, `.jsonl`, and `.ndjson` files, split names for optional
+Hugging Face dataset preview, and basic local statistics. Field Notes can export corrected
+training rows to `data/field_notes.jsonl` and local HF Dataset-style files to
+`data/hf_field_notes/`.
+
+The OCR correction hook expects a local `.csv`, `.jsonl`, or `.ndjson` prediction file with fields
+such as `source_path`, `text`, and `confidence`. The Field Notes tab can preview rows at or below a
+confidence threshold, import those uncertain rows as correction tasks, and export corrected OCR rows
+to `data/ocr_corrections.jsonl`. This is the local OCR -> Field Notes -> Training path: import
+uncertain predictions, correct them in Field Notes, then export corrected JSONL/HF Dataset files for
+future fine-tuning or evaluation.
+
+Local MCP-style tools live in `mcp_tools/tools.py`. The selected MCP path is Gradio native MCP,
+enabled by `launch(mcp_server=True)`. The documented endpoint path is `/gradio_api/mcp/sse`.
+`mcp_tools/bridge.py` exposes a local manifest and invocation helper for dataset stats, optional
+HF dataset preview, safe arithmetic, model inference, and non-executing VINDEX call planning. Full
+external MCP client verification still depends on launching the app and connecting a client.
+
+The VINDEX helper is intentionally planning-only. It validates the PRD method names, reports
+whether the local `vindex` package or `http://127.0.0.1:8765/health` endpoint is available, and
+caps risky edit parameters. It does not edit model weights.
+
+The Agent tab drafts local research-plan-implement-verify traces and paper-to-code traces, stores
+them in `data/agent_traces.jsonl`, and can export JSONL or local HF Dataset-style trace files. It
+includes safety gates and does not run shell commands, commit, push, deploy, download models, or
+call external services. Agent traces can later become persona/source material for the Desk-Pet
+extension, but no Desk-Pet runtime is implemented yet.
+
+The Export tab is a planning surface. It shows explicit `huggingface-cli`,
+`convert_hf_to_gguf.py`, and `llama-quantize` commands for the selected model and quantization,
+plus a list of files already present in the export directory. Existing files are exposed through
+the download output. It does not run those commands yet.
+
+The Train tab does not start LoRA training yet. It builds a dry-run plan from
+`config/training.yaml`, validates the dataset path, shows the planned checkpoint output directory,
+documents hardware expectations, and builds a non-executing LoRA trainer request with PEFT/TRL
+dependency status. It can also run a local evaluation by comparing newline-separated base and tuned
+responses against the built-in prompt cases. It reports exact match, can calculate perplexity from
+optional negative log likelihood values, shows a qualitative table, and appends tuned results to
+`data/eval_results.jsonl`.
+
+The vLLM tab is a local serving planner. It prepares an explicit `vllm serve ...` command, checks
+`/health`, fetches `/metrics`, logs parsed benchmark metrics through local tracking, and points the
+chat client at `/v1/chat/completions` only after you start vLLM yourself. It does not install vLLM
+or start a server on app startup.
+
+Tracing writes local events to `data/traces.jsonl` by default. The Traces tab can show recent app
+events, show JSONL trace rows, report whether optional Trackio is available, and export local
+traces to `exports/traces.jsonl`. Remote Trackio/HF sync still requires credentials and setup.
+
+## 8. Hugging Face Space Deployment
+
+The repo includes Hugging Face Space metadata in `README.md` and a local planning helper:
+
+```powershell
+.venv\Scripts\python.exe scripts\plan_hf_space.py --user <hf-user-or-org>
+```
+
+The helper validates required files and prints the manual commands for login, Space creation,
+remote setup, and push. It does not login, create a repo, push, or store tokens.
+
+Required Space files:
+
+- `app.py`
+- `requirements.txt`
+- `README.md`
+- `config/models.yaml`
+- `config/training.yaml`
+
+Target Spaces:
+
+- Workbench: `https://huggingface.co/spaces/build-small-hackathon/workbench`
+- Plant Identification Tool: `https://huggingface.co/spaces/build-small-hackathon/plant_identification_tool`
+
+Use `hf auth login` with a freshly generated token. The app does not download model weights on
+startup; model downloads happen only through explicit backend actions such as `ollama pull`,
+`hf download`, or selecting a real Transformers backend and running a prompt. Upgrade Space
+hardware if real OpenBMB text or vision inference cannot run on the default CPU tier.
+
+Trackio/HF sync path: local traces are written to JSONL first, then optional Trackio can be enabled
+in `config/training.yaml` after package availability and credentials are ready.
+
+Synthetic data and reward evaluation helpers are local Python utilities. `datasets/synthetic.py`
+can generate, validate, filter, augment, and export JSONL examples. `training/reward_eval.py` can
+score supplied responses, select best-of-N candidates, create DPO chosen/rejected pairs, and compare
+base responses against LoRA responses. They do not load reward models or call external services.
+
+## 9. How To Work With Codex
+
+Useful prompts:
+
+```text
+Read docs/TASKS.md and implement the next unchecked MVP task.
+```
+
+```text
+Read docs/IMPLEMENTATION_STATUS.md and tell me what is blocked.
+```
+
+```text
+Wire the next backend, but keep automatic model downloads disabled on startup.
+```
+
+```text
+Update the docs after the change and mark the matching checklist item.
+```
+
+```text
+If this failed, add or update a test that catches it, then fix the code.
+```
+
+## 10. What To Avoid
+
+- Do not start with the whole PRD at once.
+- Do not download huge models automatically.
+- Do not mark tasks done before running or documenting the blocker.
+- Do not add features without tests.
+- Do not add broad coverage escapes such as pragma no cover without a documented reason.
+- Do not push secrets, tokens, model caches, GGUF files, or virtual environments to git.

exports/.gitkeep

@@ -0,0 +1 @@
+

mcp_tools/__init__.py

@@ -0,0 +1 @@
+"""Local tool functions that can later be exposed through an MCP endpoint."""