docs/setup.md

# Setup Guide

## Scope

This guide covers installation and configuration of PrecedentAI's legal research and drafting workflows:

1. LangGraph graph skeleton (routing, checkpointing, human-in-the-loop)
2. Legal retrieval — keyword floor and ChromaDB + LaBSE at corpus scale
3. Judgment summarization — structured, plain-language, and Hindi bilingual variants
4. Safe drafting — interrupt-gated, citation-checked
5. Gated LLM stage — facts pass, translation, drafting, routing, planning (Groq)
6. Agentic patterns — self-corrective retrieval, multi-agent supervisor
7. Serving layer and chat UI — FastAPI, SSE streaming, usage ledger

Transcription is intentionally outside the current scope.

## Prerequisites

- Python 3.12 or newer
- `uv` package manager
- A local editor and terminal
- GPU is optional: LaBSE inference and index builds run on CPU, just slower

## Install Dependencies

From the repository root:

```bash
uv sync
```

The dependency metadata is in `pyproject.toml`.

## Environment

Copy the example environment file if you want the LLM stage:

```bash
cp .env.example .env
```

Nothing requires an API key. Retrieval, summarization, drafting, and the
whole test suite run on the deterministic floor with no keys and no network.
The selected provider's key is used only when `LEGAL_AI_ENABLE_LLM=true`:

- `GROQ_API_KEY` — the default provider
- `OPENAI_API_KEY` — when `LEGAL_AI_LLM_PROVIDER=openai`
- `NVIDIA_API_KEY` — when `LEGAL_AI_LLM_PROVIDER=nvidia`
- `AR_API_KEY` — when `LEGAL_AI_LLM_PROVIDER=agentrouter`

Do not commit `.env` or any real API keys.

## Data Setup

### Public Adalat AI Datasets

The Text-track retrieval experiments can start with public datasets from the Adalat AI Hugging Face organization:

- `adalat-ai/Indian-Legal-Retrieval-Generation`
- `adalat-ai/in22-legal`
- `adalat-ai/indian-legal-exam-benchmark`

Use only data whose license and usage terms permit the intended experiment.

### Local Legal Documents

When adding local material, organize it by source and language:

```text
data/legal/
├── constitution/
├── ipc/
├── judgments/
│   ├── hi/
│   ├── ta/
│   ├── te/
│   ├── kn/
│   ├── mr/
│   └── bn/
└── metadata/
```

Every document should have a stable source ID, court, year, language, and license/provenance metadata.

### Audio Data

No audio setup is required for the current Text-track phases. Add `data/speech/` only when starting the separate transcription work.

## Run the Graph

With no index and no keys, the graph runs on the offline fixture floor:

```bash
export LEGAL_AI_RETRIEVAL_BACKEND=keyword
uv run python -m agent.core      # routes three sample requests and prints state
```

With both indexes built (next section) the default backend is fused and the
same command searches the real corpus. The API serves both:

```bash
uv run --extra serve --extra ml --extra llm uvicorn api.main:app
# GET /health reports the live backend and whether the index is on disk
```

## Retrieval Backends

Three backends sit behind one contract (`retrieval_backend` config knob):

- **`fused` (default)** — independent ChromaDB semantic search, SQLite FTS5/BM25
search and exact identity lookup, merged at judgment level with supporting passages.
- **`semantic`** — ChromaDB + LaBSE over the indexed corpus, for dense-only comparisons.
- **`keyword`** — a 3-document in-memory fixture with zero dependencies. The
offline floor the test suite runs on (`tests/conftest.py` pins it) and the
escape hatch when there is no index on disk. It is a fixture, not a search
engine: it scores 0 against real questions.

Build the retrieval indexes — it is a separate, explicit step so model
downloads never happen at graph-compile or test time:

```bash
uv sync --extra ml                                # one-time, heavy (torch)
uv run --extra ml python scripts/build_index.py   # embeds the corpus (config.corpus_path)
```

For an existing vector index, add the independent lexical index without embedding
the corpus again:

```bash
uv run python scripts/build_index.py --lexical-only
```

The SQLite search sidecar lives alongside the Chroma collection. It is separate
from the parent-text reassembly sidecar below. Build it again when the corpus
changes; the fused backend reports a missing sidecar rather than pretending to
perform corpus-wide keyword search.

`--corpus` defaults to `LEGAL_AI_CORPUS_PATH`. Pass
`--corpus data/legal/seed/judgments.jsonl` for a 7-document smoke index.
Indexes are gitignored; budget disk and build time for the full collection.
The source-aware profile below stores text and rich provenance once in SQLite,
with vectors and filter metadata in Chroma. Builds stream bounded batches and
resume committed progress. Rebuild in a separate directory when changing the
corpus or embedding model; the builder rejects incompatible reuse.

Full-judgment reassembly (summarize, citation checks) reads parent texts
through a SQLite sidecar next to the corpus JSONL, created on first use.

## Summarization (Phase 3)

Summarization runs automatically as a **retrieve -> summarize** chain: a
"Summarize the holding in..." request retrieves the judgment first, then
the node reassembles the full parent text from the chunked corpus
(`corpus_path` config knob) and extracts structured sections from its SCR
headnote. Every returned quote is a verbatim source span carrying its
character offset.

Score the summarizer against the hand-labeled landmark references:

```bash
uv run python -m eval.run_summarize_eval
```

The golden set lives in `data/eval/summarization.jsonl` (5 judgments:
case_id, reference sections, key_points). Targets: quote support >= 0.90,
holding ROUGE-1 >= 0.50, key-point coverage >= 0.60, section F1 >= 0.40.

### Output variants

The summary output is a config knob (`LEGAL_AI_SUMMARY_VARIANT`, default
`structured`):

```bash
LEGAL_AI_SUMMARY_VARIANT=plain   # glossary-simplified, short sentences
LEGAL_AI_SUMMARY_VARIANT=hindi   # bilingual: Hindi labels + verified terms
```

Score each variant against the golden references:

```bash
uv run python -m eval.run_summarize_eval --variant plain
uv run python -m eval.run_summarize_eval --variant hindi
```

Every runner also prints its LLM spend footer (deterministic runs report
`none`). Add `--llm` to the summarize runner to run the anchor-gated
LLM facts pass per record and score `facts_llm_coverage` (token-level
recall over the golden reference facts — informational, not a gated
target, since valid narratives legitimately pick different aspects to
cover):

```bash
uv run python -m eval.run_summarize_eval --llm
```

If Groq's free-tier daily token budget is exhausted, the `--llm` pass
degrades to the deterministic floor (`llm_ok=0.000`) and the spend
footer reports the failed calls — rerun after the window resets.

## Testing

```bash
uv run --all-extras pytest -q                 # small offline behavioral suite
LEGAL_AI_RUN_EMBEDDING_TESTS=1 uv run --all-extras pytest tests/test_semantic.py -q
```

The default suite uses small fixtures and fake model clients. It covers complete
chat and streaming turns, isolated conversation history, approval and rejection,
source reading, semantic-index round trips, cited answer fallbacks, summary/OCR
regressions, translation gates, provider retries, supervisor delegation, and all
three evaluation harnesses. Two real-LaBSE checks are opt-in. Install all extras
for full offline coverage; otherwise API/index tests may be skipped.

Keep a test when it protects a user-visible behavior, a distinct failure mode,
or a bug found in practice. Extend an existing workflow test when it can check
another part of the same result. Import smokes, framework/schema checks,
visual graph topology, repeated helper permutations and test-count targets do
not belong in this suite. Model quality is measured by the evaluation harnesses.

## Retrieval Evaluation

```bash
uv run --extra ml python -m eval.run_eval --backend fused      # the product default
uv run python -m eval.run_eval --backend keyword               # fixture floor
uv run --extra ml python -m eval.run_eval --backend both      # dense versus hybrid
```

The runner exits nonzero when any docs/benchmark.md target is missed (CI-friendly).
Two golden sets live in `data/eval/`:

- `golden_retrieval_scaled.jsonl` — 8 queries originally labelled against the legacy corpus by `case_id`; the unchanged labels also evaluate the rebuilt snapshot (the runner's default)
- `golden_retrieval.jsonl` — 10 queries over the 7-doc seed corpus, kept for provenance

Each line is one JSON object: `query`, graded `relevant` (id -> 0..3), and the
`controlling` authority. Pass `--golden` to switch sets.

Every eval run also captures misses into the failure store (disable with
`--no-failures`):

```bash
uv run python -m eval.failure_loop --summary     # digest by type and harness
uv run python -m eval.failure_loop --pending     # records awaiting review
uv run python -m eval.failure_loop --review <failure_id> "note"
```

## Citation Evaluation (CourtNav QA pairs)

Scores page-level citation grounding against the 21 lawyer-verified QA pairs of
`adalat-ai/Indian-Legal-Retrieval-Generation`:

```bash
# one-time: extract PDFs to page records and build the page index
uv run --extra ml python -m eval.run_citation_eval --backend semantic
uv run --extra ml python -m eval.run_citation_eval --backend fused     # semantic + lexical re-ranking
uv run python -m eval.run_citation_eval --backend keyword              # baseline
uv run --extra ml python -m eval.run_citation_eval --mode doc          # document ranking
```

Inputs (gitignored, rebuilt from the HF dataset — see `eval/run_citation_eval.py`):
`data/hf/adalat-retrieval/` (CSV + PDFs), `data/legal/courtnav/pages.jsonl` (79
pages), and the LaBSE index at `models/courtnav/`.

## Source-aware Supreme Court snapshot

The rebuilt 2000–2025 collection contains 21,529 distinct judgments/orders and
929,151 passages. It preserves every previous case identity and adds two official
PDF recoveries. It remains a fixed Supreme Court snapshot; it does not cover all
Indian courts or establish current precedent status. See
[corpus ingestion](corpus-ingestion.md) for pinned sources, checksums, duplicate
handling, date discrepancies and OCR limits.

Install ingestion and model dependencies, plus system Poppler. Tesseract and
Poppler's rendering utilities support the image-only fallback.

```bash
uv sync --extra ingest --extra ml --extra serve --extra llm
uv run --no-sync python scripts/rebuild_source_aware.py --years 2000-2025
uv run --no-sync python scripts/build_index.py \
  --corpus data/legal/corpus/sc_2000_2025_source_aware.jsonl \
  --chroma-dir models/chroma-source-aware --collection legal_corpus \
  --compact --batch-size 512 --max-tokens 512
```

The recovery command verifies the pinned archives and publishes the merged corpus
only after extraction and old-case coverage checks pass. The index command builds
independent lexical retrieval and dense vectors. Both commands resume verified
work; the API refuses an incomplete dense build. Run the unchanged retrieval
benchmark before switching the configured corpus and index together:

```bash
uv run --no-sync python -m eval.run_eval \
  --chroma-dir models/chroma-source-aware --backend fused
```

Use `--backend both` to retain the dense-only comparison as well; each backend
reports its own target misses.

Set these values in `.env`, preserving your provider configuration:

```dotenv
LEGAL_AI_CORPUS_PATH=./data/legal/corpus/sc_2000_2025_source_aware.jsonl
LEGAL_AI_CHROMA_DIR=./models/chroma-source-aware
LEGAL_AI_CHROMA_COLLECTION=legal_corpus
LEGAL_AI_SOURCE_ARCHIVE_ROOT=./data/hf/scj
LEGAL_AI_RETRIEVAL_BACKEND=fused
```

Restart the API to reload settings. `/health` reports process liveness and file
presence, so verify an actual research turn and source PDF after switching.
The parent-text SQLite sidecar builds with bounded memory on first document
read. Retain the hash-checked source archives under the configured trusted root
for `/document/{parent_id}/pdf` to serve the extracted edition. Known archive
layouts can move to a different root without rewriting the corpus.

The original `sc_2000_2025.jsonl` and `models/chroma/` remain compatible rollback
artifacts. Restore both old paths together to revert; do not pair an old corpus
with the new retrieval index. New document IDs preserve verified case aliases
for unambiguous legacy document links.

## LLM Stage (Groq, OpenAI, NVIDIA, AgentRouter)

The deterministic floor is the default; LLM capabilities activate with
`LEGAL_AI_ENABLE_LLM=true` plus a Groq key (`GROQ_API_KEY` in `.env` or
`LEGAL_AI_GROQ_API_KEY`). Every LLM output is verified before it ships
and every failure degrades to the deterministic path with an explicit
`llm_status` — the floor is always the safe output.

```bash
# 1. Add your key to .env (free at console.groq.com/keys)
#    GROQ_API_KEY=gsk_...
# 2. Flip the switch
export LEGAL_AI_ENABLE_LLM=true

# 3. Offline test suite (stub clients, no network) — always works
uv run pytest tests/test_llm.py -q

# 4. Evaluate the real provider through the summarization harness
uv run --extra llm python -m eval.run_summarize_eval --help
# Use the --llm flag with the corpus/reference options documented below.
```

Capabilities, all behind verification gates:

- **Facts pass** (`tools/llm_passes.extract_facts_with_llm`) — LLM reads
  the judgment body to extract the facts narrative the SCR headnote
  lacks; every sentence must share a legal anchor (Art.21, s 66A, a
  year) with the judgment or it is dropped. Wired into the summarize
  node (`facts_llm` field on the summary).
- **Hindi translation** (`translate_sections_with_llm`) — LLM translates
  the structured sections; the gate requires Devanagari output and that
  every legal anchor number (21, 66A, 2015) survives — by number, so
  "Article 21" → "अनुच्छेद 21" passes. Failure falls back to the
  bilingual floor.
- **Drafting** (`tools.act.draft_petition`) — LLM drafts the GROUNDS
  from retrieved sources; the citation gate rejects any `[source_id]`
  not in the retrieved set (hallucinated authorities cannot ship), and
  the human-review interrupt stays.

The model ID (`LEGAL_AI_LLM_MODEL`, default `openai/gpt-oss-120b`) must be
available on the selected provider and account. Provider errors fall back to
source excerpts; there is no separate model-catalog validation call.

To use NVIDIA inference, install the existing LLM extra (`uv sync --extra llm`)
and set these values in the server environment or `.env`:

```dotenv
LEGAL_AI_ENABLE_LLM=true
LEGAL_AI_LLM_PROVIDER=nvidia
LEGAL_AI_LLM_MODEL=deepseek-ai/deepseek-v4.1-flash
LEGAL_AI_LLM_FALLBACK_MODEL=
LEGAL_AI_LLM_BASE_URL=
NVIDIA_API_KEY=your-nvidia-api-key
```

NVIDIA defaults to `https://integrate.api.nvidia.com/v1`, using the
[OpenAI-compatible chat endpoint](https://docs.api.nvidia.com/nim/reference/nvidia-deepseek-v4_1-flash-infer).
`LEGAL_AI_NVIDIA_API_KEY` can supply an explicit scoped key instead. Clear any
previous provider's fallback model and base URL when switching. An optional
fallback must be a model served by the same provider; without one, answer checks
use the main model. The existing completion limits, retry limits, usage ledger
and answer checks apply to NVIDIA too. It uses JSON-object mode; Groq's
model-specific strict-schema and reasoning parameters are not sent to NVIDIA.

To use AgentRouter, install the same LLM extra and set these values in the server
environment or `.env`:

```dotenv
LEGAL_AI_ENABLE_LLM=true
LEGAL_AI_LLM_PROVIDER=agentrouter
LEGAL_AI_LLM_MODEL=deepseek-v4-flash
LEGAL_AI_LLM_FALLBACK_MODEL=
LEGAL_AI_LLM_BASE_URL=
AR_API_KEY=your-agentrouter-api-key
```

The configured AgentRouter default is `https://agentrouter.org/v1`.
`LEGAL_AI_AGENTROUTER_API_KEY` can supply an explicit scoped key instead.
Choose a model ID available to your AgentRouter account; the example ID is
provider-specific and differs from NVIDIA's model ID. Clear the previous
provider's fallback and endpoint when switching, or set an explicit
`LEGAL_AI_LLM_BASE_URL` for your gateway. A fallback must be served by the same
gateway. This integration uses the OpenAI-compatible chat API and sends the
provider-specific User-Agent defined in `tools/llm_client.py`. Completion limits,
bounded retries, token accounting and answer checks also apply to AgentRouter.

## Serving Layer + Chat UI (Phase 9)

A FastAPI app wraps the graph and a minimal dark-mode chat UI is served
at `/` — streaming node progress (SSE), source cards, verbatim quotes
with offsets, and the human-review approve/reject flow for drafts.

```bash
uv sync --extra serve            # adds fastapi + uvicorn (plus llm/ml as needed)
uv run --extra serve uvicorn api.main:app --reload
# open http://127.0.0.1:8000
```

Endpoints:

- `GET /health` — liveness + active mode (llm on/off, retrieval backend)
- `POST /chat {message, thread_id}` — synchronous single-turn invoke;
  returns `interrupted: true` + review payload when drafting pauses
- `POST /stream` — SSE stream of per-node progress events, final `done`
- `POST /supervise {message, thread_id}` — multi-agent supervised run
  (Phase 8B): the supervisor LLM-plans a delegation across three real
  sub-agent graphs (researcher → analyst → drafter), re-invokes
  insufficient results with sharpened tasks, and composes the final
  payload; deterministic fallback when LLM is off
- `GET /usage` — LLM usage ledger for the process: call counts, exact
  token totals per model, and ESTIMATED dollar cost (point-in-time
  price table; token counts are the billing-grade numbers)
- `GET /history/{thread_id}` — accumulated turns from the checkpointer
- `GET /document/{parent_id}` — the whole reassembled parent judgment a
  retrieved chunk came from (metadata + full `text` + `characters`).
  Retrieval ranks chunks, so this is what lets a client show the case and
  not only the slice; 404 when the corpus is absent or the id is not in
  it, 422 for a malformed id
- `POST /review/{thread_id} {approved, note}` — resume an interrupted
  draft (approve → document generated; reject → logged, nothing drafted)

Note on Groq's free tier: the daily token budget (200k TPD) can be
exhausted by long map-reduce runs — the graph then degrades to the
deterministic floor with `llm_status: unavailable:...` until the window
resets.

## Current State

The whole Text-track scope is implemented: the graph (routing, checkpointing,
human-in-the-loop, self-corrective retrieval), semantic retrieval over the
scaled corpus, structured summarization with output variants, gated LLM
capabilities (Groq), the multi-agent supervisor, the serving layer, and the
evaluation suite. Run the graph with `uv run python -m agent.core`. The
`.env.example` file lists every `LEGAL_AI_*` knob that `shared/config.py`
reads — nothing else is consulted.