Spaces:
Running
Running
|
Download docs/model-providers.md from ignatius/curatorbot: direct link, hf CLI and curl.
- Browser
- Download file 2.79 kB
-
https://huggingface.co/spaces/ignatius/curatorbot/resolve/main/docs/model-providers.md
- Command line
-
hf download hf://spaces/ignatius/curatorbot/docs/model-providers.md
-
curl -L -o model-providers.md https://huggingface.co/spaces/ignatius/curatorbot/resolve/main/docs/model-providers.md
2.79 kB
| # Choosing the language model | |
| CuratorBot sends one request per interview to a language model, which drafts the summary, chapters, subject headings, journeys and donor checks. The validators then check every quote against the transcript, whichever model wrote it. | |
| Each institution chooses where its testimony goes. An administrator sets this once with environment variables (Space secrets and variables on Hugging Face, `.env` locally). Nothing is chosen per user. | |
| | `CURATORBOT_LLM_PROVIDER` | Key | Default model | Use it when | | |
| | --- | --- | --- | --- | | |
| | `openrouter` (default) | `OPENROUTER_API_KEY` | `anthropic/claude-sonnet-5.5`, falling back to `openai/gpt-4.1-mini` | One account reaches many models; good for demos and comparisons | | |
| | `anthropic` | `ANTHROPIC_API_KEY` | `claude-sonnet-5-5` | The institution has an Anthropic contract | | |
| | `openai` | `OPENAI_API_KEY` | `gpt-4.1-mini` | The institution has an OpenAI contract | | |
| | `compatible` | `CURATORBOT_LLM_API_KEY` (if needed) | set `CURATORBOT_LLM_MODEL` | Testimony must not leave the institution: its own gateway, or open models served with vLLM or Ollama | | |
| Other settings: | |
| - `CURATORBOT_LLM_MODEL`: use a different model from the provider's catalogue. | |
| - `CURATORBOT_LLM_FALLBACK_MODEL`: tried once if the main model fails for a reason another model could fix (an outage, a rate limit, an unreadable answer). It is not tried for account problems such as a wrong key or no credit. Set it to an empty value to switch the fallback off. Briefs record which model actually answered. | |
| - `CURATORBOT_LLM_BASE_URL`: the address of a `compatible` endpoint, for example `http://localhost:11434/v1` for Ollama. | |
| - `CURATORBOT_LLM_JSON_MODE=0`: for local servers that reject the JSON-mode option. | |
| - `CURATORBOT_LLM_MAX_TOKENS` (default 16000): the longest answer the model may write. A full interview's brief needs about 6,000; if an answer is cut off, the run falls back to the second model, which the brief records. | |
| `/api/health` shows the active provider, model and fallback. | |
| ## Data protection | |
| - **OpenRouter** forwards each request to the company running the chosen model (Anthropic, OpenAI, or a cloud host of either). In the OpenRouter account's privacy settings, allow only providers that neither train on nor retain prompts before sending pre-accession material. | |
| - **Direct contracts** (Anthropic, OpenAI) fall under the institution's own terms with that company. | |
| - **`compatible` with a local server** keeps testimony on the institution's own machines. Quality depends on the open model chosen; run the back-test (`docs/real-data-runbook.md`) before relying on it. | |
| When a model call fails, curators see a plain sentence ("Live analysis is unavailable right now: …"); the technical detail goes to the server log only. | |