Revert app defaults to Qwen
Browse filesCo-authored-by: Codex <codex@openai.com>
- README.md +16 -53
- config.py +3 -3
- tests/test_app.py +1 -1
README.md
CHANGED
|
@@ -10,9 +10,7 @@ app_file: app.py
|
|
| 10 |
pinned: false
|
| 11 |
short_description: Simplify complex legislation that affects you!
|
| 12 |
models:
|
| 13 |
-
-
|
| 14 |
-
- nvidia/llama-nemotron-embed-vl-1b-v2
|
| 15 |
-
- nvidia/llama-nemotron-rerank-1b-v2
|
| 16 |
---
|
| 17 |
|
| 18 |
# Legislation Explainer
|
|
@@ -31,7 +29,7 @@ It is created for the Hugging Face Build Small Hackathon under the `Backyard AI`
|
|
| 31 |
|
| 32 |
- Track: `Backyard AI`
|
| 33 |
- Real user: Ghanaian citizens and digital-policy stakeholders who need a clearer view of a bill's practical effects.
|
| 34 |
-
- Small-model constraint: default generation uses `
|
| 35 |
- Required surface: Gradio app, ready for Hugging Face Spaces through `app.py`.
|
| 36 |
- GitHub repo: https://github.com/KayO-GH/legislation-explainer
|
| 37 |
|
|
@@ -50,13 +48,13 @@ It is created for the Hugging Face Build Small Hackathon under the `Backyard AI`
|
|
| 50 |
|
| 51 |
## Model And Provider Notes
|
| 52 |
|
| 53 |
-
The default
|
| 54 |
|
| 55 |
-
-
|
| 56 |
-
-
|
| 57 |
-
-
|
| 58 |
|
| 59 |
-
|
| 60 |
|
| 61 |
## Local Run
|
| 62 |
|
|
@@ -65,53 +63,18 @@ pip install -r requirements.txt
|
|
| 65 |
python app.py
|
| 66 |
```
|
| 67 |
|
| 68 |
-
Set these environment variables for the default
|
| 69 |
|
| 70 |
-
- `
|
| 71 |
-
- `NEMOTRON_BASE_URL` optional, defaults to `https://router.huggingface.co/v1`
|
| 72 |
-
- `NEMOTRON_RETRIEVER_API_KEY` optional, defaults to `NEMOTRON_API_KEY`
|
| 73 |
-
- `NEMOTRON_RETRIEVER_BASE_URL` optional, defaults to `NEMOTRON_BASE_URL`
|
| 74 |
-
- `NEMOTRON_RERANKER_API_KEY` optional, defaults to `NEMOTRON_API_KEY`
|
| 75 |
-
- `NEMOTRON_RERANKER_URL` optional but recommended for the dedicated reranker path
|
| 76 |
-
|
| 77 |
-
`HF_TOKEN` still works as a fallback credential for Hugging Face-routed Nemotron endpoints.
|
| 78 |
-
|
| 79 |
-
## Modal Deployment For Nemotron
|
| 80 |
-
|
| 81 |
-
Hugging Face currently lists no hosted Inference Provider for `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`, so the practical path for the generator is to self-host it. This repo includes [modal_nemotron_service.py](/Users/Kwadwo/Documents/PROJECTS/HF-Build-Small/legislation-explainer/modal_nemotron_service.py), a standalone Modal deployment using SGLang's OpenAI-compatible server.
|
| 82 |
-
|
| 83 |
-
1. Install and authenticate Modal locally.
|
| 84 |
-
2. Create a Modal secret named `huggingface-secret` that contains your `HF_TOKEN`.
|
| 85 |
-
3. From [`legislation-explainer/`](/Users/Kwadwo/Documents/PROJECTS/HF-Build-Small/legislation-explainer/), deploy:
|
| 86 |
-
|
| 87 |
-
```bash
|
| 88 |
-
pip install modal
|
| 89 |
-
modal setup
|
| 90 |
-
modal secret create huggingface-secret HF_TOKEN=hf_xxx
|
| 91 |
-
modal deploy modal_nemotron_service.py
|
| 92 |
-
```
|
| 93 |
|
| 94 |
-
|
| 95 |
|
| 96 |
-
``
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
``
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
Recommended first use:
|
| 104 |
-
|
| 105 |
-
- Keep the generator on Modal via `NEMOTRON_BASE_URL`.
|
| 106 |
-
- Leave the retriever and reranker on their own hosted endpoints if you have them.
|
| 107 |
-
- If you do not yet have hosted retriever or reranker coverage, the app will fall back to local MiniLM embeddings and skip hosted reranking.
|
| 108 |
-
|
| 109 |
-
Useful deployment knobs:
|
| 110 |
-
|
| 111 |
-
- `NEMOTRON_MODAL_GPU` defaults to `H100`
|
| 112 |
-
- `NEMOTRON_MODAL_MIN_CONTAINERS` defaults to `0`
|
| 113 |
-
- `NEMOTRON_MODAL_CONTEXT_LENGTH` defaults to `32768`
|
| 114 |
-
- `NEMOTRON_MODAL_TARGET_INPUTS` defaults to `8`
|
| 115 |
|
| 116 |
## Space Deployment
|
| 117 |
|
|
|
|
| 10 |
pinned: false
|
| 11 |
short_description: Simplify complex legislation that affects you!
|
| 12 |
models:
|
| 13 |
+
- Qwen/Qwen3-14B
|
|
|
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
# Legislation Explainer
|
|
|
|
| 29 |
|
| 30 |
- Track: `Backyard AI`
|
| 31 |
- Real user: Ghanaian citizens and digital-policy stakeholders who need a clearer view of a bill's practical effects.
|
| 32 |
+
- Small-model constraint: default generation uses `Qwen/Qwen3-14B:cheapest` through the Hugging Face router, staying within the hackathon's `<= 32B` model cap while keeping deployment simple.
|
| 33 |
- Required surface: Gradio app, ready for Hugging Face Spaces through `app.py`.
|
| 34 |
- GitHub repo: https://github.com/KayO-GH/legislation-explainer
|
| 35 |
|
|
|
|
| 48 |
|
| 49 |
## Model And Provider Notes
|
| 50 |
|
| 51 |
+
The hackathon-safe default is Qwen3 14B through the Hugging Face router.
|
| 52 |
|
| 53 |
+
- Default generator: `Qwen/Qwen3-14B:cheapest`
|
| 54 |
+
- Default credential path: `HF_TOKEN`
|
| 55 |
+
- Default provider: `qwen`
|
| 56 |
|
| 57 |
+
Nemotron support remains optional for experimentation through the alternate provider path, but it is not the default app configuration.
|
| 58 |
|
| 59 |
## Local Run
|
| 60 |
|
|
|
|
| 63 |
python app.py
|
| 64 |
```
|
| 65 |
|
| 66 |
+
Set these environment variables for the default Qwen path:
|
| 67 |
|
| 68 |
+
- `HF_TOKEN`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
| 70 |
+
Optional Nemotron environment variables are still supported for non-default experimentation:
|
| 71 |
|
| 72 |
+
- `NEMOTRON_API_KEY`
|
| 73 |
+
- `NEMOTRON_BASE_URL`
|
| 74 |
+
- `NEMOTRON_RETRIEVER_API_KEY`
|
| 75 |
+
- `NEMOTRON_RETRIEVER_BASE_URL`
|
| 76 |
+
- `NEMOTRON_RERANKER_API_KEY`
|
| 77 |
+
- `NEMOTRON_RERANKER_URL`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
## Space Deployment
|
| 80 |
|
config.py
CHANGED
|
@@ -33,7 +33,7 @@ warnings.filterwarnings(
|
|
| 33 |
)
|
| 34 |
|
| 35 |
SUPPORTED_PROVIDERS = ["nemotron", "qwen", "openai", "anthropic", "gemini", "cohere"]
|
| 36 |
-
DEFAULT_PROVIDER: str = "
|
| 37 |
DEFAULT_NEMOTRON_MODEL = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
|
| 38 |
DEFAULT_QWEN_MODEL = "Qwen/Qwen3-14B:cheapest"
|
| 39 |
DEFAULT_CHUNK_TOKENIZER_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
|
|
@@ -105,8 +105,8 @@ PROVIDER_METADATA: list[ProviderConfig] = [
|
|
| 105 |
key_prefix=None,
|
| 106 |
display_name="Nemotron Nano 30B",
|
| 107 |
instructions=(
|
| 108 |
-
"Use NEMOTRON_API_KEY for
|
| 109 |
-
"
|
| 110 |
),
|
| 111 |
),
|
| 112 |
ProviderConfig(
|
|
|
|
| 33 |
)
|
| 34 |
|
| 35 |
SUPPORTED_PROVIDERS = ["nemotron", "qwen", "openai", "anthropic", "gemini", "cohere"]
|
| 36 |
+
DEFAULT_PROVIDER: str = "qwen"
|
| 37 |
DEFAULT_NEMOTRON_MODEL = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
|
| 38 |
DEFAULT_QWEN_MODEL = "Qwen/Qwen3-14B:cheapest"
|
| 39 |
DEFAULT_CHUNK_TOKENIZER_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
|
|
|
|
| 105 |
key_prefix=None,
|
| 106 |
display_name="Nemotron Nano 30B",
|
| 107 |
instructions=(
|
| 108 |
+
"Use NEMOTRON_API_KEY for a self-hosted or external OpenAI-compatible Nemotron endpoint. "
|
| 109 |
+
"This is optional and no longer the default app path."
|
| 110 |
),
|
| 111 |
),
|
| 112 |
ProviderConfig(
|
tests/test_app.py
CHANGED
|
@@ -127,7 +127,7 @@ def test_ask_question_bootstraps_source_document_without_analysis(monkeypatch) -
|
|
| 127 |
assert calls == [("vector-store", "Fresh source text")]
|
| 128 |
assert record["doc_text"] == "Fresh source text"
|
| 129 |
assert record["vector_store"] == "vector-store"
|
| 130 |
-
assert record["api_config"]["provider"] == "
|
| 131 |
assert "<details><summary>Supporting snippet (1)</summary>" in frames[-1][0][-1]["content"]
|
| 132 |
|
| 133 |
|
|
|
|
| 127 |
assert calls == [("vector-store", "Fresh source text")]
|
| 128 |
assert record["doc_text"] == "Fresh source text"
|
| 129 |
assert record["vector_store"] == "vector-store"
|
| 130 |
+
assert record["api_config"]["provider"] == "qwen"
|
| 131 |
assert "<details><summary>Supporting snippet (1)</summary>" in frames[-1][0][-1]["content"]
|
| 132 |
|
| 133 |
|