KayO Codex commited on
Commit
99bb596
·
1 Parent(s): e67d4d9

Revert app defaults to Qwen

Browse files

Co-authored-by: Codex <codex@openai.com>

Files changed (3) hide show
  1. README.md +16 -53
  2. config.py +3 -3
  3. tests/test_app.py +1 -1
README.md CHANGED
@@ -10,9 +10,7 @@ app_file: app.py
10
  pinned: false
11
  short_description: Simplify complex legislation that affects you!
12
  models:
13
- - nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
14
- - nvidia/llama-nemotron-embed-vl-1b-v2
15
- - nvidia/llama-nemotron-rerank-1b-v2
16
  ---
17
 
18
  # Legislation Explainer
@@ -31,7 +29,7 @@ It is created for the Hugging Face Build Small Hackathon under the `Backyard AI`
31
 
32
  - Track: `Backyard AI`
33
  - Real user: Ghanaian citizens and digital-policy stakeholders who need a clearer view of a bill's practical effects.
34
- - Small-model constraint: default generation uses `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`, with Nemotron retriever and reranker models for grounded QA.
35
  - Required surface: Gradio app, ready for Hugging Face Spaces through `app.py`.
36
  - GitHub repo: https://github.com/KayO-GH/legislation-explainer
37
 
@@ -50,13 +48,13 @@ It is created for the Hugging Face Build Small Hackathon under the `Backyard AI`
50
 
51
  ## Model And Provider Notes
52
 
53
- The default QA stack is now Nemotron-first:
54
 
55
- - Generator: `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`
56
- - Retriever: `nvidia/llama-nemotron-embed-vl-1b-v2`
57
- - Reranker: `nvidia/llama-nemotron-rerank-1b-v2`
58
 
59
- The generator uses an OpenAI-compatible endpoint, so it can run through the Hugging Face router when available or through a Modal-hosted / NIM-style deployment when you want a dedicated path. The retriever also uses an OpenAI-compatible embeddings endpoint. The reranker uses a separate HTTP endpoint so it can be pointed at a hosted Nemotron reranking service.
60
 
61
  ## Local Run
62
 
@@ -65,53 +63,18 @@ pip install -r requirements.txt
65
  python app.py
66
  ```
67
 
68
- Set these environment variables for the default Nemotron stack:
69
 
70
- - `NEMOTRON_API_KEY`
71
- - `NEMOTRON_BASE_URL` optional, defaults to `https://router.huggingface.co/v1`
72
- - `NEMOTRON_RETRIEVER_API_KEY` optional, defaults to `NEMOTRON_API_KEY`
73
- - `NEMOTRON_RETRIEVER_BASE_URL` optional, defaults to `NEMOTRON_BASE_URL`
74
- - `NEMOTRON_RERANKER_API_KEY` optional, defaults to `NEMOTRON_API_KEY`
75
- - `NEMOTRON_RERANKER_URL` optional but recommended for the dedicated reranker path
76
-
77
- `HF_TOKEN` still works as a fallback credential for Hugging Face-routed Nemotron endpoints.
78
-
79
- ## Modal Deployment For Nemotron
80
-
81
- Hugging Face currently lists no hosted Inference Provider for `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`, so the practical path for the generator is to self-host it. This repo includes [modal_nemotron_service.py](/Users/Kwadwo/Documents/PROJECTS/HF-Build-Small/legislation-explainer/modal_nemotron_service.py), a standalone Modal deployment using SGLang's OpenAI-compatible server.
82
-
83
- 1. Install and authenticate Modal locally.
84
- 2. Create a Modal secret named `huggingface-secret` that contains your `HF_TOKEN`.
85
- 3. From [`legislation-explainer/`](/Users/Kwadwo/Documents/PROJECTS/HF-Build-Small/legislation-explainer/), deploy:
86
-
87
- ```bash
88
- pip install modal
89
- modal setup
90
- modal secret create huggingface-secret HF_TOKEN=hf_xxx
91
- modal deploy modal_nemotron_service.py
92
- ```
93
 
94
- After deploy, point the app at the Modal endpoint:
95
 
96
- ```bash
97
- export NEMOTRON_API_KEY=modal-placeholder-key
98
- export NEMOTRON_BASE_URL=https://YOUR_MODAL_URL/v1
99
- ```
100
-
101
- Why the placeholder key works: the app's Nemotron client only needs an OpenAI-compatible key value to satisfy the SDK. The Modal service itself authenticates to Hugging Face through the Modal secret, not through the runtime app key.
102
-
103
- Recommended first use:
104
-
105
- - Keep the generator on Modal via `NEMOTRON_BASE_URL`.
106
- - Leave the retriever and reranker on their own hosted endpoints if you have them.
107
- - If you do not yet have hosted retriever or reranker coverage, the app will fall back to local MiniLM embeddings and skip hosted reranking.
108
-
109
- Useful deployment knobs:
110
-
111
- - `NEMOTRON_MODAL_GPU` defaults to `H100`
112
- - `NEMOTRON_MODAL_MIN_CONTAINERS` defaults to `0`
113
- - `NEMOTRON_MODAL_CONTEXT_LENGTH` defaults to `32768`
114
- - `NEMOTRON_MODAL_TARGET_INPUTS` defaults to `8`
115
 
116
  ## Space Deployment
117
 
 
10
  pinned: false
11
  short_description: Simplify complex legislation that affects you!
12
  models:
13
+ - Qwen/Qwen3-14B
 
 
14
  ---
15
 
16
  # Legislation Explainer
 
29
 
30
  - Track: `Backyard AI`
31
  - Real user: Ghanaian citizens and digital-policy stakeholders who need a clearer view of a bill's practical effects.
32
+ - Small-model constraint: default generation uses `Qwen/Qwen3-14B:cheapest` through the Hugging Face router, staying within the hackathon's `<= 32B` model cap while keeping deployment simple.
33
  - Required surface: Gradio app, ready for Hugging Face Spaces through `app.py`.
34
  - GitHub repo: https://github.com/KayO-GH/legislation-explainer
35
 
 
48
 
49
  ## Model And Provider Notes
50
 
51
+ The hackathon-safe default is Qwen3 14B through the Hugging Face router.
52
 
53
+ - Default generator: `Qwen/Qwen3-14B:cheapest`
54
+ - Default credential path: `HF_TOKEN`
55
+ - Default provider: `qwen`
56
 
57
+ Nemotron support remains optional for experimentation through the alternate provider path, but it is not the default app configuration.
58
 
59
  ## Local Run
60
 
 
63
  python app.py
64
  ```
65
 
66
+ Set these environment variables for the default Qwen path:
67
 
68
+ - `HF_TOKEN`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
 
70
+ Optional Nemotron environment variables are still supported for non-default experimentation:
71
 
72
+ - `NEMOTRON_API_KEY`
73
+ - `NEMOTRON_BASE_URL`
74
+ - `NEMOTRON_RETRIEVER_API_KEY`
75
+ - `NEMOTRON_RETRIEVER_BASE_URL`
76
+ - `NEMOTRON_RERANKER_API_KEY`
77
+ - `NEMOTRON_RERANKER_URL`
 
 
 
 
 
 
 
 
 
 
 
 
 
78
 
79
  ## Space Deployment
80
 
config.py CHANGED
@@ -33,7 +33,7 @@ warnings.filterwarnings(
33
  )
34
 
35
  SUPPORTED_PROVIDERS = ["nemotron", "qwen", "openai", "anthropic", "gemini", "cohere"]
36
- DEFAULT_PROVIDER: str = "nemotron"
37
  DEFAULT_NEMOTRON_MODEL = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
38
  DEFAULT_QWEN_MODEL = "Qwen/Qwen3-14B:cheapest"
39
  DEFAULT_CHUNK_TOKENIZER_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
@@ -105,8 +105,8 @@ PROVIDER_METADATA: list[ProviderConfig] = [
105
  key_prefix=None,
106
  display_name="Nemotron Nano 30B",
107
  instructions=(
108
- "Use NEMOTRON_API_KEY for the generator and optionally NEMOTRON_RETRIEVER_* / "
109
- "NEMOTRON_RERANKER_* settings for hosted Nemotron retrieval and reranking."
110
  ),
111
  ),
112
  ProviderConfig(
 
33
  )
34
 
35
  SUPPORTED_PROVIDERS = ["nemotron", "qwen", "openai", "anthropic", "gemini", "cohere"]
36
+ DEFAULT_PROVIDER: str = "qwen"
37
  DEFAULT_NEMOTRON_MODEL = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
38
  DEFAULT_QWEN_MODEL = "Qwen/Qwen3-14B:cheapest"
39
  DEFAULT_CHUNK_TOKENIZER_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
 
105
  key_prefix=None,
106
  display_name="Nemotron Nano 30B",
107
  instructions=(
108
+ "Use NEMOTRON_API_KEY for a self-hosted or external OpenAI-compatible Nemotron endpoint. "
109
+ "This is optional and no longer the default app path."
110
  ),
111
  ),
112
  ProviderConfig(
tests/test_app.py CHANGED
@@ -127,7 +127,7 @@ def test_ask_question_bootstraps_source_document_without_analysis(monkeypatch) -
127
  assert calls == [("vector-store", "Fresh source text")]
128
  assert record["doc_text"] == "Fresh source text"
129
  assert record["vector_store"] == "vector-store"
130
- assert record["api_config"]["provider"] == "nemotron"
131
  assert "<details><summary>Supporting snippet (1)</summary>" in frames[-1][0][-1]["content"]
132
 
133
 
 
127
  assert calls == [("vector-store", "Fresh source text")]
128
  assert record["doc_text"] == "Fresh source text"
129
  assert record["vector_store"] == "vector-store"
130
+ assert record["api_config"]["provider"] == "qwen"
131
  assert "<details><summary>Supporting snippet (1)</summary>" in frames[-1][0][-1]["content"]
132
 
133