Spaces:
Sleeping
Sleeping
Upload folder using huggingface_hub
Browse files- README.md +17 -5
- inference.py +4 -4
README.md
CHANGED
|
@@ -345,22 +345,34 @@ When `VERBOSE_TRACE=1`, inference prints compact boxed traces — prompt size, p
|
|
| 345 |
| `TRACE_BOX_WIDTH` | `76` | Width of the boxed trace output |
|
| 346 |
| `SPINNER_ENABLED` | `0` | CLI spinner while waiting for LLM response |
|
| 347 |
|
| 348 |
-
### Inference test — deployed
|
| 349 |
|
| 350 |
> ### 🧪 Deployed endpoint smoke test
|
| 351 |
>
|
| 352 |
> Team-hosted OpenAI-compatible endpoint for **`inference.py`** / **`client.py`** smoke runs (MissionCtrl server on `ENV_BASE_URL` as usual).
|
| 353 |
>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 354 |
> | Setting | Value |
|
| 355 |
> |---|---|
|
| 356 |
-
> | **`API_BASE_URL`** | `https://
|
| 357 |
-
> | **`MODEL_NAME`** | `
|
| 358 |
>
|
| 359 |
> `inference.py` normalizes dedicated HF URLs to end with **`/v1`** for the OpenAI client when the host matches `*.endpoints.huggingface.cloud`.
|
| 360 |
>
|
| 361 |
> ```bash
|
| 362 |
-
> export API_BASE_URL="https://
|
| 363 |
-
> export MODEL_NAME="
|
| 364 |
> export HF_TOKEN="hf_..." # token must be allowed to infer on this endpoint (scope / org role)
|
| 365 |
> export ENV_BASE_URL="http://localhost:7860"
|
| 366 |
> python inference.py
|
|
|
|
| 345 |
| `TRACE_BOX_WIDTH` | `76` | Width of the boxed trace output |
|
| 346 |
| `SPINNER_ENABLED` | `0` | CLI spinner while waiting for LLM response |
|
| 347 |
|
| 348 |
+
### Inference test — deployed trained model (HF dedicated Inference Endpoint)
|
| 349 |
|
| 350 |
> ### 🧪 Deployed endpoint smoke test
|
| 351 |
>
|
| 352 |
> Team-hosted OpenAI-compatible endpoint for **`inference.py`** / **`client.py`** smoke runs (MissionCtrl server on `ENV_BASE_URL` as usual).
|
| 353 |
>
|
| 354 |
+
> ### 🧭 Inference priority order (`inference.py`)
|
| 355 |
+
>
|
| 356 |
+
> ```text
|
| 357 |
+
> 1) Primary
|
| 358 |
+
> API_BASE_URL = https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud
|
| 359 |
+
> MODEL_NAME = missionctrl-env-wnc
|
| 360 |
+
>
|
| 361 |
+
> 2) Fallback (hosted model)
|
| 362 |
+
> API_BASE_URL = https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud
|
| 363 |
+
> MODEL_NAME = quen
|
| 364 |
+
> ```
|
| 365 |
+
>
|
| 366 |
> | Setting | Value |
|
| 367 |
> |---|---|
|
| 368 |
+
> | **`API_BASE_URL`** | `https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud` |
|
| 369 |
+
> | **`MODEL_NAME`** | `missionctrl-env-wnc` |
|
| 370 |
>
|
| 371 |
> `inference.py` normalizes dedicated HF URLs to end with **`/v1`** for the OpenAI client when the host matches `*.endpoints.huggingface.cloud`.
|
| 372 |
>
|
| 373 |
> ```bash
|
| 374 |
+
> export API_BASE_URL="https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud"
|
| 375 |
+
> export MODEL_NAME="missionctrl-env-wnc"
|
| 376 |
> export HF_TOKEN="hf_..." # token must be allowed to infer on this endpoint (scope / org role)
|
| 377 |
> export ENV_BASE_URL="http://localhost:7860"
|
| 378 |
> python inference.py
|
inference.py
CHANGED
|
@@ -73,15 +73,15 @@ import logging as _logging
|
|
| 73 |
# Non-empty values override `.env`, shell env, and CLI flags (see apply order below).
|
| 74 |
# Leave as "" to use env / CLI only (e.g. empty HF_TOKEN in repo; Space injects at runtime).
|
| 75 |
# =============================================================================
|
| 76 |
-
_INFERENCE_API_BASE_URL: str = "https://
|
| 77 |
-
_INFERENCE_MODEL_NAME: str = "
|
| 78 |
_INFERENCE_HF_TOKEN: str = ""
|
| 79 |
_INFERENCE_ENV_BASE_URL: str = ""
|
| 80 |
|
| 81 |
# Optional failover chain (used when the active provider errors during inference).
|
| 82 |
# Keep tokens empty in-repo; runtime secrets can still arrive from env / Spaces.
|
| 83 |
-
_INFERENCE_FALLBACK_API_BASE_URL: str = "https://
|
| 84 |
-
_INFERENCE_FALLBACK_MODEL_NAME: str = "
|
| 85 |
_INFERENCE_FALLBACK_HF_TOKEN: str = ""
|
| 86 |
|
| 87 |
_INFERENCE_FALLBACK2_API_BASE_URL: str = "https://api.groq.com/openai/v1"
|
|
|
|
| 73 |
# Non-empty values override `.env`, shell env, and CLI flags (see apply order below).
|
| 74 |
# Leave as "" to use env / CLI only (e.g. empty HF_TOKEN in repo; Space injects at runtime).
|
| 75 |
# =============================================================================
|
| 76 |
+
_INFERENCE_API_BASE_URL: str = "https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud"
|
| 77 |
+
_INFERENCE_MODEL_NAME: str = "missionctrl-env-wnc"
|
| 78 |
_INFERENCE_HF_TOKEN: str = ""
|
| 79 |
_INFERENCE_ENV_BASE_URL: str = ""
|
| 80 |
|
| 81 |
# Optional failover chain (used when the active provider errors during inference).
|
| 82 |
# Keep tokens empty in-repo; runtime secrets can still arrive from env / Spaces.
|
| 83 |
+
_INFERENCE_FALLBACK_API_BASE_URL: str = "https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud"
|
| 84 |
+
_INFERENCE_FALLBACK_MODEL_NAME: str = "quen"
|
| 85 |
_INFERENCE_FALLBACK_HF_TOKEN: str = ""
|
| 86 |
|
| 87 |
_INFERENCE_FALLBACK2_API_BASE_URL: str = "https://api.groq.com/openai/v1"
|