Jit-fnc commited on
Commit
d7cf10c
·
verified ·
1 Parent(s): 4069f26

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +17 -5
  2. inference.py +4 -4
README.md CHANGED
@@ -345,22 +345,34 @@ When `VERBOSE_TRACE=1`, inference prints compact boxed traces — prompt size, p
345
  | `TRACE_BOX_WIDTH` | `76` | Width of the boxed trace output |
346
  | `SPINNER_ENABLED` | `0` | CLI spinner while waiting for LLM response |
347
 
348
- ### Inference test — deployed Qwen (HF dedicated Inference Endpoint)
349
 
350
  > ### 🧪 Deployed endpoint smoke test
351
  >
352
  > Team-hosted OpenAI-compatible endpoint for **`inference.py`** / **`client.py`** smoke runs (MissionCtrl server on `ENV_BASE_URL` as usual).
353
  >
 
 
 
 
 
 
 
 
 
 
 
 
354
  > | Setting | Value |
355
  > |---|---|
356
- > | **`API_BASE_URL`** | `https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud` |
357
- > | **`MODEL_NAME`** | `quen` |
358
  >
359
  > `inference.py` normalizes dedicated HF URLs to end with **`/v1`** for the OpenAI client when the host matches `*.endpoints.huggingface.cloud`.
360
  >
361
  > ```bash
362
- > export API_BASE_URL="https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud"
363
- > export MODEL_NAME="quen"
364
  > export HF_TOKEN="hf_..." # token must be allowed to infer on this endpoint (scope / org role)
365
  > export ENV_BASE_URL="http://localhost:7860"
366
  > python inference.py
 
345
  | `TRACE_BOX_WIDTH` | `76` | Width of the boxed trace output |
346
  | `SPINNER_ENABLED` | `0` | CLI spinner while waiting for LLM response |
347
 
348
+ ### Inference test — deployed trained model (HF dedicated Inference Endpoint)
349
 
350
  > ### 🧪 Deployed endpoint smoke test
351
  >
352
  > Team-hosted OpenAI-compatible endpoint for **`inference.py`** / **`client.py`** smoke runs (MissionCtrl server on `ENV_BASE_URL` as usual).
353
  >
354
+ > ### 🧭 Inference priority order (`inference.py`)
355
+ >
356
+ > ```text
357
+ > 1) Primary
358
+ > API_BASE_URL = https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud
359
+ > MODEL_NAME = missionctrl-env-wnc
360
+ >
361
+ > 2) Fallback (hosted model)
362
+ > API_BASE_URL = https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud
363
+ > MODEL_NAME = quen
364
+ > ```
365
+ >
366
  > | Setting | Value |
367
  > |---|---|
368
+ > | **`API_BASE_URL`** | `https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud` |
369
+ > | **`MODEL_NAME`** | `missionctrl-env-wnc` |
370
  >
371
  > `inference.py` normalizes dedicated HF URLs to end with **`/v1`** for the OpenAI client when the host matches `*.endpoints.huggingface.cloud`.
372
  >
373
  > ```bash
374
+ > export API_BASE_URL="https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud"
375
+ > export MODEL_NAME="missionctrl-env-wnc"
376
  > export HF_TOKEN="hf_..." # token must be allowed to infer on this endpoint (scope / org role)
377
  > export ENV_BASE_URL="http://localhost:7860"
378
  > python inference.py
inference.py CHANGED
@@ -73,15 +73,15 @@ import logging as _logging
73
  # Non-empty values override `.env`, shell env, and CLI flags (see apply order below).
74
  # Leave as "" to use env / CLI only (e.g. empty HF_TOKEN in repo; Space injects at runtime).
75
  # =============================================================================
76
- _INFERENCE_API_BASE_URL: str = "https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud"
77
- _INFERENCE_MODEL_NAME: str = "qwen"
78
  _INFERENCE_HF_TOKEN: str = ""
79
  _INFERENCE_ENV_BASE_URL: str = ""
80
 
81
  # Optional failover chain (used when the active provider errors during inference).
82
  # Keep tokens empty in-repo; runtime secrets can still arrive from env / Spaces.
83
- _INFERENCE_FALLBACK_API_BASE_URL: str = "https://router.huggingface.co/v1"
84
- _INFERENCE_FALLBACK_MODEL_NAME: str = "meta-llama/Llama-3.1-8B-Instruct"
85
  _INFERENCE_FALLBACK_HF_TOKEN: str = ""
86
 
87
  _INFERENCE_FALLBACK2_API_BASE_URL: str = "https://api.groq.com/openai/v1"
 
73
  # Non-empty values override `.env`, shell env, and CLI flags (see apply order below).
74
  # Leave as "" to use env / CLI only (e.g. empty HF_TOKEN in repo; Space injects at runtime).
75
  # =============================================================================
76
+ _INFERENCE_API_BASE_URL: str = "https://pfxsogmap6do11nx.us-east4.gcp.endpoints.huggingface.cloud"
77
+ _INFERENCE_MODEL_NAME: str = "missionctrl-env-wnc"
78
  _INFERENCE_HF_TOKEN: str = ""
79
  _INFERENCE_ENV_BASE_URL: str = ""
80
 
81
  # Optional failover chain (used when the active provider errors during inference).
82
  # Keep tokens empty in-repo; runtime secrets can still arrive from env / Spaces.
83
+ _INFERENCE_FALLBACK_API_BASE_URL: str = "https://xfb9waxafjtm3p05.us-east4.gcp.endpoints.huggingface.cloud"
84
+ _INFERENCE_FALLBACK_MODEL_NAME: str = "quen"
85
  _INFERENCE_FALLBACK_HF_TOKEN: str = ""
86
 
87
  _INFERENCE_FALLBACK2_API_BASE_URL: str = "https://api.groq.com/openai/v1"