README.md

metadata

title: Llm Proxy
emoji: 🤗
colorFrom: red
colorTo: indigo
sdk: docker
pinned: false
license: mit
short_description: OpenAI-compatible proxy for Hugging Face Inference Providers

LLM proxy

The service exposes an OpenAI-like /v1/chat/completions endpoint and routes calls through Hugging Face Inference Providers.

Configuration

• HF_TOKEN: Hugging Face user token with the Inference Providers permission. HF_API_KEY remains supported for compatibility.
• API_KEYS: comma-separated bearer tokens accepted by this proxy.
• HF_MODELS: optional comma-separated alias=model-id mappings. Default: gpt-oss=openai/gpt-oss-120b:cerebras.
• HF_TIMEOUT_SECONDS: optional provider timeout, default 60.

Do not append a provider suffix unless you intentionally want to pin one. With a plain model ID, the Hugging Face router can select an available provider.

Example .env:

HF_TOKEN=hf_your_token
API_KEYS=local-secret
HF_MODELS=gpt-oss=openai/gpt-oss-120b:cerebras

Run locally with start.bat, then call:

$headers = @{ Authorization = "Bearer local-secret" }
$body = @{ model = "gpt-oss"; messages = @(@{ role = "user"; content = "Привет!" }) } | ConvertTo-Json -Depth 4
Invoke-RestMethod http://127.0.0.1:8000/v1/chat/completions -Method Post -Headers $headers -ContentType "application/json" -Body $body

Free Hugging Face accounts receive only a small monthly inference credit. Requests stop after the credit is exhausted unless additional credits are purchased.