Spaces:
Paused
Paused
|
Download README.md from dev2nd/quiz-llm-server: direct link, hf CLI and curl.
- Browser
- Download file 1.43 kB
-
https://huggingface.co/spaces/dev2nd/quiz-llm-server/resolve/main/README.md
- Command line
-
hf download hf://spaces/dev2nd/quiz-llm-server/README.md
-
curl -L -o README.md https://huggingface.co/spaces/dev2nd/quiz-llm-server/resolve/main/README.md
1.43 kB
metadata
title: Quiz LLM Server
emoji: ๐ค
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
suggested_hardware: a10g-small
pinned: false
short_description: OpenAI-compatible vLLM server for quiz generation benchmark
Quiz LLM Server (vLLM, OpenAI-compatible)
ํด์ฆ ์์ฑ ์์คํ ์ฑ๋ฅ ์ํ์ฉ ๋ก์ปฌ LLM ์๋ฒ. vLLM์ OpenAI ํธํ ์๋ํฌ์ธํธ๋ฅผ ๊ทธ๋๋ก ๋ ธ์ถํจ.
Endpoints
GET /v1/modelsPOST /v1/chat/completionsPOST /v1/completions
ํ๊ฒฝ๋ณ์ (Space โ Settings โ Variables and secrets)
| ์ด๋ฆ | ์์ ๊ฐ | ๋น๊ณ |
|---|---|---|
MODEL_ID |
Qwen/Qwen2.5-7B-Instruct |
HF ๋ชจ๋ธ ID |
MAX_MODEL_LEN |
8192 |
VRAM์ ๋ง์ถฐ ์กฐ์ |
GPU_MEM_UTIL |
0.90 |
0.0 ~ 1.0 |
HF_TOKEN |
hf_xxx (Secret์ผ๋ก ๋ฑ๋ก) |
gated ๋ชจ๋ธ ์ฌ์ฉ ์ ํ์ |
์ฐ๊ฒฐ ์์
from openai import OpenAI
client = OpenAI(
base_url="https://dev2nd-quiz-llm-server.hf.space/v1",
api_key="hf_xxx",
)
resp = client.chat.completions.create(
model="Qwen/Qwen2.5-7B-Instruct",
messages=[{"role": "user", "content": "ํด์ฆ๋ฅผ ๋ง๋ค์ด์ค"}],
)
print(resp.choices[0].message.content)