Spaces:
Paused
Paused
File size: 1,431 Bytes
6e6d3f0 1df14c8 6e6d3f0 1df14c8 6e6d3f0 1df14c8 6e6d3f0 1df14c8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | ---
title: Quiz LLM Server
emoji: ๐ค
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
suggested_hardware: a10g-small
pinned: false
short_description: OpenAI-compatible vLLM server for quiz generation benchmark
---
# Quiz LLM Server (vLLM, OpenAI-compatible)
ํด์ฆ ์์ฑ ์์คํ
์ฑ๋ฅ ์ํ์ฉ ๋ก์ปฌ LLM ์๋ฒ.
vLLM์ OpenAI ํธํ ์๋ํฌ์ธํธ๋ฅผ ๊ทธ๋๋ก ๋
ธ์ถํจ.
## Endpoints
- `GET /v1/models`
- `POST /v1/chat/completions`
- `POST /v1/completions`
## ํ๊ฒฝ๋ณ์ (Space โ Settings โ Variables and secrets)
| ์ด๋ฆ | ์์ ๊ฐ | ๋น๊ณ |
|------------------|----------------------------------|----------------------------|
| `MODEL_ID` | `Qwen/Qwen2.5-7B-Instruct` | HF ๋ชจ๋ธ ID |
| `MAX_MODEL_LEN` | `8192` | VRAM์ ๋ง์ถฐ ์กฐ์ |
| `GPU_MEM_UTIL` | `0.90` | 0.0 ~ 1.0 |
| `HF_TOKEN` | `hf_xxx` (Secret์ผ๋ก ๋ฑ๋ก) | gated ๋ชจ๋ธ ์ฌ์ฉ ์ ํ์ |
## ์ฐ๊ฒฐ ์์
```python
from openai import OpenAI
client = OpenAI(
base_url="https://dev2nd-quiz-llm-server.hf.space/v1",
api_key="hf_xxx",
)
resp = client.chat.completions.create(
model="Qwen/Qwen2.5-7B-Instruct",
messages=[{"role": "user", "content": "ํด์ฆ๋ฅผ ๋ง๋ค์ด์ค"}],
)
print(resp.choices[0].message.content)
``` |