--- title: Quiz LLM Server emoji: ๐Ÿค– colorFrom: blue colorTo: green sdk: docker app_port: 7860 suggested_hardware: a10g-small pinned: false short_description: OpenAI-compatible vLLM server for quiz generation benchmark --- # Quiz LLM Server (vLLM, OpenAI-compatible) ํ€ด์ฆˆ ์ƒ์„ฑ ์‹œ์Šคํ…œ ์„ฑ๋Šฅ ์‹œํ—˜์šฉ ๋กœ์ปฌ LLM ์„œ๋ฒ„. vLLM์˜ OpenAI ํ˜ธํ™˜ ์—”๋“œํฌ์ธํŠธ๋ฅผ ๊ทธ๋Œ€๋กœ ๋…ธ์ถœํ•จ. ## Endpoints - `GET /v1/models` - `POST /v1/chat/completions` - `POST /v1/completions` ## ํ™˜๊ฒฝ๋ณ€์ˆ˜ (Space โ†’ Settings โ†’ Variables and secrets) | ์ด๋ฆ„ | ์˜ˆ์‹œ ๊ฐ’ | ๋น„๊ณ  | |------------------|----------------------------------|----------------------------| | `MODEL_ID` | `Qwen/Qwen2.5-7B-Instruct` | HF ๋ชจ๋ธ ID | | `MAX_MODEL_LEN` | `8192` | VRAM์— ๋งž์ถฐ ์กฐ์ • | | `GPU_MEM_UTIL` | `0.90` | 0.0 ~ 1.0 | | `HF_TOKEN` | `hf_xxx` (Secret์œผ๋กœ ๋“ฑ๋ก) | gated ๋ชจ๋ธ ์‚ฌ์šฉ ์‹œ ํ•„์ˆ˜ | ## ์—ฐ๊ฒฐ ์˜ˆ์‹œ ```python from openai import OpenAI client = OpenAI( base_url="https://dev2nd-quiz-llm-server.hf.space/v1", api_key="hf_xxx", ) resp = client.chat.completions.create( model="Qwen/Qwen2.5-7B-Instruct", messages=[{"role": "user", "content": "ํ€ด์ฆˆ๋ฅผ ๋งŒ๋“ค์–ด์ค˜"}], ) print(resp.choices[0].message.content) ```