File size: 1,431 Bytes
6e6d3f0
1df14c8
 
 
 
6e6d3f0
1df14c8
 
6e6d3f0
1df14c8
6e6d3f0
 
1df14c8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
---
title: Quiz LLM Server
emoji: ๐Ÿค–
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
suggested_hardware: a10g-small
pinned: false
short_description: OpenAI-compatible vLLM server for quiz generation benchmark
---

# Quiz LLM Server (vLLM, OpenAI-compatible)

ํ€ด์ฆˆ ์ƒ์„ฑ ์‹œ์Šคํ…œ ์„ฑ๋Šฅ ์‹œํ—˜์šฉ ๋กœ์ปฌ LLM ์„œ๋ฒ„.
vLLM์˜ OpenAI ํ˜ธํ™˜ ์—”๋“œํฌ์ธํŠธ๋ฅผ ๊ทธ๋Œ€๋กœ ๋…ธ์ถœํ•จ.

## Endpoints

- `GET  /v1/models`
- `POST /v1/chat/completions`
- `POST /v1/completions`

## ํ™˜๊ฒฝ๋ณ€์ˆ˜ (Space โ†’ Settings โ†’ Variables and secrets)

| ์ด๋ฆ„             | ์˜ˆ์‹œ ๊ฐ’                          | ๋น„๊ณ                        |
|------------------|----------------------------------|----------------------------|
| `MODEL_ID`       | `Qwen/Qwen2.5-7B-Instruct`       | HF ๋ชจ๋ธ ID                 |
| `MAX_MODEL_LEN`  | `8192`                           | VRAM์— ๋งž์ถฐ ์กฐ์ •           |
| `GPU_MEM_UTIL`   | `0.90`                           | 0.0 ~ 1.0                  |
| `HF_TOKEN`       | `hf_xxx` (Secret์œผ๋กœ ๋“ฑ๋ก)       | gated ๋ชจ๋ธ ์‚ฌ์šฉ ์‹œ ํ•„์ˆ˜    |

## ์—ฐ๊ฒฐ ์˜ˆ์‹œ

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://dev2nd-quiz-llm-server.hf.space/v1",
    api_key="hf_xxx",
)

resp = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role": "user", "content": "ํ€ด์ฆˆ๋ฅผ ๋งŒ๋“ค์–ด์ค˜"}],
)
print(resp.choices[0].message.content)
```