Spaces:
Paused
Paused
Set Docker SDK and app_port
Browse files
README.md
CHANGED
|
@@ -1,12 +1,48 @@
|
|
| 1 |
---
|
| 2 |
-
title: Quiz
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: docker
|
|
|
|
|
|
|
| 7 |
pinned: false
|
| 8 |
-
|
| 9 |
-
short_description: LLM server for quiz generation benchmark
|
| 10 |
---
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Quiz LLM Server
|
| 3 |
+
emoji: ๐ค
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: green
|
| 6 |
sdk: docker
|
| 7 |
+
app_port: 7860
|
| 8 |
+
suggested_hardware: a10g-small
|
| 9 |
pinned: false
|
| 10 |
+
short_description: OpenAI-compatible vLLM server for quiz generation benchmark
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# Quiz LLM Server (vLLM, OpenAI-compatible)
|
| 14 |
+
|
| 15 |
+
ํด์ฆ ์์ฑ ์์คํ
์ฑ๋ฅ ์ํ์ฉ ๋ก์ปฌ LLM ์๋ฒ.
|
| 16 |
+
vLLM์ OpenAI ํธํ ์๋ํฌ์ธํธ๋ฅผ ๊ทธ๋๋ก ๋
ธ์ถํจ.
|
| 17 |
+
|
| 18 |
+
## Endpoints
|
| 19 |
+
|
| 20 |
+
- `GET /v1/models`
|
| 21 |
+
- `POST /v1/chat/completions`
|
| 22 |
+
- `POST /v1/completions`
|
| 23 |
+
|
| 24 |
+
## ํ๊ฒฝ๋ณ์ (Space โ Settings โ Variables and secrets)
|
| 25 |
+
|
| 26 |
+
| ์ด๋ฆ | ์์ ๊ฐ | ๋น๊ณ |
|
| 27 |
+
|------------------|----------------------------------|----------------------------|
|
| 28 |
+
| `MODEL_ID` | `Qwen/Qwen2.5-7B-Instruct` | HF ๋ชจ๋ธ ID |
|
| 29 |
+
| `MAX_MODEL_LEN` | `8192` | VRAM์ ๋ง์ถฐ ์กฐ์ |
|
| 30 |
+
| `GPU_MEM_UTIL` | `0.90` | 0.0 ~ 1.0 |
|
| 31 |
+
| `HF_TOKEN` | `hf_xxx` (Secret์ผ๋ก ๋ฑ๋ก) | gated ๋ชจ๋ธ ์ฌ์ฉ ์ ํ์ |
|
| 32 |
+
|
| 33 |
+
## ์ฐ๊ฒฐ ์์
|
| 34 |
+
|
| 35 |
+
```python
|
| 36 |
+
from openai import OpenAI
|
| 37 |
+
|
| 38 |
+
client = OpenAI(
|
| 39 |
+
base_url="https://dev2nd-quiz-llm-server.hf.space/v1",
|
| 40 |
+
api_key="hf_xxx",
|
| 41 |
+
)
|
| 42 |
+
|
| 43 |
+
resp = client.chat.completions.create(
|
| 44 |
+
model="Qwen/Qwen2.5-7B-Instruct",
|
| 45 |
+
messages=[{"role": "user", "content": "ํด์ฆ๋ฅผ ๋ง๋ค์ด์ค"}],
|
| 46 |
+
)
|
| 47 |
+
print(resp.choices[0].message.content)
|
| 48 |
+
```
|