dev2nd commited on
Commit
1df14c8
ยท
verified ยท
1 Parent(s): 99990a1

Set Docker SDK and app_port

Browse files
Files changed (1) hide show
  1. README.md +43 -7
README.md CHANGED
@@ -1,12 +1,48 @@
1
  ---
2
- title: Quiz Llm Server
3
- emoji: ๐ŸŒ–
4
- colorFrom: indigo
5
- colorTo: purple
6
  sdk: docker
 
 
7
  pinned: false
8
- license: mit
9
- short_description: LLM server for quiz generation benchmark
10
  ---
11
 
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Quiz LLM Server
3
+ emoji: ๐Ÿค–
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: docker
7
+ app_port: 7860
8
+ suggested_hardware: a10g-small
9
  pinned: false
10
+ short_description: OpenAI-compatible vLLM server for quiz generation benchmark
 
11
  ---
12
 
13
+ # Quiz LLM Server (vLLM, OpenAI-compatible)
14
+
15
+ ํ€ด์ฆˆ ์ƒ์„ฑ ์‹œ์Šคํ…œ ์„ฑ๋Šฅ ์‹œํ—˜์šฉ ๋กœ์ปฌ LLM ์„œ๋ฒ„.
16
+ vLLM์˜ OpenAI ํ˜ธํ™˜ ์—”๋“œํฌ์ธํŠธ๋ฅผ ๊ทธ๋Œ€๋กœ ๋…ธ์ถœํ•จ.
17
+
18
+ ## Endpoints
19
+
20
+ - `GET /v1/models`
21
+ - `POST /v1/chat/completions`
22
+ - `POST /v1/completions`
23
+
24
+ ## ํ™˜๊ฒฝ๋ณ€์ˆ˜ (Space โ†’ Settings โ†’ Variables and secrets)
25
+
26
+ | ์ด๋ฆ„ | ์˜ˆ์‹œ ๊ฐ’ | ๋น„๊ณ  |
27
+ |------------------|----------------------------------|----------------------------|
28
+ | `MODEL_ID` | `Qwen/Qwen2.5-7B-Instruct` | HF ๋ชจ๋ธ ID |
29
+ | `MAX_MODEL_LEN` | `8192` | VRAM์— ๋งž์ถฐ ์กฐ์ • |
30
+ | `GPU_MEM_UTIL` | `0.90` | 0.0 ~ 1.0 |
31
+ | `HF_TOKEN` | `hf_xxx` (Secret์œผ๋กœ ๋“ฑ๋ก) | gated ๋ชจ๋ธ ์‚ฌ์šฉ ์‹œ ํ•„์ˆ˜ |
32
+
33
+ ## ์—ฐ๊ฒฐ ์˜ˆ์‹œ
34
+
35
+ ```python
36
+ from openai import OpenAI
37
+
38
+ client = OpenAI(
39
+ base_url="https://dev2nd-quiz-llm-server.hf.space/v1",
40
+ api_key="hf_xxx",
41
+ )
42
+
43
+ resp = client.chat.completions.create(
44
+ model="Qwen/Qwen2.5-7B-Instruct",
45
+ messages=[{"role": "user", "content": "ํ€ด์ฆˆ๋ฅผ ๋งŒ๋“ค์–ด์ค˜"}],
46
+ )
47
+ print(resp.choices[0].message.content)
48
+ ```