Spaces:
Paused
Paused
|
Download README.md from dev2nd/quiz-llm-server: direct link, hf CLI and curl.
- Browser
- Download file 1.43 kB
-
https://huggingface.co/spaces/dev2nd/quiz-llm-server/resolve/main/README.md
- Command line
-
hf download hf://spaces/dev2nd/quiz-llm-server/README.md
-
curl -L -o README.md https://huggingface.co/spaces/dev2nd/quiz-llm-server/resolve/main/README.md
1.43 kB
| title: Quiz LLM Server | |
| emoji: ๐ค | |
| colorFrom: blue | |
| colorTo: green | |
| sdk: docker | |
| app_port: 7860 | |
| suggested_hardware: a10g-small | |
| pinned: false | |
| short_description: OpenAI-compatible vLLM server for quiz generation benchmark | |
| # Quiz LLM Server (vLLM, OpenAI-compatible) | |
| ํด์ฆ ์์ฑ ์์คํ ์ฑ๋ฅ ์ํ์ฉ ๋ก์ปฌ LLM ์๋ฒ. | |
| vLLM์ OpenAI ํธํ ์๋ํฌ์ธํธ๋ฅผ ๊ทธ๋๋ก ๋ ธ์ถํจ. | |
| ## Endpoints | |
| - `GET /v1/models` | |
| - `POST /v1/chat/completions` | |
| - `POST /v1/completions` | |
| ## ํ๊ฒฝ๋ณ์ (Space โ Settings โ Variables and secrets) | |
| | ์ด๋ฆ | ์์ ๊ฐ | ๋น๊ณ | | |
| |------------------|----------------------------------|----------------------------| | |
| | `MODEL_ID` | `Qwen/Qwen2.5-7B-Instruct` | HF ๋ชจ๋ธ ID | | |
| | `MAX_MODEL_LEN` | `8192` | VRAM์ ๋ง์ถฐ ์กฐ์ | | |
| | `GPU_MEM_UTIL` | `0.90` | 0.0 ~ 1.0 | | |
| | `HF_TOKEN` | `hf_xxx` (Secret์ผ๋ก ๋ฑ๋ก) | gated ๋ชจ๋ธ ์ฌ์ฉ ์ ํ์ | | |
| ## ์ฐ๊ฒฐ ์์ | |
| ```python | |
| from openai import OpenAI | |
| client = OpenAI( | |
| base_url="https://dev2nd-quiz-llm-server.hf.space/v1", | |
| api_key="hf_xxx", | |
| ) | |
| resp = client.chat.completions.create( | |
| model="Qwen/Qwen2.5-7B-Instruct", | |
| messages=[{"role": "user", "content": "ํด์ฆ๋ฅผ ๋ง๋ค์ด์ค"}], | |
| ) | |
| print(resp.choices[0].message.content) | |
| ``` |