Instructions to use youssefedweqd/working with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use youssefedweqd/working with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="youssefedweqd/working")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("youssefedweqd/working", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use youssefedweqd/working with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "youssefedweqd/working"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "youssefedweqd/working",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/youssefedweqd/working

SGLang

How to use youssefedweqd/working with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "youssefedweqd/working" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "youssefedweqd/working",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "youssefedweqd/working" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "youssefedweqd/working",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use youssefedweqd/working with Docker Model Runner:
```
docker model run hf.co/youssefedweqd/working
```

working / Model /LLaMA-Factory /docker /docker-cuda /Dockerfile.base

youssefedweqd

Training in progress, step 1000

cd63278 verified about 1 year ago

Raw

History Blame Contribute Delete

2.27 kB

	# Start from the pytorch official image (ubuntu-22.04 + cuda-12.4.1 + python-3.11)
	# https://hub.docker.com/r/pytorch/pytorch/tags
	FROM pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel

	# Define environments
	ENV MAX_JOBS=16
	ENV VLLM_WORKER_MULTIPROC_METHOD=spawn
	ENV DEBIAN_FRONTEND=noninteractive
	ENV NODE_OPTIONS=""
	ENV PIP_ROOT_USER_ACTION=ignore

	# Define installation arguments
	ARG APT_SOURCE=https://mirrors.tuna.tsinghua.edu.cn/ubuntu/
	ARG PIP_INDEX=https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple

	# Set apt source
	RUN cp /etc/apt/sources.list /etc/apt/sources.list.bak && \
	{ \
	echo "deb ${APT_SOURCE} jammy main restricted universe multiverse"; \
	echo "deb ${APT_SOURCE} jammy-updates main restricted universe multiverse"; \
	echo "deb ${APT_SOURCE} jammy-backports main restricted universe multiverse"; \
	echo "deb ${APT_SOURCE} jammy-security main restricted universe multiverse"; \
	} > /etc/apt/sources.list

	# Install systemctl and wget
	RUN apt-get update && \
	apt-get install -y -o Dpkg::Options::="--force-confdef" systemd wget && \
	apt-get clean

	# Install git and vim
	RUN apt-get update && \
	apt-get install -y git vim && \
	apt-get clean

	# Install gcc and g++
	RUN apt-get update && \
	apt-get install -y gcc g++ && \
	apt-get clean

	# Change pip source
	RUN pip config set global.index-url "${PIP_INDEX}" && \
	pip config set global.extra-index-url "${PIP_INDEX}" && \
	pip install --no-cache-dir --upgrade pip packaging wheel setuptools

	# Install flash-attn-2.7.4.post1 (cxx11abi=False)
	RUN wget -nv https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.6cxx11abiFALSE-cp311-cp311-linux_x86_64.whl && \
	pip install --no-cache-dir flash_attn-2.7.4.post1+cu12torch2.6cxx11abiFALSE-cp311-cp311-linux_x86_64.whl

	# Install flashinfer-0.2.2.post1+cu124 (cxx11abi=False)
	RUN wget -nv https://github.com/flashinfer-ai/flashinfer/releases/download/v0.2.2.post1/flashinfer_python-0.2.2.post1+cu124torch2.6-cp38-abi3-linux_x86_64.whl && \
	pip install --no-cache-dir flashinfer_python-0.2.2.post1+cu124torch2.6-cp38-abi3-linux_x86_64.whl

	# Reset pip config
	RUN pip config unset global.index-url && \
	pip config unset global.extra-index-url