Text Generation
Transformers
Safetensors
GGUF
English
qwen3.5
gated-deltanet
hybrid-attention
dense
tinystories
tiny-model
conversational
Instructions to use shibatch/tinyqwen35dense2m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tinyqwen35dense2m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="shibatch/tinyqwen35dense2m")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tinyqwen35dense2m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use shibatch/tinyqwen35dense2m with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf shibatch/tinyqwen35dense2m:Q4_K_M # Run inference directly in the terminal: llama cli -hf shibatch/tinyqwen35dense2m:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf shibatch/tinyqwen35dense2m:Q4_K_M # Run inference directly in the terminal: llama cli -hf shibatch/tinyqwen35dense2m:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf shibatch/tinyqwen35dense2m:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf shibatch/tinyqwen35dense2m:Q4_K_M
Use Docker
docker model run hf.co/shibatch/tinyqwen35dense2m:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use shibatch/tinyqwen35dense2m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "shibatch/tinyqwen35dense2m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinyqwen35dense2m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/shibatch/tinyqwen35dense2m:Q4_K_M
- SGLang
How to use shibatch/tinyqwen35dense2m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "shibatch/tinyqwen35dense2m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinyqwen35dense2m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "shibatch/tinyqwen35dense2m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinyqwen35dense2m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use shibatch/tinyqwen35dense2m with Ollama:
ollama run hf.co/shibatch/tinyqwen35dense2m:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use shibatch/tinyqwen35dense2m with Docker Model Runner:
docker model run hf.co/shibatch/tinyqwen35dense2m:Q4_K_M
- Lemonade
How to use shibatch/tinyqwen35dense2m with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull shibatch/tinyqwen35dense2m:Q4_K_M
Run and chat with the model
lemonade run user.tinyqwen35dense2m-Q4_K_M
List all available models
lemonade list
- Atomic Chat
File size: 1,099 Bytes
0bd3079 | 1 2 3 4 5 | {
"Once upon": "Once upon a time, there was a mighty man who lived on a farm. He had a big red ball that he used to kick the ball very far and it could not stay in the cave. One day, the man went outside and saw a big pit. The man wanted to show his friends how pretty it looked.\n\nThe man thought of a big, green field to come down. He told the man about the dog and the mighty",
"There was a little": "There was a little girl named Lily. She loved to play in the woods near her home. One day, she found a big leaf on the ground. She picked it up and held it close.\n\nLily picked up the leaf and showed it to his mom. \"Look what I found!\" she said. \"It's so pretty! Can we play with it?\"\n\nHer mom smiled and said, \"Yes, let's keep",
"One day": "One day, he was playing with his toys and running around when he saw a big, wide hole in the ground. He wanted to see what was inside the hole but he was scared.\n\nHe walked closer and noticed that it was an old piece of thread. He stepped over to the hole and shut it. Suddenly, something wobbled. It was a strange and dark shadow!\n"
} |