Instructions to use Hansda/ScreenVLM-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Hansda/ScreenVLM-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Hansda/ScreenVLM-GGUF:F16 # Run inference directly in the terminal: llama cli -hf Hansda/ScreenVLM-GGUF:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Hansda/ScreenVLM-GGUF:F16 # Run inference directly in the terminal: llama cli -hf Hansda/ScreenVLM-GGUF:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Hansda/ScreenVLM-GGUF:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf Hansda/ScreenVLM-GGUF:F16
Use Docker
docker model run hf.co/Hansda/ScreenVLM-GGUF:F16
- Llama new
- LM Studio
- Jan
- vLLM
How to use Hansda/ScreenVLM-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Hansda/ScreenVLM-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Hansda/ScreenVLM-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Hansda/ScreenVLM-GGUF:F16
- Ollama
How to use Hansda/ScreenVLM-GGUF with Ollama:
ollama run hf.co/Hansda/ScreenVLM-GGUF:F16
- Unsloth Desktop
- Docker Model Runner
How to use Hansda/ScreenVLM-GGUF with Docker Model Runner:
docker model run hf.co/Hansda/ScreenVLM-GGUF:F16
- Lemonade
How to use Hansda/ScreenVLM-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Hansda/ScreenVLM-GGUF:F16
Run and chat with the model
lemonade run user.ScreenVLM-GGUF-F16
List all available models
lemonade list
- Atomic Chat
File size: 1,461 Bytes
7c455c9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | {
"repository": "https://github.com/ggml-org/llama.cpp",
"revision": "0ec4027a2a0183de87c007069adddc06e665f7de",
"source_tree_clean": true,
"build": {
"type": "Release",
"cuda": true,
"cuda_toolkit": "13.0.88",
"cuda_arch": "120a",
"native_cpu": true,
"build_parallelism": 4
},
"python": {
"torch": "2.14.0+cu130",
"transformers": "5.17.0",
"huggingface_hub": "1.32.0",
"safetensors": "0.8.0",
"tokenizers": "0.23.2",
"numpy": "2.5.3",
"Pillow": "12.3.0",
"gguf": "0.19.0",
"sentencepiece": "0.2.2"
},
"tools": {
"llama-server": {
"sha256": "12255637a664054701e296e3b9a0cdfeda4d62380c9c7af598f4408e9f6c21bd",
"bytes": 17888
},
"llama-quantize": {
"sha256": "3048c984ce69c78e4898f94fca85949384806114a02028ef03e175c1ced54e4d",
"bytes": 17888
},
"llama-cli": {
"sha256": "26ade31d637ae6ac69d398a38dd021d5fc5a9a23aced091f8e4a1d21e4d3358a",
"bytes": 1474576
},
"llama-mtmd-cli": {
"sha256": "2d607d2a8d5a42d949340789d3bdf82295aae3010621beec7864842e77662bce",
"bytes": 106520
}
},
"model_notes": {
"special_output_flag": "--special required to retain location prefix control token",
"processor": "IDEFICS3: longest edge 2048, 512px tiles plus overview, 4x pixel shuffle, mean/std 0.5, Lanczos. Do not add Qwen-specific image-token limits.",
"tested_context": 4096,
"tested_threads": 4
}
}
|