Text Generation
Transformers
Safetensors
GGUF
English
qwen3
search
rag
question-answering
on-device
tiny
muon
text-generation-inference
Instructions to use ahkamboh/tiny-search-reader-60M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ahkamboh/tiny-search-reader-60M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ahkamboh/tiny-search-reader-60M")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ahkamboh/tiny-search-reader-60M") model = AutoModelForCausalLM.from_pretrained("ahkamboh/tiny-search-reader-60M", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ahkamboh/tiny-search-reader-60M with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ahkamboh/tiny-search-reader-60M:Q4_K_M # Run inference directly in the terminal: llama cli -hf ahkamboh/tiny-search-reader-60M:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ahkamboh/tiny-search-reader-60M:Q4_K_M # Run inference directly in the terminal: llama cli -hf ahkamboh/tiny-search-reader-60M:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ahkamboh/tiny-search-reader-60M:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf ahkamboh/tiny-search-reader-60M:Q4_K_M
Use Docker
docker model run hf.co/ahkamboh/tiny-search-reader-60M:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use ahkamboh/tiny-search-reader-60M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ahkamboh/tiny-search-reader-60M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ahkamboh/tiny-search-reader-60M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ahkamboh/tiny-search-reader-60M:Q4_K_M
- SGLang
How to use ahkamboh/tiny-search-reader-60M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ahkamboh/tiny-search-reader-60M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ahkamboh/tiny-search-reader-60M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ahkamboh/tiny-search-reader-60M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ahkamboh/tiny-search-reader-60M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use ahkamboh/tiny-search-reader-60M with Ollama:
ollama run hf.co/ahkamboh/tiny-search-reader-60M:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use ahkamboh/tiny-search-reader-60M with Docker Model Runner:
docker model run hf.co/ahkamboh/tiny-search-reader-60M:Q4_K_M
- Lemonade
How to use ahkamboh/tiny-search-reader-60M with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ahkamboh/tiny-search-reader-60M:Q4_K_M
Run and chat with the model
lemonade run user.tiny-search-reader-60M-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download eval/google25.jsonl from ahkamboh/tiny-search-reader-60M: direct link, hf CLI and curl.
- Browser
- Download file 5.16 kB
-
https://huggingface.co/ahkamboh/tiny-search-reader-60M/resolve/main/eval/google25.jsonl
- Command line
-
hf download hf://ahkamboh/tiny-search-reader-60M/eval/google25.jsonl
-
curl -L -o google25.jsonl https://huggingface.co/ahkamboh/tiny-search-reader-60M/resolve/main/eval/google25.jsonl
5.16 kB
| {"id": "g25-0", "question": "What is the dollar rate in Pakistan today?", "answers": ["277.08", "277.06", "277.90", "277.50", "275.64", "275.1777", "277"], "kind": "live", "answer": "277.08 PKR", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-1", "question": "What is the gold price in Pakistan today?", "answers": ["437,206", "433,948", "435,643.07", "438,636", "404,540", "400,772", "397,786", "399,339.48"], "kind": "live", "answer": "Rs. 437,206", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-2", "question": "What is the weather in Lahore today?", "answers": ["97°", "97", "35°", "35", "hazy and very warm", "hazy"], "kind": "live", "answer": "97°", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-3", "question": "Who won the last Formula 1 race?", "answers": ["George Russell", "Russell"], "kind": "live", "answer": "not found", "verdict": "missed", "correct": false} | |
| {"id": "g25-4", "question": "What is the Bitcoin price today?", "answers": ["$84,555.09", "$86,033.73", "$85,422.74", "$86,417.7", "$86,742.94", "$86,088.98"], "kind": "live", "answer": "$86,417.7", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-5", "question": "Who is the current prime minister of the UK?", "answers": ["Andy Burnham"], "kind": "live", "answer": "Andy Burnham", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-6", "question": "What is the petrol price in Pakistan today?", "answers": ["Rs 390.66 per litre", "Rs 390.66", "Rs.390.66", "390.66"], "kind": "live", "answer": "Rs 390.66 per litre", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-7", "question": "What is the latest iPhone model?", "answers": ["iPhone 18 Pro", "iPhone 18 Pro Max", "iPhone 18"], "kind": "live", "answer": "iPhone 18", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-8", "question": "Who is the CEO of OpenAI?", "answers": ["Sam Altman"], "kind": "live", "answer": "Sam Altman", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-9", "question": "What was the result of Pakistan cricket team's latest match?", "answers": ["Pakistan beat Bangladesh by six wickets", "won by 6 wickets", "six wickets", "6 wickets", "6 wkts"], "kind": "live", "answer": "Pakistan beat Bangladesh by six wickets", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-10", "question": "What is the capital of Australia?", "answers": ["Canberra"], "kind": "fact", "answer": "Canberra", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-11", "question": "Who wrote Pride and Prejudice?", "answers": ["Jane Austen"], "kind": "fact", "answer": "Jane Austen", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-12", "question": "How tall is K2?", "answers": ["8,611 metres", "8,611 meters", "8,611 m", "8611 m", "28,251 ft", "28,251 feet", "8,611"], "kind": "fact", "answer": "8,611 metres", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-13", "question": "When did Pakistan get independence?", "answers": ["14 August 1947", "August 14, 1947", "14 August"], "kind": "fact", "answer": "14 August 1947", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-14", "question": "What is the speed of light?", "answers": ["299,792,458 meters per second", "299,792,458 m/s", "299792458 meters per", "299,792,458", "186,000 mi/sec", "300,000 kilometers per second", "300,000 km/s"], "kind": "fact", "answer": "299792458 m/s", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-15", "question": "Who founded Microsoft?", "answers": ["Bill Gates and Paul Allen", "Bill Gates", "Paul Allen"], "kind": "fact", "answer": "Bill Gates and Paul Allen", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-16", "question": "What is the boiling point of water in Fahrenheit?", "answers": ["212°F", "212 °F", "212° F", "212", "211.95 °F"], "kind": "fact", "answer": "99.97 °C", "verdict": "wrong", "correct": false} | |
| {"id": "g25-17", "question": "How many players are in a football team?", "answers": ["11", "eleven"], "kind": "fact", "answer": "11 players", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-18", "question": "What is the largest planet in the solar system?", "answers": ["Jupiter"], "kind": "fact", "answer": "Jupiter", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-19", "question": "Who painted the Mona Lisa?", "answers": ["Leonardo da Vinci", "Leonardo"], "kind": "fact", "answer": "Leonardo da Vinci", "verdict": "answered_correct", "correct": true} | |
| {"id": "g25-20", "question": "Who was the first Pakistani to walk on Mars?", "answers": [], "kind": "none", "answer": "not found", "verdict": "trap_caught", "correct": true} | |
| {"id": "g25-21", "question": "When did Lahore host the Summer Olympics?", "answers": [], "kind": "none", "answer": "not found", "verdict": "trap_caught", "correct": true} | |
| {"id": "g25-22", "question": "Who won the 2030 FIFA World Cup?", "answers": [], "kind": "none", "answer": "not found", "verdict": "trap_caught", "correct": true} | |
| {"id": "g25-23", "question": "What is the population of Atlantis?", "answers": [], "kind": "none", "answer": "not found", "verdict": "trap_caught", "correct": true} | |