Instructions to use devAnurag/TinyLlama-Finetuned-Mixed-10k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use devAnurag/TinyLlama-Finetuned-Mixed-10k with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf devAnurag/TinyLlama-Finetuned-Mixed-10k:F16 # Run inference directly in the terminal: llama cli -hf devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf devAnurag/TinyLlama-Finetuned-Mixed-10k:F16 # Run inference directly in the terminal: llama cli -hf devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf devAnurag/TinyLlama-Finetuned-Mixed-10k:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
Use Docker
docker model run hf.co/devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use devAnurag/TinyLlama-Finetuned-Mixed-10k with Ollama:
ollama run hf.co/devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
- Unsloth Desktop
- Docker Model Runner
How to use devAnurag/TinyLlama-Finetuned-Mixed-10k with Docker Model Runner:
docker model run hf.co/devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
- Lemonade
How to use devAnurag/TinyLlama-Finetuned-Mixed-10k with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull devAnurag/TinyLlama-Finetuned-Mixed-10k:F16
Run and chat with the model
lemonade run user.TinyLlama-Finetuned-Mixed-10k-F16
List all available models
lemonade list
- Atomic Chat
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
TinyLlama Finetuned - Mixed Dataset 10k
This repository contains:
- Merged full HuggingFace model
- LoRA adapter
- GGUF formats (F16 & Q4_K_M)
- Dataset used: Mixed multilingual dataset 10K
- Architecture: TinyLlama 1.1B Chat v1.0
- Finetuned with LoRA (r=16, alpha=32)
Files
| Type | Path |
|---|---|
| Full HF model | /HF_full_model |
| LoRA Adapter | /LoRA_adapter |
| GGUF F16 | /GGUF/tinyllama_f16.gguf |
| GGUF Q4_K_M | /GGUF/tinyllama_q4_k_m.gguf |
- Downloads last month
- 3
Hardware compatibility
Log In to add your hardware
4-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support