Instructions to use ChapAF/intent-0.1-0.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ChapAF/intent-0.1-0.5b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ChapAF/intent-0.1-0.5b:F16 # Run inference directly in the terminal: llama cli -hf ChapAF/intent-0.1-0.5b:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ChapAF/intent-0.1-0.5b:F16 # Run inference directly in the terminal: llama cli -hf ChapAF/intent-0.1-0.5b:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ChapAF/intent-0.1-0.5b:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf ChapAF/intent-0.1-0.5b:F16
Use Docker
docker model run hf.co/ChapAF/intent-0.1-0.5b:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use ChapAF/intent-0.1-0.5b with Ollama:
ollama run hf.co/ChapAF/intent-0.1-0.5b:F16
- Unsloth Desktop
- Docker Model Runner
How to use ChapAF/intent-0.1-0.5b with Docker Model Runner:
docker model run hf.co/ChapAF/intent-0.1-0.5b:F16
- Lemonade
How to use ChapAF/intent-0.1-0.5b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ChapAF/intent-0.1-0.5b:F16
Run and chat with the model
lemonade run user.intent-0.1-0.5b-F16
List all available models
lemonade list
- Atomic Chat
Download config.json from ChapAF/intent-0.1-0.5b: direct link, hf CLI and curl.
- Browser
- Download file 788 Bytes
-
https://huggingface.co/ChapAF/intent-0.1-0.5b/resolve/main/config.json
- Command line
-
hf download hf://ChapAF/intent-0.1-0.5b/config.json
-
curl -L -o config.json https://huggingface.co/ChapAF/intent-0.1-0.5b/resolve/main/config.json
788 Bytes
| { | |
| "model": { | |
| "vocab_size": 32000, | |
| "sequence_len": 2048, | |
| "n_layer": 24, | |
| "n_head": 9, | |
| "n_kv_head": 9, | |
| "n_embd": 1152, | |
| "mlp_ratio": 4.0, | |
| "rope_base": 10000.0, | |
| "dropout": 0.0, | |
| "use_qk_norm": true, | |
| "use_liger": true, | |
| "attention_backend": "auto" | |
| }, | |
| "tokenizer": "meta-llama/Llama-2-7b-chat-hf", | |
| "target_tokens": 5000000000, | |
| "micro_batch_size": 28, | |
| "gradient_accumulation_steps": 1, | |
| "dtype": "bfloat16", | |
| "compile": true, | |
| "compile_mode": "default", | |
| "zero_optimizer": true, | |
| "verify_replicas": true, | |
| "matrix_lr": 0.005, | |
| "adam_lr": 5e-05, | |
| "weight_decay": 0.1, | |
| "warmup_steps": 500, | |
| "checkpoint_interval_tokens": 1000000000, | |
| "latest_interval_tokens": 500000000, | |
| "log_interval": 10, | |
| "grad_clip": 1.0, | |
| "seed": 42 | |
| } | |