Instructions to use HelloSun/lingtiny30vq3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use HelloSun/lingtiny30vq3 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf HelloSun/lingtiny30vq3 # Run inference directly in the terminal: llama cli -hf HelloSun/lingtiny30vq3
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf HelloSun/lingtiny30vq3 # Run inference directly in the terminal: llama cli -hf HelloSun/lingtiny30vq3
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf HelloSun/lingtiny30vq3 # Run inference directly in the terminal: ./build/bin/llama cli -hf HelloSun/lingtiny30vq3
Use Docker
docker model run hf.co/HelloSun/lingtiny30vq3
- Llama new
- LM Studio
- Jan
- Ollama
How to use HelloSun/lingtiny30vq3 with Ollama:
ollama run hf.co/HelloSun/lingtiny30vq3
- Unsloth Desktop
- Pi
How to use HelloSun/lingtiny30vq3 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf HelloSun/lingtiny30vq3
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "HelloSun/lingtiny30vq3" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use HelloSun/lingtiny30vq3 with Docker Model Runner:
docker model run hf.co/HelloSun/lingtiny30vq3
- Lemonade
How to use HelloSun/lingtiny30vq3 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull HelloSun/lingtiny30vq3
Run and chat with the model
lemonade run user.lingtiny30vq3-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use HelloSun/lingtiny30vq3 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf HelloSun/lingtiny30vq3
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default HelloSun/lingtiny30vq3
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use HelloSun/lingtiny30vq3 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf HelloSun/lingtiny30vq3
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "HelloSun/lingtiny30vq3" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 1,785 Bytes
511ba50 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
0.00.350.172 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.01.333.570 I cmn init: llama threadpool init, n_threads = 64
0.01.410.456 I
0.01.410.625 I system_info: n_threads = 64 (n_threads_batch = 64) / 80 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | AMX_INT8 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.01.410.641 I perplexity: tokenizing the input ..
0.02.026.275 I perplexity: tokenization took 615.627 ms
0.02.026.496 I perplexity: calculating perplexity over 64 chunks, n_ctx=512, batch_size=2048, n_seq=4
0.07.853.203 I perplexity: 5.83 seconds per pass - ETA 1.55 minutes
[1]8.1805,[2]12.6373,[3]11.8638,[4]12.3617,[5]12.6663,[6]13.1201,[7]13.8095,[8]14.2267,[9]14.9397,[10]15.5506,[11]15.7619,[12]16.0234,[13]16.0860,[14]15.7798,[15]15.4154,[16]15.8763,[17]14.5611,[18]14.6866,[19]14.8310,[20]14.5036,[21]14.3380,[22]14.1237,[23]13.6626,[24]13.2579,[25]12.8837,[26]12.7161,[27]12.0963,[28]11.9504,[29]11.9826,[30]11.7738,[31]11.9129,[32]11.9238,[33]12.0048,[34]12.2665,[35]12.3705,[36]12.5638,[37]12.6779,[38]12.8489,[39]12.7929,[40]12.7321,[41]12.7124,[42]12.6202,[43]12.7620,[44]12.7058,[45]12.6333,[46]12.8092,[47]12.7222,[48]12.9243,[49]12.9285,[50]12.9425,[51]13.0944,[52]13.1409,[53]13.1278,[54]13.1657,[55]13.2771,[56]13.3026,[57]13.3324,[58]13.3268,[59]13.4020,[60]13.4787,[61]13.5656,[62]13.6147,[63]13.6801,[64]13.8127,
1.21.268.982 I Final estimate: PPL = 13.8127 +/- 0.34818
|