Instructions to use HelloSun/Experiment_1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use HelloSun/Experiment_1 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf HelloSun/Experiment_1:Q4_K_M # Run inference directly in the terminal: llama cli -hf HelloSun/Experiment_1:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf HelloSun/Experiment_1:Q4_K_M # Run inference directly in the terminal: llama cli -hf HelloSun/Experiment_1:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf HelloSun/Experiment_1:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf HelloSun/Experiment_1:Q4_K_M
Use Docker
docker model run hf.co/HelloSun/Experiment_1:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use HelloSun/Experiment_1 with Ollama:
ollama run hf.co/HelloSun/Experiment_1:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use HelloSun/Experiment_1 with Docker Model Runner:
docker model run hf.co/HelloSun/Experiment_1:Q4_K_M
- Lemonade
How to use HelloSun/Experiment_1 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull HelloSun/Experiment_1:Q4_K_M
Run and chat with the model
lemonade run user.Experiment_1-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download onednn/results.csv from HelloSun/Experiment_1: direct link, hf CLI and curl.
- Browser
- Download file 1.47 kB
-
https://huggingface.co/HelloSun/Experiment_1/resolve/main/onednn/results.csv
- Command line
-
hf download hf://HelloSun/Experiment_1/onednn/results.csv
-
curl -L -o results.csv https://huggingface.co/HelloSun/Experiment_1/resolve/main/onednn/results.csv
1.47 kB
| backend,weights,pp512_tok_s,tg128_tok_s,ppl512,argmax_agree,threads,note | |
| llama.cpp(plain AVX512),Q4_K_M,180.50,13.39,NA,NA,16,"llama-bench -r 5; GGML_NATIVE=OFF, AMX=OFF, CPU_REPACK=OFF" | |
| llama.cpp(default AMX+repack),Q4_K_M,112.77,12.19,NA,NA,16,"llama-bench -r 5; native build" | |
| oneDNN,bf16,75.52,4.79,NA,NA,1,"llama-bench -r 5; -t 1 + OMP_NUM_THREADS=16; oneDNN AMX bf16 w" | |
| oneDNN,int8,90.09,7.67,NA,NA,1,"llama-bench -r 5; -t 1 + OMP_NUM_THREADS=16; oneDNN AMX int8 (s8/col)" | |
| oneDNN,int4,NA,NA,NA,NA,1,"no AMX int4 kernel; oneDNN 3.7 uses ref_int8 (~961 ms per 4096x4096 matmul at M=1)" | |
| OpenVINO,f32,125.60,2.22,28.8218,1.0000,16,"IR from HF fp32; INFERENCE_PRECISION_HINT=f32; tg=median of 128 steps" | |
| OpenVINO,bf16-hint,209.43,3.93,28.8452,0.9746,16,"IR from HF fp32; hint=bf16 (weights stay f32)" | |
| OpenVINO,int8-asym,219.17,4.70,29.1091,0.9512,16,"NNCF weight-only INT8_ASYM on IR; hint=f32" | |
| OpenVINO,int8-sym,226.68,NA,29.3413,0.9414,16,"NNCF weight-only INT8_SYM on IR; hint=f32" | |
| OpenVINO,int4-asym,223.65,5.20,32.6830,0.7031,16,"NNCF weight-only INT4_ASYM group=128; hint=f32" | |
| OpenVINO,int4-sym,222.48,NA,42.9488,0.7012,16,"NNCF weight-only INT4_SYM group=128; hint=f32" | |
| OpenVINO-GenAI,Q4_K_M,NA,NA,NA,NA,16,"LLMPipeline 2026.4.1 load fails: gguf_tensor_to_f16 (54 Q5_0 tensors unsupported); arch openpangu-r unsupported" | |
| OpenVINO-GenAI,f32-prefill-IR,NA,NA,NA,NA,16,"manual prefill IR loads; generate fails: Port attention_mask not found (needs stateful KV model)" | |