How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf luweigen/eurollm-1.7b-instruct-android:Q4_0# Run inference directly in the terminal:
llama cli -hf luweigen/eurollm-1.7b-instruct-android:Q4_0Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf luweigen/eurollm-1.7b-instruct-android:Q4_0# Run inference directly in the terminal:
./build/bin/llama cli -hf luweigen/eurollm-1.7b-instruct-android:Q4_0Use Docker
docker model run hf.co/luweigen/eurollm-1.7b-instruct-android:Q4_0Quick Links
EuroLLM-1.7B-Instruct — GGUF for Android
Source
- Original model: utter-project/EuroLLM-1.7B-Instruct (Apache-2.0), 35 languages.
- Converted with llama.cpp b11700 (
convert_hf_to_gguf.pyto f16, thenllama-quantize). - This repository is used by the native Android app of the
vibevoice-androidproject for sentence-by-sentence translation of speech recognition output. The app downloads the files and checks each one against a sha256 value compiled into the APK. See the original model card for intended use, limitations and the license terms.
Files
| File | What |
|---|---|
eurollm-q8_0.gguf |
Q8_0 (1.76 GB). On FLEURS (100 sentences, greedy) it matches f16 within noise |
eurollm-q4_0.gguf |
Q4_0 (1.0 GB). Noticeably weaker when Finnish or Chinese is the target (en→fi −5, fi→zh −8 BLEU vs f16) |
No other changes; the tokenizer and chat template are the original ones.
Where the files apply
- llama.cpp (CPU, OpenCL, Hexagon backends). The app uses Q8_0 on phones with ≥ 12 GB of RAM and Q4_0 otherwise.
- Use greedy decoding with the model card's ChatML prompt; sampling halves the BLEU score in our tests.
Usage and code
To be added.
- Downloads last month
- 64
Hardware compatibility
Log In to add your hardware
4-bit
8-bit
Model tree for luweigen/eurollm-1.7b-instruct-android
Base model
utter-project/EuroLLM-1.7B Finetuned
utter-project/EuroLLM-1.7B-Instruct
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf luweigen/eurollm-1.7b-instruct-android:Q4_0# Run inference directly in the terminal: llama cli -hf luweigen/eurollm-1.7b-instruct-android:Q4_0