Instructions to use cstr/ALMA-7B-R-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use cstr/ALMA-7B-R-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf cstr/ALMA-7B-R-GGUF # Run inference directly in the terminal: llama cli -hf cstr/ALMA-7B-R-GGUF
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf cstr/ALMA-7B-R-GGUF # Run inference directly in the terminal: llama cli -hf cstr/ALMA-7B-R-GGUF
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf cstr/ALMA-7B-R-GGUF # Run inference directly in the terminal: ./build/bin/llama cli -hf cstr/ALMA-7B-R-GGUF
Use Docker
docker model run hf.co/cstr/ALMA-7B-R-GGUF
- Llama new
- LM Studio
- Jan
- Ollama
How to use cstr/ALMA-7B-R-GGUF with Ollama:
ollama run hf.co/cstr/ALMA-7B-R-GGUF
- Unsloth Desktop
- Docker Model Runner
How to use cstr/ALMA-7B-R-GGUF with Docker Model Runner:
docker model run hf.co/cstr/ALMA-7B-R-GGUF
- Lemonade
How to use cstr/ALMA-7B-R-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull cstr/ALMA-7B-R-GGUF
Run and chat with the model
lemonade run user.ALMA-7B-R-GGUF-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
docs: add provenance / EU AI Act Art. 53 note
Browse files
README.md
CHANGED
|
@@ -17,3 +17,11 @@ GGUF conversion of [haoranxu/ALMA-7B-R](https://huggingface.co/haoranxu/ALMA-7B-
|
|
| 17 |
| `ALMA-7B-R-q4-k-m.gguf` | 3892 MB |
|
| 18 |
|
| 19 |
Compatible with [llama.cpp](https://github.com/ggml-org/llama.cpp) and other GGUF runtimes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
| `ALMA-7B-R-q4-k-m.gguf` | 3892 MB |
|
| 18 |
|
| 19 |
Compatible with [llama.cpp](https://github.com/ggml-org/llama.cpp) and other GGUF runtimes.
|
| 20 |
+
|
| 21 |
+
## Provenance and EU AI Act Art. 53 note
|
| 22 |
+
|
| 23 |
+
- **Upstream model:** [haoranxu/ALMA-7B-R](https://huggingface.co/haoranxu/ALMA-7B-R) — published by `haoranxu`.
|
| 24 |
+
- **Upstream licence:** `mit`. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
|
| 25 |
+
- **What was done here:** format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
|
| 26 |
+
- **Training data:** documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
|
| 27 |
+
- **Provider status:** under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.
|