Instructions to use NagusameCS/ultratensor-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use NagusameCS/ultratensor-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf NagusameCS/ultratensor-models # Run inference directly in the terminal: llama cli -hf NagusameCS/ultratensor-models
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf NagusameCS/ultratensor-models # Run inference directly in the terminal: llama cli -hf NagusameCS/ultratensor-models
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf NagusameCS/ultratensor-models # Run inference directly in the terminal: ./build/bin/llama cli -hf NagusameCS/ultratensor-models
Use Docker
docker model run hf.co/NagusameCS/ultratensor-models
- Llama new
- LM Studio
- Jan
- Ollama
How to use NagusameCS/ultratensor-models with Ollama:
ollama run hf.co/NagusameCS/ultratensor-models
- Unsloth Desktop
- Docker Model Runner
How to use NagusameCS/ultratensor-models with Docker Model Runner:
docker model run hf.co/NagusameCS/ultratensor-models
- Lemonade
How to use NagusameCS/ultratensor-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull NagusameCS/ultratensor-models
Run and chat with the model
lemonade run user.ultratensor-models-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
corrected release: model_catalog.json
Browse files- model_catalog.json +35 -35
model_catalog.json
CHANGED
|
@@ -110,53 +110,53 @@
|
|
| 110 |
{
|
| 111 |
"id": "keep16u-iq2xs",
|
| 112 |
"file": "DeepSeek-V4-Coder-keep16u-iq2xs.gguf",
|
| 113 |
-
"bytes":
|
| 114 |
-
"sha256": "
|
| 115 |
-
"quant": "IQ2_XS",
|
| 116 |
"quality_vs_full_model": {
|
| 117 |
"code": {
|
| 118 |
-
"mean_ppl_ratio": 9.
|
| 119 |
-
"mean_kld": 2.
|
| 120 |
-
"p99_kld": 11.
|
| 121 |
-
"max_kld": 16.
|
| 122 |
-
"same_top_p":
|
| 123 |
-
"ppl_q": 22.
|
| 124 |
"ppl_base": 2.482437,
|
| 125 |
-
"mean_ln_ppl_ratio": 2.
|
| 126 |
-
"seconds":
|
| 127 |
},
|
| 128 |
"math": {
|
| 129 |
-
"mean_ppl_ratio": 26.
|
| 130 |
-
"mean_kld": 3.
|
| 131 |
-
"p99_kld": 11.
|
| 132 |
-
"max_kld": 15.
|
| 133 |
-
"same_top_p": 30.
|
| 134 |
-
"ppl_q":
|
| 135 |
"ppl_base": 4.439052,
|
| 136 |
-
"mean_ln_ppl_ratio": 3.
|
| 137 |
-
"seconds":
|
| 138 |
},
|
| 139 |
"web": {
|
| 140 |
-
"mean_ppl_ratio": 20.
|
| 141 |
-
"mean_kld": 3.
|
| 142 |
-
"p99_kld": 10.
|
| 143 |
-
"max_kld": 15.
|
| 144 |
-
"same_top_p":
|
| 145 |
-
"ppl_q":
|
| 146 |
"ppl_base": 5.429595,
|
| 147 |
-
"mean_ln_ppl_ratio":
|
| 148 |
-
"seconds":
|
| 149 |
},
|
| 150 |
"multilingual": {
|
| 151 |
-
"mean_ppl_ratio": 143.
|
| 152 |
-
"mean_kld": 5.
|
| 153 |
-
"p99_kld": 12.
|
| 154 |
-
"max_kld": 15.
|
| 155 |
-
"same_top_p":
|
| 156 |
-
"ppl_q":
|
| 157 |
"ppl_base": 1.862109,
|
| 158 |
-
"mean_ln_ppl_ratio": 4.
|
| 159 |
-
"seconds":
|
| 160 |
}
|
| 161 |
}
|
| 162 |
}
|
|
|
|
| 110 |
{
|
| 111 |
"id": "keep16u-iq2xs",
|
| 112 |
"file": "DeepSeek-V4-Coder-keep16u-iq2xs.gguf",
|
| 113 |
+
"bytes": 30160640544,
|
| 114 |
+
"sha256": "ab534a1b2f87dfc64edc42f68a33f45592c2d3f85f46a0f0aee46a5c6975b855",
|
| 115 |
+
"quant": "IQ2_XS experts + Q3_K_M",
|
| 116 |
"quality_vs_full_model": {
|
| 117 |
"code": {
|
| 118 |
+
"mean_ppl_ratio": 9.099898,
|
| 119 |
+
"mean_kld": 2.362374,
|
| 120 |
+
"p99_kld": 11.31906,
|
| 121 |
+
"max_kld": 16.148975,
|
| 122 |
+
"same_top_p": 53.137,
|
| 123 |
+
"ppl_q": 22.589925,
|
| 124 |
"ppl_base": 2.482437,
|
| 125 |
+
"mean_ln_ppl_ratio": 2.208263,
|
| 126 |
+
"seconds": 853.555673122406
|
| 127 |
},
|
| 128 |
"math": {
|
| 129 |
+
"mean_ppl_ratio": 26.888559,
|
| 130 |
+
"mean_kld": 3.447439,
|
| 131 |
+
"p99_kld": 11.441486,
|
| 132 |
+
"max_kld": 15.993463,
|
| 133 |
+
"same_top_p": 30.654,
|
| 134 |
+
"ppl_q": 119.359712,
|
| 135 |
"ppl_base": 4.439052,
|
| 136 |
+
"mean_ln_ppl_ratio": 3.291701,
|
| 137 |
+
"seconds": 991.8521473407745
|
| 138 |
},
|
| 139 |
"web": {
|
| 140 |
+
"mean_ppl_ratio": 20.035827,
|
| 141 |
+
"mean_kld": 3.298229,
|
| 142 |
+
"p99_kld": 10.33832,
|
| 143 |
+
"max_kld": 15.200585,
|
| 144 |
+
"same_top_p": 30.98,
|
| 145 |
+
"ppl_q": 108.786426,
|
| 146 |
"ppl_base": 5.429595,
|
| 147 |
+
"mean_ln_ppl_ratio": 2.997522,
|
| 148 |
+
"seconds": 978.4559774398804
|
| 149 |
},
|
| 150 |
"multilingual": {
|
| 151 |
+
"mean_ppl_ratio": 143.609144,
|
| 152 |
+
"mean_kld": 5.05533,
|
| 153 |
+
"p99_kld": 12.383608,
|
| 154 |
+
"max_kld": 15.366625,
|
| 155 |
+
"same_top_p": 20.261,
|
| 156 |
+
"ppl_q": 267.415942,
|
| 157 |
"ppl_base": 1.862109,
|
| 158 |
+
"mean_ln_ppl_ratio": 4.967095,
|
| 159 |
+
"seconds": 965.1823761463165
|
| 160 |
}
|
| 161 |
}
|
| 162 |
}
|