Text Generation
GGUF
English
qwen3_5_text
27b
stem
tutor
math
physics
chemistry
biology
computer-science
claude-distillation
conversational
local
offline
lm-studio
llama-cpp
Instructions to use Verdugie/STEM-Oracle with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Verdugie/STEM-Oracle with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Verdugie/STEM-Oracle:Q4_K_M # Run inference directly in the terminal: llama cli -hf Verdugie/STEM-Oracle:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Verdugie/STEM-Oracle:Q4_K_M # Run inference directly in the terminal: llama cli -hf Verdugie/STEM-Oracle:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Verdugie/STEM-Oracle:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Verdugie/STEM-Oracle:Q4_K_M
Use Docker
docker model run hf.co/Verdugie/STEM-Oracle:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Verdugie/STEM-Oracle with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Verdugie/STEM-Oracle" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Verdugie/STEM-Oracle", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Verdugie/STEM-Oracle:Q4_K_M
- Ollama
How to use Verdugie/STEM-Oracle with Ollama:
ollama run hf.co/Verdugie/STEM-Oracle:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use Verdugie/STEM-Oracle with Docker Model Runner:
docker model run hf.co/Verdugie/STEM-Oracle:Q4_K_M
- Lemonade
How to use Verdugie/STEM-Oracle with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Verdugie/STEM-Oracle:Q4_K_M
Run and chat with the model
lemonade run user.STEM-Oracle-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Fix quant table: all three quants available (Q4_K_M, Q6_K, Q8_0)
Browse files
README.md
CHANGED
|
@@ -50,9 +50,9 @@ STEM-Oracle trains the *reasoning disposition* alongside the domain knowledge:
|
|
| 50 |
|
| 51 |
| File | Quant | Size | Notes |
|
| 52 |
|------|-------|------|-------|
|
| 53 |
-
| `STEM-Oracle-27B-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
|
| 57 |
---
|
| 58 |
|
|
@@ -82,9 +82,12 @@ Works with any GGUF-compatible runtime — LM Studio, Ollama, llama.cpp, KoboldC
|
|
| 82 |
|
| 83 |
| Setup | Quantization | VRAM/RAM | Speed | Notes |
|
| 84 |
|-------|-------------|----------|-------|-------|
|
| 85 |
-
| **RTX 4090 (24GB)** |
|
| 86 |
-
| **
|
|
|
|
|
|
|
| 87 |
| **Dual GPU** | Q8_0 | ~30 GB VRAM | Varies | Split across two 16GB+ cards. |
|
|
|
|
| 88 |
|
| 89 |
---
|
| 90 |
|
|
|
|
| 50 |
|
| 51 |
| File | Quant | Size | Notes |
|
| 52 |
|------|-------|------|-------|
|
| 53 |
+
| `STEM-Oracle-27B-Q4_K_M.gguf` | Q4_K_M | ~16 GB | Primary ship. RTX 4090 sweet spot. |
|
| 54 |
+
| `STEM-Oracle-27B-Q6_K.gguf` | Q6_K | ~21 GB | Quality tier. 32GB+ VRAM. |
|
| 55 |
+
| `STEM-Oracle-27B-Q8_0.gguf` | Q8_0 | ~28 GB | Reference quality. Serious hardware. |
|
| 56 |
|
| 57 |
---
|
| 58 |
|
|
|
|
| 82 |
|
| 83 |
| Setup | Quantization | VRAM/RAM | Speed | Notes |
|
| 84 |
|-------|-------------|----------|-------|-------|
|
| 85 |
+
| **RTX 4090 (24GB)** | Q4_K_M | ~18 GB VRAM | 15-25 t/s | Sweet spot for consumer hardware. |
|
| 86 |
+
| **RTX 4090 (24GB)** | Q6_K | ~23 GB VRAM | 10-18 t/s | Higher fidelity, tight fit. |
|
| 87 |
+
| **Apple M2/M3 Ultra** | Q4_K_M/Q6_K | 64-128 GB unified | 5-10 t/s | Full model in unified memory. |
|
| 88 |
+
| **RTX 3090/4080** | Q4_K_M | ~18 GB VRAM | 10-18 t/s | Comfortable. |
|
| 89 |
| **Dual GPU** | Q8_0 | ~30 GB VRAM | Varies | Split across two 16GB+ cards. |
|
| 90 |
+
| **CPU Only** | Q4_K_M | ~20 GB RAM | 1-3 t/s | 32GB+ system RAM. Slow but works. |
|
| 91 |
|
| 92 |
---
|
| 93 |
|