Verdugie commited on
Commit
b546c20
·
verified ·
1 Parent(s): e87a28e

Fix quant table: all three quants available (Q4_K_M, Q6_K, Q8_0)

Browse files
Files changed (1) hide show
  1. README.md +8 -5
README.md CHANGED
@@ -50,9 +50,9 @@ STEM-Oracle trains the *reasoning disposition* alongside the domain knowledge:
50
 
51
  | File | Quant | Size | Notes |
52
  |------|-------|------|-------|
53
- | `STEM-Oracle-27B-Q8_0.gguf` | Q8_0 | ~28 GB | Reference quality. |
54
-
55
- Additional quantizations (Q4_K_M, Q6_K) will be added after stress testing validates personality and accuracy survival across quant levels.
56
 
57
  ---
58
 
@@ -82,9 +82,12 @@ Works with any GGUF-compatible runtime — LM Studio, Ollama, llama.cpp, KoboldC
82
 
83
  | Setup | Quantization | VRAM/RAM | Speed | Notes |
84
  |-------|-------------|----------|-------|-------|
85
- | **RTX 4090 (24GB)** | Q8_0 | ~28 GB VRAM | 8-15 t/s | Tight fit, may need partial offload. |
86
- | **Apple M2/M3 Ultra** | Q8_0 | 64-128 GB unified | 5-10 t/s | Full model in unified memory. |
 
 
87
  | **Dual GPU** | Q8_0 | ~30 GB VRAM | Varies | Split across two 16GB+ cards. |
 
88
 
89
  ---
90
 
 
50
 
51
  | File | Quant | Size | Notes |
52
  |------|-------|------|-------|
53
+ | `STEM-Oracle-27B-Q4_K_M.gguf` | Q4_K_M | ~16 GB | Primary ship. RTX 4090 sweet spot. |
54
+ | `STEM-Oracle-27B-Q6_K.gguf` | Q6_K | ~21 GB | Quality tier. 32GB+ VRAM. |
55
+ | `STEM-Oracle-27B-Q8_0.gguf` | Q8_0 | ~28 GB | Reference quality. Serious hardware. |
56
 
57
  ---
58
 
 
82
 
83
  | Setup | Quantization | VRAM/RAM | Speed | Notes |
84
  |-------|-------------|----------|-------|-------|
85
+ | **RTX 4090 (24GB)** | Q4_K_M | ~18 GB VRAM | 15-25 t/s | Sweet spot for consumer hardware. |
86
+ | **RTX 4090 (24GB)** | Q6_K | ~23 GB VRAM | 10-18 t/s | Higher fidelity, tight fit. |
87
+ | **Apple M2/M3 Ultra** | Q4_K_M/Q6_K | 64-128 GB unified | 5-10 t/s | Full model in unified memory. |
88
+ | **RTX 3090/4080** | Q4_K_M | ~18 GB VRAM | 10-18 t/s | Comfortable. |
89
  | **Dual GPU** | Q8_0 | ~30 GB VRAM | Varies | Split across two 16GB+ cards. |
90
+ | **CPU Only** | Q4_K_M | ~20 GB RAM | 1-3 t/s | 32GB+ system RAM. Slow but works. |
91
 
92
  ---
93