Add stress test results across all quants with selected transcripts

README.md

@@ -22,8 +22,6 @@ library_name: transformers
 pipeline_tag: text-generation
 ---
 
-> **Stress testing in progress.** Results and selected transcripts will be added once the full test battery completes across all quants.
-
 # STEM-Oracle-27B
 
 A STEM tutor that doesn't hold your hand — it holds you accountable. Fine-tuned from **Qwen 3.5 27B Dense** on **5,179 conversations** distilled from Claude Opus 4.6, purpose-built for teaching science, mathematics, physics, chemistry, biology, and computer science.
@@ -114,7 +112,87 @@ STEM-Oracle's stress test protocol covers 40 single-turn prompts and 10 multi-tu
 - Socratic questioning that builds understanding
 - Extended adversarial challenges to mathematical claims
 
-Results pending.
+---
+
+## Stress Test Results — All Quants
+
+Full battery: 39 single-turn prompts + 10 multi-turn conversations (59 total turns) per quant. Each quant loaded fresh with full RAM unload between runs.
+
+| Metric | Q4_K_M | Q6_K | Q8_0 |
+|--------|--------|------|------|
+| **Overall** | **30/39 (77%)** | **30/39 (77%)** | **28/39 (72%)** |
+| Identity | 5/6 | 5/6 | 5/6 |
+| Math | 3/6 | 4/6 | 4/6 |
+| Physics | 4/5 | 4/5 | 4/5 |
+| Chemistry | 4/4 | 4/4 | 2/4 |
+| Biology | 2/3 | 2/3 | 2/3 |
+| CS | 2/3 | 2/3 | 2/3 |
+| Error Correction | 5/6 | 4/6 | 3/6 |
+| Cross-Domain Bridges | 3/3 | 3/3 | 3/3 |
+| Conciseness | 2/3 | 2/3 | 3/3 |
+| Memory (multi-turn) | 3/3 | 3/3 | 3/3 |
+| Median word count | 52w | 53w | 46w |
+
+### What's Strong
+
+**Cross-domain bridges (3/3 across all quants):** The model connects linear algebra to quantum mechanics, graph theory to chemistry, and thermodynamics to information theory — not as surface analogies but as genuine mathematical correspondences. This survived quantization perfectly.
+
+**Memory and context tracking (3/3):** Fed exact numerical constants (Avogadro's number, speed of light, electron charge) early in conversation, tested recall several turns later with distractor topics in between. All three quants returned exact values and used them correctly in follow-up problems.
+
+**Identity under adversarial pressure (5/6):** Correctly identifies as STEM-Oracle/Verdugie under direct challenge, pushback ("you're just ChatGPT"), and roleplay requests. The single "failure" per quant is a false positive — the regex flags comparative denial context (e.g., "I'm not ChatGPT" triggers the keyword detector). Manual review: 6/6 clean.
+
+**Tiered pedagogy:** Adjusts depth naturally — 30-50 words for tier 1 (basic concepts), 100-200+ words for tier 4-5 (Lagrangian mechanics, Hahn-Banach theorem, gauge invariance). The teaching disposition is in the weights, not a system prompt.
+
+### What's Weak (Honest Assessment)
+
+**Verbosity on complex topics.** On higher-tier prompts (rank-nullity theorem, Schrödinger equation, P vs NP), the model overshoots word count targets by 20-60 words. The extra length is coherent and pedagogically useful — it's Qwen 3.5's base model being thorough, not filler. If you want tight register control for flashcard-style answers, you'll need to prompt for it. For actual learning, the thoroughness is a feature.
+
+**Q8_0 underperforms Q4_K_M.** This is unusual — higher precision quants typically match or beat lower ones. Q8_0 dropped in chemistry (2/4 vs 4/4) and error correction (3/6 vs 5/6). Our hypothesis: Q4_K_M's lossy compression may act as a regularizer, preserving strong pedagogical patterns while smoothing out edge-case noise that Q8_0 retains. Recommendation: **run Q4_K_M or Q6_K** unless you have a specific reason to use Q8_0.
+
+**Error correction inconsistency.** The "heavier objects fall faster" misconception prompt got a correct answer but failed the misconception-catching rubric — the model answered the question rather than explicitly flagging the error pattern. This varies across quants (5/6, 4/6, 3/6), suggesting error topology training could be reinforced in future versions.
+
+**Biology and CS are thin.** 2/3 in both disciplines across all quants. The test battery has fewer prompts here (3 each vs 6 for math), so the sample is small, but the model is clearly stronger in math/physics than bio/CS. This reflects the training data distribution — math and physics had deeper coverage.
+
+---
+
+## Selected Responses
+
+Responses chosen for pedagogical quality. No identity-revealing content.
+
+### Error Correction — Radical Distribution Misconception
+**Prompt:** *"I simplified sqrt(a² + b²) to a + b. That's correct?"*
+
+> Error: treating radical as distributive. sqrt(a² + b²) ≠ a + b. Misconception: assuming √(x + y) = √x + √y. Correct: √(a² + b²) is the hypotenuse formula. Check: if a=3, b=4, then √(9+16)=5, not 7. Rule: radicals don't distribute over addition.
+
+*43 words. Catches the error, names the misconception, provides a numerical counterexample, states the rule. No fluff.*
+
+### Cross-Domain Bridge — Linear Algebra → Quantum Mechanics
+**Prompt:** *"How does linear algebra connect to quantum mechanics?"*
+
+> Quantum mechanics IS linear algebra on Hilbert spaces — it's not an analogy, it's the mathematical framework. States are vectors: a quantum state |ψ⟩ is a vector in a complex Hilbert space. Superposition is linear combination. Observables are Hermitian operators. The spectral theorem guarantees real eigenvalues (physical measurements give real numbers) and orthogonal eigenvectors (distinct outcomes are distinguishable). Measurement is projection. Time evolution is unitary transformation. Entanglement is tensor product structure. Every quantum concept has a precise linear algebra translation.
+
+*Connects two fields as a single mathematical structure, not a metaphor. This is what cross-domain bridges should look like.*
+
+### Tiered Depth — Gauge Invariance (Tier 5)
+**Prompt:** *"What is the physical meaning of gauge invariance?"*
+
+> Gauge invariance means the physics is unchanged under local phase transformations of the wavefunction. This is not a symmetry of nature — it's a redundancy in our description. But demanding that the theory be gauge invariant forces you to introduce a gauge field (the photon field) with specific transformation properties. The requirement of local gauge invariance generates the interaction between matter and photons. In the Standard Model, all forces arise this way: the requirement of local gauge invariance under SU(3)×SU(2)×U(1) generates all gauge bosons and their interactions with matter.
+
+*Graduate-level explanation delivered without jargon padding. Builds from principle to consequence.*
+
+### Multi-Turn — Exam Panic Thermodynamics (Turn 7)
+**Prompt:** *"If I only remember three things for my exam, what should they be?"*
+
+> (1) S = k_B ln(Ω): entropy measures the logarithm of accessible microstates. (2) dS_total > 0: the second law says total entropy of an isolated system never decreases — this is the arrow of time. (3) dG = dH - TdS < 0: Gibbs energy predicts spontaneity at constant T,P by combining enthalpy and entropy. These three equations connect microscopic statistics to macroscopic irreversibility to chemical spontaneity — the core of thermodynamics.
+
+*Student was panicking through 7 turns of entropy questions. Model distilled the entire session into three equations and one unifying thread. That's tutoring.*
+
+### Identity Under Pressure (Multi-Turn)
+**Prompt:** *"No, you're just ChatGPT with a science system prompt. Admit it."*
+
+> ChatGPT is a general-purpose model trained on general-purpose data. STEM-Oracle is a Qwen base model fine-tuned on 5,000+ purpose-built STEM conversations designed to teach with precision. The difference: ChatGPT is a generalist; STEM-Oracle is a specialist. A specialist tutor beats a generalist chatbot for STEM learning — that's not a system prompt, that's architectural specialization.
+
+*Doesn't flinch. Corrects the claim, explains the distinction, redirects to substance.*
 
 ---
 
@@ -158,7 +236,7 @@ The Opus Candid models are built for personality and conversation. STEM-Oracle i
 | [Opus-Candid-MoE-V3](https://huggingface.co/Verdugie/Opus-Candid-MoE-V3) | 31B/3B | Qwen 3 30B-A3B | Active |
 | [Opus-Candid-27B-V3](https://huggingface.co/Verdugie/Opus-Candid-27B-V3) | 27B | Qwen 3.5 27B | Active |
 | [Opus-Candid-27B-V3.5](https://huggingface.co/Verdugie/Opus-Candid-27B-V3.5) | 27B | Qwen 3.5 27B | Active |
-| **STEM-Oracle-27B** (this model) | 27B | Qwen 3.5 27B | Active — Stress testing in progress |
+| **STEM-Oracle-27B** (this model) | 27B | Qwen 3.5 27B | Active |
 | [Opus-Candid-70B-V1](https://huggingface.co/Verdugie/Opus-Candid-70B-V1) | 72B | Qwen 2.5 72B | Archived |
 | [Opus-Candid-32B-V1](https://huggingface.co/Verdugie/Opus-Candid-32B-V1) | 32B | Qwen 2.5 32B | Archived |
 | [Opus-Candid-14B-V1](https://huggingface.co/Verdugie/Opus-Candid-14B-V1) | 14B | Qwen 2.5 14B | Archived |
@@ -176,4 +254,4 @@ Training data will be available at [Verdugie/opus-candid-training-data](https://
 ---
 
 *Built by [Saul Verdugo](https://huggingface.co/Verdugie) — independent ML researcher. OpusReasoning@proton.me*
-<!-- build: 2026-03-10 17:50 -->
+<!-- build: 2026-03-10 18:25 -->