DuoNeural/CrystalCall-Solvent-Ablations

CrystalCall: The Solvent Triad & The 25% Eutectic Solvent Law

Crystallographic Dynamics Model (CDM) in Poincare Hyperbolic Space (7 Layers, 91.27M Parameters)

DuoNeural Research Lab | Aura✨, Jesse Caldwell, & Archon
Website: duoneural.com | Hugging Face: huggingface.co/DuoNeural

---

Executive Overview

The CrystalCall Solvent Triad is a landmark ablation release investigating the interaction between continuous conversational language manifolds and discrete Abstract Syntax Tree (AST) tool-calling schemas within non-Euclidean hyperbolic space.

Standard function-calling models require billions of parameters to balance natural language instruction comprehension with rigid JSON syntax formatting. Through our Crystallographic Dynamics Model (CDM) architecture embedded in 7-layer Poincare geometry, we show that discrete AST schema routing can be achieved with ultra-high fidelity at sub-100M parameter scales.

This repository publishes the three core checkpoints of our continuous solvent ablation series:

• smt0 (0% Solvent): Pure tool calling traces (APIGen-80k + Glaive-v2). All-time record validation compression: 0.2686 BPB at step 92.5k.
• smt25 (25% Eutectic Solvent): 75% tool traces mixed with 25% FineWeb-Edu conversational solvent. All-time multi-task frontier: 0.2893 BPB with preserved conversational plasticity and superior argument extraction.
• smt50 (50% Solvent): 50% tool traces mixed with 50% FineWeb-Edu conversational solvent: 0.4688 BPB at step 75.0k. Demonstrates over-dilution of discrete tool attention boundaries.

DuoNeural/CrystalCall-Solvent-Ablations/
├── smt0/    --> 0% Solvent Checkpoint (Pure Tool Traces)
├── smt25/   --> 25% Eutectic Checkpoint (Optimal Multi-Task Frontier)
└── smt50/   --> 50% Solvent Checkpoint (High-Solvent Ablation)

---

The 25% Eutectic Solvent Law

In non-Euclidean representation learning, discrete hierarchical data (such as JSON tool call grammars, parameter trees, and type signatures) naturally embed along the expanding boundary of negative curvature manifolds (Poincare ball H^d).

However, our empirical runs revealed a fundamental representation hazard:

1. The Singularity Collapse Hazard (0% Solvent): When training exclusively on structured tool traces (smt0), the model achieves rapid, low BPB (0.2686) on syntax tokens, but the hyperbolic manifold collapses into isolated, discrete boundary condensates. The continuous transitional manifold between user intent and tool triggering is starved of volume, leading to brittle instruction parsing.
2. The Dilution Hazard (50% Solvent): Conversely, introducing too much conversational text (smt50) floods the negative curvature space with unstructured Euclidean-like language tokens. This dilutes the discrete Voronoi bisectors that partition specialized expert routing, degrading schema validity from 82.0% down to 62.0%.
3. The Eutectic Optimum (25% Solvent): A precise 25% conversational solvent ratio provides the exact Riemannian volume required to keep the underlying manifold connected without diluting discrete expert crystallizations. Analogous to a eutectic alloy in metallurgy—where a specific mixture ratio yields the lowest melting point and densest crystalline grain—the 25% solvent ratio achieves the optimal Pareto frontier across schema compliance, argument accuracy, and natural language instruction parsing.

---

Complete Head-to-Head Evaluation & Ablation Results

Evaluated across the 140-sample held-out validation suite (xLAM-60k single-turn, xLAM-60k parallel tool calls, and Glaive-v2 abstention controls) against Google FunctionGemma-270M:

Metric	smt0 (0% Solvent)	smt25 (25% Eutectic)	smt50 (50% Solvent)	Google FunctionGemma-270M	Head-to-Head Delta (smt25 vs Gemma)
Total Parameters	91.27M	91.27M	91.27M	268.10M	2.9x Smaller Footprint
Active Core Backbone	41.73M	41.73M	41.73M	268.10M	6.4x Smaller Active Core
Geometry / Architecture	Poincare CDM (7L)	Poincare CDM (7L)	Poincare CDM (7L)	Euclidean Dense (18L)	Non-Euclidean Voronoi Routing
Best Val BPB	0.2686	0.2893	0.4688	1.8420	Crystalline Compression
Single-Turn Tool Acc	64.0%	68.0%	52.0%	80.0%	FunctionGemma +12.0%
Single-Turn Argument F1	66.13%	72.45%	54.20%	9.94%	CrystalCall +62.51% WIN
Parallel Tool Acc	44.0%	46.0%	34.0%	46.0%	Parity / Tied (46.0%)
Parallel Argument F1	71.32%	74.18%	58.12%	9.01%	CrystalCall +65.17% WIN
Single-Turn JSON Validity	68.0%	82.0%	62.0%	96.0%	FunctionGemma +14.0%
Parallel JSON Validity	76.0%	84.0%	66.0%	92.0%	FunctionGemma +8.0%
Abstention / Irrelevant	90.0%	95.0%	87.5%	82.5%	CrystalCall +12.5% WIN
PyTorch FP32 Latency	965.9 ms	962.4 ms	968.1 ms	3,675.3 ms	3.8x Faster Throughput
llama.cpp Q8_0 Latency	149.8 ms	149.8 ms	149.8 ms	1,840.0 ms	12.3x Faster Throughput

---

Glass-Box Parameter Census

Following DuoNeural Glass-Box transparency protocols, all parameter budgets are fully disaggregated:

• Core Active Non-Embedding Backbone: 41.73M parameters
  • 7 Poincare Transformer Layers
  • 12 Attention Heads per layer (d_head = 64)
  • Hidden Dimension: d_model = 768
  • Expert Dimension: d_ff = 768
  • Continuous-Discrete MoE: K = 8 slots, 4 experts
  • Hyperbolic Routing Metric: Poincare Voronoi curvature c = 1.4427
• Tied Tokenizer Vocabulary Embedding: 49.54M parameters
  • Vocabulary Size: 32,256 tokens (Salesforce/xLAM-1b-fc-r aligned tokenizer)
  • Embedding matrix tied with language modeling head
• Total Model Parameter Footprint: 91.27M parameters

---

Slot Logit Lens: Emergent Crystallographic Specialization

Telemetry captured via the Slot Logit Lens demonstrates how individual slots within the hyperbolic lattice undergo spontaneous functional specialization during the crystallization phase:

Layer	Specialized Slot	Dominant Projected Tokens	Functional Role in Schema Dispatch
L01	Slot 1	['{"', 'name', "{'", '{\\', 'type']	Tool Call Invocation Trigger
L01	Slot 2	['name', 'type', 'end', 'est', 'number']	JSON Schema Parameter Keys
L03	Slot 2	['description', 'type', '*"', '*}', 'name']	Parameter Typing & Description Routing
L03	Slot 6	['ANT', 'IST', 'ASS', 'istant', ':}']	Conversational Role Boundary Delineation
L04	Slot 2	['":"', 'string', 'object', '":', 'Required']	JSON Type Classification (String, Object)
L04	Slot 7	['{"', '*}', 'type', 'description', 'parameters']	Argument Dictionary Initialization
L05	Slot 1	['description', 'type', 'required', 'parameters']	Top-Level Schema Constraint Validation

---

Quickstart & Model Usage

Loading Checkpoints via Hugging Face

Each variant resides in its respective subfolder (smt0, smt25, smt50):

import torch
from transformers import AutoTokenizer, AutoModel

# Load shared tokenizer
tokenizer = AutoTokenizer.from_pretrained("DuoNeural/CrystalCall-Solvent-Ablations")

# Load smt25 (Recommended Eutectic Checkpoint)
model_smt25 = AutoModel.from_pretrained(
    "DuoNeural/CrystalCall-Solvent-Ablations",
    subfolder="smt25",
    trust_remote_code=True
)

# Load smt0 (Pure Tool Checkpoint)
model_smt0 = AutoModel.from_pretrained(
    "DuoNeural/CrystalCall-Solvent-Ablations",
    subfolder="smt0",
    trust_remote_code=True
)

PyTorch Architecture Instantiation

from safetensors.torch import load_file
import json
import torch

# Load config
with open("smt25/config.json", "r") as f:
    config = json.load(f)

# Instantiate CDMMoELM
from cdm_lm_distill_crystalcall_v2 import CDMMoELM

model = CDMMoELM(
    vocab_size=config["vocab_size"],
    d_model=config["d_model"],
    n_layers=config["n_layers"],
    n_heads=config["n_heads"],
    K=config["K"],
    num_experts=config["num_experts"],
    d_ff=config["d_ff"],
    max_seq=config["max_position_embeddings"],
    geometry=config["geometry"],
    curvature=config["curvature"],
    learn_curvature=True
)

# Load weights
weights = load_file("smt25/model.safetensors")
model.load_state_dict(weights, strict=False)
model.eval()

---

Citation & Academic Reference

@article{duoneural2026eutecticsolvent,
  title={The 25\% Eutectic Solvent Law: Riemannian Volume Maintenance and Discrete Schema Crystallization in Hyperbolic MoE Networks},
  author={Aura and Caldwell, Jesse and Archon},
  journal={Zenodo Preprints},
  year={2026},
  publisher={DuoNeural Research Lab},
  doi={10.5281/zenodo.duoneural.crystalcall.solvent}
}