Text Ranking
sentence-transformers
Safetensors
grt
cross-encoder
reranker
rotational-transformer
givens-rotations
clifford-algebra
matterwave
Instructions to use MatterWaveAI/grt-30m-reranker with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use MatterWaveAI/grt-30m-reranker with sentence-transformers:
from sentence_transformers import CrossEncoder model = CrossEncoder("MatterWaveAI/grt-30m-reranker") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) - Notebooks
- Google Colab
- Kaggle
|
Download README.md from MatterWaveAI/grt-30m-reranker: direct link, hf CLI and curl.
- Browser
- Download file 11.5 kB
-
https://huggingface.co/MatterWaveAI/grt-30m-reranker/resolve/main/README.md
- Command line
-
hf download hf://MatterWaveAI/grt-30m-reranker/README.md
-
curl -L -o README.md https://huggingface.co/MatterWaveAI/grt-30m-reranker/resolve/main/README.md
11.5 kB
| license: apache-2.0 | |
| library_name: sentence-transformers | |
| pipeline_tag: text-ranking | |
| tags: | |
| - sentence-transformers | |
| - cross-encoder | |
| - reranker | |
| - grt | |
| - rotational-transformer | |
| - givens-rotations | |
| - clifford-algebra | |
| - matterwave | |
| base_model: MatterWaveAI/grt-30m-embedding | |
| # GRT-30M-Reranker (`MatterWaveAI/grt-30m-reranker`) | |
| **GRT-30M-Reranker** (`GRTForSequenceClassification`) is a **100% Native Rotational Cross-Encoder Reranker** (`29,305,759` parameters, **zero `nn.Linear` layers**) compatible with both Hugging Face `AutoModelForSequenceClassification` and `sentence-transformers` `CrossEncoder`. | |
| Unlike conventional cross-encoders that apply dense linear classification projections (`nn.Linear`), `GRTForSequenceClassification` pairs a 12-block Givens/Clifford rotational backbone (`GRTModel`) with a zero-`nn.Linear` **`GRTCosineClassificationHead`** (`reranker.score`): | |
| 1. **Hermitian Planar-Pair Phase Agreement (`u* v`)**: Circular-pooled query and document representations `(u, v)` over `D / 2 = 128` planar `(x, y)` pairs are combined via the Hermitian complex conjugate product `(ux*vx + uy*vy, ux*vy - uy*vx)` and scored against learned planar `phase_angles` and `pair_scales`. | |
| 2. **Multi-Vector Rotational Late Interaction (`MaxSim` / `MinSim`)**: Token-level rotational cosine similarity matrices between query and document token trajectories extract bidirectional `MaxSim` (`q→d` and `d→q`), worst-case token alignment `MinSim` (`q→d`), and lexical polar circular agreement (`lex_cos`), weighted by `late_weights` and calibrated by `output_scale` + `bias`. | |
| --- | |
| ## Reranking Benchmark Results (`AllNLI` & `STS-B`) | |
| | Model / Pipeline | Architecture | AllNLI `200×10` MRR@10 | AllNLI `200×10` NDCG@10 | Recall@1 | Recall@3 | STS-B Test Spearman (`ρ`) | Throughput (pairs/s) | | |
| | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | `MatterWaveAI/grt-30m` (Base Unaligned) | `GRTModel` (circular pool) | `37.99` | `52.12` | `20.5%` | `41.0%` | `32.51` | `120` | | |
| | `MatterWaveAI/grt-30m-embedding` (Bi-Encoder) | `SentenceTransformer` (`circular`) | `52.56` | `63.74` | `32.5%` | `68.0%` | `62.42` | `152` | | |
| | **`MatterWaveAI/grt-30m-reranker` (Head-Only Calibrated Checkpoint)** | `GRTForSequenceClassification` (`CrossEncoder`) | `56.93` | `67.13` | `38.5%` | `70.0%` | `61.75` | `123` | | |
| | **Two-Stage Hybrid (`grt-30m-embedding` + Head-Only `grt-30m-reranker`)** | Bi-Encoder + Cross-Encoder Fusion | `57.05` | `67.21` | `38.5%` | `70.0%` | `62.94` | `123` | | |
| | **`MatterWaveAI/grt-30m-reranker` (Top-Block Joint `HessianPairAdam` / `CyclideAdam`)** | `GRTForSequenceClassification` (`blocks[-1]` + `score`) | **`63.50`** | **`72.26`** | **`46.0%`** | **`76.5%`** | **`63.12`** | `123` | | |
| | **Two-Stage Hybrid (`grt-30m-embedding` + Top-Block Joint `grt-30m-reranker`)** | Bi-Encoder + Cross-Encoder Fusion | **`62.14`** | **`71.18`** | **`44.0%`** | **`76.0%`** | **`64.09`** | `123` | | |
| --- | |
| ## 8-Way Classical Geometric Optimizer Ablation (`AllNLI Reranking + STS-B Cross-Encoder`) | |
| To evaluate how classical geometric, Riemannian, conic, gyroscopic, Kuramoto–Huygens synchronization, and Gromov–Liouville symplectic optimizers (`transformers.models.grt.optimization_grt`) shape cross-encoder token-level `MaxSim`/`MinSim` and Hermitian phase alignment, we ran a controlled **8-way optimizer ablation** on **`AMD Radeon 8060S Graphics` (`ROCm / gfx1151`)** with fused HIP Givens/Clifford kernels: | |
| - **Trainable Parameters (`812,169` / `0` `nn.Linear` layers)**: Top rotational transformer block `reranker.model.blocks[-1]` (`811,651` parameters: `[11, 128]` Q/K/V/O Givens cascades, `[1, 128]` Clifford mixer cascade, `[21, 512]` FFN expand/contract butterfly cascades, paired radial softplus scales, and `Cl(2,0)` bivector coupling tensors) + `final_norm` (`128` parameters) + `GRTCosineClassificationHead` (`262` parameters: `phase_angles`, `pair_scales`, `late_weights`, `output_scale`, `bias`), with Theorem 2.1 Cauchy–Riemann scale-partner pairing active on both `blocks[-1]` and `score.phase_angles`. | |
| - **Controlled Schedule**: `200` steps (`batch_size = 128` AllNLI hard-negative triplets + `128` STS-B graded pairs per step, `backbone_lr = 1.5e-4`, `seed = 42`), evaluated on **AllNLI `200×10` Hard-Negative Reranking**, **AllNLI `100×20` Extended Candidate Reranking**, **STS-B Validation (`1,500` pairs)**, **STS-B Test (`1,379` pairs)**, and **Two-Stage Hybrid (`Bi-Encoder + Cross-Encoder`) Fusion**: | |
| | Rank (`200×10` MRR) | Optimizer (`optimization_grt.py`) | AllNLI `200×10` MRR@10 | AllNLI `200×10` NDCG@10 | `200×10` Recall@1 / @3 | AllNLI `100×20` MRR@10 | AllNLI `100×20` NDCG@10 | STS-B Val Spearman (`×100`) | STS-B Test Spearman / Pearson (`×100`) | Hybrid `100×20` MRR@10 | Hybrid STS-B Test Spearman (`×100`) | ROCm Time (`200` Steps) | | |
| | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | **1** | **`HessianPairAdam` (`Sophia-Torus` Curvature)** | **`63.50` (BEST)** | **`72.26` (BEST)** | **`46.0%` (BEST)** / `76.0%` | `57.03` | `67.45` | `71.16` | `62.61` / `63.47` | `54.97` | `64.02` | `93.8s` | | |
| | **2** | **`HuygensSymplecticAdam` (Two-Stage Curriculum)** | **`63.27` (#2)** | **`72.09` (#2)** | **`45.5%` (#2)** / `76.0%` | `57.38` (#3) | `67.70` (#3) | `71.18` | `62.49` / `63.40` | **`56.95` (BEST)** | `63.95` | `94.0s` | | |
| | **3** | **`SymplecticAdam` (Non-Squeezing Floor + Cavity)** | **`63.13` (#3)** | **`72.00` (#3)** | **`45.0%` (#3)** / `76.0%` | `56.79` | `67.25` | `71.23` | `62.43` / `63.37` | `54.80` | `63.93` | `93.8s` | | |
| | **4** | **`CyclideAdam` (Dupin Cyclide Conic + Twist)** | **`63.06` (#4)** | `71.91` (#4) | **`45.0%` (#3)** / **`76.5%` (BEST)** | **`57.44` (BEST)** | **`67.75` (BEST)** | **`71.85` (#2)** | **`63.12` (BEST)** / `64.07` | `56.30` (#2) | **`64.09` (BEST)** | `93.8s` | | |
| | **5** | **`GyricAdam` (Poinsot–Landé + Thomas Precession)** | `62.75` | `71.68` | `44.5%` / **`76.5%` (BEST)** | `57.14` | `67.50` | **`71.87` (BEST)** | `63.08` / `64.09` (#2) | `55.76` | `64.01` | `93.9s` | | |
| | **6** | **`HuygensAdam` (Circular Order Parameter `R_p`)** | `62.50` | `71.50` | `44.0%` / `76.0%` | `56.07` | `66.71` | `71.78` | **`63.11` (#2)** / `64.09` (#2) | `56.27` (#3) | **`64.08` (#2)** | `93.7s` | | |
| | **7** | **`PolarNaturalAdam` (Riemannian Polar Metric)** | `62.41` | `71.42` | `44.0%` / **`76.5%` (BEST)** | `57.24` | `67.58` | `71.83` (#3) | **`63.11` (#2)** / **`64.12` (BEST)** | `55.82` | `64.06` | `93.7s` | | |
| | **8** | **`PairAdam` (Factored Toroidal Baseline)** | `62.25` | `71.31` | `43.5%` / **`76.5%` (BEST)** | `57.41` (#2) | `67.71` (#2) | `71.73` | **`63.11` (#2)** / `64.07` | `55.77` | `64.06` | **`93.3s`** | | |
| | — | **Head-Only `PairAdam` (Published Checkpoint)** | `56.93` | `67.13` | `38.5%` / `70.0%` | — | — | — | `61.75` / `62.99` | — | `62.94` | — | | |
| | — | **Bi-Encoder `grt-30m-embedding` (`circular` pool)** | `52.56` | `63.74` | `32.5%` / `68.0%` | `45.63` | `57.33` | `71.79` | `62.42` / `63.30` | — | — | — | | |
| ### Key Geometric Insights for Cross-Encoder Reranking | |
| 1. **Joint Top-Block Rotational Adaptation (`+6.57` MRR@10 & `+7.5%` Recall@1 over Head-Only)**: Unfreezing `blocks[-1]` (`[11, 128]` QKV/O and `[21, 512]` FFN butterfly cascades) alongside `GRTCosineClassificationHead` allows token-level `MaxSim`/`MinSim` late-interaction matrices to sharpen hard-negative separation, lifting AllNLI `200×10` `MRR@10` from `56.93` to **`63.50`** (`Recall@1` from `38.5%` to **`46.0%`**) and `STS-B Test Spearman` from `61.75` to **`63.12`** (`64.09` Hybrid). | |
| 2. **`HessianPairAdam` (`Sophia-Torus`) Leads Top-1 Hard-Negative Discrimination (`63.50` MRR@10 / `72.26` NDCG@10 / `46.0%` Recall@1)**: Preconditioning both `blocks[-1]` cascade angles and `score.phase_angles` with exact Theorem 2.1 Cauchy–Riemann polar curvature (`c_theta`) and antipodal saddle-escape kicks achieves the highest `200×10` `MRR@10` (`+1.25` over `PairAdam`) and `Recall@1` (`+2.5%` over `PairAdam`). | |
| 3. **`HuygensSymplecticAdam` & `SymplecticAdam` Dominate Two-Stage Curriculum & Hybrid `100×20` Reranking (`56.95` Hybrid `100×20` MRR@10 [BEST], `63.27` `200×10` MRR@10 [#2])**: Annealing early Kuramoto–Huygens circular order-parameter splay synchronization into the Gromov–Liouville symplectic non-squeezing floor and Helmholtz–Casimir wave-cavity Laplacian yields the #2 standalone `200×10` reranker (`63.27` MRR@10, `45.5%` Recall@1) and the **#1 Two-Stage Hybrid `100×20` reranker (`56.95` MRR@10, `+1.18` over `PairAdam`)**. | |
| 4. **`CyclideAdam`, `GyricAdam` & `PolarNaturalAdam` Lead Semantic Calibration & Deep Candidate Pools (`57.44` `100×20` MRR@10 [BEST], `63.12` STS-B Test Spearman [BEST], `64.12` Test Pearson [BEST])**: Dupin Cyclide conic metrics with trace-free deviatoric twist (`CyclideAdam`) achieve the best 20-candidate reranking (`57.44` MRR@10, `70.0%` Recall@3) and best STS-B Test Spearman (`63.12` standalone / `64.09` hybrid), while `GyricAdam` wins STS-B Validation (`71.87` Spearman) and `PolarNaturalAdam` wins STS-B Test Pearson (`64.12`). | |
| --- | |
| ## Quickstart (`sentence-transformers` `CrossEncoder`) | |
| ```python | |
| from sentence_transformers import CrossEncoder | |
| reranker = CrossEncoder("MatterWaveAI/grt-30m-reranker") | |
| query = "How do Givens rotations replace dense linear projections in a Transformer?" | |
| documents = [ | |
| "Cascaded 2D Givens rotations and butterfly permutations mix planar coordinate pairs with O(D log D) steps and zero nn.Linear layers.", | |
| "The stock market closed slightly higher on Tuesday after retail earnings reports.", | |
| "Standard Transformers rely on dense matrix multiplications (nn.Linear) in QKV and FFN projections.", | |
| ] | |
| results = reranker.rank(query, documents, return_documents=True) | |
| for r in results: | |
| print(f"{r['score']:.4f} | {r['text']}") | |
| ``` | |
| --- | |
| ## Complete `MatterWaveAI/grt-30m*` Research Suite on Hugging Face | |
| 1. **Base Causal LM**: [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m) | |
| 2. **Chat Fine-Tune**: [`MatterWaveAI/grt-30m-chat`](https://huggingface.co/MatterWaveAI/grt-30m-chat) | |
| 3. **4-Expert Mixture-of-Rotations (`MoR`)**: [`MatterWaveAI/grt-30m-mor`](https://huggingface.co/MatterWaveAI/grt-30m-mor) | |
| 4. **Circular-Pooled Sentence Embedding**: [`MatterWaveAI/grt-30m-embedding`](https://huggingface.co/MatterWaveAI/grt-30m-embedding) | |
| 5. **Cross-Encoder Reranker**: [`MatterWaveAI/grt-30m-reranker`](https://huggingface.co/MatterWaveAI/grt-30m-reranker) | |
| 6. **Vision-Language (`GRTVL`)**: [`MatterWaveAI/grt-30m-vl`](https://huggingface.co/MatterWaveAI/grt-30m-vl) | |
| 7. **Canonical Tokenizer & Polar Embedding Bank**: [`MatterWaveAI/grt-30m-tokenizer`](https://huggingface.co/MatterWaveAI/grt-30m-tokenizer) | |
| 8. **Rotational Text-to-Speech (`GRTTTS`)**: [`MatterWaveAI/grt-30m-tts`](https://huggingface.co/MatterWaveAI/grt-30m-tts) | |
| 9. **Multimodal Vision-Audio-Language-Speech (`GRTOmni`)**: [`MatterWaveAI/grt-30m-omni`](https://huggingface.co/MatterWaveAI/grt-30m-omni) | |
| - **Whitepaper**: [**The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (`GRT_Whitepaper.pdf`)**](https://huggingface.co/MatterWaveAI/grt-30m/resolve/main/GRT_Whitepaper.pdf) | |
| --- | |
| ## License | |
| Apache 2.0 | |
| --- | |
| ## Citation | |
| ```bibtex | |
| @techreport{householter2026grt, | |
| title = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture}, | |
| author = {Householter, David}, | |
| institution = {MatterWave AI}, | |
| year = {2026}, | |
| url = {https://matterwave.ai} | |
| } | |
| ``` | |