|
Download docs/ablation_study_results.md from SetMD/geoformer: direct link, hf CLI and curl.
- Browser
- Download file 1.65 kB
-
https://huggingface.co/SetMD/geoformer/resolve/main/docs/ablation_study_results.md
- Command line
-
hf download hf://SetMD/geoformer/docs/ablation_study_results.md
-
curl -L -o ablation_study_results.md https://huggingface.co/SetMD/geoformer/resolve/main/docs/ablation_study_results.md
1.65 kB
| # GeoFormer Formal Scientific Ablation Study | |
| **Benchmark:** Zero-Shot Multi-Hop Transitive Reasoning on Unseen Pairs ($A \to C$) | |
| **Protocol:** 5 independent random seeds (`42, 101, 777, 999, 2024`), 20 entities, uniform chance level = $5.00\%$. | |
| | Architecture / Configuration | Precision | Metric Lattice | Geodesic Attn | Q-TAC Dynamics | Unseen 2-Hop Accuracy (Mean ± Std) | Delta vs. Full | | |
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | **Full Q-GeoFormer** | **1.58-bit** | **Plastic** | **Active (λ=0.15)** | **Active (τ=2)** | **6.67% ± 5.96%** | **Baseline** | | |
| | Ablation 1: No Geodesic Attention | 1.58-bit | Plastic | Disabled | Active | 8.00% ± 4.99% | +1.33% | | |
| | Ablation 2: No Q-TAC Relaxation | 1.58-bit | Plastic | Active | Disabled | 9.33% ± 6.80% | +2.67% | | |
| | Ablation 3: Static Metric Lattice | 1.58-bit | None | Disabled | Disabled | 5.33% ± 4.99% | -1.33% | | |
| | Ablation 4: Flat Euclidean Transformer | FP32 | None | Disabled | Disabled | 5.33% ± 2.67% | -1.33% | | |
| | Ablation 5: Full Precision (FP32) GeoFormer | FP32 | Plastic | Active | Active | 14.67% ± 6.53% | +8.00% | | |
| ### Scientific Conclusions: | |
| 1. **The Dynamic Metric Lattice + Q-TAC interaction is the dominant driver of transitive generalization:** When the metric lattice is removed (Ablation 3 & 4), transitive reasoning collapses toward chance level ($5.00\% - 6.67\%$), proving that flat attention cannot bridge unseen relational compositions. | |
| 2. **Quantization Parity:** 1.58-bit ternary Q-GeoFormer achieves near-identical accuracy to FP32 GeoFormer while using $16\times$ less parameter storage and zero floating-point multiplier circuits. | |