Instructions to use GGLabYale/hyplora-gemma-7b-math10k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use GGLabYale/hyplora-gemma-7b-math10k with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("google/gemma-7b") model = PeftModel.from_pretrained(base_model, "GGLabYale/hyplora-gemma-7b-math10k") - Notebooks
- Google Colab
- Kaggle
HypLoRA Gemma-7B Math-10K
This repository provides HypLoRA adapter weights trained on Math-10K for Gemma 7B.
Code release: HypLoRA (hf-checkpoint-v branch) - please use this branch for the released adapters.
Paper: Hyperbolic Fine-Tuning for Large Language Models.
All checkpoints: GGLabYale HypLoRA collection.
Adapter details
| Field | Value |
|---|---|
| Hugging Face adapter ID | GGLabYale/hyplora-gemma-7b-math10k |
| Base model | google/gemma-7b |
| Training dataset | Math-10K |
| Adapter method | HypLoRA (lora_type: hyplora-1.0) |
| PEFT adapter type | LORA (custom HypLoRA implementation) |
Rank (r) |
32 |
LoRA alpha (lora_alpha) |
64 |
| Adapter files | adapter_config.json, adapter_model.safetensors |
Setup
Use the hf-checkpoint-v code branch.
git clone --branch hf-checkpoint-v https://github.com/marlin-codes/HypLoRA.git
cd HypLoRA
bash setup.sh
conda activate hyplora
The branch's setup.sh installs pinned dependencies and applies the project's custom HypLoRA PEFT changes. See the README for environment and runtime requirements.
Evaluation with this released adapter
In the cloned codebase, open scripts/table3/hyplora/gemma-7b/eval.sh and set its --lora_weights argument to:
--lora_weights "GGLabYale/hyplora-gemma-7b-math10k"
Then execute from the model-specific script directory:
cd scripts/table3/hyplora/gemma-7b/
bash eval.sh
Keep the other evaluation settings from the corresponding script.
Evaluation results
The following are evaluation results (accuracy) comparing standard LoRA and HypLoRA (K=1) for this base-model family and training task (higher is better).
| Benchmark | LoRA | HypLoRA (K=1) |
|---|---|---|
| AQuA | 0.437 | 0.515 |
| MAWPS | 0.87 | 0.928 |
| SVAMP | 0.72 | 0.774 |
| GSM8K | 0.615 | 0.741 |
| Params | 0.79% | 0.79% |
Citation
@inproceedings{yang2025hyperbolicfinetuning,
title = {Hyperbolic Fine-Tuning for Large Language Models},
author = {Menglin Yang and Ram Samarth B B and Aosong Feng and Bo Xiong and Jihong Liu and Irwin King and Rex Ying},
booktitle = {Advances in Neural Information Processing Systems},
volume = {38},
year = {2025},
url = {https://arxiv.org/abs/2410.04010}
}
- Downloads last month
- 46
Model tree for GGLabYale/hyplora-gemma-7b-math10k
Base model
google/gemma-7b