Instructions to use MatterWaveAI/grt-30m-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MatterWaveAI/grt-30m-lora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MatterWaveAI/grt-30m-lora")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("MatterWaveAI/grt-30m-lora", device_map="auto")

PEFT
How to use MatterWaveAI/grt-30m-lora with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use MatterWaveAI/grt-30m-lora with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MatterWaveAI/grt-30m-lora:F16
# Run inference directly in the terminal:
llama cli -hf MatterWaveAI/grt-30m-lora:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MatterWaveAI/grt-30m-lora:F16
# Run inference directly in the terminal:
llama cli -hf MatterWaveAI/grt-30m-lora:F16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf MatterWaveAI/grt-30m-lora:F16
# Run inference directly in the terminal:
./build/bin/llama cli -hf MatterWaveAI/grt-30m-lora:F16

Use Docker

docker model run hf.co/MatterWaveAI/grt-30m-lora:F16

Llama new
LM Studio
Jan

vLLM

How to use MatterWaveAI/grt-30m-lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MatterWaveAI/grt-30m-lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-30m-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MatterWaveAI/grt-30m-lora:F16

SGLang

How to use MatterWaveAI/grt-30m-lora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MatterWaveAI/grt-30m-lora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-30m-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MatterWaveAI/grt-30m-lora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-30m-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Ollama
How to use MatterWaveAI/grt-30m-lora with Ollama:
```
ollama run hf.co/MatterWaveAI/grt-30m-lora:F16
```
Unsloth Desktop
Docker Model Runner
How to use MatterWaveAI/grt-30m-lora with Docker Model Runner:
```
docker model run hf.co/MatterWaveAI/grt-30m-lora:F16
```

Lemonade

How to use MatterWaveAI/grt-30m-lora with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull MatterWaveAI/grt-30m-lora:F16

Run and chat with the model

lemonade run user.grt-30m-lora-F16

List all available models

lemonade list

Atomic Chat

grt-30m-lora / eval_results.json

davidhouseholter

Add 8-way classical geometric GRTRotationalLoRA optimizer ablation: eval_results.json

4624377 verified 9 days ago

Raw History Blame Contribute Delete

4.53 kB

{
  "model_id": "MatterWaveAI/grt-30m-lora",
  "adapter_class": "GRTRotationalLoRA",
  "base_model": "MatterWaveAI/grt-30m",
  "optimizer": "HessianPairAdam (Sophia-Torus)",
  "base_parameters": 29305498,
  "trainable_parameters": 140741,
  "total_parameters_with_lora": 29436730,
  "trainable_pct": 0.4803,
  "lora_rank": 8,
  "lora_alpha": 16.0,
  "lora_scaling": 2.0,
  "nn_linear_layers": 0,
  "nn_conv1d_layers": 0,
  "step0_identity_max_abs_diff": 0.0,
  "merge_max_abs_diff": 0.0,
  "reload_max_abs_diff": 0.0,
  "adapter_file_bytes": 587108,
  "training_steps": 80,
  "before_adaptation": {
    "assistant_loss": 6.172,
    "perplexity": 479.1388,
    "token_accuracy": 0.1954
  },
  "after_adaptation": {
    "assistant_loss": 4.7464,
    "perplexity": 115.1645,
    "token_accuracy": 0.2883
  },
  "sample_prompt": "Explain how GRTRotationalLoRA adapts a Geometric Rotation Transformer without linear layers.",
  "sample_response": "In each 2. equal 2^12^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^",
  "optimizer_ablations_8way": {
    "pair_adam": {
      "optimizer_type": "pair_adam",
      "variant": "PairAdam (Factored Toroidal Baseline)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 5.0931,
      "after_perplexity": 162.8913,
      "after_token_accuracy": 0.2134,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 12.96
    },
    "hessian_pair_adam": {
      "optimizer_type": "hessian_pair_adam",
      "variant": "HessianPairAdam (Sophia-Torus Cauchy-Riemann Curvature)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 4.5159,
      "after_perplexity": 91.4626,
      "after_token_accuracy": 0.3062,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 13.26
    },
    "polar_natural_adam": {
      "optimizer_type": "polar_natural_adam",
      "variant": "PolarNaturalAdam (Riemannian Polar Metric + Cauchy-Riemann Hessian)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 4.5078,
      "after_perplexity": 90.7201,
      "after_token_accuracy": 0.3078,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 13.34
    },
    "cyclide_adam": {
      "optimizer_type": "cyclide_adam",
      "variant": "CyclideAdam (Dupin Cyclide Conic Metric + Deviatoric Twist)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 4.6722,
      "after_perplexity": 106.9343,
      "after_token_accuracy": 0.2899,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 13.72
    },
    "symplectic_adam": {
      "optimizer_type": "symplectic_adam",
      "variant": "SymplecticAdam (Gromov-Liouville Non-Squeezing Floor + Casimir Cavity)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 3.9882,
      "after_perplexity": 53.959,
      "after_token_accuracy": 0.3534,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 13.74
    },
    "gyric_adam": {
      "optimizer_type": "gyric_adam",
      "variant": "GyricAdam (Poinsot-Lande Gyroscopic Projection + Thomas Precession)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 4.3119,
      "after_perplexity": 74.5784,
      "after_token_accuracy": 0.3176,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 13.82
    },
    "huygens_adam": {
      "optimizer_type": "huygens_adam",
      "variant": "HuygensAdam (Circular Order Parameter R_p Splay Torque)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 4.9113,
      "after_perplexity": 135.813,
      "after_token_accuracy": 0.2541,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 13.5
    },
    "huygens_symplectic_adam": {
      "optimizer_type": "huygens_symplectic_adam",
      "variant": "HuygensSymplecticAdam (Two-Stage Huygens -> Symplectic Curriculum)",
      "before_loss": 6.172,
      "before_perplexity": 479.1384,
      "before_token_accuracy": 0.1954,
      "after_loss": 4.5764,
      "after_perplexity": 97.1671,
      "after_token_accuracy": 0.3127,
      "merge_max_abs_diff": 0.0,
      "elapsed_sec": 14.22
    }
  }
}

	{
	"model_id": "MatterWaveAI/grt-30m-lora",
	"adapter_class": "GRTRotationalLoRA",
	"base_model": "MatterWaveAI/grt-30m",
	"optimizer": "HessianPairAdam (Sophia-Torus)",
	"base_parameters": 29305498,
	"trainable_parameters": 140741,
	"total_parameters_with_lora": 29436730,
	"trainable_pct": 0.4803,
	"lora_rank": 8,
	"lora_alpha": 16.0,
	"lora_scaling": 2.0,
	"nn_linear_layers": 0,
	"nn_conv1d_layers": 0,
	"step0_identity_max_abs_diff": 0.0,
	"merge_max_abs_diff": 0.0,
	"reload_max_abs_diff": 0.0,
	"adapter_file_bytes": 587108,
	"training_steps": 80,
	"before_adaptation": {
	"assistant_loss": 6.172,
	"perplexity": 479.1388,
	"token_accuracy": 0.1954
	},
	"after_adaptation": {
	"assistant_loss": 4.7464,
	"perplexity": 115.1645,
	"token_accuracy": 0.2883
	},
	"sample_prompt": "Explain how GRTRotationalLoRA adapts a Geometric Rotation Transformer without linear layers.",
	"sample_response": "In each 2. equal 2^12^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^1^",
	"optimizer_ablations_8way": {
	"pair_adam": {
	"optimizer_type": "pair_adam",
	"variant": "PairAdam (Factored Toroidal Baseline)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 5.0931,
	"after_perplexity": 162.8913,
	"after_token_accuracy": 0.2134,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 12.96
	},
	"hessian_pair_adam": {
	"optimizer_type": "hessian_pair_adam",
	"variant": "HessianPairAdam (Sophia-Torus Cauchy-Riemann Curvature)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 4.5159,
	"after_perplexity": 91.4626,
	"after_token_accuracy": 0.3062,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 13.26
	},
	"polar_natural_adam": {
	"optimizer_type": "polar_natural_adam",
	"variant": "PolarNaturalAdam (Riemannian Polar Metric + Cauchy-Riemann Hessian)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 4.5078,
	"after_perplexity": 90.7201,
	"after_token_accuracy": 0.3078,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 13.34
	},
	"cyclide_adam": {
	"optimizer_type": "cyclide_adam",
	"variant": "CyclideAdam (Dupin Cyclide Conic Metric + Deviatoric Twist)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 4.6722,
	"after_perplexity": 106.9343,
	"after_token_accuracy": 0.2899,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 13.72
	},
	"symplectic_adam": {
	"optimizer_type": "symplectic_adam",
	"variant": "SymplecticAdam (Gromov-Liouville Non-Squeezing Floor + Casimir Cavity)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 3.9882,
	"after_perplexity": 53.959,
	"after_token_accuracy": 0.3534,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 13.74
	},
	"gyric_adam": {
	"optimizer_type": "gyric_adam",
	"variant": "GyricAdam (Poinsot-Lande Gyroscopic Projection + Thomas Precession)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 4.3119,
	"after_perplexity": 74.5784,
	"after_token_accuracy": 0.3176,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 13.82
	},
	"huygens_adam": {
	"optimizer_type": "huygens_adam",
	"variant": "HuygensAdam (Circular Order Parameter R_p Splay Torque)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 4.9113,
	"after_perplexity": 135.813,
	"after_token_accuracy": 0.2541,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 13.5
	},
	"huygens_symplectic_adam": {
	"optimizer_type": "huygens_symplectic_adam",
	"variant": "HuygensSymplecticAdam (Two-Stage Huygens -> Symplectic Curriculum)",
	"before_loss": 6.172,
	"before_perplexity": 479.1384,
	"before_token_accuracy": 0.1954,
	"after_loss": 4.5764,
	"after_perplexity": 97.1671,
	"after_token_accuracy": 0.3127,
	"merge_max_abs_diff": 0.0,
	"elapsed_sec": 14.22
	}
	}
	}