Text Generation
Transformers
GGUF
PEFT
grt
lora
toroidal-lora
rotational-transformer
givens-rotations
clifford-algebra
sophia-torus
matterwave
Instructions to use MatterWaveAI/grt-30m-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MatterWaveAI/grt-30m-lora with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MatterWaveAI/grt-30m-lora")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("MatterWaveAI/grt-30m-lora", device_map="auto") - PEFT
How to use MatterWaveAI/grt-30m-lora with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use MatterWaveAI/grt-30m-lora with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf MatterWaveAI/grt-30m-lora:F16 # Run inference directly in the terminal: llama cli -hf MatterWaveAI/grt-30m-lora:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf MatterWaveAI/grt-30m-lora:F16 # Run inference directly in the terminal: llama cli -hf MatterWaveAI/grt-30m-lora:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf MatterWaveAI/grt-30m-lora:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf MatterWaveAI/grt-30m-lora:F16
Use Docker
docker model run hf.co/MatterWaveAI/grt-30m-lora:F16
- Llama new
- LM Studio
- Jan
- vLLM
How to use MatterWaveAI/grt-30m-lora with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MatterWaveAI/grt-30m-lora" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MatterWaveAI/grt-30m-lora", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MatterWaveAI/grt-30m-lora:F16
- SGLang
How to use MatterWaveAI/grt-30m-lora with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MatterWaveAI/grt-30m-lora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MatterWaveAI/grt-30m-lora", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MatterWaveAI/grt-30m-lora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MatterWaveAI/grt-30m-lora", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use MatterWaveAI/grt-30m-lora with Ollama:
ollama run hf.co/MatterWaveAI/grt-30m-lora:F16
- Unsloth Desktop
- Docker Model Runner
How to use MatterWaveAI/grt-30m-lora with Docker Model Runner:
docker model run hf.co/MatterWaveAI/grt-30m-lora:F16
- Lemonade
How to use MatterWaveAI/grt-30m-lora with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull MatterWaveAI/grt-30m-lora:F16
Run and chat with the model
lemonade run user.grt-30m-lora-F16
List all available models
lemonade list
- Atomic Chat
|
Download PROVENANCE.md from MatterWaveAI/grt-30m-lora: direct link, hf CLI and curl.
- Browser
- Download file 547 Bytes
-
https://huggingface.co/MatterWaveAI/grt-30m-lora/resolve/main/PROVENANCE.md
- Command line
-
hf download hf://MatterWaveAI/grt-30m-lora/PROVENANCE.md
-
curl -L -o PROVENANCE.md https://huggingface.co/MatterWaveAI/grt-30m-lora/resolve/main/PROVENANCE.md
547 Bytes
| # Provenance: `MatterWaveAI/grt-30m-lora` | |
| - **Base Model**: `MatterWaveAI/grt-30m` (`release/30m/model.safetensors`) | |
| - **Adapter Class**: `GRTRotationalLoRA` (`r=8`, `lora_alpha=16.0`, `tune_scales=True`) | |
| - **Optimizer**: `HessianPairAdam` (`Sophia-Torus`, `lr=1.5e-2`, `80` steps) | |
| - **Build Script**: `release/build_30m_lora.py` | |
| - **Before Adaptation**: loss `6.172`, perplexity `479.1388` | |
| - **After Adaptation**: loss `4.7464`, perplexity `115.1645` | |
| - **Merge Equivalence**: `max_abs_diff = 0.0e+00` (`0` `nn.Linear`, `0` `nn.Conv1d`) | |