README.md

metadata

title: TensorPEFT
emoji: 🧊
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Tensor-pursuit fine-tuning vs LoRA at tiny budgets
tags:
  - peft
  - fine-tuning
  - tensor-decomposition
  - lora-alternative

TensorPEFT live demo

Live race between TensorPEFT (tensor gradient-pursuit adapters) and matched-budget PEFT baselines (prompt tuning, LoRA, (IA)³, LayerNorm tuning): GPT-2 124M is fine-tuned on Bengali news with the same number of trainable parameters for every method, and validation cross-entropy decides.

• Library: Angshul/TensorPEFT — pip install git+https://huggingface.co/Angshul/TensorPEFT
• Data: Bengali news articles from soham96/Bengali_news_classifier, downloaded at start-up (not redistributed), tokenised with the GPT-2 tokenizer; 3M training tokens, held-out articles for validation.
• Base model: openai-community/gpt2.

Runs on ZeroGPU. Benchmark numbers in the Benchmark results tab were measured on CPU (fp32, 300 steps).