Spaces:
Sleeping
Sleeping
File size: 1,200 Bytes
d08983c 2320bb1 d08983c 2320bb1 d08983c 2320bb1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | ---
title: TensorPEFT
emoji: 🧊
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Tensor-pursuit fine-tuning vs LoRA at tiny budgets
tags:
- peft
- fine-tuning
- tensor-decomposition
- lora-alternative
---
# TensorPEFT live demo
Live race between **TensorPEFT** (tensor gradient-pursuit adapters) and matched-budget PEFT baselines
(prompt tuning, LoRA, (IA)³, LayerNorm tuning): GPT-2 124M is fine-tuned on Bengali news with the same number of
trainable parameters for every method, and validation cross-entropy decides.
- Library: [Angshul/TensorPEFT](https://huggingface.co/Angshul/TensorPEFT) — `pip install git+https://huggingface.co/Angshul/TensorPEFT`
- Data: Bengali news articles from [soham96/Bengali_news_classifier](https://github.com/soham96/Bengali_news_classifier),
downloaded at start-up (not redistributed), tokenised with the GPT-2 tokenizer; 3M training tokens, held-out articles for validation.
- Base model: [openai-community/gpt2](https://huggingface.co/openai-community/gpt2).
Runs on ZeroGPU. Benchmark numbers in the *Benchmark results* tab were measured on CPU (fp32, 300 steps).
|