What If the Adaptation Were a Model? ShadowPEFT in 🤗 PEFT library
• 24
AI & ML interests
Shadow LLM team from PolyU & LNU
Recent Activity
View all activity
Articles
SeanLee97
published a
Space about 12 hours ago
shadow-seven
updated a
Space about 12 hours ago
shadow-seven
posted an update 20 days ago
Post
69
What If the Adaptation Were a Model? ShadowPEFT has been integrated into the 🤗 PEFT library. Blogpost: https://huggingface.co/blog/shadow-llm/shadowpeft-peft
Article
What If the Adaptation Were a Model? ShadowPEFT in 🤗 PEFT library
shadow-llm
• • 24 Article
适配器即模型 —— ShadowPEFT 现已集成到 🤗 PEFT 库
shadow-llm
• • 16 shadow-seven
updated a
model about 2 months ago
shadow-seven
published a
model about 2 months ago
Add model card and robotics tag
🤗❤️ 1
#1 opened 6 months ago
by 
nielsr
Add paper link, task categories and GitHub repository
🤗 2
1
#2 opened 6 months ago
by nielsr
Improve model card: add metadata, documentation and sample usage
🤗 1
1
#1 opened 6 months ago
by nielsr
SeanLee97
submitted a
paper to Daily Papers 6 months ago
csroyli
authored 4
papers 6 months ago
Label Supervised LLaMA Finetuning
Paper • 2310.01208 • Published • 1
Recurrent Attention Networks for Long-text Modeling
Paper • 2306.06843 • Published
2D Matryoshka Sentence Embeddings
Paper • 2402.14776 • Published • 8
CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering
Paper • 2502.01523 • Published • 2
SeanLee97
authored a
paper 6 months ago
csroyli
authored a
paper 6 months ago
Post
8328
Our lab recently released a paper where we introduce ShadowPEFT, a new Parameter-Efficient Fine-Tuning (PEFT) paradigm tailored for edge computing scenarios.
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (2604.19254)
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (2604.19254)
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
- 7 replies
shadow-seven
updated a
collection 6 months ago
shadow-seven
updated a
model 6 months ago