Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
- Website
- Community
- Solutions
Log In
Sign Up

mengfanxu

fxmeng

ishan-pandey's profile picture

Zhke's profile picture

densenet's profile picture

·

https://fxmeng.github.io

fxmeng

AI & ML interests

None yet

Organizations

None yet

fxmeng 's collections 9

Base Model for TransMLA

TransMLA: Multi-head Latent Attention Is All You Need

Paper • 2502.07864 • Published Feb 11, 2025 • 69
fxmeng/transmla_pretrain_6B_tokens

Viewer • Updated Jul 5, 2025 • 5.94M • 147
fxmeng/transmla_pretrain_1B_tokens

Viewer • Updated Jul 5, 2025 • 1.14M • 23
fxmeng/transmla_pretrain_100m_tokens

Viewer • Updated Jul 5, 2025 • 100k • 12

PiSSA-LLaMA-3-8B

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-3-8B-r16

Text Generation • 8B • Updated May 9, 2024 • 18
fxmeng/PiSSA-Llama-3-8B-r64

Text Generation • 8B • Updated May 9, 2024 • 18
fxmeng/PiSSA-Llama-3-8B-r32

Text Generation • 8B • Updated May 9, 2024 • 16
fxmeng/PiSSA-Llama-3-8B-r128

Text Generation • 8B • Updated May 9, 2024 • 25 • 1

PiSSA-LLaMA-2-7B

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-2-7B-r16-4bit-5iter

Text Generation • 7B • Updated Apr 26, 2024 • 16
fxmeng/PiSSA-Llama-2-7B-r32-4bit-5iter

Text Generation • 7B • Updated Apr 26, 2024 • 16
fxmeng/PiSSA-Llama-2-7B-r64-4bit-5iter

Text Generation • 7B • Updated Apr 27, 2024 • 15
fxmeng/PiSSA-Llama-2-7B-r128-4bit-5iter

Text Generation • 7B • Updated Apr 27, 2024 • 19

fxmeng/PiSSA-Qwen2-7B-4bit-r128-5iter

Text Generation • 8B • Updated Jun 14, 2024 • 20
fxmeng/PiSSA-Qwen2-7B-Instruct-4bit-r128-5iter

Text Generation • 8B • Updated Jun 12, 2024 • 13
fxmeng/PiSSA-Qwen2-7B-r128

Text Generation • 8B • Updated Jun 13, 2024 • 18
fxmeng/PiSSA-Qwen2-7B-Instruct-r128

Text Generation • 8B • Updated Jun 14, 2024 • 15

Mixtral-1~8x7B-Instruct-v0.1

Substructure of mistralai/Mixtral-8x7B-Instruct-v0.1

fxmeng/Mixtral-1x7B-Instruct-v0.1

Text Generation • 7B • Updated Dec 28, 2023 • 16
fxmeng/Mixtral-2x7B-Instruct-v0.1

Text Generation • 13B • Updated Dec 28, 2023 • 19
fxmeng/Mixtral-3x7B-Instruct-v0.1

Text Generation • 19B • Updated Dec 28, 2023 • 22
fxmeng/Mixtral-4x7B-Instruct-v0.1

Text Generation • 24B • Updated Dec 28, 2023 • 21 • 1

CLOVER-Commonsense-148k

CLOVER: Constrained Learning with Orthonormal Vectors for Eliminating Redundancy

Paper • 2411.17426 • Published Nov 26, 2024
fxmeng/CLOVER-llama-2-7b-commonsense-148k

7B • Updated Feb 2, 2025 • 3
fxmeng/CLOVER-llama-13b-commonsense-148k

13B • Updated Feb 2, 2025 • 1
fxmeng/CLOVER-llama-3-8b-commonsense-148k

8B • Updated Feb 2, 2025 • 2

PiSSA-LLaMA-3-70B

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-3-70B-4bit-r64-5iter

Updated May 1, 2024
fxmeng/PiSSA-Llama-3-70B-4bit-r128-5iter

Text Generation • 72B • Updated May 1, 2024 • 25
fxmeng/PiSSA-Llama-3-70B-4bit-r64-1iter

Updated May 1, 2024
fxmeng/PiSSA-Llama-3-70B-4bit-r128-1iter

Updated May 1, 2024

PiSSA-LLaMA-3-8B-Instruct

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-3-8B-Instruct-r16

Text Generation • 8B • Updated Apr 26, 2024 • 22
fxmeng/PiSSA-Llama-3-8B-Instruct-r32

Text Generation • 8B • Updated Apr 26, 2024 • 21
fxmeng/PiSSA-Llama-3-8B-Instruct-r64

Text Generation • 8B • Updated Apr 26, 2024 • 20 • 1
fxmeng/PiSSA-Llama-3-8B-Instruct-r128

Text Generation • 8B • Updated Apr 26, 2024 • 12

https://arxiv.org/abs/2404.02948

fxmeng/pissa-dataset

Viewer • Updated Jan 8, 2025 • 844k • 1.59k • 3
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models

Paper • 2404.02948 • Published Apr 3, 2024 • 5

Base Model for TransMLA

TransMLA: Multi-head Latent Attention Is All You Need

Paper • 2502.07864 • Published Feb 11, 2025 • 69
fxmeng/transmla_pretrain_6B_tokens

Viewer • Updated Jul 5, 2025 • 5.94M • 147
fxmeng/transmla_pretrain_1B_tokens

Viewer • Updated Jul 5, 2025 • 1.14M • 23
fxmeng/transmla_pretrain_100m_tokens

Viewer • Updated Jul 5, 2025 • 100k • 12

CLOVER-Commonsense-148k

CLOVER: Constrained Learning with Orthonormal Vectors for Eliminating Redundancy

Paper • 2411.17426 • Published Nov 26, 2024
fxmeng/CLOVER-llama-2-7b-commonsense-148k

7B • Updated Feb 2, 2025 • 3
fxmeng/CLOVER-llama-13b-commonsense-148k

13B • Updated Feb 2, 2025 • 1
fxmeng/CLOVER-llama-3-8b-commonsense-148k

8B • Updated Feb 2, 2025 • 2

PiSSA-LLaMA-3-8B

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-3-8B-r16

Text Generation • 8B • Updated May 9, 2024 • 18
fxmeng/PiSSA-Llama-3-8B-r64

Text Generation • 8B • Updated May 9, 2024 • 18
fxmeng/PiSSA-Llama-3-8B-r32

Text Generation • 8B • Updated May 9, 2024 • 16
fxmeng/PiSSA-Llama-3-8B-r128

Text Generation • 8B • Updated May 9, 2024 • 25 • 1

PiSSA-LLaMA-3-70B

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-3-70B-4bit-r64-5iter

Updated May 1, 2024
fxmeng/PiSSA-Llama-3-70B-4bit-r128-5iter

Text Generation • 72B • Updated May 1, 2024 • 25
fxmeng/PiSSA-Llama-3-70B-4bit-r64-1iter

Updated May 1, 2024
fxmeng/PiSSA-Llama-3-70B-4bit-r128-1iter

Updated May 1, 2024

PiSSA-LLaMA-2-7B

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-2-7B-r16-4bit-5iter

Text Generation • 7B • Updated Apr 26, 2024 • 16
fxmeng/PiSSA-Llama-2-7B-r32-4bit-5iter

Text Generation • 7B • Updated Apr 26, 2024 • 16
fxmeng/PiSSA-Llama-2-7B-r64-4bit-5iter

Text Generation • 7B • Updated Apr 27, 2024 • 15
fxmeng/PiSSA-Llama-2-7B-r128-4bit-5iter

Text Generation • 7B • Updated Apr 27, 2024 • 19

PiSSA-LLaMA-3-8B-Instruct

Principal Singular Values and Singular Vectors Adaptation

fxmeng/PiSSA-Llama-3-8B-Instruct-r16

Text Generation • 8B • Updated Apr 26, 2024 • 22
fxmeng/PiSSA-Llama-3-8B-Instruct-r32

Text Generation • 8B • Updated Apr 26, 2024 • 21
fxmeng/PiSSA-Llama-3-8B-Instruct-r64

Text Generation • 8B • Updated Apr 26, 2024 • 20 • 1
fxmeng/PiSSA-Llama-3-8B-Instruct-r128

Text Generation • 8B • Updated Apr 26, 2024 • 12

fxmeng/PiSSA-Qwen2-7B-4bit-r128-5iter

Text Generation • 8B • Updated Jun 14, 2024 • 20
fxmeng/PiSSA-Qwen2-7B-Instruct-4bit-r128-5iter

Text Generation • 8B • Updated Jun 12, 2024 • 13
fxmeng/PiSSA-Qwen2-7B-r128

Text Generation • 8B • Updated Jun 13, 2024 • 18
fxmeng/PiSSA-Qwen2-7B-Instruct-r128

Text Generation • 8B • Updated Jun 14, 2024 • 15

https://arxiv.org/abs/2404.02948

fxmeng/pissa-dataset

Viewer • Updated Jan 8, 2025 • 844k • 1.59k • 3
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models

Paper • 2404.02948 • Published Apr 3, 2024 • 5

Mixtral-1~8x7B-Instruct-v0.1

Substructure of mistralai/Mixtral-8x7B-Instruct-v0.1

fxmeng/Mixtral-1x7B-Instruct-v0.1

Text Generation • 7B • Updated Dec 28, 2023 • 16
fxmeng/Mixtral-2x7B-Instruct-v0.1

Text Generation • 13B • Updated Dec 28, 2023 • 19
fxmeng/Mixtral-3x7B-Instruct-v0.1

Text Generation • 19B • Updated Dec 28, 2023 • 22
fxmeng/Mixtral-4x7B-Instruct-v0.1

Text Generation • 24B • Updated Dec 28, 2023 • 21 • 1

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs