Best Practices for Managing Large Model Weights and Storage Overheads

#43
by ameliahebrew - opened

Hi everyone — I’m working on a project involving multiple large transformer models (hundreds of GB total) hosted on the Hugging Face Hub + in Spaces. I’m running into issues with storage costs and speed when pulling/downloading weights, especially when switching between environments.

Some approaches I’ve looked into so far:

Hugging Face’s built-in model caching system

External object storage (e.g., AWS S3, GCS)

Compression techniques like quantization or pruning

Delta updates (only syncing changes)

I know some teams here have experimented with advanced storage solutions to handle both frequent inference and occasional large checkpoints. Could you share what has worked best for you in terms of balancing cost, speed, and reliability? Thanks in advance!

Sign up or log in to comment