gpu-vram-calculator / README.md
everxcloud's picture
Update EverXCloud description: GPU marketplace
dbb6da9 verified
|
Raw History Blame Contribute Delete
1.14 kB
---
title: LLM VRAM Calculator
emoji: 🧮
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
license: mit
short_description: Estimate the GPU memory an LLM needs and the GPUs that fit
---
# LLM VRAM Calculator
Estimate how much GPU memory a large language model needs for inference (weights, KV cache and runtime overhead) and which GPUs can hold it alone or in 2, 4 or 8 cards. Runs entirely in your browser.
Formulas:
- Weights (GB) = billions of parameters x bytes per parameter (4 in FP32, 2 in BF16/FP16, 1 in FP8/INT8, 0.5 in INT4)
- KV cache (GB) = 2 x layers x KV dimension x context tokens x batch x 2 bytes / 10^9
- Total = (weights + KV cache) x (1 + overhead %)
Full calculator with GPU recommendations: https://everxcloud.ai/en/tools/llm-vram-calculator?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator
Source code (MIT): https://github.com/everxcloud/gpu-vram-calculator
Made by [EverXCloud](https://everxcloud.ai/?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator), a US GPU marketplace to rent cloud GPUs by the hour or sell GPU capacity.