--- title: LLM VRAM Calculator emoji: 🧮 colorFrom: indigo colorTo: blue sdk: static pinned: false license: mit short_description: Estimate the GPU memory an LLM needs and the GPUs that fit --- # LLM VRAM Calculator Estimate how much GPU memory a large language model needs for inference (weights, KV cache and runtime overhead) and which GPUs can hold it alone or in 2, 4 or 8 cards. Runs entirely in your browser. Formulas: - Weights (GB) = billions of parameters x bytes per parameter (4 in FP32, 2 in BF16/FP16, 1 in FP8/INT8, 0.5 in INT4) - KV cache (GB) = 2 x layers x KV dimension x context tokens x batch x 2 bytes / 10^9 - Total = (weights + KV cache) x (1 + overhead %) Full calculator with GPU recommendations: https://everxcloud.ai/en/tools/llm-vram-calculator?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator Source code (MIT): https://github.com/everxcloud/gpu-vram-calculator Made by [EverXCloud](https://everxcloud.ai/?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator), a US GPU marketplace to rent cloud GPUs by the hour or sell GPU capacity.