Spaces:
Running
Running
|
Download README.md from everxcloud/gpu-vram-calculator: direct link, hf CLI and curl.
- Browser
- Download file 1.14 kB
-
https://huggingface.co/spaces/everxcloud/gpu-vram-calculator/resolve/main/README.md
- Command line
-
hf download hf://spaces/everxcloud/gpu-vram-calculator/README.md
-
curl -L -o README.md https://huggingface.co/spaces/everxcloud/gpu-vram-calculator/resolve/main/README.md
1.14 kB
metadata
title: LLM VRAM Calculator
emoji: 🧮
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
license: mit
short_description: Estimate the GPU memory an LLM needs and the GPUs that fit
LLM VRAM Calculator
Estimate how much GPU memory a large language model needs for inference (weights, KV cache and runtime overhead) and which GPUs can hold it alone or in 2, 4 or 8 cards. Runs entirely in your browser.
Formulas:
- Weights (GB) = billions of parameters x bytes per parameter (4 in FP32, 2 in BF16/FP16, 1 in FP8/INT8, 0.5 in INT4)
- KV cache (GB) = 2 x layers x KV dimension x context tokens x batch x 2 bytes / 10^9
- Total = (weights + KV cache) x (1 + overhead %)
Full calculator with GPU recommendations: https://everxcloud.ai/en/tools/llm-vram-calculator?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator
Source code (MIT): https://github.com/everxcloud/gpu-vram-calculator
Made by EverXCloud, a US GPU marketplace to rent cloud GPUs by the hour or sell GPU capacity.