Spaces:
Running
Running
|
Download README.md from everxcloud/gpu-vram-calculator: direct link, hf CLI and curl.
- Browser
- Download file 1.14 kB
-
https://huggingface.co/spaces/everxcloud/gpu-vram-calculator/resolve/main/README.md
- Command line
-
hf download hf://spaces/everxcloud/gpu-vram-calculator/README.md
-
curl -L -o README.md https://huggingface.co/spaces/everxcloud/gpu-vram-calculator/resolve/main/README.md
1.14 kB
| title: LLM VRAM Calculator | |
| emoji: 🧮 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| pinned: false | |
| license: mit | |
| short_description: Estimate the GPU memory an LLM needs and the GPUs that fit | |
| # LLM VRAM Calculator | |
| Estimate how much GPU memory a large language model needs for inference (weights, KV cache and runtime overhead) and which GPUs can hold it alone or in 2, 4 or 8 cards. Runs entirely in your browser. | |
| Formulas: | |
| - Weights (GB) = billions of parameters x bytes per parameter (4 in FP32, 2 in BF16/FP16, 1 in FP8/INT8, 0.5 in INT4) | |
| - KV cache (GB) = 2 x layers x KV dimension x context tokens x batch x 2 bytes / 10^9 | |
| - Total = (weights + KV cache) x (1 + overhead %) | |
| Full calculator with GPU recommendations: https://everxcloud.ai/en/tools/llm-vram-calculator?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator | |
| Source code (MIT): https://github.com/everxcloud/gpu-vram-calculator | |
| Made by [EverXCloud](https://everxcloud.ai/?utm_source=huggingface&utm_medium=space&utm_campaign=vram-calculator), a US GPU marketplace to rent cloud GPUs by the hour or sell GPU capacity. | |