Delete README.md with huggingface_hub
Browse files
README.md
DELETED
|
@@ -1,83 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
---
|
| 4 |
-
|
| 5 |
-
# LLamaOptimizerAuto
|
| 6 |
-
|
| 7 |
-
LLama.cpp optimizer toolkit with CUDA support for Windows.
|
| 8 |
-
|
| 9 |
-
## Contents
|
| 10 |
-
|
| 11 |
-
This repository contains pre-built llama.cpp binaries with GPU (CUDA 12) and multi-CPU architecture support.
|
| 12 |
-
|
| 13 |
-
### Core Binaries
|
| 14 |
-
|
| 15 |
-
| File | Description |
|
| 16 |
-
|------|-------------|
|
| 17 |
-
| `llama-server.exe` | OpenAI-compatible API server |
|
| 18 |
-
| `llama-cli.exe` | Command-line inference |
|
| 19 |
-
| `llama-bench.exe` | Benchmarking tool |
|
| 20 |
-
| `llama-quantize.exe` | Model quantization |
|
| 21 |
-
| `llama-imatrix.exe` | Importance matrix computation |
|
| 22 |
-
| `llama-gguf-split.exe` | GGUF file splitting/joining |
|
| 23 |
-
| `llama-perplexity.exe` | Perplexity calculation |
|
| 24 |
-
| `llama-tokenize.exe` | Tokenization utility |
|
| 25 |
-
| `llama-completion.exe` | Text completion |
|
| 26 |
-
| `llama-batched-bench.exe` | Batched benchmarking |
|
| 27 |
-
| `llama-llava-cli.exe` | LLaVA multimodal CLI |
|
| 28 |
-
| `llama-minicpmv-cli.exe` | MiniCPM-V multimodal CLI |
|
| 29 |
-
| `llama-mtmd-cli.exe` | Multimodal CLI |
|
| 30 |
-
| `llama-qwen2vl-cli.exe` | Qwen2-VL CLI |
|
| 31 |
-
| `llama-gemma3-cli.exe` | Gemma 3 CLI |
|
| 32 |
-
| `llama-tts.exe` | Text-to-speech |
|
| 33 |
-
| `llama-results.exe` | Results utility |
|
| 34 |
-
| `llama-template-analysis.exe` | Template analysis |
|
| 35 |
-
| `llama-fit-params.exe` | Parameter fitting |
|
| 36 |
-
| `llama-mtmd-debug.exe` | Multimodal debug |
|
| 37 |
-
| `rpc-server.exe` | RPC server for distributed inference |
|
| 38 |
-
|
| 39 |
-
### GPU Support
|
| 40 |
-
|
| 41 |
-
- `ggml-cuda.dll` — CUDA 12 backend
|
| 42 |
-
- `cublas64_12.dll`, `cublasLt64_12.dll`, `cudart64_12.dll` — CUDA 12 libraries
|
| 43 |
-
|
| 44 |
-
### CPU Architecture Support
|
| 45 |
-
|
| 46 |
-
| DLL | Architecture |
|
| 47 |
-
|-----|-------------|
|
| 48 |
-
| `ggml-cpu-x64.dll` | Generic x86-64 |
|
| 49 |
-
| `ggml-cpu-sse42.dll` | SSE 4.2 |
|
| 50 |
-
| `ggml-cpu-ivybridge.dll` | Ivy Bridge |
|
| 51 |
-
| `ggml-cpu-sandybridge.dll` | Sandy Bridge |
|
| 52 |
-
| `ggml-cpu-haswell.dll` | Haswell |
|
| 53 |
-
| `ggml-cpu-skylakex.dll` | Skylake-X |
|
| 54 |
-
| `ggml-cpu-cascadelake.dll` | Cascade Lake |
|
| 55 |
-
| `ggml-cpu-cooperlake.dll` | Cooper Lake |
|
| 56 |
-
| `ggml-cpu-icelake.dll` | Ice Lake |
|
| 57 |
-
| `ggml-cpu-cannonlake.dll` | Cannon Lake |
|
| 58 |
-
| `ggml-cpu-alderlake.dll` | Alder Lake |
|
| 59 |
-
| `ggml-cpu-sapphirerapids.dll` | Sapphire Rapids |
|
| 60 |
-
| `ggml-cpu-piledriver.dll` | AMD Piledriver |
|
| 61 |
-
| `ggml-cpu-zen4.dll` | AMD Zen 4 |
|
| 62 |
-
|
| 63 |
-
### Optimizer Tools
|
| 64 |
-
|
| 65 |
-
- `LLamaOptimizer.exe` — Auto optimizer for llama.cpp configurations
|
| 66 |
-
- `LlamaOptGUI(1).exe` — GUI version of the optimizer
|
| 67 |
-
|
| 68 |
-
## Quick Start
|
| 69 |
-
|
| 70 |
-
1. Download the files
|
| 71 |
-
2. Run `llama-server.exe` with your GGUF model:
|
| 72 |
-
|
| 73 |
-
```bash
|
| 74 |
-
llama-server.exe -m your_model.gguf -ngl 99 --host 0.0.0.0 --port 8080
|
| 75 |
-
```
|
| 76 |
-
|
| 77 |
-
3. Access the API at `http://localhost:8080`
|
| 78 |
-
|
| 79 |
-
## Requirements
|
| 80 |
-
|
| 81 |
-
- Windows x64
|
| 82 |
-
- NVIDIA GPU with CUDA 12 support (for GPU acceleration)
|
| 83 |
-
- Sufficient RAM/VRAM for your model
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|