mohammad commited on
Commit
47555ea
·
verified ·
1 Parent(s): 31baece

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -14
README.md CHANGED
@@ -62,7 +62,7 @@ pipeline_tag: text-generation
62
  <img src="https://img.shields.io/badge/🎬 Video_Gen-🔜-yellow" alt="Video Gen">
63
  </p>
64
 
65
- <i>Research-assisted model selection · Quantized for consumer hardware · Ready to deploy</i>
66
 
67
  </div>
68
 
@@ -70,9 +70,13 @@ pipeline_tag: text-generation
70
 
71
  ## 📖 About This Toolkit
72
 
73
- This repository is a **hand-curated toolkit** of open-source AI models specifically selected and quantized to run entirely locally on consumer-grade hardware — specifically an **NVIDIA RTX 3060 (12GB VRAM) with 32GB system RAM**. Every model in this collection has been tested and verified to fit within these constraints while delivering the best possible output quality.
74
 
75
- The selection process was **research-assisted** — leveraging AI tools to systematically evaluate benchmarks, quantization trade-offs, and real-world performance across hundreds of candidate models before narrowing down to the ones you see here. The goal was simple: **maximum capability per GB of VRAM**.
 
 
 
 
76
 
77
  ### 🎯 Design Philosophy
78
 
@@ -80,8 +84,8 @@ Each category follows a dual-tier architecture:
80
 
81
  | Tier | Purpose | What It Means |
82
  |------|---------|---------------|
83
- | 🏆 **Quality** | Best possible output | The highest-quality model that still fits in 12GB VRAM with room for context |
84
- | ⚡ **Speed** | Fastest iteration | The fastest model that fits comfortably, ideal for rapid prototyping and quick tasks |
85
 
86
  > **Note on Coding models:** The Coding Agents category includes two quality-tier models (Architect & Executor + Detective & Debugger). The "speed" variants in that category, while labeled for speed, are still powerful enough to serve as competent rapid-response code assistants — they simply trade some reasoning depth for significantly faster token generation.
87
 
@@ -201,7 +205,7 @@ The "brains" of your agent network. These models decide which worker to call, wh
201
  |-------|------|-------------|------|
202
  | [Agents-A1 35B MoE](https://huggingface.co/Chungulus/Agents-A1-IQ4_XS-imatrix-gguf-fable5-calibrated) | 35B (MoE) | IQ4_XS | **CEO Agent** — high-level autonomous planning and task decomposition. Calibrated with imatrix for superior quantization fidelity |
203
  | [GLM-4.7-Flash](https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF) | Flash | Q4_K_XL | **Router** — ultra-fast model for routing decisions, intent classification, and dispatching tasks to appropriate workers |
204
- | [Hermes 4.3 36B](https://huggingface.co/NousResearch/Hermes-4.3-36B-GGUF) | 36B | Q4_K_M | **Quality Orchestrator** — the best-in-class model for complex multi-step reasoning, tool use, and agent coordination. Excellent function calling |
205
 
206
  ### ⚙️ Agentic Workers
207
 
@@ -272,7 +276,7 @@ RAG (Retrieval-Augmented Generation) is how you give your AI **access to your ow
272
  - **Sparse retrieval** — BM25-style keyword matching (great for exact term matches)
273
  - **ColBERT retrieval** — late interaction for fine-grained token-level relevance scoring
274
 
275
- This multi-modal retrieval approach means you get the best of both semantic understanding and keyword precision in a single model. It supports 100+ languages out of the box.
276
 
277
  ### 🔄 2. Reranker — [BGE-Reranker-V2-M3](https://huggingface.co/Heisenberg0314/bge-reranker-v2-m3)
278
 
@@ -376,7 +380,7 @@ A complete speech understanding pipeline that starts with audio preprocessing (c
376
  | 🎙️ Vocal Isolation | [Demucs V4 (ONNX)](https://huggingface.co/MrCitron/demucs-v4-onnx) | Studio-quality vocal separation — removes background music, noise, and other speakers. Essential for noisy recordings |
377
  | 🔇 Silence/Noise Cutter | [Silero VAD (ONNX)](https://huggingface.co/istupakov/silero-vad-onnx) | Voice Activity Detection — automatically cuts silence and non-speech segments, reducing processing time and improving transcription accuracy |
378
 
379
- ### 🏆 Quality Tier — Best Accuracy
380
 
381
  | Scope | Model | Size | Highlights |
382
  |-------|-------|------|------------|
@@ -503,9 +507,6 @@ These are **abliterated** (uncensored) versions of the models found in other cat
503
 
504
  All models in this toolkit are sourced from Hugging Face. During the initial download process, direct URLs were not recorded. The original sources were **recovered by matching SHA-256 hashes** against Hugging Face repositories. While some links may point to slightly different file versions within the same quantization family, the hash-verified content is identical to what is hosted in this toolkit.
505
 
506
- <details>
507
- <summary>🎯 Click to expand full source URL table</summary>
508
-
509
  ### 1 — Universal Foundation LLMs
510
 
511
  | File | Source |
@@ -592,8 +593,6 @@ All models in this toolkit are sourced from Hugging Face. During the initial dow
592
  | Huihui-Qwen3-VL-8B-Instruct-abliterated.Q8_0.gguf | [mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF](https://huggingface.co/mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF/blob/main/Huihui-Qwen3-VL-8B-Instruct-abliterated.Q8_0.gguf) |
593
  | Huihui-Qwen3-VL-8B-Instruct-abliterated.mmproj-f16.gguf | [mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF](https://huggingface.co/mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF/blob/main/Huihui-Qwen3-VL-8B-Instruct-abliterated.mmproj-f16.gguf) |
594
 
595
- </details>
596
-
597
  ---
598
 
599
  ## 🛠️ Recommended Tools for Running These Models
@@ -635,6 +634,6 @@ All models in this toolkit are the work of their respective creators and are use
635
 
636
  ---
637
 
638
- <i>Built with research-assisted model selection · Every model verified on RTX 3060 12GB</i>
639
 
640
  </div>
 
62
  <img src="https://img.shields.io/badge/🎬 Video_Gen-🔜-yellow" alt="Video Gen">
63
  </p>
64
 
65
+ <i>AI-assisted model selection · Quantized for consumer hardware · A suggested starting point</i>
66
 
67
  </div>
68
 
 
70
 
71
  ## 📖 About This Toolkit
72
 
73
+ This repository is a **suggested toolkit** of open-source AI models assembled to run entirely locally on consumer-grade hardware — specifically an **NVIDIA RTX 3060 (12GB VRAM) with 32GB system RAM**. Each model was chosen to fit within these constraints while offering strong output quality for the VRAM budget.
74
 
75
+ The selection process was **AI-assisted** — I consulted several AI assistants to weigh benchmarks, quantization trade-offs, and model capabilities, then settled on the common recommendations you see here. This is **not a definitively optimal combination**: better models may already exist, and stronger ones are released regularly. Treat it as a well-reasoned starting point rather than a final answer — your own testing on your own hardware is the real benchmark.
76
+
77
+ ### 🔄 Updates & Longevity
78
+
79
+ This toolkit is **periodically refreshed** as stronger or more efficient models are released. When a model gets clearly outclassed, it gets swapped out — so the lineup you see today may differ from a few months down the line.
80
 
81
  ### 🎯 Design Philosophy
82
 
 
84
 
85
  | Tier | Purpose | What It Means |
86
  |------|---------|---------------|
87
+ | 🏆 **Quality** | Strong output | A high-quality model that fits in 12GB VRAM with room for context |
88
+ | ⚡ **Speed** | Fast iteration | A fast model that fits comfortably, ideal for rapid prototyping and quick tasks |
89
 
90
  > **Note on Coding models:** The Coding Agents category includes two quality-tier models (Architect & Executor + Detective & Debugger). The "speed" variants in that category, while labeled for speed, are still powerful enough to serve as competent rapid-response code assistants — they simply trade some reasoning depth for significantly faster token generation.
91
 
 
205
  |-------|------|-------------|------|
206
  | [Agents-A1 35B MoE](https://huggingface.co/Chungulus/Agents-A1-IQ4_XS-imatrix-gguf-fable5-calibrated) | 35B (MoE) | IQ4_XS | **CEO Agent** — high-level autonomous planning and task decomposition. Calibrated with imatrix for superior quantization fidelity |
207
  | [GLM-4.7-Flash](https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF) | Flash | Q4_K_XL | **Router** — ultra-fast model for routing decisions, intent classification, and dispatching tasks to appropriate workers |
208
+ | [Hermes 4.3 36B](https://huggingface.co/NousResearch/Hermes-4.3-36B-GGUF) | 36B | Q4_K_M | **Quality Orchestrator** — a capable model for complex multi-step reasoning, tool use, and agent coordination. Excellent function calling |
209
 
210
  ### ⚙️ Agentic Workers
211
 
 
276
  - **Sparse retrieval** — BM25-style keyword matching (great for exact term matches)
277
  - **ColBERT retrieval** — late interaction for fine-grained token-level relevance scoring
278
 
279
+ This multi-modal retrieval approach means you combine semantic understanding and keyword precision in a single model. It supports 100+ languages out of the box.
280
 
281
  ### 🔄 2. Reranker — [BGE-Reranker-V2-M3](https://huggingface.co/Heisenberg0314/bge-reranker-v2-m3)
282
 
 
380
  | 🎙️ Vocal Isolation | [Demucs V4 (ONNX)](https://huggingface.co/MrCitron/demucs-v4-onnx) | Studio-quality vocal separation — removes background music, noise, and other speakers. Essential for noisy recordings |
381
  | 🔇 Silence/Noise Cutter | [Silero VAD (ONNX)](https://huggingface.co/istupakov/silero-vad-onnx) | Voice Activity Detection — automatically cuts silence and non-speech segments, reducing processing time and improving transcription accuracy |
382
 
383
+ ### 🏆 Quality Tier — High Accuracy
384
 
385
  | Scope | Model | Size | Highlights |
386
  |-------|-------|------|------------|
 
507
 
508
  All models in this toolkit are sourced from Hugging Face. During the initial download process, direct URLs were not recorded. The original sources were **recovered by matching SHA-256 hashes** against Hugging Face repositories. While some links may point to slightly different file versions within the same quantization family, the hash-verified content is identical to what is hosted in this toolkit.
509
 
 
 
 
510
  ### 1 — Universal Foundation LLMs
511
 
512
  | File | Source |
 
593
  | Huihui-Qwen3-VL-8B-Instruct-abliterated.Q8_0.gguf | [mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF](https://huggingface.co/mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF/blob/main/Huihui-Qwen3-VL-8B-Instruct-abliterated.Q8_0.gguf) |
594
  | Huihui-Qwen3-VL-8B-Instruct-abliterated.mmproj-f16.gguf | [mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF](https://huggingface.co/mradermacher/Huihui-Qwen3-VL-8B-Instruct-abliterated-GGUF/blob/main/Huihui-Qwen3-VL-8B-Instruct-abliterated.mmproj-f16.gguf) |
595
 
 
 
596
  ---
597
 
598
  ## 🛠️ Recommended Tools for Running These Models
 
634
 
635
  ---
636
 
637
+ <i>Assembled with AI-assisted model selection · Curated for RTX 3060 12GB · Updated as stronger models arrive</i>
638
 
639
  </div>