NIM AI
NIM AI focuses on engineering compute-efficient, high-density reasoning models and autonomous agent runtimes designed to run locally on consumer hardware and edge devices.
Mission
Modern frontier models are powerful, but their infrastructure footprint limits widespread, private, and low-latency deployment. NIM AI builds purpose-driven Small Language Models (SLMs) that maximize logical density, deterministic execution, and token throughput per watt.
- Edge First: Built strictly within the bounds of consumer GPUs and unified memory architectures (sub-8 GB VRAM envelopes).
- Domain Specialization: Specialized model architectures tailored for autonomous software development, algorithmic decomposition, and deterministic tool use without generalist parameter taxes.
- Open & Deployable: Packaged in high-fidelity quantized formats (
Q8_0,Q4_K_MGGUF) for one-command integration with runtimes like Ollama andllama.cpp.
Flagship Releases
| Model | Size | Quant | Target Use Case | Deployment |
|---|---|---|---|---|
| NIM-1 | 3.09B | Q8_0 GGUF |
Low-latency triage, structured extraction, edge CLI intelligence | ollama run hf.co/N-I-M-AI/NIM-1-3B:NIM-1-3B-Q8_0.gguf |
| NIM-2 Coder | 7.61B | Q4_K_M GGUF |
Autonomous software engineering, algorithmic design, unit test synthesis | ollama run hf.co/N-I-M-AI/NIM-2-Coder-7B:NIM-2-Coder-7B-Q4_K_M.gguf |
Tech Stack & Architecture
- Fine-Tuning: Custom Triton backpropagation kernels, rank-stabilized QLoRA (
r=16,alpha=32), and ChatML instruction formatting. - Execution Engines: Ollama,
llama.cpp, Hugging Face Transformers. - Hardware Philosophy: Built and benchmarked directly on accessible consumer silicon (NVIDIA RTX 4060 8 GB / Apple Silicon) to guarantee reproducible local execution.
Connect & Contribute
- Hugging Face Hub: huggingface.co/N-I-M-AI
- GitHub Organization: github.com/N-I-M-AI
- License: Permissive open-source research and commercial deployment under the Apache 2.0 license.