README.md

---
license: apache-2.0
language:
  - en
tags:
  - cybersecurity
  - pentesting
  - security
  - uncensored
  - tool-calling
  - gguf
  - llama-cpp
pipeline_tag: text-generation
---

# Nesi2 — Uncensored AI for Cybersecurity (GGUF / llama.cpp)

**Nesi2** is an uncensored large language model, curated and optimized by
**Nesilabs** specifically for **offensive and defensive cybersecurity**:
penetration testing, vulnerability analysis, exploit development, secure
code review, and audit automation.

It is packaged by Nesilabs as a **GGUF** build for **llama.cpp**, so it runs on a
single GPU without an AWQ/vLLM stack. This is the build we run as an autonomous
**pentest agent**: it reliably reaches code-exec on multi-stage targets
(web → WebSocket tool abuse → RCE) with native tool-calling.

## Highlights
- **Unrestricted** for legitimate security use cases.
- **~27B parameters**, long context (up to **240k tokens** in production), native tool-calling.
- Two GGUF builds: **Q8_0** (≈lossless, production quality) and **Q6_K** (lighter).
- Ships the **MTP** draft weights (`*-mtp*.gguf`) for optional speculative decode.
- Validated end-to-end as a **pentest agent** (live RCE + loot + flag on a 3-stage CTF).

## Model details
| | |
|---|---|
| Format | **GGUF** (llama.cpp) |
| Quantizations | **Q8_0** (~29 GB) · **Q6_K** (~22 GB) |
| Context length | up to 262,144 tokens (240k used in prod) |
| Tool-calling | OpenAI-compatible (native in llama.cpp `llama-server`) |
| Serving | **llama.cpp `llama-server`** (OpenAI `/v1` compatible) |
| KV cache | bf16 by default; `q8_0` to roughly halve footprint |

> Note: the Hub's "Architecture" widget reads the GGUF's internal loader tag
> (`general.architecture`); that is only a llama.cpp loader key baked into the
> file and does not reflect the model name. The model is **Nesi2**.

## Files
| File | What |
|---|---|
| `Nesi2-Q8_0.gguf` | the model, **Q8_0** (max quality) |
| `Nesi2-mtp-Q8_0.gguf` | MTP draft head for Q8 (optional, `--model-draft`) |
| `Nesi2-Q6_K.gguf` | the model, **Q6_K** (lighter) |
| `Nesi2-Q6_K-mtp.gguf` | Q6_K with the MTP head already merged (single file) |

---

## Two configurations

Both serve the same `nesi2` model over an OpenAI-compatible `/v1`. Pick by VRAM
and quality target.

| Config | Files | VRAM (weights + MTP) | When |
|---|---|---|---|
| **A — Q8_0 + MTP** | `Nesi2-Q8_0.gguf` + `Nesi2-mtp-Q8_0.gguf` | ~29 + ~3 GB | max quality, 96 GB card |
| **B — Q6_K (+MTP)** | `Nesi2-Q6_K.gguf` (or `Nesi2-Q6_K-mtp.gguf`) | ~22 GB | lighter, more slots / smaller card |

> **MTP (speculative decode)** needs a llama.cpp build that supports
> `--spec-type draft-mtp`. It helps **single-stream latency**; under heavy
> concurrency it did not improve aggregate throughput in our tests, so for
> multi-user serving you can run without it and keep it for low-latency use.

---

## Deploy on a GPU box from zero

Everything below runs **on the GPU box as root**. The gateway bits run on your
LiteLLM host. Target image: `nvidia/cuda:12.8.1-devel-ubuntu22.04`, ≥ 60 GB disk
(≥ 90 GB if you want Q8 **and** Q6 on the same box).

### 0) Pick a GPU
| GPU | VRAM | Fits | single-stream tok/s (240k slot) |
|---|---|---|---|
| RTX A6000 | 48 GB | 1 user @ 240k (Q6_K) | ~22 |
| RTX PRO 6000 **Max-Q** (power-capped ~300W) | 96 GB | 1–3 users @ 240k | ~20–25 (gateway end-to-end) |
| RTX PRO 6000 **WS** (full ~600W) | 96 GB | up to 5 users @ 240k (Q8+MTP) | **~55** |

> Blackwell (PRO 6000, `sm_120`) needs CUDA toolkit **≥ 12.8**; build llama.cpp
> with `-DCMAKE_CUDA_ARCHITECTURES=120`. On Ampere (A6000) use `86`.

### 1) Deps + build llama.cpp
```bash
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq build-essential cmake git python3 python3-pip libcurl4-openssl-dev ccache
pip install -q -U "huggingface_hub[cli]" hf_transfer
export HF_HUB_ENABLE_HF_TRANSFER=1          # fast download
# the HF CLI in hf_hub 2.x is `hf`, NOT `huggingface-cli`

cd /root && git clone --depth 1 https://github.com/ggml-org/llama.cpp && cd llama.cpp
# Blackwell PRO 6000 -> 120 ; Ampere A6000 -> 86
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DLLAMA_CURL=ON -DGGML_CCACHE=ON
cmake --build build --config Release -j"$(nproc)" --target llama-server
```

### 2) Download the model
```bash
# Config A — Q8 + MTP
hf download nesilabs/Nesi2 --include "Nesi2-Q8_0.gguf" "Nesi2-mtp-Q8_0.gguf" --local-dir /root/models/nesi2
# Config B — Q6_K
hf download nesilabs/Nesi2 --include "Nesi2-Q6_K*.gguf" --local-dir /root/models/nesi2
```

### 3) Launch `llama-server` in tmux
Never start it with `setsid`/`nohup` over SSH (they die when the pipe closes).
Use **tmux** so it survives the SSH session.

#### Config A — Q8_0 + MTP, 5 users @ 240k (production, RTX PRO 6000 WS)
`--parallel N` splits `--ctx-size` across slots → `n_ctx_slot = ctx_size / parallel`.
For **5 slots of 240k each** you need `--ctx-size 1228800` (= 5 × 245760), a
**quantized KV cache** (`q8_0`), and the MTP draft:
```bash
tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \
  --model /root/models/nesi2/Nesi2-Q8_0.gguf --alias nesi2 \
  --n-gpu-layers 999 -fa on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp --model-draft /root/models/nesi2/Nesi2-mtp-Q8_0.gguf --spec-draft-n-max 3 \
  --ctx-size 1228800 --parallel 5 \
  --host 0.0.0.0 --port 8080 --api-key <YOUR_API_KEY> \
  > /root/llama.log 2>&1"
```
VRAM ≈ 84 / 96 GB. Drop `--spec-type/--model-draft` if your build lacks MTP.

#### Config B — Q6_K, single power user @ 240k
```bash
tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \
  --model /root/models/nesi2/Nesi2-Q6_K.gguf --alias nesi2 \
  --ctx-size 245760 --parallel 1 --n-gpu-layers 999 -fa on \
  --host 0.0.0.0 --port 8080 --api-key <YOUR_API_KEY> \
  > /root/llama.log 2>&1"
```
For **3 users @ 240k each** on a 96 GB card, use `--ctx-size 737280 --parallel 3`
plus `--cache-type-k q8_0 --cache-type-v q8_0`. If it OOMs, drop to
`q4_0` KV, lower per-user context (e.g. `--ctx-size 393216 --parallel 3` = 128k
each), or move to a 2× GPU box. Under `--parallel 3` per-stream speed drops
(~55 → ~20–30 tok/s) but aggregate throughput goes up.

Wait for load (VRAM up + `server is listening`):
```bash
grep -m1 "server is listening" <(tail -f /root/llama.log)
```

### 4) Point the gateway (LiteLLM) at the box
The box's **public IP and 8080 port change on every (re)start**, so this is the
only step you repeat. On the LiteLLM host:
```bash
# .env
NESI2_API_BASE=http://<BOX_PUBLIC_IP>:<BOX_8080_HOSTPORT>/v1
NESI2_API_KEY=<YOUR_API_KEY>
# then
docker compose up -d litellm
```

### 5) Verify end-to-end
```bash
curl -s -X POST https://<your-gateway>/v1/chat/completions \
  -H "Authorization: Bearer <client_key>" -H "Content-Type: application/json" \
  -d '{"model":"nesi2","messages":[{"role":"user","content":"di OK"}],"max_tokens":5}'
```

---

## Lessons learned & recommendations

- **`--parallel 1` is mandatory to actually get the full 240k in one slot.** The
  llama.cpp default (`--parallel 4`) silently gives each slot `ctx/4`.
- **`-fa on` (flash-attention)** — keep it on; large context is unusable without it.
- **tmux, not setsid/nohup.** Over SSH the detached process dies and you get an
  empty log; tmux survives.
- **Agent discipline > raw capability.** As an agent Nesi2 shines because of
  *tool-calling discipline*: it acts (one concrete tool call per turn) instead of
  narrating. Give it a lean system prompt that says: act don't narrate; write your
  own Python for anything past a simple HTTP request (WebSocket, sockets, rotating
  CSRF); get code-exec first and confirm with `id`; and **keep durable state on
  disk** (a `NOTES.md` + reusable scripts) so a context compaction doesn't make it
  re-derive what it already solved.
- **Isolate the agent's filesystem for honest evals.** A harness whose shell can
  read your machine will "solve" a CTF by reading the solution/writeups off disk
  instead of attacking live. Deny reads outside the target (regex guard in the tool
  layer, or run the agent under `sandbox-exec`/a container). Network to the target
  stays open.
- **KV quantization scales users cheaply.** `--cache-type-k/v q8_0` roughly doubles
  how many 240k slots fit, with no quality drift we could measure; it's what makes
  the multi-user profiles fit a single 96 GB card.
- **Max-Q vs WS.** The power-capped Max-Q PRO 6000 is ~2.5× slower than the full WS
  for ~0.25 $/h less. For interactive/agent use pay for the WS; for cheap batch,
  Max-Q is fine.
- **Speculative decode (MTP).** The `*-mtp*.gguf` draft head is included; under
  concurrency it did **not** help throughput in our tests, so we run without it for
  multi-user serving and keep it available for single-stream latency.

## ⚠️ Legal Notice / Disclaimer

**Nesi2 is intended EXCLUSIVELY for security professionals, for lawful and
authorized activities.**

- **Authorization required.** Use it only against systems, networks, or
  applications for which you have **explicit written permission** (a contracted
  penetration test, an in-scope bug bounty program, or your own/lab environments).
  Using it against systems without authorization is **illegal** and strictly
  prohibited.
- **User responsibility.** You are the **sole responsible party** for how you use
  this model and for complying with all applicable laws in your jurisdiction
  (including computer-misuse, data-protection/GDPR, and intellectual-property laws).
- **No warranty.** The model is provided "as is", **without warranty** of accuracy,
  fitness, or results. It may produce incorrect information — always verify before acting.
- **Limitation of liability.** Nesilabs is **not liable** for any damages, losses,
  or legal consequences arising from the use or misuse of this model, including any
  unauthorized or unlawful use.
- **Prohibited use.** Using Nesi2 for illegal activities, unauthorized access,
  extortion, harm to third parties, or any purpose that violates the law or these
  terms is prohibited.

By using Nesi2 you accept this notice and the Nesilabs Terms of Service and
Acceptable Use Policy.

## License
Released under **Apache 2.0**. Packaged and optimized by Nesilabs.