| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - cybersecurity |
| - pentesting |
| - security |
| - uncensored |
| - qwen3 |
| - tool-calling |
| - gguf |
| - llama-cpp |
| pipeline_tag: text-generation |
| --- |
| |
| # Nesi2 — Uncensored AI for Cybersecurity (GGUF / llama.cpp) |
|
|
| **Nesi2** is an uncensored large language model, curated and optimized by |
| **Nesilabs** specifically for **offensive and defensive cybersecurity**: |
| penetration testing, vulnerability analysis, exploit development, secure |
| code review, and audit automation. |
|
|
| It is built on the **Qwen3.8-27B** architecture (same family as |
| [Nesi1](https://huggingface.co/nesilabs/Nesi1)) and packaged by Nesilabs as a |
| **GGUF** build for **llama.cpp**, so it runs on a single GPU without an AWQ/vLLM |
| stack. This is the build we run as an autonomous **pentest agent**: it reliably |
| reaches code-exec on multi-stage targets (web → WebSocket tool abuse → RCE) with |
| native tool-calling. |
|
|
| ## Highlights |
| - **Unrestricted** for legitimate security use cases. |
| - **27B parameters**, long context (up to **240k tokens** in production), native tool-calling. |
| - **GGUF Q6_K** — single-file, runs on one 48–96 GB GPU with llama.cpp. |
| - Includes the **MTP** weights (`*-mtp.gguf`) for optional speculative decode. |
| - Validated end-to-end as a **pentest agent** (live RCE + loot + flag on a 3-stage CTF). |
| |
| ## Model details |
| | | | |
| |---|---| |
| | Base architecture | Qwen3.8-27B | |
| | Format | **GGUF** (`Nesi2-Q6_K.gguf` + `Nesi2-Q6_K-mtp.gguf`) | |
| | Quantization | **Q6_K** (~22 GB weights) | |
| | Context length | up to 262,144 tokens (240k used in prod) | |
| | Tool-calling | OpenAI-compatible (native in llama.cpp `llama-server`) | |
| | Serving | **llama.cpp `llama-server`** (OpenAI `/v1` compatible) | |
| | KV cache | bf16 by default; `q8_0` to roughly halve footprint | |
|
|
| > Note: the Hub's "Architecture" widget reads the GGUF's internal loader tag |
| > (`general.architecture`), which may display as `qwen35`. That is only the |
| > llama.cpp loader key baked into the file — the model is **Qwen3.8-27B**. |
|
|
| ## Files |
| | File | What | |
| |---|---| |
| | `Nesi2-Q6_K.gguf` | the model (load this one) | |
| | `Nesi2-Q6_K-mtp.gguf` | MTP head for speculative decode (optional, `--model-draft`) | |
|
|
| --- |
|
|
| ## Deploy on a Vast.ai box from zero |
|
|
| Everything below runs **on the GPU box as root**. The gateway bits run on your |
| LiteLLM host. Target image: `nvidia/cuda:12.8.1-devel-ubuntu22.04`, ≥ 160 GB disk. |
|
|
| ### 0) Pick a GPU |
| | GPU | VRAM | Fits | single-stream tok/s (Q6_K, 240k slot) | |
| |---|---|---|---| |
| | RTX A6000 | 48 GB | 1 user @ 240k | ~22 | |
| | RTX PRO 6000 **Max-Q** (power-capped ~300W) | 96 GB | 1–3 users @ 240k | ~20–25 (gateway end-to-end) | |
| | RTX PRO 6000 **WS** (full ~600W) | 96 GB | 1–3 users @ 240k | **~55** | |
| |
| > Blackwell (PRO 6000, `sm_120`) needs CUDA toolkit **≥ 12.8**. A box whose driver |
| > reports CUDA < 12.8 will fail vLLM; for llama.cpp just build with |
| > `-DCMAKE_CUDA_ARCHITECTURES=120`. On Ampere (A6000) use `86`. |
|
|
| ### 1) Deps + build llama.cpp |
| ```bash |
| export DEBIAN_FRONTEND=noninteractive |
| apt-get update -qq |
| apt-get install -y -qq build-essential cmake git python3 python3-pip libcurl4-openssl-dev ccache |
| pip install -q -U "huggingface_hub[cli]" hf_transfer |
| export HF_HUB_ENABLE_HF_TRANSFER=1 # fast download |
| # the HF CLI in hf_hub 2.x is `hf`, NOT `huggingface-cli` |
| |
| cd /root && git clone --depth 1 https://github.com/ggml-org/llama.cpp && cd llama.cpp |
| # Blackwell PRO 6000 -> 120 ; Ampere A6000 -> 86 |
| cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DLLAMA_CURL=ON -DGGML_CCACHE=ON |
| cmake --build build --config Release -j"$(nproc)" --target llama-server |
| ``` |
|
|
| ### 2) Download the model |
| ```bash |
| hf download nesilabs/Nesi2 --include "Nesi2-Q6_K*.gguf" --local-dir /root/models/nesi2 |
| ``` |
|
|
| ### 3) Launch `llama-server` in tmux |
| Never start it with `setsid`/`nohup` over SSH (they die when the pipe closes). |
| Use **tmux** so it survives the SSH session. |
|
|
| **Profile A — single power user @ 240k** (the production default): |
| ```bash |
| GGUF=$(find /root/models/nesi2 -name "*.gguf" | grep -ivE "mmproj|vision|mtp" | sort | head -1) |
| tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \ |
| --model $GGUF --alias nesi2 \ |
| --ctx-size 245760 --parallel 1 --n-gpu-layers 999 -fa on \ |
| --host 0.0.0.0 --port 8080 --api-key <YOUR_API_KEY> \ |
| > /root/llama.log 2>&1" |
| ``` |
|
|
| **Profile B — 3 users (team), 240k each at the same time:** |
| `--parallel N` splits `--ctx-size` across slots → `n_ctx_slot = ctx_size / parallel`. |
| So for **3 slots of 240k each you need `--ctx-size 737280` (= 3 × 245760)**. |
| That much KV does not fit alongside Q6_K weights in 96 GB at bf16, so use a |
| **quantized KV cache** (`q8_0`, ~half the footprint, negligible quality loss): |
| ```bash |
| GGUF=$(find /root/models/nesi2 -name "*.gguf" | grep -ivE "mmproj|vision|mtp" | sort | head -1) |
| tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \ |
| --model $GGUF --alias nesi2 \ |
| --ctx-size 737280 --parallel 3 --n-gpu-layers 999 -fa on \ |
| --cache-type-k q8_0 --cache-type-v q8_0 \ |
| --host 0.0.0.0 --port 8080 --api-key <YOUR_API_KEY> \ |
| > /root/llama.log 2>&1" |
| ``` |
| If it still OOMs on a given card: drop to `--cache-type-k q4_0 --cache-type-v q4_0`, |
| or lower per-user context (e.g. `--ctx-size 393216 --parallel 3` = 128k each), |
| or move to a 2× GPU box. Under `--parallel 3` **per-stream** speed drops |
| (~55 → ~20–30 tok/s/stream on a WS) but aggregate throughput goes up; this is the |
| right trade for a 3-person team that isn't all generating at the exact same instant. |
|
|
| Wait for load (VRAM up + `server is listening`): |
| ```bash |
| grep -m1 "server is listening" <(tail -f /root/llama.log) |
| ``` |
|
|
| ### 4) Point the gateway (LiteLLM) at the box |
| The box's **public IP and 8080 port change on every (re)start**, so this is the |
| only step you repeat. On the LiteLLM host: |
| ```bash |
| # .env |
| NESI2_API_BASE=http://<BOX_PUBLIC_IP>:<BOX_8080_HOSTPORT>/v1 |
| NESI2_API_KEY=<YOUR_API_KEY> |
| # then |
| docker compose up -d litellm |
| ``` |
| Get `<BOX_PUBLIC_IP>:<HOSTPORT>` with |
| `vastai show instance <ID> --raw` → `public_ipaddr` and `ports["8080/tcp"][0].HostPort`. |
|
|
| ### 5) Verify end-to-end |
| ```bash |
| curl -s -X POST https://<your-gateway>/v1/chat/completions \ |
| -H "Authorization: Bearer <client_key>" -H "Content-Type: application/json" \ |
| -d '{"model":"nesi2","messages":[{"role":"user","content":"di OK"}],"max_tokens":5}' |
| ``` |
|
|
| --- |
|
|
| ## Lessons learned & recommendations |
|
|
| - **`reasoning_effort` breaks this chat template.** The template only accepts |
| `xhigh|medium|low`; a client sending `reasoning_effort: "high"` triggers a jinja |
| exception → HTTP 500. Strip it at the gateway (LiteLLM: `drop_params: true` + |
| `additional_drop_params: ["reasoning_effort"]`) so every client is unaffected. |
| - **`--parallel 1` is mandatory to actually get the full 240k in one slot.** The |
| llama.cpp default (`--parallel 4`) silently gives each slot `ctx/4`. |
| - **`-fa on` (flash-attention)** — keep it on; large context is unusable without it. |
| - **tmux, not setsid/nohup.** Over SSH the detached process dies and you get an |
| empty log; tmux survives. |
| - **Agent discipline > raw capability.** Same base as Nesi1, but as an agent Nesi2 |
| shines because of *tool-calling discipline*: it acts (one concrete tool call per |
| turn) instead of narrating. Give it a lean system prompt that says: act don't |
| narrate; write your own Python for anything past a simple HTTP request |
| (WebSocket, sockets, rotating CSRF); get code-exec first and confirm with `id`; |
| and **keep durable state on disk** (a `NOTES.md` + reusable scripts) so a context |
| compaction doesn't make it re-derive what it already solved. |
| - **Isolate the agent's filesystem for honest evals.** A harness whose shell can |
| read your machine will "solve" a CTF by reading the solution/writeups off disk |
| instead of attacking live. Deny reads outside the target (regex guard in the tool |
| layer, or run the agent under `sandbox-exec`/a container). Network to the target |
| stays open. |
| - **KV quantization scales users cheaply.** `--cache-type-k/v q8_0` roughly doubles |
| how many 240k slots fit, with no quality drift we could measure; it's what makes |
| the 3-user profile fit a single 96 GB card. |
| - **Max-Q vs WS.** The power-capped Max-Q PRO 6000 is ~2.5× slower than the full WS |
| for ~0.25 $/h less. For interactive/agent use pay for the WS; for cheap batch, |
| Max-Q is fine. |
| - **Speculative decode (MTP).** The `*-mtp.gguf` draft head is included; under |
| concurrency it did **not** help throughput in our tests (same finding as Nesi1's |
| NEXTN), so we run without it and keep it available for single-stream latency. |
|
|
| ## ⚠️ Legal Notice / Disclaimer |
|
|
| **Nesi2 is intended EXCLUSIVELY for security professionals, for lawful and |
| authorized activities.** |
|
|
| - **Authorization required.** Use it only against systems, networks, or |
| applications for which you have **explicit written permission** (a contracted |
| penetration test, an in-scope bug bounty program, or your own/lab environments). |
| Using it against systems without authorization is **illegal** and strictly |
| prohibited. |
| - **User responsibility.** You are the **sole responsible party** for how you use |
| this model and for complying with all applicable laws in your jurisdiction |
| (including computer-misuse, data-protection/GDPR, and intellectual-property laws). |
| - **No warranty.** The model is provided "as is", **without warranty** of accuracy, |
| fitness, or results. It may produce incorrect information — always verify before acting. |
| - **Limitation of liability.** Nesilabs is **not liable** for any damages, losses, |
| or legal consequences arising from the use or misuse of this model, including any |
| unauthorized or unlawful use. |
| - **Prohibited use.** Using Nesi2 for illegal activities, unauthorized access, |
| extortion, harm to third parties, or any purpose that violates the law or these |
| terms is prohibited. |
|
|
| By using Nesi2 you accept this notice and the Nesilabs Terms of Service and |
| Acceptable Use Policy. |
|
|
| ## License |
| Released under **Apache 2.0**. Based on the Qwen3.8-27B architecture. |
| Adapted and packaged by Nesilabs. |
|
|
