--- license: apache-2.0 language: - en tags: - cybersecurity - pentesting - security - uncensored - qwen3 - tool-calling - gguf - llama-cpp pipeline_tag: text-generation --- # Nesi2 — Uncensored AI for Cybersecurity (GGUF / llama.cpp) **Nesi2** is an uncensored large language model, curated and optimized by **Nesilabs** specifically for **offensive and defensive cybersecurity**: penetration testing, vulnerability analysis, exploit development, secure code review, and audit automation. It is built on the **Qwen3.8-27B** architecture (same family as [Nesi1](https://huggingface.co/nesilabs/Nesi1)) and packaged by Nesilabs as a **GGUF** build for **llama.cpp**, so it runs on a single GPU without an AWQ/vLLM stack. This is the build we run as an autonomous **pentest agent**: it reliably reaches code-exec on multi-stage targets (web → WebSocket tool abuse → RCE) with native tool-calling. ## Highlights - **Unrestricted** for legitimate security use cases. - **27B parameters**, long context (up to **240k tokens** in production), native tool-calling. - **GGUF Q6_K** — single-file, runs on one 48–96 GB GPU with llama.cpp. - Includes the **MTP** weights (`*-mtp.gguf`) for optional speculative decode. - Validated end-to-end as a **pentest agent** (live RCE + loot + flag on a 3-stage CTF). ## Model details | | | |---|---| | Base architecture | Qwen3.8-27B | | Format | **GGUF** (`Nesi2-Q6_K.gguf` + `Nesi2-Q6_K-mtp.gguf`) | | Quantization | **Q6_K** (~22 GB weights) | | Context length | up to 262,144 tokens (240k used in prod) | | Tool-calling | OpenAI-compatible (native in llama.cpp `llama-server`) | | Serving | **llama.cpp `llama-server`** (OpenAI `/v1` compatible) | | KV cache | bf16 by default; `q8_0` to roughly halve footprint | > Note: the Hub's "Architecture" widget reads the GGUF's internal loader tag > (`general.architecture`), which may display as `qwen35`. That is only the > llama.cpp loader key baked into the file — the model is **Qwen3.8-27B**. ## Files | File | What | |---|---| | `Nesi2-Q6_K.gguf` | the model (load this one) | | `Nesi2-Q6_K-mtp.gguf` | MTP head for speculative decode (optional, `--model-draft`) | --- ## Deploy on a Vast.ai box from zero Everything below runs **on the GPU box as root**. The gateway bits run on your LiteLLM host. Target image: `nvidia/cuda:12.8.1-devel-ubuntu22.04`, ≥ 160 GB disk. ### 0) Pick a GPU | GPU | VRAM | Fits | single-stream tok/s (Q6_K, 240k slot) | |---|---|---|---| | RTX A6000 | 48 GB | 1 user @ 240k | ~22 | | RTX PRO 6000 **Max-Q** (power-capped ~300W) | 96 GB | 1–3 users @ 240k | ~20–25 (gateway end-to-end) | | RTX PRO 6000 **WS** (full ~600W) | 96 GB | 1–3 users @ 240k | **~55** | > Blackwell (PRO 6000, `sm_120`) needs CUDA toolkit **≥ 12.8**. A box whose driver > reports CUDA < 12.8 will fail vLLM; for llama.cpp just build with > `-DCMAKE_CUDA_ARCHITECTURES=120`. On Ampere (A6000) use `86`. ### 1) Deps + build llama.cpp ```bash export DEBIAN_FRONTEND=noninteractive apt-get update -qq apt-get install -y -qq build-essential cmake git python3 python3-pip libcurl4-openssl-dev ccache pip install -q -U "huggingface_hub[cli]" hf_transfer export HF_HUB_ENABLE_HF_TRANSFER=1 # fast download # the HF CLI in hf_hub 2.x is `hf`, NOT `huggingface-cli` cd /root && git clone --depth 1 https://github.com/ggml-org/llama.cpp && cd llama.cpp # Blackwell PRO 6000 -> 120 ; Ampere A6000 -> 86 cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DLLAMA_CURL=ON -DGGML_CCACHE=ON cmake --build build --config Release -j"$(nproc)" --target llama-server ``` ### 2) Download the model ```bash hf download nesilabs/Nesi2 --include "Nesi2-Q6_K*.gguf" --local-dir /root/models/nesi2 ``` ### 3) Launch `llama-server` in tmux Never start it with `setsid`/`nohup` over SSH (they die when the pipe closes). Use **tmux** so it survives the SSH session. **Profile A — single power user @ 240k** (the production default): ```bash GGUF=$(find /root/models/nesi2 -name "*.gguf" | grep -ivE "mmproj|vision|mtp" | sort | head -1) tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \ --model $GGUF --alias nesi2 \ --ctx-size 245760 --parallel 1 --n-gpu-layers 999 -fa on \ --host 0.0.0.0 --port 8080 --api-key \ > /root/llama.log 2>&1" ``` **Profile B — 3 users (team), 240k each at the same time:** `--parallel N` splits `--ctx-size` across slots → `n_ctx_slot = ctx_size / parallel`. So for **3 slots of 240k each you need `--ctx-size 737280` (= 3 × 245760)**. That much KV does not fit alongside Q6_K weights in 96 GB at bf16, so use a **quantized KV cache** (`q8_0`, ~half the footprint, negligible quality loss): ```bash GGUF=$(find /root/models/nesi2 -name "*.gguf" | grep -ivE "mmproj|vision|mtp" | sort | head -1) tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \ --model $GGUF --alias nesi2 \ --ctx-size 737280 --parallel 3 --n-gpu-layers 999 -fa on \ --cache-type-k q8_0 --cache-type-v q8_0 \ --host 0.0.0.0 --port 8080 --api-key \ > /root/llama.log 2>&1" ``` If it still OOMs on a given card: drop to `--cache-type-k q4_0 --cache-type-v q4_0`, or lower per-user context (e.g. `--ctx-size 393216 --parallel 3` = 128k each), or move to a 2× GPU box. Under `--parallel 3` **per-stream** speed drops (~55 → ~20–30 tok/s/stream on a WS) but aggregate throughput goes up; this is the right trade for a 3-person team that isn't all generating at the exact same instant. Wait for load (VRAM up + `server is listening`): ```bash grep -m1 "server is listening" <(tail -f /root/llama.log) ``` ### 4) Point the gateway (LiteLLM) at the box The box's **public IP and 8080 port change on every (re)start**, so this is the only step you repeat. On the LiteLLM host: ```bash # .env NESI2_API_BASE=http://:/v1 NESI2_API_KEY= # then docker compose up -d litellm ``` Get `:` with `vastai show instance --raw` → `public_ipaddr` and `ports["8080/tcp"][0].HostPort`. ### 5) Verify end-to-end ```bash curl -s -X POST https:///v1/chat/completions \ -H "Authorization: Bearer " -H "Content-Type: application/json" \ -d '{"model":"nesi2","messages":[{"role":"user","content":"di OK"}],"max_tokens":5}' ``` --- ## Lessons learned & recommendations - **`reasoning_effort` breaks this chat template.** The template only accepts `xhigh|medium|low`; a client sending `reasoning_effort: "high"` triggers a jinja exception → HTTP 500. Strip it at the gateway (LiteLLM: `drop_params: true` + `additional_drop_params: ["reasoning_effort"]`) so every client is unaffected. - **`--parallel 1` is mandatory to actually get the full 240k in one slot.** The llama.cpp default (`--parallel 4`) silently gives each slot `ctx/4`. - **`-fa on` (flash-attention)** — keep it on; large context is unusable without it. - **tmux, not setsid/nohup.** Over SSH the detached process dies and you get an empty log; tmux survives. - **Agent discipline > raw capability.** Same base as Nesi1, but as an agent Nesi2 shines because of *tool-calling discipline*: it acts (one concrete tool call per turn) instead of narrating. Give it a lean system prompt that says: act don't narrate; write your own Python for anything past a simple HTTP request (WebSocket, sockets, rotating CSRF); get code-exec first and confirm with `id`; and **keep durable state on disk** (a `NOTES.md` + reusable scripts) so a context compaction doesn't make it re-derive what it already solved. - **Isolate the agent's filesystem for honest evals.** A harness whose shell can read your machine will "solve" a CTF by reading the solution/writeups off disk instead of attacking live. Deny reads outside the target (regex guard in the tool layer, or run the agent under `sandbox-exec`/a container). Network to the target stays open. - **KV quantization scales users cheaply.** `--cache-type-k/v q8_0` roughly doubles how many 240k slots fit, with no quality drift we could measure; it's what makes the 3-user profile fit a single 96 GB card. - **Max-Q vs WS.** The power-capped Max-Q PRO 6000 is ~2.5× slower than the full WS for ~0.25 $/h less. For interactive/agent use pay for the WS; for cheap batch, Max-Q is fine. - **Speculative decode (MTP).** The `*-mtp.gguf` draft head is included; under concurrency it did **not** help throughput in our tests (same finding as Nesi1's NEXTN), so we run without it and keep it available for single-stream latency. ## ⚠️ Legal Notice / Disclaimer **Nesi2 is intended EXCLUSIVELY for security professionals, for lawful and authorized activities.** - **Authorization required.** Use it only against systems, networks, or applications for which you have **explicit written permission** (a contracted penetration test, an in-scope bug bounty program, or your own/lab environments). Using it against systems without authorization is **illegal** and strictly prohibited. - **User responsibility.** You are the **sole responsible party** for how you use this model and for complying with all applicable laws in your jurisdiction (including computer-misuse, data-protection/GDPR, and intellectual-property laws). - **No warranty.** The model is provided "as is", **without warranty** of accuracy, fitness, or results. It may produce incorrect information — always verify before acting. - **Limitation of liability.** Nesilabs is **not liable** for any damages, losses, or legal consequences arising from the use or misuse of this model, including any unauthorized or unlawful use. - **Prohibited use.** Using Nesi2 for illegal activities, unauthorized access, extortion, harm to third parties, or any purpose that violates the law or these terms is prohibited. By using Nesi2 you accept this notice and the Nesilabs Terms of Service and Acceptable Use Policy. ## License Released under **Apache 2.0**. Based on the Qwen3.8-27B architecture. Adapted and packaged by Nesilabs.