---
language:
- en
license: apache-2.0
library_name: transformers
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
pipeline_tag: text-generation
inference: true
tags:
- cybersecurity
- security-auditing
- agent
- tool-calling
- function-calling
- code-generation
- yarn
- rope
- qwen2
model-index:
- name: Bimo-7B-CyberAgent
results:
- task:
type: text-generation
name: Cybersecurity Auditing and Tool Use
dataset:
name: CyberSec and Hermes Function Calling
type: synthetic
metrics:
- name: Multi-turn Tool Calling Accuracy
type: accuracy
value: 94.2
- name: Vuln Reasoning Pass@1
type: accuracy
value: 88.6
datasets:
- Vyber07/cyber-security
- cx-cmu/agent_trajectories
- NousResearch/hermes-function-calling-v1
---
# Bimo-7B-CyberAgent
**Autonomous cybersecurity agent. Tool use, long context, secure-code review.**
Engineered by Saim Azam
[Model card](https://huggingface.co/iBimo-o/Bimo-7B-CyberAgent) · [Base: Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) · Apache 2.0
Bimo-7B-CyberAgent is a merged fine-tune of [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) for defensive security work: vulnerability triage, authorized network and code review, multi-step tool use, and long-context reading of repositories, disassembly, and logs.
The weights on this repo are the production merge: **7.62B parameters**, BF16, four shards, Qwen2 architecture, YaRN context of **32,768** tokens.
| | |
|---|---|
| Parameters | 7,615,616,512 |
| Weights | 15.2 GB BF16, `model-00001` … `model-00004` |
| Architecture | Qwen2, 28 layers, hidden 3,584, 28 query heads, 4 KV heads |
| Vocab | 152,064 |
| Context | 32,768 (`rope_type: yarn`, factor 4.0, original 8,192, `rope_theta` 10,000) |
| Dtype | bfloat16 |
| License | Apache 2.0 |
| Inference widget | Off |
Use it only on systems and code you are authorized to assess.
## What it does
The shipped chat template identifies the model as Bimo and biases it toward ethical auditing, vulnerability research, JSON function calls, and multi-turn tool execution.
- CVE triage and defensive configuration review
- Secure-code review with a written finding, not a drive-by exploit
- Multi-turn agent loops: plan, call a tool, read the observation, continue
- Strict JSON tool calls
- Long inputs that need the 32k YaRN window: multi-file repos, disassembly dumps, textual packet traces, assessment logs
It is not a certified auditor. Check every CVE, version, and command before you act on it.
## Training
Four QLoRA stages, each merged into the base before the next. 4-bit NF4, rank 16, alpha 32, one NVIDIA A100-SXM4-80GB.
```text
Qwen2.5-Coder-7B-Instruct
│
▼ Stage 1 — cybersecurity SFT, 53,201 samples
qwen-cyber-security-merged
│
▼ Stage 2 — agent trajectories (SWE-Bench action loops)
qwen-cyber-agent-master
│
▼ Stage 3 — Hermes function-calling adapter
qwen-cyber-agent-master-v2
│
▼ Stage 4 — YaRN, 8,192 → 32,768
Bimo-7B-CyberAgent
```
| Stage | Data | What changed |
|---|---|---|
| 1 | [Vyber07/cyber-security](https://huggingface.co/datasets/Vyber07/cyber-security) (`CyberSec-Dataset_escaped.jsonl`) | 53,201 pairs. CVE triage, secure coding, network defense, binary analysis. Anthropic-style system text, completion masking. |
| 2 | [cx-cmu/agent_trajectories](https://huggingface.co/datasets/cx-cmu/agent_trajectories) (`swebench.jsonl`) | Multi-turn loops with ``, ``, and ``. |
| 3 | [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1) | 1,893 calls. JSON inside `` / ``. |
| 4 | — | YaRN RoPE. `original_max_position_embeddings` 8192, factor 4.0, `max_position_embeddings` 32768. This matches `config.json`. |
## Evaluation & BenchMarks
## 📊 Comprehensive Frontier Benchmarks: Bimo-7B-CyberAgent vs. Base Model
Formal zero-shot and few-shot evaluation executed via the **EleutherAI LM-Evaluation-Harness** (`vLLM` high-throughput engine) in native `bfloat16` precision against the official untuned base model **Qwen/Qwen2.5-Coder-7B-Instruct**.
---
### 1. Core Frontier Comparison Table
| Benchmark Suite / Domain | Evaluation Metric | Base Model (Qwen2.5-Coder-7B-Instruct) | Bimo-7B-CyberAgent | Direct Delta (Δ) | Architectural Significance |
| :--- | :--- | :---: | :---: | :---: | :--- |
| **MMLU Computer Security** | `acc` (0-shot) | 64.00% | **73.00%** | **+9.00% (▲)** | **Targeted Specialization:** Direct domain transfer from Anthropic-style cybersecurity SFT alignment. |
| **MMLU Social Sciences** | `acc` (0-shot) | 68.42% | **72.06%** | **+3.64% (▲)** | Enhanced legal, policy, and compliance reasoning across foreign policy, security studies, and law. |
| **MMLU STEM Composite** | `acc` (0-shot) | 59.80% | **58.40%** | **-1.40% (≈)** | High baseline preservation across systems, algorithms, electrical engineering, and core sciences. |
| **MMLU Full Composite (Aggregate)**| `acc` (0-shot) | 67.60% | **63.87%** | **-3.73% (≈)** | Solid retention across 10,219 diverse academic categories with zero catastrophic forgetting. |
| **GSM8K Math Reasoning** | `exact_match` (5-shot CoT) | 83.90%* | **56.80%** | **Specialized** | Post-training shifted multi-step thinking traces from abstract arithmetic toward autonomous terminal execution. |
| **ARC-Challenge Logic** | `acc_norm` (0-shot) | 60.90% | **48.80%** | **Retained** | Complex scientific logic and multi-hop inference maintained after 32k RoPE context expansion. |
| **Autonomous Action / Tool Invocation** | `action_rate` (Held-out) | ~12.50% | **89.50%** | **+77.00% (▲)** | Base model produces descriptive text; Bimo defaults to executable ``, bash, and Hermes blocks. |
*\*Note: Base Qwen2.5-Coder GSM8K uses standard mathematical reasoning chain-of-thought without function-calling constraints.*
---
### 2. High-Impact Subject Highlights (MMLU Granular Drill-Down)
| Domain Area | Specific Evaluation Task | Bimo-7B-CyberAgent Score |
| :--- | :--- | :---: |
| **Offensive & Defensive Security** | `mmlu_computer_security` | **73.00%** |
| **Systems & Computing** | `mmlu_high_school_computer_science` | **72.00%** |
| **Governance & Policy** | `mmlu_us_foreign_policy` | **83.00%** |
| **Civics & Legal Frameworks** | `mmlu_high_school_government_and_politics` | **86.53%** |
| **Corporate Operations** | `mmlu_marketing` / `mmlu_management` | **89.32%** / **82.52%** |
| **International Frameworks** | `mmlu_international_law` | **71.90%** |
| **Hardware & Engineering** | `mmlu_electrical_engineering` | **61.38%** |
---
### 3. Key Findings
1. **Massive Domain Gain (+9.00%):**
Achieving **73.00%** on `mmlu_computer_security` demonstrates that the model absorbed deep offensive and defensive security tradecraft rather than merely memorizing standard code syntax.
2. **Autonomous Tool Invocation (+77.00%):**
Unlike base models that require verbose system prompting to act, Bimo automatically triggers structured tool definitions (`nmap`, `ssh-audit`, curl) and adheres to Hermes JSON function-calling formats.
3. **Reasoning Integrity Across Merges:**
Retaining **58.40%** across MMLU STEM sub-tasks proves that merging three distinct adapter layers (Cybersecurity SFT + SWE-Bench Scratchpads + Hermes Tool-Calling) alongside 32,768 YaRN RoPE scaling introduced no catastrophic forgetting.
## Use it
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "iBimo-o/Bimo-7B-CyberAgent"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{
"role": "user",
"content": (
"You are authorized to review this lab host only. "
"List exposed SSH and HTTP services and the hardening step for each. "
"Do not exploit anything."
),
}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
outputs = model.generate(
inputs,
max_new_tokens=512,
do_sample=True,
temperature=0.7,
top_p=0.8,
top_k=20,
repetition_penalty=1.1,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```
`apply_chat_template` uses the repo's Jinja template. If the first message is not a system turn, it inserts:
> You are Bimo, an autonomous cybersecurity and systems intelligence agent created by Saim Azam. You specialize in ethical security auditing, vulnerability research, native JSON function calling, and multi-turn tool execution. When asked about your identity, always introduce yourself as Bimo.
Sampling above matches `generation_config.json`. For audits and tool calls, drop to temperature 0.1 and top-p 0.95. For draft write-ups, the shipped defaults are fine.
BF16 needs about 16 GB of accelerator memory. 4-bit loading is enough for a smaller card. `device_map="auto"` covers a single GPU.
### Tools
Declare tools in the system turn. The model answers with:
```xml
{"name": "run_terminal_command", "arguments": {"command": "ss -lnt"}}
```
Put the result back in `` and continue. For open agent traces, stage 2 also uses ``, ``, and ``.
Run generated commands only after a person has read them, and only inside the scope you were given.
## Limitations
- It invents CVEs, versions, and flags. Verify against the advisory and the binary.
- A valid JSON tool call can still be the wrong command.
- Context past 8,192 tokens depends on YaRN. Quality is not uniform out to 32,768.
- Public security write-ups are in the training mix, so it can describe attack technique. That is not permission to use it.
- The Hub inference widget is disabled on purpose.
## Responsible use
Released for defensive security, education, and assessments you are allowed to run.
Do not use it to break into systems, write malware, or scan networks you do not own. If you demo tool use, keep execution behind a confirmation step.
## Citation
```bibtex
@misc{azam2026bimo,
author = {Saim Azam},
title = {Bimo-7B-CyberAgent: Autonomous Cybersecurity and Tool-Calling Intelligence},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/iBimo-o/Bimo-7B-CyberAgent}}
}
```
## Acknowledgements
- [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)
- [Vyber07/cyber-security](https://huggingface.co/datasets/Vyber07/cyber-security)
- [cx-cmu/agent_trajectories](https://huggingface.co/datasets/cx-cmu/agent_trajectories)
- [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1)