dungnvt commited on
Commit
45fbc69
·
verified ·
1 Parent(s): 23712ef

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +27 -7
README.md CHANGED
@@ -27,19 +27,19 @@ model-index:
27
  type: custom
28
  metrics:
29
  - type: strict_pass_rate
30
- value: 88.46
31
  name: Strict Pass Rate
32
  - type: tool_call_parse_rate
33
  value: 100.0
34
  name: Tool Call Parse Rate
35
  - type: semantic_command_accuracy
36
- value: 88.46
37
  name: Semantic Command Accuracy
38
  ---
39
 
40
  # Kali-Terminus-v2
41
 
42
- **Kali-Terminus-v2** is a fine-tuned [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) model specialized for Kali Linux terminal command execution. It converts natural language instructions into precise shell commands with the `exec` tool-call format.
43
 
44
  This is the **Executor Agent** in a multi-agent penetration-testing system — it translates one bounded instruction into one structured terminal action.
45
 
@@ -62,10 +62,10 @@ Evaluated across 5 scenarios (26 turns) covering network recon, system enumerati
62
 
63
  | Metric | Score |
64
  |---|---|
65
- | **Strict Pass Rate** | **88.46%** |
66
  | Tool Call Parse Rate | 100.0% |
67
  | Exactly One Tool Call | 100.0% |
68
- | Semantic Command Accuracy | 88.46% |
69
 
70
  ### Comparison with V1
71
 
@@ -73,7 +73,7 @@ Evaluated across 5 scenarios (26 turns) covering network recon, system enumerati
73
  |---|---|---|
74
  | Kali-Terminus-v1 (FunctionGemma-270M) | 65.8% | — |
75
  | Kali-Terminus-v1 (LFM2.5-350M) | 50.0% | — |
76
- | **Kali-Terminus-v2** | **88.46%** | **+38.5 pp over v1 LFM2.5** |
77
 
78
  ## Dataset
79
 
@@ -113,9 +113,29 @@ inputs = tokenizer.apply_chat_template(
113
  outputs = model.generate(**inputs, max_new_tokens=256)
114
  response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=False)
115
  print(response)
116
- # Expected: <call:exec>{"command": "nmap -p 80 192.168.1.1"}</call:exec>
117
  ```
118
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
  ## Intended Use
120
 
121
  - **Executor Agent** in multi-agent penetration-testing systems
 
27
  type: custom
28
  metrics:
29
  - type: strict_pass_rate
30
+ value: 92.31
31
  name: Strict Pass Rate
32
  - type: tool_call_parse_rate
33
  value: 100.0
34
  name: Tool Call Parse Rate
35
  - type: semantic_command_accuracy
36
+ value: 92.31
37
  name: Semantic Command Accuracy
38
  ---
39
 
40
  # Kali-Terminus-v2
41
 
42
+ **Kali-Terminus-v2** is a fine-tuned [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) model specialized for Kali Linux terminal command execution. It converts natural language instructions into precise shell commands with native LFM2.5 `<|tool_call_start|>`/`<|tool_call_end|>` JSON tool calls.
43
 
44
  This is the **Executor Agent** in a multi-agent penetration-testing system — it translates one bounded instruction into one structured terminal action.
45
 
 
62
 
63
  | Metric | Score |
64
  |---|---|
65
+ | **Strict Pass Rate** | **92.31%** |
66
  | Tool Call Parse Rate | 100.0% |
67
  | Exactly One Tool Call | 100.0% |
68
+ | Semantic Command Accuracy | 92.31% |
69
 
70
  ### Comparison with V1
71
 
 
73
  |---|---|---|
74
  | Kali-Terminus-v1 (FunctionGemma-270M) | 65.8% | — |
75
  | Kali-Terminus-v1 (LFM2.5-350M) | 50.0% | — |
76
+ | **Kali-Terminus-v2** | **92.31%** | **+42.3 pp over v1 LFM2.5** |
77
 
78
  ## Dataset
79
 
 
113
  outputs = model.generate(**inputs, max_new_tokens=256)
114
  response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=False)
115
  print(response)
116
+ # Expected: <|tool_call_start|>{"name": "exec", "arguments": {"command": "nmap -p 80 192.168.1.1"}}<|tool_call_end|>
117
  ```
118
 
119
+ ## Tool Call Format
120
+
121
+ Kali-Terminus-v2 produces native **LFM2.5 tool calls** wrapped in `<|tool_call_start|>` and `<|tool_call_end|>` tokens, output as JSON:
122
+
123
+ ```
124
+ <|tool_call_start|>{"name": "exec", "arguments": {"command": "nmap -p 80 192.168.1.1"}}<|tool_call_end|>
125
+ ```
126
+
127
+ A system instruction `Output function calls as JSON.` is prepended so the model emits JSON rather than the default Pythonic calls.
128
+
129
+ ## Tool Call Format
130
+
131
+ Kali-Terminus-v2 produces native **LFM2.5 tool calls** wrapped in `<|tool_call_start|>` and `<|tool_call_end|>` tokens, output as JSON:
132
+
133
+ ```
134
+ <|tool_call_start|>{"name": "exec", "arguments": {"command": "nmap -p 80 192.168.1.1"}}<|tool_call_end|>
135
+ ```
136
+
137
+ A system instruction `Output function calls as JSON.` is prepended so the model emits JSON rather than the default Pythonic calls.
138
+
139
  ## Intended Use
140
 
141
  - **Executor Agent** in multi-agent penetration-testing systems