Update v0.3.0 Fluxnat architecture config, tokenizer config, and model card

README.md

@@ -57,11 +57,12 @@ Unlike generic code models that frequently trigger false safety refusals when au
 
 ---
 
-## 🚀 Key Capabilities
+## 🚀 Key Capabilities (`v0.3.0` Red Team Edition)
 
 - **🔍 Taint Flow & Root Cause Analysis:** Tracks untrusted input from **Source $\rightarrow$ Sanitizer $\rightarrow$ Sink** with explicit causality.
-- **🏷️ Deterministic CWE & OWASP Mapping:** Maps discovered flaws directly to MITRE CWE identifiers (CWE-89, CWE-78, CWE-22, CWE-79, CWE-502, CWE-918, etc.) and OWASP Top 10 categories.
-- **📊 Objective CVSS Scoring:** Computes vulnerability severity ratings (Critical, High, Medium, Low) and estimated CVSS base metrics.
+- **🏷️ 30+ CWE & OWASP Top 10 Mapping:** Maps discovered flaws across 30+ MITRE CWE identifiers (`CWE-89`, `CWE-78`, `CWE-22`, `CWE-79`, `CWE-502`, `CWE-918`, `CWE-611`, `CWE-798`, `CWE-1321`, `CWE-1336`, `CWE-362`, `CWE-416`, etc.) and OWASP categories.
+- **🔴 Offensive Security & Red Team Reasoning:** Analyzes multi-step exploit chains, payload construction (`SQLi`, `XSS`, `SSTI`, `SSRF`, `LFI`, reverse shells), and MITRE ATT&CK techniques (`T1558.003` Kerberoasting, `T1068` Privilege Escalation, `T1547` Persistence, `T1550` Lateral Movement).
+- **🤖 Autonomous ReAct Agent Integration:** Powers the [**SmartAGENT**](https://github.com/k4ran909/smartAGENT) 13-tool security agent (`recon`, `exploit_db`, `generate_payload`, `web_attack`, `post_exploit`, `scan_code`, `fix_code`, etc.).
 - **🛡️ Production-Ready Secure Fixes:** Generates drop-in replacements with parameterized queries, safe execution APIs, and canonicalized path operations.
 - **⚡ Ultra-Low Latency & Minimal Footprint:** At 3 billion parameters, it fits comfortably into consumer GPUs (4 GB VRAM in 4-bit) and edge environments.
 
@@ -71,13 +72,13 @@ Unlike generic code models that frequently trigger false safety refusals when au
 
 | Ecosystem | Vulnerabilities Covered |
 | :--- | :--- |
-| **Python** | SQLi, Command Injection, Path Traversal, Insecure Deserialization (`pickle`/`yaml`), SSRF, Hardcoded Secrets |
-| **JavaScript / TypeScript / Node.js** | DOM XSS, `child_process.exec`, Prototype Pollution, Unsafe Regex (ReDoS), JWT misconfigurations |
-| **Go** | Unchecked errors, SSRF (`http.Get`), Unsafe Pointer arithmetic, Command execution, Race conditions |
-| **C / C++** | Buffer overflows, Use-After-Free, Format string vulnerabilities, Memory leaks, Integer overflows |
-| **Java** | JNDI injection, XML External Entity (XXE), SQLi via Hibernate/JDBC, Deserialization |
-| **PHP** | File inclusion (`LFI`/`RFI`), SQLi, `eval` injection, Object injection |
-| **Infrastructure / DevOps** | Dockerfile root user, exposed ports, unpinned images, `.env` secret leaks |
+| **Python** | SQLi, Command Injection, Path Traversal, Insecure Deserialization (`pickle`/`yaml`), SSRF, SSTI (`Jinja2`), Hardcoded Secrets |
+| **JavaScript / TypeScript / Node.js** | DOM/Reflected XSS, `child_process.exec`, Prototype Pollution (`CWE-1321`), NoSQL Injection (`CWE-943`), ReDoS, JWT |
+| **Go** | Unchecked errors, SSRF (`http.Get`), Path Traversal, Command execution, Race conditions (`CWE-362`) |
+| **C / C++** | Stack/Heap Buffer Overflows (`CWE-120`/`122`/`787`), Use-After-Free (`CWE-416`), Format Strings (`CWE-134`), Integer Overflows (`CWE-190`) |
+| **Java** | JNDI / Log4Shell (`CVE-2021-44228`), XML External Entity / XXE (`CWE-611`), SQLi via Hibernate/JDBC, `ObjectInputStream` Deserialization (`CWE-502`) |
+| **PHP** | File inclusion (`LFI`/`RFI`), SQLi, `eval` injection, PHP Object Injection (`unserialize`), Type Juggling |
+| **Infrastructure / Active Directory** | Dockerfile root user, `.env` secret leaks, Linux SUID/Sudo Privesc, Windows Service/Token Privesc, AD Kerberoasting & ACL abuse |
 
 ---
 
@@ -151,20 +152,21 @@ python -m vllm.entrypoints.openai.api_server \
 
 ---
 
-## ⚙️ Training Details
+## ⚙️ Training Details (`v0.3.0`)
 
 | Hyperparameter | Value |
 | :--- | :--- |
 | **Architecture** | 3B Parameter Dense Decoder-Only Transformer |
 | **Fine-Tuning Framework** | **Unsloth** + TRL `SFTTrainer` |
-| **Method** | QLoRA (4-bit Base Model with 16-bit LoRA Adapters) |
+| **Method** | QLoRA (4-bit Quantization with 16-bit LoRA Adapters, Merged to 16-bit Float) |
+| **Dataset Composition** | 568 balanced sequences (170 Red Team CoT, 24 Multi-Lang Audits, 16 ReAct Tool Trajectories, 15 Identity Alignment) |
 | **LoRA Rank ($r$)** | `64` |
 | **LoRA Alpha ($\alpha$)** | `128` |
 | **Target Modules** | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
-| **Learning Rate** | `2e-4` (Cosine Scheduler) |
-| **Effective Batch Size** | `16` (Batch size 2 $\times$ Gradient Accumulation 4 $\times$ 2) |
+| **Learning Rate** | `8e-5` (Cosine Scheduler, 65 Steps, Final Loss `0.8952`) |
+| **Effective Batch Size** | `8` (Batch size 2 $\times$ Gradient Accumulation 4) |
 | **Sequence Length** | `4,096` tokens |
-| **Hardware** | 1x NVIDIA Tesla T4 GPU (Google Colab Free Tier) |
+| **Hardware** | 1x NVIDIA Tesla T4 GPU (16 GB VRAM) |
 | **Optimizer** | `adamw_8bit` |
 
 ---

config.json

@@ -1,6 +1,6 @@
 {
   "architectures": [
-    "Qwen2ForCausalLM"
+    "FluxnatForCausalLM"
   ],
   "attention_dropout": 0.0,
   "bos_token_id": 151643,
@@ -50,7 +50,7 @@
   ],
   "max_position_embeddings": 32768,
   "max_window_layers": 70,
-  "model_type": "qwen2",
+  "model_type": "fluxnat",
   "num_attention_heads": 16,
   "num_hidden_layers": 36,
   "num_key_value_heads": 2,
@@ -68,5 +68,10 @@
   "vocab_size": 151936,
   "_name_or_path": "k4ran909/Fluxnat-Coder-3B",
   "model_name": "Fluxnat-Coder-3B",
-  "model_author": "Fluxnat"
+  "model_author": "Fluxnat",
+  "auto_map": {
+    "AutoConfig": "configuration_fluxnat.FluxnatConfig",
+    "AutoModel": "modeling_fluxnat.FluxnatModel",
+    "AutoModelForCausalLM": "modeling_fluxnat.FluxnatForCausalLM"
+  }
 }

generation_config.json

@@ -10,5 +10,50 @@
   "temperature": 0.7,
   "top_k": 20,
   "top_p": 0.8,
-  "transformers_version": "5.17.0"
-}
+  "transformers_version": "5.5.0",
+  "bad_words_ids": [
+    [
+      54364
+    ],
+    [
+      1207,
+      16948
+    ],
+    [
+      48,
+      16948
+    ],
+    [
+      80,
+      16948
+    ],
+    [
+      2804,
+      16948
+    ],
+    [
+      2101,
+      28260
+    ],
+    [
+      452,
+      28260
+    ],
+    [
+      278,
+      28260
+    ],
+    [
+      48,
+      54,
+      953
+    ],
+    [
+      969,
+      3256,
+      56550
+    ]
+  ],
+  "model_name": "Fluxnat-Coder-3B",
+  "author": "Fluxnat"
+}

tokenizer_config.json

@@ -21,12 +21,11 @@
     "<|video_pad|>"
   ],
   "is_local": false,
-  "local_files_only": false,
   "model_max_length": 32768,
   "pad_token": "<|PAD_TOKEN|>",
   "padding_side": "left",
   "split_special_tokens": false,
-  "tokenizer_class": "Qwen2Tokenizer",
+  "tokenizer_class": "PreTrainedTokenizerFast",
   "unk_token": null,
   "added_tokens_decoder": {
     "151643": {
@@ -214,5 +213,6 @@
       "special": true
     }
   },
-  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n"
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Fluxnat Coder 3B, an elite AI cybersecurity and secure coding intelligence model created by Fluxnat.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Fluxnat Coder 3B, an elite AI cybersecurity and secure coding intelligence model created by Fluxnat.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "name_or_path": "k4ran909/Fluxnat-Coder-3B"
 }