README.md

# DevOps Qwen — Fine-Tuned Qwen2.5-3B-Instruct

A Qwen2.5-3B-Instruct model fine-tuned with **LoRA (Low-Rank Adaptation)** on a DevOps-focused dataset.

The model is designed for practical DevOps, Cloud, Linux, Docker, Kubernetes, Terraform, CI/CD, networking, monitoring, and troubleshooting questions.

## Model Details

| Property            | Value                          |
| ------------------- | ------------------------------ |
| Base Model          | Qwen/Qwen2.5-3B-Instruct       |
| Model Type          | Causal Language Model          |
| Fine-Tuning         | LoRA                           |
| LoRA Rank           | 8                              |
| LoRA Alpha          | 16                             |
| LoRA Dropout        | 0.05                           |
| Target Modules      | q_proj, k_proj, v_proj, o_proj |
| Training Epochs     | 1                              |
| Max Sequence Length | 512                            |
| Quantization        | Q4_K_M                         |
| Format              | GGUF                           |
| Approx. Model Size  | 1.9 GB                         |
| Runtime             | Ollama / llama.cpp             |
| Primary Use         | DevOps AI Assistant            |

## What is this model?

This model is a specialized version of Qwen2.5-3B-Instruct trained on DevOps-oriented examples.

The goal of the fine-tuning is to improve the model's ability to provide practical responses for:

* Linux administration
* AWS
* GCP
* Docker
* Kubernetes
* Terraform
* Git
* Jenkins
* CI/CD
* Networking
* HTTP troubleshooting
* Monitoring
* Production troubleshooting
* Cloud infrastructure

The model is intended to provide answers with:

1. Root cause or explanation
2. Exact commands where appropriate
3. Short explanation of commands
4. Production-safe troubleshooting steps

## Fine-Tuning Approach

The model was fine-tuned using **LoRA — Low-Rank Adaptation**.

Instead of updating the entire base model, LoRA trains a small set of additional parameters while keeping most of the original model frozen.

```text
Qwen2.5-3B-Instruct
        |
        v
   LoRA Training
        |
        v
   LoRA Adapter
        |
        v
Merge Adapter + Base Model
        |
        v
   Merged Model
```

This approach reduces training memory and computational requirements compared with full fine-tuning.

## Training Configuration

```text
Base Model:
Qwen/Qwen2.5-3B-Instruct

LoRA:
r = 8
alpha = 16
dropout = 0.05

Target modules:
q_proj
k_proj
v_proj
o_proj

Epochs:
1

Batch size:
1

Gradient accumulation:
4

Learning rate:
2e-4

Maximum sequence length:
512
```

## Model Conversion

After LoRA training, the adapter was merged with the base model.

The merged Hugging Face model was then converted to GGUF using llama.cpp.

```text
LoRA Adapter
     |
     v
Merged Hugging Face Model
     |
     v
GGUF F16
     |
     v
Q4_K_M Quantization
     |
     v
qwen-devops-q4_k_m.gguf
```

### GGUF

**GGUF (GPT-Generated Unified Format)** is an efficient model format commonly used for local LLM inference with llama.cpp and compatible runtimes.

### Q4_K_M

Q4_K_M is a 4-bit quantization format.

It reduces model storage and memory requirements while maintaining a useful level of model quality for local inference.

Approximate sizes:

```text
Merged Hugging Face Model  ≈ 12 GB
GGUF F16                   ≈ 5.8 GB
Q4_K_M GGUF                ≈ 1.9 GB
```

## Hardware Used

The model was developed and tested in a CPU-only environment.

```text
CPU:
AMD EPYC 7543

CPU cores available:
12

RAM:
~57 GB

GPU:
None

CUDA:
False

Python:
3.10.14
```

## Usage with Ollama

Download the GGUF model from this repository.

Create a `Modelfile`:

```text
FROM ./qwen-devops-q4_k_m.gguf

PARAMETER temperature 0.2
PARAMETER top_k 20
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 4096

SYSTEM """
You are a senior DevOps and Cloud engineer.

Give practical and accurate technical answers.

For Linux, AWS, Docker, Kubernetes, Terraform, Git,
Jenkins, CI/CD, monitoring, networking and troubleshooting:

- Explain the root cause.
- Give exact commands when appropriate.
- Explain commands briefly.
- Do not invent information.
- If you don't know something, clearly say so.
- Prefer safe production-ready solutions.
"""
```

Create the Ollama model:

```bash
ollama create devops-qwen -f Modelfile
```

Run:

```bash
ollama run devops-qwen
```

## Example

Question:

```text
How do I troubleshoot a 502 Bad Gateway error from an AWS ALB?
```

The model is intended to provide a structured troubleshooting approach such as:

```text
1. Check ALB target health
2. Verify application is listening on the expected port
3. Check security groups
4. Check target response
5. Review ALB access logs
6. Review application logs
7. Test the target directly
8. Check health-check configuration
```

Example commands may include:

```bash
ss -lntp
curl -v http://127.0.0.1:8080/
curl -v http://TARGET_PRIVATE_IP:8080/
```

## Ollama API

Non-streaming request:

```bash
curl http://localhost:11434/api/generate \
  -d '{
    "model": "devops-qwen",
    "prompt": "How do I check disk usage in Linux?",
    "stream": false
  }'
```

Streaming request:

```bash
curl http://localhost:11434/api/generate \
  -d '{
    "model": "devops-qwen",
    "prompt": "How do I troubleshoot Kubernetes CrashLoopBackOff?",
    "stream": true
  }'
```

## llama.cpp

The GGUF model can also be used with llama.cpp:

```bash
./llama-cli \
  -m qwen-devops-q4_k_m.gguf
```

## Recommended Generation Parameters

For technical and DevOps questions:

```text
temperature     = 0.2
top_k           = 20
top_p           = 0.9
repeat_penalty  = 1.1
context         = 4096
```

Lower temperature is used to encourage more deterministic and consistent technical responses.

## Fine-Tuning vs RAG

This model should not be considered a replacement for RAG.

Fine-tuning is useful for:

* Response style
* Domain behavior
* Task patterns
* DevOps troubleshooting patterns
* Command-oriented responses

RAG is useful for:

* Company documentation
* Current infrastructure information
* Internal runbooks
* AWS architecture documentation
* Frequently changing configuration
* Private knowledge bases

Recommended architecture:

```text
User
 |
 v
Chat UI
 |
 v
n8n / FastAPI
 |
 v
RAG Retriever
 |
 v
Vector Database
 |
 v
Relevant DevOps Documents
 |
 v
Context
 |
 v
devops-qwen
 |
 v
Final Answer
```

## Intended Use

This model is intended for:

* DevOps assistants
* Cloud troubleshooting assistants
* Linux support
* Infrastructure automation
* CI/CD assistance
* Kubernetes troubleshooting
* Terraform assistance
* Internal technical assistants
* RAG-based DevOps assistants

## Limitations

The model is relatively small at approximately 3B parameters.

It may:

* Make incorrect technical assumptions
* Produce outdated information
* Generate commands that require environment-specific changes
* Fail on complex infrastructure architecture
* Require RAG or external tools for current infrastructure information

Always verify commands before running them in production.

For production environments, use appropriate:

* Backups
* Change management
* Testing
* Access controls
* Approval processes

## Security

Do not provide the model with:

* AWS access keys
* Private SSH keys
* Passwords
* API tokens
* Database credentials
* TLS private keys
* Other secrets

When integrating this model with automation, use least-privilege credentials and approval controls for destructive operations.

## Project Pipeline

```text
DevOps Dataset
      |
      v
JSONL Validation
      |
      v
Train / Validation Split
      |
      v
Qwen2.5-3B-Instruct
      |
      v
LoRA Fine-Tuning
      |
      v
LoRA Adapter
      |
      v
Merge
      |
      v
Merged Model
      |
      v
GGUF F16
      |
      v
Q4_K_M
      |
      v
Ollama
      |
      v
devops-qwen
      |
      v
API / n8n / RAG
```

## Benchmark

The project includes an automated benchmark comparing:

```text
qwen2.5:3b
       VS
devops-qwen
```

The benchmark contains 10 DevOps questions covering:

* Linux
* AWS ALB
* Docker
* CPU/RAM
* Disk usage
* Terraform
* Kubernetes
* HTTP
* Production troubleshooting

Benchmark output:

```text
benchmark_results.json
```

## Model Card Summary

```text
Model:
DevOps Qwen

Base:
Qwen2.5-3B-Instruct

Fine-Tuning:
LoRA

Format:
GGUF

Quantization:
Q4_K_M

Size:
~1.9 GB

Runtime:
Ollama / llama.cpp

Domain:
DevOps / Cloud / Infrastructure

Recommended:
CPU local inference + RAG
```

## License

This model is derived from Qwen2.5-3B-Instruct.

Users should review and comply with the applicable Qwen model license and its terms before using or redistributing this model, particularly for commercial use.

The fine-tuning dataset and any additional project components may have their own applicable terms.

## Disclaimer

This model is an experimental DevOps-focused AI assistant. It is not a substitute for production change-control procedures or expert review.

Always validate generated commands and infrastructure changes before applying them to production systems.