Rapnss DevOps-Ultra-125M 🚀

DevOps-Ultra-125M is a highly optimized, lightweight 125 Million parameter language model designed specifically for Python code generation and code editing. It was built from scratch and trained entirely by Rapnss.

This model features a completely custom Transformer architecture (RapnssCodeTransformer v2) that incorporates state-of-the-art AI technologies like RoPE, SwiGLU, and RMSNorm, making it highly efficient for its size.

---

📌 Model Details

• Model Name: DevOps-Ultra-125M
• Parent Company: Rapnss
• Model Type: Causal Language Model (Auto-regressive Next-Token Prediction)
• Parameter Count: ~125 Million
• Language: Python
• Architecture: Custom RapnssCodeTransformer (Built in PyTorch)

Architecture Enhancements (v2)

Unlike a standard GPT-2 model, DevOps-Ultra-125M uses modern LLM techniques inspired by LLaMA and Mistral:

1. Rotary Position Embeddings (RoPE): Replaces standard absolute position embeddings for a much better understanding of sequence context and length.
2. SwiGLU Activations: Replaces standard GELU/ReLU in the Feed-Forward networks for improved reasoning capabilities.
3. RMSNorm (Root Mean Square Normalization): Replaces standard Layer Normalization for faster and more stable training.

---

💻 Hardware & Training Details

DevOps-Ultra-125M was purposefully designed to be trained and run on edge devices and consumer hardware.

• Hardware Used: NVIDIA RTX 3050 Laptop GPU (4GB VRAM)
• Training Time: ~1.5 Hours
• Epochs: 2
• Dataset: Fine-tuned from scratch on iamtarun/python_code_instructions_18k_alpaca (18,000+ instruction-following Python code snippets).
• VRAM Optimizations: Trained using Automatic Mixed Precision (AMP) and Gradient Accumulation (Effective Batch Size of 8) to fit within strict 4GB VRAM limitations without crashing.

---

🚀 How to Use the Model

Because DevOps-Ultra-125M uses a completely custom architecture, you cannot load it using the standard Hugging Face AutoModelForCausalLM. You must download the rapnss_model.py architecture file alongside the .pt weights.

1. Requirements

Ensure you have PyTorch and Transformers installed:

pip install torch transformers

2. Loading the Model

Place the rapnss_model.py file in your directory and run the following code:

import torch
from transformers import AutoTokenizer
from rapnss_model import RapnssCodeTransformer, RapnssConfig

# 1. Load the GPT-2 Tokenizer
tokenizer = AutoTokenizer.from_pretrained("gpt2")
tokenizer.pad_token = tokenizer.eos_token

# 2. Initialize the Rapnss Architecture
config = RapnssConfig(
    vocab_size=tokenizer.vocab_size,
    max_seq_len=512
)
model = RapnssCodeTransformer(config)

# 3. Load the Weights
# Make sure you have downloaded 'Rapnss-Coder-125M.pt' from this repository
model.load_state_dict(torch.load("Rapnss-Coder-125M.pt", map_location='cpu'))
model.eval()

# 4. Generate Code
instruction = "Write a Python program to print Hello World!"
prompt = f"Instruction: {instruction}\nOutput:\n"
input_ids = tokenizer.encode(prompt, return_tensors="pt")

output_ids = model.generate(input_ids, max_new_tokens=50, temperature=0.7)
response = tokenizer.decode(output_ids[0], skip_special_tokens=True)

print(response)

---

⚠️ Limitations & Bias

DevOps-Ultra-125M is a Proof-of-Concept model.

• Semantic Logic: With only 125 million parameters, the model is incredibly small. While it has successfully learned the syntax, grammar, and structural formatting of Python code, it lacks the parameter capacity for advanced reasoning or solving complex coding algorithms.
• Scope: It only knows Python. It cannot generate HTML, CSS, JavaScript, or general conversational text.
• Use Case: It is best used for educational purposes, architecture testing, or running extremely fast on CPU-only free-tier servers.

---

Developed with ❤️ by Rapnss