README.md

---
datasets:
- stellaray777/1000s-websites
language:
- en
base_model:
- deepseek-ai/deepseek-coder-6.7b-base
tags:
- code
- web-design
- frontend
- html
- css
- javascript
- lora
- peft
---

# DeepSeek Coder 6.7B - Website Design Fine-tuned

A fine-tuned version of DeepSeek Coder 6.7B specifically trained to generate website designs based on design specifications and requirements.

## Model Description

This model is a LoRA fine-tuned version of `deepseek-ai/deepseek-coder-6.7b-base` trained on a dataset of website designs. The model has been specialized to understand design requirements (industry, tone, layout, etc.) and generate appropriate HTML/CSS/JavaScript implementations for brand-specific websites.

- **Base Model**: `deepseek-ai/deepseek-coder-6.7b-base`
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **Training Dataset**: `stellaray777/1000s-websites`
- **Quantization**: 4-bit (NF4) with BitsAndBytes

## Training Details

### Training Configuration

- **Epochs**: 3
- **Batch Size**: 1 (gradient accumulation: 4)
- **Learning Rate**: 2e-4
- **LoRA Configuration**:
  - `r`: 8
  - `alpha`: 16
  - `target_modules`: ["q_proj", "v_proj"]
  - `lora_dropout`: 0.05
- **Max Sequence Length**: 2048 tokens
- **Optimizer**: paged_adamw_8bit
- **Learning Rate Scheduler**: cosine
- **Warmup Steps**: 50

### Training Infrastructure

- **Quantization**: 4-bit NF4 quantization for memory efficiency
- **Gradient Checkpointing**: Enabled
- **Mixed Precision**: FP16

## Intended Use

This model is designed for:

- Generating website designs based on design specifications
- Creating HTML/CSS/JavaScript code for brand-specific websites
- Understanding design requirements (industry, tone, layout, photo usage, motion) and translating them to code
- Assisting front-end developers in rapid website prototyping

## How to Use

### Installation

```bash
pip install transformers torch peft bitsandbytes accelerate
```

### Basic Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = "deepseek-ai/deepseek-coder-6.7b-base"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    load_in_4bit=True,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Load LoRA weights
model = PeftModel.from_pretrained(model, "stellaray777/1000s-websites")

# Prepare input
messages = [
    {
        "role": "system",
        "content": "You are a senior creative front-end engineer who designs brand-specific websites."
    },
    {
        "role": "user",
        "content": "Industry: Healthcare\nTone: Professional, Trustworthy\nPage type: Landing page\nLayout: Grid-based\nPhoto usage: Medium\nTask: Design the website based on the provided HTML structure and styling."
    }
]

# Generate response
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.7)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

### Interactive Chatbot

Use the provided test script:

```bash
# From Hugging Face Hub
python src/test_trained_model.py

# From local cloned model
python src/test_trained_model.py --local
```

## Limitations

- The model is fine-tuned on a specific dataset of website designs and may not generalize well to all design styles
- Generated code may require manual review and adjustments
- The model may not always produce production-ready code and should be used as a starting point
- Performance depends on the quality and specificity of the design requirements provided

## Model Card Contact

For questions, issues, or contributions, please refer to the main project repository.

## Citation

If you use this model, please cite:

```bibtex
@misc{deepseek-coder-website-design,
  title={DeepSeek Coder 6.7B - Website Design Fine-tuned},
  author={Stellaray777},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/stellaray777/1000s-websites}}
}
```