File size: 4,222 Bytes
a2f88df 450a4c4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 | ---
datasets:
- stellaray777/1000s-websites
language:
- en
base_model:
- deepseek-ai/deepseek-coder-6.7b-base
tags:
- code
- web-design
- frontend
- html
- css
- javascript
- lora
- peft
---
# DeepSeek Coder 6.7B - Website Design Fine-tuned
A fine-tuned version of DeepSeek Coder 6.7B specifically trained to generate website designs based on design specifications and requirements.
## Model Description
This model is a LoRA fine-tuned version of `deepseek-ai/deepseek-coder-6.7b-base` trained on a dataset of website designs. The model has been specialized to understand design requirements (industry, tone, layout, etc.) and generate appropriate HTML/CSS/JavaScript implementations for brand-specific websites.
- **Base Model**: `deepseek-ai/deepseek-coder-6.7b-base`
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **Training Dataset**: `stellaray777/1000s-websites`
- **Quantization**: 4-bit (NF4) with BitsAndBytes
## Training Details
### Training Configuration
- **Epochs**: 3
- **Batch Size**: 1 (gradient accumulation: 4)
- **Learning Rate**: 2e-4
- **LoRA Configuration**:
- `r`: 8
- `alpha`: 16
- `target_modules`: ["q_proj", "v_proj"]
- `lora_dropout`: 0.05
- **Max Sequence Length**: 2048 tokens
- **Optimizer**: paged_adamw_8bit
- **Learning Rate Scheduler**: cosine
- **Warmup Steps**: 50
### Training Infrastructure
- **Quantization**: 4-bit NF4 quantization for memory efficiency
- **Gradient Checkpointing**: Enabled
- **Mixed Precision**: FP16
## Intended Use
This model is designed for:
- Generating website designs based on design specifications
- Creating HTML/CSS/JavaScript code for brand-specific websites
- Understanding design requirements (industry, tone, layout, photo usage, motion) and translating them to code
- Assisting front-end developers in rapid website prototyping
## How to Use
### Installation
```bash
pip install transformers torch peft bitsandbytes accelerate
```
### Basic Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load base model and tokenizer
base_model = "deepseek-ai/deepseek-coder-6.7b-base"
model = AutoModelForCausalLM.from_pretrained(
base_model,
load_in_4bit=True,
device_map="auto",
trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(base_model)
# Load LoRA weights
model = PeftModel.from_pretrained(model, "stellaray777/1000s-websites")
# Prepare input
messages = [
{
"role": "system",
"content": "You are a senior creative front-end engineer who designs brand-specific websites."
},
{
"role": "user",
"content": "Industry: Healthcare\nTone: Professional, Trustworthy\nPage type: Landing page\nLayout: Grid-based\nPhoto usage: Medium\nTask: Design the website based on the provided HTML structure and styling."
}
]
# Generate response
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```
### Interactive Chatbot
Use the provided test script:
```bash
# From Hugging Face Hub
python src/test_trained_model.py
# From local cloned model
python src/test_trained_model.py --local
```
## Limitations
- The model is fine-tuned on a specific dataset of website designs and may not generalize well to all design styles
- Generated code may require manual review and adjustments
- The model may not always produce production-ready code and should be used as a starting point
- Performance depends on the quality and specificity of the design requirements provided
## Model Card Contact
For questions, issues, or contributions, please refer to the main project repository.
## Citation
If you use this model, please cite:
```bibtex
@misc{deepseek-coder-website-design,
title={DeepSeek Coder 6.7B - Website Design Fine-tuned},
author={Stellaray777},
year={2024},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/stellaray777/1000s-websites}}
}
```
|