Blame for README.md

---
language:
- en
- zh
tags:
- hk-address-parser
- address-parsing
- peft
- lora
- qwen
- qwen2.5
- causal-lm
license: mit
base_model: Qwen/Qwen2.5-3B-Instruct
library_name: transformers
pipeline_tag: text-generation
widget:
- text: "九龍觀塘區雲漢街61號南寧大樓地庫01舖"
  example_title: "Chinese Address Example"
- text: "ROOM 2107, 42/F, WINNING HEIGHTS, 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES"
  example_title: "English Address Example"
---

# Qwen2.5-3B-Instruct Address Formatter (LoRA Adapter)

This repository contains the LoRA adapter weights for fine-tuning the `Qwen/Qwen2.5-3B-Instruct` model on the task of **Hong Kong address parsing and formatting**.

The model is designed to split unstructured, messy address text into two clean, structured lines following Hong Kong addressing conventions:
- **Line 1**: Specific location details (e.g., floor, unit, building name)
- **Line 2**: General location details (e.g., street, district, region)

---

## Model Details

| Attribute | Value |
|-----------|-------|
| **Base Model** | [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) |
| **Fine-tuning Method** | LoRA (Low-Rank Adaptation) |
| **Task** | Address Parsing & Formatting |
| **Language Support** | English, Traditional Chinese, Simplified Chinese |
| **Quantization** | 4-bit (NF4) for training; supports fp16/4-bit for inference |
| **License** | MIT |

### Training Configuration

| Hyperparameter | Value |
|----------------|-------|
| LoRA Rank (`r`) | 8 |
| LoRA Alpha (`lora_alpha`) | 16 |
| LoRA Dropout | 0.15 |
| Target Modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
| Learning Rate | 2e-5 |
| Epochs | 3 |
| Batch Size (per device) | 4 |
| Gradient Accumulation Steps | 2 |
| Optimizer | paged_adamw_8bit |
| Scheduler | Cosine |
| Weight Decay | 0.01 |
| Max Sequence Length | 256 |

### Post-Processing Features

The associated inference pipeline includes intelligent post-processing:
- **Character Pooling**: Matches model output characters to the original input.
- **Omission Detection & Re-insertion**: Identifies missing characters and re-inserts them into the correct line.
- **Language Detection**: Handles Chinese and English addresses with different strategies.
- **Traditional Chinese Conversion**: Converts Simplified Chinese to Traditional using OpenCC.
- **English Spell Checking**: Identifies and suggests corrections for misspelled words.

---

## Intended Use

This model is intended for **Hong Kong address formatting** tasks, particularly for cleaning and structuring address data for logistics, e-commerce, and administrative applications.

### Quick Start

Load the adapter with the base model using the `peft` library:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Load base model (with optional 4-bit quantization for memory efficiency)
base_model_name = "Qwen/Qwen2.5-3B-Instruct"
adapter_name = "ymlee13/Qwen2.5-3B-Instruct_Address_Formatter"

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=quantization_config,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, adapter_name)

# Inference
def format_address(address_text):
    messages = [
        {"role": "system", "content": "You are an address formatting assistant. You always return the formatted address with 'Line 1:' and 'Line 2:'."},
        {"role": "user", "content": address_text},
    ]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=100, pad_token_id=tokenizer.eos_token_id)
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    # Parse lines
    import re
    l1_match = re.search(r"Line\s*1:\s*(.+)", response, re.IGNORECASE)
    l2_match = re.search(r"Line\s*2:\s*(.+)", response, re.IGNORECASE)
    line1 = l1_match.group(1).strip() if l1_match else ""
    line2 = l2_match.group(1).strip() if l2_match else ""
    return line1, line2

# Example usage
print(format_address("九龍觀塘區雲漢街61號南寧大樓地庫01舖"))
# Output: ('南寧大樓地庫01舖', '九龍觀塘區雲漢街61號')
```
For a complete inference pipeline with batch processing and advanced post-processing, please see the full project on GitHub: ymlee13/address_parsing

---

## Training Data

The model was fine-tuned on a dataset of 570+ Hong Kong addresses generated from real geospatial data.

### Data Sources

- **ALS-GeoJSON**: Hong Kong government dataset containing detailed address information.
- **Synthetic Generation**: Custom script (gen_real_address.ipynb) was used to create structured examples from the raw geospatial features.

### Data Format

Each training example consists of:
- Input: A messy, unstructured address string.
- Output: A structured JSON object with line1 and line2.
- 
Example:
``` json
  "input": "九龍觀塘區雲漢街61號南寧大樓地庫01舖",
  "output": {"line1": "南寧大樓地庫01舖", "line2": "九龍觀塘區雲漢街61號"}
```

---

## Performance & Evaluation

### Evaluation Metrics

The model was evaluated on a held-out test set using character-level similarity (SequenceMatcher ratio).

| Metric | Score (Typical) |
|--------|-----------------|
| Average Line 1 Similarity | 75-90% |
| Average Line 2 Similarity | 75-90% |
| Strict Both Lines Match | 60-75% |
| Inference Speed | ~0.5-1.0s/address |

### Test Examples

### Chinese Address

Input:
```text
九龍觀塘區雲漢街61號南寧大樓地庫01舖
```

Output:
```text
Line 1: 南寧大樓地庫01舖
Line 2: 九龍觀塘區雲漢街61號
```

### English Address

Input:
```text
ROOM 2107, 42/F, WINNING HEIGHTS, 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES
```

Output:
```text
Line 1: FLAT 2107, 42/F, WINNING HEIGHTS
Line 2: 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES
```

---

## Limitations & Known Issues

- **Geographic Scope**: Primarily optimized for Hong Kong addresses. Performance may degrade on addresses from other regions.
- **Format Variations**: The model may struggle with highly unconventional or incomplete addresses.
- **Language Mixing**: While it handles both Chinese and English, addresses with heavy code-switching may produce less consistent results.
- **Hardware**: 4-bit quantization is recommended for GPUs with <16GB VRAM. For older GPUs (e.g., Tesla P40/Pascal), the inference script automatically disables 4-bit and uses fp16 to avoid compatibility issues.

---

## Usage Recommendations

### Hardware Requirements

| Mode | VRAM (Approx.) |
|--------|-----------------|
| Training (4-bit) | 12-14 GB |
| Inference (4-bit) | 6-8 GB |
| Inference (fp16) | 10-12 GB |

### Environment

```bash
# Install dependencies
pip install torch transformers accelerate peft bitsandbytes
```

### Citation

If you use this model in your research or project, please consider citing:
```bbibtex
@misc{ymlee13_2024_address_parser,
  author = {Lee, Y.M.},
  title = {Qwen2.5-3B-Instruct Address Formatter},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ymlee13/Qwen2.5-3B-Instruct_Address_Formatter}}
}
```

---

## Acknowledgements
- Qwen Team for the excellent base model.
- Hugging Face for the transformers and peft libraries.
- Hong Kong Government for providing the ALS-GeoJSON dataset.
- Open Source Community for tools like trl, bitsandbytes, and accelerate.

---

## Contact & Support
For questions, issues, or collaborations:
- Author: ymlee13
- GitHub: ymlee13/address_parsing
- Hugging Face: ymlee13