Update README.md

README.md

@@ -19,4 +19,217 @@ widget:
   example_title: "Chinese Address Example"
 - text: "ROOM 2107, 42/F, WINNING HEIGHTS, 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES"
   example_title: "English Address Example"
----
+---
+
+# Qwen2.5-3B-Instruct Address Formatter (LoRA Adapter)
+
+This repository contains the LoRA adapter weights for fine-tuning the `Qwen/Qwen2.5-3B-Instruct` model on the task of **Hong Kong address parsing and formatting**.
+
+The model is designed to split unstructured, messy address text into two clean, structured lines following Hong Kong addressing conventions:
+- **Line 1**: Specific location details (e.g., floor, unit, building name)
+- **Line 2**: General location details (e.g., street, district, region)
+
+---
+
+## Model Details
+
+| Attribute | Value |
+|-----------|-------|
+| **Base Model** | [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) |
+| **Fine-tuning Method** | LoRA (Low-Rank Adaptation) |
+| **Task** | Address Parsing & Formatting |
+| **Language Support** | English, Traditional Chinese, Simplified Chinese |
+| **Quantization** | 4-bit (NF4) for training; supports fp16/4-bit for inference |
+| **License** | MIT |
+
+### Training Configuration
+
+| Hyperparameter | Value |
+|----------------|-------|
+| LoRA Rank (`r`) | 8 |
+| LoRA Alpha (`lora_alpha`) | 16 |
+| LoRA Dropout | 0.15 |
+| Target Modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
+| Learning Rate | 2e-5 |
+| Epochs | 3 |
+| Batch Size (per device) | 4 |
+| Gradient Accumulation Steps | 2 |
+| Optimizer | paged_adamw_8bit |
+| Scheduler | Cosine |
+| Weight Decay | 0.01 |
+| Max Sequence Length | 256 |
+
+### Post-Processing Features
+
+The associated inference pipeline includes intelligent post-processing:
+- **Character Pooling**: Matches model output characters to the original input.
+- **Omission Detection & Re-insertion**: Identifies missing characters and re-inserts them into the correct line.
+- **Language Detection**: Handles Chinese and English addresses with different strategies.
+- **Traditional Chinese Conversion**: Converts Simplified Chinese to Traditional using OpenCC.
+- **English Spell Checking**: Identifies and suggests corrections for misspelled words.
+
+---
+
+## Intended Use
+
+This model is intended for **Hong Kong address formatting** tasks, particularly for cleaning and structuring address data for logistics, e-commerce, and administrative applications.
+
+### Quick Start
+
+Load the adapter with the base model using the `peft` library:
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+
+# Load base model (with optional 4-bit quantization for memory efficiency)
+base_model_name = "Qwen/Qwen2.5-3B-Instruct"
+adapter_name = "ymlee13/Qwen2.5-3B-Instruct_Address_Formatter"
+
+quantization_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_use_double_quant=True,
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    base_model_name,
+    quantization_config=quantization_config,
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(base_model_name)
+model = PeftModel.from_pretrained(model, adapter_name)
+
+# Inference
+def format_address(address_text):
+    messages = [
+        {"role": "system", "content": "You are an address formatting assistant. You always return the formatted address with 'Line 1:' and 'Line 2:'."},
+        {"role": "user", "content": address_text},
+    ]
+    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+    
+    with torch.no_grad():
+        outputs = model.generate(**inputs, max_new_tokens=100, pad_token_id=tokenizer.eos_token_id)
+    
+    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    # Parse lines
+    import re
+    l1_match = re.search(r"Line\s*1:\s*(.+)", response, re.IGNORECASE)
+    l2_match = re.search(r"Line\s*2:\s*(.+)", response, re.IGNORECASE)
+    line1 = l1_match.group(1).strip() if l1_match else ""
+    line2 = l2_match.group(1).strip() if l2_match else ""
+    return line1, line2
+
+# Example usage
+print(format_address("九龍觀塘區雲漢街61號南寧大樓地庫01舖"))
+# Output: ('南寧大樓地庫01舖', '九龍觀塘區雲漢街61號')
+```
+For a complete inference pipeline with batch processing and advanced post-processing, please see the full project on GitHub: ymlee13/address_parsing
+
+---
+## Training Data
+The model was fine-tuned on a dataset of 570+ Hong Kong addresses generated from real geospatial data.
+
+### Data Sources
+- **ALS-GeoJSON**: Hong Kong government dataset containing detailed address information.
+- **Synthetic Generation**: Custom script (gen_real_address.ipynb) was used to create structured examples from the raw geospatial features.
+
+### Data Format
+Each training example consists of:
+- Input: A messy, unstructured address string.
+- Output: A structured JSON object with line1 and line2.
+
+Example:
+``` json
+  "input": "九龍觀塘區雲漢街61號南寧大樓地庫01舖",
+  "output": {"line1": "南寧大樓地庫01舖", "line2": "九龍觀塘區雲漢街61號"}
+```
+
+---
+## Performance & Evaluation
+
+### Evaluation Metrics
+The model was evaluated on a held-out test set using character-level similarity (SequenceMatcher ratio).
+
+| Metric | Score (Typical) |
+|--------|-----------------|
+| Average Line 1 Similarity | 75-90% |
+| Average Line 2 Similarity | 75-90% |
+| Strict Both Lines Match | 60-75% |
+| Inference Speed | ~0.5-1.0s/address |
+
+### Test Examples
+
+### Chinese Address
+Input:
+```text
+九龍觀塘區雲漢街61號南寧大樓地庫01舖
+```
+
+Output:
+```text
+Line 1: 南寧大樓地庫01舖
+Line 2: 九龍觀塘區雲漢街61號
+```
+
+### English Address
+Input:
+```text
+ROOM 2107, 42/F, WINNING HEIGHTS, 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES
+```
+
+Output:
+```text
+Line 1: FLAT 2107, 42/F, WINNING HEIGHTS
+Line 2: 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES
+```
+
+---
+## Limitations & Known Issues
+- **Geographic Scope**: Primarily optimized for Hong Kong addresses. Performance may degrade on addresses from other regions.
+- **Format Variations**: The model may struggle with highly unconventional or incomplete addresses.
+- **Language Mixing**: While it handles both Chinese and English, addresses with heavy code-switching may produce less consistent results.
+- **Hardware**: 4-bit quantization is recommended for GPUs with <16GB VRAM. For older GPUs (e.g., Tesla P40/Pascal), the inference script automatically disables 4-bit and uses fp16 to avoid compatibility issues.
+
+---
+## Usage Recommendations
+
+### Hardware Requirements
+| Mode | VRAM (Approx.) |
+|--------|-----------------|
+| Training (4-bit) | 12-14 GB |
+| Inference (4-bit) | 6-8 GB |
+| Inference (fp16) | 10-12 GB |
+
+### Environment
+```bash
+# Install dependencies
+pip install torch transformers accelerate peft bitsandbytes
+```
+
+### Citation
+If you use this model in your research or project, please consider citing:
+```bbibtex
+@misc{ymlee13_2024_address_parser,
+  author = {Lee, Y.M.},
+  title = {Qwen2.5-3B-Instruct Address Formatter},
+  year = {2024},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/ymlee13/Qwen2.5-3B-Instruct_Address_Formatter}}
+}
+```
+
+## Acknowledgements
+- Qwen Team for the excellent base model.
+- Hugging Face for the transformers and peft libraries.
+- Hong Kong Government for providing the ALS-GeoJSON dataset.
+- Open Source Community for tools like trl, bitsandbytes, and accelerate.
+
+## Contact & Support
+For questions, issues, or collaborations:
+- Author: ymlee13
+- GitHub: ymlee13/address_parsing
+- Hugging Face: ymlee13