ymlee13/Qwen2.5-3B-Instruct_Address_Formatter

Qwen2.5-3B-Instruct Address Formatter (LoRA Adapter)

This repository contains the LoRA adapter weights for fine-tuning the Qwen/Qwen2.5-3B-Instruct model on the task of Hong Kong address parsing and formatting.

The model is designed to split unstructured, messy address text into two clean, structured lines following Hong Kong addressing conventions:

• —Line 1: Specific location details (e.g., floor, unit, building name)
• —Line 2: General location details (e.g., street, district, region)

Model Details

Training Configuration

Post-Processing Features

The associated inference pipeline includes intelligent post-processing:

• —Character Pooling: Matches model output characters to the original input.
• —Omission Detection & Re-insertion: Identifies missing characters and re-inserts them into the correct line.
• —Language Detection: Handles Chinese and English addresses with different strategies.
• —Traditional Chinese Conversion: Converts Simplified Chinese to Traditional using OpenCC.
• —English Spell Checking: Identifies and suggests corrections for misspelled words.

Intended Use

This model is intended for Hong Kong address formatting tasks, particularly for cleaning and structuring address data for logistics, e-commerce, and administrative applications.

Quick Start

Load the adapter with the base model using the peft library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Load base model (with optional 4-bit quantization for memory efficiency)
base_model_name = "Qwen/Qwen2.5-3B-Instruct"
adapter_name = "ymlee13/Qwen2.5-3B-Instruct_Address_Formatter"

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=quantization_config,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, adapter_name)

# Inference
def format_address(address_text):
    messages = [
        {"role": "system", "content": "You are an address formatting assistant. You always return the formatted address with 'Line 1:' and 'Line 2:'."},
        {"role": "user", "content": address_text},
    ]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=100, pad_token_id=tokenizer.eos_token_id)
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    # Parse lines
    import re
    l1_match = re.search(r"Line\s*1:\s*(.+)", response, re.IGNORECASE)
    l2_match = re.search(r"Line\s*2:\s*(.+)", response, re.IGNORECASE)
    line1 = l1_match.group(1).strip() if l1_match else ""
    line2 = l2_match.group(1).strip() if l2_match else ""
    return line1, line2

# Example usage
print(format_address("九龍觀塘區雲漢街61號南寧大樓地庫01舖"))
# Output: ('南寧大樓地庫01舖', '九龍觀塘區雲漢街61號')

For a complete inference pipeline with batch processing and advanced post-processing, please see the full project on GitHub: ymlee13/address_parsing

Training Data

The model was fine-tuned on a dataset of 570+ Hong Kong addresses generated from real geospatial data.

Data Sources

• —ALS-GeoJSON: Hong Kong government dataset containing detailed address information.
• —Synthetic Generation: Custom script (genrealaddress.ipynb) was used to create structured examples from the raw geospatial features.

Data Format

Each training example consists of:

• —Input: A messy, unstructured address string.
• —Output: A structured JSON object with line1 and line2.
• — Example:

  "input": "九龍觀塘區雲漢街61號南寧大樓地庫01舖",
  "output": {"line1": "南寧大樓地庫01舖", "line2": "九龍觀塘區雲漢街61號"}

Performance & Evaluation

Evaluation Metrics

The model was evaluated on a held-out test set using character-level similarity (SequenceMatcher ratio).

Test Examples

Chinese Address

Input:

九龍觀塘區雲漢街61號南寧大樓地庫01舖

Output:

Line 1: 南寧大樓地庫01舖
Line 2: 九龍觀塘區雲漢街61號

English Address

Input:

ROOM 2107, 42/F, WINNING HEIGHTS, 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES

Output:

Line 1: FLAT 2107, 42/F, WINNING HEIGHTS
Line 2: 277 CASTLE PEAK ROAD, TSUEN WAN, NEW TERRITORIES

Limitations & Known Issues

• —Geographic Scope: Primarily optimized for Hong Kong addresses. Performance may degrade on addresses from other regions.
• —Format Variations: The model may struggle with highly unconventional or incomplete addresses.
• —Language Mixing: While it handles both Chinese and English, addresses with heavy code-switching may produce less consistent results.
• —Hardware: 4-bit quantization is recommended for GPUs with <16GB VRAM. For older GPUs (e.g., Tesla P40/Pascal), the inference script automatically disables 4-bit and uses fp16 to avoid compatibility issues.

Usage Recommendations

Hardware Requirements

Environment

# Install dependencies
pip install torch transformers accelerate peft bitsandbytes

Citation

If you use this model in your research or project, please consider citing:

@misc{ymlee13_2024_address_parser,
  author = {Lee, Y.M.},
  title = {Qwen2.5-3B-Instruct Address Formatter},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ymlee13/Qwen2.5-3B-Instruct_Address_Formatter}}
}

Acknowledgements

• —Qwen Team for the excellent base model.
• —Hugging Face for the transformers and peft libraries.
• —Hong Kong Government for providing the ALS-GeoJSON dataset.
• —Open Source Community for tools like trl, bitsandbytes, and accelerate.

Contact & Support

For questions, issues, or collaborations:

• —Author: ymlee13
• —GitHub: ymlee13/address_parsing
• —Hugging Face: ymlee13