otelk/mt-error-detect-correct

1---2language:3- en4- de5pipeline_tag: text-generation6library_name: transformers7tags:8- machine-translation9- error-detection10- error-correction11- lora12- qwen313---14# MT Error Detection and Correction Models15 16This repository contains models for machine translation error detection and error correction.17 18## Repository Structure19 20```text21error_detection/22├── model/   # Model for translation error detection23└── lora/    # GRPO LoRA adapter for error detection24 25error_correction/26├── model/   # Model for translation error correction27└── lora/    # GRPO LoRA adapter for error correction28```29## Usage30 31The repository contains two components:32 33- `error_detection`: machine translation error detection34- `error_correction`: machine translation error correction35 36### Installation37 38```bash39pip install transformers huggingface_hub vllm40```41 42### Load a Model43 44First download the repository:45 46```python47import os48from huggingface_hub import snapshot_download49from transformers import AutoTokenizer50from vllm import LLM51from vllm.lora.request import LoRARequest52 53repo_dir = snapshot_download(54    repo_id="otelk/mt-error-detect-correct"55)56```57 58For error detection:59 60```python61model_dir = os.path.join(repo_dir, "error_detection", "model")62adapter_dir = os.path.join(repo_dir, "error_detection", "lora")63```64 65For error correction:66 67```python68model_dir = os.path.join(repo_dir, "error_correction", "model")69adapter_dir = os.path.join(repo_dir, "error_correction", "lora")70```71 72```python73tokenizer = AutoTokenizer.from_pretrained(74    model_dir,75    trust_remote_code=True76)77 78llm = LLM(79    model=model_dir,80    tokenizer=model_dir,81    dtype="bfloat16",82    enable_lora=True,83    max_lora_rank=16,84    trust_remote_code=True85)86 87lora_request = LoRARequest(88    "mt_lora",89    1,90    adapter_dir91)92```93 94The model can then be used with vLLM by passing the corresponding `lora_request` during generation.95 96```python97outputs = llm.generate(98    prompts,99    sampling_params=sampling_params,100    lora_request=lora_request101)102```