Modelpublic

eole-nlp/wmt23-cometkiwi-da-xl-eole

sourceHugging Facecc-by-nc-sa-4.0updated 3mo agoView on Hugging Face

0likes4downloads

Model Card

WMT23 COMET-KIWI DA XL (EOLE)

This is Unbabel/wmt23-cometkiwi-da-xl converted to EOLE format.

No model weights were modified. This repository contains a format conversion for use with EOLE's native transformer_encoder_scorer implementation.

This is not an upstream unbabel-comet checkpoint layout. Use it with EOLE.

Model Details


Original model	Unbabel/wmt23-cometkiwi-da-xl
EOLE architecture	`transformer_encoder_scorer`
Scoring type	`comet`
Class identifier	`unified_metric`
EOLE scorer	`EOLE-COMET-KIWI`
Requires reference	No
Encoder	XLM-R XL style encoder

Usage

Requires an EOLE version with native COMET scorer support.

Validation Metric

yaml

valid_metrics: ["EOLE-COMET-KIWI"]
comet_model: eole-nlp/wmt23-cometkiwi-da-xl-eole
comet_batch_size: 32

Direct Scoring

COMET-KIWI is reference-free. Use --src for source sentences and --tgt for machine translations.

bash

eole predict \
  --model_path eole-nlp/wmt23-cometkiwi-da-xl-eole \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --output /path/to/scores.txt \
  --with_score

By default, direct scoring writes one segment score per input line. To emit a single aggregate system score, add --score_level system:

bash

eole predict \
  --model_path eole-nlp/wmt23-cometkiwi-da-xl-eole \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --output /path/to/system-score.txt \
  --with_score \
  --score_level system

score_level: system writes one numeric line containing the arithmetic mean of the segment scores, similar to Unbabel COMET's --only_system mode but using EOLE's score-file format.

Conversion

Converted with:

bash

eole convert COMET \
  --model Unbabel/wmt23-cometkiwi-da-xl \
  --output wmt23-cometkiwi-da-xl-eole

Parity

This conversion was parity-checked against the upstream Unbabel COMET runtime. On a 100-line WMT17 slice, native EOLE scoring matched the upstream baseline within floating point tolerance:

Metric	Value
System score delta	`7.69e-08`
Sentence MAE	`2.67e-07`
Sentence max abs	`1.01e-06`

Original Model and Attribution

This model is a conversion of Unbabel/wmt23-cometkiwi-da-xl. The model weights were converted without modification.

This converted repository follows the upstream model license: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC-BY-NC-SA-4.0). Use is subject to the upstream non-commercial, attribution, and share-alike terms. Please refer to the original model card and license for training data, intended use, limitations, and citation details.

Unbabel COMET: https://github.com/Unbabel/COMET