Modelpublic

eole-nlp/wmt22-cometkiwi-da-eole-fp16

sourceHugging Facecc-by-nc-sa-4.0updated 3mo agoView on Hugging Face

0likes5downloads

Model Card

WMT22 COMET-KIWI DA FP16 (EOLE)

This is Unbabel/wmt22-cometkiwi-da converted to EOLE format.

Model weights were dtype-cast to fp16 during EOLE conversion. This repository contains a format and dtype conversion for use with EOLE's native transformer_encoder_scorer implementation.

This is not an upstream unbabel-comet checkpoint layout. Use it with EOLE.

Model Details


Original model	Unbabel/wmt22-cometkiwi-da
EOLE architecture	`transformer_encoder_scorer`
Weight dtype	`fp16`
Scoring type	`comet`
Class identifier	`unified_metric`
EOLE scorer	`EOLE-COMET-KIWI`
Requires reference	No
Encoder	XLM-R large style encoder

Usage

Requires an EOLE version with native COMET scorer support.

Validation Metric

yaml

valid_metrics: ["EOLE-COMET-KIWI"]
comet_model: eole-nlp/wmt22-cometkiwi-da-eole-fp16
comet_batch_size: 64

Direct Scoring

COMET-KIWI is reference-free. Use --src for source sentences and --tgt for machine translations.

bash

eole predict \
  --model_path eole-nlp/wmt22-cometkiwi-da-eole-fp16 \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --output /path/to/scores.txt \
  --with_score

By default, direct scoring writes one segment score per input line. To emit a single aggregate system score, add --score_level system:

bash

eole predict \
  --model_path eole-nlp/wmt22-cometkiwi-da-eole-fp16 \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --output /path/to/system-score.txt \
  --with_score \
  --score_level system

score_level: system writes one numeric line containing the arithmetic mean of the segment scores, similar to Unbabel COMET's --only_system mode but using EOLE's score-file format.

Conversion

Converted with:

bash

eole convert COMET \
  --model Unbabel/wmt22-cometkiwi-da \
  --dtype fp16 \
  --output wmt22-cometkiwi-da-eole-fp16

Parity

The fp32 EOLE conversion was parity-checked against the upstream Unbabel COMET runtime. On a 100-line WMT17 slice, native EOLE scoring matched the upstream baseline within floating point tolerance:

Metric	Value
System score delta	`1.01e-08`
Sentence MAE	`1.28e-07`
Sentence max abs	`9.54e-07`

The fp16 artifacts are intended to provide smaller EOLE model files and fp16 runtime behavior. Run the EOLE COMET parity harness against these fp16 repositories if exact fp16 deltas are required.

Original Model and Attribution

This model is a conversion of Unbabel/wmt22-cometkiwi-da. The model weights were converted to EOLE format and dtype-cast to fp16.

This converted repository follows the upstream model license: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC-BY-NC-SA-4.0). Use is subject to the upstream non-commercial, attribution, and share-alike terms. Please refer to the original model card and license for training data, intended use, limitations, and citation details.

Unbabel COMET: https://github.com/Unbabel/COMET