Apppublic

themechanism/script_fidelity_rate

sourceHugging Facemitupdated 5mo agoView on Hugging Face

0likes

App README

Script Fidelity Rate

This directory is the Hugging Face Evaluate metric module for Script Fidelity Rate (SFR).

The Python package is published as script-fidelity on PyPI: <https://pypi.org/project/script-fidelity/>. The import name is script_fidelity.

python

import evaluate

sfr = evaluate.load("./metrics/script_fidelity_rate", module_type="metric")
result = sfr.compute(
    predictions=["کابل کې ښه هوا ده", "this is romanized output"],
    language="ps_af",
)
print(result["sfr_percent"])

Hub use:

python

import evaluate

sfr = evaluate.load("themechanism/script_fidelity_rate", module_type="metric")
sfr.compute(predictions=["کابل کې ښه هوا ده"], language="pashto")

Use SFR with WER and CER, not instead of them. SFR checks whether output is in the intended script. It does not measure lexical accuracy.