Modelpublic

ao9000/gpt2-cda-gender

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes15downloads

Model Card

Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)

—Dataset: Wikipedia-10
—Target modules = ["query", "value"]

{
    "epoch": 0.06122556358609606,
    "total_flos": 3.3909236563968e+16,
    "train_loss": 0.9592373611629009,
    "train_runtime": 5840.6966,
    "train_samples": 1045315,
    "train_samples_per_second": 10.958,
    "train_steps_per_second": 0.342
}

Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)

Training script: https://github.com/ao9000/bias-bench/blob/main/experiments/run_clm.py