monsoon-nlp/ar-seq2seq-gender-decoder

1---2language: ar3---4 5# ar-seq2seq-gender (decoder)6 7This is a seq2seq model (decoder half) to "flip" gender in **first-person** Arabic sentences.8The model can augment your existing Arabic data, or generate counterfactuals9to test a model's decisions (would changing the gender of the subject or speaker change output?).10 11Intended Examples:12- 'أنا سعيد' <=> 'انا سعيدة'13- 'ركض إلى المتجر' <=> 'ركضت إلى المتجر'14 15People's names, gender pronouns, gendered words (father, mother), and many other values are currently unchanged by this model. Future versions may be trained on more data.16 17## Sample Code18 19```20import torch21from transformers import AutoTokenizer, EncoderDecoderModel22 23model = EncoderDecoderModel.from_encoder_decoder_pretrained(24  "monsoon-nlp/ar-seq2seq-gender-encoder",25  "monsoon-nlp/ar-seq2seq-gender-decoder",26  min_length=4027)28tokenizer = AutoTokenizer.from_pretrained('monsoon-nlp/ar-seq2seq-gender-decoder') # same as MARBERT original29 30input_ids = torch.tensor(tokenizer.encode("أنا سعيدة")).unsqueeze(0)31generated = model.generate(input_ids, decoder_start_token_id=model.config.decoder.pad_token_id)32tokenizer.decode(generated.tolist()[0][1 : len(input_ids[0]) - 1])33> 'انا سعيد'34```35 36https://colab.research.google.com/drive/1S0kE_2WiV82JkqKik_sBW-0TUtzUVmrV?usp=sharing37 38## Training39 40I originally developed41<a href="https://github.com/MonsoonNLP/el-la">a gender flip Python script</a>42for Spanish sentences, using43<a href="https://huggingface.co/dccuchile/bert-base-spanish-wwm-uncased">BETO</a>,44and spaCy. More about this project: https://medium.com/ai-in-plain-english/gender-bias-in-spanish-bert-1f4d7678061745 46The Arabic model encoder and decoder started with weights and vocabulary from47<a href="https://github.com/UBC-NLP/marbert">MARBERT from UBC-NLP</a>,48and was trained on the49<a href="https://camel.abudhabi.nyu.edu/arabic-parallel-gender-corpus/">Arabic Parallel Gender Corpus</a>50from NYU Abu Dhabi. The text is first-person sentences from OpenSubtitles, with parallel51gender-reinflected sentences generated by Arabic speakers.52 53Training notebook: https://colab.research.google.com/drive/1TuDfnV2gQ-WsDtHkF52jbn699bk6vJZV54 55## Non-binary gender56 57This model is useful to generate male and female text samples, but falls58short of capturing gender diversity in the world and in the Arabic59language. This subject is discussed in the bias statement of the60<a href="https://www.aclweb.org/anthology/2020.gebnlp-1.12/">Gender Reinflection paper</a>.61