elliepreed/french_english_sequential

library_name: transformers tags:

• —gpt2
• —causal-lm
• —bilingual
• —sentencepiece
• —french
• —english pipeline_tag: text-generation datasets:
• —climb-mao/babylm-fra
• —elliepreed/l2-corpus-10m license: other # change to "apache-2.0" or "mit" if that's correct model-index:
• —name: FrenchEnglishsequential – 128k steps results: [] ---
• —French + English (GPT-2 style) sequential model Small bilingual GPT-2–style language model trained on French and English with SentencePiece tokenizers.

This model is trained on both French 🇫🇷 and English 🇬🇧, but it does not come with a single AutoTokenizer. Instead, we provide two SentencePiece tokenizers:

tokenizers/french.model tokenizers/english.model You can load either depending on the language you want to work with.

• —Load the model from transformers import AutoModelForCausalLM import torch modelid = "elliepreed/bgpt-french-english" device = "cuda" if torch.cuda.isavailable() else "cpu" model = AutoModelForCausalLM.frompretrained(modelid).to(device).eval()

• —Load both tokenizers import sentencepiece as spm from huggingfacehub import hfhubdownload frpath = hfhubdownload(modelid, "tokenizers/french.model") enpath = hfhubdownload(modelid, "tokenizers/english.model") spfr = spm.SentencePieceProcessor(modelfile=frpath) spen = spm.SentencePieceProcessor(modelfile=en_path)

Example: French generation prompt = "Paris est" ids = spfr.encode(prompt, outtype=int) + [spfr.eosid()] inputids = torch.tensor([ids], device=device) out = model.generate( inputids, maxnewtokens=40, dosample=True, topp=0.95, temperature=0.9, eostokenid=spfr.eosid(), padtokenid=spfr.padid(), ) print("FR:", sp_fr.decode(out[0].tolist()[len(ids):]))