CoolFace
Modelpublic

RichardErkhov/postbot_-_emailgen-pythia-410m-deduped-gguf

sourceHugging Faceupdated 2y agoView on Hugging Face
0likes298downloads
Model Card

Quantization made by Richard Erkhov.

Github

Discord

Request more models

emailgen-pythia-410m-deduped - GGUF

  • —Model creator: https://huggingface.co/postbot/
  • —Original model: https://huggingface.co/postbot/emailgen-pythia-410m-deduped/

Original model description: --- language:

  • —en license: apache-2.0 tags:
  • —generatedfromtrainer datasets:
  • —postbot/multi-emails-hq metrics:
  • —accuracy widget:
  • —text: 'Good Morning Professor Beans,

Hope you are doing well. I just wanted to reach out and ask if differential calculus will be on the exam' example_title: email to prof

  • —text: 'Hey <NAME>,

Thank you for signing up for my weekly newsletter. Before we get started, you''ll have to confirm your email address.' example_title: newsletter

  • —text: 'Hi <NAME>,

I hope this email finds you well. I wanted to reach out and ask about office hours' example_title: office hours

  • —text: 'Greetings <NAME>,

I hope you had a splendid evening at the Company sausage eating festival. I am reaching out because' example_title: festival

  • —text: 'Good Morning Harold,

I was wondering when the next' example_title: event

  • —text: URGENT - I need the TPS reports example_title: URGENT
  • —text: 'Hi Archibald,

I hope this email finds you extremely well.' example_title: emails that find you

  • —text: 'Hello there.

I just wanted to reach out and check in to' example_title: checking in

  • —text: 'Hello <NAME>,

I hope this email finds you well. I wanted to reach out and see if you''ve enjoyed your time with us' example_title: work well

  • —text: 'Hi <NAME>,

I hope this email finds you well. I wanted to reach out and see if we could catch up' example_title: catch up

  • —text: I'm <NAME> and I just moved into the area and wanted to reach out and get some details on where I could get groceries and exampletitle: grocery pipelinetag: text-generation base_model: EleutherAI/pythia-410m-deduped model-index:
  • —name: multi-emails-hq-pythia-410m-deduped-r1 results: [] ---

emailgen-pythia-410m-deduped

![colab](https://colab.research.google.com/gist/pszemraj/94b0e6b95437896f800a65ae2e5f9ab4/emailgen-pythia-410m-deduped.ipynb )

This model is a fine-tuned version of EleutherAI/pythia-410m-deduped on email data. It achieves the following results on the evaluation set:

  • —Loss: 2.1018
  • —Accuracy: 0.6157
  • —perplexity: 8.181

Model description

  • —fine-tuned on dataset of emails for 4 epochs
  • —intended use: "text completion" of partially written emails

Usage example

python
from transformers import pipeline

model_tag = "postbot/emailgen-pythia-410m-deduped"
generator = pipeline(
    "text-generation",
    model=model_tag,
)

prompt = """
Hello, 

Following up on the bubblegum shipment."""

result = generator(
    prompt,
)  # generate
print(result[0]["generated_text"])

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

MetricValue
Avg.26.65
ARC (25-shot)27.9
HellaSwag (10-shot)40.04
MMLU (5-shot)27.35
TruthfulQA (0-shot)38.2
Winogrande (5-shot)52.09
GSM8K (5-shot)0.0
DROP (3-shot)0.99