CoolFace
Datasetpublic

sentence-transformers/stackexchange-duplicates

Dataset Card for Stack Exchange Duplicates This dataset contains the Stack Exchange Duplicates dataset in three formats that are easily used with Sentence Transformers to train embedding models. The data was originally extracted using the Stack Exchange API and taken from embedding-training-data. Each pair contains data from two Stack Exchange posts that were marked as duplicates. title-title-pair only has the titles, body-body-pair only the bodies, and post-post-pair has both.… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/stackexchange-duplicates.

sourceHugging Faceupdated 2y agoView on Hugging Face
2likes321downloads
Dataset Card

Dataset Card for Stack Exchange Duplicates

This dataset contains the Stack Exchange Duplicates dataset in three formats that are easily used with Sentence Transformers to train embedding models. The data was originally extracted using the Stack Exchange API and taken from embedding-training-data. Each pair contains data from two Stack Exchange posts that were marked as duplicates. title-title-pair only has the titles, body-body-pair only the bodies, and post-post-pair has both.

Dataset Subsets

title-title-pair subset

  • —Columns: "title1", "title2"
  • —Column types: str, str
  • —Examples:
python
    {
      'title1': 'what is the advantage of using the GPU rendering options in Android?',
      'title2': 'Can anyone explain all these Developer Options?',
    }
  • —Collection strategy: Reading the Stack Exchange Duplicates (title, title) dataset from embedding-training-data.
  • —Deduplified: No

body-body-pair subset

  • —Columns: "body1", "body2"
  • —Column types: str, str
  • —Examples:
python
    {
      'body1': "I've been wondering about this for years.  It seems like a pretty obvious question, so I'm surprised not to have found it addressed among the other Tolkien minutiae on this site.  Hopefully I haven't missed it, but anyway, here goes...  In Tolkien's Middle-Earth writings, Evil cannot create things, only twist and warp what already exists.  Thus, Orcs are twisted Elves, Trolls are twisted Ents, etc.  So then, what's the original source for Dragons?  They look pretty original to me!  The only template that seems even remotely possible is the Eagles, as they're both powerful fliers, but the connection seems very remote indeed.  Also, as twisted copies Orcs and Trolls are markedly inferior to Elves and Ents respectively, but I'm not aware of any text describing Dragons as inferior to Eagles.",
      'body2': 'All that I know of Smaug is that he (she?) came out of nowhere to attack and conquer Erebor. Where exactly did he come from? In fact, what are the origins of dragons? Did Ilúvatar create them or did they come from somewhere else?',
    }
  • —Collection strategy: Reading the Stack Exchange Duplicates (body, body) dataset from embedding-training-data.
  • —Deduplified: No

post-post-pair subset

  • —Columns: "post1", "post2"
  • —Column types: str, str
  • —Examples:
python
    {
      'post1': 'Hyperdrive vs Warp drive. Which is fastest? In Star Trek Warp Factor 36 is the highest FTL speed while in star wars I think the fastest canon was 0.4. So which is faster?',
      'post2': "Is a warp drive faster than a hyperdrive? What's faster: a warp drive from Star Trek, or a hyperdrive from Star Wars?",
    }
  • —Collection strategy: Reading the Stack Exchange Duplicates (title+body, title+body) dataset from embedding-training-data.
  • —Deduplified: No