MongoDB/mongodb-docs

Overview

This dataset consists of a small subset of MongoDB's technical documentation.

Dataset Structure

The dataset consists of the following fields:

• —sourceName: The source of the document.
• —url: Link to the article.
• —action: Action taken on the article.
• —body: Content of the article in Markdown format.
• —format: Format of the content.
• —metadata: Metadata such as tags, content type etc. associated with the document.
• —title: Title of the document.
• —updated: The last updated date of the document.

Usage

This dataset can be useful for prototyping RAG applications. This is a real sample of data we have used to build the MongoDB Documentation Chatbot.

Ingest Data

To experiment with this dataset using MongoDB Atlas, first create a MongoDB Atlas account.

You can then use the following script to load this dataset into your MongoDB Atlas cluster:

import os
from pymongo import MongoClient
import datasets
from datasets import load_dataset
from bson import json_util


uri = os.environ.get('MONGODB_ATLAS_URI')
client = MongoClient(uri)
db_name = 'your_database_name'  # Change this to your actual database name
collection_name = 'mongodb_docs'

collection = client[db_name][collection_name]

dataset = load_dataset("MongoDB/mongodb-docs")

insert_data = []

for item in dataset['train']:
    doc = json_util.loads(json_util.dumps(item))
    insert_data.append(doc)

    if len(insert_data) == 1000:
        collection.insert_many(insert_data)
        print("1000 records ingested")
        insert_data = []

if len(insert_data) > 0:
    collection.insert_many(insert_data)
    insert_data = []

print("Data ingested successfully!")