promehedi/quran_embeddings
Quran Embeddings Dataset This repository contains vector embeddings for the Holy Quran, generated using OpenAI's embedding model. These embeddings can be used for semantic search, question answering, and other natural language processing tasks related to Quranic text. Dataset Information The dataset consists of a single JSON file: quran_embeddings.json: Contains embeddings for each verse (ayah) of the Quran with associated metadata Metadata Structure… See the full description on the dataset page: https://huggingface.co/datasets/promehedi/quran_embeddings.
Quran Embeddings Dataset
This repository contains vector embeddings for the Holy Quran, generated using OpenAI's embedding model. These embeddings can be used for semantic search, question answering, and other natural language processing tasks related to Quranic text.
Dataset Information
The dataset consists of a single JSON file:
quran_embeddings.json: Contains embeddings for each verse (ayah) of the Quran with associated metadata
Metadata Structure
Each verse embedding includes the following metadata:
- Surah (chapter) name
- Surah number
- Ayah (verse) number
- Juz number
- Revelation location (Meccan/Medinan)
- Original Arabic text
- English translation
Usage Example
The embeddings can be loaded into a vector database like Chroma for similarity search:
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings
from langchain.schema import Document
import json
# Load the embeddings
with open("quran_embeddings.json", "r", encoding="utf-8") as f:
quran_data = json.load(f)
documents = []
for verse in quran_data:
content = verse.get("english_translation", "")
metadata = {
"type": "quran",
"surah": verse.get("surah_name"),
"surah_number": verse.get("surah_number"),
"ayah_number": verse.get("ayah_number"),
"juz_number": verse.get("juz_number"),
"revelation": verse.get("revelation_type"),
"arabic": verse.get("arabic_text"),
}
documents.append(Document(page_content=content, metadata=metadata))
# Initialize embedding function
embedding_function = OpenAIEmbeddings()
# Create vector database
db = Chroma(
collection_name="quran_collection",
embedding_function=embedding_function,
persist_directory="./db"
)
# Add documents to the database (if not already added)
db.add_documents(documents)
# Example query
results = db.similarity_search("What does the Quran say about patience?", k=3)Credits
This dataset builds upon the work from QuranGPT by Hazem Abdelkawy. The original project aims to make Islamic texts more accessible through natural language processing technologies.
License
These embeddings are provided for research, educational, and non-commercial purposes. Please respect the sanctity of the Quranic text when using this dataset.