CoolFace
Datasetpublic

metehan777/aeo-geo-rag

AEO/GEO RAG Knowledge Base Source: metehan777/cc-aeo-geo-fulltext-CC-MAIN-2026-21 (Common Crawl CC-MAIN-2026-21, AEO/GEO filtered) Contents chunks.parquet — 1,187,728 text chunks with metadata (domain, url, q_score, etc.) embeddings.npy — float32 normalized embeddings, shape (1187728, 384) Embedding model BAAI/bge-small-en-v1.5 (384-dim, normalized) Chunking 512 tokens (~2048 chars) with 64 token overlap HTML/boilerplate cleaned… See the full description on the dataset page: https://huggingface.co/datasets/metehan777/aeo-geo-rag.

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes25downloads
Dataset Card

AEO/GEO RAG Knowledge Base

Source: metehan777/cc-aeo-geo-fulltext-CC-MAIN-2026-21 (Common Crawl CC-MAIN-2026-21, AEO/GEO filtered)

Contents

  • —chunks.parquet — 1,187,728 text chunks with metadata (domain, url, q_score, etc.)
  • —embeddings.npy — float32 normalized embeddings, shape (1187728, 384)

Embedding model

BAAI/bge-small-en-v1.5 (384-dim, normalized)

Chunking

  • —512 tokens (~2048 chars) with 64 token overlap
  • —HTML/boilerplate cleaned (nav, cookie, footer, scripts)

Usage

python
import numpy as np
import pandas as pd
from sentence_transformers import SentenceTransformer

chunks = pd.read_parquet("chunks.parquet")
embs = np.load("embeddings.npy")
model = SentenceTransformer("BAAI/bge-small-en-v1.5")

query = "How to optimize for AI search engines?"
q_emb = model.encode([query], normalize_embeddings=True)
scores = (embs @ q_emb.T).ravel()
top_k = scores.argsort()[-10:][::-1]
for i in top_k:
    print(f"score={scores[i]:.3f} | {chunks.iloc[i]['domain']}")
    print(chunks.iloc[i]['text'][:200])