CoolFace
Datasetpublic

johannhartmann/pgturbohybrid_dbpedia_colbert

johannhartmann/pgturbohybrid_dbpedia_colbert Precomputed DBpedia ColBERT multivectors for pgturbohybrid benchmark runs. The dataset stores packed little-endian float16 values for the document and query embeddings. Importing these rows into PostgreSQL avoids llama.cpp embedding generation during retrieval/index benchmarks. This compact export stores half-precision values and reconstructs turbohybrid_multivector values on import. Use it for benchmark loading where avoiding runtime… See the full description on the dataset page: https://huggingface.co/datasets/johannhartmann/pgturbohybrid_dbpedia_colbert.

sourceHugging Faceotherupdated 4mo agoView on Hugging Face
0likes28downloads
Dataset Card

johannhartmann/pgturbohybriddbpediacolbert

Precomputed DBpedia ColBERT multivectors for pgturbohybrid benchmark runs.

The dataset stores packed little-endian float16 values for the document and query embeddings. Importing these rows into PostgreSQL avoids llama.cpp embedding generation during retrieval/index benchmarks.

This compact export stores half-precision values and reconstructs turbohybrid_multivector values on import. Use it for benchmark loading where avoiding runtime model generation is more important than bit-exact float32 round-trips.

Contents

  • —Documents: 1,000,000
  • —Queries: 381
  • —Qrels: 6,411
  • —Embedding dimension: 128
  • —Embedding format: float16_le

Files

  • —docs/*.parquet: doc_id, title, body, colbert_values_f16, colbert_dim, colbert_count
  • —queries/*.parquet: query_id, query_text, colbert_values_f16, colbert_dim, colbert_count
  • —qrels/*.parquet: query_id, doc_id, relevance
  • —metadata.json: source model, row counts, schema version, and export command

Model

  • —GGUF: johannhartmann/SauerkrautLM-Multi-ColBERT-15m-GGUF
  • —Source model: VAGOsolutions/SauerkrautLM-Multi-ColBERT-15m

Import

sh
hf download johannhartmann/pgturbohybrid_dbpedia_colbert --type dataset \
  --local-dir .nix-dev/hf-datasets/dbpedia-colbert-multivector-1m

nix --extra-experimental-features 'nix-command flakes' develop .#bench --command \
  th-dbpedia-colbert-hf-dataset import \
  --source .nix-dev/hf-datasets/dbpedia-colbert-multivector-1m