leo20000306/KANT_Domain_Corpus

KANT RAG Corpus Release

This Hugging Face dataset contains the retrieval materials used by the KANT RAG experiments. The main payload is split into 512 MiB parts so it can be uploaded and downloaded reliably.

The release includes:

• —rag_release_materials.tar.zst.part-*: split parts of the self-contained RAG materials archive.
• —SHA256SUMS.parts: checksums for the split archive parts.
• —SHA256SUMS.original_archives: checksums for the reconstructed archives.

Reconstruct

Reconstruct and verify the main archive:

cat rag_release_materials.tar.zst.part-* > rag_release_materials.tar.zst
sha256sum -c SHA256SUMS.original_archives

Extract the RAG materials:

tar --zstd -xf rag_release_materials.tar.zst

After extraction, the directory is:

rag_release_materials/
  corpus_records/
    <corpus_name>/
      records.jsonl
      manifest.json
  faiss_indices/
    <corpus_name>/
      fused_index_flatip.faiss
      manifest.json
  images/
    <corpus_name>/
      ...
  retrieval_cache/
    <kant_task>/
      retrieval_by_query.jsonl
      sample_to_query.jsonl
      summary.json

Contents

corpus_records/ contains the retrieval corpus rows. Each records.jsonl line is one image-text record, and its line number is the FAISS row id used by the index and the retrieval cache.

images/ contains the package-relative image files referenced by corpus_records/*/records.jsonl.

faiss_indices/ contains exact FAISS IndexFlatIP indices over fused Qwen3-VL text/image embeddings. Each index row aligns with the corresponding corpus_records/<corpus_name>/records.jsonl row.

retrieval_cache/ contains the precomputed top-5 retrieval results for KANT training queries, so users can consume the RAG results without rebuilding embeddings or running FAISS search.

Task To Corpus Mapping

Corpus Summary

File Formats

corpus_records/<corpus_name>/records.jsonl

Each line is one corpus record. Example, abridged:

{
  "id": "openpmc_00000007",
  "image_path": "images/openpmc_50k_qwen3vl_official/openpmc_00000007.jpg",
  "text": "Sub-caption: Mammography revealing an isodense and irregular spiculated mass ...",
  "metadata": {
    "dataset": "Open-PMC",
    "source_repo": "vector-institute/open-pmc",
    "source_shard": "train-0133.tar",
    "source_stem": "train-0133-000007",
    "pmc_id": "PMC4214494",
    "source_image": "train-0133-000007.jpg",
    "modality": "R",
    "ahash": "f8fcfcfcf8f8f0e0"
  }
}

Important fields:

• —id: stable corpus record id.
• —image_path: image path relative to the extracted rag_release_materials/ directory.
• —text: text used as the retrieval document content.
• —metadata: source-specific provenance.

faiss_indices/<corpus_name>/manifest.json

Each manifest describes the corresponding FAISS index. The indices are exact inner-product indices over fused normalized text/image representations.

Key fields include:

{
  "backend": "faiss.IndexFlatIP",
  "exact_search": true,
  "metric": "inner_product",
  "cosine_equivalent": true,
  "fusion": "sqrt_weighted_concat",
  "score_formula": "text_weight * dot(query, text_vector) + image_weight * dot(query, image_vector)",
  "text_weight": 0.5,
  "image_weight": 0.5,
  "num_records": 50000,
  "source_dim": 2048,
  "fused_dim": 4096
}

The FAISS row id is the zero-based line number in corpus_records/<corpus_name>/records.jsonl.

retrieval_cache/<kant_task>/retrieval_by_query.jsonl

Each line stores the final top-5 retrieval results for one deduplicated query. Example, abridged:

{
  "task": "llava_med",
  "query_key": "6964e68ede98680ff80626f6cde182086128e5cd",
  "query": "What is the purpose of the flow diagram?",
  "query_mode": "human",
  "search_mode": "fused",
  "top_k": 5,
  "backend": "faiss",
  "embedding_loader": "official_qwen3_vl_embedding.Qwen3VLEmbedder",
  "hits": [
    {
      "rank": 1,
      "index": 41156,
      "score": 0.3184305429458618,
      "text_score": 0.355814129114151,
      "image_score": 0.28104692697525024,
      "record_id": "openpmc_00029249",
      "text": "Sub-caption: Clinical isolates with positive partial modified acid-fast staining ...",
      "metadata": {
        "dataset": "Open-PMC",
        "pmc_id": "PMC7664795"
      }
    }
  ]
}

Important fields:

• —query_key: SHA-style key for a deduplicated query.
• —query: query text embedded for retrieval.
• —hits: ranked retrieved records.
• —hits[].index: row id in the task's mapped corpus_records/<corpus_name>/records.jsonl.
• —hits[].score: fused retrieval score.
• —hits[].text_score / hits[].image_score: modality-specific scores before fusion.
• —hits[].record_id: id of the retrieved corpus record.
• —hits[].text and hits[].metadata: copied retrieval content for convenient direct use.

retrieval_cache/<kant_task>/sample_to_query.jsonl

Each line maps one original KANT training sample to a deduplicated query in retrieval_by_query.jsonl.

{
  "sample_index": 0,
  "id": "27374136_fig01",
  "image": "27374136_fig01.jpg",
  "query_key": "6964e68ede98680ff80626f6cde182086128e5cd",
  "query": "What is the purpose of the flow diagram?"
}

Use sample_to_query.jsonl when multiple training samples share the same retrieval query. Join by query_key to fetch the cached top-5 results from retrieval_by_query.jsonl.

Typical Usage

To use cached RAG results for a task:

1. 1.Open retrieval_cache/<kant_task>/sample_to_query.jsonl.
2. 2.Find the row for the desired sample_index.
3. 3.Use its query_key to find the row in retrieval_cache/<kant_task>/retrieval_by_query.jsonl.
4. 4.Read hits directly, or map hits[].index back to corpus_records/<corpus_name>/records.jsonl if package-relative image paths are needed.

To rebuild retrieval from FAISS:

1. 1.Load faiss_indices/<corpus_name>/fused_index_flatip.faiss.
2. 2.Embed the query with the same embedding model and prompt recorded in retrieval_cache/<kant_task>/summary.json.
3. 3.Search the FAISS index.
4. 4.Interpret returned row ids using corpus_records/<corpus_name>/records.jsonl.