RKB109/rag-evaluation-lab-20260720-model

RAG Evaluation Lab Baseline Model

Model Description

This repository contains a small, transparent prototype model for RAG systems often ship without a stable regression set or failure taxonomy.

The model combines per-label token weights with IDF-weighted evidence retrieval. It was generated for reproducible architecture demonstrations and does not call a hosted LLM.

Evaluation

• —Held-out synthetic examples: 4
• —Accuracy: 0.75
• —Intended metrics: failureclassaccuracy, citationcoverage, releasegatepassrate

Intended Use

• —Architecture prototyping
• —CI and evaluation examples
• —Local baseline comparisons
• —Educational experimentation

Hugging Face Task Coverage

• —text-classification
• —question-answering
• —text-ranking
• —summarization

Limitations and Risks

Synthetic cases validate the harness, not a production RAG system. Teams must add representative domain examples.

The dataset is synthetic and small. Do not use this model for consequential decisions without representative data, expert review, and production-grade evaluation.

Reproducibility

The linked GitHub repository includes train.py, the exact dataset split, evaluation code, and the model JSON format.