biodatlab/ec-raft

EC-RAFT: Automated Generation of Clinical Trial Eligibility Criteria

Model Description

EC-RAFT is a fine-tuned Retrieval-Augmented Fine-Tuning (RAFT) model based on LLaMA-3.1-8B-Instruct architecture. It is designed to automatically generate structured, high-quality clinical trial eligibility criteria (EC) directly from trial titles and descriptions.

EC-RAFT integrates domain-specific retrieval with synthesized intermediate reasoning steps, enabling it to produce clinically relevant and contextually appropriate EC sets.

Fine-tuning Details

• —Original Model: LLaMA-3.1-8B-Instruct
• —Datasets used for fine-tuning:
• —ClinicalTrials.gov (267,347 trials, 2000–2024) biodatlab/ec-raft-dataset
• —Retrieval corpus constructed using SciNCL model
• —Intermediate reasoning steps R generated using Gemini-1.5-flash-002
• —Fine-tuning method:
• —Retrieval-Augmented Fine-Tuning (RAFT)
• —Low-Rank Adaptation (LoRA)

Model Performance

Evaluated on a held-out ClinicalTrials.gov test split:

• —Outperforms zero-shot LLaMA-3.1 and Gemini-1.5-flash baselines
• —Outperforms fine-tuned LLaMA and Meditron baselines
• —Clinically validated: LLM-as-a-Judge scores highly correlated with human physician evaluation

Intended Use

• —Assist researchers, trial designers, and sponsors in drafting clinical trial eligibility criteria.
• —Automate EC generation to reduce manual effort and improve consistency.
• —Support clinical trial design transparency and quality.
• —Enable integration with trial registry platforms, clinical trial matching systems, and EC recommendation tools.

Limitations

• —Requires human validation of generated EC before clinical use.
• —Trained on public ClinicalTrials.gov data — may not generalize well to:
• —Rare or novel diseases
• —Specialized or non-standard trial designs
• —Non-public trial data
• —Optimized for English-language clinical trials.
• —As with any LLM-based system, risks include hallucination, subtle errors, and domain shifts.
• —Evaluation metrics (BERTScore, LLM-as-a-Judge) are proxies — not full substitutes for domain expert review.

Acknowledgments

This model was developed using resources provided by:

• —RAVIS Technology for feedback and collaboration.
• —Faculty of Medicine Ramathibodi Hospital
• —NSTDA Supercomputer Center (ThaiSC), Project \#pv814001

We also acknowledge the contributions of the broader open-source community whose tools and prior works on RAFT, SciNCL, LoRA, LLaMA-3, and biomedical NLP made this project possible.