datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Clinical_trials_anchor-positive-pairs_EmbeddingModel-data_final
Dataset details:-
This dataset is the final version of anchor(query)-positive(chunk) pair data w.r.t fine tuning embedding model for clinical trials dataset.
It includes best of both 4 anchors-consolidated positive chunk/nctId dataset-->first dataset and
5 anchors-3 positive chunk/nctId--->Second dataset.
The 1st dataset(consolidated title +summary+ inclusion criteria chunk) suffered with pre-processing bottlenecks :-
rendering huge chunks upto 15k characeters.
missing on… See the full description on the dataset page: https://huggingface.co/datasets/vab46/Clinical_trials_anchor-positive-pairs_EmbeddingModel-data_final.Clinical_trials_anchor-positive-pairs_EmbeddingModel-data2
Dataset details:-
This is the iteration 2 of the previous dataset on same repository.
Overall similar usecase:-(i)embedding model fine-tune/training for CT(clinical trials) domain, thereby aiding downstream tsak like ranked retrieval document generation
comparing 2 or more anchors, anchor vs chunks via cosine similarity.
Key changes(from earlier version):-
Greater granulation of chunks so to have:- (i) cleaner directed retreival from fine tuned embedding model;… See the full description on the dataset page: https://huggingface.co/datasets/vab46/Clinical_trials_anchor-positive-pairs_EmbeddingModel-data2.Clinical_trials_anchor-positive-pairs_EmbeddingModel-data
Dataset details:-
This dataset is basically mapping of anchor-positive pair chunks.
It consists of consolidated "title"+"summary"+"inclusion_criteria" for each CT idi.e. clinical trials(unique "nctId") as chunks.
Each of the aformentioned chunks have 4 questions(anchors) branched to it(here 1-to-1 normalized mapping of those).
The data is particularly useful in order to fine tune an embedding model for CT domain. This further helps in (i)ranked retreival genesis of CT RAGs… See the full description on the dataset page: https://huggingface.co/datasets/vab46/Clinical_trials_anchor-positive-pairs_EmbeddingModel-data.