vab46/Clinical_trials_anchor-contextORpositive-ground-truth_LLM_LORA-junk_handled_ft

Dataset details:-

• —This dataset is the 2nd iteration following bugs in **1st dataset**.
• —The initial data suffered with followoing cases:-

(i) The failed referenceanswers generation(due error totalling 23) primarly because of 2 reasons/exceptions:-\   (a) There was normal limit(300 in 1st request) and worst case limit(450 in 3rd request) number of tokens for consolidated 4 refernceanswers per chunk and its 4 corresponding answers. However certain answers breached this higher limit(450) and this caused json truncation error(where answer generation stopped in between).\   (b)Mismatch between input anchor set and output refernceanswer set.Egs:- 3 output refAnswers to 4 anchors.

(ii) Single word answer:-   (a) In age/temporal related question. For eg:- What is upper age restriction for patient going this epilipsy drug trial?--->55(total 5 instances).\   (b) In eligiblity/qualification question. For eg:- Is the person with asthama comorbidity eligible for this clinical trial study?--->False(total 10 instances).

(iii) If no answer feasible(given context and query), blank space left(2 instances).

• —The issue(i) was resolved using individual context and query passage via teacher model(Qwen-2.5-7B instruct), minor tweaks in prompt and increasing the max_tokens to 500-700. On the other hand issues(ii) and (iii) were handled both manually and via teacher model.

• —This changes made way to present dataset. This has been used in 2nd iteration of **RAG CT candidate model/generator fine tuning_epoch1** **RAG CT candidate model/generator fine tuning_final** given retrieved context(ranked) and query. Futher individual use case of answer generation with just context is also feasible.