KushieBoi/QA_Low_Resource_FatimaFellowship
Model Experimentation and Analysis 1. Model Experimentations I chose the task of Q&A, branching into two categories: general and specific. I tested the output, i.e., the LLM's response against the expected output. I have created the dataset in a CSV file. The model used is : https://huggingface.co/Andron00e/YetAnother_Open-Llama-3B-LoRA-OpenOrca 2. "Blind spots" In this dataset, model is not able to predict well on very-specific information, like… See the full description on the dataset page: https://huggingface.co/datasets/KushieBoi/QA_Low_Resource_FatimaFellowship.
Model Experimentation and Analysis
1. Model Experimentations
I chose the task of Q&A, branching into two categories: general and specific. I tested the output, i.e., the LLM's response against the expected output. I have created the dataset in a CSV file. The model used is : https://huggingface.co/Andron00e/YetAnother_Open-Llama-3B-LoRA-OpenOrca
2. "Blind spots"
In this dataset, model is not able to predict well on very-specific information, like dates or years and scientific information, for instance.
3. Solution
The scope of a "perfect" Q&A is very difficult. By "perfect", I mean the model answers ideally and correct information for every question asked. Instead, we should hope for a model that answers questions only it knows and not answer incorrectly (A similar analogy from verification vs validation from Human-Computer Interaction).
4. Code Notebook
The notebook can be accessed from: https://colab.research.google.com/drive/1ePvADfbJQ9D4gCIvLyIqNNhaq4pqmifX?usp=sharing