mcnckc/llm-course-hw3-lora

Модель OuteAI/Lite-Oute-1-300M-Instruct дообученная на датасете cardiffnlp/tweet_eval, задача классификации сентимента твита, вывести одно из трех слов - negative, neutral, positive.

Дообучение

Модель дообучалась при помощи LoRA.

• —Ранг LoRA = 8
• —alpha=16
• —LoRA применялась только к весам Key, Value в attention
• —BATCH_SIZE = 16
• —LEARNING_RATE = 2e-4
• —NUM_EPOCHS = 2
• —AdamW

Метрика на валидации

F1=0.53

Примеры генерации

Tweet: "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin" \ Label: positive \ Output: \ positive \ positive \ positive

Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ" \ Label: neutral \ Output: \ neutral \ neutral \ neutral \ neut

Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night. \ Label: neutral \ Output: \ neutral \ positive \ positive \ pos

Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays \ Label: neutral \ Output: \ neutral \ neutral \ neutral \ neut

Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017" \ Label: positive \ Output: \ neutral \ neutral \ neutral \ neut