PEFT/quantization
6
1{2 "cells": [3 {4 "cell_type": "markdown",5 "metadata": {6 "id": "WE5GJ6s7y0Xo"7 },8 "source": [9 "## Fine-tune large models using 🤗 `peft` adapters, `transformers` & `bitsandbytes`\n",10 "\n",11 "In this tutorial we will cover how we can fine-tune large language models using the very recent `peft` library and `bitsandbytes` for loading large models in 8-bit.\n",12 "The fine-tuning method will rely on a recent method called \"Low Rank Adapters\" (LoRA), instead of fine-tuning the entire model you just have to fine-tune these adapters and load them properly inside the model. \n",13 "After fine-tuning the model you can also share your adapters on the 🤗 Hub and load them very easily. Let's get started!"14 ]15 },16 {17 "cell_type": "markdown",18 "metadata": {19 "id": "TfBzP8gWzkpv"20 },21 "source": [22 "### Install requirements\n",23 "\n",24 "First, run the cells below to install the requirements:"25 ]26 },27 {28 "cell_type": "code",29 "execution_count": 1,30 "metadata": {31 "colab": {32 "base_uri": "https://localhost:8080/"33 },34 "id": "otj46qRbtpnd",35 "outputId": "2aa109f6-3f4e-4887-a16e-336f51e7cc9a"36 },37 "outputs": [38 {39 "name": "stdout",40 "output_type": "stream",41 "text": [42 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.3/76.3 MB\u001b[0m \u001b[31m10.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",43 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.8/462.8 KB\u001b[0m \u001b[31m25.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",44 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.7/199.7 KB\u001b[0m \u001b[31m25.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",45 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m23.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",46 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.0/213.0 KB\u001b[0m \u001b[31m26.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",47 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.0/132.0 KB\u001b[0m \u001b[31m18.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",48 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.6/140.6 KB\u001b[0m \u001b[31m20.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",49 "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",50 " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",51 " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",52 " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",53 " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",54 " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",55 "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m72.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",56 "\u001b[?25h Building wheel for transformers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",57 " Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n"58 ]59 }60 ],61 "source": [62 "!pip install -q bitsandbytes datasets accelerate\n",63 "!pip install -q git+https://github.com/huggingface/transformers.git@main git+https://github.com/huggingface/peft.git"64 ]65 },66 {67 "cell_type": "markdown",68 "metadata": {69 "id": "FOtwYRI3zzXI"70 },71 "source": [72 "### Model loading\n",73 "\n",74 "Here let's load the `opt-6.7b` model, its weights in half-precision (float16) are about 13GB on the Hub! If we load them in 8-bit we would require around 7GB of memory instead."75 ]76 },77 {78 "cell_type": "code",79 "execution_count": 4,80 "metadata": {81 "colab": {82 "base_uri": "https://localhost:8080/",83 "height": 408,84 "referenced_widgets": [85 "d4de260ffd8a440eb87eb900fc1bb1d3",86 "8602b545a9f8474dbb3cc178ac0b8e60",87 "b46919912ee54f6f9f2ce9080be1c61a",88 "50374e3ab81c4626a182e61fc03b94ce",89 "2144bc2897dc40b29f060e30ace12275",90 "949ca70002ca4472bbc21fea4d7ac745",91 "49943c9dadca43a584b3f354ba45280c",92 "6123e53fb26b41f0af9a3a3348ae1afd",93 "285ef943d540400ab827c462945a259c",94 "95727290446244ccb9626f4594949675",95 "61b54aa6c9e94ee1bc45c15a9e3f7917",96 "fc2d5ffe254d425b939252ec46ec27cc",97 "f65af2e868244edeb0cc9402534874a8",98 "e466054f08004bbcabb24e400cb3c7fc",99 "6ea40800dfd849e3b106bae71fc53ae3",100 "722a01f42b7d4c38836a4546ecb38108",101 "1bd5179cdb474b65aa06eca3520ad37b",102 "04d367124a3b419ab1fa1dfd4f9004c3",103 "4ea667f48b9f4e1f9da7c5a0d3025b85",104 "9b96c63630654773acd38b8b88371f28",105 "cb69ae47666a4603a07a8778e2ae7d6e",106 "8f5e9f2d11d54fd2a08dcbff9f6da05c",107 "c6f712eadc4d49019b2bd355968cc155",108 "5fd979c05fd34311af877ce1a988ead8",109 "179165ff4c0e4586aaf3a40b8502f428",110 "cad1d8326e474a4f9ee13db9005121dd",111 "b049ac0b3d2e44ba9b1fc1cb2dd3de62",112 "921c268ecd724b6a8869dae8f81d558a",113 "4cf85a85c0764f15b5134c81b360e910",114 "5936d9eb59e147eea6482006decfe0ee",115 "fad61c68edc84c8197afeafded84280d",116 "451309d8b3ca4cd9b6f538640477039e",117 "c7621e14aa16421d9758321e433b92e4",118 "5aa74b9b30614172b07f88873cf89471",119 "cf39fb025e3d4635a5695135a56d9f64",120 "deaf9732f33446a3be015d2ec16aba76",121 "bdfd856dd8ee4ae09205ad9d1b9cc806",122 "7a49f7a55b054b6d829f290a1a426a7a",123 "5b695486a0ac442b8b6a8ef2ebf4e57a",124 "03be7cb91cba4afab795aab7aa242ee1",125 "8f7890f54d514f80b1eb17905cf0f964",126 "ae660d51267544a89f0ad199cc12b6fa",127 "d5bb8d7359274c8e9cc79df563175137",128 "98d5a80ec50c46c18f9ee991e2982115",129 "e73e5388182040a8937ccf1748171a87",130 "f797f7b8b13f4b3cb871522d34498631",131 "41f47c864e094cacb1c550d37ddfe80e",132 "104d982b444947fe8e4fbb2c2f082616",133 "2cf8581583c641fb95d1e16aff7d4cd1",134 "11808e9097424dd0b22a5af6c77813f3",135 "dc08d237860e4788a8ceeff4518c2612",136 "7bbf2c40a4ae46fe9e5885db08975263",137 "f00cbc4a89f5492787ec489da65ff70b",138 "8a7cd2194113493a841b00d034b5f1ba",139 "e9673363e85448d494ac9ac5d7ba0efb",140 "a994beafbf3f4c20880a7bbe3898db36",141 "7769c261781f4f5483c7e9d58c1a5573",142 "4a713a8fe16f4e81aa841c69711fa136",143 "08d036904ecf47ed88e129ba6e2b285c",144 "9f1182fdddab43b59ee98bc701965a17",145 "c2302f535b114ad780bfa440445c2e28",146 "8fe032f285ba4858b8efd68119c217b0",147 "4ca695fca3d140c6ab4e1e1d34df807b",148 "b77ceaf55dc04810963cdd01126478f6",149 "0ccdefcb25e14d229b2634ffae4a6d3d",150 "11d6b952503c4824b36b66e228f87599",151 "1e9391f6c89c4d08859ef3413edb19be",152 "039bbda2402f469eb21ba7ec7ea589a6",153 "5da6eef8fb0048219159f38a68727b64",154 "c006e62ee6d04831b2b89273ef04a8ac",155 "af0634c2539a43989902daef47776901",156 "62fea00ef0364af287e6097b964d00c4",157 "9e5ef73f8b244845a6b9002fa5c35d15",158 "5cb09c84e1e144e0a92580fb5e1ce2fe",159 "0f561c1660744251a8f710b69d434c87",160 "9e5f870ef80242f5af09fad70f84ea62",161 "6f173ca73dd545deb22b8cd0470d925f",162 "4e6d5943bc374b388b93ed115e44b6a5",163 "cbe2a6ea41834e95a27d6f02c3c0eeec",164 "7beb5f4efefc4593abac253df74d1405",165 "370cefbaeebf41f582b7507ad493055f",166 "e3df9dda16e244aa9b61d54f9e21ef40",167 "23d2ad64a17041b7a006dad1e041e0a1",168 "c1fd6a1234274a44b838a09f3f5380c6",169 "81bb51d088374394becd9a45ec3b17d4",170 "9e762779e5434bb7afcc295b61c2f4e4",171 "f539b7a4665449de9eac209a20629969",172 "32d528db79ad4f6f836ab2e0df5ac426",173 "1ca7684b79c5438fa06b047bd2b3283f",174 "a07688185bff4c4b8cbed3af3b4cf802",175 "0272f1d9f93f4dd788363a8409cdfd69",176 "27b41d23d2c64127ba3ae8464958f855",177 "2b54032c0d8e4a2897aed1ac1c79af14",178 "ed8fa1048e814f2fa3666899fc42e55a",179 "97daf559100c44ac983562fea93c5fac",180 "72e511b775604d899ff5b3fa2ebe9fc4",181 "da946f86590447d2ab98b9da468fa66b",182 "54fe79d5c7254117a2209927a7248dd4",183 "1d122e4eaad54e06961288484f31e18b",184 "d46b5725c35142a89617e46c0e8d3679",185 "c5493c23fd5542738ffd1ff5f09a6a67",186 "a1a80d3460984c2496ada5a634875934",187 "598c5584ffba4f26815c4e87bb1595c4",188 "fe93f25323604447be0bb1d24a0c2c59",189 "00c2e2d3ee8b45818ba84da12c6b11e2",190 "6ccea64c2e614a9fbdcc2f716cecaea0",191 "63fc9a9eebca4f2db2ed8a385fc5e204",192 "9a4860dfeac944db85e6e532599bc1cb",193 "3b946e1bbab24629b98307275fbe7cbb",194 "d9d36f8ff5f747bf90fbc8a7d35a6664"195 ]196 },197 "id": "cg3fiQOvmI3Q",198 "outputId": "135a7675-6a4d-4786-b5dc-34cb867f40c7"199 },200 "outputs": [201 {202 "data": {203 "application/vnd.jupyter.widget-view+json": {204 "model_id": "bee2f575b3e64c30b2f3afa137802406",205 "version_major": 2,206 "version_minor": 0207 },208 "text/plain": [209 "Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]"210 ]211 },212 "metadata": {},213 "output_type": "display_data"214 }215 ],216 "source": [217 "import os\n",218 "\n",219 "import torch\n",220 "import torch.nn as nn\n",221 "import bitsandbytes as bnb\n",222 "from transformers import AutoTokenizer, AutoConfig, AutoModelForCausalLM\n",223 "\n",224 "model = AutoModelForCausalLM.from_pretrained(\"facebook/opt-6.7b\", load_in_8bit=True)\n",225 "\n",226 "tokenizer = AutoTokenizer.from_pretrained(\"facebook/opt-6.7b\")"227 ]228 },229 {230 "attachments": {},231 "cell_type": "markdown",232 "metadata": {233 "id": "9fTSZntA1iUG"234 },235 "source": [236 "### Prepare model for training\n",237 "\n",238 "Some pre-processing needs to be done before training such an int8 model using `peft`, therefore let's import an utiliy function `prepare_model_for_int8_training` that will: \n",239 "- Casts all the non `int8` modules to full precision (`fp32`) for stability\n",240 "- Add a `forward_hook` to the input embedding layer to enable gradient computation of the input hidden states\n",241 "- Enable gradient checkpointing for more memory-efficient training"242 ]243 },244 {245 "cell_type": "code",246 "execution_count": 5,247 "metadata": {248 "id": "T-gy-LxM0yAi"249 },250 "outputs": [],251 "source": [252 "from peft import prepare_model_for_int8_training\n",253 "\n",254 "model = prepare_model_for_int8_training(model)"255 ]256 },257 {258 "cell_type": "markdown",259 "metadata": {260 "id": "KwOTr7B3NlM3"261 },262 "source": [263 "### Apply LoRA\n",264 "\n",265 "Here comes the magic with `peft`! Let's load a `PeftModel` and specify that we are going to use low-rank adapters (LoRA) using `get_peft_model` utility function from `peft`."266 ]267 },268 {269 "cell_type": "code",270 "execution_count": 6,271 "metadata": {272 "id": "4W1j6lxaNnxC"273 },274 "outputs": [],275 "source": [276 "def print_trainable_parameters(model):\n",277 " \"\"\"\n",278 " Prints the number of trainable parameters in the model.\n",279 " \"\"\"\n",280 " trainable_params = 0\n",281 " all_param = 0\n",282 " for _, param in model.named_parameters():\n",283 " all_param += param.numel()\n",284 " if param.requires_grad:\n",285 " trainable_params += param.numel()\n",286 " print(\n",287 " f\"trainable params: {trainable_params} || all params: {all_param} || trainable%: {100 * trainable_params / all_param}\"\n",288 " )"289 ]290 },291 {292 "cell_type": "code",293 "execution_count": 7,294 "metadata": {295 "colab": {296 "base_uri": "https://localhost:8080/"297 },298 "id": "4iwHGzKBN6wk",299 "outputId": "039f7175-14c9-42b4-a078-80d27aab161c"300 },301 "outputs": [302 {303 "name": "stdout",304 "output_type": "stream",305 "text": [306 "trainable params: 8388608 || all params: 6666862592 || trainable%: 0.12582542214183376\n"307 ]308 }309 ],310 "source": [311 "from peft import LoraConfig, get_peft_model\n",312 "\n",313 "config = LoraConfig(\n",314 " r=16, lora_alpha=32, target_modules=[\"q_proj\", \"v_proj\"], lora_dropout=0.05, bias=\"none\", task_type=\"CAUSAL_LM\"\n",315 ")\n",316 "\n",317 "model = get_peft_model(model, config)\n",318 "print_trainable_parameters(model)"319 ]320 },321 {322 "cell_type": "markdown",323 "metadata": {324 "id": "QdjWif4CVXR6"325 },326 "source": [327 "### Training"328 ]329 },330 {331 "cell_type": "markdown",332 "metadata": {333 "id": "b_RPiO7f3ClX"334 },335 "source": []336 },337 {338 "cell_type": "code",339 "execution_count": 9,340 "metadata": {341 "colab": {342 "base_uri": "https://localhost:8080/",343 "height": 1000,344 "referenced_widgets": [345 "f357166c6e5f43f39d0a287ca6d6f60e",346 "805a10c2fd794ff692e8ebeefd65f2eb",347 "3730f843399d4ba48e98383563283e94",348 "f855aced7ca2485ea720604359deaa18",349 "6d728366de1a4bacb1ba1939c5e0146f",350 "c57c7cf35bf04f3bb3b2b0d8ce7feb31",351 "2113970533604749a808fa1b98269b0e",352 "fc15c6d6eb3049a3b8542b332dd8a3f2",353 "b6c06e70d7ed48f1840086489300cb3d",354 "49504531deaf4449938bea751d1ec4e7",355 "7dfe540a75864cf390b3bed20ab1dcd9",356 "c81d20fe47ce4b7594427830d71504d7",357 "a14542c8431c48b48a614cfd0d41f03c",358 "856f3dcf949741acb394f252186a1d7e",359 "865bae11c917492a9a1ef7286a493bd5",360 "2561b7a7c1694f229d30d2b1eeb14b2f",361 "96bd12acd63a4232b2f4ac159cc4a768",362 "746c8a2fd0ee458bad85cc75ac333e43",363 "ce6de6f9ddde4a6d8094a2b96eac3a4e",364 "62ceac028f144120a24e75afbaccf306",365 "2f953abe023240558580dcff3f0c033e",366 "c2b42681b8bb47e3895d6105240c5812",367 "ae58b3f129644729aa2f890b1712fc6e",368 "74a968ce2dc34189b7dfbb39276f5138",369 "d4f5b19f75e246df9c688f625792e8ba",370 "5cd7a5bbab5f4f2daa53d984568ea630",371 "7288f4fd6bb44089baef9f20f50e0e04",372 "57ff7d85e56f49239d1eeedd43b88d22",373 "4d5cbca8d5e647669faeea32de761dcd",374 "a6ce291698ad460394433a49000c1d25",375 "aa54b2a9b43848b0902d135beffb806b",376 "8cd2073cd3304b2ca4b997127aa88bd1",377 "f51c7f18977447e2bca36e1da3e1be4f",378 "b8c29c1accf4479da91d393dd59ca82b",379 "5fa269b8704e4509bf8cd918b657ae23",380 "ac91fc78fb04436b80a0a2cbe4380c2b",381 "5db87509bf56435eafaea4cf1153a5d3",382 "840981ef419a48918cf000f89807890e",383 "2c27ab87b28946e39681a29cc6f14ea9",384 "c4ab408eb1344da0bb15a9a6760818e4",385 "d54e8d69575f49eb977da64abc5ceb0c",386 "548299d96190406391cab49ad29cc5d6",387 "c41a9d785e884ab0a58117d17ac7d228",388 "256c56849e8545018514fd52f1247501",389 "041944011c204558a4315c27ec5dabce",390 "1247ee7aeac5406e93e22313d54ef54a",391 "ed2736d862a94d8f9db9ba6037016071",392 "ddc333530c13446a91cf332846bfa22f",393 "3d2325879cfa48048e81c44e2c2444d5",394 "7573358a50bd446486b4c5528a298fae",395 "663c338f84c94b63bc05b0fb6835d99f",396 "3185bd8ecbde4f26b8ed0f92cf79e14f",397 "ddc36fdbdd634dc489f658bead61e7ee",398 "d7e33c29d410414eb452d121edd9920e",399 "d3511ce1754b41969e5c36a5b33ac466",400 "9225dd50bcea435289fa2eba64087076",401 "a23144916f02459b966b9830f0a1d64c",402 "c9b718882fec4254bce1f33fa9373921",403 "f27905d0073e493cb9dcd174c0f15e35",404 "0f634721a70d4e749cb616e55ab747b3",405 "e1f62cbd805d4b8aa9aba7e345c21c82",406 "893d8cb7857147ba82ac86d140f69a5c",407 "4884bf82f5814e049c47cf6d496aab08",408 "432fc8277ebc492f91d6b46ed073ccb4",409 "c423a3cc0b504828b11077c77268ed92",410 "668dd47eec9942bcad0af209772cf8e6",411 "a618f430e06645eb9c95bf16bb6ea59a",412 "bba20d9bf1974a7f8d1ebcb9f5c4cd49",413 "5cec06ccf9074e019ea1f7daf17a0319",414 "faf24b3ed994422f8dd806ae0cc30531",415 "79f7929f423d493caf86b548e91e8a42",416 "23cb4d44c4534bae80357c8a9f6b86ba",417 "4fa82dbe7d3a4b3996b1fa1cf9c23498",418 "6eee224b244a4cafb3fc0f3b16eb0d03",419 "73344836205f4569a4113c24985bd5b1",420 "86dff0987bf040da99c8f2846da26d86",421 "63f8ad255d2147128bdc26f47fdf2528"422 ]423 },424 "id": "AQ_HCYruWIHU",425 "outputId": "e9baadaf-e202-413d-87f0-74c730d78408"426 },427 "outputs": [428 {429 "data": {430 "application/vnd.jupyter.widget-view+json": {431 "model_id": "f357166c6e5f43f39d0a287ca6d6f60e",432 "version_major": 2,433 "version_minor": 0434 },435 "text/plain": [436 "Downloading readme: 0%| | 0.00/5.55k [00:00<?, ?B/s]"437 ]438 },439 "metadata": {},440 "output_type": "display_data"441 },442 {443 "name": "stderr",444 "output_type": "stream",445 "text": [446 "WARNING:datasets.builder:Using custom data configuration Abirate--english_quotes-6e72855d06356857\n"447 ]448 },449 {450 "name": "stdout",451 "output_type": "stream",452 "text": [453 "Downloading and preparing dataset json/Abirate--english_quotes to /root/.cache/huggingface/datasets/Abirate___json/Abirate--english_quotes-6e72855d06356857/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51...\n"454 ]455 },456 {457 "data": {458 "application/vnd.jupyter.widget-view+json": {459 "model_id": "c81d20fe47ce4b7594427830d71504d7",460 "version_major": 2,461 "version_minor": 0462 },463 "text/plain": [464 "Downloading data files: 0%| | 0/1 [00:00<?, ?it/s]"465 ]466 },467 "metadata": {},468 "output_type": "display_data"469 },470 {471 "data": {472 "application/vnd.jupyter.widget-view+json": {473 "model_id": "ae58b3f129644729aa2f890b1712fc6e",474 "version_major": 2,475 "version_minor": 0476 },477 "text/plain": [478 "Downloading data: 0%| | 0.00/647k [00:00<?, ?B/s]"479 ]480 },481 "metadata": {},482 "output_type": "display_data"483 },484 {485 "data": {486 "application/vnd.jupyter.widget-view+json": {487 "model_id": "b8c29c1accf4479da91d393dd59ca82b",488 "version_major": 2,489 "version_minor": 0490 },491 "text/plain": [492 "Extracting data files: 0%| | 0/1 [00:00<?, ?it/s]"493 ]494 },495 "metadata": {},496 "output_type": "display_data"497 },498 {499 "data": {500 "application/vnd.jupyter.widget-view+json": {501 "model_id": "041944011c204558a4315c27ec5dabce",502 "version_major": 2,503 "version_minor": 0504 },505 "text/plain": [506 "Generating train split: 0 examples [00:00, ? examples/s]"507 ]508 },509 "metadata": {},510 "output_type": "display_data"511 },512 {513 "name": "stdout",514 "output_type": "stream",515 "text": [516 "Dataset json downloaded and prepared to /root/.cache/huggingface/datasets/Abirate___json/Abirate--english_quotes-6e72855d06356857/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data.\n"517 ]518 },519 {520 "data": {521 "application/vnd.jupyter.widget-view+json": {522 "model_id": "9225dd50bcea435289fa2eba64087076",523 "version_major": 2,524 "version_minor": 0525 },526 "text/plain": [527 " 0%| | 0/1 [00:00<?, ?it/s]"528 ]529 },530 "metadata": {},531 "output_type": "display_data"532 },533 {534 "data": {535 "application/vnd.jupyter.widget-view+json": {536 "model_id": "a618f430e06645eb9c95bf16bb6ea59a",537 "version_major": 2,538 "version_minor": 0539 },540 "text/plain": [541 " 0%| | 0/3 [00:00<?, ?ba/s]"542 ]543 },544 "metadata": {},545 "output_type": "display_data"546 },547 {548 "name": "stderr",549 "output_type": "stream",550 "text": [551 "The model is loaded in 8-bit precision. To train this model you need to add additional modules inside the model such as adapters using `peft` library and freeze the model weights. Please check the examples in https://github.com/huggingface/peft for more details.\n",552 "max_steps is given, it will override any value given in num_train_epochs\n",553 "Using cuda_amp half precision backend\n",554 "The following columns in the training set don't have a corresponding argument in `PeftModelForCausalLM.forward` and have been ignored: author, tags, quote. If author, tags, quote are not expected by `PeftModelForCausalLM.forward`, you can safely ignore this message.\n",555 "/usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",556 " warnings.warn(\n",557 "***** Running training *****\n",558 " Num examples = 2508\n",559 " Num Epochs = 2\n",560 " Instantaneous batch size per device = 4\n",561 " Total train batch size (w. parallel, distributed & accumulation) = 16\n",562 " Gradient Accumulation steps = 4\n",563 " Total optimization steps = 200\n",564 " Number of trainable parameters = 8388608\n",565 "/usr/local/lib/python3.8/dist-packages/bitsandbytes/autograd/_functions.py:298: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization\n",566 " warnings.warn(f\"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization\")\n"567 ]568 },569 {570 "data": {571 "text/html": [572 "\n",573 " <div>\n",574 " \n",575 " <progress value='153' max='200' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",576 " [153/200 26:04 < 08:06, 0.10 it/s, Epoch 0.97/2]\n",577 " </div>\n",578 " <table border=\"1\" class=\"dataframe\">\n",579 " <thead>\n",580 " <tr style=\"text-align: left;\">\n",581 " <th>Step</th>\n",582 " <th>Training Loss</th>\n",583 " </tr>\n",584 " </thead>\n",585 " <tbody>\n",586 " <tr>\n",587 " <td>1</td>\n",588 " <td>2.364400</td>\n",589 " </tr>\n",590 " <tr>\n",591 " <td>2</td>\n",592 " <td>2.200400</td>\n",593 " </tr>\n",594 " <tr>\n",595 " <td>3</td>\n",596 " <td>2.302300</td>\n",597 " </tr>\n",598 " <tr>\n",599 " <td>4</td>\n",600 " <td>2.184700</td>\n",601 " </tr>\n",602 " <tr>\n",603 " <td>5</td>\n",604 " <td>1.878700</td>\n",605 " </tr>\n",606 " <tr>\n",607 " <td>6</td>\n",608 " <td>2.307200</td>\n",609 " </tr>\n",610 " <tr>\n",611 " <td>7</td>\n",612 " <td>2.193800</td>\n",613 " </tr>\n",614 " <tr>\n",615 " <td>8</td>\n",616 " <td>2.446200</td>\n",617 " </tr>\n",618 " <tr>\n",619 " <td>9</td>\n",620 " <td>2.458900</td>\n",621 " </tr>\n",622 " <tr>\n",623 " <td>10</td>\n",624 " <td>2.020000</td>\n",625 " </tr>\n",626 " <tr>\n",627 " <td>11</td>\n",628 " <td>1.941200</td>\n",629 " </tr>\n",630 " <tr>\n",631 " <td>12</td>\n",632 " <td>1.931000</td>\n",633 " </tr>\n",634 " <tr>\n",635 " <td>13</td>\n",636 " <td>2.055900</td>\n",637 " </tr>\n",638 " <tr>\n",639 " <td>14</td>\n",640 " <td>1.975100</td>\n",641 " </tr>\n",642 " <tr>\n",643 " <td>15</td>\n",644 " <td>2.015100</td>\n",645 " </tr>\n",646 " <tr>\n",647 " <td>16</td>\n",648 " <td>2.095600</td>\n",649 " </tr>\n",650 " <tr>\n",651 " <td>17</td>\n",652 " <td>1.768300</td>\n",653 " </tr>\n",654 " <tr>\n",655 " <td>18</td>\n",656 " <td>2.155700</td>\n",657 " </tr>\n",658 " <tr>\n",659 " <td>19</td>\n",660 " <td>2.402300</td>\n",661 " </tr>\n",662 " <tr>\n",663 " <td>20</td>\n",664 " <td>2.124600</td>\n",665 " </tr>\n",666 " <tr>\n",667 " <td>21</td>\n",668 " <td>2.314900</td>\n",669 " </tr>\n",670 " <tr>\n",671 " <td>22</td>\n",672 " <td>1.908500</td>\n",673 " </tr>\n",674 " <tr>\n",675 " <td>23</td>\n",676 " <td>2.078800</td>\n",677 " </tr>\n",678 " <tr>\n",679 " <td>24</td>\n",680 " <td>1.941900</td>\n",681 " </tr>\n",682 " <tr>\n",683 " <td>25</td>\n",684 " <td>1.879800</td>\n",685 " </tr>\n",686 " <tr>\n",687 " <td>26</td>\n",688 " <td>1.927500</td>\n",689 " </tr>\n",690 " <tr>\n",691 " <td>27</td>\n",692 " <td>1.371400</td>\n",693 " </tr>\n",694 " <tr>\n",695 " <td>28</td>\n",696 " <td>1.977600</td>\n",697 " </tr>\n",698 " <tr>\n",699 " <td>29</td>\n",700 " <td>2.055000</td>\n",701 " </tr>\n",702 " <tr>\n",703 " <td>30</td>\n",704 " <td>1.915800</td>\n",705 " </tr>\n",706 " <tr>\n",707 " <td>31</td>\n",708 " <td>1.958100</td>\n",709 " </tr>\n",710 " <tr>\n",711 " <td>32</td>\n",712 " <td>2.195900</td>\n",713 " </tr>\n",714 " <tr>\n",715 " <td>33</td>\n",716 " <td>2.001000</td>\n",717 " </tr>\n",718 " <tr>\n",719 " <td>34</td>\n",720 " <td>2.025000</td>\n",721 " </tr>\n",722 " <tr>\n",723 " <td>35</td>\n",724 " <td>1.576900</td>\n",725 " </tr>\n",726 " <tr>\n",727 " <td>36</td>\n",728 " <td>1.879800</td>\n",729 " </tr>\n",730 " <tr>\n",731 " <td>37</td>\n",732 " <td>1.821600</td>\n",733 " </tr>\n",734 " <tr>\n",735 " <td>38</td>\n",736 " <td>1.727800</td>\n",737 " </tr>\n",738 " <tr>\n",739 " <td>39</td>\n",740 " <td>1.995700</td>\n",741 " </tr>\n",742 " <tr>\n",743 " <td>40</td>\n",744 " <td>1.698600</td>\n",745 " </tr>\n",746 " <tr>\n",747 " <td>41</td>\n",748 " <td>2.129300</td>\n",749 " </tr>\n",750 " <tr>\n",751 " <td>42</td>\n",752 " <td>2.025800</td>\n",753 " </tr>\n",754 " <tr>\n",755 " <td>43</td>\n",756 " <td>1.696500</td>\n",757 " </tr>\n",758 " <tr>\n",759 " <td>44</td>\n",760 " <td>1.984700</td>\n",761 " </tr>\n",762 " <tr>\n",763 " <td>45</td>\n",764 " <td>2.051100</td>\n",765 " </tr>\n",766 " <tr>\n",767 " <td>46</td>\n",768 " <td>2.054400</td>\n",769 " </tr>\n",770 " <tr>\n",771 " <td>47</td>\n",772 " <td>1.765600</td>\n",773 " </tr>\n",774 " <tr>\n",775 " <td>48</td>\n",776 " <td>2.063100</td>\n",777 " </tr>\n",778 " <tr>\n",779 " <td>49</td>\n",780 " <td>1.746900</td>\n",781 " </tr>\n",782 " <tr>\n",783 " <td>50</td>\n",784 " <td>1.873000</td>\n",785 " </tr>\n",786 " <tr>\n",787 " <td>51</td>\n",788 " <td>2.391300</td>\n",789 " </tr>\n",790 " <tr>\n",791 " <td>52</td>\n",792 " <td>2.494100</td>\n",793 " </tr>\n",794 " <tr>\n",795 " <td>53</td>\n",796 " <td>2.072300</td>\n",797 " </tr>\n",798 " <tr>\n",799 " <td>54</td>\n",800 " <td>1.808000</td>\n",801 " </tr>\n",802 " <tr>\n",803 " <td>55</td>\n",804 " <td>1.911900</td>\n",805 " </tr>\n",806 " <tr>\n",807 " <td>56</td>\n",808 " <td>2.168100</td>\n",809 " </tr>\n",810 " <tr>\n",811 " <td>57</td>\n",812 " <td>2.166100</td>\n",813 " </tr>\n",814 " <tr>\n",815 " <td>58</td>\n",816 " <td>1.921500</td>\n",817 " </tr>\n",818 " <tr>\n",819 " <td>59</td>\n",820 " <td>1.856000</td>\n",821 " </tr>\n",822 " <tr>\n",823 " <td>60</td>\n",824 " <td>1.652800</td>\n",825 " </tr>\n",826 " <tr>\n",827 " <td>61</td>\n",828 " <td>1.605000</td>\n",829 " </tr>\n",830 " <tr>\n",831 " <td>62</td>\n",832 " <td>2.032500</td>\n",833 " </tr>\n",834 " <tr>\n",835 " <td>63</td>\n",836 " <td>1.822100</td>\n",837 " </tr>\n",838 " <tr>\n",839 " <td>64</td>\n",840 " <td>1.623600</td>\n",841 " </tr>\n",842 " <tr>\n",843 " <td>65</td>\n",844 " <td>1.923200</td>\n",845 " </tr>\n",846 " <tr>\n",847 " <td>66</td>\n",848 " <td>2.053200</td>\n",849 " </tr>\n",850 " <tr>\n",851 " <td>67</td>\n",852 " <td>2.114300</td>\n",853 " </tr>\n",854 " <tr>\n",855 " <td>68</td>\n",856 " <td>1.807700</td>\n",857 " </tr>\n",858 " <tr>\n",859 " <td>69</td>\n",860 " <td>1.857800</td>\n",861 " </tr>\n",862 " <tr>\n",863 " <td>70</td>\n",864 " <td>1.854600</td>\n",865 " </tr>\n",866 " <tr>\n",867 " <td>71</td>\n",868 " <td>2.023000</td>\n",869 " </tr>\n",870 " <tr>\n",871 " <td>72</td>\n",872 " <td>1.864900</td>\n",873 " </tr>\n",874 " <tr>\n",875 " <td>73</td>\n",876 " <td>1.769300</td>\n",877 " </tr>\n",878 " <tr>\n",879 " <td>74</td>\n",880 " <td>1.837700</td>\n",881 " </tr>\n",882 " <tr>\n",883 " <td>75</td>\n",884 " <td>1.742200</td>\n",885 " </tr>\n",886 " <tr>\n",887 " <td>76</td>\n",888 " <td>1.895900</td>\n",889 " </tr>\n",890 " <tr>\n",891 " <td>77</td>\n",892 " <td>1.922800</td>\n",893 " </tr>\n",894 " <tr>\n",895 " <td>78</td>\n",896 " <td>2.325300</td>\n",897 " </tr>\n",898 " <tr>\n",899 " <td>79</td>\n",900 " <td>2.231200</td>\n",901 " </tr>\n",902 " <tr>\n",903 " <td>80</td>\n",904 " <td>2.309500</td>\n",905 " </tr>\n",906 " <tr>\n",907 " <td>81</td>\n",908 " <td>1.945700</td>\n",909 " </tr>\n",910 " <tr>\n",911 " <td>82</td>\n",912 " <td>2.072100</td>\n",913 " </tr>\n",914 " <tr>\n",915 " <td>83</td>\n",916 " <td>1.917400</td>\n",917 " </tr>\n",918 " <tr>\n",919 " <td>84</td>\n",920 " <td>2.004600</td>\n",921 " </tr>\n",922 " <tr>\n",923 " <td>85</td>\n",924 " <td>1.951700</td>\n",925 " </tr>\n",926 " <tr>\n",927 " <td>86</td>\n",928 " <td>1.450600</td>\n",929 " </tr>\n",930 " <tr>\n",931 " <td>87</td>\n",932 " <td>1.785600</td>\n",933 " </tr>\n",934 " <tr>\n",935 " <td>88</td>\n",936 " <td>1.668000</td>\n",937 " </tr>\n",938 " <tr>\n",939 " <td>89</td>\n",940 " <td>1.903100</td>\n",941 " </tr>\n",942 " <tr>\n",943 " <td>90</td>\n",944 " <td>1.709800</td>\n",945 " </tr>\n",946 " <tr>\n",947 " <td>91</td>\n",948 " <td>2.312900</td>\n",949 " </tr>\n",950 " <tr>\n",951 " <td>92</td>\n",952 " <td>2.092100</td>\n",953 " </tr>\n",954 " <tr>\n",955 " <td>93</td>\n",956 " <td>2.319600</td>\n",957 " </tr>\n",958 " <tr>\n",959 " <td>94</td>\n",960 " <td>1.603100</td>\n",961 " </tr>\n",962 " <tr>\n",963 " <td>95</td>\n",964 " <td>1.740000</td>\n",965 " </tr>\n",966 " <tr>\n",967 " <td>96</td>\n",968 " <td>1.670500</td>\n",969 " </tr>\n",970 " <tr>\n",971 " <td>97</td>\n",972 " <td>1.611600</td>\n",973 " </tr>\n",974 " <tr>\n",975 " <td>98</td>\n",976 " <td>1.728900</td>\n",977 " </tr>\n",978 " <tr>\n",979 " <td>99</td>\n",980 " <td>2.285200</td>\n",981 " </tr>\n",982 " <tr>\n",983 " <td>100</td>\n",984 " <td>1.957800</td>\n",985 " </tr>\n",986 " <tr>\n",987 " <td>101</td>\n",988 " <td>1.676700</td>\n",989 " </tr>\n",990 " <tr>\n",991 " <td>102</td>\n",992 " <td>1.656300</td>\n",993 " </tr>\n",994 " <tr>\n",995 " <td>103</td>\n",996 " <td>1.612400</td>\n",997 " </tr>\n",998 " <tr>\n",999 " <td>104</td>\n",1000 " <td>1.848900</td>\n",1001 " </tr>\n",1002 " <tr>\n",1003 " <td>105</td>\n",1004 " <td>1.870000</td>\n",1005 " </tr>\n",1006 " <tr>\n",1007 " <td>106</td>\n",1008 " <td>1.954000</td>\n",1009 " </tr>\n",1010 " <tr>\n",1011 " <td>107</td>\n",1012 " <td>2.192200</td>\n",1013 " </tr>\n",1014 " <tr>\n",1015 " <td>108</td>\n",1016 " <td>1.637600</td>\n",1017 " </tr>\n",1018 " <tr>\n",1019 " <td>109</td>\n",1020 " <td>1.208700</td>\n",1021 " </tr>\n",1022 " <tr>\n",1023 " <td>110</td>\n",1024 " <td>2.254200</td>\n",1025 " </tr>\n",1026 " <tr>\n",1027 " <td>111</td>\n",1028 " <td>1.832100</td>\n",1029 " </tr>\n",1030 " <tr>\n",1031 " <td>112</td>\n",1032 " <td>2.119600</td>\n",1033 " </tr>\n",1034 " <tr>\n",1035 " <td>113</td>\n",1036 " <td>2.126400</td>\n",1037 " </tr>\n",1038 " <tr>\n",1039 " <td>114</td>\n",1040 " <td>1.915700</td>\n",1041 " </tr>\n",1042 " <tr>\n",1043 " <td>115</td>\n",1044 " <td>1.587500</td>\n",1045 " </tr>\n",1046 " <tr>\n",1047 " <td>116</td>\n",1048 " <td>1.564800</td>\n",1049 " </tr>\n",1050 " <tr>\n",1051 " <td>117</td>\n",1052 " <td>1.742700</td>\n",1053 " </tr>\n",1054 " <tr>\n",1055 " <td>118</td>\n",1056 " <td>1.712600</td>\n",1057 " </tr>\n",1058 " <tr>\n",1059 " <td>119</td>\n",1060 " <td>1.727900</td>\n",1061 " </tr>\n",1062 " <tr>\n",1063 " <td>120</td>\n",1064 " <td>2.361500</td>\n",1065 " </tr>\n",1066 " <tr>\n",1067 " <td>121</td>\n",1068 " <td>2.070300</td>\n",1069 " </tr>\n",1070 " <tr>\n",1071 " <td>122</td>\n",1072 " <td>1.878500</td>\n",1073 " </tr>\n",1074 " <tr>\n",1075 " <td>123</td>\n",1076 " <td>1.846600</td>\n",1077 " </tr>\n",1078 " <tr>\n",1079 " <td>124</td>\n",1080 " <td>2.061700</td>\n",1081 " </tr>\n",1082 " <tr>\n",1083 " <td>125</td>\n",1084 " <td>2.149700</td>\n",1085 " </tr>\n",1086 " <tr>\n",1087 " <td>126</td>\n",1088 " <td>1.940600</td>\n",1089 " </tr>\n",1090 " <tr>\n",1091 " <td>127</td>\n",1092 " <td>2.098300</td>\n",1093 " </tr>\n",1094 " <tr>\n",1095 " <td>128</td>\n",1096 " <td>1.734100</td>\n",1097 " </tr>\n",1098 " <tr>\n",1099 " <td>129</td>\n",1100 " <td>2.111700</td>\n",1101 " </tr>\n",1102 " <tr>\n",1103 " <td>130</td>\n",1104 " <td>1.887600</td>\n",1105 " </tr>\n",1106 " <tr>\n",1107 " <td>131</td>\n",1108 " <td>1.716300</td>\n",1109 " </tr>\n",1110 " <tr>\n",1111 " <td>132</td>\n",1112 " <td>2.070000</td>\n",1113 " </tr>\n",1114 " <tr>\n",1115 " <td>133</td>\n",1116 " <td>1.782200</td>\n",1117 " </tr>\n",1118 " <tr>\n",1119 " <td>134</td>\n",1120 " <td>1.955200</td>\n",1121 " </tr>\n",1122 " <tr>\n",1123 " <td>135</td>\n",1124 " <td>1.762900</td>\n",1125 " </tr>\n",1126 " <tr>\n",1127 " <td>136</td>\n",1128 " <td>1.954700</td>\n",1129 " </tr>\n",1130 " <tr>\n",1131 " <td>137</td>\n",1132 " <td>1.687100</td>\n",1133 " </tr>\n",1134 " <tr>\n",1135 " <td>138</td>\n",1136 " <td>1.979100</td>\n",1137 " </tr>\n",1138 " <tr>\n",1139 " <td>139</td>\n",1140 " <td>1.634600</td>\n",1141 " </tr>\n",1142 " <tr>\n",1143 " <td>140</td>\n",1144 " <td>1.801200</td>\n",1145 " </tr>\n",1146 " <tr>\n",1147 " <td>141</td>\n",1148 " <td>1.954100</td>\n",1149 " </tr>\n",1150 " <tr>\n",1151 " <td>142</td>\n",1152 " <td>1.833900</td>\n",1153 " </tr>\n",1154 " <tr>\n",1155 " <td>143</td>\n",1156 " <td>2.051400</td>\n",1157 " </tr>\n",1158 " <tr>\n",1159 " <td>144</td>\n",1160 " <td>1.921200</td>\n",1161 " </tr>\n",1162 " <tr>\n",1163 " <td>145</td>\n",1164 " <td>1.787500</td>\n",1165 " </tr>\n",1166 " <tr>\n",1167 " <td>146</td>\n",1168 " <td>1.825400</td>\n",1169 " </tr>\n",1170 " <tr>\n",1171 " <td>147</td>\n",1172 " <td>1.363400</td>\n",1173 " </tr>\n",1174 " <tr>\n",1175 " <td>148</td>\n",1176 " <td>1.977400</td>\n",1177 " </tr>\n",1178 " <tr>\n",1179 " <td>149</td>\n",1180 " <td>1.768300</td>\n",1181 " </tr>\n",1182 " <tr>\n",1183 " <td>150</td>\n",1184 " <td>2.226700</td>\n",1185 " </tr>\n",1186 " <tr>\n",1187 " <td>151</td>\n",1188 " <td>1.945500</td>\n",1189 " </tr>\n",1190 " </tbody>\n",1191 "</table><p>"1192 ],1193 "text/plain": [1194 "<IPython.core.display.HTML object>"1195 ]1196 },1197 "metadata": {},1198 "output_type": "display_data"1199 }1200 ],