Prithvi-Raj-Dixit/Large_Language_Model_LLMs
0
1Most modern LLMs are based on transformer architecture which is nothing but deep neural network architecture introduced in 2017 paper2Attention is all you need. 3 4Original Transformer(Research paper): Developed for machine translation(translating English to German and French).5 6Schematic of Transformer architecture:7 8 9 ----------------------------10 | "Das est ein Beispiel" | (The complete output)11 ----------------------------12 ↑ 13 --------------------- ---------------------14 4 | Embeddings | | Decoder | 8 (Generated translated text15 --------------------- --------------------- one word at a time)16 ↑ ↑ 17 18 ---------------------- --------------------------19 3 | Encoder | | Preprocessing steps | 7 (Input text is prepared20 ---------------------- -------------------------- for decoder)21 ↑ ↑ 22 23 ----------------------- ---------------------24 2 | Preprocessing steps | | Input text | 625 ----------------------- ---------------------26 ↑ ↑27 28 ------------------------ ---------------------29 1 | "This is an example" | | "Das est ein" | 5 30 ------------------------ ---------------------31 (Input text to be translated) (Partial output text32 Model completes one 33 word at a time)34 35 36 371 - Input text to be translated382 - Input text prepared for encoder(tokenization)393 - Produces text encodings used by decoder(Tokens are passed to encoder. Encoder converts those tokens to vector embeddings)404 - Encoder returns embedding vector as input to decoder41(Left side of Transformer is to convert tokens to vector embeddings42 so that symentic meaning of words are captured)43 445 - Partial output is produced because model completes one word at a time456 - Input text is prepared for decoder467 - Decoder also receives vector embeddings from left side and also partial output47 and generated output one word at a time)488 - Complete output is produced49 50 51Transformer architecture = Encoder + Decoder52 53------------- --------------54| Encoder | - Encodes input text into vectors | Decoder | - Generates output text from encoded vectors55------------- --------------56 57Key part of Transformer model: Self attention mechanism58 59- Allows model to weigh the importance of different words/tokens relative to each other60- Enables model to capture long range dependencies61 62 63Later variations of Transformer architecture64 65BERT(Bi-directional encoder representations from transformers) - Predicts hidden words in a given sentence (Left side)66GPT models(Generative pretrained Transformers) - Generates new words (Right side because we have all the left information as input from user)67 68Both above models are originated from Transformer model.69 70 71Transformers vs LLMs72- Not all transformers are LLMs73- Transformers are also used for Computer Vision(Image recognition)74 75- Not all LLMs are transformers76- LLMs can also be based on recurrent or CNN architecture as well77 78 79 RNN LSTM80Recurrent Neural ---------> Long Short-Term ---------> Transformers81Networks Memory Networks (2017)82(1980) (1997) 83 84RNN- The big difference is that RNN have feedback loops and updates its memory.85LSTM- It uses two separates paths to make predictions one short one long.86