Subhashsah/seq2seq-en-es
2
1---2license: mit3language:4- en5- es6---7# Neural Machine Translation with Attention 🚀8 9A PyTorch implementation of a Sequence-to-Sequence model with Attention for English-Spanish translation.10 11121314 15## 🌟 Features16 17- **Bidirectional GRU Encoder**: Captures context from both directions of the input sequence18- **Attention Mechanism**: Helps the model focus on relevant parts of the input sequence19- **Teacher Forcing**: Implements curriculum learning for better training stability20- **Dynamic Batching**: Efficient training with variable sequence lengths21- **Hugging Face Integration**: Uses MarianTokenizer for robust text processing22 23## 🏗️ Architecture24 25The model consists of three main components:26 271. **Encoder**: Bidirectional GRU network that processes input sequences282. **Attention**: Computes attention weights for each encoder state293. **Decoder**: GRU network that generates translations using attention context30 31```plaintext32Input → Encoder → Attention → Decoder → Translation33 ↑ ↑ ↑34 Embeddings Context Attention Weights35```36 37## 🚀 Quick Start38 391. Clone the repository:40```bash41git clone https://github.com/yourusername/nmt-attention.git42cd nmt-attention43```44 452. Install dependencies:46```bash47pip install torch transformers datasets48```49 503. Train the model:51```python52python train.py53```54 554. Translate text:56```python57from translate import translate58text = "How are you?"59translated = translate(model, text, tokenizer)60print(translated)61 62# Loading a saved model63model = Seq2Seq(encoder, decoder, device)64model.load_state_dict(torch.load('LSTM_text_generator.pth'))65model.eval()66```67 68## 📊 Model Performance69 70Training metrics after 10 epochs:71- Initial Loss: 11.14772- Final Loss: 3.52773- Training Time: ~2 hours on NVIDIA V10074 75## 🔧 Hyperparameters76 77```python78BATCH_SIZE = 3279LEARNING_RATE = 1e-380CLIP = 1.081N_EPOCHS = 1082ENC_EMB_DIM = 25683DEC_EMB_DIM = 25684ENC_HID_DIM = 51285DEC_HID_DIM = 51286```87 88## 📚 Dataset89 90Using the `loresiensis/corpus-en-es` dataset from Hugging Face Hub, which provides English-Spanish sentence pairs for training.91 92## 🤝 Contributing93 941. Fork the repository952. Create your feature branch (`git checkout -b feature/amazing-feature`)963. Commit your changes (`git commit -m 'Add amazing feature'`)974. Push to the branch (`git push origin feature/amazing-feature`)985. Open a Pull Request99 100## 📝 License101 102This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.103 104## 🙏 Acknowledgments105 106- [Attention Is All You Need](https://arxiv.org/abs/1706.03762) paper107- Hugging Face for the transformers library and datasets108- PyTorch team for the amazing deep learning framework109 110---111⭐️ If you found this project helpful, please consider giving it a star!