**Deep Learning for Machine Translation**

*Hassan Sajjad *&* Fahim Dalvi* (Hamad Bin Khalifa University Qatar)

Statistical methods have dominated the field of machine translation for almost a decade now. These methods use a parallel corpus, i.e. a set of sentence pairs, where a sentence pair consists of a source language sentence, and its corresponding target language translation. The main objective of these methods has been to learn a mapping between the source and target words, and then use this mapping to generate translations of new source sentences. As recently as a couple of years ago, Deep Neural Networks have dethroned the Phrase based methods, and have been shown to give state-of-the-art results for machine translation.

In this lecture series, we will first cover the basics of statistical machine translation to establish the intuition behind machine translation. We will then cover the basics of neural network models – word embedding and neural language model. Finally, we will learn an end-to-end translation system based completely on deep neural networks. In the last part of the lecture series, we will learn to peek into these neural systems and analyze what they learn about the intricacies of a language like morphology and syntax, without ever explicitly seeing these details in the training data. We will see how to adapt these models quickly to the required domain without retraining the model from scratch.

**Background reading**

**Slides**

**Lecture 0**– Introduction & Roadmap [slides]**Lecture 1**– Language & Translation [slides]**Lecture 2**– Language Modeling [slides]- Python tutorial [iPython/Jupyter Notebook]
- Python tutorial as a PDF [non-editable]

**Lecture 3**– Machine Learning [slides]- Decision Boundary Exercise [iPython/Jupyter Notebook]
- Optimization functions demonstration [iPython/Jupyter Notebook]

**Lecture 4**– Machine Learning II [slides]- Linear Classifier with MSE [iPython/Jupyter Notebook]
- Linear Classifier with Hinge loss [iPython/Jupyter Notebook]
- Linear Classifier with Softmax and Cross Entropy [iPython/Jupyter Notebook]

**Lecture 5**– Machine Learning and Neural Networks [slides]- Efficient Linear Classifier with Softmax and Cross Entropy [iPython/Jupyter Notebook]
- Neural Network [iPython/Jupyter Notebook]
- Neural Network with Keras toolkit [iPython/Jupyter Notebook]

**Lecture 6**– Neural Network Language Models [slides]- Language Modeling with Keras [iPython/Jupyter Notebook]

**Lecture 7**– Sequence to Sequence [slides]**Lecture 8**– Practical Neural MT [slides]**Lecture 9**– Analysis of Neural MT [slides]**Lecture 10**– Recent Advancements [slides]