LSTMs & Transformers

Stay tuned for slides from the third lecture!

Additional materials

If you want to dig a bit deeper, here are (optional!) supplementary readings, a general textbook on deep learning and the paper that first introduced backpropagation for training neural networks:

• Hochreiter & Schmidhuber (1997) Long Short-Term Memory

• Vaswani et al. (2017) Attention is all you need

• Devlin et al. (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

• Radford et al. (2019) Language Models are Unsupervised Multitask Learners (GPT-2)