Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 14 – Transformers and Self-Attention скачать в хорошем качестве

Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 14 – Transformers and Self-Attention

For more information about Stanford’s Artificial Intelligence professional and graduate programs, visit: https://stanford.io/3niIw41 Professor Christopher Manning, Stanford University, Ashish Vaswani & Anna Huang, Google http://onlinehub.stanford.edu/ Professor Christopher Manning Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and of Computer Science Director, Stanford Artificial Intelligence Laboratory (SAIL) To follow along with the course schedule and syllabus, visit: http://web.stanford.edu/class/cs224n/... 0:00 Introduction 2:07 Learning Representations of Variable Length Data 2:28 Recurrent Neural Networks 4:51 Convolutional Neural Networks? 14:06 Attention is Cheap! 16:05 Attention head: Who 16:26 Attention head: Did What? 16:35 Multihead Attention 17:34 Machine Translation: WMT-2014 BLEU 19:07 Frameworks 19:31 Importance of Residuals 23:26 Non-local Means 26:18 Image Transformer Layer 30:56 Raw representations in music and language 37:52 Attention: a weighted average 40:08 Closer look at relative attention 42:41 A Jazz sample from Music Transformer 44:42 Convolutions and Translational Equivariance 45:12 Relative positions Translational Equivariance 50:21 Sequential generation breaks modes. 50:32 Active Research Area #naturallanguageprocessing #deeplearning