Hands-On NLP with Transformers and Huggingface: Part 2 (Lewis Tunstall, ML Engineer скачать в хорошем качестве

Hands-On NLP with Transformers and Huggingface: Part 2 (Lewis Tunstall, ML Engineer

Meetup.com: https://www.meetup.com/NLP-Zurich/ Linkedin:   / nlp-zurich   Twitter:   / nlp_zurich   Materials: Slides and notebooks can be found at this GitHub repo: https://github.com/huggingface/workshops Sign up for a Hugging Face account: https://huggingface.co/join Register for the Hugging Face Spaces feature: https://huggingface.co/join?next=/spa... Sign up to Google Colaboratory (or another GPU provider): https://colab.research.google.com/not... Summary: Natural language processing (NLP) is currently undergoing a period of rapid progress, with new benchmarks broken on a monthly basis. The main driver of these breakthroughs is the Transformer architecture, which relies on a self-attention mechanism to encode the importance of symbol order in text sequences. When pretrained on a large corpus of unlabelled text, these novel architectures have proven to be an effective way to capture long-term dependencies, sentiment and other syntactic structures in natural language. For example, the SQuAD benchmark for question-answering is dominated by Transformer models, some of which achieve superhuman performance! Since Transformers are straightforward to parallelize across elements of the input sequence, they can be trained at much larger scales than recurrent or convolutional neural networks. With the advent of transfer learning for NLP, these large pretrained models can be then “fine-tuned” on a wide variety of tasks and languages, all with modest compute. However, until recently it was not easy to integrate Transformers in practical applications. Each model would typically be associated with a single codebase and API, making it difficult to change between models. To address this need, the team at HuggingFace have developed the open-source Transformers library to make these powerful models accessible to the wider machine learning community. The library provides a unified API that makes it simple to fine-tune these models on downstream tasks, and provides a centralised model Hub for users to share and utilise pretrained models. These developments in fundamental research and open-source software indicate that Transformers are likely to become a cornerstone of NLP. With a hands-on approach, this workshop will provide you with a foundation on how Transformers work, what tasks they are good for, and how to integrate them in your applications. About the Instructors: Lewis Tunstall is a Machine Learning Engineer at Hugging Face, focused on developing open-source tools and making them accessible to the wider community. A former theoretical physicist, he has over 10 years experience translating complex subject matter to lay audiences and has taught machine learning to university students at both the graduate and undergraduate levels. Leandro von Werra is a Machine Learning Engineer at Hugging Face. He has years of industry experience bringing NLP projects to production by working across the whole machine learning stack, and is the creator of a popular Python library that combines Transformers with reinforcement learning.