How to Train Sparse Large Language Models with Vithu Thangarasa скачать в хорошем качестве

How to Train Sparse Large Language Models with Vithu Thangarasa

In this episode of the Cerebras podcast we explore the latest research in sparse neural networks. We discuss: • What is sparsity and why is it important in training neural networks • The latest progress in applying sparsity to large language models • Our latest paper “SPDF: Sparse Pre-training and Dense Fine-tuning For Large Language Models” https://arxiv.org/abs/2303.10464 • Why Cerebras hardware is uniquely suited to training large sparse models • Future directions in sparsity research Speakers: Vithu Thangarasa (@vithursant19) - Senior ML Research Scientist, Cerebras James Wang (@draecomino) - Senior Product Marketing Manager, Cerebras