Intro to Triton: A MyTorch Sidequest! скачать в хорошем качестве

Intro to Triton: A MyTorch Sidequest!

In our quest to build a deep learning framework, we have hit a roadblock! Training is too slow and needs too much memory for large models! What we want to do is reproduce GPT2 (124M) so its time to pull our fused kernels. We could do them in CUDA but Triton is so much easier and accessible so that's what we will do! Today we set the stage to understand some of the fundamentals of GPU programming and what to expect with Triton! Timestamps: 00:00:00 - Introduction 00:01:00 - GPUs and Embarrassingly Parallel Problems 00:02:30 - GPUs vs CPUs 00:04:20 - Vector Sum CUDA Code 00:05:38 - Warps 00:07:00 - All Data is a Vector! 00:08:30 - Compute Bound Problems 00:09:30 - Memory Bound Problems 00:10:15 - Memory Coalescing 00:12:20 - Fused Kernels 00:15:25 - Why Triton? 00:15:45 - CUDA vs Triton Example 00:23:45 - Our Approach 00:24:45 - Learning Plan Socials! X   / data_adventurer   Instagram   / nixielights   Linkedin   / priyammaz   Discord   / discord   🚀 Github: https://github.com/priyammaz 🌐 Website: https://www.priyammazumdar.com/