How do LLMs add numbers? скачать в хорошем качестве

How do LLMs add numbers?

Largely based on Anthropic's research "On the biology of a Large Language Model", we dive into the process of how LLMs handle addition, including main takeaways from self-attention and the replacement model - cross-layer transcoder for better interpretability of neural networks. In the self-attention matrix multiplication animation, we use 4-dimension toy embeddings for illustration purposes. Featured in this video: diffy chat: https://diffy.chat/ On the Biology of a Large Language Model: https://transformer-circuits.pub/2025... Transformer explainer: https://poloclub.github.io/transforme... manim animations (Thanks to Grant Sanderson for this amazing animation library from 3b1b!) : https://github.com/leannchen86/how-do... 00:38 what this video is about 00:56 tokenization 01:35 self-attention 03:46 feed-forward network's polysemantic nature (mlp) 04:21 cross-layer transcoder (replacement model) 04:31 sparse features in clt 5:41 how are sparse features activated? 6:22 do LLMs tell us what they "think"? 6:51 limitations of the replacement model (clt) 7:10 tool-calling is here to stay