Inside the mind of an LLM | Emanuele Fabbiani скачать в хорошем качестве

Inside the mind of an LLM | Emanuele Fabbiani

In 2024, significant advances were made in understanding large language models (LLMs). Three key discoveries stand out: 1. English as Internal Representation: Llama 2 LLMs consistently use English as their internal language, regardless of the input or output language. 2. Monosemantic Features: Claude 3 Sonnet and GPT-4's internal representations have been condensed into monosemantic features. This allows for a clear understanding of which parts of the model relate to specific topics and enables adjustments in their relative importance. 3. LLMs Memorize Unusual Data: Recent research reveals why LLMs memorize outliers, such as names and personal information. This phenomenon explains instances where GPT-4 has repeated personal information when prompted to repeat the same word indefinitely. This talk will discuss how such discoveries were performed, and their profound implications for privacy and security in LLM applications. Emanuele Fabbiani presented this talk in Codemotion Milan 2024 ___________ 🌟 Discover all Codemotion community activities: https://community-en.codemotion.it 🦸‍♂️ Become #CodemotionContributor: https://community-en.codemotion.it/co... 🧡 Join our community: https://community-en.codemotion.it/co... 🫶 Become #CodemotionPartner: https://community-en.codemotion.it/pa... #CodemotionConference #CodemotionMilan #CodemotionCommunity #Codemotion #TechConference