Emergent Analogical Reasoning in Transformers скачать в хорошем качестве

Emergent Analogical Reasoning in Transformers

This study investigates the emergence of analogical reasoning in Transformers by formalizing analogy as a functor mapping between categories within a synthetic task environment. The researchers observe that unlike compositional reasoning, which improves consistently with model scale, analogical reasoning appears as a distinct third stage of learning that is highly sensitive to data structure and optimization settings. Mechanistically, this capability relies on the geometric alignment of entity embeddings across different domains, a process characterized by a significant decrease in Dirichlet Energy and the application of vector arithmetic to transfer relational structures. These findings extend to pretrained large language models, where similar patterns of structural alignment evolve across network layers during in-context learning, moving analogy from an abstract cognitive concept to a measurable phenomenon in neural networks. https://arxiv.org/pdf/2602.01992