IAIFI Colloquium: Feature Geometry guides Model Design in Deep Learning скачать в хорошем качестве

IAIFI Colloquium: Feature Geometry guides Model Design in Deep Learning

Melanie Weber, Assistant Professor of Applied Mathematics and of Computer Science, Harvard University Friday, November 14, 2025, 2:00pm–3:00pm, MIT Kolker Room (26-414) Feature Geometry guides Model Design in Deep Learning The geometry of learned features can provide crucial insights on model design in deep learning. In this talk, we discuss two recent lines of work that reveal how the evolution of learned feature geometry during training both informs and is informed by architecture choices. First, we explore how deep neural networks transform the input data manifold by tracking its evolving geometry through discrete approximations via geometric graphs that encode local similarity structure. Analyzing the graphs’ geometry reveals that as networks train, the models’ nonlinearities drive geometric transformations akin to a discrete Ricci flow. This perspective yields practical insights for early stopping and network depth selection informed by data geometry. The second line of work concerns learning under symmetry, including permutation symmetry in graphs or translation symmetry in images. Group-convolutional architectures can encode such structure as inductive biases, which can enhance model efficiency. However, with increased depth, conventional group convolutions can suffer from instabilities that manifest as loss of feature diversity. A notable example is oversmoothing in graph neural networks. We discuss unitary group convolutions, which provably stabilize feature evolution across layers, enabling the construction of deeper networks that are stable during training. Sign up for IAIFI's mailing list: https://mailman.mit.edu/mailman/listi...