Exploiting Similarity in Federated Learning скачать в хорошем качестве

Exploiting Similarity in Federated Learning

Sebastian Stich (CISPA) https://simons.berkeley.edu/talks/seb... Learning from Heterogeneous Sources We provide a brief introduction to local update methods developed for federated optimization and discuss their worst-case complexity. Surprisingly, these methods often perform much better in practice than predicted by theoretical analyses using classical assumptions. Recent years have revealed that their performance can be better described using refined notions that capture the similarity among client objectives. In this talk, we introduce a generic framework based on a distributed proximal point algorithm, which consolidates many of our insights and allows for the adaptation of arbitrary centralized optimization algorithms to the convex federated setting, including accelerated variants. Our theoretical analysis shows that the derived methods enjoy faster convergence when the degree of similarity among clients is high. Based on joint work with Xiaowen Jiang and Anton Rodomanov.