Visual Foundation Model Flywheel скачать в хорошем качестве

Visual Foundation Model Flywheel

Abstract: Current Visual Foundation Model (VFM) development typically follows a linear pipeline: data curation, pre-training/modeling, and deployment to downstream use cases. While this streamlined approach has enabled remarkable progress through scale, it faces significant challenges due to increasingly complex use cases and inherent data scarcity. In this talk, I will introduce the VFM Flywheel, a new paradigm designed to address these challenges and foster the next generation of VFMs. Specifically, I propose establishing critical feedback loops connecting three essential stages: 1) domain-specific insights from downstream use cases should directly inform pre-training and model development, 2) pre-training should more effectively leverage existing data, including unlabeled data, 3) downstream use cases should directly guide the data curation practices for pre-training. I will conclude by highlighting future directions and emerging opportunities enabled by the VFM Flywheel. Ultimately, this paradigm offers a promising path toward developing more capable, efficient, and safer visual intelligence systems. Bio: Jason Ren is a research scientist at Apple. He received his Ph.D. from the University of Illinois Urbana-Champaign (UIUC), advised by Alex Schwing and Shenlong Wang. His research interests lie at the intersection of Computer Vision and Machine Learning, with a focus on generative modeling, efficient ML, and 3D vision. During his PhD, he interned at NVIDIA Research, Facebook AI Research, Adobe Research, and Apple. He is a recipient of the Yee Fellowship and the Yunni & Maxine Pao Fellowship. For more information, please visit https://jason718.github.io