DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation скачать в хорошем качестве

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

This research introduces DreamBooth, a method for personalizing text-to-image diffusion models using only a few images of a specific subject. The technique fine-tunes the model to associate a unique identifier with the subject, enabling the generation of novel, photorealistic images of the subject in diverse contexts. A key component is a class-specific prior preservation loss, which prevents the model from "forgetting" how to generate diverse images of the same class as the subject. DreamBooth demonstrates strong performance in subject recontextualization, artistic rendering, and view synthesis. Experiments show it surpasses existing methods in subject and prompt fidelity, preserving key visual features while adhering to text prompts. This approach allows users to create customized images of themselves and their belongings. Nataniel Ruiz Yael Pritch Yuanzhen Li Michael Rubinstein Varun Jampani Kfir Aberman 1 Google Research 2 Boston University arXiv:2208.12242v2 [cs.CV] 15 Mar 2023 Abstract Large text-to-image models achieved a remarkable leap in the evolution of AI, enabling high-quality and diverse synthesis of images from a given text prompt. However, these models lack the ability to mimic the appearance of subjects in a given reference set and synthesize novel rendi- tions of them in different contexts. In this work, we present a new approach for “personalization” of text-to-image dif fusion models. Given as input just a few images of a sub ject, we fine-tune a pretrained text-to-image model such that it learns to bind a unique identifier with that specific sub ject. Once the subject is embedded in the output domain of the model, the unique identifier can be used to synthesize novel photorealistic images of the subject contextualized in different scenes. By leveraging the semantic prior embed ded in the model with a new autogenous class-specific prior preservation loss, our technique enables synthesizing the subject in diverse scenes, poses, views and lighting condi tions that do not appear in the reference images. We ap ply our technique to several previously-unassailable tasks, including subject recontextualization, text-guided view syn thesis, and artistic rendering, all while preserving the sub ject’s key features. We also provide a new dataset and eval uation protocol for this new task of subject-driven genera tion. Project page: https://dreambooth.github.io/