One-Minute Video Generation with Test-Time Training (Apr 2025) скачать в хорошем качестве

One-Minute Video Generation with Test-Time Training (Apr 2025)

Title: One-Minute Video Generation with Test-Time Training (Apr 2025) Link: http://arxiv.org/abs/2504.05298v1 Date: April 2025 Summary: This paper introduces Test-Time Training (TTT) layers to enable Transformer models to generate coherent one-minute videos from text storyboards. They curated a dataset of Tom and Jerry cartoons as proof of concept and found that TTT layers outperform baselines like Mamba, Gated DeltaNet, and sliding-window attention layers. Key Topics: Video Generation Test-Time Training Diffusion Transformer Long Context Modeling RNN Transformer Tom and Jerry dataset Chapters: 00:00 - Introduction 00:16 - Current Video Generation Limitations 00:41 - One Minute Video Generation 01:13 - Test Time Training Layers 01:39 - Tom and Jerry Cartoons 02:16 - TTT Layers Explained 03:00 - Dynamic Hidden State 03:38 - Update Rule 04:34 - Test Time Training 04:54 - Self-Supervised Task 05:37 - Learnable Parameters 06:15 - Full Video Generation Model 06:46 - Learnable Gates 07:28 - Diffusion Models 08:01 - Bi-directional TTT layers 08:09 - Generating A Minute-Long Video 09:03 - Training 09:59 - Tom and Jerry Dataset 10:41 - Parallelize the Training 11:18 - Performance 11:30 - Movie Gen Benchmark 12:05 - TTT MLP Stack Up 12:36 - Temporal Consistency 13:11 - TTT MLP Isn't Perfect 13:51 - Conclusion 14:34 - Further Improvements