VL-JEPA Explained: Why Meta is Stopping Token Generation скачать в хорошем качестве

VL-JEPA Explained: Why Meta is Stopping Token Generation

In this video, we break down VL-JEPA (Vision-Language Joint Embedding Predictive Architecture), a new model from Meta FAIR that challenges the standard way Vision-Language Models (VLMs) operate. Instead of generating text token-by-token like a standard LLM, VL-JEPA predicts continuous embeddings in an abstract latent space. This approach decouples visual understanding from text generation, resulting in a model that is faster, more efficient, and capable of real-time video understanding. Key Topics Covered: • The Core Concept: Moving from autoregressive token generation (turn-by-turn directions) to latent embedding prediction (GPS coordinates). • The Architecture: A look at the 4 main components—the X-Encoder (Vision), the Predictor (based on Llama-3), the Y-Encoder (Text Target), and the lightweight Y-Decoder. • The Training Process: How the model uses InfoNCE loss to align embeddings and the two-stage training pipeline (Pretraining + Supervised Finetuning). • Performance: How VL-JEPA achieves comparable results to standard VLMs with 50% fewer trainable parameters and reduces decoding operations by ~2.85x using "Selective Decoding". Paper Referenced: "VL-JEPA: Joint Embedding Predictive Architecture for Vision-language" Authors: Delong Chen, Mustafa Shukor, et al. (Meta FAIR, HKUST, Sorbonne Université, NYU). https://arxiv.org/pdf/2512.10942