TiDAR: Hybrid Diffusion + AR Decoding for LLMs скачать в хорошем качестве

TiDAR: Hybrid Diffusion + AR Decoding for LLMs

In this AI Research Roundup episode, Alex discusses the paper: 'TiDAR: Think in Diffusion, Talk in Autoregression' TiDAR tackles the trade-off between diffusion LLMs’ parallel token drafting and autoregressive models’ higher sequence quality. It’s a single-model hybrid that drafts tokens in parallel via diffusion and verifies them autoregressively in one pass using a structured causal–bidirectional attention mask. Each step partitions the sequence into a causal prefix, tokens to verify, and a pre-drafted block; drafts use bidirectional attention (pDiff) while verification uses causal AR probabilities (pAR) with KV cache updates and evictions. Training masks the diffusion block fully, enabling dense diffusion loss alongside next-token prediction for stable joint learning. Paper URL: https://arxiv.org/abs/2511.08923 #AI #MachineLearning #DeepLearning #LLM #DiffusionModels #Autoregressive #Decoding #InferenceEfficiency