Kimi K2.5: Visual Agentic Intelligence (Paper review) скачать в хорошем качестве

Kimi K2.5: Visual Agentic Intelligence (Paper review)

Kimi K2.5 from Moonshot AI just dropped — 1.04 trillion parameters, 32 billion active per token, 128K context, open weights. It uses essentially the same architecture as DeepSeek V3 (same MoE, same MLA, same SwiGLU), yet outperforms models that cost 10x more. How? In this video I break down every piece of the system: 0:00 — Hook: Same Architecture, Different Tricks 1:00 — DeepSeek V3 vs Kimi K2 Architecture Comparison 2:32 — MoE Deep Dive: 384 Experts, 48x Sparsity 4:12 — Multi-head Latent Attention + MuonClip Optimizer 6:06 — MoonViT-3D: Native Resolution Vision Encoder 7:42 — Early Fusion vs Late Fusion (the result that broke assumptions) 9:36 — Zero-Vision SFT: Text-only training improves vision (???) 11:22 — Joint Multimodal RL: Vision RL improves text benchmarks 12:50 — TOGGLE: 30% fewer tokens, same performance 14:20 — Agent Swarm + PARL: 4.5x faster parallel agents 16:50 — Benchmarks vs GPT-4.5, Claude Opus, Gemini 2.5 18:20 — Cost & Deployment: 76% cheaper than Claude Opus 19:58 — Conclusion Key findings: MuonClip delivered zero loss spikes across 15.5T tokens Early fusion (10% vision from day 1) crushed late fusion on every benchmark Text-only fine-tuning improved vision performance; adding vision examples made it worse Vision RL training improved pure text benchmarks (MMLU-Pro +1.7%, GPQA +2.1%) Agent Swarm: up to 100 sub-agents, 1500 tool calls per task, 4.5x speedup TOGGLE: 25-30% token reduction with negligible quality loss Paper: https://arxiv.org/abs/2507.xxxxx Model weights: https://huggingface.co/moonshotai/Kim... Moonshot API: https://platform.moonshot.ai #kimik2 #KimiK25 #MoonshotAI #llm #ai #deeplearning #MixtureOfExperts #MoE #DeepSeekV3 #opensource #TransformerArchitecture #machinelearning #agi #multimodalai #reinforcementlearning #aiagents #MuonClip #NativeVision #mla