AA247 - AI is a Poor Team-Player: Stanford's CooperBench Experiment скачать в хорошем качестве

AA247 - AI is a Poor Team-Player: Stanford's CooperBench Experiment

AI agents failed spectacularly at teamwork, performing ~50% worse than one solo agent! This week, we're discussing Stanford’s CooperBench study (a benchmark, testing whether AI agents can collaborate on real coding tasks across Python, TypeScript, Go, and Rust) and why AI-developer coordination collapses, even with a constant chat. Listen or watch as Product Manager Brian Orlando and Enterprise Business Agility Consultant Om Patel dig into the methods and findings of Stanford’s 2026 CooperBench experiment and learn about the three capability gaps that caused these failures: • Expectation Failures (42%): Agents ignored shared plans or misunderstood scope • Commitment Failures (32%): Promised work was never completed • Communication Failures (26%): Silence, spam, or hallucinations The experiment's findings seem to confirm human-refined agile practices. The episode ends with a concrete call to action: stop treating AI as teammates. Use them as solo contributors. And if you must coordinate? Build working agreements, not handoffs. This episode is for anyone navigating the AI hype cycle and wondering if swarms of agents are going to coordinate everyone out of a job! SOURCE CooperBench: Benchmarking AI Agents' Cooperation (Stanford University & SAP Labs US) https://cooperbench.com/ https://cooperbench.com/static/pdfs/m... #Agile #AI #ProductManagement CHAPTERS 00:00:00 - Introduction and Episode Overview 00:01:24 - Welcome to Arguing Agile 00:01:58 - Today's Topic: Stanford's CooperBench Study 00:02:20 - What is CooperBench? 00:03:05 - 1. The Experiment Setup and Baseline 00:04:41 - Human Teams as the Gold Standard 00:06:00 - Expectations 00:06:52 - Experiment Design and Controls 00:07:28 - Real-World Teaming Implications 00:08:19 - The Curse of Coordination Emerges 00:09:52 - Two Agents Performed 50% Worse 00:10:41 - Success Rates Across Different Models 00:11:54 - Questioning Why Medium Difficulty Tasks Hit Hardest? 00:13:14 - Why Medium Tasks Failed Most 00:14:09 - Coordination Overhead Negates Parallelization 00:15:13 - Three Capability Gaps Identified 00:16:33 - Expectation Failures: The Largest Category 00:19:57 - Commitment Failures Explained 00:19:56 - Communication Failures: The Smallest Category 00:20:55 - Why Communication Failures Are Overblamed 00:22:07 - Expectations: A 3000-Word Communication Problem 00:24:54 - Commitment Failures: The Other Unspoken Problem 00:27:09 - True Communication Failures 00:28:50 - Reflecting on the 3-category Split 00:29:29 - The Real Problem: Ignoring Shared Information 00:30:49 - Constant Communication was No Help 00:33:08 - Coordinated, Yet Still Unsuccessful 00:36:13 - Observations & Planning 00:37:02 - Working Agreements 00:39:28 - Learning From Successful Agent Interactions 00:40:16 - Three Emergent Patterns of Coordination 00:46:45 - Don't Call it a Handoff 00:48:22 - Learnable Patterns 00:49:40 - Conclusions of the Research 00:50:55 - Wrap-Up LINKS YouTube:    / @arguingagile   Spotify: https://open.spotify.com/show/362QvYO... Apple: https://podcasts.apple.com/us/podcast... INTRO MUSIC Toronto Is My Beat By Whitewolf (Source: https://ccmixter.org/files/whitewolf2...) CC BY 4.0 DEED (https://creativecommons.org/licenses/...)