"Not Me" Podcast Episode #10: The End of Context Windows скачать в хорошем качестве

"Not Me" Podcast Episode #10: The End of Context Windows

Hey, it’s Vlad. Everyone’s obsessed with building bigger AI brains. More parameters. Longer context windows. Better reasoning. But here’s what MIT just figured out: you don’t need a bigger brain. You need a smarter one. Researchers Alex Zhang, Tim Kraska, and Omar Khattab from MIT CSAIL dropped a paper that’s getting way less attention than it deserves. It’s called Recursive Language Models, or RLMs. And it flips everything we know about AI limitations on its head. The Problem Nobody Solved Let’s start with the ugly truth. Every AI model you use has a memory problem. GPT-5? It chokes after 272,000 tokens. Claude? Same ballpark. Even with these “massive” context windows, the models get dumber the more you feed them. It’s called context rot. Think about it. You paste a 50-page document and ask a simple question. The model starts hallucinating. Missing obvious facts. Getting confused. Why? Because cramming everything into the context window is like forcing someone to read a 10,000-page encyclopedia cover-to-cover before answering your question. It’s absurd. We’ve been treating AI like a student with a strict word limit on their exam. No wonder it struggles. The MIT Breakthrough Here’s where it gets interesting. MIT asked a different question: What if the AI didn’t have to read everything at once? What if it could treat the prompt as an external environment? A workspace. A filing cabinet can be explored strategically. That’s RLM. Instead of feeding GPT-5 your entire 10-million-token corpus directly, you store it as a Python variable. The model never sees it in the prompt. Instead, it writes code to peek at specific sections. Grep through for patterns. Chunk it up. And here’s the kicker: it can spawn sub-models to investigate specific parts. It’s recursion. The model calls itself. Over and over. Each layer handles a smaller, more manageable piece. Like hiring a research team instead of forcing one person to do everything. Sound familiar? It’s the same philosophy behind sub-agents I wrote about recently. Stop making one AI do everything. Orchestrate. The Numbers Don’t Lie Let’s talk results. On the OOLONG benchmark, which is designed to torture AI with long context tasks, here’s what happened: Base GPT-5? Crashed and burned. Near zero performance. GPT-5 with RLM? 58% F1 score. From essentially nothing to majority correct. That’s not an improvement. That’s a resurrection. On the BrowseComp-Plus benchmark, RLM handled over 10 million tokens. Two orders of magnitude beyond the context window. And it did it for roughly the same cost as running the base model. Sometimes cheaper. 91.33% accuracy on a task where the base model literally couldn’t fit the input. This isn’t incremental progress. This is a paradigm shift. Why Programmatic Decomposition Beats Everything You might ask: Why not just summarize the context? Compress it? They tried that. It’s called context compaction. Here’s the problem. Every time you summarize, you lose information. It’s entropy. Irreversible. Summarization agents on the same benchmarks? 70% at best. Often worse. RLM doesn’t summarize. It delegates. Big difference. The model actively decides what to look at. Uses regex filters. Keyword searches. Strategic sampling. It behaves less like a student cramming for an exam and more like a senior researcher with a team of assistants. And because each sub-call runs with a fresh context window, there’s no pollution. No context rot. Each recursive agent stays sharp. What Most People Overlook Here’s the thing that’s flying under the radar. This approach is model-agnostic. RLM works with GPT-5. With Qwen. With Claude. Open-source, closed-source, doesn’t matter. It’s an inference strategy, not an architecture change. You don’t need to retrain anything. And the cost structure is fascinating. Using GPT-5-mini for recursive calls while GPT-5 handles the final synthesis? Cheaper than running GPT-5 on truncated input. Better results. Lower price. That’s the arbitrage nobody’s talking about. The Bitter Lesson, Again Alex Zhang called this a “bitter-lesson-pilled approach.” If you don’t know Rich Sutton’s Bitter Lesson, it’s simple: general methods that leverage computation beat specialized hand-engineered solutions. Every time. RLM fits perfectly. Instead of designing clever compression schemes or specialized architectures, you give the model tools and let it figure out the strategy. The model learns to peek first. Scan for relevant sections. Delegate the hard parts. Build up answers iteratively. No human had to specify these behaviors. They emerge naturally when you give the model the right environment. That’s the meta-lesson here. Stop constraining AI. Start enabling it. Practical Implications So what does this mean for you? If you’re building with AI, pay attention. Long-horizon agents, the ones that need to process weeks or ...