The Agent Paradox - Why Moderna's Most Productive AI Systems Aren't Agents скачать в хорошем качестве

The Agent Paradox - Why Moderna's Most Productive AI Systems Aren't Agents

Eric Ma, who leads Research Data Science in the Data Science and AI group at Moderna, joins Hugo on moving past the hype of autonomous agents to build reliable, high-value workflows. We discuss: Reliable Workflows: Prioritize rigid workflows over dynamic AI agents to ensure reliability and minimize stochasticity in production environments; Permission Mapping: The true challenge in regulated environments is security, specifically mapping permissions across source documents, vector stores, and model weights; Trace Log Risk: LLM execution traces pose a regulatory risk, inadvertently leaking restricted data like trade secrets or personal information; High-Value Data Work: LLMs excel at transforming archived documents and freeform forms into required formats, offloading significant “janitorial” work from scientists; “Non-LLM” First: Solve problems with simpler tools like Python or ML models before LLMs to ensure robustness and eliminate generative AI stochasticity; Contextual Evaluation: Tailor evaluation rigor to consequences; low-stakes tools can be “vibe-checked,” while patient safety outputs demand exhaustive error characterization; Serverless Biotech Backbone: Serverless infrastructure like Modal and reactive notebooks such as Marimo empowers biotech data scientists for rapid deployment without heavy infrastructure overhead. Check out more here: https://hugobowne.substack.com/p/epis... Want to learn more? Join the final cohort of our Building AI Applications course in Q1, 2026 (25% off for listeners): https://maven.com/hugo-stefan/buildin... 00:00 Defining Agents and Workflows 02:04 Challenges in Regulated Environments 04:24 Eric Ma's Role at Moderna, Leading Research Data Science in the Data Science and AI Group 12:37 Document Reformatting and Automation 15:42 Data Security and Permission Mapping 20:05 Choosing the Right Model for Production 20:41 Evaluating Model Changes with Benchmarks 23:10 Vibe-Based Evaluation vs. Formal Testing 27:22 Security and Fine-Tuning in LLMs 28:45 Challenges and Future of Fine-Tuning 34:00 Security Layers and Information Leakage 37:48 Wrap-Up and Final Remarks