How to build with the NEW gpt-realtime + MCP, Tools, Image Input скачать в хорошем качестве

How to build with the NEW gpt-realtime + MCP, Tools, Image Input

Build a talking voice agent with the NEW OpenAI Realtime API (gpt‑realtime) and the OpenAI Agents SDK in TypeScript. We wire voice, tool calls, image input, and a remote MCP server (Home Assistant) using production‑safe ephemeral keys—end to end. Demo source code: https://github.com/iamarcel/gpt-realt... If this helps you ship, tap like and drop what you built in the comments (voice agents, home automations, dashboards, whatever). Subscribing now genuinely helps the channel grow. Moo. What you’ll learn Realtime session setup in TypeScript: Agent + Session + connect() Secure auth: server‑generated ephemeral client secrets (never ship your API key) History and messaging: audio, text, and image input; handling transcripts vs text Tools two ways: frontend tools (execute in browser) vs remote MCP tools (server‑side) Optional user approvals for tool calls (UX pattern) Practical pitfalls: image messages in history, local vs remote MCP, keeping secrets secret Commands/prereqs Install: pnpm add openai @openai/agents-realtime Server env: set OPENAI_API_KEY (server only), create per‑session client secret Stack: TypeScript + your preferred React setup (I show TanStack Start) Chapters 0:00 Demo 1:02 Project Overview 1:34 Core API Setup 4:33 Secure API Keys 7:34 Session Connection 9:23 Agent Instructions 11:02 Chat History 15:22 Client-Side Tools 18:54 Server-Side Tools (MCP) 23:08 Combined Tooling 24:14 Text & Image Input 27:38 Conclusion