How to Fully Test AI Agents with IBM watsonx Orchestrate ADK — Hands-On Evaluation Framework скачать в хорошем качестве

How to Fully Test AI Agents with IBM watsonx Orchestrate ADK — Hands-On Evaluation Framework

Curious how to test AI agents end-to-end using IBM’s watsonx Orchestrate ADK? In this 35 min hands-on walk-through, I’ll show you how to: Define Stories (i.e. user scenarios) for your agents Automatically generate synthetic test cases from those stories Set up a sample backend + a local Python tool for lookups Import & configure your Agent, connect Tools & REST APIs Run the ADK’s built-in Evaluation Framework and inspect results Useful Links & Resources *Read the full blog post: Testing AI Agents with the watsonx Orchestrate ADK – Evaluation Framework A Hands-on Example*: https://suedbroecker.net/2025/09/21/t... IBM’s ADK Overview & Evaluation Framework docs: https://developer.watson-orchestrate.... Langfuse observability integration with watsonx Orchestrate ADK : https://github.com/langfuse/langfuse Galaxium Travels Evaluation Example https://github.com/thomassuedbroecker... Table of Contents: 00:00 - Intro & who am I 00:25 - Date information 00:55 - What is the Evaluation Framework 02:42 - Overview of the example application 03:26 - Folder structure of the example 03:53 - Starting point of the example 04:23 - Link to setup example 04:38 - ADK version used in this video 05:01 - Overview of all steps 05:53 - Main component overview 06:40 - Creating a tool (part 1) 10:39 - Testing the tool locally 11:57 - Saving requirements.txt 12:44 - Importing the tool! 15:00 - Adding a connection 17:50 - Importing an agent 19:50 - Generating synthetic test cases! 21:01 - Building the stories file 25:47 - Running the evaluation! 28:43 - Inspecting test results 30:06 - Langfuse observability 32:18 - Summary of key findings 33:03 - Outro Quick Checklist: Testing AI Agents with watsonx Orchestrate ADK ✅ Define clear user stories that include context, goal, and expected interaction ✅ Build or import tools & REST APIs your agent needs ✅ Generate synthetic test cases automatically from your stories ✅ Set up backend + local environment to test tools locally ✅ Use Evaluation Framework to simulate “trajectories” → compare actual behavior to ground-truth data ✅ Check for correct ordering of tool calls + correct input parameters ✅ Evaluate summary quality: final answers should be relevant, concise, accurate developer.watson-orchestrate.ibm.com ✅ Use “ground truth datasets” with expected conversations, tool-calls, goal dependencies etc. ✅ Inspect evaluation results, logs, detect where errors occur → iterate to improve #AIAgents #watsonxOrchestrate #IBM #ADK #AgentTesting #SyntheticTestCases #EvaluationFramework #Langfuse #ToolIntegration #MachineLearningOps #AgentDevelopment #DeveloperTutorial #PythonTools #RESTAPIs #Observability #AgentOrchestration