DEEPPLANNING: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints скачать в хорошем качестве

DEEPPLANNING: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints

DESCRIBE DEEPPLANNING, A NEW BENCHMARK DESIGNED TO ASSESS THE LONG-TERM PLANNING ABILITY OF LARGE-SCALE LANGUAGE MODEL (LLM) AGENTS. Unlike traditional assessment methods that focused on the use of single tools, this system tests the agent's competencies through complex and realistic scenarios such as multi-day travel planning and multi-shopping. Agents are tasked with proactively collecting information and coordinating details while optimizing global constraints such as limited budget and time. Studies have shown that even state-of-the-art models have great difficulty maintaining the consistency of their plans in situations of complex constraints. In particular, it emphasizes that models provide more efficient solutions when leveraging reasoning capabilities, but still do not fully overcome errors that occur in the long-term execution process. This benchmark aims to provide sophisticated metrics and data sets for the development of more reliable agents. https://arxiv.org/pdf/2601.18137