ERNIE 5.0: A Trillion-Parameter Unified Multimodal Foundation Model скачать в хорошем качестве

ERNIE 5.0: A Trillion-Parameter Unified Multimodal Foundation Model

The technical report details the design and performance of ERNIE 5.0, an artificial intelligence model that integrates text, images, video, and audio. Based on the ultra-houst Mixture-of-Experts (MoE) structure, the model simultaneously understands and generates multi-modal data, and adopts a flexible architecture that supports elastic depth and width for efficient learning. In particular, we introduced next-generation frame and scale prediction technology for visual data and hierarchical codec prediction method for audio to achieve optimal performance for each media characteristic. In addition, we applied sophisticated optimization techniques such as *non-biased replay buffer (U-RB)* and adaptive hint-based learning to improve inefficiencies that occur in the reinforcement learning (RL) process. As a result, ERNIE 5.0 has demonstrated outstanding competitiveness that competes with or surpasses existing powerful models in a wide range of benchmarks, including language reasoning, knowledge acquisition, and visual and sound generation. All these technological innovations are summarized as a result of maximizing the operational efficiency of large models through distributed infrastructure and low-precision training technology. https://arxiv.org/pdf/2602.04705