Deploying Large Language Models to Mobile Phones with Memory Offloading скачать в хорошем качестве

Deploying Large Language Models to Mobile Phones with Memory Offloading

Carnegie Mellon University 15-821/18-843: Mobile and Pervasive Computing, Fall 2025 Students: Hongyi Jin and Ruihang Lai Mentor: Babu Pillai This project implements LLM weight offloading on mobile phones, thereby enabling mobile deployment of models that cannot fit mobile phone memory. The implementation supports a wide range of Apple devices, including iPhone, iPad, and Mac. Modern LLMs are all based on the transformer architecture, which consist of many identical layers. Therefore, we can divide a model into multiple stages. Before running a stage, we evict the model weights of the previous stage and load the model weights of the current stage. By doing this, the maximum memory requirement is bounded by the largest stage.