NEON: Zero-Shot Compression Agent for DenseNetworks - Dr. Gilad Katz​, Ben Gurion University скачать в хорошем качестве

NEON: Zero-Shot Compression Agent for DenseNetworks - Dr. Gilad Katz​, Ben Gurion University

This lecture was part of the AutoML conference, organized by the MDLI community. Link: https://bit.ly/AutoMLConf Network pruning is often used to reduce the inference cost of large models and enable neural architectures to run on end-devices such as mobile phones. We present NEON, a novel iterative pruning approach using deep reinforcement learning(DRL). While most reinforcement learning-based pruning solutions only analyze the one network they aim to prune, NEONtrains itself on a large set of randomly-generated architectures, and is, therefore, less prone to overfitting. To avoid the long running times often required to train DRL models for each new dataset, we train NEON offline on multiple datasets and then apply it without additional training (i.e., zero-shot) on others. Additionally, we propose a novel reward function that enables users to clearly define their pruning/performance trade-off preferences. Our evaluation, conducted on a set of 28 diverse datasets, shows that NEON significantly outperforms recent top-performing solutions in the pruning of fully-connected networks.