SANE2022 | Scott Wisdom - Unsupervised Sound Separation: A New Paradigm for Audio Processing скачать в хорошем качестве

SANE2022 | Scott Wisdom - Unsupervised Sound Separation: A New Paradigm for Audio Processing

Scott Wisdom, senior research scientist at Google in Cambridge, MA, presents his work on unsupervised sound separation at the SANE 2022 workshop in Kendall Square, October 6, 2022. More info on the SANE workshop series: http://www.saneworkshop.org/ Abstract: Historically, sound separation models have been trained using synthetic mixtures of isolated reference signals. This supervised approach generally works well when matched isolated data is abundant, but for many domains such isolated data is not available, and it is difficult to simulate real conditions. I will describe our recent breakthrough in unsupervised sound separation, mixture invariant training (MixIT), which allows neural networks to learn to separate audio mixtures into their constituent sources without requiring isolated reference signals. Unsupervised sound separation is a potent new paradigm for audio processing, enabling a number of new directions. These include scaling up training data for universal sound separation, improved bird species classification by separating birdsong, adapting separation models to real-world meeting data, and a state-of-the-art audio-visual on-screen separation model called AudioScope, which can isolate the sounds of visible objects in a video regardless of their class. I will discuss our experiments using MixIT for these directions and describe several exciting avenues for future research.