Machine Learning with Scarce Annotations: Applications in Speech Processing, Bioacoustics, and ... скачать в хорошем качестве

Machine Learning with Scarce Annotations: Applications in Speech Processing, Bioacoustics, and ...

by Ricard Marxer, University of Toulon, France Summary: This talk will provide an overview of our research tackling the challenge of working with minimal or no human annotations. I will begin by discussing our initial work on computational models of music perception, which led to my interest in textless speech processing. I will demonstrate how self-supervised learning and weak supervision techniques have been pivotal in addressing questions related to handwritten and acoustic unit discovery, modeling intelligibility at the microscopic level, and developing spoken language models. Next, I will present our studies on bioacoustics, where we train deep learning models on audio recordings with limited annotations. We utilize the spatialization of multi-channel data and unsupervised clustering to significantly reduce the need for human annotation. This approach has enabled us to gain insights into hard-to-study species like sperm whales and to characterize the long-term evolution of the fin whale song. Finally, I will cover our latest work in computer vision, specifically in the context of ocean exploration and visual relocalization of underwater unmanned vehicles. I will highlight the challenges we face in tasks such as pose estimation and color restoration, where the lack of ground truth references and the unique properties of light transport in underwater environments make these problems particularly intriguing. Throughout the talk, I will illustrate how our use of advanced machine learning techniques and innovative approaches has allowed us to make significant strides in these diverse fields, despite the scarcity of human annotations. Bio: Ricard Marxer is a full professor at the University of Toulon, a researcher at the Laboratoire d'Informatique et Systèmes (LIS) CNRS UMR 7020, and the head of the DYNamics of Information (DYNI) research team. He previously worked as a postdoctoral researcher with the Speech and Hearing Group at the University of Sheffield and obtained his PhD from the Music Technology Group at Universitat Pompeu Fabra. His research focuses on the development and application of machine learning techniques to study human behavior and environmental interactions. A major area of interest is the perception of sound, music, and speech, with the long-term goal of understanding language development. Marxer employs advanced deep learning and reinforcement learning techniques as computational models to mimic and explain human behavior. These methods also analyze large datasets, ranging from bioacoustic recordings to underwater imaging. A common theme in his research is addressing the scarcity of human annotations by utilizing self-supervised approaches and leveraging domain expertise. Marxer is a co-founder and steering committee member of the Vocal Interactivity in and between Humans, Animals, and Robots (VIHAR) community. He is also the founder and current director of the Erasmus Mundus Master’s in Marine and Maritime Intelligent Robotics (MIR). Find out more information related to our research at the LIVIA website: https://liviamtl.ca/