OSC Colloquium: George Barbastathis, "AI-enhanced vision: seeing the invisible" скачать в хорошем качестве

OSC Colloquium: George Barbastathis, "AI-enhanced vision: seeing the invisible"

If you point your camera to a scene, and the camera registers nothing—does it mean that nothing was really there? Hardly! The camera pixels measure “raw” light intensity where the encoded information often is much richer than a human observer could tell just by looking at the pixels on a screen. Which algorithms, then, should one apply to decode the raw intensity and reveal the hidden scene? In this seminar, I will describe how to use Deep Neural Networks (DNNs), a form of Machine Learning (ML) algorithm, to perform this decoding. During the training stage of the DNN, physically generated objects are used to produce the encoded raw intensities. From these pairs of objects and raw intensities the DNN learns the association between the scenes and their encoded representations. After training, given a new scene, the DNN decodes it correctly to produce a final reconstructed image that is meaningful to a human observer. With my research group, we applied this approach to three challenging instances of invisibility: transparent objects, also known as “phase objects,” whose raw intensities are highly rippled diffraction patterns; phase objects that are also very dark, i.e. the diffraction patterns are also highly attenuated; and objects hidden behind or surrounded by diffusers, e.g. frosted glass or multiple layers of glass patterned with sharp light-scattering features. It is important to emphasize that in our work ML is not used in the traditional way to interpret the scenes; rather, it is used to form interpretable representations of scenes in situations where traditional ML would be helpless due to physical limitations in the optics. The cooperation of ML with physical models proved to be very powerful in this work and, beyond, is certain to impact many fundamental and applied aspects of physical and life sciences and engineering.