Faculty Interview 2012: Geometric Capture of Human Performances - Hao Li скачать в хорошем качестве

Faculty Interview 2012: Geometric Capture of Human Performances - Hao Li

Faculty Candidate Seminars, Department of Computer Science, Columbia University, New York, 03/2012. Summary. In this century dominated by the Internet and social media, omnipresent information about ourselves and others increasingly influence our actions and social behavior. Soon, our homes will be equipped with 3D sensors that monitor our movements and, since our appearance and movements telegraph our mood, health, and habits, these sensors will also be able to infer our intentions. My research consists of developing novel methods for high quality digitization of human performances using 3D scanning technology with emphasis on recovering highly complex facial expressions. Until recently, the only reliable way to obtain human motion was to track markers that are sparsely attached to the body. Not only does this approach involve a tedious calibration process, but also it fails to recover valuable details between markers. Driven by the development of real-time 3D sensors, I developed a method that reconstructs both geometry and motion in parallel without involving markers. The method builds a highly realistic digital model in motion with sub-millimeter accurate details aggregated from all observations. At the core of this technology lies a so called non-rigid registration algorithm that automatically brings arbitrary pairs of shapes into alignment. Besides being used to study the statistics of human shapes (U.S. Department of Defense), this technique lead to the development of CloneCam, the next generation facial performance capture system in visual effects (Lucasfilm/ILM). To enable interactive applications and deployment in everyday environments, I extended the system to work under severe lighting conditions and be resistant to unforeseeable occlusions by scene objects and other users. The marker-free system effectively maps low-quality and noisy input data obtained from Microsoft's Kinect sensor to realistic facial expressions using a probabilistic animation prior. My work has generated several collaborations outside the discipline of computer graphics, namely in biomechanics. My reconstruction techniques are licensed by a leading medical imaging company (Elekta/C-RAD) for radiation therapy; the software has been deployed in over 50 hospitals worldwide. Along the lines of biomedical imaging, I recently collaborated with Washington University on a novel framework for fiducial free measurement of cardiac surface mechanics (stress and strain) using an ultra-fast 3D imaging system. The key contribution of this work includes the description of precise 3D cardiac motions at unparalleled resolution and frame rates (200,000 tracked samples at 667 Hz)—immediately impacting the studies between whole-heart surface mechanics, cardiac electrophysiology, and cardiac diseases. The computation of a consistent parameterization of heart motions will enable an important breakthrough in cardiology, namely the simultaneous capture of heart excitation and contraction. Like video cameras, 3D sensors will become an integral part of every computer, cellphone, car, living room, and airport, streaming and analyzing terabytes of data about us and our friends every second over the Internet. The future lies in providing mechanisms to collect massive data sets of detailed dynamic geometries and to learn how to use them to improve our lifestyles.