Abstract:
In this talk, I will present two new techniques we have developed recently: video background separation, and video super-resolution. We aim at applications for mobile and home broadband Internet users, and believe that these techniques have a great potential as enabling tools for bandwidth reduction, privacy protection, video quality enhancement, personalized video content editing and hallucination, etc.

For video background separation, We take the following approaches to provide computationally affordable solutions. First, we assume that the input video sequence is composed of two major motion layers which correspond to the foreground and the background, respectively. Second, we strive to compute sparse motion layers first, using our joint spatio-temporal linear regression method on sparse image features such as edge and corner points extracted from each of the video frames. This method aims to dramatically reduce the computational cost, and to generate more reliable and temporally smoother motion layers. Third, once the two sparse motion layers have been identified, we create the corresponding dense motion layers by using the Markov Random Field (MRF) model. The MRF model assigns the rest of the pixels to either of the motion layers by considering both the color attributes and the spatial relations between each pixel and its surrounding edge/corner points.

For video super-resolution, we adopt the learning-based approach to transform the low-resolution input video into a high-resolution one using a training sample dictionary. We propose to construct a non-linear regression function to model the relationship between low-resolution and high-resolution image patches using the Gaussian process method. Once the regression function is obtained, we can construct a super-resolution version of the low-resolution video in near real-time speed. Our experimental results have revealed that we can simply extend the resolution of the input video into 3 to 4 times without remarkable archifects.

Bio:
YIHONG GONG received his B.S., M.S., and Ph.D. degrees in Electronic Engineering from the University of Tokyo in 1987, 1989, and 1992, respectively. He then joined the Nanyang Technological University of Singapore, where he worked as an assistant professor in the School of Electrical and Electronic Engineering for four years. From 1996 to 1998, he worked for the Robotics Institute, Carnegie Mellon University as a project scientist. He was a principal investigator for both the Informedia Digital Video Library project and the Experience-On-Demand project funded in multi-million dollars by NSF, DARPA, NASA, and other government agencies. In 1999, he joined NEC Laboratories America, and has been leading the Multimedia Processing group since then. In 2006, he became the department head to lead the entire Cupertino branch of the labs. His research interests include multimedia content analysis, and machine learning applications. The major research achievements from his group include news video summarization, sports highlight detection, data clustering, and SmartCatch video surveillance that led to a successful spin-off.

He is among the first group of researchers in the world initiating research studies on content-based image retrieval, sports video highlight detection, and text/video content summarization. Among his publications, he has more than 10 papers that have received more than 50 citations by peer researchers around the world. His paper on soccer highlight detection has become a classical paper in this subject area.