Speaker: Dr. Mitsunori Ogihara When: Friday, Nov 30th, 2007 Time: 2:00pm Where: ECS 243

Abstract:
Using entropy of traffic distributions has been shown to aid a wide variety of network monitoring applications such as anomaly detection, clustering to reveal interesting patterns, and traffic classification. However, realizing this potential benefit in practice requires accurate algorithms that can operate on high-speed links, with low CPU and memory requirements. Estimating the entropy in a streaming model (read-once, limited storage) to enable such fine-grained traffic analysis has been a challenging problem.

This talk presents algorithms for solving two types of network traffic entropy estimation problems. The first is the problem of estimating entropy between one origin and one destination. I present two algorithms for randomly approximating the entropy in a time and space efficient manner, applicable for use on very high speed (greater than OC-48) links. The first algorithm, inspired by the celebrated algorithm of Alon, Matias, and Szegedy for estimating frequency moments, has strong theoretical guarantees on the error and resource usage. The second algorithm utilizes the observation that the efficiency can be substantially enhanced by separating the high-frequency items (or elephants), from the low-frequency items (or mice). Evaluations on real-world traffic traces from different deployment scenarios demonstrate the utility of our approaches.

The second problem is concerned with estimation of the traffic between every origin-destination pair. I will present an algorithm based on the Lp-sketch of Indyk with significant additional innovations. The algorithm is designed to work with high link speeds of up to 10 million packets per second using commodity CPU/memory at a reasonable cost. The algorithm is shown to be very accurate in practice via simulations, using traffic traces collected at a tier-1 ISP backbone link.

Biography:
Mitsunori Ogihara is Professor of Computer Science at the University of Miami and Director for Data Mining in the University's newly created Center for Computational Sciences. He received his Ph.D. degree in Information Sciences from Tokyo Institute of Technology in 1993. Since 1994 he has been a Computer Science faculty member at the University of Rochester, where he received tenure in 1998, became Full Professor in 2001, and served as department chair between 1999 and 2007. He is a recipient of an NSF CAREER Award. He has published two books (one co-authored), 50+ journal articles, and 70+ conference articles. He is on the editorial board of International Journal of Foundations of Computer Science (World Scientific Press) and Theory of Computing Systems (Springer).