Recognition of Humans and Their Activities Using Video

The recognition of humans and their activities from video sequences is currently a very active area of research because of its applications in video surveillance, design of realistic entertainment systems, multimedia communications, and medical diagnosis. In this lecture, we discuss the use of face and gait signatures for human identification and recognition of human activities from video sequences. We survey existing work and describe some of the more well-known methods in these areas. We also describe our own research and outline future possibilities. In the area of face recognition, we start with the traditional methods for image-based analysis and then describe some of the more recent developments related to the use of video sequences, 3D models, and techniques for representing variations of illumination. We note that the main challenge facing researchers in this area is the development of recognition strategies that are robust to changes due to pose, illumination, disguise, and aging. Gait recognition is a more recent area of research in video understanding, although it has been studied for a long time in psychophysics and kinesiology. The goal for video scientists working in this area is to automatically extract the parameters for representation of human gait. We describe some of the techniques that have been developed for this purpose, most of which are appearance based. We also highlight the challenges involved in dealing with changes in viewpoint and propose methods based on image synthesis, visual hull, and 3D models. In the domain of human activity recognition, we present an extensive survey of various methods that have been developed in different disciplines like artificial intelligence, image processing, pattern recognition, and computer vision. We then outline our method for modeling complex activities using 2D and 3D deformable shape theory. The wide application of automatic human identification and activity recognition methods will require the fusion of different modalities like face and gait, dealing with the problems of pose and illumination variations, and accurate computation of 3D models. The last chapter of this lecture deals with these areas of future research.

Introduction.- Human Recognition Using Face.- Human Recognition Using Gait.- Human Activity Recognition.- Future Research Directions.- Conclusions.

Rama Chellappa received the B.E. (Hons.) degree from the University of Madras, India, in 1975 and the M.E. (Distinction) degree from the Indian Institute of Science, Bangalore, in 1977. He received the M.S.E.E. and Ph.D. degrees in elec trical engineering from Purdue University, West Lafayette, IN, in 1978 and 1981, respectively. Since 1991 he has been a Professor of Electrical Engineering and an Affil iate Professor of Computer Science at the University of Maryland, College Park. He is also affiliated with the Center for Automation Research (Director) and the Institute for Advanced Computer Studies (permanent member). Prior to joining the University of Maryland, he was an Assistant (1981–1986) and Associate Pro fessor (1986–1991) and Director of the Signal and Image Processing Institute (1988–1990) with the University of Southern California, Los Angeles. Over the last 23 years he has published numerous book chapters, peer reviewed journal and conference papers. He has edited a collection of papers on Digital Image Process ing (published by IEEE Computer Society Press), coauthored (with Y.T. Zhou) a research monograph on Artificial Neural Networks for Computer Vision, published by Springer-Verlag, and coedited (with A.K. Jain) a book on Markov Random fields, published by Academic Press. His current research interests are face and gait analysis, 3D modeling from video, automatic target recognition from stationary and moving platforms, surveillance and monitoring, hyper spectral processing, image understanding, and commercial applications of image processing and understanding. Dr. Chellappa has served as an associate editor of the IEEE Transactionson Signal Processing, Pattern Analysis and Machine intelligence, Image Processing, and Neural Networks. He was a coeditor-in-chief of Graphical Models and Image Process ing. He also served as the editor-in-chief of IEEE Transactions on Pattern Analysis and Machine Intelligence during 2001–2004. He has also served as a member of the IEEE Signal Processing Society Board of Governors during 1996–1999 and as its Vice President of Awards and Membership during 2002–2004. He has received several awards, including NSF Presidential Young Investigator Award, an IBM Faculty Development Award, the 1990 Excellence in Teaching Award from the School of Engineering at USC, the 1992 Best Industry Related Paper Award from the International Association of Pattern Recognition (with Q. Zheng), and the 2000 Technical Achievement Award from the IEEE Signal Processing Society. He was elected as a Distinguished Faculty Research Fellow (1996–1998) and as a Distinguished Scholar–Teacher (2003) at the University of Maryland. He is a fellow of the International Association for Pattern Recognition. He has served as a General the Technical Program Chair for Server IEEE international and national conferences and workshops. Amit K. Roy-Chowdhury received the B.E. degree in electrical engineering from Jadavpur University, India, in 1995; the M.E. degree in systems science and automation from the Indian Institute of Science, Bangalore, in 1997; and Ph.D. from the University of Maryland, College Park, in 2002. His Ph.D. thesis was on sta[1]tistical error characterization of 3D modeling from monocular video sequences. His current research interests are in motion and illumination modeling in video sequences, computational models for human activity recognition, and video sensor networks. Since 2004 he has been an Assistant Professor in the Department of Electrical Engineering, University of California, Riverside. In 2003 he was with the Center for Automation Research, University of Maryland, College Park, as a Research Associate, where he worked in projects related to face, gait, and activity recognition. He is the author of a number of papers, book chapters, and magazine THE AUTHORS 173 articles on motion analysis, 3D modeling, and object recognition. He is a reviewer for journals in signal and image processing and computer vision and has served on the Program Committees of major international conferences in these areas. He was a recipient of University of California Regents’ Faculty Fellowship Award in 2004–2005. S. KEVIN ZHOU S. Kevin Zhou is a Research Scientist at Siemens Corporate Research, Princeton, New Jersey. He received his B.E. degree in electronic engineering from the University of Science and Technology of China, Hefei, China, in 1994; M.E. degree in computer engineering from the National University of Singapore in 2000; and Ph.D. in electrical engineering from the University of Maryland at College Park in 2004. He has broad research interests in signal/image/video processing, computer vision, pattern recognition, machine learning, and statistical inference and computing. He has published more than 30 technical papers and book chapters on echocardiography image processing, visual recognition (in particular face recognition under unconstrained conditions, such as video sequences, illuminatio