Computer Vision and Machine Learning with RGB-D Sensors

The combination of high-resolution visual and depth sensing, supported by machine learning, opens up new opportunities to solve real-world problems in computer vision.

This authoritative text/reference presents an interdisciplinary selection of important, cutting-edge research on RGB-D based computer vision. Divided into four sections, the book opens with a detailed survey of the field, followed by a focused examination of RGB-D based 3D reconstruction, mapping and synthesis. The work continues with a section devoted to novel techniques that employ depth data for object detection, segmentation and tracking, and concludes with examples of accurate human action interpretation aided by depth sensors.

Topics and features: discusses the calibration of color and depth cameras, the reduction of noise on depth maps, and methods for capturing human performance in 3D; reviews a selection of applications which use RGB-D information to reconstruct human figures, evaluate energy consumption, and obtain accurate action classification; presents an innovative approach for 3D object retrieval, and for the reconstruction of gas flow from multiple Kinect cameras; describes an RGB-D computer vision system designed to assist the visually impaired, and another for smart-environment sensing to assist elderly and disabled people; examines the effective features that characterize static hand poses, and introduces a unified framework to enforce both temporal and spatial constraints for hand parsing; proposes a new classifier architecture for real-time hand pose recognition, and a novel hand segmentation and gesture recognition system.

Part I: Surveys

3D Depth Cameras in Vision: Benefits and Limitations of the Hardware
Achuta Kadambi, Ayush Bhandari and Ramesh Raskar

A State-of-the-Art Report on Multiple RGB-D Sensor Research and on Publicly Available RGB-D Datasets
Kai Berger

Part II: Reconstruction, Mapping and Synthesis

Calibration Between Depth and Color Sensors for Commodity Depth Cameras
Cha Zhang and Zhengyou Zhang

Depth Map Denoising via CDT-Based Joint Bilateral Filter
Andreas Koschan and Mongi Abidi

Human Performance Capture Using Multiple Handheld Kinects
Yebin Liu, Genzhi Ye, Yangang Wang, Qionghai Dai and Christian Theobalt

Human Centered 3D Home Applications via Low-Cost RGBD Cameras
Zhenbao Liu, Shuhui Bu and Junwei Han

Matching of 3D Objects Based on 3D Curves
Christian Feinen, Joanna Czajkowska, Marcin Grzegorzek and Longin Jan Latecki

Using Sparse Optical Flow for Two-Phase Gas Flow Capturing with Multiple Kinects
Kai Berger, Marc Kastner, Yannic Schroeder and Stefan Guthe

Part III: Detection, Segmentation and Tracking

RGB-D Sensor-Based Computer Vision Assistive Technology for Visually Impaired Persons
Yingli Tian

RGB-D Human Identification and Tracking in a Smart Environment
Jungong Han and Junwei Han

Part IV: Learning-Based Recognition

Feature Descriptors for Depth-Based Hand Gesture Recognition
Fabio Dominio, Giulio Marin, Mauro Piazza and Pietro Zanuttigh

Hand Parsing and Gesture Recognition with a Commodity Depth Camera
Hui Liang and Junsong Yuan

Learning Fast Hand Pose Recognition
Eyal Krupka, Alon Vinnikov, Ben Klein, Aharon Bar Hillel, Daniel Freedman, Simon Stachniak and Cem Keskin

Realtime Hand-Gesture Recognition Using RGB-D Sensor
Yuan Yao, Fan Zhang and Yun Fu

Dr. Ling Shao is a Senior Lecturer (Associate Professor) in the Department of Electronic and Electrical Engineering at the University of Sheffield, UK. His publications include the Springer title Multimedia Interaction and Intelligent User Interfaces.

Dr. Jungong Han is a Senior Scientist at Civolution Technology, Eindhoven, and a Guest Researcher at the Eindhoven University of Technology, Netherlands.

Dr. Pushmeet Kohli is a Senior Researcher in the Machine Learning and Perception Group at Microsoft Research Cambridge and an Associate in the Psychometrics Centre at the University of Cambridge, UK.

Dr. Zhengyou Zhang, IEEE Fellow and ACM Fellow, is a Principal Researcher and Research Manager of the Multimedia, Interaction, and Communication Group at Microsoft Research Redmond, WA, USA.