Interaction Design for 3D User Interfaces The World of Modern Input Devices for Research, Applications, and Game Development

In this new era of computing, where the iPhone, iPad, Xbox Kinect, and similar devices have changed the way to interact with computers, many questions have risen about how modern input devices can be used for a more intuitive user interaction. Interaction Design for 3D User Interfaces: The World of Modern Input Devices for Research, Applications, and Game Development addresses this paradigm shift by looking at user interfaces from an input perspective. The book is divided into four parts: Theory of input devices and user interfaces, with an emphasis on multi-touch interaction Advanced topics on reducing noise on input devices using Kalman Filters A collection of hands-on approaches that allows the reader to gain experience with some devices A case study examining speech as input Most of the chapters contain exercises that provide practical experience to enhance knowledge of the material in the related chapter. With its hands-on approach and the affordability of the required hardware, this book is an excellent flexible resource for both the novice and the expert in 3D user input device development. Researchers and practitioners will gain a much deeper understanding about user input devices and user interfaces. Game developers and software designers will find new techniques to improve their products by adding intuitive user interaction mechanisms to their games and applications. In addition to the resources provided in the book, its companion website, http://3DInputBook.com, provides additional resources, which include: additional exercises and project ideas, additional chapters, source code, and class instructors’ resources. The additional resources are provided to keep helping you with new research and new technology as it becomes available to help you stay up to date.

THEORYIntroductionThe VisionHuman–Computer InteractionDefinitionsFurther ReadingInput InterfacesIntroductionInput TechnologiesUser Input InterfacesInput DevicesInput RecognitionVirtual DevicesInput TaxonomiesFurther ReadingOutput Interfaces and Displays3D Output InterfacesDisplaysFurther ReadingComputer GraphicsComputer GraphicsFurther Reading3D InteractionIntroduction3D ManipulationFurther Reading3D Navigation3D TravelWayfinding3D Navigation: User StudiesFurther ReadingDescriptive and Predictive ModelsIntroductionPredictive ModelsDescriptive ModelsFurther ReadingMulti-TouchIntroductionHardwareMulti-Touch and Its ApplicationsFigures of Large TabletTop DisplaysFurther ReadingMulti-Touch for Stereoscopic DisplaysUnderstanding 3D TouchTouching ParallaxesMulti-Touch Above the TabletopInteraction with Virtual ShadowsPerceptual Illusions for 3D Touch InteractionPen and Multi-Touch Modeling and RecognitionIntroductionThe Dollar FamilyProton++ and MoreFETOUCHUsing Multi-Touch with PetriNetsBackgroundPeNTa: Petri NetsFurther ReadingEye Gaze Tracking as Input in Human–Computer InteractionPrinciple of OperationPost-Processing of POG Data: Fixation IdentificationEmerging Uses of EGT in HCI: Affective SensingFurther ReadingBrain–Computer Interfaces: Considerations for the Next Frontier in Interactive Graphics and GamesFrances Lucretia Van ScoyIntroductionNeuroscience ResearchImplications of EEG and fMRI-Based Research for the Brain–Computer InterfaceNeuroheadsetsA Simple Approach to Recognizing Specific Brain Activities Using Low-End Neuroheadsets and Simple Clustering TechniquesEvidence of Feasibility of Using EEG Data to Recognize Active Brain RegionsConclusionFor Further ReadingADVANCED TOPICSMath for 3D InputSteven P. Landers and David RiekstsIntroductionAxis ConventionsVectorsMatricesAxis Angle RotationsTwo Vector OrientationCalibration of Three Axis SensorsSmoothingFurther ReadingIntroduction to Digital Signal ProcessingIntroductionWhat Is a Signal?Classification of SignalsApplications of Digital Signal ProcessingNoiseSignal Energy and PowerMathematical Representation of Elementary SignalsSampling TheoremNyquist–Shannon TheoremAliasingQuantizationFourier AnalysisFast Fourier Transformz-TransformConvolutionFurther ReadingThree Dimensional RotationsIntroductionThree Dimensional RotationCoordinate SystemsEuler AnglesQuaternionsFurther ReadingMEMS Inertial Sensors and Magnetic SensorsIntroductionInertial SensorsMEMS Inertial Sensor ErrorsMagnetometersMEMS Magnetometer ErrorsFurther ReadingKalman FiltersIntroductionLeast Squares EstimatorKalman FilterDiscrete Kalman FilterExtended Kalman FilterFurther ReadingQuaternions and Sensor FusionIntroductionQuaternion-Based Kalman FilterQuaternion-Based Extended Kalman FilterConversion between Euler and QuaternionFurther ReadingHANDS-ONHands-On: Inertial Sensors for 3D InputPaul W. YostIntroductionMotion Sensing and Motion CaptureTypes of Motion Sensing TechnologyInertial Sensor Configurations for InputHands-On: YEI 3-Space SensorsHands-On: YEI Prio for Whole-Body InputFurther ReadingSimple Hands-On Project with Unity 3D and Oculus RiftNonnarit O-larnnithipongInstallation and System RequirementsGetting StartedCreating Game SceneLighting, Camera and SkyboxGameObject and Basic Action ScriptGraphic User Interface (GUI)Oculus Rift Integration for UnityFurther ReadingHands-On Approach with Leap MotionFrank E. HernandezWhat Is Leap MotionInstallationHands-On Mini-ProjectFurther ReadingHands-On Approach with Kinect Sensor v2Frank E. HernandezWhat Is the Kinect SensorInstallationHands-On Mini-ProjectFurther ReadingCreating Home-Brew Devices with Arduino MicrocontrollersSudarat TangnimitchokMicrocontrollerAnalog SensorSerial CommunicationHands-On Project: Ultrasonic Proximity SensorAutonomous Bicycle with Gyroscope SensorPanuwat Janwattanapong and Mercedes CabrerizoIntroductionAU Self-Balancing Bicycle (AUSB)Data ProcessingSystem Implementation and ResultsConclusionFurther ReadingExerciseInput Implementation DetailsInput DevicesMulti-Touch ImplementationWorking with a 3D Graphics Engine: OGREECHoSS: Experiment ModuleFurther ReadingCASE STUDY: SPEECH AS INPUTMultimodal Human-Like Conversational InterfacesUgan Yasavur and Christine LisettiDialogue Management OverviewDialogue Management in Health Dialogue SystemsTask-Based Spoken Dialog SystemsEmbodied Conversational AgentsBrief Interventions for Alcohol ProblemsConclusionAdaptive Dialogue Systems for HealthUgan Yasavur and Christine LisettiApproachReinforcement Learning BackgroundMarkov Decision ProcessesModeling World with Interconnected MDPsAgent and Dialogue Strategy LearningReward Function DesignSpeech Recognition and Language ModelDialog CorpusConclusionCONTENTS OF ONLINE SUPPLEMENTARY MATERIALThe World of Gesture RecognitionInput Frameworks and Development Considerations for Input TechnologiesHands-On: Multi-Touch and Pen Using Microsoft Windows 8 and Windows 10 with Microsoft Surface Pro Hands-On: Intel RealSense SDK.APPENDICESDisplaysJorge ChernicharoFixed DisplaysPortable DisplaysHybrid SystemsCreating Your Own Virtual Reality HeadsetKarrel MullerIntroductionGoogle Cardboard

Francisco R. Ortega, PhD, is a postdoctorate research fellow at Florida International University, Miami, where he received his PhD in computer science. He is the current director of the Open Human-Interface Device Laboratory at Florida International University (http://openhid.com). He was a member of the Digital Signal Processing Laboratory at FIU, and has over 17 years of experience in software development and systems integration. His interests are in 3D user interfaces, input devices, human–computer interaction, 3D navigation, and input modeling, among other interests. He has multiple publications in journals, lecture notes, and conference proceedings. Fatemeh Abyarjoo, PhD, received her PhD in Electrical engineering from Florida International University, Miami, where she was also a research assistant in the Digital Signal Processing Laboratory, focusing on sensor fusion for human motion tracking. She is currently a Fraud Risk Data Scientist, focusing on financial data analyzing. Her research interests are data mining, data analysis, statistical modeling, sensor fusion and wearable devices. She is a former Open Science Data Cloud PIRE National Science Foundation Fellow. Armando Barreto, PhD, is a faculty member of the Electrical and Computer Engineering Department at Florida International University, Miami, as well as the director of FIU’s Digital Signal Processing Laboratory. He earned his PhD in electrical engineering from the University of Florida, Gainesville. His work has focused on applying DSP techniques to the facilitation of human–computer interactions, particularly for the benefit of individuals with disabilities. He has developed human–computer interfaces based on the processing of signals and has developed a system that adds spatialized sounds to the icons in a computer interface to facilitate access by individuals with "low vision." He is a senior member of the Institute of Electrical and Electronics Engineers and the Association for Computing Machinery. Naphtali Rishe, PhD, is Eminent Chair Professor of Computer Science at Florida International University, Miami. He has authored three books on database design and geography and has edited five books on database management and high performance computing. He holds four US patents on database querying, semantic database performance, Internet data extraction, and computer medicine. He has also authored 300 papers in journals and proceedings on databases, software engineering, Geographic Information Systems, Internet, and life sciences. His TerraFly project—a 50-terabyte database of aerial imagery and Web-based GIS—has been extensively covered by worldwide press. Malek Adjouadi, PhD,is a professor with the Department of Electrical and Computer Engineering at Florida International University, Miami. He received his PhD from the Electrical Engineering Department at The University of Florida, Gainesville. He is the founding director of the Center for Advanced Technology and Education funded by the National Science Foundation. His earlier work on computer vision to help persons with blindness led to his testimony to the US Senate on the committee of Veterans Affairs on the subject of technology to help persons with disabilities. His research interests are in image and signal processing with applications in neuroscience and assistive technology research.