Stepping into Virtual Reality

This book is the fruit of many years of experience on the creation of synthetic worldsandvirtualrealities.Ourgoalistotransmitthisexperiencetostudents and ease the learning curve required to master the creation of Virtual Reality (VR) applications. We start by providing some brief answers to key questions such as: where did VR come from? what are the main concepts that help us understand this research ?eld? what are the current VR trends? and last but not least, how can we create virtual worlds? Throughout the book we consider the terms “virtual environment,” “v- tualworld”and“VRapplication”asequivalentconcepts:computer-generated environments mainly composed of interactive computer graphics, designed to physically and/or psychologically immerse one or more users in an alternative reality. The ?rst part of the book makes a review of the basic theoretical and practical concepts involved in the visual aspect of virtual environments. We startbypresentingthebasicmathematicalfoundationsforthesynthesisof3D graphics,includingbasicmodelingandrenderingtechniques.Acomprehensive review of the state of the art of computer animation closes this part.

Contents

1 Introduction

1.1 Virtual Reality: The Science of Illusion

1.2 Basic Concepts

1.2.1 Immersion

1.2.2 Presence

1.3 A Brief History of Virtual Reality

1.4 Reality-Virtuality Continuum

1.5 Metaverse

 1.5.1 Metaverse and its Future

 1.5.2 Major Technologies to Empower the Metaverse

Part I Fundamentals

2 Computer Graphics

2.1 Mathematics

2.1.1 Coordinate Systems

2.1.2 Vectors, Transformations and Matrices

2.1.3 Angular Representations

2.1.4 Projections

2.2 3D Modeling

2.2.1 Geometric Representations

2.2.2 Curves

2.2.3 Surfaces

2.3 3D Rendering

2.3.1 Local Illumination Model

2.3.2 Global Illumination Model

2.3.3 Textures

2.3.4 Rendering Pipeline

2.4 Game Engines

3 Computer Animation

3.1 Introduction

3.1.1 Animation Types: Real Time and Image by Image

3.1.2 Articulated Bodies and Virtual Characters

3.2 Motion Control Methods

3.3 Motion Capture and Performance Animation

3.3.1 Optical Motion Capture Systems

3.3.2 Non-Optical Motion Capture

3.3.3 Motion Capture Advantages and Disadvantages

3.4 Key-Frame Animation

3.4.1 Shape Interpolation and Parametric Keyframe Animation

3.4.2 Kochanek-Bartels Spline Interpolation

3.5 Inverse Kinematics

3.6 Motion Retargeting

3.7 Procedural Animation

3.8 Physics-Based Animation

3.9 Behavioral Animation

Part II Virtual Worlds

4 Virtual Characters

4.1 Virtual Humans in Virtual Environments

4.2 Character Skinning

4.2.1 Skeleton-Based Deformations

4.2.2 Data-Driven Methods

4.2.3 Physics-Based Approaches

4.3 Locomotion

4.3.1 Locomotion Generation

4.3.2 PCA-Based Locomotion

4.4 Virtual Human-Object interaction

4.4.1 Feature Modeling and Smart Objects

4.4.2 Grasping

4.4.3 Motion Planning

4.5 Facial Animation

4.6 Autonomous Characters

4.6.1 Why Autonomous Virtual Characters?

4.6.2 Properties of Autonomous Virtual Characters

4.6.3 Behaviors for Autonomous Virtual Characters

4.7 Crowd Simulation

5 Architecture of Virtual Reality Systems

5.1 Scene Graph-Based Systems

5.2 Semantic Virtual Environments

5.3 Generic System Architecture for VR Systems

5.4 Distributed Virtual Environments

5.4.1 Communication Architecture

5.4.2 Interest Management

5.4.3 Concurrency Control

5.4.4 Data Replication

5.4.5 Load Distribution

5.5 Metaverse

5.5.1 The Concept of Avatar

5.5.2 Online Worlds

5.6 Metaverse and the Internet of Things (IoT)

6 Mixed Realities

6.1 Augmented Reality and Augmented Virtuality

6.2 Tracking Techniques

6.2.1 Markers-Based Tracking

6.2.2 Marker-Less Tracking

6.3 Mixed Reality Tool Kits

6.4 Deep Learning and 3D Hand Pose

Part III Perceiving Virtual Worlds

7. Senses and Sensors

7.1 Introduction

7.2 The 5 senses

7.3 The Extra Senses

 7.3.1 Proprioception

 7.3.2 Equilibrioception

 7.3.3 Pallesthesia

 7.3.4 Thermoception

 7.3.5 Nociception

 7.3.6 Chronoception

8. Vision

8.1 Graphical Display Technologies

8.1.1 Cathode-Ray Tubes

8.1.2 Liquid Crystal Displays

8.1.3 Plasma Displays

8.2 Virtual Reality Displays

8.2.1 Head-Mounted Displays

8.2.2 Fish Tank VR

8.2.3 Handheld Displays

8.2.4 Large Projection Screens

8.2.5 CAVE Systems

9 Audition

9.1 The Need for Sound in VR

9.2 Recording and Reproduction of Spatial Sound

9.3 Synthesis of Spatial Sound

9.3.1 Sound Rendering

9.3.2 Head-Related Transfer Function

9.3.3 3D Sound Imaging

9.3.4 Utilization of Loudspeaker Location

9.4 Sound Systems for VR

9.4.1 Sound Hardware

9.4.2 Sound Engines

 9.5 Immersive Sound

10 Touch

10.1 The Need for Touch in VR

10.2 Data Gloves

10.3 Haptic Rendering

10.3.1 History of Haptic Rendering

10.4 Haptic Interfaces

10.4.1 Vibrotactile Displays

10.4.2 Tactile Displays

10.4.3 Kinesthetic Displays

11 Smell and Taste

11.1 The Need for Smells and Tastes in VR

11.2 Smell Interfaces

11.3 Taste interfaces

12 Brain-Computer Interfaces

12.1 BCI Principles

12.2. BCI in Virtual Reality

Part IV Applications

13 Health Sciences

13.1 Virtual Surgery

 13.1.2 Principles of Virtual Surgery

 13.1.2 Case Studies in Virtual Surgery

 13.1.3 Commercial Virtual Surgery Simulators

13.1.4 The Intelligent Digital Surgeon

13.2 Virtual Rehabilitation and Therapy

13.2.1 Physiotherapy

13.2.2 Psychological Therapy

13.2.3 Autism and Virtual Dolphins

13.3 Virtual Anatomy

14 Cultural Heritage

14.1 Virtual Campeche and Calakmul

14.2 Virtual Dunhuang

14.3 Terracotta Soldiers

14.4 EU-INCO CAHRISMA and ERATO

14.5 EU-HORIZON 2020 Mingei

15 Other VR Applications

15.1 Vehicle Simulators

15.2 Manufacturing

15.3 Entertainment

15.4 Virtual Reality and Sport

Mario A. Gutiérrez A. obtained a Ph.D. in Computer Science, with specialization in Virtual Reality from EPFL, Switzerland. He has co-authored several peer-reviewed international conference papers, scientific journal articles and books covering topics related to Computer Graphics, Virtual Reality and Human-Computer Interaction. He has served as assistant professor in Computer Graphics and Virtual Reality at Tecnológico de Monterrey in Mexico. In addition to his academic experience, Mario Gutierrez has worked for more than 10 years in the software and consumer electronics industries, focusing on research and development projects aimed at developing innovative computer peripheral devices and software for Virtual Reality and spatial computing applications. He is currently Sr. Software Engineer at Logitech Europe S.A., based in Lausanne, Switzerland.

 

Dr. Frédéric Vexo is a computer scientist, accomplished entrepreneur, and respected authority in the field of human-computer interaction. Born in 1974 in eastern France, his early fascination with computers sparked a lifelong passion for exploring their capabilities. He pursued a Ph.D. in computer science, immersing himself in the realms of physics and mathematics to create Virtual Worlds. As a senior researcher at the Swiss Federal Institute of Science in Lausanne, Switzerland, Dr Vexo led a talented team of graduate and Ph.D. students in pioneering research on virtual reality. Dr Vexo’s expertise is widely recognised through his extensive publications, including co-authoring books and over 100 scientific papers. He is a sought-after speaker at international conferences, sharing his knowledge and shaping the discourse on human-computer interaction. Passionate about education, Dr Vexo has dedicated himself to nurturing the next generation of innovators. He has taught graduate and undergraduate students in the captivating field of Human Computer Interaction, serving as an invited professor and senior researcher at institutions such as Tec Monterrey in Toluca, Mexico, and Keio University in Yokohama, Japan. Today, Dr. Vexo leverages his extensive experience as an advisor and strategist, guiding companies through digital transformation and supporting disruptive startups.

Prof. Daniel Thalmann is a renowned Swiss and Canadian computer scientist. He is currently an honorary professor at the EPFL in Switzerland and the executive director of R&D at MIRALab Sarl. He is co-editor-in-chief of Wiley's Journal of Computer Animation and Virtual Worlds (CAVW) and on the editorial boards of several other journals. Daniel Thalmann is programme chair and co-chair of CASA 2023 and CGI2023.

After receiving his Ph.D. in Computer Science from the University of Geneva in 1977, Daniel Thalmann began at the University of Montreal in Canada.  He later became a professor at EPFL, Switzerland, where he founded the Virtual Reality Lab (VRlab).  From 2009 to 2017, he was a visiting professor at the Nanyang Technological University in Singapore. 

Daniel Thalmann has served on numerous programme committees and as programme chair and co-chair of several conferences, including IEEE VR, ACM VRST, and ACM VRCAI. He and his students have published several seminal papers in virtual reality, computer graphics and animation. He is co-editor and co-author of many books including 'Crowd Simulation' and 'Stepping into Virtual Reality', published by Springer.

Throughout his successful career, Professor Daniel Thalmann has received many awards, including an Honorary Doctorate from Paul Sabatier University in Toulouse, France in 2003, the Eurographics Distinguished Career Award in 2010, the Canadian Human Computer Communications Society Achievement Award in 2012, and the CGI Career Achievement Award in 2015. More can be found on Daniel Thalmann in Wikipedia.