Interactive Sketch-based Interfaces and Modelling for Design

Sketching is a natural and intuitive communication tool used for expressing concepts and ideas that are difficult to communicate through text or speech alone. In design applications, drawings are used at various stages of the design process: from the early concept drawings scribbled on a piece of paper to immersive interactions in which users manipulate and adjust the 3D form of an object in virtual or augmented reality environments. This variety in drawing activities brings about the need for different interpretation strategies that support not only the sketching activity itself, but also allow sketch-based interactions, such as sketch-based queries, to take place. In this book, we explore the different drawing approaches used in design and the algorithms required for processing and interpreting the different sketches and drawings in design. The book is divided into two parts. The first part focuses on sketching in the 2D domain. This includes the digitization of offline and paperbased sketches, techniques for online sketch recognition, observations of user drawing habits, algorithms for inferring depth from 2D drawings, as well as non-photorealistic rendering techniques that are then applied to sketch-based queries. The second part of the book focuses on 3D sketching in virtual or augmented reality spaces. Here, we present the processing and rendering of the 3D strokes, the different interaction devices available for 3D sketching, and look at different applications where immersive 3D sketching has been applied with success.

1 IntroductionAlexandra Bonnici and Kenneth P. Camilleri I Sketches Drawn on 2D Media2 Simplification and Vectorization of Sketched DrawingsAlexandra Bonnici and Kenneth P. Camilleri 3 Online Interpretation of Sketched DrawingsT. Metin Sezgin 4 Reconstruction from Sketched DrawingsAlexandra Bonnici and Kenneth P. Camilleri 5 Reconciling Search Results with User SketchesPaul L. Rosin and Juncheng Liu II 3D Sketching6 Introduction to 3D SketchingRahul Arora, Mayra Donaji Barrera Machuca, Philipp Wacker, Daniel Keefe, et al. 7 Input Processing and Geometric Representations for 3D SketchesJohann Habakuk Israel, Mayra Donaji Barrera Machuca, Rahul Arora, et al. 8 Interaction Devices and Techniques for 3D SketchingMayra Donaji Barrera Machuca, Rahul Arora, Philipp Wacker, et al. 9 3D Sketching Application ScenariosPhilipp Wacker, Rahul Arora, Mayra Donaji Barrera Machuca, et al. 10 ConclusionsAlexandra Bonnici and Kenneth P. Camilleri

Alexandra Bonnici received her B.Eng. (Hons) in 2004, M.Phil. (Melit.) in Engineering in 2008 and Ph.D. (Melit.) in 2015 on work related to the vectorization and interpretation of sketches with artistic cues. Alexandra’s research work is the fields of computer vision and machine learning. Specific research interests include the machine interpretation of drawings and sketches, text, and music document processing, as well as the development of applications that assist the teaching and learning of music. Alexandra is a Senior Lecturer at the University of Malta and is currently serving as Head of Department of the Department of Systems and Control Engineering. She is also the coordinator of the Faculty’s Certificate in Engineering Sciences. She is also active in promoting STEM education, having set up the Engineering Technology Clubs at the Faculty of Engineering. Alexandra is a Senior Member of the Institute of Electronic and Electrical Engineers (IEEE, USA), a member of the Association for Computing Machinery (ACM, USA) and the Eurographics Association. She is also a steering committee member of the ACM Symposium on Document Engineering. Professor Kenneth P. Camilleri graduated with a B.Elec.Eng. (Hons) degree in electrical engineering from the University of Malta in 1991, an M.Sc. in Signal Processing and Machine Intelligence, and Ph.D. degrees from the University of Surrey, UK in 1994 and 1999, respectively. Professor Camilleri’s research work is in the fields of signal processing, computer vision and machine learning. His specific research interests include automatic drawing interpretation, eye gaze tracking, human computer interfacing based on electrophysiological signals such as the electroencephalogram (EEG) and the electrooculogram (EOG), and biomedical signal and image analysis for physiological measurement and the extraction of disease biomarkers. He has published research in these areas in over 160 international peer-reviewed publications. Professor Camilleri is a founding member of the Department of Systems and Control Engineering, and of the Centre for Biomedical Cybernetics, both at the University of Malta. He presently serves as Director of the Centre for Biomedical Cybernetics. Professor Camilleri is a Senior Member of the Institute of Electronic and Electrical Engineers (IEEE, USA), the Society for Photo-Optical Instrumentation Engineering (SPIE, USA), and the Association for Computing Machinery (ACM, USA), and a member of the Institution of Engineering and Technology (IET, UK).