Multimodal Interactive Systems Management

This book provides a synthesis of the multifaceted field of interactive multimodal information management. The subjects treated include spoken language processing, image and video processing, document and handwriting analysis, identity information and interfaces. The book concludes with an overview of the highlights of the progress of the field during the past ten years, as well as the problems that are now under investigation and that offer the most promising results for the future. The book is addressed to the graduate student/postdoc level, but much of the book will be accessible to all those with a general background in information processing.

Interactive Multimodal Information Management: Shaping the VisionMeeting capture, analysis and accessThe IM2 Swiss National Center of Competence in ResearchRelated international projects and consortiaHuman-Computer Interaction and Human FactorsHuman Factors in Multimodal Information ManagementRole of human factorsProminent research topics in human factorsMethodological approach in human factorsEmpirical studiesDiscussion and implicationsUser Attention During Mobile Video ConsumptionModeling user behaviorData acquisition experimentData processing and resultsConclusionsWizard of Oz Evaluations of the Archivus Meeting BrowserThe Archivus meeting browserMultimodal Wizard of Oz evaluationConclusionsDocument-Centric and Multimodal Meeting AssistantsThe Smart Meeting Minutes applicationDocument centric meeting browsingCross-meeting and ego-centric browsingMultimodal user interfaces prototyping for online meeting assistantsThe Communication Board applicationConclusionSemantic Meeting Browsers and AssistantsThe JFerret framework and browserTQB: a transcript-based query and browsing interfaceEvaluation of meeting browsersAutomatic meeting browsers and assistantsConclusions and perspectivesMultimedia Information RetrievalIntroductionMultimedia information retrieval: from information to user satisfactionInteraction log mining: from user satisfaction to improved information retrievalMultimedia information retrieval in a wider contextVisual and Multimodal Analysis of Human Appearance and BehaviorFace Recognition for BiometricsIntroductionFace processing in a nutshellFrom face detection to face recognitionStatistical generative models for face recognitionCross-pollination to other problemsOpen data and softwareConclusion and future workFacial Expression AnalysisIntroduction and state-of-the-artRecognizing action unitsModeling human perception of static facial expressionsConclusionSoftware for Automatic Gaze and Face/Object TrackingGaze trackingFace tracking in real environmentsApplication to autism spectrum disorderConclusionLearning to Learn New Models of Human Activities in Indoor SettingsIntroductionRelated workProposed approachActivity tracking for unusual event detectionKnowledge transfer for unusual event learningExperimentsConclusionNonverbal Behavior AnalysisIntroduction: a brief history of nonverbal behavior research in IM2VFOA recognition for communication analysis in meeting rooms and beyondSocial signal processingBehavioral analysis of video bloggingFinal remarksMultimodal Biometric Person RecognitionIntroductionBiometric classification with quality measuresModeling reliability with Bayesian networksA-stack: biometric recognition in the score-age-quality classification spaceConclusionsMedical Image AnnotationIntroductionMultiple cues for image annotationExploiting the hierarchical structure of data: confidence-based opinion fusionFacing the class imbalance problem: virtual examplesExperimentsConclusionsSpeech, Language, and Document ProcessingSpeech ProcessingMethods for automatic speech recognitionFront-end processing of speechPosterior-based automatic speech recognitionThe Juicer decoderConclusionsResearch Trends in Speaker DiarizationGoals and applications of speaker diarizationA state-of-the-art speaker diarization systemResearch problems in speaker diarizationConclusions and perspectivesSpeaker Diarization of Large CorporaTwo-stage cross-meeting diarizationSpeaker linkingExperimental resultsConclusionsLanguage Processing in DialoguesObjectives of language analysis in meetingsDialogue actsDiscourse particlesThematic episodes and hot spotsSemantic cross-modal alignmentConclusion and perspectivesOffline Handwriting RecognitionIntroductionOffline word recognitionFrom word to text recognitionFrom text to documentsConclusionsOnline Handwriting Analysis and RecognitionIntroductionDatabase acquisitionOnline mode detectionOnline handwriting recognitionWriter identificationConclusionAnalysis of Printed DocumentsExtracting and reorganizing digital content from printable documentsTagging the information extracted from digital documents . .Video recording alignment and the temporal dimension of printable documentsAligning digital documents with audio recordingsFrom printable documents to cross-media alignment and indexingConclusionAssessmentsIt was Worth it! Assessment of the Impact of IM2Motivation and procedureThe assessment questionsSynthesis of the interviewsConclusionTechnology Transfer: Turning Science into ProductsVisual recognition on mobile devices: kooaba AGJoining capture and webcast: Klewel SABusiness experience meets technology: KeyLemon SAXED and Dolores for ebooks: sugarcube Information Technology SarlFrom speech to text: Koemei SALook me in the eye: Pomelo SARLThe Association for Interactive Multimodal Information ManagementThe International Create ChallengeConclusion and PerspectivesLooking back on the initial motivations of IM2Scientific achievements of IM2Structural achievementsTechnology transfer achievementsPerspectives