Bourlard H., Popescu-Belis A. (eds.) Interactive Multimodal Information Management

EPFL Press, 2014, -350 p.
IM2 is concerned with the development of natural multimodal interfaces for human-computer interaction. By multimodal we mean the different technologies that coordinate natural input modes (such as speech, pen, touch, hand gestures, head and body movements, and eventually physiological sensors) with multimedia system output (such as speech, sounds, and images). Ultimately, these multimodal interfaces should flexibly accommodate a wide range of users, tasks, and environments for which any single mode may not suffice. The ideal interface should primarily be able to deal with more comprehensive and realistic forms of data, including mixed data types (i.e., data from different input modalities such as image and audio)..
As part of IM2, we are also focusing on computer-enhanced human-to-human interaction. Indeed, understanding human-human interaction is fundamental to the long-term pursuit of powerful and natural multimodal interfaces for human-computer interaction. In addition to making rich, socially-enhanced analyses of group process ripe for exploitation, our advances in speech, video, and language processing, as well as the tools for working with multimodal data, will improve research and development in many related areas..
The field of multimodal interaction covers a wide range of critical activities and applications, including recognition and interpretation of spoken, written and gestural language, particularly when used to interface with multimedia information systems, and biometric user authentication (protecting information access). As addressed by IM2, management of multimedia information systems is a wide-ranging and important research area that includes not only the multimodal interaction described above, but also multimedia document analysis, indexing, and information retrieval. The development of this technology is necessarily multi-disciplinary, requiring the collaborative contributions of experts in engineering, computer science, and linguistics.
Interactive Multimodal Information Management: Shaping the Vision.
Human-Computer Interaction and Human Factors.
Human Factors in Multimodal Information Management.
User Attention During Mobile Video Consumption.
Wizard of Oz Evaluations of the Archivus Meeting Browser.
Document-Centric and Multimodal Meeting Assistants.
Semantic Meeting Browsers and Assistants.
Multimedia Information Retrieval.
Visual and Multimodal Analysis of Human Appearance and Behavior.
Face Recognition for Biometrics.
Facial Expression Analysis.
Software for Automatic Gaze and Face/Object Tracking.
Learning to Learn New Models of Human Activities in Indoor Settings.
Nonverbal Behavior Analysis.
Multimodal Biometric Person Recognition.
Medical Image Annotation.
Speech, Language, and Document Processing.
Speech Processing.
Research Trends in Speaker Diarization.
Speaker Diarization of Large Corpora.
Language Processing in Dialogues.
Online Handwriting Recognition.
Online Handwriting Analysis and Recognition.
Analysis of Printed Documents.
Assessments.
It was Worth it! Assessment of the Impact of IM2.
Technology Transfer: Turning Science into Products.
Conclusion and Perspectives.