Tzovaras D. (ed.) Multimodal User Interfaces. From Signals to Interaction Signals and Communication Technology

Springer, 2008, -248 p.Human interfacing with the environment and with other humans is undoubtedly, fully multimodal. All human senses participate, even if some of then dominate, to the everyday human operations of perception, action and interaction. Interaction with the computer or computer-mediated interaction with others has been based for decades in a limited set of modalities and customised devices. Recent technological advancement in the areas of signal processing, computer vision and human- computer interaction, however, has made multimodal interfaces a reality. Multimodal interaction is a multidisciplinary research area integrating efforts of various sciences such as cognition, psychology, computer science and engineering.
This book reports one of the main outcomes of an attempt to bring closer experts from the HCI and the signal processing fields. Multimodal signal processing is fused with multimodal HCI to produce new knowledge in the field of multimodal interfaces. This knowledge goes all the way from signals to interaction and paves the way for a multiview exploration of the topic by scientists of both fields.
The book presents the most recent results of the SIMILAR EU funded Network of Excellence (with the contribution of one Chapter from groups participating in the HUMAINE Network of Excellence) on three main pillars in multimodal interfaces research i. e. theory, software development and application frameworks. The contributions in this book are fully towards the ten Grand Challenges identified by the SIMILAR NoE in the SIMILAR Dreams book published on 2005.Multimodality Theory
Information-Theoretic Framework for Multimodal Signal Processing
Multimodality for Plastic User Interfaces: Models, Methods, and Principles
Face and Speech Interaction
Recognition of Emotional States in Natural Human-Computer Interaction
Two SIMILAR Different Speech and Gestures Multimodal Interfaces
Multimodal User Interfaces in Ubiquitous Environments
Software Engineering for Multimodal Interactive Systems
Gestural Interfaces for Hearing-Impaired Communication
Modality Replacement Framework for Applications for the Disabled
A medical Component-based Framework for Image Guided Surgery
Multimodal Interfaces for Laparoscopic Training