Abstract
The objective of the present study was to develop a data-driven framework for MMI (multimodal interaction) system development. In the area of software engineering, data-driven development methods have proven to be reusable and extensible in the object-oriented paradigm. We have designed an object-oriented data modeling language for MMI systems and an extended elements of HTML for interaction description. The proposed framework uses the extended Rails framework as a generator of the MVC (Model-View-Controller) elements from the proposed objectoriented data modeling language. As the generated view code is written in extended HTML, we also implemented a speech browser for the extended HTML. We evaluated the framework by rapidly implementing a messaging system and examined its extensibility by a functionality expansion test.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Masahiro Araki, Kazunori Komatani, Taishi Hirata and Shuji Doshit: A Dialogue Library for Task-oriented Spoken Dialogue Systems, In Proc. IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems, 1–7, (1999)
Masahiro Araki, Akiko Kouzawa and Kenji Tachibana: Proposal of a multimodal interaction description language for various interac-tive agents, IEICE transactions on information and systems, E88-D(11), 2469–2476 (2005)
Masahiro Araki, Tsuneo Nitta, Kouichi Katsurada, Takuya Nishimoto, Tetsuo Amakasu and Shinnichi Kawamoto: Proposal of a Hierarchical Architecture for Multimodal Interactive Systems. W3C MMI workshop. (2007) Available via http://www.w3.org/2007/08/mmi-arch/slides/ araki.pdfCited5May2011
Masahiro Araki and Takashi Hattori: Proposal of a Practical Spoken Dialogue System Development Method: Data-management Centered Approach, In W. Minker et al. (eds) Spoken Dialogue Systems Technology and Design, Springer, 187–211 (2010)
Masahiro Araki, Kana Shibahara and Yuko Mizukami: Spoken dialogue system for learning Braille, In Proc. COMPSAC 2011 (to appear)
Jim Barnett (eds.): Multimodal Architecture and Interfaces. W3C Working Draft 25 January 2011. Available via http://www.w3.org/TR/ mmi-arch/Cited5May2011
Philip Hanna, Ian O’Neill, Xingkun Liu and Michael McTear: Developing Extensible and Reusable Spoken Dialogue Components: An Examination of the Queen’s Communicator, In Proc. Interspeech, 1865–1868 (2005)
Tobias Heinroth, Dan Denich and Gregor Bertrand: Ontology-based Spoken Dialogue Modeling, In Proc. the 1st IWSDS, (2009)
Koichi Katsurada, Hiroaki Adachi, Kunitoshi Sato, Hirobumi Yamada and Tsuneo Nitta: Rapid Prototyping Tool for Constructing Web-based MMI Applications, In Proc. Interspeech, 1861–1864 (2005)
Michael McTear: Spoken Dialogue Technology, Springer. (2004)
Bertrand Meyer: Object-Oriented Software Construction (2nd Edition). Prentice-Hall International Series in Computer Science. (2000)
Stephanie Seneff, Ed Hurley, Raymond Lau, Christine Pao, Philipp Schmid, and Victor Zue: GALAXY-II: A Reference Architecture for Conversational System Development. In Proc. ICSLP, 931–934 (1998)
Daniel Sonntag, Robert Nesselrath, Gerhard Sonnenberg, and Gerd Herzog: Supporting a Rapid Dialogue System Engineering Process, In Proc. the 1st IWSDS, (2009)
Acknowledgements
The present research was supported in part by the Ministry of Education, Science, Sports, and Culture through a Grant-in-Aid for Scientific Research (C), 22500153, 2010.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this paper
Cite this paper
Araki, M., Mizukami, Y. (2011). Development of a Data-driven Framework for Multimodal Interactive Systems. In: Delgado, RC., Kobayashi, T. (eds) Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1335-6_11
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1335-6_11
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-1334-9
Online ISBN: 978-1-4614-1335-6
eBook Packages: EngineeringEngineering (R0)