Abstract
In this paper we argue for conducting empirical studies into the actual use and usefulness of multimodal user interfaces. We have started formulating some important questions and attempting to answer three of them using the prototype multimodal user interface EDWARD. We investigated questions concerning efficiency, experience, speed, and error for multimodal and unimodal interaction. The preliminary results indicate that multimodality is indeed a useful approach.
The original version was written while both authors were working at the NICI, University of Nijmegen, PO Box 9104 HE Nijmegen, The Netherlands.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allgayer, J., et al. (1989), ‘XTRA: a natural-language access system to expert systems'. In International Journal of Man-Machine Studies, 31, 161–195.
Bos, E. (1993), Easier said or done? Studies in multimodal human-computer interaction, Doctoral Dissertation. Rijks Universiteit Leiden (NICI Technical Report 93-02. Nijmegen: NICI).
Bos, E., Huls C., and Claassen, W. (1994), ‘EDWARD: full integration of language and action in a multimodal interface'. In International Journal of Human-Computer Studies 40, 473–495.
Claassen, W., Bos, E., and Huls, C. (1990), The Pooh Way in human-computer interaction: towards multimodal interfaces, SPIN-MMC Research report 5, Nijmegen:NICI.
Cohen, P. (1992), ‘The role of natural language in a multimodal interface'. In Proceedings of the 5th annual ACM symposium on user interface software and technology (UIST'92), New York: ACM Press.
Desain, P. (1988), ‘Direct manipulation and the design of user interfaces'. In Communication and Cognition — Artificial Intelligence, 5, 225–246.
Huls, C. and Bos, E. (1993), ‘EDWARD: A multimodal interface', in Proceedings of the Twente Workshop on Language Technology 5 (TWLT5), 89–98.
Huls, C., Bos, E., and Damen, H. (1993), ‘Fully integrated multimodality: a case study'. In HCI International'93, Orlando, Florida.
Huls, C., Bos, E., and Dijkstra, A. (1994), ‘Talking Pictures: An empirical study into the usefulness of natural language output in a graphical interface'. In Workshop Notes of the AAAI workshop Integration of Natural Language and Vision Processing, McKevitt, P. (ed.), 83–90.
Hutchins, E. (1989), ‘Metaphors for Interface Design'. In The Structure of Multimodal Dialogue, Taylor, M.M., Néel, F., and Bouwhuis, D.G. (eds.), Amsterdam: North-Holland, 11–28.
Marslen-Wilson, W., Levy, E., and Tyler, L.K. (1982), ‘Producing interpretable discourse: The establishment and maintenance of reference'. In Speech, place, and action, Jarvella, R.J. and Klein, W. (eds.), Chichester: John Wiley and Sons Ltd.
Stock, O. (1991), ‘Natural language and exploration of an information space: the ALFresco interactive system'. In Proceedings of the Twelfth international joint conference on artificial intelligence (IJCAI'91, Sydney, 24–30 Aug.), San Mateo, CA: Morgan Kaufmann Inc., 972–978.
Walker, M.A. (1989), ‘Natural language in a desktop environment'. In Designing and using human-computer interfaces and knowledge based systems, Proceedings of the HCI International '89, Salvendy, G., and Smith, M.J. (eds.), Amsterdam, Elsevier Science Publishers, 502–509.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag
About this paper
Cite this paper
Huls, C., Bos, E. (1998). Studies into full integration of language and action. In: Bunt, H., Beun, RJ., Borghuis, T. (eds) Multimodal Human-Computer Communication. CMC 1995. Lecture Notes in Computer Science, vol 1374. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0052326
Download citation
DOI: https://doi.org/10.1007/BFb0052326
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64380-7
Online ISBN: 978-3-540-69764-0
eBook Packages: Springer Book Archive