MEMODULES as Tangible Shortcuts to Multimedia Information | SpringerLink
Skip to main content
Springer Nature Link
Log in

MEMODULES as Tangible Shortcuts to Multimedia Information

  • Chapter
Human Machine Interaction

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5440))

Abstract

Tangible User Interfaces (TUIs) are emerging as a new paradigm for facilitating user interaction with the digital world by providing intuitive means to link the physical and digital worlds. The MEMODULES project has the objective of developing, experimenting and evaluating the concept of tangible shortcuts (reminders), facilitating (a) the control of devices in the everyday life and also (b) the categorization of information in order to ease or improve information access and retrieval. The project aims at facilitating the user interaction with multimedia information by supporting both the creation and management of tangible links to digital content. Moreover, our research investigates the opportunity of a more complex, multi-sensorial combination of objects and multimedia information by combining multiple interaction modalities - such as voice and gesture - with interactive information visualizations. In order to address these issues we propose a user-oriented framework, called Memodules Framework, enabling end users to turn everyday objects into Memodules. The framework provides a single platform that combines end-user programming, tangible interaction, multimodality and personal information management issues. Memodules framework is built upon MemoML (Memodules Markup Language) and SMUIML (Synchronized Multimodal User Interaction Markup Language) models, which guarantee framework flexibility, extensibility and evolution over time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ballagas, A.R., Ringel, M., Stone, M., Borchers, J.: iStuff: A Physical User Interface Toolkit for Ubiquitous Computing Environments. In: Conference on Human factors in computing systems (2003)

    Google Scholar 

  2. Benoit, A., Bonnaud, L., Caplier, L., Damousis, I., Tzovaras, D., Jourde, F., Nigay, L., Serrano, M., Lawson, J.-Y.: Multimodal Signal Processing and Interaction for a Driving Simulator: Component-based Architecture. Journal on Multimodal User Interfaces 1(1) (2007)

    Google Scholar 

  3. Boardman, R., Sasse, M.A.: Stuff goes into the computer and doesn’t come out: a cross-tool study of personal information management. In: SIGCHI conference on Human factors in computing systems, pp. 583–590. ACM Press, New York (2004)

    Google Scholar 

  4. Bolt, R.: Put-that-there: voice and gesture at the graphics interface. Computer Graphics 14(3) (1980)

    Google Scholar 

  5. Bouchet, J., Nigay, L., Ganille, T.: ICARE Software Components for Rapidly Developing Multimodal Interfaces. In: ICMI 2004, State College, Pennsylvania, USA (2004)

    Google Scholar 

  6. Bourguet, M.L.: A Toolkit for Creating and Testing Multimodal Interface Designs. In: 15th Annual Symposium on User Interface Software and Technology, Paris, pp. 29–30 (2002)

    Google Scholar 

  7. Cohen, P.R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., Clow, J.: QuickSet: multimodal interaction for distributed applications. In: Fifth ACM international Conference on Multimedia, Seattle, USA (1997)

    Google Scholar 

  8. Cselle, G., Albrecht, K., Wattenhofer, R.: Buzztrack: topic detection and tracking in email. In: 12th international conference on Intelligent user interfaces, pp. 190–197. ACM Press, New York (2007)

    Google Scholar 

  9. Dredze, M., Lau, T., Kushmerick, N.: Automatically classifying emails into activities. In: 11th international conference on Intelligent user interfaces, pp. 70–77. ACM Press, New York (2006)

    Google Scholar 

  10. Dumais, S., Cutrell, E., Cadiz, J., Jancke, G., Sarin, R., Robbins, D.C.: Stuff I’ve seen: a system for personal information retrieval and re-use. In: SIGIR 2003: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pp. 72–79. ACM Press, New York (2003)

    Chapter  Google Scholar 

  11. Dumas, B., Lalanne, D., Ingold, R.: Prototyping Multimodal Interfaces with SMUIML Modeling Language. In: CHI 2008 Workshop on UIDLs for Next Generation User Interfaces (CHI 2008 workshop), Florence, Italy (2008)

    Google Scholar 

  12. Dumas, B., Lalanne, D., Guinard, D., Ingold, R., Koenig, R.: Strengths and Weaknesses of Software Architectures for the Rapid Creation of Tangible and Multimodal Interfaces. In: Proceedings of 2nd international conference on Tangible and Embedded Interaction, Bonn, Germany, February 19 - 21, pp. 47–54 (2008)

    Google Scholar 

  13. Flippo, F., Krebs, A., Marsic, I.: A Framework for Rapid Development of Multimodal Interfaces. In: ICMI 2003, Vancouver, BC, November 5-7 (2003)

    Google Scholar 

  14. Gemmell, J., Bell, G., Lueder, R., Drucker, S., Wong, C.: MyLifeBits: fulfilling the Memex vision. In: MULTIMEDIA 2002: Proceedings of the tenth ACM international conference on Multimedia, pp. 235–238. ACM Press, New York (2002)

    Chapter  Google Scholar 

  15. Glushko, R.J., McGrath, T.: Document Engineering: Analyzing and Designing Documents for Business Informatics and Web Services. MIT Press, Cambridge (2005)

    Google Scholar 

  16. Greenberg, S., Fitchett, C.: Phidgets: easy development of physical interfaces through physical widgets. In: User Interface Software & Technology, CHI Letters (2001)

    Google Scholar 

  17. Greenberg, S.: Rapidly prototyping Single Display Groupware through the SDGToolkit. In: Fifth Conference on Australasian User interfaces (AUIC 2004). Dunedin, NZ (2004)

    Google Scholar 

  18. Holmquist, L.E., Antifakos, S., Schiele, B., Michahelles, F., Beigl, M., Gaye, L., Gellersen, H.-W., Schmidt, A., Strohbach, M.: Building Intelligent Environments with Smart-Its. In: SIGGRAPH 2003, Emerging Technologies exhibition (2003)

    Google Scholar 

  19. van den Hoven, E., Eggen, B.: Design Recommendations for Augmented Memory Systems. In: Designing for Collective Remembering Workshop at CHI 2006, Montréal, Quebec, Canada (2006)

    Google Scholar 

  20. van den Hoven, E., Eggen, B.: Personal souvenirs as Ambient Intelligent objects. In: Joint Conference on Smart Objects and Ambient intelligence: innovative Context-Aware Services, Usages and Technologies, SOc-EUSAI 2005, Grenoble, France, vol. 121, pp. 123–128. ACM Press, New York (2005)

    Google Scholar 

  21. Humble, J., Crabtree, A., Hemmings, T., Åkesson, K.-P., Koleva, B., Rodden, T., Hansson, P.: Playing with the Bits - User-configuration of Ubiquitous Domestic Environments. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864. Springer, Heidelberg (2003)

    Google Scholar 

  22. Ishii, H., Ullmer, B.: Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In: Conference on Human Factors and Computing Systems (1997)

    Google Scholar 

  23. Kaltenbrunner, M., Bencina, R.: ReacTIVision: A Computer-Vision Framework for Table-Based Tangible Interaction. In: Proceedings of TEI 2007. Baton Rouge, Louisiana (2007)

    Google Scholar 

  24. Kerr. B.: Thread arcs: an email thread visualization. In: IEEE Symposium on Information Visualization (INFOVIS), pp. 211–218 (2003)

    Google Scholar 

  25. Klemmer, S.R., Li, J., Lin, J., Landay, J.A.: Papier-Mâché: Toolkit Support for Tangible Input. In: CHI Letters: CHI 2004, vol. 6(1) (2004)

    Google Scholar 

  26. Krahnstoever, N., Kettebekov, S., Yeasin, M., Sharma, R.: A real-time framework for natural multimodal interaction with large screen displays. In: ICMI 2002, Pittsburgh, PA, USA (2002)

    Google Scholar 

  27. Krueger, R.A.: Focus Groups: a practical guide for applied research. Sage publications, Newbury Park (1988)

    Google Scholar 

  28. Lalanne, D., Rigamonti, M., Evequoz, F., Dumas, B., Ingold, R.: An ego-centric and tangible approach to meeting indexing and browsing. In: Popescu-Belis, A., Renals, S., Bourlard, H. (eds.) MLMI 2007. LNCS, vol. 4892, pp. 84–95. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  29. Lamming, M., Flynn, M.: Forget-me-not: intimate computing in support of human memory. In: FRIEND21 Symposium on Next Generation Human Interfaces (1994)

    Google Scholar 

  30. Mavrommati, I., Kameas, A., Markopoulos, P.: An editing tool that manages devices associations in an in-home environment. In: Personal and Ubiquitous Computing, vol. 8(3-4), pp. 255–263. ACM Press/Springer-Verlag, London (2004)

    Google Scholar 

  31. Mazalek, A.: Tangible Toolkits: Integrating Application Development across Diverse Multi-User and Tangible Interaction Platforms. In: Let’s Get Physical Workshop, DCC 2006, Eindhoven, Netherlands (2006)

    Google Scholar 

  32. Microsoft. Windows desktop search 3.01 (2007), http://www.microsoft.com/windows/products/winfamily/desktopsearch/default.mspx

  33. Mugellini, E., Rubegni, E., Gerardi, S., Abou Khaled, O.: Using Personal Objects as Tangible Interfaces for Memory Recollection and Sharing. In: 1st International Conference on Tangible and Embedded Interaction 2007, TEI 2007, Baton Rouge, Louisiana, USA (2007)

    Google Scholar 

  34. Mugellini, E., Rubegni, E., Abou Khaled, O.: Tangibl: 12th International Conference on Human-Computer Interaction, HCI International 2007, Beijing, China (2007)

    Google Scholar 

  35. Noy, N., McGuinness, D.L.: Ontology Development 101: A Guide to Creating Your First Ontology. Technical Report, Stanford University, Stanford, CA, 94305 (2001)

    Google Scholar 

  36. Perer, A., Shneiderman, B., Oard, D.W.: Using rhythms of relationships to understand e-mail archives. Journal American Society Inf. Sci. Technol. 57(14), 1936–1948 (2006)

    Article  Google Scholar 

  37. Rekimoto, J.: Time-machine computing: a time-centric approach for the information environment. In: 12th annual ACM symposium on User interface software and technology, pp. 45–54. ACM, New York (1999)

    Google Scholar 

  38. Ringel, M., Cutrell, E., Dumais, S.T., Horvitz, E.: Milestones in Time: The Value of Landmarks in Retrieving Information from Personal Stores. In: Rauterberg, M., Menozzi, M., Wesson, J. (eds.) INTERACT, Zurich, Switzerland, pp. 184–191. IOS Press, Amsterdam (2003)

    Google Scholar 

  39. Rodden, T., Crabtree, A., Hemmings, T., Koleva, B., Humble, J., Åkesson, K.-P., Hansson, P.: Between the dazzle of a new building and its eventual corpse: assembling the ubiquitous home. In: ACM Symposium on Designing Interactive Systems, Cambridge, Massachusetts. ACM Press, New York (2004)

    Google Scholar 

  40. Rudolph, L.: Project Oxygen: Pervasive, Human-Centric Computing – An Initial Experience. In: Dittrich, K.R., Geppert, A., Norrie, M.C. (eds.) CAiSE 2001. LNCS, vol. 2068, pp. 1–12. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  41. Shaer, O., Leland, N., Calvillo-Gamez, E.H., Jacob, R.J.K.: The TAC Paradigm: Specifying Tangible User Interfaces. Personal and Ubiquitous Computing Journal (2004)

    Google Scholar 

  42. Smith, G., Czerwinski, M., Meyers, B., Robbins, D., Robertson, G., Tan, D.S.: FacetMap: A Scalable Search and Browse Visualization. IEEE Transactions on Visualization and Computer Graphics 12(5), 797–804 (2006)

    Article  Google Scholar 

  43. Sohn, T., Dey, A.K.: iCAP: An Informal Tool for Interactive Prototyping of Context-Aware Applications. In: Extended Abstracts of CHI 2003, pp. 974–975 (2003)

    Google Scholar 

  44. Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM Systems Journal 39, 915–931 (2000)

    Article  Google Scholar 

  45. Viegas, F.B., Golder, S., Donath, J.: Visualizing email content: portraying relationships from conversational histories. In: SIGCHI conference on Human Factors in computing systems, pp. 979–988. ACM Press, New York (2006)

    Chapter  Google Scholar 

  46. Walker, W., Lamere, P., Kwok, P., Raj, B., Singh, R., Gouvea, E., Wolf, P., Woelfel, J.: Sphinx-4: A flexible open source framework for speech recognition. Sun Microsystems, Tech. Rep. TR-2004-139 (2004)

    Google Scholar 

  47. Wellner, P.: Interacting with paper on the DigitalDesk. Communications of the ACM 36(7), 87–96 (1993)

    Article  Google Scholar 

  48. Whittaker, S., Sidner, C.: Email overload: exploring personal information management of email. In: SIGCHI conference on Human factors in computing systems, pp. 276–283. ACM Press, New York (1996)

    Google Scholar 

  49. Rubegni, E., Mugellini, E., Abou Khaled, O.: Travelling as social activity, results of the survey. Technical Report (September 2006)

    Google Scholar 

  50. Cohen, G.: Memory in the real world. Psychology Press, Hove (1996)

    Google Scholar 

  51. Baddeley, A.: Human Memory: Theory and Practice. Psychology Press, UK (1997)

    Google Scholar 

  52. Lalanne, D., van den Hoven, E.: Supporting Human Memory with Interactive Systems. In: Proceedings of the British HCI conference 2007, Lancaster, UK, vol. 2, pp. 215–216 (2007)

    Google Scholar 

  53. Rusted, J.M., Sheppard, L.M.: Action-based memory in people with dementia: A longitudinal look at tea-making. Neurocase 8, 111–126 (2002)

    Article  Google Scholar 

  54. Conway, M.A., Pleydell-Pearce, C.W.: The construction of autobiographical memories in the self-memory system. Psychological Review 107, 261–288 (2000)

    Article  Google Scholar 

  55. Ochsner, K.N., Schacter, D.L.: Remembering emotional events: A social cognitive neuroscience approach. In: Davidson, R.J., et al. (eds.) Handbook of Affective Sciences, pp. 643–660. Oxford University Press, Oxford (2003)

    Google Scholar 

  56. van den Hoven, E., Eggen, B.: Digital Photo Browsing with Souvenirs. In: Proceedings of the Interact 2003, pp. 1000–1004 (2003)

    Google Scholar 

  57. van den Hoven, E., Eggen, B.: Informing Augmented Memory System design through Autobiographical Memory theory. Personal and Ubiquitous Computing journal (2007), www.personal-ubicomp.com

  58. Glos, J., Cassell, J.: Rosebud: Technological Toys for Storytelling. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 1997), pp. 359–360. ACM Press, New York (1997)

    Google Scholar 

  59. van den Hoven, E., Eggen, B.: Tangible Computing in Everyday Life: Extending Current Frameworks for Tangible User Interfaces with Personal Objects. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L., et al. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 230–242. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  60. Lamming, M., Flynn, M.: Forget-me-not: intimate computing in support of human memory. In: FRIEND21: International symposium on next generation human interface, Meguro Gajoen, Japan, pp. 125–128 (1994)

    Google Scholar 

  61. Whittaker, S., Bellotti, V., Gwizdka, J.: Email in Personal Information Management. Communications of the ACM 49(1), 68–73 (2006)

    Article  Google Scholar 

  62. Lalanne, D., Ingold, R.: Structuring Multimedia Archives With Static Documents. ERCIM News: Multimedia Informatics 62(62), 19–20 (2005)

    Google Scholar 

  63. Ullmer, B.: Tangible Interfaces for Manipulating Aggregates of Digital Information. Doctoral dissertation. MIT Media Lab., USA (2002)

    Google Scholar 

  64. Ljungstrand, P., Redström, J., Holmquist, L.E.: WebStickers: Using Physical Tokens to Access, Manage and Share Bookmarks to the Web. In: Designing Augmented Reality Environments, DARE 2000, pp. 23–31 (2000)

    Google Scholar 

  65. Streitz, N., Geissler, J., Holmer, T., Konomi, S., Müller-Tomfelde, C., Reischl, W., Rexroth, P., Seitz, P., Steinmetz, R.: i-LAND: An interactive Landscape for Creativity and Innovation. In: Proceedings of the ACM Conference of Human Factors in Computing Systems (CHI 1999), pp. 120–127 (1999)

    Google Scholar 

  66. Petersen, M.-G., Grønbæk, K.: Shaping the Ambience of Homes with Domestic Hypermedia. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 218–229. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  67. Stevens, M.M., Abowd, G.D., Truong, K.N., Vollmer, F.: Getting into the living memory box: family archives & holistic design. Personal and Ubiquitous Computing journal 7, 210–216 (2003)

    Article  Google Scholar 

  68. Ullmer, B.: Models and Mechanisms for Tangible User Interfaces, Masters thesis. MIT Media Lab., USA (1997)

    Google Scholar 

  69. van den Hoven, E.: Graspable Cues for Everyday Recollecting, PhD Thesis, Department of Industrial Design, Eindhoven University of Technology, The Netherlands (2004) ISBN 90-386-1958-8

    Google Scholar 

  70. Bationo, A., Kahn, J., Decortis, F.: Traveling narrative as a multi-sensorial experience: A user centered approach of smart objects. In: Proceedings of Smart Objects Conference (sOc 2003), pp. 160–163 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Applied Sciences of Western Switzerland, Switzerland

    Elena Mugellini, Sandro Gerardi, Anne Le Calvé, Alexandre Boder & Omar Abou Khaled

  2. University of Fribourg, Switzerland

    Denis Lalanne, Bruno Dumas, Florian Evéquoz & Rolf Ingold

Authors
  1. Elena Mugellini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Denis Lalanne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno Dumas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Florian Evéquoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sandro Gerardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anne Le Calvé
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexandre Boder
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rolf Ingold
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Omar Abou Khaled
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Fribourg, Bd. de Pérolles 90, CH-1700, Fribourg, Switzerland

    Denis Lalanne  & Jürg Kohlas  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Mugellini, E. et al. (2009). MEMODULES as Tangible Shortcuts to Multimedia Information. In: Lalanne, D., Kohlas, J. (eds) Human Machine Interaction. Lecture Notes in Computer Science, vol 5440. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00437-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-00437-7_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-00436-0

  • Online ISBN: 978-3-642-00437-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics