Touching graphical applications: bimanual tactile interaction on the HyperBraille pin-matrix display | Universal Access in the Information Society Skip to main content
Springer Nature Link
Log in

Touching graphical applications: bimanual tactile interaction on the HyperBraille pin-matrix display

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Novel two-dimensional tactile displays enable blind users to not only get access to the textual but also to the graphical content of a graphical user interface. Due to the higher amount of information that can be presented in parallel, orientation and exploration can be more complex. In this paper we present the HyperBraille system, which consists of a pin-matrix device as well as a graphical screen reader providing the user with appropriate presentation and interaction possibilities. To allow for a detailed analysis of bimanual interaction strategies on a pin-matrix device, we conducted two user studies with a total of 12 blind people. The task was to fill in .pdf forms on the pin-matrix device by using different input methods, namely gestures, built-in hardware buttons as well as a conventional PC keyboard. The forms were presented in a semigraphic view type that not only contains Braille but also tactile widgets in a spatial arrangement. While completion time and error rate partly depended on the chosen input method, the usage of special reading strategies seemed to be independent of it. A direct comparison of the system and a conventional assistive technology (screen reader with single-line Braille device) showed that interaction on the pin-matrix device can be very efficient if the user is trained. The two-dimensional output can improve access to .pdf forms with insufficient accessibility as the mapping of input controls and the corresponding labels can be supported by a spatial presentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Many blind users in Germany already know the navigation bar under the name ‘Easy Access Bar’ from single-line Braille devices (Braillex devices from Papenmeier, see http://www.papenmeier.de/product-line.html) where it is used for scrolling.

  2. Common accessibility framework for Windows.

  3. See Microsoft MSDN Library: Add-Ins and Extensibility, URL: http://msdn.microsoft.com/en-us/library/bb384241.aspx.

  4. Text input was realized by standard keyboard.

  5. Sole exception: participant 5 in PD keys condition.

  6. See VoiceOver screen reader for OS X, URL: http://www.apple.com/accessibility/osx/voiceover/.

References

  1. Albert, P.: Math class: an application for dynamic tactile graphics. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) Computers Helping People with Special Needs. Lecture Notes in Computer Science, pp. 1118–1121. Springer, Berlin (2006)

    Chapter  Google Scholar 

  2. Asakawa, C., Leporini, B.: Screen readers. In: Stephanidis, C. (ed.) The Universal Access Handbook, chap. 28, pp. 1–17. CRC Press, Boca Raton (2009)

  3. Bliss, J.: A relatively high-resolution reading aid for the blind. IEEE Trans. Man Mach. Syst. 10(1), 1–9 (1969)

    Article  Google Scholar 

  4. Bornschein, J.: BrailleIO: a tactile display abstraction framework. In: Proceedings of the International Workshop on Tactile/Haptic User Interfaces for Tabletops and Tablets (2014)

  5. Bornschein, J., Prescher, D., Weber, G.: Collaborative creation of digital tactile graphics. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 117–126. ACM (2015)

  6. Boyd, L.H., Boyd, W.L., Vanderheiden, G.C.: The graphical user interface: crisis, danger and opportunity. J. Vis. Impair. Blind. 496–502 (1990)

  7. Buscher, G., Biedert, R., Heinesch, D., Dengel, A.: Eye tracking analysis of preferred reading regions on the screen. In: CHI’10 Extended Abstracts on Human Factors in Computing Systems, pp. 3307–3312. ACM (2010)

  8. Buxton, W., Myers, B.: A study in two-handed input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 321–326. ACM (1986)

  9. Card, S.K., English, W.K., Burr, B.J.: Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a crt. Ergonomics 21(8), 601–613 (1978)

    Article  Google Scholar 

  10. Chang, D., Nesbitt, K.V., Wilkins, K.: The gestalt principle of continuation applies to both the haptic and visual grouping of elements. In: EuroHaptics Conference, 2007 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2007. Second Joint, pp. 15–20. IEEE (2007)

  11. Chang, D., Nesbitt, K.V., Wilkins, K.: The gestalt principles of similarity and proximity apply to both the haptic and visual grouping of elements. In: Proceedings of the Eight Australasian Conference on User interface-Volume 64, pp. 79–86. Australian Computer Society, Inc. (2007)

  12. Cichoń, K., Sobecki, J., Szymański, J.M.: Gesture tracking and recognition in touchscreens usability testing. In: Proceedings of the International Conference on Multimedia, Interaction, Design and Innovation, p. 9. ACM (2013)

  13. Davidson, P.W., Appelle, S., Haber, R.N.: Haptic scanning of braille cells by low-and high-proficiency blind readers. Res. Dev. Disabil. 13(2), 99–111 (1992)

    Article  Google Scholar 

  14. Gunzenhäuser, R., Weber, G.: Graphical user interfaces for blind people. In: Brunnstein, K., Raubold, E. (eds.) 13th World Computer Congress, vol. 2, pp. 450–457 (1994)

  15. Jacob, R.J.: The use of eye movements in human–computer interaction techniques: what you look at is what you get. ACM Trans. Inf. Syst. 9(2), 152–169 (1991)

    Article  Google Scholar 

  16. Klöpfer, K.: Ein multifunktionaler Büroarbeitsplatz für Blinde (a multifunctional work place for the blind). Ph.D. thesis, Universität Stuttgart, Institut für Informatik (1987). (in German)

  17. Kochanek, D.: A hypertext system for the blind newspaper reader. In: Zagler, W. (ed.) Computers for Handicapped Persons, pp. 285–293 (1992)

  18. Köhlmann, W., Lucke, U.: Alternative concepts for accessible virtual classrooms for blind users. In: 15th International Conference on Advanced Learning Technologies, pp. 413–417. IEEE (2015)

  19. Kraus, M., Völkel, T., Weber, G.: An off-screen model for tactile graphical user interfaces. In: ICCHP ’08: Proceedings of the 11th International Conference on Computers Helping People with Special Needs, pp. 865–872. Springer, Berlin (2008)

  20. Leganchuk, A., Zhai, S., Buxton, W.: Manual and cognitive benefits of two-handed input: an experimental study. ACM Trans. Comput. Hum. Interact. 5(4), 326–359 (1998)

    Article  Google Scholar 

  21. Loitsch, C., Weber, G.: Viable haptic UML for blind people. In: ICCHP ’12: Proceedings of the 13th International Conference on Computers Helping People with Special Needs, pp. 509–516. Springer, Berlin (2012)

  22. Miao, M., Köhlmann, W., Schiewe, M., Weber, G.: Tactile paper prototyping with blind subjects. In: HAID ’09 (Haptic and Audio Interaction Design), LNCS, vol. 5763, pp. 81–90. Springer, Berlin (2009)

  23. Molina-Rueda, A., Magallanes, Y., Sánchez, J.A., Enriquez, D.F.: Using heat maps for studying user preferences in vertical and horizontal multi-touch surfaces. In: International Conference on Electronics, Communications and Computing (CONIELECOMP), pp. 21–26. IEEE (2013)

  24. Mousty, P., Bertelson, P.: A study of braille reading: 1. Reading speed as a function of hand usage and context. Q. J. Exp. Psychol. 37(2), 217–233 (1985)

    Article  Google Scholar 

  25. Pohl, M., Schmitt, M., Diehl, S.: Comparing the readability of graph layouts using eyetracking and task-oriented analysis. In: Computational Aesthetics, pp. 49–56 (2009)

  26. Pölzer, S., Miesenberger, K.: 2D presentation techniques of mind-maps for blind meeting participants. Stud. Health Technol. Inform. 217, 533–538 (2014)

    Google Scholar 

  27. Prescher, D.: Ein taktiles Fenstersystem mit Multitouch-Bedienung (A tactile window system for multitouch operation). Diploma Thesis, Department of Computer Science, Dresden (2009). (in German)

  28. Prescher, D.: Redesigning input controls of a touch-sensitive pin-matrix device. In: Proceedings of the International Workshop on Tactile/Haptic User Interfaces for Tabletops and Tablets, pp. 19–24. Citeseer (2014)

  29. Prescher, D., Weber, G.: Locating widgets in different tactile information visualizations. In: ICCHP ’16: Proceedings of the 15th International Conference on Computers Helping People with Special Needs, pp. 100–107. Springer (2016)

  30. Prescher, D., Weber, G., Spindler, M.: A tactile windowing system for blind users. In: ASSETS ’10: Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 91–98. ACM (2010)

  31. Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall, Inc., Upper Saddle River, NJ, USA (1993)

    MATH  Google Scholar 

  32. Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., Strope, B.: “Your word is my command”: Google search by voice: a case study. In: Advances in Speech Recognition, pp. 61–90. Springer (2010)

  33. Schiewe, M., Köhlmann, W., Nadig, O., Weber, G.: What you feel is what you get: mapping GUIs on planar tactile displays. In: UAHCI ’09: Proceedings of the 5th International Conference on Universal Access in Human–Computer Interaction. Part II, pp. 564–573. Springer, Berlin (2009)

  34. Schweikhardt, W.: Interaktives Erkunden tastbarer Grafiken durch Blinde (interactive exploration of tactual graphics by the blind). In: Software-Ergonomie ’85, Mensch-Computer-Interaktion, Tagung III/1985 des German Chapter of the ACM, pp. 366–375. Teubner (1985). (in German)

  35. Shimada, S., Murase, H., Yamamoto, S., Uchida, Y., Shimojo, M., Shimizu, Y.: Development of directly manipulable tactile graphic system with audio support function. In: ICCHP ’10: Proceedings of the 12th International Conference on Computers Helping People with Special Needs, pp. 451–458. Springer, Berlin (2010)

  36. Spindler, M., Kraus, M., Weber, G.: A graphical tactile screen-explorer. In: ICCHP ’10: Proceedings of the 12th International Conference on Computers Helping People with Special Needs, pp. 474–481. Springer, Berlin (2010)

  37. Takagi, N., Morii, S., Motoyoshi, T.: A study of input and scrolling methods for tactile graphics editors available for visually impaired people. In: International Conference on Systems, Man, and Cybernetics, pp. 2333–2337. IEEE (2015)

  38. Taras, C., Raschke, M., Schlegel, T., Ertl, T.: Running graphical desktop applications on tactile graphics displays made easy. In: Proceedings of 3rd International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2010), pp. 25–26. Citeseer (2010)

  39. Völkel, T., Weber, G., Baumann, U.: Tactile graphics revised: the novel BrailleDis 9000 pin-matrix device with multitouch input. In: ICCHP ’08: Proceedings of the 11th International Conference on Computers Helping People with Special Needs, pp. 835–842. Springer, Berlin (2008)

  40. Weber, G.: Reading and pointing—modes of interaction for blind users. In: Ritter, G. (ed.) Proceedings of 11th World Computer Congress, Information Processing ’89, pp. 535–540. Elsevier, North Holland (1989)

    Google Scholar 

  41. Weber, G.: Adapting direct manipulation for blind users. In: CHI ’93: INTERACT ’93 and CHI ’93 Conference Companion on Human Factors in Computing Systems, pp. 21–22. ACM, New York (1993)

  42. Wormsley, D.: Reading rates of young braille-reading children. J. Vis. Impair. Blind. 90(3), 278–282 (1996)

    Google Scholar 

  43. Zeng, L., Weber, G.: Audio-haptic browser for a geographical information system. In: ICCHP ’10: Proceedings of the 12th International Conference on Computers Helping People with Special Needs, pp. 466–473. Springer, Berlin (2010)

Download references

Acknowledgements

We thank all the blind participants. The HyperBraille project was sponsored by the German Ministry of Economy and Technology (BMWi) under the Grant No. 01MT07004. Only the authors of this paper are responsible for its content.

Author information

Authors and Affiliations

  1. Technische Universität Dresden, Nöthnitzer Straße 46, 01187, Dresden, Germany

    Denise Prescher, Jens Bornschein & Gerhard Weber

  2. University of Potsdam, Potsdam, Germany

    Wiebke Köhlmann

Authors
  1. Denise Prescher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jens Bornschein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wiebke Köhlmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerhard Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Denise Prescher.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Prescher, D., Bornschein, J., Köhlmann, W. et al. Touching graphical applications: bimanual tactile interaction on the HyperBraille pin-matrix display. Univ Access Inf Soc 17, 391–409 (2018). https://doi.org/10.1007/s10209-017-0538-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-017-0538-8

Keywords

Search

Navigation