{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:50:41Z","timestamp":1740149441137,"version":"3.37.3"},"reference-count":43,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T00:00:00Z","timestamp":1619568000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"University of Rijeka","award":["uniri-tehnic-18-15"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"In this paper, we investigate the possibilities for augmenting interaction around the mobile device, with the aim of enabling input techniques that do not rely on typical touch-based gestures. The presented research focuses on utilizing a built-in magnetic field sensor, whose readouts are intentionally affected by moving a strong permanent magnet around a smartphone device. Different approaches for supporting magnet-based Around-Device Interaction are applied, including magnetic field fingerprinting, curve-fitting modeling, and machine learning. We implemented the corresponding proof-of-concept applications that incorporate magnet-based interaction. Namely, text entry is achieved by discrete positioning of the magnet within a keyboard mockup, and free-move pointing is enabled by monitoring the magnet\u2019s continuous movement in real-time. The related solutions successfully expand both the interaction language and the interaction space in front of the device without altering its hardware or involving sophisticated peripherals. A controlled experiment was conducted to evaluate the provided text entry method initially. The obtained results were promising (text entry speed of nine words per minute) and served as a motivation for implementing new interaction modalities. The use of neural networks has shown to be a better approach than curve fitting to support free-move pointing. We demonstrate how neural networks with a very small number of input parameters can be used to provide highly usable pointing with an acceptable level of error (mean absolute error of 3 mm for pointer position on the smartphone display).<\/jats:p>","DOI":"10.3390\/s21093087","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T02:29:07Z","timestamp":1619663347000},"page":"3087","source":"Crossref","is-referenced-by-count":0,"title":["Augmenting Around-Device Interaction by Geomagnetic Field Built-in Sensor Utilization"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3456-8369","authenticated-orcid":false,"given":"Sandi","family":"Ljubic","sequence":"first","affiliation":[{"name":"Faculty of Engineering, University of Rijeka, Vukovarska 58, HR-51000 Rijeka, Croatia"},{"name":"Center for Artificial Intelligence and Cybersecurity, University of Rijeka, R. Matejcic 2, HR-51000 Rijeka, Croatia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1513-0337","authenticated-orcid":false,"given":"Franko","family":"Hr\u017ei\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Rijeka, Vukovarska 58, HR-51000 Rijeka, Croatia"},{"name":"Center for Artificial Intelligence and Cybersecurity, University of Rijeka, R. Matejcic 2, HR-51000 Rijeka, Croatia"}]},{"given":"Alen","family":"Salkanovic","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Rijeka, Vukovarska 58, HR-51000 Rijeka, Croatia"},{"name":"Center for Artificial Intelligence and Cybersecurity, University of Rijeka, R. Matejcic 2, HR-51000 Rijeka, Croatia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4758-7972","authenticated-orcid":false,"given":"Ivan","family":"\u0160tajduhar","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Rijeka, Vukovarska 58, HR-51000 Rijeka, Croatia"},{"name":"Center for Artificial Intelligence and Cybersecurity, University of Rijeka, R. Matejcic 2, HR-51000 Rijeka, Croatia"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jones, B., Sodhi, R., Forsyth, D., Bailey, B., and Maciocci, G. (2012, January 21\u201324). Around Device Interaction for Multiscale Navigation. Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, San Francsico, CA, USA.","DOI":"10.1145\/2371574.2371589"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Butler, A., Izadi, S., and Hodges, S. (2008, January 19\u201322). SideSight: Multi-\u201ctouch\u201d Interaction around Small Devices. Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, Monterey, CA, USA.","DOI":"10.1145\/1449715.1449746"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hasan, K., Ahlstr\u00f6m, D., Kim, J., and Irani, P. (2017, January 6\u201311). AirPanes: Two-Handed Around-Device Interaction for Pane Switching on Smartphones. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3026029"},{"key":"ref_4","unstructured":"Hasan, K., Ahlstr\u00f6m, D., and Irani, P. (May, January 27). Ad-Binning: Leveraging around Device Space for Storing, Browsing and Retrieving Mobile Device Content. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France."},{"key":"ref_5","unstructured":"Kotz\u00e9, P., Marsden, G., Lindgaard, G., Wesson, J., and Winckler, M. (2013, January 2\u20136). Thumbs Up: 3D Gesture Input on Mobile Phones Using the Front Facing Camera. Proceedings of the 14th IFIP TC 13 International Conference, Cape Town, South Africa."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, J., Zhao, K., Zhang, X., and Peng, C. (2014, January 16\u201319). Ubiquitous Keyboard for Small Mobile Devices: Harnessing Multipath Fading for Fine-Grained Keystroke Localization. Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services, Bretton Woods, NH, USA.","DOI":"10.1145\/2594368.2594384"},{"key":"ref_7","unstructured":"Google (2021, March 27). Project Soli. Available online: https:\/\/atap.google.com\/soli\/."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lien, J., Gillian, N., Karagozler, M.E., Amihood, P., Schwesig, C., Olson, E., Raja, H., and Poupyrev, I. (2016). Soli: Ubiquitous Gesture Sensing with Millimeter Wave Radar. ACM Trans. Graph., 35.","DOI":"10.1145\/2897824.2925953"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ahlstr\u00f6m, D., Hasan, K., and Irani, P. (2014, January 23\u201326). Are You Comfortable Doing That? Acceptance Studies of around-Device Gestures in and for Public Settings. Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices and Services, Toronto, ON, Canada.","DOI":"10.1145\/2628363.2628381"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bianchi, A., and Oakley, I. (2013, January 10\u201313). Designing Tangible Magnetic Appcessories. Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, Barcelona, Spain.","DOI":"10.1145\/2460625.2460667"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hwang, S., Ahn, M., and Wohn, K. (2013, January 8\u201311). MagGetz: Customizable Passive Tangible Controllers on and around Conventional Mobile Devices. Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, UK.","DOI":"10.1145\/2501988.2501991"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Harrison, C., and Hudson, S.E. (2009, January 4\u20137). Abracadabra: Wireless, High-Precision, and Unpowered Finger Input for Very Small Mobile Devices. Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, Victoria, BC, Canada.","DOI":"10.1145\/1622176.1622199"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ketabdar, H., Y\u00fcksel, K.A., and Roshandel, M. (2010, January 7\u201310). MagiTact: Interaction with Mobile Devices Based on Compass (Magnetic) Sensor. Proceedings of the 15th International Conference on Intelligent User Interfaces, Hong Kong, China.","DOI":"10.1145\/1719970.1720048"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ketabdar, H., Roshandel, M., and Y\u00fcksel, K.A. (2010, January 7\u201310). MagiWrite: Towards Touchless Digit Entry Using 3D Space around Mobile Devices. Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, Lisbon, Portugal.","DOI":"10.1145\/1851600.1851701"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"23","DOI":"10.4018\/jmhci.2013070102","article-title":"MagiThings: Gestural Interaction with Mobile Devices Based on Using Embedded Compass Magnetic Field Sensor","volume":"5","author":"Ketabdar","year":"2013","journal-title":"Int. J. Mob. Hum. Comput. Interact."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Abdelnasser, H., Khalafallah, A., and Yousef, M. (2018, January 19\u201323). MagController: A System for Touchless Mobile Devices Control Using the Magnetic Field. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece.","DOI":"10.1109\/PERCOMW.2018.8480128"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ketabdar, H., Moghadam, P., Naderi, B., and Roshandel, M. (2012, January 21\u201324). Magnetic Signatures in Air for Mobile Devices. Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, San Francsico, CA, USA.","DOI":"10.1145\/2371664.2371705"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ketabdar, H., Chang, H., Moghadam, P., Roshandel, M., and Naderi, B. (2012, January 21\u201324). MagiGuitar: A Guitar That is Played in Air!. Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, San Francsico, CA, USA.","DOI":"10.1145\/2371664.2371704"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Smus, B., and Riederer, C. (2015, January 7\u201311). Magnetic Input for Mobile Virtual Reality. Proceedings of the 2015 ACM International Symposium on Wearable Computers, Osaka, Japan.","DOI":"10.1145\/2802083.2808395"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Abdelnasser, H., Youssef, M., and Harras, K. (2016, January 27\u201330). MagBoard: Magnetic-based Ubiquitous Homomorphic Off-the-shelf Keyboard. Proceedings of the 2016 13th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), London, UK,.","DOI":"10.1109\/SAHCN.2016.7733000"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Abdelnasser, H., Harras, K., and Moustafa, Y. (2020, January 22\u201325). MagStroke: A Magnetic Based Virtual Keyboard for Off-the-Shelf Smart Devices. Proceedings of the 2020 17th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), Como, Italy.","DOI":"10.1109\/SECON48991.2020.9158416"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Abe, T., Shizuki, B., and Tanako, J. (2016, January 7\u201312). Input Techniques to the Surface around a Smartphone using a Magnet Attached on a Stylus. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2851581.2892376"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hwang, S., Bianchi, A., Ahn, M., and Wohn, K. (2013, January 27\u201330). MagPen: Magnetically Driven Pen Interactions on and around Conventional Smartphones. Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, Munich, Germany.","DOI":"10.1145\/2493190.2493194"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yoon, S.h., Huo, K., and Ramani, K. (2016, January 14\u201317). TMotion: Embedded 3D Mobile Input using Magnetic Sensing Technique. Proceedings of the TEI\u201916: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, Eindhoven, The Netherlands.","DOI":"10.1145\/2839462.2839463"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sun, K., Yu, C., Shi, W., Liu, L., and Shi, Y. (2018, January 14\u201317). Lip-interact: Improving mobile device interaction with silent speech commands. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, Berlin, Germany.","DOI":"10.1145\/3242587.3242599"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mayer, S., Le, H.V., and Henze, N. (2017, January 17\u201320). Estimating the finger orientation on capacitive touchscreens using convolutional neural networks. Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces, Brighton, UK.","DOI":"10.1145\/3132272.3134130"},{"key":"ref_27","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_29","unstructured":"Zaremba, W., Sutskever, I., and Vinyals, O. (2014). Recurrent neural network regularization. arXiv."},{"key":"ref_30","unstructured":"iFixit (2021, March 27). Samsung Galaxy S9+ Teardown. Available online: https:\/\/www.ifixit.com\/Teardown\/Samsung+Galaxy+S9++Teardown\/104308."},{"key":"ref_31","unstructured":"AKM (2021, March 27). AK09916 3-Axis Electronic Compass. Available online: https:\/\/datasheetspdf.com\/pdf-file\/1074997\/AKM\/AK09916\/1."},{"key":"ref_32","unstructured":"Soukoreff, R.W., and MacKenzie, I.S. (May, January 27). Metrics for Text Entry Research: An Evaluation of MSD and KSPC, and a New Unified Error Metric. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"MacKenzie, I.S., and Soukoreff, R.W. (2003, January 5\u201310). Phrase Sets for Evaluating Text Entry Techniques. Proceedings of the CHI \u201903 Extended Abstracts on Human Factors in Computing Systems, Ft. Lauderdale, FL, USA.","DOI":"10.1145\/765891.765971"},{"key":"ref_34","unstructured":"NASA (2021, March 27). TLX: Task Load Index, Available online: https:\/\/humansystems.arc.nasa.gov\/groups\/tlx\/."},{"key":"ref_35","unstructured":"Sauro, J. (2021, April 15). 10 Things to Know about the NASA TLX. Available online: https:\/\/measuringu.com\/nasa-tlx\/."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/0734-189X(85)90016-7","article-title":"Topological structural analysis of digitized binary images by border following","volume":"30","author":"Suzuki","year":"1985","journal-title":"Comput. Vision Graph. Image Process."},{"key":"ref_37","unstructured":"Toussaint, G.T. (1983, January 24\u201326). Solving geometric problems with the rotating calipers. Proceedings of the MELECON \u201983\u2014Mediterranean Electrotechnical Conference, Athens, Greece."},{"key":"ref_38","first-page":"1089","article-title":"No unbiased estimator of the variance of k-fold cross-validation","volume":"5","author":"Bengio","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_39","unstructured":"Agarap, A.F. (2018). Deep learning using rectified linear units (relu). arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Han, J., and Moraga, C. (1995, January 7\u20139). The influence of the sigmoid function parameters on the speed of backpropagation learning. Proceedings of the International Workshop on Artificial Neural Networks, Malaga-Torremolinos, Spain.","DOI":"10.1007\/3-540-59497-3_175"},{"key":"ref_41","unstructured":"Saito, K., Ushiku, Y., Harada, T., and Saenko, K. (2017). Adversarial dropout regularization. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Vanderdonckt, J., Roselli, P., and P\u00e9rez-Medina, J.L. (2018, January 16\u201320). !FTL, an Articulation-Invariant Stroke Gesture Recognizer with Controllable Position, Scale, and Rotation Invariances. Proceedings of the 20th ACM International Conference on Multimodal Interaction, Boulder, CO, USA.","DOI":"10.1145\/3242969.3243032"},{"key":"ref_43","unstructured":"ISO (2012). Ergonomics of Human-System Interaction\u2014Part 411: Evaluation Methods for the Design of Physical Input Devices, ISO. Report Number ISO\/TS 9241-411:2012."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/3087\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:58:39Z","timestamp":1720745919000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/3087"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,28]]},"references-count":43,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["s21093087"],"URL":"https:\/\/doi.org\/10.3390\/s21093087","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,4,28]]}}}