Abstract
A variety of sensors are built into intelligent mobile devices. However, these sensors can be used as side channels for inferring information. Researchers have shown that some touchscreen information, such as PIN and unlock pattern, can be speculated by background applications with motion sensors. Those attacks mainly focus on the restricted-area input interface (e.g., virtual keyboard). To date, the privacy risk in the unrestricted-area input interface does not receive sufficient attention.
In this paper, we investigate such privacy risk and design an unrestricted-area information speculation framework, called Handwritten Information Awareness (HIAWare). HIAWare exploits the sensors’ signals that are affected by handwriting actions to speculate the handwritten characters. To alleviate the impact of different handwriting habits, we utilize the generality patterns of characters. Furthermore, to mitigate the impact of holding posture in handwriting, we propose a user-independent posture-aware approach. As a result, HIAWare can attack any victim without obtaining the victim’s information in advance. The experiments show that the speculation accuracy of HIAWare is close to 90.0%, demonstrating the viability of HIAWare.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Android Developers: Motion sensors — android developers. https://developer.android.com/guide/topics/sensors/sensors_motion
Aviv, A.J., Sapp, B., Blaze, M., Smith, J.M.: Practicality of accelerometer side channels on smartphones. In: Proceedings of ACSAC, pp. 41–50 (2012)
Cai, L., Chen, H.: Touchlogger: inferring keystrokes on touch screen from smartphone motion. In: Proceedings of HotSec (2011)
Chen, D., et al.: Magleak: a learning-based side-channel attack for password recognition with multiple sensors in IIoT environment. IEEE Trans. Ind. Inform. (2020)
Chen, J., Fang, Y., He, K., Du, R.: Charge-depleting of the batteries makes smartphones recognizable. In: Proceedings of ICPADS, pp. 33–40 (2017)
Chen, Y., Jin, X., Sun, J., Zhang, R., Zhang, Y.: POWERFUL: mobile app fingerprinting via power analysis. In: Proceedings of INFOCOM, pp. 1–9 (2017)
Chen, Z., Zhu, Q., Soh, Y.C., Zhang, L.: Robust human activity recognition using smartphone sensors via CT-PCA and Misc SVM. IEEE Trans. Ind. Inform. 13(6), 3070–3080 (2017)
Du, H., Li, P., Zhou, H., Gong, W., Luo, G., Yang, P.: WordRecorder: accurate acoustic-based handwriting recognition using deep learning. In: Proceedings of INFOCOM, pp. 1448–1456 (2018)
Hafez, A.: Information inference based on barometer sensor in android devices. dissertation, University of Alberta (2020). https://era.library.ualberta.ca/items/15d8d051-45ab-4b1f-ba8a-005688e92f05
Javed, A.R., Beg, M.O., Asim, M., Baker, T., Al-Bayatti, A.H.: AlphaLogger: detecting motion-based side-channel attack using smartphone keystrokes. J. Ambient Intell. Humanized Comput. 1–14 (2020). https://doi.org/10.1007/s12652-020-01770-0
Mehrnezhad, M., Toreini, E., Shahandashti, S.F., Hao, F.: Stealing PINs via mobile sensors: actual risk versus user perception. Int. J. Inf. Secur. 17(3), 291–313 (2017). https://doi.org/10.1007/s10207-017-0369-x
Mehrnezhad, M., Toreini, E., Shahandashti, S.F., Hao, F.: Touchsignatures: identification of user touch actions and pins based on mobile sensor data via javascript. J. Inf. Sec. Appl. 26, 23–38 (2016)
Ping, D., Sun, X., Mao, B.: TextLogger: inferring longer inputs on touch screen using motion sensors. In: Proceedings of WiSec, pp. 24:1–24:12 (2015)
Qimai: Apple store app downloads analysis (2019). https://www.qimai.cn/
Qin, Y., Yue, C.: Website fingerprinting by power estimation based side-channel attacks on Android 7. In: Proceedings of TrustCom, pp. 1030–1039 (2018)
Quispe, K.G.M., Lima, W.S., Batista, D.M., Souto, E.: MBOSS: a symbolic representation of human activity recognition using mobile sensors. Sensors 18(12), 4354 (2018)
Schmitt, E., Voigt-Antons, J.-N.: Predicting tap locations on touch screens in the field using accelerometer and gyroscope sensor readings. In: Moallem, A. (ed.) HCII 2020. LNCS, vol. 12210, pp. 637–651. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50309-3_43
Spreitzer, R., Moonsamy, V., Korak, T., Mangard, S.: Systematic classification of side-channel attacks: a case study for mobile devices. IEEE Commun. Surv. Tutorials 20(1), 465–488 (2018)
Spreitzer, R., Kirchengast, F., Gruss, D., Mangard, S.: ProcHarvester: fully automated analysis of procfs side-channel leaks on Android. In: Proceedings of ASIACCS, pp. 749–763 (2018)
Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Patt. Recogn. Lett. 119, 3–11 (2019)
Xu, Z., Bai, K., Zhu, S.: Taplogger: inferring user inputs on smartphone touchscreens using on-board motion sensors. In: Proceedings of WiSec, pp. 113–124 (2012)
Yu, T., Jin, H., Nahrstedt, K.: Writinghacker: audio based eavesdropping of handwriting via mobile devices. In: Proceedings of UbiComp, pp. 463–473 (2016)
Zhao, R., Yue, C., Han, Q.: Sensor-based mobile web cross-site input inference attacks and defenses. IEEE Trans. Inf. Forensics Secur. 14(1), 75–89 (2019)
Zhou, M., Wang, Q., Yang, J., Li, Q., Xiao, F., Wang, Z., Chen, X.: Patternlistener: cracking android pattern lock using acoustic signals. In: Proceedings of CCS, pp. 1775–1787 (2018)
Acknowledgments
This research was supported in part by the National Natural Science Foundation of China under grants No. 61772383, U1836202, 62076187.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J., Jiang, P., He, K., Zeng, C., Du, R. (2021). HIAWare: Speculate Handwriting on Mobile Devices with Built-In Sensors. In: Gao, D., Li, Q., Guan, X., Liao, X. (eds) Information and Communications Security. ICICS 2021. Lecture Notes in Computer Science(), vol 12918. Springer, Cham. https://doi.org/10.1007/978-3-030-86890-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-86890-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86889-5
Online ISBN: 978-3-030-86890-1
eBook Packages: Computer ScienceComputer Science (R0)