Abstract
Completely automated public turing test to tell computers and humans apart (CAPTCHA) is widely used to prevent malicious automated attacks on various online services. Text- and image-CAPTCHAs have shown broader acceptability due to usability and security factors. However, recent progress in deep learning implies that text-CAPTCHAs can easily be exposed to various fraudulent attacks. Thus, image-CAPTCHAs are getting research attention to enhance usability and security. In this work, the neural-style transfer (NST) is adapted for designing an image-CAPTCHA algorithm to enhance security while maintaining human performance. In NST-rendered image-CAPTCHAs, existing methods inquire a user to identify or localize the salient object (e.g., content) which is solvable effortlessly by off-the-shelf intelligent tools. Contrarily, we propose a Style Matching CAPTCHA (SMC) that asks a user to select the style image which is applied in the NST method. A user can solve a random SMC challenge by understanding the semantic correlation between the content and style output as a cue. The performance in solving SMC is evaluated based on the 1368 responses collected from 152 participants through a web-application. The average solving accuracy in three sessions is 95.61%; and the average response time for each challenge per user is 6.52 s, respectively. Likewise, a Smartphone Application (SMC-App) is devised using the proposed method. The average solving accuracy through SMC-App is 96.33%, and the average solving time is 5.13 s. To evaluate the vulnerability of SMC, deep learning-based attack schemes using Convolutional Neural Networks (CNN), such as ResNet-50 and Inception-v3 are simulated. The average accuracy of attacks considering various studies on SMC using ResNet-50 and Inception-v3 is 37%, which is improved over existing methods. Moreover, in-depth security analysis, experimental insights, and comparative studies imply the suitability of the proposed SMC.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Acien, A., Morales, A., Fierrez, J., Vera-Rodriguez, R., Delgado-Mohatar, O.: Becaptcha: Behavioral bot detection using touchscreen and mobile sensors benchmarked on humidb. Eng. Appl. Artif. Intell. 98, 104058 (2021). https://doi.org/10.1016/j.engappai.2020.104058
Alnfiai, M., Alassery, F.: Tapcaptcha: non-visual captcha on touchscreens for visually impaired people. Journal on Multimodal User Interfaces pp. 1–14 (2022). https://doi.org/10.1007/s12193-022-00394-2
Baird, H.S., Bentley, J.L.: Implicit captchas. In: Document Recognition and Retrieval XII, vol. 5676, pp. 191–196. International Society for Optics and Photonics (2005). https://doi.org/10.1117/12.590944
Bera, A., Bhattacharjee, D., Nasipuri, M.: Person recognition using alternative hand geometry. International Journal of Biometrics 6(3), 231–247 (2014). https://doi.org/10.1504/IJBM.2014.064403
Bera, A., Bhattacharjee, D., Nasipuri, M.: Hand biometric verification with hand image-based captcha. In: Advanced Computing and Systems for Security, pp. 3–18. Springer (2018). https://doi.org/10.1007/978-981-10-8180-4_1
Bera, A., Bhattacharjee, D., Shum, H.P.: Two-stage human verification using handcaptcha and anti-spoofed finger biometrics with feature selection. Expert Syst. Appl. 171, 114583 (2021). https://doi.org/10.1016/j.eswa.2021.114583
Breitinger, F., Tully-Doyle, R., Hassenfeldt, C.: A survey on smartphone user’s security choices, awareness and education. Computers & Security 88, 101647 (2020). https://doi.org/10.1016/j.cose.2019.101647
Chen, F., Wang, Y., Xu, S., Wang, F., Sun, F., Jia, X.: Style transfer network for complex multi-stroke text. Multimedia Systems pp. 1–10 (2023). https://doi.org/10.1007/s00530-023-01047-4
Chen, H., Jiang, B., Chen, H.: Stylecaptcha: Captcha based on stylized images to defend against deep networks. In: Proceedings of the 2020 ACM-IMS on Foundations of Data Science Conference, pp. 161–170 (2020). https://doi.org/10.1145/3412815.3416895
Chen, H., Zhao, L., Wang, Z., Zhang, H., Zuo, Z., Li, A., Xing, W., Lu, D.: Dualast: Dual style-learning networks for artistic style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 872–881 (2021)
Chen, H.Y., Fang, I., Cheng, C.M., Chiu, W.C., et al.: Self-contained stylization via steganography for reverse and serial style transfer. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2163–2171 (2020)
Cheng, Z., Gao, H., Liu, Z., Wu, H., Zi, Y., Pei, G.: Image-based captchas based on neural style transfer. IET Inf. Secur. 13(6), 519–529 (2019). https://doi.org/10.1049/iet-ifs.2018.5036
Conti, M., Guarisco, C., Spolaor, R.: Captchastar! a novel captcha based on interactive shape discovery. In: International Conference on Applied Cryptography and Network Security, pp. 611–628. Springer (2016). https://doi.org/10.1007/978-3-319-39555-5_33
Conti, M., Pajola, L., Tricomi, P.P.: Captcha attack: Turning captchas against humanity. arXiv preprint arXiv:2201.04014 (2022). https://doi.org/10.48550/arXiv.2201.04014
Dang, V.N., Galati, F., Cortese, R., Di Giacomo, G., Marconetto, V., Mathur, P., Lekadir, K., Lorenzi, M., Prados, F., Zuluaga, M.A.: Vessel-captcha: an efficient learning framework for vessel annotation and segmentation. Med. Image Anal. 75, 102263 (2022). https://doi.org/10.1016/j.media.2021.102263
Datta, R., Li, J., Wang, J.Z.: Imagination: a robust image-based captcha generation system. In: Proceedings of the 13th annual ACM international conference on Multimedia, pp. 331–334. ACM (2005). https://doi.org/10.1145/1101149.1101218
Deng, X.: Enhancing image quality via style transfer for single image super-resolution. IEEE Signal Process. Lett. 25(4), 571–575 (2018). https://doi.org/10.1109/LSP.2018.2805809
Elson, J., Douceur, J.R., Howell, J., Saul, J.: Asirra: a captcha that exploits interest-aligned manual image categorization. CCS 7, 366–374 (2007). https://doi.org/10.1145/1315245.1315291
Feng, Y., Cao, Q., Qi, H., Ruoti, S.: Sencaptcha: A mobile-first captcha using orientation sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4(2), 1–26 (2020). https://doi.org/10.1145/3397312
Gao, H., Cao, F., Zhang, P.: Annulus: A novel image-based captcha scheme. In: 2016 IEEE Region 10 Conference (TENCON), pp. 464–467. IEEE (2016). https://doi.org/10.1109/TENCON.2016.7848042
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2414–2423 (2016)
Godi, M., Joppi, C., Giachetti, A., Pellacini, F., Cristani, M.: Texel-att: Representing and classifying element-based textures by attributes. arXiv preprint arXiv:1908.11127 (2019). https://doi.org/10.48550/arXiv.1908.11127
Golle, P.: Machine learning attacks against the asirra captcha. In: Proceedings of the 15th ACM conference on Computer and communications security, pp. 535–542 (2008). https://doi.org/10.1145/1455770.1455838
Goswami, G., Powell, B.M., Vatsa, M., Singh, R., Noore, A.: Facedcaptcha: Face detection based color image captcha. Futur. Gener. Comput. Syst. 31, 59–68 (2014). https://doi.org/10.1016/j.future.2012.08.013
Goswami, G., Powell, B.M., Vatsa, M., Singh, R., Noore, A.: Fr-captcha: Captcha based on recognizing human faces. PLoS ONE 9(4), e91708 (2014). https://doi.org/10.1371/journal.pone.0091708
Gougeon, T., Lacharme, P.: A simple attack on captchastar. In: International Conference on Information Systems Security and Privacy, pp. 66–85. Springer (2018). https://doi.org/10.1007/978-3-030-25109-3_4
Gupta, A., Johnson, J., Alahi, A., Fei-Fei, L.: Characterizing and improving stability in neural style transfer. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4067–4076 (2017). https://doi.org/10.48550/arXiv.1705.02092
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)
Hosam, O., Abousamra, R.: Enhancing deep training of image landmarking with image captcha. In: 2022 8th International Conference on Information Technology Trends (ITT), pp. 88–93. IEEE (2022). https://doi.org/10.1109/ITT56123.2022.9863967
Hou, X., Shen, L., Sun, K., Qiu, G.: Deep feature consistent variational autoencoder. In: 2017 IEEE winter conference on applications of computer vision (WACV), pp. 1133–1141. IEEE (2017). https://doi.org/10.1109/WACV.2017.131
Jia, X., Xiao, J., Wu, C.: Tics: text–image-based semantic captcha synthesis via multi-condition adversarial learning. The Visual Computer pp. 1–13 (2022). https://doi.org/10.1007/s00371-021-02061-1
Jin, X., Han, R., Duan, Y., Ning, N., Li, X.: Ar captcha: Recognizing robot by augmented reality. Concurrency and Computation: Practice and Experience 33(15), e5585 (2021). https://doi.org/10.1002/cpe.5585
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., Song, M.: Neural style transfer: A review. IEEE Trans. Visual Comput. Graphics 26(11), 3365–3385 (2019). https://doi.org/10.1109/TVCG.2019.2921336
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp. 694–711. Springer (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Kong, X., Deng, Y., Tang, F., Dong, W., Ma, C., Chen, Y., He, Z., Xu, C.: Exploring the temporal consistency of arbitrary style transfer: A channelwise perspective. IEEE Transactions on Neural Networks and Learning Systems (2023). https://doi.org/10.1109/TNNLS.2022.3230084
Kumar, M., Jindal, M., Kumar, M.: Distortion, rotation and scale invariant recognition of hollow hindi characters. Sādhanā 47(2), 92 (2022). https://doi.org/10.1007/s12046-022-01847-w
Kumar, M., Jindal, M., Kumar, M.: A systematic survey on captcha recognition: types, creation and breaking techniques. Archives of Computational Methods in Engineering 29(2), 1107–1136 (2022). https://doi.org/10.1007/s11831-021-09608-4
Kumar, M., Jindal, M., Kumar, M.: An efficient technique for breaking of coloured hindi captcha. Soft Computing pp. 1–26 (2023). https://doi.org/10.1007/s00500-023-07844-3
Kumar, M., Jindal, M.K., Kumar, M.: A novel attack on monochrome and greyscale devanagari captchas. Transactions on Asian and Low-Resource Language Information Processing 20(4), 1–30 (2021). https://doi.org/10.1145/3439798
Kumar, M., Jindal, M.K., Kumar, M.: Design of innovative captcha for hindi language. Neural Computing and Applications pp. 1–36 (2022). https://doi.org/10.1007/s00521-021-06686-0
Lewis, J.: Fast normalized cross-correlation, 1995. In: Vision Interface, vol. 2010, pp. 120–123 (2010)
Li, C., Chen, X., Wang, H., Wang, P., Zhang, Y., Wang, W.: End-to-end attack on text-based captchas based on cycle-consistent generative adversarial network. Neurocomputing 433, 223–236 (2021). https://doi.org/10.1016/j.neucom.2020.11.057
Liu, X.C., Yang, Y.L., Hall, P.: Learning to warp for style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3702–3711 (2021)
Ma, Y., Zhao, C., Li, X., Basu, A.: Rast: Restorable arbitrary style transfer via multi-restoration. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 331–340 (2023)
Mallika, Ubhi, J.S., Aggarwal, A.K.: Neural style transfer for image within images and conditional gans for destylization. Journal of Visual Communication and Image Representation 85, 103483 (2022). https://doi.org/10.1016/j.jvcir.2022.103483
Mori, G., Malik, J.: Recognizing objects in adversarial clutter: Breaking a visual captcha. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., vol. 1, pp. I–I. IEEE (2003). https://doi.org/10.1109/CVPR.2003.1211347
Okada, M., Matsuyama, S.: New captcha for smartphones and tablet pc. In: 2012 IEEE Consumer Communications and Networking Conference (CCNC), pp. 34–35. IEEE (2012). https://doi.org/10.1109/CCNC.2012.6181038
Osadchy, M., Hernandez-Castro, J., Gibson, S., Dunkelman, O., Pérez-Cabo, D.: No bot expects the deepcaptcha! introducing immutable adversarial examples with applications to captcha. Cryptology ePrint Archive (2016)
Polakis, I., Ilia, P., Maggi, F., Lancini, M., Kontaxis, G., Zanero, S., Ioannidis, S., Keromytis, A.D.: Faces in the distorting mirror: Revisiting photo-based social authentication. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, pp. 501–512 (2014). https://doi.org/10.1145/2660267.2660317
Rathor, V.S., Garg, B., Patil, M., Sharma, G.: Security analysis of image captcha using a mask r-cnn-based attack model. Int. J. Ad Hoc Ubiquitous Comput. 36(4), 238–247 (2021)
Ray, P., Giri, D., Kumar, S., Sahoo, P.: Fp-captcha: An improved captcha design scheme based on face points. In: International Conference on Information Technology and Applied Mathematics, pp. 218–233. Springer (2019). https://doi.org/10.1007/978-3-030-34152-7_17
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)
Ruder, M., Dosovitskiy, A., Brox, T.: Artistic style transfer for videos. In: German conference on pattern recognition, pp. 26–36. Springer (2016). https://doi.org/10.1007/978-3-319-45886-1_3
Rui, Y., Liu, Z.: Artifacial: Automated reverse turing test using facial features. Multimedia Syst. 9(6), 493–502 (2004). https://doi.org/10.1145/957013.957075
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp. 618–626 (2017)
Shet, V.: Are you a robot? introducing no captcha recaptcha. Google Security Blog 3, 12 (2014)
Shi, C., Xu, X., Ji, S., Bu, K., Chen, J., Beyah, R., Wang, T.: Adversarial captchas. IEEE Transactions on. Cybernetics (2021). https://doi.org/10.1109/TCYB.2021.3071395
Shirali-Shahreza, M., Shirali-Shahreza, S.: Captcha for blind people. In: 2007 IEEE International Symposium on Signal Processing and Information Technology, pp. 995–998. IEEE (2007). https://doi.org/10.1109/ISSPIT.2007.4458048
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014). https://doi.org/10.48550/arXiv.1409.1556
Singh, A., Jaiswal, V., Joshi, G., Sanjeeve, A., Gite, S., Kotecha, K.: Neural style transfer: A critical review. IEEE Access (2021). https://doi.org/10.1109/ACCESS.2021.3112996
Sivakorn, S., Polakis, I., Keromytis, A.D.: I am robot:(deep) learning to break semantic image captchas. In: 2016 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 388–403. IEEE (2016). https://doi.org/10.1109/EuroSP.2016.37
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826 (2016)
Tang, M., Gao, H., Zhang, Y., Liu, Y., Zhang, P., Wang, P.: Research on deep learning techniques in breaking text-based captchas and designing image-based captcha. IEEE Trans. Inf. Forensics Secur. 13(10), 2522–2537 (2018). https://doi.org/10.1109/TIFS.2018.2821096
Terada, K., Okabe, Y., Matsumoto, Y.: Is puzzle-based captcha secure against attacks based on cnn? In: 2023 International Conference on Information Networking (ICOIN), pp. 358–362. IEEE (2023). https://doi.org/10.1109/ICOIN56518.2023.10049032
Thompson, N., McGill, T.J., Wang, X.: “security begins at home’’: Determinants of home computer and mobile device security behavior. computers & security 70, 376–391 (2017). https://doi.org/10.1016/j.cose.2017.07.003
Uzun, E., Chung, S.P.H., Essa, I., Lee, W.: rtcaptcha: A real-time captcha based liveness detection system. In: NDSS (2018)
Von Ahn, L., Blum, M., Hopper, N.J., Langford, J.: Captcha: Using hard ai problems for security. In: International Conference on the Theory and Applications of Cryptographic Techniques, pp. 294–311. Springer (2003). https://doi.org/10.1007/3-540-39200-9
Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: recaptcha: Human-based character recognition via web security measures. Science 321(5895), 1465–1468 (2008). https://doi.org/10.1126/science.1160379
Wang, P., Gao, H., Guo, X., Yuan, Z., Nian, J.: Improving the security of audio captchas with adversarial examples. IEEE Trans. Dependable Secure Comput. (2023). https://doi.org/10.1109/TDSC.2023.3236367
Wang, P., Gao, H., Shi, Z., Yuan, Z., Hu, J.: Simple and easy: transfer learning-based attacks to text captcha. IEEE Access 8, 59044–59058 (2020). https://doi.org/10.1109/ACCESS.2020.2982945
Wang, P., Gao, H., Xiao, C., Guo, X., Gao, Y., Zi, Y.: Extended research on the security of visual reasoning captcha. IEEE Transactions on Dependable and Secure Computing pp. 1–17 (2023). https://doi.org/10.1109/TDSC.2023.3238408
Wang, P., Gao, H., Xiao, C., Guo, X., Gao, Y., Zi, Y.: Extended research on the security of visual reasoning captcha. IEEE Trans. Dependable Secure Comput. (2023). https://doi.org/10.1109/TDSC.2023.3238408
Wang, P., Li, Y., Vasconcelos, N.: Rethinking and improving the robustness of image style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 124–133 (2021). https://doi.org/10.48550/arXiv.2104.05623
Wang, Y., Wei, Y., Zhang, M., Liu, Y., Wang, B.: Make complex captchas simple: A fast text captcha solver based on a small number of samples. Inf. Sci. 578, 181–194 (2021). https://doi.org/10.1016/j.ins.2021.07.040
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Weichbroth, P., Łysik, Ł: Mobile security: Threats and best practices. Mob. Inf. Syst. 2020, 1–15 (2020). https://doi.org/10.1155/2020/8828078
Xu, X., Liu, L., Li, B.: A survey of captcha technologies to distinguish between human and computer. Neurocomputing 408, 292–307 (2020). https://doi.org/10.1016/j.neucom.2019.08.109
Zhang, J., Tsai, M.Y., Kitchat, K., Sun, M.T., Sakai, K., Ku, W.S., Surasak, T., Thaipisutikul, T.: A secure annuli captcha system. Computers & Security 125, 103025 (2023). https://doi.org/10.1016/j.cose.2022.103025
Zhang, K., Zheng, Y.: Information Security: 7th International Conference, ISC 2004, Palo Alto, CA, USA, September 27-29, 2004, Proceedings, vol. 3225. Springer (2004)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017). https://doi.org/10.1109/TIP.2017.2662206
Zhang, P., Gao, H., Cheng, Z., Cao, F.: Two novel image-based captcha schemes based on visual effects. In: CCF Chinese Conference on Computer Vision, pp. 14–25. Springer (2017). https://doi.org/10.1007/978-981-10-7305-2_2
Zhu, B., Liu, J., Li, Q., Li, S., Xu, N.: Image-based captcha exploiting context in object recognition (2013). US Patent 8,483,518
Zhu, B.B., Yan, J., Li, Q., Yang, C., Liu, J., Xu, N., Yi, M., Cai, K.: Attacks and design of image recognition captchas. In: Proceedings of the 17th ACM conference on Computer and communications security, pp. 187–200. ACM (2010). https://doi.org/10.1145/1866307.1866329
Zi, Y., Gao, H., Cheng, Z., Liu, Y.: An end-to-end attack on text captchas. IEEE Trans. Inf. Forensics Secur. 15, 753–766 (2019). https://doi.org/10.1109/TIFS.2019.2928622
Acknowledgements
The authors would like to thank the Editors and anonymous reviewers for their valuable comments. The authors are also thankful to the volunteers, experts, and participants who have provided their responses and suggestions for this research.
Author information
Authors and Affiliations
Contributions
A. conceptualization, methodology, prepared main manuscript. B. conceptualization, methodology, supervision, writing. C. and D. review and editing. A–D. All the authors reviewed the manuscript
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ray, P., Bera, A., Giri, D. et al. Style matching CAPTCHA: match neural transferred styles to thwart intelligent attacks. Multimedia Systems 29, 1865–1895 (2023). https://doi.org/10.1007/s00530-023-01075-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-023-01075-0