Reinforcement learning of non-additive joint steganographic embedding costs with attention mechanism | Science China Information Sciences Skip to main content
Springer Nature Link
Log in

Reinforcement learning of non-additive joint steganographic embedding costs with attention mechanism

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Image steganography is the art and science of secure communication by concealing information within digital images. In recent years, the techniques of steganographic cost learning have developed rapidly. Although the existing methods can learn satisfactory additive costs, the interplay of different pixels’ embedding impacts has not been considered, so the potential of learning may not be fully exploited. To overcome this limitation, in this paper, a reinforcement learning paradigm called JoPoL (joint policy learning) is proposed to extend the idea of additive cost learning to a non-additive situation. JoPoL aims to capture the interactions within pixel blocks by defining embedding policies and evaluating contributions of embedding impacts on a block level rather than a pixel level. Then, a policy network is utilized to learn optimal joint embedding policies for pixel blocks through interactions with the environment. Afterwards, these policies can be converted into joint embedding costs for practical message embedding. The structure of the policy network is designed with an effective attention mechanism and incorporated with the domain knowledge derived from traditional non-additive steganographic methods. The environment is responsible for assigning rewards according to the impacts of the sampled joint embedding actions, which are evaluated by the gradient information of a neural network-based steganalyzer. Experimental results show that the proposed non-additive method JoPoL significantly outperforms the existing additive methods against both feature-based and CNN-based steganalzyers over different payloads.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Shen C X, Zhang H G, Feng D G, et al. Survey of information security. Sci China Ser F-Inf Sci, 2007, 50: 273–298

    Article  Google Scholar 

  2. Filler T, Judas J, Fridrich J. Minimizing additive distortion in steganography using syndrome-trellis codes. IEEE Trans Inform Forensic Secur, 2011, 6: 920–935

    Article  Google Scholar 

  3. Holub V, Fridrich J. Designing steganographic distortion using directional filters. In: Proceedings of IEEE International Workshop on Information Forensics and Security, Costa Adeje, 2012. 234–239

  4. Holub V, Fridrich J, Denemark T. Universal distortion function for steganography in an arbitrary domain. EURASIP J Info Secur, 2014, 2014: 1

    Article  Google Scholar 

  5. Li B, Wang M, Huang J W, et al. A new cost function for spatial image steganography. In: Proceedings of IEEE International Conference on Image Processing, Paris, 2014. 4026–4210

  6. Fridrich J, Kodovsky J. Multivariate Gaussian model for designing additive distortion for steganography. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, 2013. 2949–2953

  7. Sedighi V, Cogranne R, Fridrich J. Content-adaptive steganography by minimizing statistical detectability. IEEE Trans Inform Forensic Secur, 2016, 11: 221–234

    Article  Google Scholar 

  8. Qin X H, Li B, Huang J W. A new spatial steganographic scheme by modeling image residuals with multivariate Gaussian model. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, 2019. 2617–2621

  9. Pevny T, Filler T, Bas P. Using high-dimensional image models to perform highly undetectable steganography. In: Proceedings of International Workshop on Information Hiding, Calgary, 2010. 161–177

  10. Kouider S, Chaumont M, Puech W. Adaptive steganography by oracle (ASO). In: Proceedings of IEEE International Conference on Multimedia & Expo, San Jose, 2013. 1–6

  11. Li B, Wang M, Li X L, et al. A strategy of clustering modification directions in spatial image steganography. IEEE Trans Inform Forensic Secur, 2015, 10: 1905–1917

    Article  Google Scholar 

  12. Denemark T, Fridrich J. Improving steganographic security by synchronizing the selection channel. In: Proceedings of the 3rd ACM Workshop on Information Hiding and Multimedia Security, Portland Oregon, 2015. 5–14

  13. Zhang W M, Zhang Z, Zhang L L, et al. Decomposing joint distortion for adaptive steganography. IEEE Trans Circ Syst Video Technol, 2017, 27: 2274–2280

    Article  Google Scholar 

  14. Li W X, Zhang W M, Chen K J, et al. Defining joint distortion for JPEG steganography. In: Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security, 2018. 5–16

  15. Wang Y F, Zhang W M, Li W X, et al. Non-additive cost functions for JPEG steganography based on block boundary maintenance. IEEE Trans Inform Forensic Secur, 2021, 16: 1117–1130

    Article  Google Scholar 

  16. Mo X B, Tan S Q, Li B, et al. MCTSteg: a Monte Carlo tree search-based reinforcement learning framework for universal non-additive steganography. IEEE Trans Inform Forensic Secur, 2021, 16: 4306–4320

    Article  Google Scholar 

  17. Kodovsky J, Fridrich J, Holub V. Ensemble classifiers for steganalysis of digital media. IEEE Trans Inform Forensic Secur, 2012, 7: 432–444

    Article  Google Scholar 

  18. Cogranne R, Sedighi V, Fridrich J, et al. Is ensemble classifier needed for steganalysis in high-dimensional feature spaces? In: Proceedings of IEEE International Workshop on Information Forensics & Security, Rome, 2015. 1–6

  19. Luo X Y, Liu F L, Yang C F, et al. Modification ratio estimation for a category of adaptive steganography. Sci China Inf Sci, 2010, 53: 2472–2484

    Article  MathSciNet  Google Scholar 

  20. Fridrich J, Kodovsky J. Rich models for steganalysis of digital images. IEEE Trans Inform Forensic Secur, 2012, 7: 868–882

    Article  Google Scholar 

  21. Tang W X, Li H D, Luo W Q, et al. Adaptive steganalysis against WOW embedding algorithm. In: Proceedings of ACM Workshop Information Hiding and Multimedia Security, Salzburg, 2014. 91–96

  22. Denemark T, Sedighi V, Holub V, et al. Selection-channel-aware rich model for steganalysis of digital images. In: Proceedings of IEEE International Workshop on Information Forensics and Security (WIFS), Atlanta, 2014. 48–53

  23. Tang W X, Li H D, Luo W Q, et al. Adaptive steganalysis based on embedding probabilities of pixels. IEEE Trans Inform Forensic Secur, 2016, 11: 734–745

    Article  Google Scholar 

  24. Li B, Li Z P, Zhou S J, et al. New steganalytic features for spatial image steganography based on derivative filters and threshold LBP operator. IEEE Trans Inform Forensic Secur, 2018, 13: 1242–1257

    Article  Google Scholar 

  25. Zhang H, Ping X J, Xu M K, et al. Steganalysis by subtractive pixel adjacency matrix and dimensionality reduction. Sci China Inf Sci, 2014, 57: 048101

    Google Scholar 

  26. Ren Y L, Zhang X P, Gu D W, et al. Efficient outsourced extraction of histogram features over encrypted images in cloud. Sci China Inf Sci, 2021, 64: 139105

    Article  Google Scholar 

  27. Luo X Y, Liu F L, Yang C F, et al. Image universal steganalysis based on best wavelet packet decomposition. Sci China Inf Sci, 2010, 53: 634–647

    Article  MathSciNet  Google Scholar 

  28. Zhong K, Feng G R, Shen L Q, et al. Deep learning for steganalysis based on filter diversity selection. Sci China Inf Sci, 2018, 61: 129105

    Article  Google Scholar 

  29. Tan S Q, Li B. Stacked convolutional auto-encoders for steganalysis of digital images. In: Proceedings of Signal and Information Processing Association Annual Summit and Conference (APSIPA), Siem Reap, 2014. 1–4

  30. Qian Y L, Dong J, Wang W, et al. Deep learning for steganalysis via convolutional neural networks. In: Proceedings of SPIE, San Francisco, 2015

  31. Xu G S, Wu H Z, Shi Y Q. Structural design of convolutional neural networks for steganalysis. IEEE Signal Process Lett, 2016, 23: 708–712

    Article  Google Scholar 

  32. Yedroudj M, Comby F, Chaumont M. Yedroudj-Net: an efficient CNN for spatial steganalysis. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, 2018. 15–20

  33. Boroumand M, Chen M, Fridrich J. Deep residual network for steganalysis of digital images. IEEE Trans Inform Forensic Secur, 2019, 14: 1181–1193

    Article  Google Scholar 

  34. Xu G S. Deep convolutional neural network to detect J-UNIWARD. In: Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, New York, 2017. 67–73

  35. Ye J, Ni J Q, Yi Y. Deep learning hierarchical representations for image steganalysis. IEEE Trans Inform Forensic Secur, 2017, 12: 2545–2557

    Article  Google Scholar 

  36. You W K, Zhang H, Zhao X F. A siamese CNN for image steganalysis. IEEE Trans Inform Forensic Secur, 2021, 16: 291–306

    Article  Google Scholar 

  37. Li Q J, Feng G R, Ren Y L, et al. Embedding probability guided network for image steganalysis. IEEE Signal Process Lett, 2021, 28: 1095–1099

    Article  Google Scholar 

  38. Tang W X, Tan S Q, Li B, et al. Automatic steganographic distortion learning using a generative adversarial network. IEEE Signal Process Lett, 2017, 24: 1547–1551

    Article  Google Scholar 

  39. Yang J H, Ruan D Y, Huang J W, et al. An embedding cost learning framework using GAN. IEEE Trans Inform Forensic Secur, 2020, 15: 839–851

    Article  Google Scholar 

  40. Tang W W, Li B, Barni M, et al. An automatic cost learning framework for image steganography using deep reinforcement learning. IEEE Trans Inform Forensic Secur, 2021, 16: 952–967

    Article  Google Scholar 

  41. Tang W X, Li B, Barni M, et al. Improving cost learning for JPEG steganography by exploiting JPEG domain knowledge. IEEE Trans Circ Syst Video Technol, 2021. doi: https://doi.org/10.48550/arXiv.2105.03867

  42. Tang W X, Li B, Tan S Q, et al. CNN-based adversarial embedding for image steganography. IEEE Trans Inform Forensic Secur, 2019, 14: 2074–2087

    Article  Google Scholar 

  43. Bernard S, Bas P, Klein J, et al. Explicit optimization of min max steganographic game. IEEE Trans Inform Forensic Secur, 2021, 16: 812–823

    Article  Google Scholar 

  44. Li W X, Zhang W M, Li L, et al. Designing near-optimal steganographic codes in practice based on polar codes. IEEE Trans Commun, 2020, 68: 3948–3962

    Article  Google Scholar 

  45. Mnih V, Heess N, Graves A, et al. Recurrent models of visual attention. 2014. ArXiv:1406.6247

  46. Xu K, Ba J, Kiros R, et al. Show, attend and tell: neural image caption generation with visual attention. In: Proceedings of International Conference on Machine Learning, 2015. 2048–2057

  47. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Proceedings of International Conference on Learning Representations, 2015

  48. Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018. 7132–7141

  49. Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision, 2018. 286–301

  50. Wang Q, Wu B G, Zhu P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020. 11531–11539

  51. Bas P, Filler T, Pevny T. Break our steganographic system: the ins and outs of organizing BOSS. In: Proceedings of International Workshop on Information Hiding, Prague, 2011. 59–70

  52. Bas P, Furon T. Bows-2. 2007. http://bows2.ec-lille.fr

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 62002075, 61872244, 61872099, U19B2022), Guangdong Basic and Applied Basic Research Foundation (Grant No. 2019B151502001), and Shenzhen R&D Program (Grant No. JCYJ20200109105008228).

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, 510006, China

    Weixuan Tang

  2. Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen Key Laboratory of Media Security, Shenzhen University, Shenzhen, 518060, China

    Bin Li, Weixiang Li & Jiwu Huang

  3. Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, 518060, China

    Bin Li & Jiwu Huang

  4. School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, 510006, China

    Yuangen Wang

Authors
  1. Weixuan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weixiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuangen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiwu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, W., Li, B., Li, W. et al. Reinforcement learning of non-additive joint steganographic embedding costs with attention mechanism. Sci. China Inf. Sci. 66, 132305 (2023). https://doi.org/10.1007/s11432-021-3453-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3453-5

Keywords

Search

Navigation