A quantum system control method based on enhanced reinforcement learning | Soft Computing Skip to main content
Springer Nature Link
Log in

A quantum system control method based on enhanced reinforcement learning

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Traditional quantum system control methods often face different constraints, and are easy to cause both leakage and stochastic control errors under the condition of limited resources. Reinforcement learning has been proved as an efficient way to complete the quantum system control task. To learn a satisfactory control strategy under the condition of limited resources, a quantum system control method based on enhanced reinforcement learning (QSC-ERL) is proposed. The states and actions in reinforcement learning are mapped to quantum states and control operations in quantum systems. By using new enhanced neural networks, reinforcement learning can quickly achieve the maximization of long-term cumulative rewards, and a quantum state can be evolved accurately from an initial state to a target state. According to the number of candidate unitary operations, the three-switch control is used for simulation experiments. Compared with other methods, the QSC-ERL achieves close to 1 fidelity learning control of quantum systems, and takes fewer episodes to quantum state evolution under the condition of limited resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Enquiries about data availability should be directed to the authors.

References

  • Abualigah L, Diabat A, Mirjalili S, Abd Elaziz M, Gandomi AH (2021a) The arithmetic optimization algorithm. Comput Methods Appl Mech Eng 376:113609. https://doi.org/10.1016/j.cma.2020.113609

    Article  MathSciNet  MATH  Google Scholar 

  • Abualigah L, Diabat A, Sumari P, Gandomi A (2021b) Applications, deployments, and integration of internet of drones (iod): a review. IEEE Sens J. https://doi.org/10.1109/JSEN.2021.3114266

  • Abualigah L, Elsayed Abd Elaziz M, Sumari P, Geem ZW, Gandomi A (2021c) Reptile search algorithm (rsa): a nature-inspired meta-heuristic optimizer. Expert Syst Appl 191:116158. https://doi.org/10.1016/j.eswa.2021.116158

    Article  Google Scholar 

  • Abualigah L, Yousri D, Abd Elaziz M, Ewees AA, Al-qaness MA, Gandomi AH (2021d) Aquila optimizer: a novel meta-heuristic optimization algorithm. Comput Ind Eng 157:107250. https://doi.org/10.1016/j.cie.2021.107250

    Article  Google Scholar 

  • An Z, Zhou D (2019) Deep reinforcement learning for quantum gate control. EPL (Europhysics Letters) 126(6):60002

    Article  Google Scholar 

  • An Z, Song HJ, He QK, Zhou D (2021) Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning. Phys Rev A 103(1):012404

    Article  Google Scholar 

  • Bukov M (2018) Reinforcement learning for autonomous preparation of floquet-engineered states: inverting the quantum kapitza oscillator. Phys Rev B 98(22):224305

    Article  Google Scholar 

  • Bukov M, Day AG, Sels D, Weinberg P, Polkovnikov A, Mehta P (2018) Reinforcement learning in different phases of quantum control. Phys Rev X 8(3):031086

    Google Scholar 

  • Cárdenas-López FA, Lamata L, Retamal JC, Solano E (2018) Multiqubit and multilevel quantum reinforcement learning with quantum technologies. PLoS ONE 13(7):e0200455

    Article  Google Scholar 

  • Chakrabarti R, Rabitz H (2007) Quantum control landscapes. Int Rev Phys Chem 26(4):671–735

    Article  Google Scholar 

  • Chen C, Dong D, Li HX, Chu J, Tarn TJ (2013) Fidelity-based probabilistic q-learning for control of quantum systems. IEEE Trans Neural Netw Learn Syst 25(5):920–933

    Article  Google Scholar 

  • Chu S (2002) Cold atoms and quantum control. Nature 416(6877):206–210

    Article  Google Scholar 

  • Chunlin C, Frank J, Daoyi D (2012) Hybrid control of uncertain quantum systems via fuzzy estimation and quantum reinforcement learning. In: Proceedings of the 31st Chinese Control Conference, IEEE, pp 7177–7182

  • D’Alessandro D, Dahleh M (2001) Optimal control of two-level quantum systems. IEEE Trans Autom Control 46(6):866–876

    Article  MathSciNet  Google Scholar 

  • Dong D, Chen C, Li H, Tarn TJ (2008) Quantum reinforcement learning. IEEE Trans Syst Man Cybern Part B (Cybernetics) 38(5):1207–1220

    Article  Google Scholar 

  • Dong D, Chen C, Tarn TJ, Pechen A, Rabitz H (2008) Incoherent control of quantum systems with wavefunction-controllable subspaces via quantum reinforcement learning. IEEE Trans Syst Man Cybern Part B (Cybernetics) 38(4):957–962

    Article  Google Scholar 

  • Fang W, Pang L, Yi W (2020) Survey on the application of deep reinforcement learning in image processing. J Artif Intell 2(1):39–58

    Article  Google Scholar 

  • Fösel T, Tighineanu P, Weiss T, Marquardt F (2018) Reinforcement learning with neural networks for quantum feedback. Phys Rev X 8(3):031084

    Google Scholar 

  • Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Hu B, Zhao H, Yang Y, Zhou B, Raj ANJ (2020) Multiple faces tracking using feature fusion and neural network in video. Intell Autom Soft Comput 26(6):1549–1560

    Article  Google Scholar 

  • Li Z, Zhang J, Zhang K, Li Z (2018) Visual tracking with weighted adaptive local sparse appearance model via spatio-temporal context learning. IEEE Trans Image Process 27(9):4478–4489

    Article  MathSciNet  Google Scholar 

  • Ma H, Chen C (2020) Several developments in learning control of quantum systems. In: 2020 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 4165–4172

  • Meng F, Cong S (2022) Control design for state transition of open quantum system. J Phys Conf Series 2183:012005

    Article  Google Scholar 

  • Michael MH, Silveri M, Brierley R, Albert VV, Salmilehto J, Jiang L, Girvin SM (2016) New class of quantum error-correcting codes for a bosonic mode. Phys Rev X 6(3):031006

    Google Scholar 

  • Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

  • Niu MY, Boixo S, Smelyanskiy VN, Neven H (2019) Universal quantum control through deep reinforcement learning. npj Quant Inform 5(1):1–8

    Article  Google Scholar 

  • Palittapongarnpim P, Wittek P, Sanders BC (2017) Robustness of learning-assisted adaptive quantum-enhanced metrology in the presence of noise. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 294–299

  • Patsch S, Maniscalco S, Koch CP (2020) Simulation of open-quantum-system dynamics using the quantum zeno effect. Phys Rev Res 2(2):023133

    Article  Google Scholar 

  • Rabitz H, de Vivie-Riedle R, Motzkus M, Kompa K (2000) Whither the future of controlling quantum phenomena? Science 288(5467):824–828

    Article  Google Scholar 

  • Roslund J, Rabitz H (2009) Gradient algorithm applied to laboratory quantum control. Phys Rev A 79(5):053417

    Article  Google Scholar 

  • Singh SP, Sutton RS (1996) Reinforcement learning with replacing eligibility traces. Mach Learn 22(1):123–158

  • Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press

  • Sutton RS, McAllester DA, Singh SP, Mansour Y (2000) Policy gradient methods for reinforcement learning with function approximation. In: Advances in neural information processing systems, pp 1057–1063

  • Torosov BT, Shore BW, Vitanov NV (2021) Coherent control techniques for two-state quantum systems: a comparative study. Phys Rev A 103(3):033110

    Article  MathSciNet  Google Scholar 

  • Tsubouchi M, Momose T (2008) Rovibrational wave-packet manipulation using shaped midinfrared femtosecond pulses toward quantum computation: optimization of pulse shape by a genetic algorithm. Phys Rev A 77(5):052326

    Article  Google Scholar 

  • Vedaie SS, Palittapongarnpim P, Sanders BC (2018) Reinforcement learning for quantum metrology via quantum control. In: 2018 IEEE photonics society summer topical meeting series (SUM), IEEE, pp 163–164

  • Vrajitoarea A, Huang Z, Groszkowski P, Koch J, Houck AA (2020) Quantum control of an oscillator using a stimulated josephson nonlinearity. Nat Phys 16(2):211–217

    Article  Google Scholar 

  • Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292

    MATH  Google Scholar 

  • Xu F, Zhang X, Xin Z, Yang A (2019) Investigation on the chinese text sentiment analysis based on convolutional neural networks in deep learning. Comput Mater Contin 58(3):697–709

    Google Scholar 

  • Yu S, Albarrán-Arriagada F, Retamal JC, Wang YT, Liu W, Ke ZJ, Meng Y, Li ZP, Tang JS, Solano E et al (2019) Reconstruction of a photonic qubit state with reinforcement learning. Adv Quant Technol 2(7–8):1800074

    Article  Google Scholar 

  • Zhang XM, Wei Z, Asad R, Yang XC, Wang X (2019) When does reinforcement learning stand out in quantum control? A comparative study on state preparation. npj Quant Inform 5(1):1–7

    Article  Google Scholar 

  • Zhang Y, Wang Z (2020) Hybrid malware detection approach with feedback-directed machine learning. Inf Sci 63(139103):1–139103

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers and editors for their comments that improved the quality of this paper. This work is supported by the National Natural Science Foundation of China (62071240, 61802175), and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Wenjie Liu, Bosi Wang & Yebo Ge

  2. Engineering Research Center of Digital Forensics Ministry of Education, Nanjing, 210044, China

    Wenjie Liu

  3. School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Jihao Fan

  4. Department of Artificial Intelligence, Faculty of Computers and Artificial Intelligence, South Valley University, Hurghada Branch, Egypt

    Mohammed Zidan

Authors
  1. Wenjie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bosi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jihao Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yebo Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohammed Zidan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjie Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical statement

Articles do not rely on clinical trials.

Human and animal participants

All submitted manuscripts contain research which do not involve human participants and/or animal experimentation.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, W., Wang, B., Fan, J. et al. A quantum system control method based on enhanced reinforcement learning. Soft Comput 26, 6567–6575 (2022). https://doi.org/10.1007/s00500-022-07179-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-07179-5

Keywords

Search

Navigation