A general framework for ADMM acceleration | Numerical Algorithms Skip to main content
Springer Nature Link
Log in

A general framework for ADMM acceleration

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The Alternating Direction Multipliers Method (ADMM) is a very popular algorithm for computing the solution of convex constrained minimization problems. Such problems are important from the application point of view, since they occur in many fields of science and engineering. ADMM is a powerful numerical tool, but unfortunately its main drawback is that it can exhibit slow convergence. Several approaches for its acceleration have been proposed in the literature and in this paper we present a new general framework devoted to this aim. In particular, we describe an algorithmic framework that makes possible the application of any acceleration step while still having the guarantee of convergence. This result is achieved thanks to a guard condition that ensures the monotonic decrease of the combined residual. The proposed strategy is applied to image deblurring problems. Several acceleration techniques are compared; to the best of our knowledge, some of them are investigated for the first time in connection with ADMM. Numerical results show that the proposed framework leads to a faster convergence with respect to other acceleration strategies recently introduced for ADMM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  2. Chen, C., Chan, R.H., Ma, S., Yang, J.: Inertial proximal ADMM for linearly constrained separable convex optimization. SIAM J. Imag. Sci. 8(4), 2239–2267 (2015)

    Article  MathSciNet  Google Scholar 

  3. Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Imag. Sci. 7(3), 1588–1623 (2014)

    Article  MathSciNet  Google Scholar 

  4. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Img. Sci. 2, 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  5. Nesterov, Y.: Introductory Lectures on Convex Optimization: a Basic Course. Kluwer Academic Publishers (2004)

  6. Biggs, D.S.C., Andrews, M.: Acceleration of iterative image restoration algorithms. Appl. Opt. 36, 1766–1775 (1997)

    Article  Google Scholar 

  7. Dell’Acqua, P.: ν acceleration of statistical iterative methods for image restoration. Signal Image Vid Process 10, 927–934 (2016)

    Article  Google Scholar 

  8. Hanke, M.: Accelerated Landweber iterations for the solutions of ill-posed equations. Numer. Math. 60, 341–373 (1991)

    Article  MathSciNet  Google Scholar 

  9. Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73, 615–624 (1951)

    Article  MathSciNet  Google Scholar 

  10. Brezinski, C., Redivo-Zaglia, M.: The simplified topological ε-algorithms for accelerating sequences in a vector space. SIAM J. Sci. Comput. 36, A2227–A2247 (2014)

    Article  MathSciNet  Google Scholar 

  11. Chan, R.H., Tao, M., Yuan, X.: Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers. SIAM J. Imaging Sci. 6(1), 680–697 (2013)

    Article  MathSciNet  Google Scholar 

  12. Lanza, A., Morigi, S., Pragliola, M., Sgallari, F.: Space-variant TV regularization for image restoration. In: European Congress on Computational Methods in Applied Sciences and Engineering, pp 160–169. Springer (2017)

  13. Almeida, M.S., Figueiredo, M.: Deconvolving images with unknown boundaries using the alternating direction method of multipliers. IEEE Trans. Image Process. 22(8), 3074–3086 (2013)

    Article  MathSciNet  Google Scholar 

  14. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  15. Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via non-convex regularization with parameter selection. J. Math. Imag. Vis. 56(2), 195–220 (2016)

    Article  MathSciNet  Google Scholar 

  16. Almeida, M.S., Figueiredo, M.A.: Frame-based image deblurring with unknown boundary conditions using the alternating direction method of multipliers. In: 2013 20th IEEE International Conference on Image Processing (ICIP), pp 582–585. IEEE (2013)

  17. Hansen, P.C., Nagy, J.G., O’leary, D.P.: Deblurring Images: Matrices, Spectra, and Filtering, vol. 3. Siam, Philadelphia (2006)

    Book  Google Scholar 

  18. He, B., Yuan, X.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130(3), 567–577 (2015)

    Article  MathSciNet  Google Scholar 

  19. Gazzola, S., Karapiperi, A.: Image reconstruction and restoration using the simplified ε-algorithm. Appl. Math. Comput. 274, 539–555 (2016)

    MathSciNet  MATH  Google Scholar 

  20. Shanks, D.: Nonlinear transformations of divergent and slowly convergent sequences. J. Math. Phys. 34, 1–42 (1955)

    Article  MathSciNet  Google Scholar 

  21. Wynn, P.: On a device for computing the em(Sn) transformation. MTAC 10, 91–96 (1956)

    MathSciNet  MATH  Google Scholar 

  22. Brezinski, C.: Généralisations de la transformation de Shanks, de la table de Padé et de l’ε-algorithme. Calcolo 12, 317–360 (1975)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The work of the first author has been partially founded by the Project Young Researchers “Reconstruction of sparse data” of the group GNCS of INdAM.

Author information

Authors and Affiliations

  1. Dipartimento di Matematica e Informatica, Università degli Studi di Cagliari, Via Ospedale 72 - Palazzo delle Scienze, 09124, Cagliari, Italy

    Alessandro Buccini

  2. Libera Università di Bolzano, Piazza Domenicani 3, 39100, Bolzano, Italy

    Pietro Dell’Acqua

  3. Dipartimento di Scienza e Alta Tecnologia, Università degli Studi dell’Insubria, Via Valleggio 11, 22100, Como, Italy

    Marco Donatelli

Authors
  1. Alessandro Buccini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pietro Dell’Acqua
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Donatelli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pietro Dell’Acqua.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Buccini, A., Dell’Acqua, P. & Donatelli, M. A general framework for ADMM acceleration. Numer Algor 85, 829–848 (2020). https://doi.org/10.1007/s11075-019-00839-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-019-00839-y

Keywords

Search

Navigation