A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization | Computational Optimization and Applications Skip to main content
Log in

A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper we consider finding an approximate second-order stationary point (SOSP) of general nonconvex conic optimization that minimizes a twice differentiable function subject to nonlinear equality constraints and also a convex conic constraint. In particular, we propose a Newton-conjugate gradient (Newton-CG) based barrier-augmented Lagrangian method for finding an approximate SOSP of this problem. Under some mild assumptions, we show that our method enjoys a total inner iteration complexity of \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-11/2})\) and an operation complexity of \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-11/2}\min \{n,\epsilon ^{-5/4}\})\) for finding an \((\epsilon ,\sqrt{\epsilon })\)-SOSP of general nonconvex conic optimization with high probability. Moreover, under a constraint qualification, these complexity bounds are improved to \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-7/2})\) and \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-7/2}\min \{n,\epsilon ^{-3/4}\})\), respectively. To the best of our knowledge, this is the first study on the complexity of finding an approximate SOSP of general nonconvex conic optimization. Preliminary numerical results are presented to demonstrate superiority of the proposed method over first-order methods in terms of solution quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availibility

The codes for generating the random data and implementing the algorithms in the numerical section are available from the first author upon request.

Notes

  1. In fact, a total inner iteration complexity of \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-7})\) and an operation complexity \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-7}\min \{n,\epsilon ^{-1}\})\) were established in [70] for finding an \((\epsilon ,\epsilon )\)-SOSP of problem (1) with high probability; see [70, Theorem 4(ii), Corollary 3(ii), Theorem 5] Nevertheless, they can be easily modified to obtain the aforementioned complexity for finding an \((\epsilon ,\sqrt{\epsilon })\)-SOSP of (1) with high probability.

  2. The operation complexity of the barrier method [43] is measured by the amount of fundamental operations consisting of matrix–vector products, matrix multiplications, Cholesky factorizations, and backward or forward substitutions to a triangular linear system.

  3. The arithmetic complexity of Cholesky factorizations for a positive definite matrix is \({{\,\mathrm{\mathcal {O}}\,}}(n^3)\) in general, while the arithmetic complexity of matrix–vector products and backward or forward substitutions is at most \({{\,\mathrm{\mathcal {O}}\,}}(n^2)\), where n is the number of rows of the matrix.

  4. It shall be mentioned that the total numbers of Cholesky factorizations are only \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-7/2})\) and \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\epsilon ^{-11/2})\) respectively for the case where constraint qualification holds or not. See Subsections 5.3 and 5.4 for details.

  5. The \(\lambda ^{k+1}\) is also called a safeguarded Lagrangian multiplier, which has been used in the literature for designing some AL methods (e.g., see [3, 13, 44, 47]). It has been shown to enjoy many practical and theoretical advantages (e.g., see [13]).

  6. The vectorization of a matrix is the column vector obtained by stacking the columns of the matrix on top of one another.

References

  1. Agarwal, N., Allen-Zhu, Z., Bullins, B., Hazan, E., Ma, T.: Finding approximate local minima faster than gradient descent. In: Proceedings of the 49th annual ACM SIGACT symposium on theory of computing, pp. 1195–1199 (2017)

  2. Allen-Zhu, Z., Li, Y.: Neon2: finding local minima via first-order oracles. Adv. Neural Info. Process. Syst. 31, 3716–3726 (2018)

  3. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On augmented Lagrangian methods with general lower-level constraints. SIAM J. Optim. 18(4), 1286–1309 (2008)

    MathSciNet  Google Scholar 

  4. Andreani, R., Haeser, G., Ramos, A., Silva, P.J.: A second-order sequential optimality condition associated to the convergence of optimization algorithms. IMA J. Numer. Anal. 37(4), 1902–1929 (2017)

    MathSciNet  Google Scholar 

  5. Argáez, M., Tapia, R.: On the global convergence of a modified augmented Lagrangian linesearch interior-point Newton method for nonlinear programming. J. Optim. Theory Appl 114(1), 1–25 (2002)

    MathSciNet  Google Scholar 

  6. Armand, P., Omheni, R.: A mixed logarithmic barrier-augmented Lagrangian method for nonlinear optimization. J. Optim. Theory Appl. 173(2), 523–547 (2017)

    MathSciNet  Google Scholar 

  7. Armand, P., Tran, N.N.: Rapid infeasibility detection in a mixed logarithmic barrier-augmented Lagrangian method for nonlinear optimization. Optim. Methods Softw. 34(5), 991–1013 (2019)

    MathSciNet  Google Scholar 

  8. Bertsekas, D.P.: Nonlinear Programming. Athena Scientific. (1995)

  9. Bhojanapalli, S., Neyshabur, B., Srebro, N.: Global optimality of local search for low rank matrix recovery. Adv. Neural Info. Process. Syst. 29, 3873–3881 (2016)

  10. Bian, W., Chen, X., Ye, Y.: Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization. Math. Program. 149(1), 301–327 (2015)

    MathSciNet  Google Scholar 

  11. Birgin, E.G., Gardenghi, J., Martínez, J.M., Santos, S.A., Toint, P.L.: Evaluation complexity for nonlinear constrained optimization using unscaled KKT conditions and high-order models. SIAM J. Optim. 26(2), 951–967 (2016)

    MathSciNet  Google Scholar 

  12. Birgin, E.G., Haeser, G., Ramos, A.: Augmented Lagrangians with constrained subproblems and convergence to second-order stationary points. Comput. Optim. and Appl. 69(1), 51–75 (2018)

    MathSciNet  Google Scholar 

  13. Birgin, E. G., Martínez, J.M.: Practical augmented Lagrangian methods for constrained optimization. SIAM (2014)

  14. Birgin, E.G., Martínez, J.M.: The use of quadratic regularization with a cubic descent condition for unconstrained optimization. SIAM J. Optim. 27(2), 1049–1074 (2017)

    MathSciNet  Google Scholar 

  15. Bonnans, J.F., Launay, G.: Sequential quadratic programming with penalization of the displacement. SIAM J. Optim. 5(4), 792–812 (1995)

    MathSciNet  Google Scholar 

  16. Bueno, L.F., Martínez, J.M.: On the complexity of an inexact restoration method for constrained optimization. SIAM J. Optim. 30(1), 80–101 (2020)

    MathSciNet  Google Scholar 

  17. Byrd, R.H., Schnabel, R.B., Shultz, G.A.: A trust region algorithm for nonlinearly constrained optimization. SIAM J. Numer. Anal. 24(5), 1152–1170 (1987)

    MathSciNet  Google Scholar 

  18. Carmon, Y., Duchi, J.: Gradient descent finds the cubic-regularized nonconvex newton step. SIAM J. Optim. 29(3), 2146–2178 (2019)

    MathSciNet  Google Scholar 

  19. Carmon,Y., Duchi, J.C., Hinder, O., Sidford, A.:“Convex until proven guilty": dimension-free acceleration of gradient descent on non-convex functions. In: International conference on machine learning, pp. 654–663. PMLR, (2017)

  20. Carmon, Y., Duchi, J.C., Hinder, O., Sidford, A.: Accelerated methods for nonconvex optimization. SIAM J. Optim. 28(2), 1751–1772 (2018)

    MathSciNet  Google Scholar 

  21. Cartis, C., Gould, N.I., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127(2), 245–295 (2011)

    MathSciNet  Google Scholar 

  22. Cartis, C., Gould, N.I., Toint, P.L.: On the evaluation complexity of cubic regularization methods for potentially rank-deficient nonlinear least-squares problems and its relevance to constrained nonlinear optimization. SIAM J. Optim. 23(3), 1553–1574 (2013)

    MathSciNet  Google Scholar 

  23. Cartis, C., Gould, N.I., Toint, P.L.: On the complexity of finding first-order critical points in constrained nonlinear optimization. Math. Program. 144(1), 93–106 (2014)

    MathSciNet  Google Scholar 

  24. Cartis, C., Gould, N.I., Toint, P.L.: On the complexity of finding first-order critical points in constrained nonlinear optimization. Math. Program. 144(1), 93–106 (2014)

    MathSciNet  Google Scholar 

  25. Cartis, C., Gould, N.I., Toint, P.L.: On the evaluation complexity of constrained nonlinear least-squares and general constrained nonlinear optimization using second-order methods. SIAM J. Numer. Anal. 53(2), 836–851 (2015)

    MathSciNet  Google Scholar 

  26. Cartis, C., Gould, N.I., Toint, P.L.: Evaluation complexity bounds for smooth constrained nonlinear optimization using scaled KKT conditions, high-order models and the criticality measure \(\chi \). In: Demetriou, I., Pardalos, P. (eds.) Approximation and Optimization: Algorithms, Complexity and Applications, pp. 5–26. Springer, NY (2019)

    Google Scholar 

  27. Cartis, C., Gould, N.I., Toint, P.L.: Optimality of orders one to three and beyond: characterization and evaluation complexity in constrained nonconvex optimization. J. Complex. 53, 68–94 (2019)

    MathSciNet  Google Scholar 

  28. Chen, X., Guo, L., Lu, Z., Ye, J.J.: An augmented Lagrangian method for non-Lipschitz nonconvex programming. SIAM J. Numer. Anal. 55(1), 168–193 (2017)

    MathSciNet  Google Scholar 

  29. Chi, Y., Lu, Y.M., Chen, Y.: Nonconvex optimization meets low-rank matrix factorization: an overview. IEEE Trans. Signal Process. 67(20), 5239–5269 (2019)

    MathSciNet  Google Scholar 

  30. Cifuentes, D., Moitra, A.: Polynomial time guarantees for the Burer-Monteiro method. (2019) arXiv:1912.01745

  31. Coleman, T.F., Liu, J., Yuan, W.: A new trust-region algorithm for equality constrained optimization. Comput. Optim. Appl. 21(2), 177–199 (2002)

    MathSciNet  Google Scholar 

  32. Conn, A.R., Gould, G., Toint, P.L.: LANCELOT: a Fortran package for large-scale nonlinear optimization. Springer Science & Business Media, NY (2013)

    Google Scholar 

  33. Conn, A.R., Gould, N.I., Toint, P.L.: A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds. Math. Comput. 66(217), 261–288 (1997)

    MathSciNet  Google Scholar 

  34. Curtis, F.E., Robinson, D.P., Royer, C.W., Wright, S.J.: Trust-region Newton-CG with strong second-order complexity guarantees for nonconvex optimization. SIAM J. Optim. 31(1), 518–544 (2021)

    MathSciNet  Google Scholar 

  35. Curtis, F.E., Robinson, D.P., Samadi, M.: A trust region algorithm with a worst-case iteration complexity of \(\cal{O} (\epsilon ^{-3/2})\) for nonconvex optimization. Math. Program. 1(162), 1–32 (2016)

    MathSciNet  Google Scholar 

  36. Curtis, F.E., Robinson, D.P., Samadi, M.: Complexity analysis of a trust funnel algorithm for equality constrained optimization. SIAM J. Optim. 28(2), 1533–1563 (2018)

    MathSciNet  Google Scholar 

  37. De Carvalho, E.P., dos Santos Júnior, A., Ma, T.F.: Reduced gradient method combined with augmented Lagrangian and barrier for the optimal power flow problem. Appl. Math. Comput. 200(2), 529–536 (2008)

    MathSciNet  Google Scholar 

  38. P. Dvurechensky and M. Staudigl. Hessian barrier algorithms for non-convex conic optimization. (2021) arXiv:2111.00100

  39. Goldfarb, D., Polyak, R., Scheinberg, K., Yuzefovich, I.: A modified barrier-augmented Lagrangian method for constrained minimization. Comput. Optim. Appl. 14(1), 55–74 (1999)

    MathSciNet  Google Scholar 

  40. F. Goyens, A. Eftekhari, and N. Boumal. Computing second-order points under equality constraints: revisiting Fletcher’s augmented Lagrangian. (2022) arXiv:2204.01448

  41. Grapiglia, G.N., Yuan, Y.-X.: On the complexity of an augmented Lagrangian method for nonconvex optimization. IMA J. Numer. Anal. 41(2), 1546–1568 (2021)

    MathSciNet  Google Scholar 

  42. Haeser, G., Liu, H., Ye, Y.: Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary. Math. Program. 178(1), 263–299 (2019)

    MathSciNet  Google Scholar 

  43. He, C., Lu, Z.: A Newton-CG based barrier method for finding a second-order stationary point of nonconvex conic optimization with complexity guarantees. SIAM J. Optim. 33(2), 1191–1222 (2023)

    MathSciNet  Google Scholar 

  44. He, C., Lu, Z., Pong, T.K.: A Newton-CG based augmented Lagrangian method for finding a second-order stationary point of nonconvex equality constrained optimization with complexity guarantees. SIAM J. Optim. 33(3), 1734–1766 (2023)

    MathSciNet  Google Scholar 

  45. Huck, A., Guillaume, M., Blanc-Talon, J.: Minimum dispersion constrained nonnegative matrix factorization to unmix hyperspectral data. IEEE Trans. Geosci. Remote Sens. 48(6), 2590–2602 (2010)

    Google Scholar 

  46. C. Jin, P. Netrapalli, and M. I. Jordan. Accelerated gradient descent escapes saddle points faster than gradient descent. In: Conference on learning theory, pp. 1042–1085. (2018)

  47. Kanzow, C., Steck, D.: An example comparing the standard and safeguarded augmented Lagrangian methods. Oper. Res. Lett. 45(6), 598–603 (2017)

    MathSciNet  Google Scholar 

  48. Kuczyński, J., Woźniakowski, H.: Estimating the largest eigenvalue by the power and Lanczos algorithms with a random start. SIAM J. Matrix Anal. Appl. 13(4), 1094–1122 (1992)

    MathSciNet  Google Scholar 

  49. Kuhlmann, R., Büskens, C.: A primal-dual augmented Lagrangian penalty-interior-point filter line search algorithm. Math. Method Oper. Res. 87(3), 451–483 (2018)

    MathSciNet  Google Scholar 

  50. Liu, X., Xia, W., Wang, B., Zhang, L.: An approach based on constrained nonnegative matrix factorization to unmix hyperspectral data. IEEE Trans. Geosci. Remote Sens. 49(2), 757–772 (2010)

    Google Scholar 

  51. Lu, S., Razaviyayn, M., Yang, B., Huang, K., Hong, M.: Finding second-order stationary points efficiently in smooth nonconvex linearly constrained optimization problems. Adv. Neural Info. Process. Syst. 33, 2811–2822 (2020)

    Google Scholar 

  52. Lu, Z., Zhang, Y.: An augmented Lagrangian approach for sparse principal component analysis. Math. Program. 135(1), 149–193 (2012)

    MathSciNet  Google Scholar 

  53. Martínez, J.M., Raydan, M.: Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization. J. Glob. Optim. 68(2), 367–385 (2017)

    MathSciNet  Google Scholar 

  54. Miao, L., Qi, H.: Endmember extraction from highly mixed data using minimum volume constrained nonnegative matrix factorization. IEEE Trans. Geosci. Remote Sens. 45(3), 765–777 (2007)

    Google Scholar 

  55. Moguerza, J.M., Prieto, F.J.: An augmented Lagrangian interior-point method using directions of negative curvature. Math. Program. 95(3), 573–616 (2003)

    MathSciNet  Google Scholar 

  56. Nesterov, Y., Nemirovskii, A.: Interior-point polynomial algorithms in convex programming. SIAM, Philadelphia (1994)

    Google Scholar 

  57. Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108(1), 177–205 (2006)

    MathSciNet  Google Scholar 

  58. O’Neill, M., Wright, S.J.: A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees. IMA J. Numer. Anal. 41(1), 84–121 (2021)

    MathSciNet  Google Scholar 

  59. Park, D., Kyrillidis, A., Carmanis, C., Sanghavi, S.: Non-square matrix sensing without spurious local minima via the burer-monteiro approach. In Artificial intelligence and statistics, pp. 65–74. PMLR, (2017)

  60. Royer, C.W., O’Neill, M., Wright, S.J.: A Newton-CG algorithm with complexity guarantees for smooth unconstrained optimization. Math. Program. 180(1), 451–488 (2020)

    MathSciNet  Google Scholar 

  61. Royer, C.W., Wright, S.J.: Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim. 28(2), 1448–1477 (2018)

    MathSciNet  Google Scholar 

  62. Ruszczynski, A.: Nonlinear optimization. Princeton University Press, NJ (2011)

    Google Scholar 

  63. M. F. Sahin, A. Eftekhari, A. Alacaoglu, F. Latorre, and V. Cevher. An inexact augmented Lagrangian framework for nonconvex optimization with nonlinear constraints. Adv. Neural Info. Process. Syst. 32, 632–650 (2019)

  64. Thanh, O. V., Gillis, N., Lecron, F.: Bounded simplex-structured matrix factorization. In ICASSP 2022-2022 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp. 9062–9066. IEEE, (2022)

  65. Vanderbei, R.J., Shanno, D.F.: An interior-point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13(1), 231–252 (1999)

    MathSciNet  Google Scholar 

  66. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)

    MathSciNet  Google Scholar 

  67. Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107(3), 391–408 (2006)

    MathSciNet  Google Scholar 

  68. Wright, S.J., Nowak, R.D., Figueiredo, M.A.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57(7), 2479–2493 (2009)

    MathSciNet  Google Scholar 

  69. Xie, P., Wright, S.J.: Complexity of projected Newton methods for bound-constrained optimization. (2021) arXiv:2103.15989

  70. Xie, Y., Wright, S.J.: Complexity of proximal augmented Lagrangian for nonconvex optimization with nonlinear equality constraints. J. Sci. Comput. 86(3), 1–30 (2021)

    MathSciNet  Google Scholar 

  71. Xu, Y., Jin, R., Yang, Y.: NEON+: Accelerated gradient methods for extracting negative curvature for non-convex optimization. (2017) arXiv:1712.01033

  72. Yang, L., Sun, D., Toh, K.C.: SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints. Math. Program. Comput. 7(3), 331–366 (2015)

    MathSciNet  Google Scholar 

  73. Zhao, X., Sun, D., Toh, K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20(4), 1737–1765 (2010)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhaosong Lu.

Ethics declarations

Conflict of interest

The third author is an editorial board member of this journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work of this author was partially supported by NSF Award IIS-2211492

The work of this author was partially supported by NSF Award IIS-2211491.

Appendices

Appendix

A capped conjugate gradient method

Algorithm 3
figure c

A capped conjugate gradient method

In this part we present the capped CG method proposed in [60, Algorithm 1] for solving a possibly indefinite linear system (15). As briefly discussed in Sect. 4, the capped CG method finds either an approximate solution to (15) or a sufficiently negative curvature direction of the associated matrix H. More details about this method can be found in [60, Section 3.1].

The following theorem presents the iteration complexity of Algorithm 3, whose proof can be found in [44, Theorem A.1], and thus omitted here.

Theorem A.1

(iteration complexity of Algorithm 3) Consider applying Algorithm 3 with the optional input \(U=0\) to the linear system (15) with \(g\ne 0\), \(\varepsilon >0\), and H being an \(n\times n\) symmetric matrix. Then the number of iterations of Algorithm 3 is \({\widetilde{{{\,\mathrm{\mathcal {O}}\,}}}}(\min \{n,\sqrt{\Vert H\Vert /\varepsilon }\})\).

A randomized Lanczos based minimum eigenvalue oracle

Algorithm 4
figure d

A randomized Lanczos based minimum eigenvalue oracle

In this part we present the randomized Lanczos method proposed in [60, Section 3.2], which can be used as a minimum eigenvalue oracle for Algorithm 1. As mentioned in Sect. 4, this oracle either outputs a sufficiently negative curvature direction of H or certifies that H is nearly positive semidefinite with high probability. More details about it can be found in [60, Section 3.2].

The following theorem justifies that Algorithm 4 is a suitable minimum eigenvalue oracle for Algorithm 1. Its proof is identical to that of [60, Lemma 2] and thus omitted.

Theorem B.1

[Iteration complexity of Algorithm 4] Consider Algorithm 4 with tolerance \(\varepsilon >0\), probability parameter \(\delta \in (0,1)\), and symmetric matrix \(H\in {{\,\mathrm{\mathbb {R}}\,}}^{n\times n}\) as its input. Then it either finds a sufficiently negative curvature direction v satisfying \(v^THv\le -\varepsilon /2\) and \(\Vert v\Vert =1\) or certifies that \(\lambda _{\min }(H)\ge -\varepsilon \) holds with probability at least \(1-\delta \) in at most \(N(\varepsilon ,\delta )\) iterations, where \(N(\varepsilon ,\delta )\) is defined in (111).

Notice that generally, computing \(\Vert H\Vert \) in Algorithm 4 may not be cheap when n is large. Nevertheless, \(\Vert H\Vert \) can be efficiently estimated via a randomization scheme with high confidence (e.g., see the discussion in [60, Appendix B3]).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, C., Huang, H. & Lu, Z. A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization. Comput Optim Appl 89, 843–894 (2024). https://doi.org/10.1007/s10589-024-00603-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-024-00603-6

Keywords

Mathematics Subject Classification