Abstract
Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. While the cross-entropy loss is usually justified from a probabilistic perspective, this paper shows an alternative and more direct interpretation of this loss in terms of t-norms and their associated generator functions, and derives a general relation between loss functions and t-norms. In particular, the presented work shows intriguing results leading to the development of a novel class of loss functions. These losses can be exploited in any supervised learning task and which could lead to faster convergence rates that the commonly employed cross-entropy loss.
This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 825619.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bach, S.H., Broecheler, M., Huang, B., Getoor, L.: Hinge-loss markov random fields and probabilistic soft logic. J. Mach. Learn. Res. 18, 1–67 (2017)
Beliakov, G., Pradera, A., Calvo, T.: Aggregation Functions: A Guide for Practitioners, vol. 221. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73721-6
Calvo, T., Kolesárová, A., Komorníková, M., Mesiar, R.: Aggregation operators: properties classes and construction methods. In: Calvo, T., Mayor, G., Mesiar, R. (eds.) Aggregation Operators, pp. 3–104. Springer, Heidelberg (2002). https://doi.org/10.1007/978-3-7908-1787-4_1
Diligenti, M., Gori, M., Sacca, C.: Semantic-based regularization for learning and inference. Artif. Intell. 244, 143–165 (2017)
Donadello, I., Serafini, L., d’Avila Garcez, A.: Logic tensor networks for semantic image interpretation. In: IJCAI International Joint Conference on Artificial Intelligence, pp. 1596–1602 (2017)
Giannini, F., Diligenti, M., Gori, M., Maggini, M.: On a convex logic fragment for learning and reasoning. IEEE Trans. Fuzzy Syst. 27, 1407–1416 (2018)
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learn., vol. 1. MIT Press, Cambridge (2016)
Grabisch, M., Marichal, J.L., Mesiar, R., Pap, E.: Aggregation functions: means. Inf. Sci. 181(1), 1–22 (2011)
Hájek, P.: Metamathematics of Fuzzy Logic, vol. 4. Springer, Dordrecht (2013). https://doi.org/10.1007/978-94-011-5300-3
Jenei, S.: A note on the ordinal sum theorem and its consequence for the construction of triangular norms. Fuzzy Sets Syst. 126(2), 199–205 (2002)
Klement, E.P., Mesiar, R., Pap, E.: Triangular norms. position paper i: basic analytical and algebraic properties. Fuzzy Sets Syst. 143(1), 5–26 (2004)
Klement, E.P., Mesiar, R., Pap, E.: Triangular norms. position paper ii: general constructions and parameterized families. Fuzzy Sets Syst. 145(3), 411–438 (2004)
Klement, E.P., Mesiar, R., Pap, E.: Triangular norms position paper iii: continuous t-norms. Fuzzy Sets Syst. 145(3), 439–454 (2004)
Klement, E.P., Mesiar, R., Pap, E.: Triangular Norms, vol. 8. Springer, Dordrecht (2013). https://doi.org/10.1007/978-94-015-9540-7
Kolb, S., Teso, S., Passerini, A., De Raedt, L.: Learning SMT (IRA) constraints using smt solvers. In: IJCAI. pp. 2333–2340 (2018)
Koller, D., et al.: Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)
Novák, V., Perfilieva, I., Mockor, J.: Mathematical Principles of Fuzzy Logic, vol. 517. Springer, New York (2012). https://doi.org/10.1007/978-1-4615-5217-8
Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1), 107–136 (2006)
Torra, V., Narukawa, Y.: Modeling Decisions: Information Fusion and Aggregation Operators. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-68791-7
Xu, J., Zhang, Z., Friedman, T., Liang, Y., Broeck, G.V.d.: A semantic loss function for deep learning with symbolic knowledge. arXiv preprint arXiv:1711.11157 (2017)
Yang, F., Yang, Z., Cohen, W.W.: Differentiable learning of logical rules for knowledge base reasoning. In: Advances in Neural Information Processing Systems, pp. 2319–2328 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Giannini, F., Marra, G., Diligenti, M., Maggini, M., Gori, M. (2020). On the Relation Between Loss Functions and T-Norms. In: Kazakov, D., Erten, C. (eds) Inductive Logic Programming. ILP 2019. Lecture Notes in Computer Science(), vol 11770. Springer, Cham. https://doi.org/10.1007/978-3-030-49210-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-49210-6_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-49209-0
Online ISBN: 978-3-030-49210-6
eBook Packages: Computer ScienceComputer Science (R0)