Abstract
This paper considers the problem of constructing classifiers for road side assistance capable of providing reliability values for classifications of individual instances. In this context we analyze the existing approaches to reliable classification based on the conformity framework [16,18,19,27]. As a result we propose an approach that allows the framework to be applied to any type of classifiers so that the classification-reliability values can be computed for each class. The experiments show that the approach outperforms the existing approaches to reliable classification for road side assistance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Baskiotis, N., Sebag, M.: C4.5 competence map: a phase transition-inspired approach. In: Proceedings of the 21th International Conference on Machine Learning (ICML 2004), pp. 73–80 (2004)
Bay, S., Pazzani, M.: Characterizing model errors and differences. In: Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 196–201 (2000)
Bosnic, Z., Kononenko, I.: An overview of advances in reliability estimation of individual predictions in machine learning. Intell. Data Anal. 13(2), 385–401 (2009)
Domingos, P., Pazzani, M.: Beyond independence: conditions for the optimality of the simple bayesian classifier. In: Proceedings of the Thirteenth International Conference on Machine Learning (ICML 1996), pp. 105–112. Morgan Kaufmann, San Francisco (1996)
Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. Wiley, Chichester (2000)
Fawcett, T.: Roc graphs: Notes and practical considerations for researchers. Tech. Rep. HPL-2003-4, HP Laboratories (2003), http://citeseer.ist.psu.edu/fawcett04roc.html
Ferri, C., Hernndez-Orallo, J.: Cautious classifiers. In: Proceedings of the 1st International Workshop on ROC Analysis in Artificial Intelligence (ROCAI 2004), pp. 27–36 (2004)
Friedel, C., Ruckert, U., Kramer, S.: Cost curves for abstaining classifiers. In: Lachiche, N., Ferri, C., Macskassy, S. (eds.) Proceedings of the ICML 2006 Workshop on ROC Analysis (ROCML 2006), Pittsburgh, USA (June 29, 2006)
Hullermeier, E.: Instance-based prediction with guaranteed confidence. In: Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004), pp. 97–101 (2004)
Kaptein, A.: Meta-Classifier Approach to Reliable Text Classification. Master’s thesis, Maastricht University, The Netherlands (2005)
Kukar, M.: Quality assessment of individual classifications in machine learning and data mining. Knowl. Inf. Syst. 9(3), 364–384 (2006)
Perner, P.: Proceedings of 10th Industrial Conference on Data Mining (ICDM 2010), Berlin, Germany. July 12-14, IBaI Report (2010)
Proedru, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 381–390. Springer, Heidelberg (2002)
Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann, San Mateo (1993)
Saunders, C., Gammerman, A., Vovk, V.: Transduction with confidence and credibility. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 722–726 (1999)
Shafer, G., Vovk, V.: A tutorial on conformal prediction. Journal of Machine Learning Research 9, 371–421 (2008)
Smirnov, E., Nalbantov, G., Nikolaev, N.: k-Version-space multi-class classification based on k-consistency tests. In: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2010 (2010) (accepted)
Smirnov, E., Kaptein, A.: Theoretical and experimental study of a meta-typicalness approach for reliable classification. In: Proceedings of the IEEE International Workshop on Reliability Issues in Knowledge Discovery (RIKD 2006), pp. 739–743. IEEE Computer Society, Los Alamitos (2006)
Smirnov, E., Nalbantov, G., Kaptein, A.: Meta-conformity approach to reliable classification. Intell. Data Anal. 13(6), 901–915 (2009)
Smirnov, E., Sprinkhuizen-Kuyper, I., Nalbantov, G., Vanderlooy, S.: Version space support vector machines. In: Proceedings of the 17th European Conference on Artificial Intelligence (ECAI 2006), pp. 809–810 (2006)
Smirnov, E., Sprinkhuizen-Kuyper, I., Nikolaev, N.: Generalizing version space support vector machines for non-separable data. In: Proceedings of the IEEE International Workshop on Reliability Issues in Knowledge Discovery (RIKD 2006), pp. 744–748. IEEE Computer Society, Los Alamitos (2006)
Smirnov, E., Vanderlooy, S., Sprinkhuizen-Kuyper, I.: Meta-typicalness approach to reliable classification. In: Proceedings of the 17th European Conference on Artificial Intelligence (ECAI 2006), pp. 811–812 (2006)
Tzikas, D., Kukar, M., Likas, A.: Transductive reliability estimation for kernel based classifiers. In: Berthold, M.R., Shawe-Taylor, J., Lavrač, N. (eds.) IDA 2007. LNCS, vol. 4723, pp. 37–47. Springer, Heidelberg (2007)
Vanderlooy, S., Sprinkhuizen-Kuyper, I., Smirnov, E.: An analysis of reliable classifiers through ROC isometrics. In: Lachiche, N., Ferri, C., Macskassy, S. (eds.) Proceedings of the ICML 2006 Workshop on ROC Analysis (ROCML 2006), Pittsburgh, USA, June 29, pp. 55–62 (2006)
Vanderlooy, S., Sprinkhuizen-Kuyper, I., Smirnov, E.: Reliable classifiers in ROC space. In: Saeys, Y., Tsiporkova, E., Baets, B.D., de Peer, Y.V. (eds.) Proceedings of the 15th Annual Machine Learning Conference of Belgium and the Netherlands (BENELEARN 2006), Ghent, Belgium, May 11-22, pp. 113–120 (2006)
Vapnik, V.: Statistical Learning Theory. John Wiley, NY (1998)
Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, Heidelberg (2005)
Wolpert, D.: Stacked generalization. Neural Networks 5, 241–260 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Smirnov, E., Nikolaev, N., Nalbantov, G. (2010). Single-Stacking Conformity Approach to Reliable Classification. In: Dicheva, D., Dochev, D. (eds) Artificial Intelligence: Methodology, Systems, and Applications. AIMSA 2010. Lecture Notes in Computer Science(), vol 6304. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15431-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-15431-7_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15430-0
Online ISBN: 978-3-642-15431-7
eBook Packages: Computer ScienceComputer Science (R0)