Abstract
We investigate the implementation of multi-label classification algorithms with a reject option, as a mean to reduce the time required to human annotators and to attain a higher classification accuracy on automatically classified samples than the one which can be obtained without a reject option. Based on a recently proposed model of manual annotation time, we identify two approaches to implement a reject option, related to the two main manual annotation methods: browsing and tagging. In this paper we focus on the approach suitable to tagging, which consists in withholding either all or none of the category assignments of a given sample. We develop classification reliability measures to decide whether rejecting or not a sample, aimed at maximising classification accuracy on non-rejected ones. We finally evaluate the trade-off between classification accuracy and rejection rate that can be attained by our method, on three benchmark data sets related to text categorisation and image annotation tasks.
Chapter PDF
Similar content being viewed by others
References
Aronson, A., Rogers, W., Lang, F., Névéol, A.: 2008 report to the board of scientific counselors (2008), http://ii.nlm.nih.gov/IIPublications.shtml
Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Chow, C.K.: On optimum recognition error and reject tradeoff. IEEE Transactions in Information Theory 16(1), 41–16 (1970)
Fan, R.E., Lin, C.J.: A study on threshold selection for multi-label. Tech. rep., National Taiwan University (2007)
Fumera, G., Pillai, I., Roli, F.: Classification with reject option in text categorisation systems. In: Int. Conf. Image Analysis and Proc. (2003)
Fumera, G., Pillai, I., Roli, F.: A Two-Stage Classifier with Reject Option for Text Categorisation. In: Structural, Syntactic, and Statistical Patt. Rec. (2004)
Lewis, D.D., Schapire, R.E., Callan, J.P., Papka, R.: Training algorithms for linear text classifiers. In: SIGIR, pp. 298–306 (1996)
Nowak, S., Huiskes, M.: New strategies for image annotation: Overview of the photo annotation task at imageclef 2010. Working Notes of CLEF 2010 (2010)
Pudil, P., Novovicova, J., Blaha, S., Kittler, J.V.: Multistage pattern recognition with reject option. In: ICPR, pp. II:92–II:95 (1992)
Ruiz, M., Aronson, A.: User-centered evaluation of the medical text indexing (mti) system (2007), http://ii.nlm.nih.gov/IIPublications.shtml
Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34(1), 1–47 (2002)
Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. In: Data Mining and Knowledge Discovery Handbook, pp. 667–685 (2010)
Yan, R., Natsev, A., Campbell, M.: An efficient manual image annotation approach based on tagging and browsing. In: Workshop on Multimedia Inf. Retr. on The Many Faces of Multimedia Semantics, pp. 13–20 (2007)
Yang, Y.: A study of thresholding strategies for text categorization. In: Int. Conf. on Research and Development in Information Retrieval, New York, USA (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pillai, I., Fumera, G., Roli, F. (2011). A Classification Approach with a Reject Option for Multi-label Problems. In: Maino, G., Foresti, G.L. (eds) Image Analysis and Processing – ICIAP 2011. ICIAP 2011. Lecture Notes in Computer Science, vol 6978. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24085-0_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-24085-0_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24084-3
Online ISBN: 978-3-642-24085-0
eBook Packages: Computer ScienceComputer Science (R0)
