Abstract
Estimation by analogy EBA (effort estimation by analogy) is one of the proven methods for effort prediction in software engineering; in AI this would be called Case-Based Reasoning. In this paper we consider effort predictions using the EBA () method AQUA and pay attention to two aspects: (i) The influence of the set of analogs on the quality of prediction. The set of analogs is determined by a learning process incorporating the number of nearest neighbors and the threshold of the similarity measure used, (ii) Analyzing and understanding the conditions under which the prediction can be expected to be the most or the least accurate.
We study two types of learning: One for finding the “best” set of analogs, and one for finding out factors for reliability. While both questions are relevant for different areas and disciplines, the focus of the paper is on estimation of effort in software engineering. For EBA method AQUA, the cases can be features or past projects characterized by attributes of various type. Classical estimation approaches just investigate the overall estimated quality of a system. However, in that case information is missing if and why estimation was performing the way it did. Bad estimates are often due to external influences. Therefore it is valuable for to find out under which conditions the estimates are more or less reliable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Li, J.Z.: A Flexible Method for Software Effort Estimation by Analogy. PhD Thesis, University of Calgary, Department of Computer Science, Calgary (2007)
Li, J.Z., Ruhe, G., Al-Emran, A., Richter, M.M.: A flexible method for software effort estimation by analogy. Empirical Software Engineering 12(1), 65–106 (2007)
Angelis, L., Stamelos., I., Morisio, M.: Building a Software Cost Estimation Model Based on Categorical Data. In: METRICS 2001: Proceedings of the IEEE 7th International Symposium on Software Metrics, England, UK, pp. 4–15 (2001)
Ruhe, M., Jeffery, R., Wieczorek, I.: Cost Estimation for Web Application. In: ICSE 2003: Proceedings of 25th International Conference on Software Engineering, Oregon, USA, pp. 285–294 (2003)
Conte, S.D., Dunsmore, H., Shen, V.Y.: Software engineering metrics and models. Benjamin-Cummings Publishing Co. Inc. (1986)
Mendes, E., Watson, I., Chris, T., Nile, M., Steve, C.: A Comparative Study of Cost Estimation Models for Web Hypermedia Applications. Empirical Software Engineering 8(2), 163–196 (2003)
Shepperd, M., Schofield, C.: Estimating Software Project Effort Using Analogies. IEEE Transactions on Software Engineering 23(12), 736–743 (1997)
Walkerden, F., Jeffery, R.: An Empirical Study of Analogy-based Software Effort Estimation. Empirical Software Engineering 4(2), 135–158 (1999)
Delany, S.J., Cunningham, P., Wilke, W.: The Limits of CBR in Software Project Estimation. In: 6th German Workshop On Case-Based Reasoning, https://www.cs.tcd.ie/publications/tech-reports/reports.99/TCD-CS-1999-21.pdf
Kadoda, G., Michelle, C., Chen, L., Shepperd, M.: Experiences Using Case-Based Reasoning to Predict Software Project Effort. In: EASE 2000, Staffordshire, UK (2000)
Leake, D.B., Sooriamurthi, R.: Case dispatching versus case-base merging: when MCBR matters. International Journal on Artificial Intelligence Tools 13(1), 237–254 (2004)
Richter, M.M.: Foundations of Similarity and Utility. In: Proc. FLAIRS 2007, pp. 30–37. AAAI Press, Menlo Park (2007)
Cheetham, W.: Case Based Reasoning with Confidence. In: Blanzieri, E., Portinale, L. (eds.) EWCBR 2000. LNCS (LNAI), vol. 1898, pp. 15–25. Springer, Heidelberg (2000)
Wettschereck, D., Aha, D.W.: Weighting features. In: ICCBR 1995. Springer, Heidelberg (1995)
Stahl, A.: Learning of Knowledge-Intensive Similarity Measures in Case-Based Reasoning. Dissertation Kaiserslautern (2003)
Richter, M.M.: Similarity. In: Perner, P. (ed.) Case-Based Reasoning on Signals and Images, pp. 25–90. Springer, Heidelberg (2007)
Basili, V.R., Caldiera, G., Rombach, H.D.: The Goal Question Metric Approach. Encyclopedia of Software Engineering. John Wiley & Sons, Inc., Chichester (1994)
Efron, B., Gong, G.: A Leisurely Look at the Bootstrap, the Jackknife, and Cross-Validation. The American Statistician 37(1), 36–48 (1983)
Song, Q., Shepperd, M., Mair, C.: Using Grey Relational Analysis to Predict Software Effort with Small Data Sets. In: METRICS 2005: Proceedings of the 11th IEEE International Software Metrics Symposium, Como, Italy, pp. 35–45 (2005)
Mackas, B.: Prediction Accuracy Validation. In: Richter, M.M. (ed.) Teaching Machine Learning, Technical Report, University of Calgary (2007)
SNNS user manual, http://www.ra.informatik.uni-tuebingen.de/SNNS/UserManual/UserManual.html
Baldi, P., Hornik, K.: Neural networks and principal component analysis: learning from examples without local minima. Neural Network 2(1), 53–58 (1989)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, J., Mackas, B., Richter, M.M., Ruhe, G. (2008). Cases, Predictions, and Accuracy Learning and Its Application to Effort Estimation. In: Althoff, KD., Bergmann, R., Minor, M., Hanft, A. (eds) Advances in Case-Based Reasoning. ECCBR 2008. Lecture Notes in Computer Science(), vol 5239. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85502-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-85502-6_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85501-9
Online ISBN: 978-3-540-85502-6
eBook Packages: Computer ScienceComputer Science (R0)