Regression Models of Software Development Effort Estimation Accuracy and Bias

Abstract

This paper describes models whose purpose is to explain the accuracy and bias variation of an organization’s estimates of software development effort through regression analysis. We collected information about variables that we believed would affect the accuracy or bias of estimates of the performance of tasks completed by the organization. In total, information about 49 software development tasks was collected. We found that the following conditions led to inaccuracies in estimates: (1) Estimates were provided by a person in the role of “software developer” instead of “project leader”, (2) The project had as its highest priority time-to-delivery instead of quality or cost, and (3) The estimator did not participate in the completion of the task. The following conditions led to an increased bias towards under-estimation: (1) Estimates were provided by a person with the role of “project leader” instead of “software developer”. (2) The estimator assessed the accuracy of own estimates of similar, previously completed tasks to be low (more than 20% error). Although all variables included in the models were significant p < 0.1), the explanatory and predictive power of both models was poor, that is, most of the variance in the accuracy and bias of estimates was not explained or predicted by our models. In addition, there were several important threats to the validity of the coefficients suggested by the models. An analysis of the estimators’ own descriptions of the reasons for achieved estimation accuracy on each task suggests that it will be difficult to include all important estimation accuracy and bias factors in regression-based models. It is, for this reason, not realistic to expect such models to replace human judgment in estimation uncertainty assessments and as input to plans for the improvement of estimates. It is, nevertheless, possible that the type of formal analysis and regression-based models presented in this paper may, in some cases, be useful as support for human judgment.