{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,19]],"date-time":"2025-03-19T12:33:03Z","timestamp":1742387583799},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[1999,7]]},"abstract":"A new approach to software reliability estimation is presented that combines operational testing with stratified sampling in order to reduce the number of program executions that must be checked manually for conformance to requirements. Automatic cluster analysis is applied to execution profiles in order to stratify captured operational executions. Experimental results are reported that suggest this approach can significantly reduce the cost of estimating reliability.<\/jats:p>","DOI":"10.1145\/310663.310667","type":"journal-article","created":{"date-parts":[[2002,7,27]],"date-time":"2002-07-27T11:29:00Z","timestamp":1027769340000},"page":"263-283","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":50,"title":["Estimation of software reliability by stratified sampling"],"prefix":"10.1145","volume":"8","author":[{"given":"Andy","family":"Podgurski","sequence":"first","affiliation":[{"name":"Case Western Reserve Univ., Cleveland, OH"}]},{"given":"Wassim","family":"Masri","sequence":"additional","affiliation":[{"name":"Case Western Reserve Univ., Cleveland, OH"}]},{"given":"Yolanda","family":"McCleese","sequence":"additional","affiliation":[{"name":"Case Western Reserve Univ., Cleveland, OH"}]},{"given":"Francis G.","family":"Wolff","sequence":"additional","affiliation":[{"name":"Case Western Reserve Univ., Cleveland, OH"}]},{"given":"Charles","family":"Yang","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge"}]}],"member":"320","published-online":{"date-parts":[[1999,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/356876.356879"},{"key":"e_1_2_1_2_1","volume-title":"Cluster Analysis for Applications","author":"ANDERBERG M. R.","unstructured":"ANDERBERG , M. R. 1973. Cluster Analysis for Applications . Academic Press, Inc. , New York, NY . ANDERBERG, M. R. 1973. Cluster Analysis for Applications. Academic Press, Inc., New York, NY."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the International Conference on Reliable Software","author":"BROWN J. R.","year":"1975","unstructured":"BROWN , J. R. AND LIPOW , M. 1975 . Testing for software reliability . In Proceedings of the International Conference on Reliable Software ( Los Angeles, CA, Apr.). 518-527. 10.1145\/800027.808475 BROWN, J. R. AND LIPOW, M. 1975. Testing for software reliability. In Proceedings of the International Conference on Reliable Software (Los Angeles, CA, Apr.). 518-527. 10.1145\/800027.808475"},{"key":"e_1_2_1_4_1","volume-title":"Sampling Techniques","author":"COCHRAN W. G.","unstructured":"COCHRAN , W. G. 1977. Sampling Techniques . John Wiley & amp; Sons, Inc., New York, NY. COCHRAN, W. G. 1977. Sampling Techniques. John Wiley & Sons, Inc., New York, NY."},{"issue":"4","key":"e_1_2_1_5_1","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/C-M.1978.218136","article-title":"Hints on test data selection: Help for the practising programmer","volume":"11","author":"DEMILLO R. A.","year":"1978","unstructured":"DEMILLO , R. A. , LIPTON , R. J. , AND SAYWARD , F. G. 1978 . Hints on test data selection: Help for the practising programmer . Computer 11 , 4 , 34 - 41 . DEMILLO, R. A., LIPTON, R. J., AND SAYWARD, F. G. 1978. Hints on test data selection: Help for the practising programmer. Computer 11, 4, 34-41.","journal-title":"Computer"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1109\/TR.1980.5220756","article-title":"Quantifying software validity by sampling","volume":"2","author":"DURAN J. W.","year":"1980","unstructured":"DURAN , J. W. AND WIORKOWSKI , J. J. 1980 . Quantifying software validity by sampling . IEEE Trans. Reliab. R-29 , 2 ( June ), 141 - 144 . DURAN, J. W. AND WIORKOWSKI, J. J. 1980. Quantifying software validity by sampling. IEEE Trans. Reliab. R-29, 2 (June), 141-144.","journal-title":"IEEE Trans. Reliab. R-29"},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1109\/TSE.1984.5010257","article-title":"An evaluation of random testing","volume":"4","author":"DURAN J. W.","year":"1984","unstructured":"DURAN , J. W. AND NTAFOS , S. C. 1984 . An evaluation of random testing . IEEE Trans. Softw. Eng. SE-IO , 4 ( July ), 438 - 444 . DURAN, J. W. AND NTAFOS, S. C. 1984. An evaluation of random testing. IEEE Trans. Softw. Eng. SE-IO, 4 (July), 438-444.","journal-title":"IEEE Trans. Softw. Eng. SE-IO"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.221133"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.245738"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.62448"},{"key":"e_1_2_1_11_1","first-page":"68","volume-title":"Proceedings of the 14th International Conference on Software Engineering (ICSE '92","author":"HARROLD M. J.","year":"1992","unstructured":"HARROLD , M. J. , MCGREGOR , J. D. , AND FITZPATRICK , K.J. 1992 . Incremental testing of object-oriented class structures . In Proceedings of the 14th International Conference on Software Engineering (ICSE '92 , Melbourne, Australia, May 11-15), T. Montgomery, Ed. ACM Press, New York, NY , 68 - 80 . 10.1145\/143062.143093 HARROLD, M. J., MCGREGOR, J. D., AND FITZPATRICK, K.J. 1992. Incremental testing of object-oriented class structures. In Proceedings of the 14th International Conference on Software Engineering (ICSE '92, Melbourne, Australia, May 11-15), T. Montgomery, Ed. ACM Press, New York, NY, 68-80. 10.1145\/143062.143093"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1109\/TSE.1980.230467","article-title":"Functional program testing","author":"HOWDEN W. E.","year":"1980","unstructured":"HOWDEN , W. E. 1980 . Functional program testing . IEEE Trans. Softw. Eng. SE-6, 2 (Mar.) , 162 - 169 . HOWDEN, W. E. 1980. Functional program testing. IEEE Trans. Softw. Eng. SE-6, 2 (Mar.), 162-169.","journal-title":"IEEE Trans. Softw. Eng. SE-6, 2 (Mar.)"},{"key":"e_1_2_1_13_1","volume-title":"Finding Groups in Data","author":"KAUFMAN L.","unstructured":"KAUFMAN , L. AND ROUSSEEUW , P.J. 1990. Finding Groups in Data . John Wiley & amp; Sons, Inc., New York, NY. KAUFMAN, L. AND ROUSSEEUW, P.J. 1990. Finding Groups in Data. John Wiley & Sons, Inc., New York, NY."},{"key":"e_1_2_1_14_1","volume-title":"The C Programming Language","author":"KERNIGHAN B. W.","unstructured":"KERNIGHAN , B. W. AND RITCHIE , D. M. 1988. The C Programming Language . Prentice Hall Press , Upper Saddle River, NJ. KERNIGHAN, B. W. AND RITCHIE, D. M. 1988. The C Programming Language. Prentice Hall Press, Upper Saddle River, NJ."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.120314"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.199724"},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/0026-2714(78)91139-3","article-title":"Estimating software reliability from test data","volume":"17","author":"NELSON E. N.","year":"1978","unstructured":"NELSON , E. N. 1978 . Estimating software reliability from test data . Mircroelectron. Reliab. 17 , 67 - 74 . NELSON, E. N. 1978. Estimating software reliability from test data. Mircroelectron. Reliab. 17, 67-74.","journal-title":"Mircroelectron. Reliab."},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","first-page":"558","DOI":"10.2307\/2342192","article-title":"On two different aspects of the representative method: The method of stratified sampling and the method of purposive selection","volume":"97","author":"NEYMAN J.","year":"1934","unstructured":"NEYMAN , J. 1934 . On two different aspects of the representative method: The method of stratified sampling and the method of purposive selection . J. Royal Stat. Soc. B. 97 , 558 - 606 . NEYMAN, J. 1934. On two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. J. Royal Stat. Soc. B. 97, 558-606.","journal-title":"J. Royal Stat. Soc. B."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/167049.167076"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1985.232226"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 5th International Conference on Software Engineering","author":"RICHARDSON D. J.","year":"1981","unstructured":"RICHARDSON , D. J. AND CLARKE , L.A. 1981 . A partition analysis method to increase program reliability . In Proceedings of the 5th International Conference on Software Engineering ( Los Alamitos, CA). 244-253. RICHARDSON, D. J. AND CLARKE, L.A. 1981. A partition analysis method to increase program reliability. In Proceedings of the 5th International Conference on Software Engineering (Los Alamitos, CA). 244-253."},{"key":"e_1_2_1_22_1","volume-title":"Model Assisted Survey Sampling","author":"SARNDAL C.-E.","unstructured":"SARNDAL , C.-E. , SWENSSON , B. , AND WRETMAN , J. 1992. Model Assisted Survey Sampling . Springer-Verlag , New York, NY . SARNDAL, C.-E., SWENSSON, B., AND WRETMAN, J. 1992. Model Assisted Survey Sampling. Springer-Verlag, New York, NY."},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1109\/TSE.1978.231481","article-title":"An analysis of competing software reliability models","author":"SCHICK G. J.","year":"1978","unstructured":"SCHICK , G. J. AND WOLVERTON , R.W. 1978 . An analysis of competing software reliability models . IEEE Trans. Softw. Eng. SE-4, 2 (Mar.) , 104 - 120 . SCHICK, G. J. AND WOLVERTON, R.W. 1978. An analysis of competing software reliability models. IEEE Trans. Softw. Eng. SE-4, 2 (Mar.), 104-120.","journal-title":"IEEE Trans. Softw. Eng. SE-4, 2 (Mar.)"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.126769"},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/154183.154262","volume-title":"Proceedings of the 1993 International Symposium on Software Testing and Analysis (ISSTA","author":"TH VENOD-FOSSE P.","year":"1993","unstructured":"TH VENOD-FOSSE , P. AND WAESELYNCK , H. 1993 . STATEMATE applied to statistical software testing . In Proceedings of the 1993 International Symposium on Software Testing and Analysis (ISSTA , Cambridge, MA , June 28-30, 1993), T. Ostrand and E. Weyuker, Eds. ACM Press, New York, NY, 99 - 109 . 10.1145\/154183.154262 TH VENOD-FOSSE, P. AND WAESELYNCK, H. 1993. STATEMATE applied to statistical software testing. In Proceedings of the 1993 International Symposium on Software Testing and Analysis (ISSTA, Cambridge, MA, June 28-30, 1993), T. Ostrand and E. Weyuker, Eds. ACM Press, New York, NY, 99-109. 10.1145\/154183.154262"},{"key":"e_1_2_1_27_1","unstructured":"THOMPSON S. K. 1992. Sampling. John Wiley & Sons Inc. New York NY. THOMPSON S. K. 1992. Sampling. John Wiley & Sons Inc. New York NY."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.238569"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.83906"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/310663.310667","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,23]],"date-time":"2023-04-23T17:32:30Z","timestamp":1682271150000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/310663.310667"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1999,7]]},"references-count":28,"journal-issue":{"issue":"3","published-print":{"date-parts":[[1999,7]]}},"alternative-id":["10.1145\/310663.310667"],"URL":"https:\/\/doi.org\/10.1145\/310663.310667","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[1999,7]]},"assertion":[{"value":"1999-07-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}