Explaining counterexamples using causality | Formal Methods in System Design Skip to main content
Springer Nature Link
Log in

Explaining counterexamples using causality

  • Published:
Formal Methods in System Design Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

When a model does not satisfy a given specification, a counterexample is produced by the model checker to demonstrate the failure. A user must then examine the counterexample trace, in order to visually identify the failure that it demonstrates. If the trace is long, or the specification is complex, finding the failure in the trace becomes a non-trivial task. In this paper, we address the problem of analyzing a counterexample trace and highlighting the failure that it demonstrates. Using the notion of causality introduced by Halpern and Pearl, we formally define a set of causes for the failure of the specification on the given counterexample trace. These causes are marked as red dots and presented to the user as a visual explanation of the failure. We study the complexity of computing the exact set of causes, and provide a polynomial-time algorithm that approximates it. We then analyze the output of the algorithm and compare it to the one expected by the definition. The algorithm is implemented as a feature in the IBM formal verification platform RuleBase PE, where the visual explanations are an integral part of every counterexample trace. Our approach is independent of the tool that produced the counterexample, and can be applied as a light-weight external layer to any model checking tool, or used to explain simulation traces.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Red dots are used in IBM’S RuleBase PE for displaying counterexample explanations.

References

  1. Prosyd: Property-based system design (2005). http://www.prosyd.org/

  2. Ball T, Naik M, Rajamani S (2003) From symptom to cause: Localizing errors in counterexample traces. In: Principles of programming languages, pp 97–105

    Google Scholar 

  3. Ben-David S, Eisner C, Geist D, Wolfsthal Y (2003) Model checking at IBM. Form Methods Syst Des 22(2):101–108

    Article  MATH  Google Scholar 

  4. Chechik M, Gurfinkel A (2005) A framework for counterexample generation and exploration. In: Proceedings of FASE’05, pp 217–233

    Google Scholar 

  5. Chockler H, Grumberg O, Yadgar A (2008) Efficient automatic STE refinement using responsibility. In: TACAS, pp 233–248

    Google Scholar 

  6. Chockler H, Halpern JY, Kupferman O (2008) What causes a system to satisfy a specification? ACM Trans Comput Log 9(3)

  7. Clarke E, Emerson E (1981) Design and synthesis of synchronization skeletons using branching time temporal logic. In: Proc workshop on logic of programs, Lecture notes in computer science, vol 131. Springer, Berlin, pp 52–71

    Chapter  Google Scholar 

  8. Clarke E, Grumberg O, McMillan K, Zhao X (1995) Efficient generation of counterexamples and witnesses in symbolic model checking. In: Proc 32nd design automation conference. IEEE Comput Soc, Los Alamitos, pp 427–432

    Google Scholar 

  9. Copty F, Irron A, Weissberg O, Kropp N, Kamhi G (2001) Efficient debugging in a formal verification environment. In: Proceedings of CHARME’01, pp 275–292

    Google Scholar 

  10. Dershowitz N, Hanna Z, Nadel A (2006) A scalable algorithm for minimal unsatisfiable core extraction. In: Proceedings of 9th international conference on theory and applications of satisfiability testing (SAT), pp 36–41

    Chapter  Google Scholar 

  11. Dong Y, Ramakrishnan CR, Smolka SA (2003) Model checking and evidence exploration. In: IEEE conference and workshops on engineering computer based systems, IEEE, Huntsville, pp 214–223

    Google Scholar 

  12. Eisner C, Fisman D (2006) A practical introduction to PSL. Series on integrated circuits and systems. Springer, Berlin

    Google Scholar 

  13. Eisner C, Fisman D, Havlicek J, Lustig Y, McIsaac A, Campenhout DV (2003) Reasoning with temporal logic on truncated paths. In: Proc 15th intl conference on computer aided verification (CAV’03). Lect notes in comp sci, vol 2725. Springer, Boulder, pp 27–39

    Google Scholar 

  14. Eiter T, Lukasiewicz T (2001) Complexity results for structure-based causality. In: Proc 7th international joint conference on artificial intelligence, pp 35–40

    Google Scholar 

  15. Eiter T, Lukasiewicz T (2002) Causes and explanations in the structural-model approach: Tractable cases. In: Uncertainty in artificial intelligence: proceedings of the 18th conference (UAI-2002). Morgan Kaufmann, Edmonton

    Google Scholar 

  16. Griesmayer A, Staber S, Bloem R (2007) Automated fault localization for C programs. Electron Notes Theor Comput Sci 174(4):95–111

    Article  Google Scholar 

  17. Groce A (2004) Error explanation with distance metrics. In: TACAS

    Google Scholar 

  18. Groce A, Chaki S, Kroening D, Strichman O (2006) Error explanation with distance metrics. STTT 8(3):229–247

    Article  Google Scholar 

  19. Groce A, Kroening D (2004) Making the most of BMC counterexamples. In: SGSH

    Google Scholar 

  20. Hall N (2002) Two concepts of causation. In: Collins J, Hall N, Paul LA (eds) Causation and counterfactuals. MIT Press, Cambridge

    Google Scholar 

  21. Halpern J, Pearl J (2001) Causes and explanations: a structural-model approach—part 1: Causes. In: Uncertainty in artificial intelligence: proceedings of the seventeenth conference (UAI-2001). Morgan Kaufmann, San Francisco, pp 194–202

    Google Scholar 

  22. Halpern J, Pearl J (2005) Causes and explanations: a structural-model approach—part 1: Causes. Br J Philos Sci 56:843–887

    Google Scholar 

  23. Hume D (1939) A treatise of human nature. John Noon, London

    Google Scholar 

  24. Jin H, Ravi K, Somenzi F (2002) Fate and free will in error traces. In: TACAS02, pp 445–458

    Google Scholar 

  25. Kupferman O, Vardi M, Wolper P (2000) An automata-theoretic approach to branching-time model checking. J ACM 47(2):312–360

    Article  MathSciNet  MATH  Google Scholar 

  26. Manna Z, Pnueli A (1995) The temporal logic of reactive and concurrent systems: safety. Springer, New York

    Google Scholar 

  27. Pnueli A (1977) The temporal logic of programs. In: 18th IEEE symposium on foundation of computer science, pp 46–57

    Google Scholar 

  28. Quielle J, Sifakis J (1982) Specification and verification of concurrent systems in CESAR. In: 5th international symposium on programming

    Google Scholar 

  29. RuleBase PE Homepage. http://www.haifa.il.ibm.com/projects/verification/RB_Homepage

  30. Shen S, Qin Y, Li S (2005) A faster counterexample minimization algorithm based on refutation analysis. In: DATE05, pp 672–677

    Google Scholar 

  31. Staber S, Bloem R (2007) Fault localization and correction with QBF. In: SAT, pp 355–368

    Google Scholar 

  32. Sülflow A, Fey G, Bloem R, Drechsler R (2008) Using unsatisfiable cores to debug multiple design errors. In: ACM great lakes symposium on VLSI, pp 77–82

    Google Scholar 

  33. Wang C, Yang Z, Ivancic F, Gupta A (2006) Whodunit? Causal analysis for counterexamples. In: ATVA, pp 82–95

    Google Scholar 

  34. Wolper P (1985) The tableau method for temporal logic: an overview. Log Anal 110–111:119–136

    MathSciNet  Google Scholar 

Download references

Acknowledgements

We thank Cindy Eisner for helpful discussions on the truncated semantics. Shoham Ben-David is grateful to the Azrieli Foundation for the award of an Azrieli Fellowship.

Author information

Authors and Affiliations

  1. IBM Research, Mount Carmel, Haifa, 31905, Israel

    Ilan Beer, Hana Chockler & Avigail Orni

  2. The Hebrew University, Jerusalem, 91904, Israel

    Shoham Ben-David

  3. David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada

    Richard Trefler

Authors
  1. Ilan Beer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shoham Ben-David
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hana Chockler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Avigail Orni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Trefler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hana Chockler.

Additional information

A preliminary version of the paper appeared in the proceedings of CAV 2009.

The research of S. Ben-David was supported in part by the Ontario Ministry of Research and Innovation and by the Azrieli Foundation.

The research of R. Trefler was supported in part by the Natural Sciences and Engineering Research Council of Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Beer, I., Ben-David, S., Chockler, H. et al. Explaining counterexamples using causality. Form Methods Syst Des 40, 20–40 (2012). https://doi.org/10.1007/s10703-011-0132-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10703-011-0132-2

Keywords

Search

Navigation