Full Bayesian Significance Testing for Neural Networks

Authors

  • Zehua Liu School of Computer Science and Engineering, Beihang University, Beijing, China
  • Zimeng Li School of Computer Science and Engineering, Beihang University, Beijing, China
  • Jingyuan Wang School of Computer Science and Engineering, Beihang University, Beijing, China School of Economics and Management, Beihang University, Beijing, China Key Laboratory of Data Intelligence and Management (Beihang University), Ministry of Industry and Information Technology, Beijing, China
  • Yue He Department of Computer Science and Technology, Tsinghua University, Beijing, China

DOI:

https://doi.org/10.1609/aaai.v38i8.28731

Keywords:

DMKM: Representing, Reasoning, and Using Provenance, Trust

Abstract

Significance testing aims to determine whether a proposition about the population distribution is the truth or not given observations. However, traditional significance testing often needs to derive the distribution of the testing statistic, failing to deal with complex nonlinear relationships. In this paper, we propose to conduct Full Bayesian Significance Testing for neural networks, called nFBST, to overcome the limitation in relationship characterization of traditional approaches. A Bayesian neural network is utilized to fit the nonlinear and multi-dimensional relationships with small errors and avoid hard theoretical derivation by computing the evidence value. Besides, nFBST can test not only global significance but also local and instance-wise significance, which previous testing methods don't focus on. Moreover, nFBST is a general framework that can be extended based on the measures selected, such as Grad-nFBST, LRP-nFBST, DeepLIFT-nFBST, LIME-nFBST. A range of experiments on both simulated and real data are conducted to show the advantages of our method.
AAAI-24 / IAAI-24 / EAAI-24 Proceedings Cover

Downloads

Published

2024-03-24

How to Cite

Liu, Z., Li, Z., Wang, J., & He, Y. (2024). Full Bayesian Significance Testing for Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 38(8), 8841-8849. https://doi.org/10.1609/aaai.v38i8.28731

Issue

Vol. 38 No. 8: AAAI-24 Technical Tracks 8

Section

AAAI Technical Track on Data Mining & Knowledge Management