<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhou-shah-2023-solvability">
    <titleInfo>
        <title>The Solvability of Interpretability Evaluation Metrics</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yilun</namePart>
        <namePart type="family">Zhou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Julie</namePart>
        <namePart type="family">Shah</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Feature attribution methods are popular for explaining neural network predictions, and they are often evaluated on metrics such as comprehensiveness and sufficiency. In this paper, we highlight an intriguing property of these metrics: their solvability. Concretely, we can define the problem of optimizing an explanation for a metric, which can be solved by beam search. This observation leads to the obvious yet unaddressed question: why do we use explainers (e.g., LIME) not based on solving the target metric, if the metric value represents explanation quality? We present a series of investigations showing strong performance of this beam search explainer and discuss its broader implication: a definition-evaluation duality of interpretability concepts. We implement the explainer and release the Python solvex package for models of text, image and tabular domains.</abstract>
    <identifier type="citekey">zhou-shah-2023-solvability</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-eacl.182</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-eacl.182</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>2399</start>
            <end>2415</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Solvability of Interpretability Evaluation Metrics
%A Zhou, Yilun
%A Shah, Julie
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Findings of the Association for Computational Linguistics: EACL 2023
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F zhou-shah-2023-solvability
%X Feature attribution methods are popular for explaining neural network predictions, and they are often evaluated on metrics such as comprehensiveness and sufficiency. In this paper, we highlight an intriguing property of these metrics: their solvability. Concretely, we can define the problem of optimizing an explanation for a metric, which can be solved by beam search. This observation leads to the obvious yet unaddressed question: why do we use explainers (e.g., LIME) not based on solving the target metric, if the metric value represents explanation quality? We present a series of investigations showing strong performance of this beam search explainer and discuss its broader implication: a definition-evaluation duality of interpretability concepts. We implement the explainer and release the Python solvex package for models of text, image and tabular domains.
%R 10.18653/v1/2023.findings-eacl.182
%U https://aclanthology.org/2023.findings-eacl.182
%U https://doi.org/10.18653/v1/2023.findings-eacl.182
%P 2399-2415

Markdown (Informal)

[The Solvability of Interpretability Evaluation Metrics](https://aclanthology.org/2023.findings-eacl.182) (Zhou & Shah, Findings 2023)

• The Solvability of Interpretability Evaluation Metrics (Zhou & Shah, Findings 2023)

ACL

• Yilun Zhou and Julie Shah. 2023. The Solvability of Interpretability Evaluation Metrics. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2399–2415, Dubrovnik, Croatia. Association for Computational Linguistics.