Selective Self-attention with BIO Embeddings for Distant Supervision Relation Extraction

Abstract

Distant supervision method is proposed to label instances automatically, which could operate relation extraction without human annotations. However, the training data generated in this way intrinsically include massive noise. To alleviate this problem, attention mechanism is employed by most prior works that achieves significant improvements but could be still imcompetent for one-sentence bags which means only one sentence within a bag. To this end, in this paper, we propose a novel neural relation extraction method employing BIO embeddings and a selective self-attention with fusion gate mechanism to fix the aformentioned defects in previous attention methods. First, in addition to commonly adopted embedding methods in input layer, we propose to add BIO embeddings to enrich the input representation. Second, a selective self-attention mechanism is proposed to capture context dependency information and combined with PCNN via a Fusion Gate module to enhance the representation of sentences. Experiments on the NYT dataset demonstrate the effectiveness of our proposed methods and our model achieves consistent improvements for relation extraction compared to the state-of-the-art methods.