User Trust and Judgments in a Curated Database with Explicit Provenance

Abstract

We focus on human-in-the-loop, information-integration settings where users gather and evaluate data from a broad variety of sources and where the levels of trust in sources and users change dynamically. In such settings, users must use their judgment as they collect and modify data. As an example, a battlefield information officer preparing a report to inform his or her superiors about the current state of affairs must gather and integrate data from many (including non-computerized) sources. By tracking multiple sources for individual values, the officer may eliminate a value from the current state whenever all of the sources where this value was found are no longer trusted. We define a conceptual model for a curated database with provenance for such settings, the Multi-granularity, Multi-provenance Model (MMP), which supports multiple insertions and multiple (copy-and-)paste operations for a single database element, captures the external source for all operations, and includes a Data Confidence Language that allows users to confirm or doubt values to record their atomic judgments about the data. In this paper, we briefly summarize the MMP model and show how it can be extended to support potentially complex operations including compound judgment operators (such as merging tuples to achieve entity resolution), while capturing a complete record of data provenance.