Choose Your Words Wisely: Leveraging Embedded Dialog Trajectories to Enhance Performance in Open-Domain Conversations

Abstract

Human conversations are notoriously nondeterministic, and identical conversation histories can nevertheless accept dozens, if not hundreds, of distinct valid responses. In this paper, we present and expand upon Conversational Scaffolding, a response scoring method that capitalizes on this fundamental linguistic property. We envision a conversation as a set of trajectories through embedding space. Our method leverages the analogical structure encoded within language model representations to prioritize possible conversational responses with respect to these trajectories. Specifically, we locate candidate responses based on their linear offsets relative to the scaffold sentence pair with the greatest cosine similarity to the current conversation history. In an open-domain dialog setting, we are able to show that our method outperforms both an Approximate Nearest-Neighbor approach and a naive nearest neighbor baseline. We demonstrate our method’s performance on a retrieval-based dialog task using a retrieval dataset containing 19,665 randomly-selected sentences. We further introduce a comparative analysis of algorithm performance as a function of contextual alignment strategy, with accompanying discussion.