The Study and Design of Human-AI Thought Partnerships

Abstract

Humans have long thought together. Advances in AI supports multi-turn conversation in natural language fundamentally alters our relationship with AI and teases the possibility that AI systems too can think together with us: moving from tools for thought to potentially being "partners'' in thought. What is needed to build truly human-compatible AI systems that can function as "thought partners'' -- that meet our expectations and complement our limitations? In this thesis, we make a case for how addressing this question requires deep engagement with human cognition and AI system behavior. This new form of human-AI thought partnerships demands an even richer awareness of the uncertainty in our ever-changing world. To empower effective and responsible human-AI partnerships in practice, we argue it is essential to treat uncertainty as a first-class citizen in development and deployment. We lay out a series of desiderata for what we may want in AI thought partnerships that can engage with people, in an uncertain world. We first take steps to explore these desiderata in the context of a comparatively simpler AI system (concept-based models) that offers a controlled and externally-valid playground for exploring the impact of the form of human uncertainty on the ability of AI systems to uptake such feedback and improve downstream performance. We find that this particular class of model struggle to handle human uncertainty and that while training with soft targets can improve test-time robustness, there remains a core gap in uptake receptiveness. Similarly, we uncover an important trade-off in the degree of uncertainty we elicit from people: attempting to capture too many alternative possibilities can hamper the learning signal, whereas too coarse falls prey to brittleness. We highlight how these data can inform human-AI partnership design. We then pivot to focus on evaluation: human-AI thought partnerships with their capacity multi-turn conversation, demand new methods for scalable human-centric evaluation. We contribute a new platform for interactive evaluation, in the context of mathematics. Our evaluations highlight the importance of a deep understanding of people (and their own uncertainty) for faithful evaluation of thought partnerships to determine whether such systems meet the desiderata we lay out for good thought partners. We apply these insights to the development of adaptive interfaces to help guide people's use of AI. We then take a step back and rethink what it may take to scalably build AI thought partners that meet our desiderata. We point to modeling motifs from computational cognitive science that can support just that. As a proof-of-concept, we highlight early computational modeling, drawn richly from cognitive science, for how we may be able to scalably synthesize models of the world and that align with people’s uncertainty over new problems. We close with future directions informed by this work towards the design of meaningful long-term human-AI thought partnerships that adapt with us and empower new forms of joyful collaborative cognition.