Repository logo
 

Explanations for Autonomous Agents


Type

Thesis

Change log

Authors

Raymond, Alexandre 

Abstract

Recent years have seen an accelerated development of agents and systems capable of sophisticated autonomous behaviour. As the consequences of such agents' actions begin to manifest in society, the need for understanding their decisions motivates the study of mechanisms for obtaining explanations that are compatible with human reasoning. However, the design of explainable systems often does not consider the impact that explanations could bring to machine and human agents alike. This thesis explores this challenge.

Our approach begins by looking at decentralised environments with complex regulations, where explanations must be exchanged to ensure orderly interactions between agents. To convert human rulesets into machine-compatible reasoning mechanisms, we propose an argumentation-based human-agent architecture to map human regulations into a culture for artificial agents with explainable behaviour. Our user studies in a hybrid, explainable, human-agent setting show that system complexity is a determining factor for the usefulness of explanations for humans. For autonomous agents, privacy and partial observability can introduce a notion of subjective unfairness in decentralised systems. We show that this effect can also be mitigated with the use of effective explanations.

In like manner, we look at Reinforcement Learning (RL) agents and investigate the possibility to orient the learning mechanism with explainable features. We call this process Explanation-Aware Experience Replay (XAER) and demonstrate that explanation engineering can be used in lieu of reward engineering for environments with explainable features. Further, we extend this concept into multi-agent RL and show how exchanging explanations in environments with partial observability can be used to obtain a more robust and effective collective behaviour.

Our conclusion is that the design of explainable systems should not only consider the generation of explanations, but also their consumption. Explanations can serve as tools for communicating precise and distilled information, and the insights gained by human agents could also be gained by machine agents, especially in systems with decentralised agency or partial knowledge.

Description

Date

2022-06-27

Advisors

Prorok, Amanda
Gunes, Hatice

Keywords

argumentation frameworks, autonomous agents, explanations, multi-agent systems

Qualification

Doctor of Philosophy (PhD)

Awarding Institution

University of Cambridge
Sponsorship
Royal Commission for the Exhibition of 1851 and L3Harris ASV