Repository logo
 

Direct Human-AI Comparison in the Animal-AI Environment.

Published version
Peer-reviewed

Change log

Authors

Voudouris, Konstantinos 
Crosby, Matthew 
Beyret, Benjamin 
Hernández-Orallo, José 
Shanahan, Murray 

Abstract

Artificial Intelligence is making rapid and remarkable progress in the development of more sophisticated and powerful systems. However, the acknowledgement of several problems with modern machine learning approaches has prompted a shift in AI benchmarking away from task-oriented testing (such as Chess and Go) towards ability-oriented testing, in which AI systems are tested on their capacity to solve certain kinds of novel problems. The Animal-AI Environment is one such benchmark which aims to apply the ability-oriented testing used in comparative psychology to AI systems. Here, we present the first direct human-AI comparison in the Animal-AI Environment, using children aged 6-10 (n = 52). We found that children of all ages were significantly better than a sample of 30 AIs across most of the tests we examined, as well as performing significantly better than the two top-scoring AIs, "ironbar" and "Trrrrr," from the Animal-AI Olympics Competition 2019. While children and AIs performed similarly on basic navigational tasks, AIs performed significantly worse in more complex cognitive tests, including detour tasks, spatial elimination tasks, and object permanence tasks, indicating that AIs lack several cognitive abilities that children aged 6-10 possess. Both children and AIs performed poorly on tool-use tasks, suggesting that these tests are challenging for both biological and non-biological machines.

Description

Keywords

AI benchmarks, Animal-AI Olympics, artificial intelligence, cognitive AI, comparative cognition, human-AI comparison, out-of-distribution testing

Journal Title

Front Psychol

Conference Name

Journal ISSN

1664-1078
1664-1078

Volume Title

13

Publisher

Frontiers Media SA
Sponsorship
ESRC (ES/P000738/1)