A key focus in AI is building machines and software capable of being autonomous, especially in complex and dynamic environments where, e.g., self driving cars, trading systems, and social care robots operate. Such autonomous systems are able to independently make decisions and act on them with limited human intervention, balancing the pursuit of long-term goals (proactiveness) with rapid response to environmental changes (reactiveness) (Fisher et al. 2021). The notion of an autonomous system is synonymous with the notion of an ‘autonomous software agent’ (Fisher et al. 2021), and a class of domain-specific language called an Agent-Oriented Programming Language (AOPL) has proved to be one of the most successful approaches to building such systems. AOPLs provide abstractions over Object-Oriented Programming, by modelling complex systems through the ‘intentional stance’ – human-like mental attitudes such as beliefs, goals, and intentions, enabling users understand, explain, predict, and program behaviour by abstracting from the detail (objects, attributes, etc.). Indeed, giving people this ability helps build trustworthy AI systems, particularly those that people can trust to have been designed and programmed to be lawful, ethical, and robust, ensuring adherence to applicable laws, regulations, and ethical principles, and operating in a safe, secure and reliable manner.