In this response we particularly focus on defence and those in adjacent markets systems that integrate increasingly capable artificial intelligence (AI), especially those based on machine learning (ML). Many systems that the Ministry of Defence (MoD) is likely to procure over the next 5-10 years will integrate AI and ML; these systems are likely to both be strategically important and to introduce new vulnerabilities. These vulnerabilities are likely to pose significant national security risks over the next few decades, both for the UK and the UK’s allies. These systems are the focus of much of our work, and where we hope to add our expertise to the Committee’s Inquiry. We make the following recommendations to protect against premature and/or unsafe procurement and deployment of ML-based systems:

• Improve systemic risk assessment in defence procurement.
• Ensure clear lines of responsibility so that senior officials can be held responsible for errors caused in the procurement chain and are therefore incentivised to reduce them;
• Acknowledge potential shifts in international standards for autonomous systems, and build flexible procurement standards accordingly.
• Update the MoD’s definition of lethal autonomous weapons - the Integrated Security, Defence and Foreign Policy Review provides an excellent opportunity to bring the UK in line with its allies.