Global agriculture is poised to benefit from the rapid advance and diffusion of artificial intelligence (AI) technologies. AI in agriculture could improve crop management and agricultural productivity through plant phenotyping, rapid diagnosis of plant disease, efficient application of agrochemicals and assistance for growers with location-relevant agronomic advice. However, the ramifications of machine learning (ML) models, expert systems and autonomous machines for farms, farmers and food security are poorly understood and under-appreciated. Here, we consider systemic risk factors of AI in agriculture. Namely, we review risks relating to interoperability, reliability and relevance of agricultural data, unintended socio-ecological consequences resulting from ML models optimized for yields, and safety and security concerns associated with deployment of ML platforms at scale. As a response, we suggest risk-mitigation measures, including inviting rural anthropologists and applied ecologists into the technology design process, applying frameworks for responsible and human-centred innovation, setting data cooperatives for improved data transparency and ownership rights, and initial deployment of agricultural AI in digital sandboxes.