Repository logo

Autonomous detection and sorting of litter using deep learning and soft robotic grippers

Published version

Change log


Almanzor, Elijah 
Anvo, Nzebo Richard 
Thuruthel, Thomas George 
Iida, Fumiya 


jats:pRoad infrastructure is one of the most vital assets of any country. Keeping the road infrastructure clean and unpolluted is important for ensuring road safety and reducing environmental risk. However, roadside litter picking is an extremely laborious, expensive, monotonous and hazardous task. Automating the process would save taxpayers money and reduce the risk for road users and the maintenance crew. This work presents LitterBot, an autonomous robotic system capable of detecting, localizing and classifying common roadside litter. We use a learning-based object detection and segmentation algorithm trained on the TACO dataset for identifying and classifying garbage. We develop a robust modular manipulation framework by using soft robotic grippers and a real-time visual-servoing strategy. This enables the manipulator to pick up objects of variable sizes and shapes even in dynamic environments. The robot achieves greater than 80% classified picking and binning success rates for all experiments; which was validated on a wide variety of test litter objects in static single and cluttered configurations and with dynamically moving test objects. Our results showcase how a deep model trained on an online dataset can be deployed in real-world applications with high accuracy by the appropriate design of a control framework around it.</jats:p>


Peer reviewed: True

Acknowledgements: The authors would like to thank Costain Group PLC and National Highways as the partners in this Prosperity Partnership.


Robotics and AI, AI-driven control, soft robotics, deep learning, visual servoing, litter picking

Journal Title

Frontiers in Robotics and AI

Conference Name

Journal ISSN


Volume Title


Frontiers Media SA