Repository logo

Autonomous detection and sorting of litter using deep learning and soft robotic grippers.

Accepted version



Change log


Anvo, Nzebo Richard 
Thuruthel, Thomas George 
Iida, Fumiya 


Road infrastructure is one of the most vital assets of any country. Keeping the road infrastructure clean and unpolluted is important for ensuring road safety and reducing environmental risk. However, roadside litter picking is an extremely laborious, expensive, monotonous and hazardous task. Automating the process would save taxpayers money and reduce the risk for road users and the maintenance crew. This work presents LitterBot, an autonomous robotic system capable of detecting, localizing and classifying common roadside litter. We use a learning-based object detection and segmentation algorithm trained on the TACO dataset for identifying and classifying garbage. We develop a robust modular manipulation framework by using soft robotic grippers and a real-time visual-servoing strategy. This enables the manipulator to pick up objects of variable sizes and shapes even in dynamic environments. The robot achieves greater than 80% classified picking and binning success rates for all experiments; which was validated on a wide variety of test litter objects in static single and cluttered configurations and with dynamically moving test objects. Our results showcase how a deep model trained on an online dataset can be deployed in real-world applications with high accuracy by the appropriate design of a control framework around it.



AI-driven control, deep learning, litter picking, soft robotics, visual servoing

Journal Title

Front Robot AI

Conference Name

Journal ISSN


Volume Title


Frontiers Media
EPSRC (EP/V056441/1)
European Commission Horizon 2020 (H2020) Future and Emerging Technologies (FET) (828818)
EPSRC (via University Of Lincoln) (EP/S023917/1)
This work was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) [grant number EP/V056441/1], the SHERO project, a Future and Emerging Technologies (FET) programme of the European Commission [grant agreement ID 828818], AgriFoRwArdS Centre for Doctoral Training programme under the UKRI grant [EP/S023917/1], and the Jersey Farmers Union. The authors would like to thank Costain Group PLC and National Highways as the partners in this Prosperity Partnership.
Is supplemented by: