Repository logo
 

Real-Time Onboard Object Detection for Augmented Reality: Enhancing Head-Mounted Display with YOLOv8

Accepted version
Peer-reviewed

Type

Conference Object

Change log

Authors

Lysakowski, M 
Zywanowski, K 
Banaszczyk, A 
Nowicki, MR 
Skrzypczynski, P 

Abstract

This paper introduces a software architecture for real-time object detection using machine learning (ML) in an augmented reality (AR) environment. Our approach uses the recent state-of-the-art YOLOv8 network that runs onboard on the Microsoft HoloLens 2 head-mounted display (HMD). The primary motivation behind this research is to enable the application of advanced ML models for enhanced perception and situational awareness with a wearable, hands-free AR platform. We show the image processing pipeline for the YOLOv8 model and the techniques used to make it real-time on the resource-limited edge computing platform of the headset. The experimental results demonstrate that our solution achieves real-time processing without needing offloading tasks to the cloud or any other external servers while retaining satisfactory accuracy regarding the usual mAP metric and measured qualitative performance.

Description

Keywords

Journal Title

Proceedings - IEEE International Conference on Edge Computing

Conference Name

2023 IEEE Symposium on intelligent Edge Computing and Communications (EDGE)

Journal ISSN

2767-9918

Volume Title

Publisher

Sponsorship
The study has been supported by funding provided through an unrestricted gift by Meta.