With the proliferation of IoT devices in households, network-level management is essential for users' security and control. Identifying IoT devices through their network profiles enables the detection of anomalies, such as hacking attempts, misconfigurations, or firmware updates. However, the variability in the types and numbers of devices across households makes training separate models for each household or a single global model impractical. Solutions based on single multiclass classification are not scalable considering the diverse range of devices in households and the constant introduction of new devices. In this technical report, we propose a system that employs separate binary classification models for each device. We evaluate its accuracy in classifying the network traffic over a long period of time. We also investigate the decline in accuracy over time and propose mitigation strategies. Furthermore, we assess the models' sensitivity to changes in network traffic patterns.

To address this, our system utilizes scalable binary classification models
that can be adjusted to individual households by downloading only the
necessary device-specific models. The system is protocol-agnostic and
capable of classifying network traffic, whether it is local network
communication or over the Internet. Our results show a promising F1 score of
90-95% on the trained dataset, with accuracy remaining around 80% even
after three months. This indicates that periodic model retraining every
three months is sufficient.

Additionally, we evaluate the system's ability to detect non-device traffic
and find that it can detect deviations even when only 30\% of the traffic
does not belong to the device. This demonstrates the system's sensitivity to
changes in an IoT device's network communication profile, providing users
with potential insights into device-related issues.