ConTag has generated open datasets for visual machine learning (ML) specific to the construction industry. ML technology has enabled a revolutionary leap in many digital economies generating growth in activity and business mainly for the ITC sector. Part of the growth is generated through sharing of IP, knowledge, tools and datasets. We want to adopt this approach for the digital construction sector. ConTag provides visual and 3D training datasets for training deep neural networks (DNNs) and provides weights for pre-trained networks. The research output is to support visual tagging of assets from reality capture data. Such automatically generated semantic information can be used to generate or populate digital twins in the example scenarios. The first dataset is a collection of fire safety equipment typically found in indoor environments. The dataset contains the classified images, per-pixel label images and bounding box data for object detection. The second dataset is a synthetic 3D point cloud of an outdoor urban street scenario. The dataset contains the point cloud data and per-point label data. We expect this shared and open datasets to kick-start further ML developments in both academia and industry. It is intended as a seed point for collaborative research.