Over the past decades, the construction industry lags further and further behind the manufacturing sector when productivity is considered. This is due to internal factors that take place on-site. Almost all of them are directly related to the way that productivity is monitored. Current practices for monitoring labour productivity are labour intensive, time - cost consuming and error prone. They are mainly reactive processes initiated after the detection of a negatively influencing factor. Although research studies have been performed towards leveraging these limitations, a gap still exists in monitoring labour productivity of multiple workers at the same time accurately, unobtrusively, cost and time efficiently. This thesis proposes a framework to address this gap. It hypothesizes that task productivity of construction workers can be monitored through their trajectory data. The proposed framework uses as input, video data streamed from cameras with overlapping field of view. It consists of two main methods. The output of the first is the input of the second. The first method tracks the location of workers across the range of a jobsite over time and returns their 4D trajectories. Such type of tracking requires that workers are matched under a unique ID not only across successive frames of a single camera (intra tracking) but also across multiple cameras (inter tracking). Existing tag-less studies fail to track construction workers due to the challenging nature of their working environments. Therefore, two novel computer vision-based algorithms are developed to perform both the intra and the inter camera tracking. The second method of the proposed framework converts the 4D trajectories of workers into productivity information. These trajectories are clustered into work cycles with an accuracy of 95%, recall of 76% and precision of 76%. Such work cycles depict the actual execution of tasks. The overall proposed framework features an average accuracy of 95% in terms of determining the total time workers spend on construction-related tasks.