Construction progress is predominantly measured with manual site surveys. These surveys are labor-intensive, on-site manual investigations. The generated survey reports are subjective and approximate because they are based on the surveyors' individual experiences. This paper presents a novel method that can automatically count the number of bricks on a façade for reducing the cost and increasing the reliability of progress surveys. The method uses video data taken from a user's mobile phone to detect bricks on a façade in each video frame by using color thresholding, edge detection, and filtering of rectangular shapes and sizes. Then, it compares the difference between consecutive frames to add counts when new bricks appear and to avoid double counting. The proposed method was implemented and tested on on-site videos of red brick façades, and resulted in 99.8% precision and 98.7% recall. The results demonstrate the suitability of this method for progress monitoring of brick façade construction.