This article compares a conventional diesel-fuelled heavy goods vehicle with one that runs on compressed gas (CG): either Compressed Natural Gas (CNG) or biomethane. It includes an evaluation of the transport performance, carbon emissions, and total costs. 41 CG trucks were trialled by a supermarket, delivering goods from their distribution centre. One of the vehicles was instrumented with a gas flow meter, which measured the gas consumption, and a smartphone based data logger, which collected data from the truck's Fleet Management System (FMS) interface. Similarly, a baseline diesel truck was instrumented. Based on the measurements that were collected in-service, fuel consumption models were developed and validated. Using the validated models, the CG and diesel trucks were compared for their transport performance, costs and carbon emissions. In addition, the vehicles were evaluated using telematics data and refuelling data from a year of transport operations for all 41 CG trucks and 9 diesel trucks. The results show that for a tonne.km of in-service (mostly long haul) transport work, compared to the diesel trucks, the CG trucks spend on average 22% more energy and their fuel cost is 30% lower for biomethane. When running on CNG, their equivalent carbon emissions is 10% lower than the baseline diesel, and when running on biomethane, their equivalent carbon emissions is 78% lower. Compared to long haul, the benefits are lower for regional operations, and are the lowest for city centre operations where using CNG, the CG truck's equivalent carbon emissions is higher than the diesel truck.