More than 9 million people live in shelters globally, often in extremely hot climates. The thermal performance of shelters is often overlooked in the design process, despite being a consideration second only to safety in surveys of camp dwellers. Indeed, indoor temperatures exceeding 40°C have been recorded in previous studies. To aid in improving conditions, the roles building simulation and prototyping could play in forecasting shelter thermal performance as part of a new shelter design process are examined. The thermal performance of prototypes, built in the refugee camp of Azraq, was monitored during the hot season to evaluate four design approaches: (1) “blind” (uncalibrated) models, (2) calibrated models, (3) on-site design-variants and (4) off-site prototypes. These included the original shelter and six design alternatives implementing different overheating countermeasures. The results demonstrate that blind models are sensitive to the judgement of uncertainties but were still qualitatively useful. Model calibration vastly improves the agreement and significantly enhances forecasts of performance for the design alternatives, which remained similar across examined climates. It is therefore concluded that simulation and prototyping, either on-site or off-site, should be adopted within the shelter design process before mass deployment, to create better-living conditions for their dwellers.