Background. Several psychometric instruments are available for the diagnostic interview of subjects at ultra high risk (UHR) of psychosis. Their diagnostic comparability is unknown. Methods. All referrals to the OASIS (London) or CAMEO (Cambridgeshire) UHR services from May 13 to Dec 14 were interviewed for a UHR state using both the CAARMS 12/2006 and the SIPS 5.0. Percent overall agreement, kappa, the McNemar-Bowker χ (2) test, equipercentile methods, and residual analyses were used to investigate diagnostic outcomes and symptoms severity or frequency. A conversion algorithm (CONVERT) was validated in an independent UHR sample from the Seoul Youth Clinic (Seoul). Results. There was overall substantial CAARMS-versus-SIPS agreement in the identification of UHR subjects (n = 212, percent overall agreement = 86%; kappa = 0.781, 95% CI from 0.684 to 0.878; McNemar-Bowker test = 0.069), with the exception of the brief limited intermittent psychotic symptoms (BLIPS) subgroup. Equipercentile-linking table linked symptoms severity and frequency across the CAARMS and SIPS. The conversion algorithm was validated in 93 UHR subjects, showing excellent diagnostic accuracy (CAARMS to SIPS: ROC area 0.929; SIPS to CAARMS: ROC area 0.903). Conclusions. This study provides initial comparability data between CAARMS and SIPS and will inform ongoing multicentre studies and clinical guidelines for the UHR psychometric diagnostic interview.