De-identified Bayesian personal identity matching for privacy-preserving record linkage despite errors: development and validation
Published version
Peer-reviewed
Repository URI
Repository DOI
Type
Change log
Authors
Abstract
jats:titleAbstract</jats:title>jats:sec jats:titleBackground</jats:title> jats:pEpidemiological research may require linkage of information from multiple organizations. This can bring two problems: (1) the information governance desirability of linkage without sharing direct identifiers, and (2) a requirement to link databases without a common person-unique identifier.</jats:p> </jats:sec>jats:sec jats:titleMethods</jats:title> jats:pWe develop a Bayesian matching technique to solve both. We provide an open-source software implementation capable of de-identified probabilistic matching despite discrepancies, via fuzzy representations and complete mismatches, plus de-identified deterministic matching if required. We validate the technique by testing linkage between multiple medical records systems in a UK National Health Service Trust, examining the effects of decision thresholds on linkage accuracy. We report demographic factors associated with correct linkage.</jats:p> </jats:sec>jats:sec jats:titleResults</jats:title> jats:pThe system supports dates of birth (DOBs), forenames, surnames, three-state gender, and UK postcodes. Fuzzy representations are supported for all except gender, and there is support for additional transformations, such as accent misrepresentation, variation for multi-part surnames, and name re-ordering. Calculated log odds predicted a proband’s presence in the sample database with an area under the receiver operating curve of 0.997–0.999 for non-self database comparisons. Log odds were converted to a decision via a consideration threshold jats:italicθ</jats:italic> and a leader advantage threshold jats:italicδ</jats:italic>. Defaults were chosen to penalize misidentification 20-fold versus linkage failure. By default, complete DOB mismatches were disallowed for computational efficiency. At these settings, for non-self database comparisons, the mean probability of a proband being correctly declared to be in the sample was 0.965 (range 0.931–0.994), and the misidentification rate was 0.00249 (range 0.00123–0.00429). Correct linkage was positively associated with male gender, Black or mixed ethnicity, and the presence of diagnostic codes for severe mental illnesses or other mental disorders, and negatively associated with birth year, unknown ethnicity, residential area deprivation, and presence of a pseudopostcode (e.g. indicating homelessness). Accuracy rates would be improved further if person-unique identifiers were also used, as supported by the software. Our two largest databases were linked in 44 min via an interpreted programming language.</jats:p> </jats:sec>jats:sec jats:titleConclusions</jats:title> jats:pFully de-identified matching with high accuracy is feasible without a person-unique identifier and appropriate software is freely available.</jats:p> </jats:sec>
Description
Acknowledgements: We thank Johnny Downs, Gos Micklem, Senka Njegovan-Rajnic, Bristena Oprisanu, Matt Seigel, Naaman Tammuz, and Blaise Thomson for helpful discussion, and Phil Alsop, Natalie Barden, Chris Carling, Rachel Kyd, Linda Jones, Iliana Rokkou, Mary-Beth Sherwood, and Ben Underwood for support. We thank two anonymous referees for their helpful comments.