Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes.
Authors
Garrett, A
Choi, S
King, L
Loveday, C
Torr, B
Burghel, GJ
Durkie, M
Callaway, A
Robinson, R
Drummond, J
Berry, I
Wallace, A
Eccles, D
Tischkowitz, M
Whiffin, N
Ware, JS
Hanson, H
Turnbull, C
CanVIG-Uk
Publication Date
2021-11Journal Title
Genet Med
ISSN
1098-3600
Publisher
Elsevier BV
Volume
23
Issue
11
Pages
2096-2104
Language
en
Type
Article
This Version
VoR
Metadata
Show full item recordCitation
Cubuk, C., Garrett, A., Choi, S., King, L., Loveday, C., Torr, B., Burghel, G., et al. (2021). Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes.. Genet Med, 23 (11), 2096-2104. https://doi.org/10.1038/s41436-021-01265-z
Abstract
PURPOSE: Where multiple in silico tools are concordant, the American College of Medical Genetics and Genomics/Association for Molecular Pathology (ACMG/AMP) framework affords supporting evidence toward pathogenicity or benignity, equivalent to a likelihood ratio of ~2. However, limited availability of "clinical truth sets" and prior use in tool training limits their utility for evaluation of tool performance. METHODS: We created a truth set of 9,436 missense variants classified as deleterious or tolerated in clinically validated high-throughput functional assays for BRCA1, BRCA2, MSH2, PTEN, and TP53 to evaluate predictive performance for 44 recommended/commonly used in silico tools. RESULTS: Over two-thirds of the tool-threshold combinations examined had specificity of <50%, thus substantially overcalling deleteriousness. REVEL scores of 0.8-1.0 had a Positive Likelihood Ratio (PLR) of 6.74 (5.24-8.82) compared to scores <0.7 and scores of 0-0.4 had a Negative Likelihood Ratio (NLR) of 34.3 (31.5-37.3) compared to scores of >0.7. For Meta-SNP, the equivalent PLR = 42.9 (14.4-406) and NLR = 19.4 (15.6-24.9). CONCLUSION: Against these clinically validated "functional truth sets," there was wide variation in the predictive performance of commonly used in silico tools. Overall, REVEL and Meta-SNP had best balanced accuracy and might potentially be used at stronger evidence weighting than current ACMG/AMP prescription, in particular for predictions of benignity.
Keywords
Article, article
Sponsorship
Cancer Research UK (C61296/A27223)
Identifiers
s41436-021-01265-z, 1265
External DOI: https://doi.org/10.1038/s41436-021-01265-z
This record's URL: https://www.repository.cam.ac.uk/handle/1810/330003
Rights
Licence:
http://creativecommons.org/licenses/by/4.0/
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.