Repository logo
 

Prevalence of Multiplicity and Appropriate Adjustments Among Cardiovascular Randomized Clinical Trials Published in Major Medical Journals.

Published version
Peer-reviewed

Type

Article

Change log

Authors

Khan, Muhammad Shahzeb 
Khan, Maaz Shah 
Ansari, Zunaira Navid 
Siddiqi, Tariq Jamal 
Khan, Safi U 

Abstract

IMPORTANCE: Multiple analyses in a clinical trial can increase the probability of inaccurately concluding that there is a statistically significant treatment effect. However, to date, it is unknown how many randomized clinical trials (RCTs) perform adjustments for multiple comparisons, the lack of which could lead to erroneous findings. OBJECTIVES: To assess the prevalence of multiplicity and whether appropriate multiplicity adjustments were performed among cardiovascular RCTs published in 6 medical journals with a high impact factor. DESIGN, SETTING, AND PARTICIPANTS: In this cross-sectional study, cardiovascular RCTs were selected from all over the world, characterized as North America, Western Europe, multiregional, and rest of the world. Data were collected from past issues of 3 cardiovascular journals (Circulation, European Heart Journal, and Journal of the American College of Cardiology) and 3 general medicine journals (JAMA, The Lancet, and The New England Journal of Medicine) with high impact factors published between August 1, 2015, and July 31, 2018. Supplements and trial protocols of each of the included RCTs were also searched for multiplicity. Data were analyzed December 20 to 27, 2018. EXPOSURES: Data from the selected RCTs were extracted and verified independently by 2 researchers using a structured data instrument. In case of disagreement, a third reviewer helped to achieve consensus. An RCT was considered to have multiple treatment groups if it had more than 2 arms; multiple outcomes were defined as having more than 1 primary outcome, and multiple analyses were defined as analysis of the same outcome variable in multiple ways. Multiplicity was examined only for the analysis of the primary end point. MAIN OUTCOMES AND MEASURES: Outcomes of interest were percentages of primary analyses that performed multiplicity adjustment of primary end points. RESULTS: Of 511 cardiovascular RCTs included in this analysis, 300 (58.7%) had some form of multiplicity; of these 300, only 85 (28.3%) adjusted for multiplicity. Intervention type and funding source had no statistically significant association with the reporting of multiplicity risk adjustment. Trials that assessed mortality vs nonmortality outcomes were more likely to contain a multiplicity risk in their primary analysis (66.3% [177 of 267] vs 50.4% [123 of 244]; P < .001), and larger trials vs smaller trials were less likely to make any adjustments for multiplicity (35.6% [52 of 146] vs 21.4% [33 of 154]; P = .001). CONCLUSIONS AND RELEVANCE: Findings from this study suggest that cardiovascular RCTs published in medical journals with high impact factors demonstrate infrequent adjustments to correct for multiple comparisons in the primary end point. These parameters may be improved by more standardized reporting.

Description

Keywords

Biomedical Research, Cardiovascular Diseases, Cross-Sectional Studies, Humans, Journal Impact Factor, Periodicals as Topic, Prevalence, Publishing, Randomized Controlled Trials as Topic

Journal Title

JAMA Netw Open

Conference Name

Journal ISSN

2574-3805
2574-3805

Volume Title

3

Publisher

American Medical Association (AMA)
Sponsorship
MRC (unknown)
Medical Research Council (MR/N028171/1)