Show simple item record

dc.contributor.authorChristie, Alec
dc.contributor.authorAmano, Tatsuya
dc.contributor.authorMartin, Philip
dc.contributor.authorShackelford, Gorm
dc.contributor.authorSimmons, Benno
dc.contributor.authorSutherland, William
dc.date.accessioned2022-03-08T00:30:24Z
dc.date.available2022-03-08T00:30:24Z
dc.date.issued2022
dc.identifier.issn0021-8901
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/334740
dc.description.abstract1. In Christie et al. (2019), we used simulations to quantitatively compare the bias of commonly used study designs in ecology and conservation. Based on these simulations, we proposed ‘accuracy weights’ as a potential way to account for study design validity in meta-analytic weighting methods. Pescott & Stewart (2021) raised concerns that these weights may not be generalisable and still lead to biased meta-estimates. Here we respond to their concerns and demonstrate why developing alternative weighting methods is key to the future of evidence synthesis. 2. We acknowledge that our simple simulation unfairly penalised Randomised Controlled Trial (RCT) relative to Before-After Control-Impact (BACI) designs as we assumed that the parallel trends assumption held for BACI designs. We point to an empirical follow-up study in which we more fairly quantify differences in biases between different study designs. However, we stand by our main findings that Before-After (BA), Control-Impact (CI), and After designs are quantifiably more biased than BACI and RCT designs. We also emphasise that our 'accuracy weighting’ method was preliminary and welcome future research to incorporate more dimensions of study quality. 3. We further show that over a decade of advances in quality effect modelling, which Pescott & Stewart (2021) omit, highlights the importance of research such as ours in better understanding how to quantitatively integrate data on study quality directly into meta-analyses. We further argue that the traditional methods advocated for by Pescott & Stewart (2021) (e.g., manual risk-of-bias assessments and inverse-variance weighting) are subjective, wasteful, and potentially biased themselves. They also lack scalability for use in large syntheses that keep up-to-date with the rapidly growing scientific literature. 4. Synthesis and applications. We suggest, contrary to Pescott & Stewart’s narrative, that moving towards alternative weighting methods is key to future-proofing evidence synthesis through greater automation, flexibility, and updating to respond to decision-makers needs – particularly in crisis disciplines in conservation science where problematic biases and variability exist in study designs, contexts, and metrics used. Whilst we must be cautious to avoid misinforming decision-makers, this should not stop us investigating alternative weighting methods that integrate study quality data directly into meta-analyses. To reliably and pragmatically inform decision-makers with science, we need efficient, scalable, readily automated, and feasible methods to appraise and weight studies to produce large-scale living syntheses of the future.
dc.description.sponsorshipAustralian Research Council Future Fellowship Royal Commission for the Exhibition of 1851 Research Fellowship Grantham Foundation for the Protection of the Environment Natural Environment Research Council The David and Claudia Harding Foundation Kenneth Miller Trust Arcadia Fund
dc.publisherWiley
dc.rightsAll Rights Reserved
dc.rights.urihttp://www.rioxx.net/licenses/all-rights-reserved
dc.titleInnovation and forward-thinking are needed to improve traditional synthesis methods: a response to Pescott & Stewart
dc.typeArticle
dc.publisher.departmentDepartment of Zoology
dc.date.updated2022-03-04T17:23:22Z
prism.publicationNameJournal of Applied Ecology
dc.identifier.doi10.17863/CAM.82170
dcterms.dateAccepted2022-02-21
rioxxterms.versionofrecord10.1111/1365-2664.14154
rioxxterms.versionAM
dc.contributor.orcidChristie, Alec [0000-0002-8465-8410]
dc.contributor.orcidSutherland, William [0000-0002-6498-0437]
dc.identifier.eissn1365-2664
rioxxterms.typeJournal Article/Review
pubs.funder-project-idNERC (1945942)
pubs.funder-project-idNatural Environment Research Council (1945942)
pubs.funder-project-idNERC (NE/L002507/1)
cam.issuedOnline2022-03-22
cam.orpheus.success2022-04-29: Embargo updated
cam.orpheus.counter4
cam.depositDate2022-03-04
pubs.licence-identifierapollo-deposit-licence-2-1
pubs.licence-display-nameApollo Repository Deposit Licence Agreement
datacite.issourceof.doi10.1111/1365-2664.14154
rioxxterms.freetoread.startdate2023-03-22


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record