Dr Malcolm Macleod: The reproducibility opportunity скачать в хорошем качестве

Dr Malcolm Macleod: The reproducibility opportunity

A talk on the reproducibility opportunity by Dr Malcolm Macleod as part of the UK Reproducibility Network launch on the 12th September 2018. It is important for research users to know how likely it is that reported research findings are true. Early definitions of “reproducibility” related to the re-analysis of existing data following the same analytical procedures, while “replication” was held to require the collection of new data, following the same methods. However, the interchangeable use of these terms means that “reproducible research” has a broader meaning.[1] Articulating this broader definition, consider a core “reproducibility hierarchy”: (1) reanalysis of an existing dataset (“reproducibility of analysis”); (2) new data collection in experiments as identical as possible to the first (“reproducibility of experimental findings”); and (3) deliberate variation of experimental conditions or analytical approaches (“robustness”). The main focus of recent concern relates to “reproducibility of experimental findings”. In retrospective observational[2, 3] and prospective[4-6] studies it has not been possible to confirm many findings previously considered to be “true”. Failed reproducibility of experimental findings in biomedicine occurs if the originator study was falsely positive (by chance, or because of bias), or because of some unknown (latent) independent variable. In this case, what was intended as a test of “replication of experimental findings” was a test of “robustness”. This latter explanation promises the discovery of a previously unknown facet of the process being studied.[7] Sensible approaches include efforts to (1) increase the probability that published research is true (through the development of organised research improvement strategies); (2) establish a framework to select efficiently which research findings we should attempt to replicate (by establishing whether there are study characteristics which predict whether or not a key research finding can be replicated); and (3) develop strategies to evaluate the robustness of key research findings (based on pre-registered, probably multicentre studies, with deliberate heterogeneity). This would provide an opportunity substantially to increase the value of existing and of future research. 1. Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl Med 2016;8(341):341ps12. doi: 10.1126/scitranslmed.aaf5027 2. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature 2012;483(7391):531-33. 3. Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov 2011;10(9):712-7c1. 4. Camerer CF, Dreber A, Forsell E, et al. Evaluating replicability of laboratory experiments in economics. Science 2016;351(6280):1433-6. doi: 10.1126/science.aaf0918 5. Errington TM, Iorns E, Gunn W, et al. An open investigation of the reproducibility of cancer biology research. Elife 2014;3 doi: 10.7554/eLife.04333 6. Open Science Collaboration. Estimating the reproducibility of psychological science. Science 2015;349(6251):aac4716. 7. Lithgow GJ, Driscoll M, Phillips P. A long journey to reproducible results. Nature 2017;548(7668):387-88. doi: 10.1038/548387a