Evaluating evidence in medicine

Evidence-based medicine has transformed the way in which statistical evidence is used in medicine. Hierarchies of evidence are now routinely used by medical researchers and health policy makers to assess evidence for the effectiveness of treatments and health policies: studies that simply observe patients after treatment are ranked lower than studies that randomly decide who to treat, and these in turn rank lower than studies that review the evidence obtained by a series of trials. Evidence hierarchies have become so widely endorsed that they are now being used across the social sciences and in public policy, as well as in medicine.

While there has been some debate about which sorts of trials should be placed at the top of the hierarchy, this project focuses on the bottom level, which is normally occupied by evidence that is not obtained from a statistical trial. In our view, while it is appropriate to relegate anecdotal evidence and hearsay to this lowest level, other, better quality evidence is also being ignored, simply because it is often not obtained from statistical trials.

In particular, evidence of the underlying physiological and biochemical mechanisms is often classified as inferior to statistical evidence. This is because evidence of mechanisms is normally obtained, not simply via statistical trials, but in a complex way, by integrating a mixture of laboratory experiments, basic scientific knowledge and case studies as well as past trials. Recent work suggests that it is wrong to view evidence of mechanisms as inferior. Philosophers of causality and historians of medicine have argued that evidence of mechanisms is required alongside statistical evidence in order to evaluate whether treatments or health policies are effective. This is because such evidence helps to determine whether positive results of a trial are due to genuine effectiveness or are simply a statistical blip; such evidence is also crucial when designing and interpreting a statistical trial, and when determining effectiveness in a new population or a particular patient

But how can one formulate explicit guidelines for considering mechanistic evidence alongside statistical evidence? One reason why non-statistical evidence is relegated to the bottom of the hierarchies is that it is very hard to weigh against evidence obtained from statistical trials. In this project we seek to understand how to evaluate mechanistic evidence alongside statistical evidence in medical research and health policy.

This AHRC-funded project ran from 1st June 2015 – 31 May 2018.

Doctoral training initiative: evidence and its quality.

Research Questions

Overarching research question [Q]: How can evidence of mechanisms be considered alongside evidence of correlation to evaluate causal claims in medical research and health policy?
The answer to this overarching question will build on answers to following questions:
- [EM: Evidence of Mechanisms]: What is evidence of a mechanism, and how do we get it?
- [QE: Quality of Evidence]: How can quality of evidence be characterised?
- [PC: Philosophy of Causality]: Which accounts of causality best fit the programme for integrating evidence of mechanisms with evidence of correlation?

People

The Centre for Reasoning at the University of Kent: Christian Wallmann, Michael Wilde, Jon Williamson
The department of Science and Technology Studies at University College London (UCL): Brendan Clarke, Athena Drakou, Donald Gillies, Phyllis Illari, Charles Norell
The department of Philosophy at the University of Amsterdam (UvA): Federica Russo
The National Institute for Health and Clinical Excellence (NICE): Beth Shaw
The International Agency for Research on Cancer (IARC): Kurt Straif
The Institute of Public Health at Cambridge University: Mike Kelly
The Medical School at Leiden University: Jan Vandenbroucke

Publications

Volumes

Veli-Pekka Parkkinen, Christian Wallmann, Michael Wilde, Brendan Clarke, Phyllis Illari, Michael P. Kelly, Charles Norell, Federica Russo, Beth Shaw and Jon Williamson: Evaluating evidence of mechanisms in medicine: Principles and procedures, Springer, 2018.

Donald Gillies: Causality, probability and medicine. Routledge, 2018.

Stuart Glennan and Phyllis Illari (eds): Routledge Handbook of Mechanisms and Mechanical Philosophy, Routledge 2018.

Jimenez-Buedo M. and Russo F. (eds) Causality and modelling. Special issue of Disputatio 9 (47), 2017.

Articles

Jon Williamson: Evidential Proximity, Independence, and the evaluation of carcinogenicity, Journal of Evaluation in Clinical Practice 25(6):955-961, 2019. doi: 10.1111/jep.13226

Jon Williamson: Establishing causal claims in medicine, International Studies in the Philosophy of Science 32(1): 33-61, 2019. doi: 10.1080/02698595.2019.1630927

Mark R. Tonelli & Jon Williamson: Mechanisms in clinical practice: use and justification, Medicine, Health Care and Philosophy, 2019. doi: 10.1007/s11019-019-09915-5

Veli-Pekka Parkkinen & Jon Williamson: Extrapolating from model organisms in pharmacology, in La Caze, A., & Osimani, B., (eds), Uncertainty in pharmacology: epistemology, methods, and decisions, Springer, 2020, pp. 59-78. ISBN: 978-3-030-29178-5

Jon Williamson: Establishing the teratogenicity of Zika and evaluating causal criteria, Synthese. doi: 10.1007/s11229-018-1866-9

Jeffrey K. Aronson, Adam La Caze, Michael P. Kelly, Veli-Pekka Parkkinen and Jon Williamson: The use of mechanistic evidence in drug approval, Journal of Evaluation in Clinical Practice 24(5):1166-1176, 2018. doi: 10.1111/jep.12960

Michael P. Kelly, 2018. The need for a rationalist turn in evidence-based medicine, Journal of Evaluation in Clinical Practice, DOI 10.1111/jep.12974

Jeffrey K. Aronson, Adam La Caze, Michael P. Kelly, Veli-Pekka Parkkinen and Jon Williamson: The use of mechanistic evidence in drug approval, Journal of Evaluation in Clinical Practice, . doi: 10.1111/jep.12960

KELLY, M.P. & KELLY, R.S. (2018) Quantifying social influences throughout the life course: action, structure and ‘omics’, in, (eds), Meloni M., Cromby, J., Fitzgerald, P., Lloyd, S. (eds) The Palgrave Handbook of Biology and Society, London: Palgrave Macmillan. pp 587-609. https://link.springer.com/book/10.1057%2F978-1-137-52879-7

KRIZNIK, N.M., KINMONTH, A.L., LING, T., KELLY, M.P. (2018) Moving beyond individual choice in policies to reduce health inequalities: the integration of dynamic with individual explanations, Journal of Public Health. https://academic.oup.com/jpubhealth/advance-article/doi/10.1093/pubmed/fdy045/4931230?guestAccessKey=af9f5249-b3b7-4270-92db-421e9c8fb5ac

KELLY, M.P., (2018) How to make the first thousand days count, Health Promotion Journal of Australia Health Promot J Austral. 2018; 00:1–5. https://doi.org/10.1002/hpja.58

Jan-Willem Romeijn & Jon Williamson: Intervention and Identifiability in Latent Variable Modelling, Minds and Machines, 2018. doi: 10.1007/s11023-018-9460-y

Vineis P. and Russo F. (2018) Epigenetics and the Exposome: environmental exposure in disease etiology. In Oxford Research Encyclopaedia of Environmental Science. Oxford University Press. DOI: 10.1093/acrefore/9780199389414.013.325

Johnson R.B., Russo F., Schoonenboom J. (2017), Causality in mixed methods research: the meeting of philosophy, science, and practice. Journal of Mixed Methods Research. DOI: 10.1177/1558689817719610

Russo F. and Vineis P. (2017) Opportunities and challenges of molecular epidemiology. In G. Boniolo and M. J. Nathan (eds) Philosophy of Molecular Medicine: Foundational Issues in Research and Practice. Routledge, ch.12.

Iliadis A. and Russo F. (2016) Critical data studies: An introduction. Big Data & Society, July-December 2016, 1-7.

Donald Gillies (2017) Mechanisms in Medicine, Axiomathes, 27, pp. 621-634.

Donald Gillies (2018) Discovering Cures in Medicine. In David Danks and Emiliano Ippoliti (Eds.) Building Theories. Heuristics and Hypotheses in Science, Springer, pp. 83-100.

BLUE, S., SHOVE, E., CARMONA, C. KELLY, M.P. (2016) Theories of practice and public health: understanding (un) healthy practices, Critical Public Health; 26: 36-50. DOI: 10.1080/09581596.2014.980396 http://dx.doi.org/10.1080/09581596.2014.980396.

THRELFALL, A., MEAH, S., FISCHER, A.J., COOKSON, R., RUTTER, H., KELLY, M.P. (2015) The appraisal of public health interventions: the use of theory, Journal of Public Health, 37: 166-171. http://jpubhealth.oxfordjournals.org/content/37/1/166.full.pdf+html

MARTEAU, T.M., HOLLANDS, G.J., KELLY, M.P. (2015) Changing population behavior and reducing health disparities: Exploring the potential of “choice architecture” interventions, in Kaplan, RM, Spittel, M, & David, DH. (eds). Population Health: Behavioral and Social Science Insights, AHRQ Publication No. 15-0002. Rockville, MD: Agency for Healthcare Research and Quality and Office of Behavioral and Social Sciences Research, National Institutes of Health. pp105- 126.

KELLY, M.P.., HEATH, I., HOWICK, J., GREENHALGH, T. (2015) The importance of values in evidence-based medicine, BMC Medical Ethics; BMC Medical Ethics.2015, 16:69. DOI: 10.1186/s12910-015-0063-3 URL: http://www.biomedcentral.com/1472-6939/16/69

SQUIRES, H., CHILCOTT, J., AKEHURST, R., BURR, J., KELLY, M.P. (2016) A systematic literature review of the key challenges for developing the structure of Public Health economic models, International Journal of Public Health. 61: 289-298. DOI 10.1007/s00038-015-0775-7 http://link.springer.com/article/10.1007%2Fs00038-015-0775-7

KELLY, M.P. (2016) Understanding the mechanisms underpinning health inequalities: lessons from economics. Global Health Promotion; 23: 3-5

MICHIE, S CAREY, R., JOHNSTON, M, ROTHMAN, A., de BRUIN, M., KELLY, M.P., CONNELL, L. (2016) From theory-inspired to theory-based interventions: A protocol for developing and testing a methodology for linking behaviour change techniques to theoretical mechanisms of action, Annals of Behavioral Medicine; doi:10.1007/s12160-016-9816-6 http://rdcu.be/nitY

HOLLANDS, G., BIGNARDI, G., JOHNSTON, M., KELLY, M.P., OGILVIE, D., PETTICREW, M., PRESTWICH, A., SHEMILT, I., SUTTON, S., MARTEAU, T. The TIPPME intervention typology for changing environments to change behaviour, Nature Human Behaviour. https://www.nature.com/articles/s41562-017-0140

KELLY, M.P. & RUSSO, F. Causal narratives in public health: the difference between mechanisms of aetiology and mechanisms of prevention in non-communicable diseases. Sociology of Health and Illness. http://onlinelibrary.wiley.com/doi/10.1111/1467-9566.12621/pdf

MICHIE, S., THOMAS, J., JOHNSTON, M., MAC AONGHUSA, P., SHAWE-TAYLOR, J., KELLY, M.P., DELERIS, L.A., FINNERTY, A.N., MARQUES, M.M., NORRIS, E., O’MARA-EVES, A., WEST, R. The Human Behaviour-Change Project: Harnessing the power of Artificial Intelligence and Machine Learning for evidence synthesis and interpretation, Implementation Science. 12:121 DOI 10.1186/s13012-017-0641-5 http://rdcu.be/wRUc

Wallmann C.: A Bayesian solution to the conflict of narrowness and precision in direct inference. Journal for General Philosophy of Science, 48(3): 485-500, (2017).

Christian Wallmann & Jon Williamson: Four approaches to the reference class problem, in Gábor Hofer-Szabó & Leszek Wroński (eds), Making it Formally Explicit: Probability, Causality and Indeterminism, Springer, 2017, pp. 61-81. ISBN 978-3-319-55486-0

Corraini P, Olsen M, Pedersen L, Dekkers OM, Vandenbroucke JP. Effect modification, interaction and mediation: an overview of theoretical insights for clinical investigators. Clin Epidemiol. 2017 Jun 8;9:331-338. doi: 10.2147/CLEP.S129728. eCollection 2017. PubMed PMID: 28652815; PubMed Central
PMCID: PMC5476432.

Nørgaard M, Ehrenstein V, Vandenbroucke JP. Confounding in observational studies based on large health care databases: problems and potential solutions – a primer for the clinician. Clin Epidemiol. 2017 Mar 28;9:185-193. doi:
10.2147/CLEP.S129879. eCollection 2017. PubMed PMID: 28405173; PubMed Central PMCID: PMC5378455.

Pearce N, Vandenbroucke JP. Causation, mediation and explanation. Int J Epidemiol. 2016 Dec 1;45(6):1915-1922. doi: 10.1093/ije/dyw281. PubMed PMID: 27864404.

Vandenbroucke JP, Broadbent A, Pearce N. Causality and causal inference in epidemiology: the need for a pluralistic approach. Int J Epidemiol. 2016 Dec 1;45(6):1776-1786. doi: 10.1093/ije/dyv341. PubMed PMID: 26800751.

Veli-Pekka Parkkinen & Michael Wilde: Extrapolation and the Russo–Williamson thesis, Synthese, 2017.

Donald Gillies: Evidence of mechanism in the evaluation of streptomycin and thalidomide, Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences 66: 52-62, 2017.

Veli-Pekka Parkkinen, Federica Russo & Christian Wallmann: Scientific Disagreement and Evidential Pluralism: Lessons from the Studies on Hypercholesterolemia, Humana.Mente Journal of Philosophical Studies 32 75–116, 2017.

Paolo Vineis, Phyllis Illari & Federica Russo: Causality in cancer research: a journey through models in molecular epidemiology and their philosophical interpretation, Emerging Themes in Epidemiology 14:7, 2017, DOI: 10.1186/s12982-017-0061-7

Donald Gillies 2016. Establishing Causality in Medicine and Koch’s Postulates, International Journal of History and Philosophy of Medicine, 6: 10603, pp. 1-13.

Clarke, B. and Russo, F. 2017. Causation in Medicine, in Marcum, J. (ed.) 2017 The Bloomsbury Companion to Contemporary Philosophy of Medicine. London: Bloomsbury, Chapter 12, pp. 297-322

Clarke, B. and Russo, F. 2017. Mechanisms and Bioedicine in Glennan, S. & Illari, P. (eds) 2017 Routledge Companion to Mechanisms. London: Routledge, chapter 24.

Sozudoğru, E. and Clarke, B. forthcoming. Uncertainty in drug discovery: strategies, heuristics and technologies. In Osimani, B. (ed.) Uncertainty in Pharmacology. Pickering & Chatto.

Michel Mouchart, Guillaume Wunsch & Federica Russo: Controlling Variables in Social Systems – A Structural Modelling Approach, Bulletin de Methodologie Sociologique 132:5–25, 2016. doi: 10.1177/0759106316662811

Russo F.: Statistical generalizations in epidemiology: philosophical analysis. In Handbook of the Philosophy of Medicine. Edited by T. Schramme and S. Edwards. 2016. doi:10.1007/978-94-017-8706-2_39-2

Michael Wilde & Jon Williamson: Models in medicine, in H. Kincaid, J. Simon & M. Solomon (eds), The Routledge Companion to Philosophy of Medicine. Routledge, pp. 271-284, 2016. ISBN: 978-1-13-884679-1

Phyllis Illari: Mechanisms in medicine, to be published in the Routledge Handbook of Philosophy of Medicine, edited by Miriam Solomon, Jeremy Simon and Harold Kincaid.

Illari P. and Russo F. Causality and information. In The Routledge Handbook of Philosophy of Information. Edited by L. Floridi. Forthcoming.

Federica Russo: Causation and Correlation in Medical Science: Theoretical Problems, in Thomas Schramme, Steven Edwards (eds), Handbook of the Philosophy of Medicine. Springer, in press.

Michael Wilde & Jon Williamson: Evidence and Epistemic Causality, in A. von Eye & W. Wiedermann (eds), Statistics and Causality: methods for applied empirical research, pp. 33-44. Wiley, 2016. ISBN: 978-1-118-94704-3

Michael Kelly, Rachel Kelly & Federica Russo: The integration of social, behavioural and biological mechanisms in models of pathogenesis, Perspectives in Biology and Medicine 57: 308-28, 2015.

Clarke, B. (2016), Discovery in Medicine, in Miriam Solomon; Jeremy Simon & Harold Kincaid, ed., ‘Routledge Companion to the Philosophy of Medicine’, Routledge, London.

Donald Gillies: The Interpretation of Probability in Causal Models for Medicine, in Miriam Solomon, Jeremy R. Simon, and Harold Kincaid (Eds.) Routledge Companion to Philosophy of Medicine, 2016, pp. 71-80.

Donald Gillies: The Propensity Interpretation, in Alan Hájek and Christopher Hitchcock (Eds.) Oxford Handbook of Probability and Philosophy, 2016.