Evaluating evidence in medicine

Evidence-based medicine has transformed the way in which statistical evidence is used in medicine. Hierarchies of evidence are now routinely used by medical researchers and health policy makers to assess evidence for the effectiveness of treatments and health policies: studies that simply observe patients after treatment are ranked lower than studies that randomly decide who to treat, and these in turn rank lower than studies that review the evidence obtained by a series of trials. Evidence hierarchies have become so widely endorsed that they are now being used across the social sciences and in public policy, as well as in medicine.

While there has been some debate about which sorts of trials should be placed at the top of the hierarchy, this project focuses on the bottom level, which is normally occupied by evidence that is not obtained from a statistical trial. In our view, while it is appropriate to relegate anecdotal evidence and hearsay to this lowest level, other, better quality evidence is also being ignored, simply because it is often not obtained from statistical trials.

In particular, evidence of the underlying physiological and biochemical mechanisms is often classified as inferior to statistical evidence. This is because evidence of mechanisms is normally obtained, not simply via statistical trials, but in a complex way, by integrating a mixture of laboratory experiments, basic scientific knowledge and case studies as well as past trials. Recent work suggests that it is wrong to view evidence of mechanisms as inferior. Philosophers of causality and historians of medicine have argued that evidence of mechanisms is required alongside statistical evidence in order to evaluate whether treatments or health policies are effective. This is because such evidence helps to determine whether positive results of a trial are due to genuine effectiveness or are simply a statistical blip; such evidence is also crucial when designing and interpreting a statistical trial, and when determining effectiveness in a new population or a particular patient

But how can one formulate explicit guidelines for considering mechanistic evidence alongside statistical evidence? One reason why non-statistical evidence is relegated to the bottom of the hierarchies is that it is very hard to weigh against evidence obtained from statistical trials. In this project we seek to understand how to evaluate mechanistic evidence alongside statistical evidence in medical research and health policy.

This AHRC-funded project runs from 1st June 2015 – 31 May 2018.

For latest news please see the blog of the EBM+ consortium.

Doctoral training initiative: evidence and its quality.

Research Questions

  • Overarching research question [Q]: How can evidence of mechanisms be considered alongside evidence of correlation to evaluate causal claims in medical research and health policy?
    The answer to this overarching question will build on answers to following questions:

    • [EM: Evidence of Mechanisms]: What is evidence of a mechanism, and how do we get it?
    • [QE: Quality of Evidence]: How can quality of evidence be characterised?
    • [PC: Philosophy of Causality]: Which accounts of causality best fit the programme for integrating evidence of mechanisms with evidence of correlation?

People

  • The Centre for Reasoning at the University of Kent: Christian Wallmann, Michael Wilde, Jon Williamson
  • The department of Science and Technology Studies at University College London (UCL): Brendan Clarke, Athena Drakou, Donald Gillies, Phyllis Illari
  • The department of Philosophy at the University of Amsterdam (UvA): Federica Russo
  • The National Institute for Health and Clinical Excellence (NICE): Beth Shaw
  • The International Agency for Research on Cancer (IARC): Kurt Straif
  • The Institute of Public Health at Cambridge University: Mike Kelly
  • The Medical School at Leiden University: Jan Vandenbroucke

Publications

Stuart Glennan and Phyllis Illari (eds): Routledge Handbook of Mechanisms, Routledge 2016.

Russo F.: Statistical generalizations in epidemiology: philosophical analysis. In Handbook of the Philosophy of Medicine. Edited by T. Schramme and S. Edwards. 2016. doi:10.1007/978-94-017-8706-2_39-2

Michael Wilde & Jon Williamson: Models in medicine, in H. Kincaid, J. Simon & M. Solomon (eds), The Routledge Companion to Philosophy of Medicine. In press. 

Phyllis Illari: Mechanisms in medicine, to be published in the Routledge Handbook of Philosophy of Medicine, edited by Miriam Solomon, Jeremy Simon and Harold Kincaid.

Illari P. and Russo F. Causality and information. In The Routledge Handbook of Philosophy of Information. Edited by L. Floridi. Forthcoming.

Federica Russo: Causation and Correlation in Medical Science: Theoretical Problems, in Thomas Schramme, Steven Edwards (eds), Handbook of the Philosophy of Medicine. Springer, in press. 

Michael Wilde & Jon Williamson: Evidence and Epistemic Causality, in A. von Eye & W. Wiedermann (eds), Statistics and Causality: methods for applied empirical research, pp. 33-44. Wiley, 2016.   ISBN: 978-1-118-94704-3

Michael Kelly, Rachel Kelly & Federica Russo: The integration of social, behavioural and biological mechanisms in models of pathogenesis, Perspectives in Biology and Medicine 57: 308-28, 2015.  

Clarke, B. & Russo, F. (2016), Causation in Medicine, in James Marcum, ed., ‘Companion to Contemporary Philosophy of Medicine’, Bloomsbury, London.

Clarke, B. (2016), Discovery in Medicine, in Miriam Solomon; Jeremy Simon & Harold Kincaid, ed., ‘Routledge Companion to the Philosophy of Medicine’, Routledge, London.

Donald Gillies: The Interpretation of Probability in Causal Models for Medicine, in Miriam Solomon, Jeremy R. Simon, and Harold Kincaid (Eds.) Routledge Companion to Philosophy of Medicine, 2016.

Donald Gillies:  The Propensity Interpretation, in Alan Hájek and Christopher Hitchcock (Eds.) Oxford Handbook of Probability and Philosophy, 2016.

Activities

20 June 2016: Workshop. New frontiers for evaluating evidence in medicine. UCL.

16 May 2016: Workshop. Explanation and evidence of mechanisms across the sciences, University of Kent.

12 May 2016: Workshop. Processes, University of Kent.

21 January 2016:  EBM+ workshop. Amsterdam.

13 July 2015: Project kick-off workshop. Canterbury.

8-9 January 2015: EBM+ Workshop. Canterbury

 AHRCreasoning-logo-pics