Mechanisms and the Evidence Hierarchy

UK Arts and Humanities Research Council project 2012

Contributors: Brendan Clarke, Donald Gillies, Phyllis Illari, Federica Russo, Jon Williamson (PI)

Evidence-based medicine is a relatively recent technique for supporting clinical decisions by the conscientious, explicit, and judicious use of current best evidence (Sackett et al. 1996. BMJ. 312: 71). This best evidence usually has a very specific meaning: the best evidence available to support decision-making in medicine is that arising from clinical trials, where treatments are tested on large numbers of patients. On the other hand, evidence of mechanisms usually characterized as knowledge gained from experimental investigations in the laboratory is held to be of low quality by the EBM practitioner.

However, recent work in the philosophy of causality has suggested that this hierarchical interpretation of evidence is problematic, while even within medicine there is interest in evidence that can complement evidence gained in clinical trials.

Decisions about treatment make a difference to the health of individuals. Therefore it is of utmost importance to develop a concept of evidence that maximizes the available sources of evidence (trials, results of lab experiments) and minimizes the risks of errors in various medical decisions.

In this project, we aim to investigate the relationship between evidence-based medicine, evidence of mechanisms, and causality from a number of different theoretical and practical perspectives including philosophy of causality, philosophy and history of medicine, and medical practice.

Events:

  • Workshop: Mechanisms and the evidence hierarchy III, University College London, 10 January 2013.

    Speakers:

  • Public engagement event: The hierarchy of evidence, Archaeology Lecture Theatre, Archaeology building, UCL, 5.30-7.30pm Monday 10th September 2012.

    Chair: Phyllis Illari (Science and Technology Studies, UCL)

    Speakers:

    The most wide-ranging change in medical practice in the past two decades has been the introduction of evidence-based medicine (EBM). The EBM project, which calls for the explicit examination of evidence to guide healthcare decisions, has made a significant difference to the practice of healthcare. However, the complexity and sheer quantity of medical evidence means that tools for assessing this evidence are of crucial importance to this project. For example, the quality of evidence supporting a particularly health intervention might be assessed by examining the method by which that evidence is produced. Typically, this kind of ranking suggests that randomised control trials (or meta-analysis of randomised control trials) will produce better evidence than other trial methodologies. This means that evidence produced by other means, such as that arising from laboratory science, or from observational studies, is usually regarded as unsuitable when it comes to clinical decision making.

    However, recent philosophical work has cast some doubt on the wisdom of relying on just one form of evidence when considering complex medical interventions. For instance, it seems possible to improve the reliability of evidence gained from clinical trials by judicious use of evidence gleaned from laboratory investigation. This suggests that current schemes of ranking biomedical evidence may be capable of some improvement.
    This evening event presents some results of a preliminary project designed to investigate these questions, together with some lively discussion about the role of ranking evidence in supporting good scientific practice.

    Thanks to the generosity of the Arts and Humanities Research Council, the event will be followed by a reception in the Wilkins Lower Refectory [see here for directions].
    The event is free, and there is no need to book. Contact Brendan Clarke (b.clarke@ucl.ac.uk) for any queries.

  • Conference: ECitS 2012: Evidence and Causality in the Sciences, University of Kent, UK, 5-7 September 2012.
  • Workshop: Mechanisms and the evidence hierarchy II, University of Kent, UK, 1 June 2012.

    Speakers:

  • Workshop: Mechanisms and the evidence hierarchy I, University College London, 5 March 2012.

    Speakers:

Publications:

Brendan Clarke, Donald Gillies, Phyllis Illari, Federica Russo & Jon Williamson: Mechanisms and the Evidence Hierarchy, Topoi, 33(2):339-360, 2014. doi: 10.1007/s11245-013-9220-9

Evidence-based medicine (EBM) makes use of explicit procedures for grading evidence for causal claims. Normally, these procedures categorise evidence of correlation produced by statistical trials as better evidence for a causal claim than evidence of mechanisms produced by other methods. We argue, in contrast, that evidence of mechanisms needs to be viewed as complementary to, rather than inferior to, evidence of correlation. In this paper we first set out the case for treating evidence of mechanisms alongside evidence of correlation in explicit protocols for evaluating evidence. Next we provide case studies which exemplify the ways in which evidence of mechanisms complements evidence of correlation in practice. Finally, we put forward some general considerations as to how the two sorts of evidence can be more closely integrated by EBM.

Brendan Clarke, Donald Gillies, Phyllis Illari, Federica Russo & Jon Williamson: The evidence that evidence-based medicine omits, Preventative Medicine 57:745-747, 2013. DOI 10.1016/ j.ypmed.2012.10.020.

According to current hierarchies of evidence for EBM, evidence of correlation (e.g., from RCTs) is always more important than evidence of mechanisms when evaluating and establishing causal claims. We argue that evidence of mechanisms needs to be treated alongside evidence of correlation. This is for three reasons. First, correlation is always a fallible indicator of causation, subject in particular to the problem of confounding; evidence of mechanisms can in some cases be more important than evidence of correlation when assessing a causal claim. Second, evidence of mechanisms is often required in order to obtain evidence of correlation (for example, in order to set up and evaluate RCTs). Third, evidence of mechanisms is often required in order to generalise and apply causal claims.

While the EBM movement has been enormously successful in making explicit and critically examining one aspect of our evidential practice, i.e., evidence of correlation, we wish to extend this line of work to make explicit and critically examine a second aspect of our evidential practices: evidence of mechanisms.

Brendan Clarke, Bert Leuridan & Jon Williamson: Modelling mechanisms with causal cycles, Synthese 191(8): 1651-1681, 2014. . doi: 10.1007/s11229-013-0360-7

Mechanistic philosophy of science views a large part of scientific activity as engaged in modelling mechanisms. While science textbooks tend to offer qualitative models of mechanisms, there is increasing demand for models from which one can draw quantitative predictions and explanations. Casini et al. (2011) put forward the Recursive Bayesian Net (RBN) formalism as well suited to this end. The RBN formalism is an extension of the standard Bayesian net formalism, an extension that allows for modelling the hierarchical nature of mechanisms. Like the standard Bayesian net formalism, it models causal relationships using directed acyclic graphs. Given this appeal to acyclicity, causal cycles pose a prima facie problem for the RBN approach. This paper argues that the problem is a significant one given the ubiquity of causal cycles in mechanisms, but that the problem can be solved by combining two sorts of solution strategy in a judicious way.

Donald Gillies & Aidan Sudbury: Should causal models always be Markovian? The case of multi-causal forks, European Journal for the Philosophy of Science 3(3):275-308, 2013. doi: 10.1007/s13194-013-0068-z

The mechanistic and causal accounts of explanation are often conflated to yield a `causal-mechanical’ account. This paper prizes them apart and asks: if the mechanistic account is correct, how can causal explanations be explanatory? The answer to this question varies according to how causality itself is understood. It is argued that difference-making, mechanistic, dualist and inferentialist accounts of causality all struggle to yield explanatory causal explanations, but that an epistemic account of causality is more promising in this regard..

Jon Williamson: How can causal explanations explain?, Erkenntnis78(2):257-275, 2013. doi: 10.1007/s10670-013-9512-x

The mechanistic and causal accounts of explanation are often conflated to yield a `causal-mechanical’ account. This paper prizes them apart and asks: if the mechanistic account is correct, how can causal explanations be explanatory? The answer to this question varies according to how causality itself is understood. It is argued that difference-making, mechanistic, dualist and inferentialist accounts of causality all struggle to yield explanatory causal explanations, but that an epistemic account of causality is more promising in this regard..

Federica Russo & Jon Williamson: EnviroGenomarkers, The interplay between mechanisms and difference making in establishing causal claims, Medicine Studies: International Journal for the History, Philosophy and Ethics of Medicine & Allied Sciences, 3:249–262, 2012.

According to Russo and Williamson (2007, 2011a,b), in order to establish a causal claim of the form `C is a cause of E’, one needs evidence that there is an underlying mechanism between C and E as well as evidence that C makes a difference to E. This thesis has been used to argue that hierarchies of evidence, as championed by evidence-based movements, tend to give primacy to evidence of difference making over evidence of mechanism, and are flawed because the two sorts of evidence are required and they should be treated on a par.
An alternative approach gives primacy to evidence of mechanism over evidence of difference making. In this paper we argue that this alternative approach is equally flawed, again because both sorts of evidence need to be treated on a par. As an illustration of this parity we explain how scientists working in the `EnviroGenomarkers’ project constantly make use of the two evidential components in a dynamic and intertwined way. We argue that such an interplay is needed not only for causal assessment but also for policy purposes.

Acknowledgements:

We are very grateful to the UK Arts and Humanities Research Council for funding this research.

AHRC reasoning-logo-pics