Author Archives: Brendan Clarke

About Brendan Clarke

Lecturer in history and philosophy of medicine at UCL

Seminar announcement: Donald Gillies on Causality, Propensity, and Simpson’s Paradox, 30 Sept 2014

This is part of the (excellent) seminar series on Probabilities, Propensities, and Conditionals convened by Mauricio Suárez at the Institute of Philosophy. You can find more details at their website: http://philosophy.sas.ac.uk/about/ppc-seminar-donald-gillies-30-Sept

30 September 2014, 17:15 – 19:00

Objective Probability, and Conditional Reasoning Seminar: Room G34, Senate House, WC1
Causality, Propensity, and Simpson’s Paradox
Donald Gillies (UCL)

Contemporary medicine uses indeterministic causes, i.e. causes which do not always give rise to their effects.  For example, smoking causes lung cancer but only about 5% of smokers get lung cancer.  Indeterministic causes have to be linked to probabilities, but the nature of this link is problematic.  Seemingly correct principles connecting causes to probabilities turn out to be liable to counter-examples.  The present paper explores this problem by interpreting the probabilities involved as propensities.  This associates the problem of linking causality and probability closely with Simpson’s paradox, thereby suggesting a way in which the problem might be resolved.

What’s the difference between data and evidence

This is a question that came up while I was writing a talk about the difficulties that might be encountered when translating evidence policies from one context to another for my home department’s Annual Research Day a year or so ago. You can find a copy of the slides here.

The plan was to say something about the way that EBM has influenced non-medical decision-making. The original rationale for EBM was a) to de-emphasise individual judgement, based on clinical experience, as a sufficient foundation for making care decisions and b) to instead base care decisions on evidence, particularly that arising from clinical trials. To quote perhaps the most widely-cited paper on the subject, EBM is the:

“conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al 1996)

However, a cursory glance at the topics of article citing Sackett – all 9845 of them, at the time of writing – suggest that there is a growing interest in exporting this method of making decisions far outside the original context of medicine. These include papers on education policy, social work and – most interesting of all – dealing with architecture as a means of crime control. While an analysis of the reasons for this wide circulation are fascinating (and hopefully the subject of a later post), they’re a bit beyond what I want to talk about here. Instead, I want to simply claim that EBM’s tools and tactics have had a really wide circulation in the last 10 years or so, with the most visible new locus of practice in the evidence-based policy (EBP) movement.

Yet this change in application poses tough questions about translation. How should EBM – a method that depends on practices that are pretty specific to medicine – be modified to give useful answers to those making decisions in other contexts? A further puzzle concerns the role of philosophers of science in all this. While there are many questions here that might benefit from a philosophical treatment of of one kind or another, the contribution from philosophers have not been terribly helpful to this conversation.Given that I really believe that philosophers can and do meaningfully contribute to this kind of conversation, I will conclude by suggesting a few ways that we might provide a more useful (and more critical) contribution to the philosophy of evidence-based something. To illustrate this, I’d like to talk about one specific question thrown up by the circulation of practices from EBM to EBP. This starts with an ostensibly simple question: what’s the difference between data and evidence?

The data-evidence distinction

Why care about this distinction? Well, it appears to be one that gets made very frequently in EBP. We can find lots of examples of practitioners making distinctions between data and evidence. My quick web search this afternoon threw up examples including one by the UN’s Data Unity Network, or the South Downs National Park Authority or the Marine Management Organisation.

But it’s not very clear from these examples exactly how this distinction gets made. Is the distinction something that comes over to EBP from EBM? Well, I think the short answer here is ‘no’. I can’t find a detailed analysis of any such data/evidence distinction in the EBM literature. However, my intuition (and perhaps one that I might be able to defend if pushed) is something like this: EBM proponents typically claim that evidence alone should be used when making decisions about healthcare (look at the Sackett quote above). Yet this evidence often depends on data gathered during, for instance, clinical trials. Here then, data and evidence can be locally distinguished. Information about individual trial subjects is data. But once aggregated via appropriate statistical work, and reported as the result of a trial, it becomes evidence, which can then be used to address a clinical question.

This local distinction isn’t very helpful outside EBM. Perhaps because EBP decisions often involve looking at processes only measurable at a group level (in economics, for instance), the EBM distinction between individual data and group evidence is unlikely to be applicable. So the data/evidence distinction that is being made in the examples above can’t just be made in the same way as it is in EBM. Can we find some more general way of distinguishing data from evidence by looking at the literature on the philosophy of evidence?

Philosophers and the data-evidence distinction

Well, at the outset, looking to philosophers of science for help with this question appears promising. There is a great deal of philosophical work on evidence, and some of it contains distinctions between data and evidence. Perhaps it might be possible to translate some of this work to the EBP context? Let’s take a closer look at some of this philosophical work. I’ve picked a pair of ways of making the data-evidence distinction that have appeared in the philosophy of probability literature:

Mayo’s error-statistical philosophy of evidence

Mayo’s idea is that evidence describes a special sub-set of our data. More precisely, when a particular hypothesis is tested using a particular set of data (arising from a clinical trial, say), that data becomes evidence in relation to a particular hypothesis.

data x are evidence for a hypothesis H to the extent that H passes a severe test with x. (Mayo 2004: 79)

This seems a pretty plausible way of making the data/evidence distinction that might be suitable for either EBM or EBP.

Subjective Bayesian view of evidence

This view essentially distinguishes data from evidence by defining evidence in a way that (negatively) defines evidence. Here, the primitive concept is the acceptance of some evidential statement. Anything that leads to that statement is (basically) irrelevant, or at least not defined. For us, this might well include data.

The Bayesian theory of support is a theory of how the acceptance as true of some evidential statement affects your belief in some hypothesis. How you came to accept the truth of the evidence, and whether you are correct in accepting it as true, are matters that, from the point of view of the theory, are simply irrelevant. (Howson and Urbach 1993: 419)

Here, then, the idea is that evidence is constituted by those statements that affect belief in some hypothesis. Everything that leads to these statements – data, for example – is lumped together as an irrelevance. Like Mayo’s distinction, this also seems a pretty plausible way of making the data/evidence distinction that might be suitable for either EBM or EBP.

So what’s the problem?

Given that both ways of distinguishing data and evidence seem (at least) plausible, which should we prefer to use in practice? For the examples cited, this is where things start to get a bit tricky. As I’ve hinted above, each of these distinctions is rooted in a different theory of probability. Mayo’s distinction comes from the frequentist Neyman-Pearson tradition, while Howson and Urbach’s comes from subjective Bayesianism. Given that both methods appear to provide us with a means of making clear distinctions between data and evidence, the decision about how to make this distinction presumably follows from an earlier decision to adopt one or other general theory of probability.

But picking a general theory of probability is no small matter, either philosophically (see Gillies 2000 for background) or practically. At the very least, the choice of theory shapes the kinds of statistical methods that are appropriate, leading to all kinds of implications for experimental design and so on. And suggesting that we decide how to distinguish data from evidence by first deciding on a general theory of probability is not terribly helpful either (in any case, these kinds of discussions usually regress into ‘theory x is better than theory y‘ foot-stamping). So it is not clear to me just which way of making the distinction we should prefer. However, a more local conclusion is a bit more positive: any distinction that we draw between data and evidence should probably follow whichever general theory of probability is in use.

References

Gillies, D. 2000. Philosophical Theories of Probability. Routledge.

Howson and Urbach 1993. Scientific Reasoning: the Bayesian Approach. Open Court

Mayo 2004. “An Error-Statistical Philosophy of Evidence,” in Taper and Lele (eds.) The Nature of Scientific Evidence: Statistical, Philosophical and Empirical Considerations. University Of Chicago Press: 79-118.

Sackett, D., Rosenberg, W., Gray, J., Haynes, R., and Richardson, W. 1996. Evidence based medicine: what it is and what it isn’t. British Medical Journal, 312(7023): 71-2.

Why association is only half the story

I want to develop a point from Jon’s earlier post. A central theme of this project is that association (a correlation found in a drug trial, for example) is only half the story about causation. As Jon mentioned, there are many reasons that an observed correlation might be non-causal (like sampling errors, confounding, and so on). Here, I want to explore a case where a non-causal correlations was taken as sufficient reason for accepting a causal claim.

Cervical cancer

Cervical cancer is caused by infection with human papillomavirus (HPV). This claim was first made in the early 1980s by Harald zur Hausen, a German virologist. You can have a look at the original paper (Durst et al, 1983), as well as some information about the half share of the 2008 Nobel prize in Physiology or Medicine which he won for this work. When I started studying medicine in the late 1990s, the causal link between HPV and cervical cancer was common knowledge. So when I began researching the history of cervical cancer for my PhD (which you can read online if you’re keen on that kind of thing), it was a shock to discover that HPV was not the only virus that had been associated with cervical cancer.

Between about 1970 and 1985, herpes simplex virus (HSV) was generally accepted as the cause of cervical cancer. For example, you can peruse the forty or so papers that make up the proceedings of the 1972 American Cancer Society conference ‘Herpesvirus and cervical cancer’ in Cancer Research, which demonstrate the existence of a thriving research program on HSV and cervical cancer. I’ll discuss the significance of this below, but for now I want to introduce the question that first bothered me when I started this research: why did anyone think that HSV might cause cervical cancer?

HSV and cervical cancer?

The roots of the claim that HSV might cause cervical cancer came from some observed correlations between certain sexual behaviours and the risk of developing the disease. In fact, cervical cancer has long been noted to behave more like an infectious disease than a typical cancer. Perhaps the most interesting series of observations of this kind was produced by Rigoni-Stern in 1842 (available in English translation as Stavola, 1987), which described a series of cases in Verona (1760-1839) that showed much higher rates of cervical cancer in married women than in nuns. One possible explanation for this difference was the celibacy practised by the nuns. Other studies during the nineteenth and early twentieth centuries found that other behaviours related to sex also seemed to modify the chance of developing cervical cancer. In general, the more sex an individual had had, the greater their risk of getting cervical cancer. So being married, having sex in adolescence, contracting other sexually transmitted infections (like syphilis) and having a large number of children positively correlated with the disease, while abstinence from sex negatively correlated with the disease.

By the time that mass population screening for cervical cancer was introduced in the mid-twentieth century, these sexual risk factors had been extensively researched. One great quote from the Aberdeenshire cervical cancer research project sums up the thinking typical at the time:

The cancer patient is characterised by more marital misadventures, divorce and separation, more pre-marital coitus and deliveries and more sexual partners. (Aitken-Swan and Baird, 1966: 656)

So perhaps cervical cancer was a consequence of a sexually transmitted disease. While the usual suspects (syphilis and the like) did not seem to account for it, research in different contexts suggested that herpes viruses might cause many kinds of cancer. The details of this are rather complicated (and probably something for another post), but the upshot was that (in the mid-twentieth century) herpes viruses seemed the most likely suspects as causes of cancer in humans. Happily for researchers at the time, this seemed to provide a causal explanation for the correlation between (sexually transmitted) HSV and cervical cancer (see, for example, Kessler, 1976).

Not much of a mechanism

So infection with HSV was an attractive explanation for these sexual risk factors. But was it also the cause of cervical cancer? Well, the lack of correlation between other sexually transmitted infections with cancer of the cervix suggested that correlation wasn’t just an accident, but was instead due to a causal relationship (Rawls et al, 1973: 1482). Other evidence, like serology, the mutagenic power of HSV, the detection of fragments of HSV DNA in cervical cancer cells, and the causal role played in other tumours by herpes viruses, seemed to support this causal claim. Yet its details remained elusive. Most of the papers from the 1973 Cancer Research volume mentioned above tried, but failed, to detect some specific evidence of a the mechanism linking the virus with the disease. And the details of this mechanism remained elusive, as we might expect. Yet the claim that HSV caused cervical cancer persisted well into the 1980s, and lead to significant resistance when other causal claims (like that involving HPV) were mooted. In conclusion, when combined with the plausibility of possible mechanisms involving HSV, the correlation between HSV infection and cervical cancer meant that it was unthinkable that HSV did not cause cervical cancer.

Conclusion

It’s pretty uncontroversial to say that we should distrust brute correlations, or mistake a correlation for a causal relation. But there are other, more subtle, issues that this case raises that I think we should be similarly mindful of. The first of these is the difference between plausible, and actual, mechanisms. HSV was linked to cervical cancer by an extremely plausible mechanism. But no actual mechanism was found. Good mechanisms in this context are specific and local: and we should be extremely cautious about mechanisms that are purely plausible. The boundaries here are pretty vague, though, and a future research goal for me is to try and come to grips with the difference between plausible and actual mechanisms.

The second issue that I’d like to raise by way of conclusion concerns being explicit about causal evidence. The HSV case is an example where, despite a great deal of research, no specific evidence mechanistically linking HSV and cervical cancer was found. However, this lack of evidence is not readily apparent from individual papers in the literature. Health researchers have recently adopted many strategies to more effectively review evidence of correlations (like meta-analysis and systematic reviews of trials). I imagine that a similar strategy for explicitly considering evidence of mechanism would have been valuable for HSV researchers as a way of detecting a persistent absence of evidence in the face of inquiry.

References

Aitken-Swan, J, and Baird, D. 1966. “Cancer of the Uterine Cervix in Aberdeenshire. Aetiological Aspects.” British Journal of Cancer, 20(4): 642–59.

Dürst, M, Gissmann, L, Ikenberg, H, and zur Hausen, H. 1983. A papillomavirus DNA from a cervical carcinoma and its prevalence in cancer biopsy samples from different geographic regions. PNAS 80(12): 3812–3815.

Kessler, II. 1976. Human Cervical Cancer as a Venereal Disease. Cancer Research. 36: 783-91.

Rawls, WE, Adam, E, and Melnick, JL. 1973. “An Analysis of Seroepidemiological Studies of Herpesvirus Type 2 and Carcinoma of the Cervix.” Cancer Research, 33(6): 1477–82.

Stavola, BD, 1987. “Statistical Facts about Cancers on which Doctor Rigoni-Stern based his Contribution to the Surgeons’ Subgroup of the IV Congress of the Italian Scientists on 23 September 1842. (translation).” Statistics in Medicine, 6(8): 881–4.