Georg Ivanovas From Autism to Humanism - systems theory in medicine

2.5 The limits of evidence based medicine

previous -- home -- content -- next

d) lack of significance and predictability

Even if studies are carried out correctly the question remains whether their results are true for a given population. The Simpson paradox says that probabilities that are true for two populations might be no longer be true for the two populations together (Bogomolny; Malinas/Bigelow 2008). If something is statistically good for men and good for women, it might be as well be bad for people (Baker/Kramer 2002). Or if A bests B and B bests C in different randomised trails the conclusion that A bests C is a fallacy (Baker/Kramer 2002). As a study provides only valid results for the population of the study, it is of restricted value for any other population. E.g., the predictive accuracy of the Framingham study, which is solid and large, overestimates the coronary risks assigned to the individuals of the United Kingdom (Brindle et al 2003). All conclusions made on the basis of the Framingham study are only true for Framingham in this period of time. For any other population it is but a hint. “You must be careful in extrapolating from the results of trials done in selective patients, mostly in the United States, to the real world and most other countries” (Chaner in Smith 2004b).

Radical critics (Beck-Bornholdt/Dubben 2003) declare medical research based on statistics as deeply dubious. They formally prove that the probability of error for a study, although high, is not ascertainable at all. According to them, statistics lead medicine into a dead end. Other critics argue similarly showing that significance and probability of effectiveness have nothing to do with each other and that results are extremely vague (Weihe 2004).

This can be translated into medical practice:

Furthermore, there is no individual predictability in statistics. Beck-Bornholdt and Dubben state: “Large numbers show a statistically exact result. But nobody knows for whom it is the case. Small numbers show a statistically unsuitable result, but we know whom it concerns. Difficult to say which kind of ignorance is more useless” (Beck-Bornholdt/Dubben: 218, my translation). These doubts are also shared by other statisticians and scientists (Gigerenzer 2003, Bagshaw/Bellomo 2008, Kienle 2008), although their criticism is mostly more decent.

Cluster trails (Bland 2004) try to balance the mentioned disadvantages. But as the problem is not technical but epistemological, such a solution will not provide more sound results.

As statistical results are already of limited use for a single disease, they are even less useful in complex diseases. “Many patients in primary care have two or more diseases, even without a common biological basis (comorbidity or multimorbidity), which is particularly the case in elderly people--78% of the population aged 80 years and older. These coexisting disorders typically do not have a biological link, but nevertheless affect treatment. What is the evidence to follow in the management of an 82-year-old patient with chronic obstructive pulmonary disease and type 2 diabetes, because treatment with corticosteroids could interfere with control of glycaemia?“ (Messeneer et al 2003).

previous -- home -- content -- next