Georg Ivanovas From Autism to Humanism - systems theory in medicine

5. Empirical medicine

previous -- home -- content -- next

5.7 Efficacy of CAM – the methodological problem

Most methods of CAM are not acknowledged by the scientific community. This is partly due to their theories. But it is also due to the fact that most of these methods are not able to prove their efficacy through randomised controlled trials (RCTs). “Some maintain that complementary medicine cannot be evidence-based in the conventional sense of the word; that "softer" types of evidence need to be taken into consideration as well); that placebo effects must not be dismissed as nonbeneficial; that the healing encounter includes significant factors that may never be quantifiable; that "the scientific method cannot measure hope, divine intervention, or the power of belief." And, obviously, research in complementary medicine "must consider social, cultural, political, and economic contexts” (Ernst 2003a).

Although proponents of EBM believe that “saying something cannot be tested by trials simply betrays ignorance of the scientific method - you can construct a trial to test practically anything” (Ernst cited in Carter 1996), there is a methodological problem which makes it nearly impossible to apply RCTs to regulative therapies (Fønnebø et al 2007). This is of special interest in the context of this work as a systemic medicine will probably also have a regulative character. That is, a systemic medicine will encounter the same difficulties with EBM as CAM has.

In the first place, there is a certain bias of orthodox medicine to prove the infectivity of CAM (Linde et al 2001b). This can be done with a lot of statistical techniques (overview in Kiene 2001: 85-100). Two major publications proving the infectivity of homeopathy shall demonstrate this:

242 asthmatic patients with positive prick test for house dust mite were given a potency of 30C of house dust mite. The result showed no difference between the placebo and the verum group (Lewith et al 2002). The bias in this study is the following: First, there was no individualisation. All patients were given the same remedy, something incompatible with homeopathy and any other strategic therapy. Second, a remedy was used (house dust mite) which is not used in homeopathy. As the authors were aware of their unusual procedure, they stated: “This treatment is not usual homoeopathic practice but offers a testable model for differentiating between infinitesimal homoeopathic dilutions and placebo.” That is, they constructed a setting that can be controlled, although it has nothing to do with homeopathy. This would exclude any statement on homeopathy or at least make the results very doubtful. Nevertheless, the related editorial came to the conclusion “that new trials of homoeopathic medicines against placebo are no longer a research priority” as research funding is a scarce resource (Feder/Katz 2002). This comes despite the fact that homeopathy has proved to be effective under practitioner’s condition in allergic rhinitis (Lewith et al 2002). In this case is applicable what Robert Temple has said: "When you have a trial like that you can learn two things...You can tell whether your drug is better than a placebo, and you can also tell whether the study is a useful study” (Taubes 1995 cited after Kiene 2001:86).

The second publication was a meta-analysis overviewing the available trials for homeopathic treatment. The study used 110 studies for the review. Due to statistical or scientific shortcomings the authors discarded 96 studies. On the basis of 14 studies they found a certain evidence for their assumption that homeopathic treatment is only a placebo therapy (Shang et al 2005). The related editorial proclaimed “The end of homeopathy” (Lancet editorial 2005). However, the study itself had severe shortcomings. For example, it was not mentioned which studies were included into the meta-analysis and the authors and the Lancet were reluctant to reveal them. Only after months of international protest the data for the meta-analysis were published on the web-side of the authors but removed after short. They had used studies like the house dust mite study and similar others which had nothing to do with the strategic concept of homeopathy. The material revealed also further shortcomings. (Dellmour 2006). A comparable article in favour of homeopathy would have been retracted immediately.

Interestingly, Lancet had published a more carefully made meta-analysis in favour of homeopathy nine years earlier (Linde et al 1997). This meta-analysis was ignored by the medical community, whereas the negative outcomes made it into the headlines.

Simultaneously with the cited negative meat-analysis, BMJ published a meta-analysis on antidepressant showing that they do not prove better than a placebo (Moncrieff/Kirsch 2005). But nobody declared the end of the antidepressant therapy. We see here a biased atmosphere making sound discussions on the efficacy of CAM difficult.

The major problem in judging CAM is that most of its methods use an individualizing therapy, something incompatible with randomisation. Even a simple body massage has to be individually adapted for every patient, otherwise it cannot be successful. What is good for one person can be harmful for another. So how to evaluate a study which compares the effect of a standardized physiotherapy (massage, warmth, cold, etc.) with the simple, standardized instruction to stay active? The result that physiotherapy is not superior to this simple advice (Frost et al 2004), might not be of great relevance. Important, however, is that both treatments (standardized therapy and standardized advice) are bad medicine.

The fact that a schematic therapy is bad medicine has already been stressed for balneology (chap 5.4.a) and emergency medicine (chap. 2.1.f). As methods like psychotherapy, acupuncture or homeopathy are even more dependent on an individualization they can less be judged with a simple randomised study. This might partly explain why these methods work well under practitioners’ conditions (Güthlin et al 2004, Vickers et al 2004) – have better results with lower costs (Witt et al 2005, Ratcliff et al 2006) – but fail to prove effective when put into the corset of randomised trials. The higher the methodological standard of a trail is, the less effective are such therapies (Süß 2004).

Besides individualisation another factor is important. In hamsters it has been shown that wounds heal quicker if the wounded is in a normal social context. If the animals were in a glass tube (comparable to a hospital based medicine?) the wound became even worse (Detillion et al 2004). In the second case a therapeutic intervention might prove effective, but could turn out to be a placebo in the first case. That is, the more supporting the frame conditions are, the less specific a therapy will work. As it is generally accepted that regulative therapies like psychotherapy, homeopathy and acupuncture provide a supporting atmosphere whereas hospitals often do the opposite it is difficult to compare their results of efficacy control.

To illustrate how RCTs miss the principles of an individualized therapy I published the following example (Ivanovas 2001a): In headache acetylsalicylate may be tested against placebo. But it makes no sense to test the homeopathic remedy Bryonia D6 against placebo, as every patient has to receive an individualised remedy. On the same logical level would be to give in a general practice with all kind of diseases (depression, asthma, peptic ulcer, colds etc) one group acetylsalicylate and the other placebo. The results after one year would say nothing about the effectiveness of acetylsalicylic acid in special or of drug therapy in general. This example prompted strong objections, claiming I would reject randomisation (Schuck et al 2001). But there is no randomisation in an individualizing therapy. Thus, the specific approach is not appropriate for many CAM therapies (Paterson/Dieppe 2005).

But there is also a formal problem: A study can only come to conclusions on the issue they measure. When a potency of house dust mite in an allergy against house dust mite does not prove better than a placebo then the result is only that a potency of house dust mite is not superior to placebo iunder these conditions. It does not say anything about a potency of house dust mite in any other diseases, nor does it allow conclusions about potencies in general or on homeopathy in special. In the same way, a study which does not show a positive effect of acetylsalicylic acid in breast cancer (Cool et al 2005), cannot evaluate the effectiveness of acetylsalicylic acid as such, and the effectiveness of pharmacotherapy in general. That is, most designs of RCTs in CAM violate the basics of logical typing (chap. 3.2).

There have been attempts to perform studies according to the necessities of CAM. The Munich Headache Study was a carefully performed study. It fulfilled the a priori criteria of statisticians and homeopaths. The study consisted of patients who suffered from headache with an average of 23 years, had 2-3 times headache per week with an duration of more than 8 hours. Patients had, in average, a lot of premedication. Participants had no special high expectation in the result of the study. The normal medication was interrupted and patients were treated according to the approved regimen of the German Migraine Society. These patients were treated for 12 weeks where a patient was seen twice by the homeopathic team, once at the beginning, once after 4 weeks where the prescription could be changed. The study showed no superiority of the homeopathic remedy to a placebo. However, in both groups there was a reduction of headache of about 25% in frequency, duration and intensity (Walach et al 1997). The result showed that the homeopathic treatment worked, but not the remedy.

Afterwards, as always when a study does not show the expected results, there had been many objections by homeopaths such as

Independent of the soundness of these arguments the study poses a problem for homeopaths and for orthodox medicine. The homeopathic therapy was highly effective, low in side effects and performed much better than the standard drug therapy. But it did not work in a specific way. The paradox that something is effective although it is not effective is typical for a violation of logical types. There has to be some logical confusion, as nobody would claim to ban orthodox drug therapy although this would be the only legitimate conclusion according to EBM standards.

The result of the Munich Headache Study is not singular. Quite a lot of studies come to the result that the CAM treatment is effective but not better than a sham intervention. The German Acupuncture Trials (Gerac), the largest ever undertaken study on acupuncture came to this conclusion. Patients with osteoarthritis of the knee and low back pain were treated either with a conventional therapy, or with acupuncture or with placebo-acupuncture where a point was needled not existing in Chinese medicine. After half a year the correct acupuncture performed not better than the placebo acupuncture, but both were much better than conventional pain therapy (Hackenbroch 2004). This was verified with studies on gonarthorosis (Bermann et al 2005, Scharf et al 2006), on migraine (Linde et al. 2005; Diener et al 2006), on different forms of chronic pain (Melchart et al. 2006) and on low back pain (Haake et al 2007).

The possible explanations demonstrate the difficulty and confusion around the issue:


The results of the Munich headache study and the German Acupuncture Trials remind the dodo verdict of psychotherapy. The dodo verdict says that psychotherapy works but the effect is independent of the technique, i.e., all kind of therapies work about the same.

The early positive results of psychotherapy were always individual case reports. No efficacy control existed. A first blow shocked the psychotherapeutic community when Eysenck published in 1952 an article, stating that psychotherapy is not more effective than no therapy and might even be less effective (Eysenck 1952). Since then such results are regularly found (Stevens et al 2000). For example,

Mostly such trails are of low quality and also Eysenck has been criticized mainly on a formal level (Mc Neilly/Howard 1992). The arguments are much the same as seen in CAM, concerning issues like individualization, specific treatment, the role of the therapist etc.

The last decades, especially in the age of EBM a lot of research on the efficacy of psychotherapy was done. But the results resemble more “a lottery” (Goldbeck-Wood/Fonagy 2004). Two comprehensive meta-analyses based on the available studies came to opposite results. For one the effectiveness of psychotherapy is beyond doubt (Lambert 2003). The other finds little evidence for most methods. Only behaviourism and cognitive therapy perform well (Grawe et al.1995). Of course the work of Grawe was criticised, mainly by those methods that failed (Tschuschke et al 1998).

Grawe’s meta-analysis is in so far interesting as it is comparable with the work of Ernst for CAM. Ernst holds the only chair of CAM in Europe (University of Exeter) and tries to apply strict methods of EBM. He finds only herbal therapy convincingly effective. Strikingly, behaviourism and herbal therapy resemble most the concept of the specific therapy used in orthodox medicine. This is a further hint that the efficacy control of EBM (in the traditional way) is only suitable for a certain methodology but fails in other respects.

Both works, the meta-analysis of Grawe et al. and of the research team of Ernst (Jonas/Levin 1999; University of Exeter 2004) give overviews on psychotherapy, respectively CAM where everything is neatly summed up according to some criteria. Grawe et al and Ernst do not take any methodological problems into account and are probably not aware of them. They misunderstand that their approach does not mirror efficacy but their tool of investigation.

This can be demonstrated with music therapy. Music therapy has a lot of good observed effects, especially in traumatised patients (Case/Else 2003) or after stroke (Särkämö et al 2008). Sacks, in his account of the inability to move his leg after the inner representation had been lost (chap. 2.2), describes how he overcame this inability suddenly by hearing the music of Mendelssohn (Sacks 1984: 146). Sacks cites Novalis who said that every disease is a musical problem and healing is a musical solution (Sacks 1984: 137).

This shall be further illustrated by the following example: A five year old autistic boy who had never uttered a word had been send to music therapy. The therapist accompanied the screaming and yelling of the boy with related sounds on the piano. Thus, he was able to ‘communicate’ in the terms of the boy. Slowly they developed a stronger communication. Finally the boy sang the first words in his life “a song”. With this technique the boy developed a certain ability to communicate (Neugebauer 1994 cited after Kiene 2001: 54-55). From the point of view of evidence-based medicine such a narrative is without any relevance. The development of the boy could have been accidental, or the effect of the therapy might have been due to a placebo effect. In any case, it is impossible to educe the efficacy of music therapy from such single cases even, well, for once, there might be a certain plausibility. But a prove of efficacy? Never!

Music does a lot more (chap. 5.4.a). Amateur chorus singing increases IgA and decreases cortisol levels, i.e., boosts the immune system (Kreutz et al 2004). Music helps to develop verbal and cognitive abilities in childhood (Ho et al 2003) and better cognitive abilities in childhood are correlated with better health later (Martin et al 2004). But also the recitation of poetry synchronizes respiration and heart rate (Cysarz et al 2004). Making music together leads to a synchronization of the brains (Lindenberger et al. 2009), to a kind of mirroring, where a lot of information is exchanged that never can be quantified. That is, music and rhythms might influence inner rhythms in a multitude of ways. Neurophysiology developed related models recently (Peretz/Zatorre 2003, Sloboda 2003, Dobbs 2008).

But how should all this be proved in a randomised trail? Studies like “The effect of Mendelssohn’s music in postoperative movement disorders. A prospective, randomised, (double-blind,) placebo-controlled trail” (the placebo might be the evening news) or “The use of the piano in the treatment of autistic boys. A prospective, randomised, (double-blind,) placebo-controlled trail” (the placebo might be a triangle) are absolutely absurd. They misunderstand what is happening in life and in the therapeutic relation.

Trails are not able to model individual reactions. This is why music therapy will never show the same effects as a behaviouristic intervention to combat agoraphobia, although music therapy might be much more beneficial for the health of the patient. Music therapy might induce a learning on a higher level, whereas behaviouristic and cognitive approaches might change only the surrogate parameter leaving the adaptation untouched.

CAM, like every method (Sackett/Wennberger 1997) must find its own design of efficacy control. The main problem will be to be solve the question how individualization might be conceptualised? Newer attempts like formal case studies (Thompson 2004) or Cognition-Based Medicine (Kienle 2005) try to overcome the usual restrictions. But only time will tell whether such methods can really be translated into medical practice.

previous -- home -- content -- next