Alleged Ideological Bias In Homeopathy Reviews

Research News

Alleged Ideological Bias In Homeopathy Reviews

16 January, 2014 by David Finer

Homeopathy is generally superior to placebo, particularly in immune-related conditions. But it has been discounted by ideologically biased researchers, who have excluded almost all published studies, referring to virtual data, and using inappropriate statistics, writes physician Robert Hahn in a critical analysis in Forschende Komplementärmedizin.

In a previous article, I C has referred to some of the homeopathy studies. In this article, we probe deeper into the evidence, summarizing an article by Swedish physician and Robert Hahn in the international peer-reviewed  journal of complementary medicine research Forschende Komplementärmedizin.

In his analysis, Hahn looks at meta-analyses, including pooled data from placebo-controlled clinical trials of homeopathy as well as ensuing debate articles.

Some Basic ConceptsFirst, some basic concepts to hopefully assist the reader. Typically, in the life sciences, treatment effects are reported with reference to

• odds ratios (OR) and
• confidence intervals (CI).

Furthermore, in order to gain greater certainty of evidence, rather than relying on a single study, many studies are evaluated together. Their results are ”pooled”. According to the traditional ”hierarchy of evidence”, so called randomized controlled trials (RCTs) are preferred.

An RCT is a specific type of scientific experiment, in which study subjects are randomly allocated to receive one or other of the alternative treatments under study. An RCT may contain control groups, in which groups receiving the experimental treatment are compared with control groups receiving no treatment (a placebo-controlled study) or a previously tested treatment (a positive-control study).

Returning to Hahn´s discussion of the homeopathic literature, different studies of the ”same” disease or remedy may vary greatly in design, the scope, focus, and population size (and hence the validity and reliability of their results). This calls for some sophisticated tools of analysis, but – as Hahn demonstrates – also opens up for a certain measure of arbitrariness.

An odds ratio (OR) is a measure of association between an exposure and an outcome. Simply put, an OR above 1 signifies that the ”exposure” is associated with a higher likelihood of a positive outcome. In the case we are discussing here, the exposure is the homeopathic treatment.

The confidence interval (CI) is used to estimate the precision of the OR. It can also be called the margin of error. It tells you the possible range around the estimate given and also how stable/reliable the estimate is. Typically, a 95 percent CI indicates the likelihood (95 percent) that the observed result falls in a certain interval, known as the confidence interval.

A large confidence interval indicates a low level of precision of the association signified by the OR, whereas a small confidence interval indicates a higher precision of the OR.

Linde Study Positive for HomeopathyIn 1997, Klaus Linde and co-workers identified 89 clinical trials that showed an overall odds ratio of 2.45 in favor of homeopathy over placebo. Hence, an odds ratio of 2.45 indicates a high likelihood of the superiority of homeopathy over placebo.

When all data were pooled, the OR was 2.45 in favor of homeopathy and the 95 percent CI 2.05–2.93. After correction for publication bias, the OR decreased to 1.78 (CI 1.03–3.10). When only the 26 studies of highest quality were included, the benefit was somewhat weaker but still statistically significant, OR 1.66 (CI 1.33–2.08).

There was a trend toward smaller benefit from studies of the highest quality, but the 10 trials with the highest quality (as measure by a so called Jadad score) still showed homeopathy had a statistically significant effect.

In Hahn´s view, these results provoked academics to try and analyse the material in alternative ways, in their efforts to disprove the positive results. They excluded a lot of them, often to the point that they based their conclusions on only 5–10 percent of the material, or on virtual data. In 1998, Edzard Ernst – a former practitioner of homeopathy, well-known for his almost universally negative studies of alternative medicine – selected five studies using highly diluted remedies from the original 89, and concluded that homeopathy had no effect.

Virtual Data PreferredIn 2000, Ernst and Pittler sought to invalidate the statistically significant superiority of homeopathy over placebo in the 10 studies with the highest quality as measured by the so called Jadad score (OR 2.00 (1.37–2.91).

The new argument was that the Jadad score and OR in favor of homeopathy seemed to follow a straight line, and hence should theoretically show zero effect. This reasoning argued that the assumed data are more correct than the real data.

The ultimate argument against homeopathy is the ‘funnel plot’ published by Aijing Shang’s research group in 2005. However, writes Hahn, the funnel plot is flawed when applied to a mixture of diseases. This is because studies with expected strong treatments effects are, for ethical reasons, powered lower than studies with expected weak or unclear treatment effects.

Cucherat et al identified 118 RCTs as potentially evaluable but excluded 85 percent of them, usually because the primary endpoint was deemed unclear. The overall treatment effect in the 17 studies was still highly statistically significant in favor of homeopathy. The authors remained skeptical about homeopathy although their data, even after most of the statistical power was removed by excluding 86 percent of the clinical trials, showed that the therapy is superior to placebo.

The meta-analysis published by Shang et al. in 2005 identified essentially the same set of clinical trials as Linde et al. with the addition of some more recent material. The group identified 165 publications and – in the final process – excluded all except 8 studies, without clearly explaining why, Hahn maintains.

In addition, they relied on the flawed funnel plot reasoning. Despite these shortcomings in the Shang et al study, their calculations have been widely used to claim that homeopathy lacks clinical effect.

Tempting for the SkepticHahn agrees that poor quality studies and those that lack the necessary data should be excluded. However, he says, it is also tempting for the skeptic to ruin any evidence that might be for the effectiveness of homeopathy.

When a lot of studies are excluded, the mix of conditions in the remaining studies becomes very important for the conclusion, perhaps unduly so. We must remember, Hahn explains, that the overall conclusion made in these meta-analyses relates to the overall efficacy of a heterogeneous group of treatments for a heterogeneous group of diseases.

Hence, the results of a single study in a meta-analysis of various studies can make the difference between accepting or rejecting the effectiveness of the treatment under study, in this case homeopathy.

Hahn gives the example of a single study of muscle soreness in 400 long-distance runners, without which the result would have shown the statistically significant superiority of homeopathy over placebo.

Hahn further critiques how Shang and collaborators arrived at the number of 8 studies to include in the final analysis. He finds strong reasons to believe that the authors may have manipulated the study selection until eventually they found the desired result.He bases these allegations on statements made by the group on the usefulness of homeopathy in a previous report and in public.

Evaluation in Two StepsHahn makes the point that therapies should be evaluated in two steps, one being an objective summary of the evidence for the efficacy of the therapy, the second making recommendations for use, clinical guidelines and so on.

The second step is more subjective and integrates the evidence as well as scientific, ethical, economic, and practical perspectives. Authorities often ask separate constellations of researchers or stakeholders to performs these steps. It might otherwise be tempting for researchers to distort the evidence when other considerations disagree with the evidence.

However, Hahn believes that it has been common for these roles to overlap in the metaanalyses of pooled clinical data about homeopathy. That is why, he maintains, the conclusions are so different and the debate so emotional.

He explains that his own impetus to write on the subject was evoken 2011 as a reaction to a campaign by skeptics in Sweden against homeopathy in the Summer of 2011. This involved i.a. a highly pubilicized action with among others the famous and very popular Swedish physicist/astronaut Christer Fuglesang.

Staged a ”Suicide Attempt”Fuglesang and a number of skeptical colleagues staged a dramatic happening in connection with the so called Almedalen week on the island of Gotland in the Summer of 2011. During this week, which is typically visited by the whole of the Swedish political, media and non-.governmental elite, they staged a “suicide attempt”, involving ingesting large amounts of homeopathic remedies.

After surviving the exercise, they announced that they had proven that the remedies taken lacked any effects. In connection with this event, skeptics claimed that homeopathy is poorly studied, and that those studies that do exist fail to show a positive treatment effect, and are of low quality.

None of these claims are true, Hahn observes. Finally, based on his critical analysis of the studies of Ernst and Shang, described avove, Hahn warns that ideological biases color the evaluations of homeopathy by these researchers.

Future meta-analyses should focus on areas where homeopathy tends to be effective instead of diluting the results with data from diseases in which an effect is unlikely. Socio-scientific  issues are also relevant to discuss, Hahn feels.

Homeopathy Better Than PlaceboHe concludes that homeopathy is not very strong but seems to show better effects than placebo. This is particularly true in conditions that either are known or can be assumed to arise from the immunological system. According to Linde and collaborators this could include allergic rhinitis,rheumatology, dermatology (except warts), and certain neurological disorders, such as seasickness and migraine. However, benefit is more uncertain or absent in studies of asthma, surgery, gastrointestinal disorders, anesthesiology, and gynecology.

Finally, Hahn asserts, the RCT is not the best form for evaluating the efficacy of homeopathic remedies, as well as the efficiency of more complex interventions in complementary medicine. Traditional therapy might be a better comparator than placebo as CAM therapies often show a large nonspecific effect.

Hahn also refers to the collection of 263 high-standard RCTs of homeopathy by Mathie and collaborators for future guidance on which conditions might be suitable for homeopathic treatments and whether placebo or other treatments should serve as controls.

Until then, writes Hahn, homeopaths might do well to focus on the areas and treatments where placebo-controlled studies have shown a better outcome than placebo. In reality, homeopathic treatments are usually individualized while most of the performed clinical trials are non-individualized. Therefore, the practice is not evidence-based, regardless of the results of all the trials.

David Finer

Read more

Hahn R. Homeopathy: meta-analyses of pooled clinical data. Forschende Komplementärmedizin / Research in Complementary Medicine. 01/2013; 20(5):376-81.

Mathie RT, Hacke D, Clausen J, Nicolai T, Riley DS, Fisher P. Randomised controlled trials of homeopathy in humans: characterising the research journal literature for systematic review. Homeopathy.Vol 102, Issue 1, January 2013, Pages 3–24.

Tags: alternative medicine, bias, CAM, CI, communication, confidence interval, conflict, controversy, David Finer, Edzard Ernst, evidence, Forschende Komplementärmedizin, homeopathy, ideology, odds ratio, OR,placebo, randomized controlled trial, RCT, review, Robert Hahn

http://www.integrativecare.se/en/alleged-ideological-bias-in-homeopathy-reviews-3609