In the Pursuit of Evidence
by Kate Chatfield
This paper was first published in 2007, at a time when homeopathy in the UK was receiving a considerable amount of adverse publicity. Unfortunately, the situation is not greatly improved and currently homeopaths in many other countries around the world are facing a similar challenge. Repeatedly, critics ask the same question: “Where is your evidence?” Consideration of this question led me to become involved with homeopathy research many years ago, but increasingly I found that my background in philosophy compelled me to challenge the very concepts of evidence, proof and science that were driving the research agenda. What follows is an account of evidence-based medicine, some of the implications for homeopathy, and philosophical challenges for the foundations of our assumptions. It has been modified slightly from the original 2007 paper in the light of subsequent developments.
In spite of the potential for benefit, its widespread usage, and the apparent acceptance of homeopathy by the general public, the subject can lead to emotionally charged debates. In some parts of the world there are particularly vocal objectors and it has been described variously as a form of ‘medical heresy’ (Stambolovic 1996), no better than ‘superstition’ (Singh and Ernst 2008 ), and ‘pseudoscientific’ (Colquhoun and Isbell 2007, Hall 2009).
The primary objection to homeopathy focuses upon the lack of a rigorous evidence base as is required for conventional medical health care (Ernst and Cohen 2002, Smith 2008, Ernst 2011). No new conventional medications are introduced without rigorous testing through established scientific methods such as clinical trials. The practice of homeopathy, however, is primarily rooted in case-based, empirical evidence, passed down through generations. This form of knowledge, whether in written or purely verbal form, is subject to testing for effectiveness and safety only when applied in the real world to individuals and it does not satisfy those who demand that all healthcare interventions should be rooted in a ‘scientific’ evidence base.
The introduction of evidence-based medicine (EBM) has been heralded as the most recent revolutionary phase in health care, described as a ‘paradigm shift’ that will change medical practice for years ahead (Guyatt et al 2002). This movement was initiated in 1992 when a group of physician-researchers, known as the Evidence-Based Medicine Working Group (EBMWG 1992) published an article urging physicians to base clinical decisions purely on evidence. In this way unreliable, intuitive judgements would be replaced with rational calculation and the use of research.
The notion of EBM is predicated upon the assumption that there is a hierarchy of evidence for medical interventions, such that certain types of evidence are deemed of greater value than others. At the top of this hierarchy are placed results from randomised controlled trials (RCTs) and systematic reviews or meta-analyses of these trials. The synthesis of large amounts of clinical trial data into manageable systematic reviews or meta-analyses is meant to revolutionise medical practice and offer objective and politically transparent criteria for treatment choice and funding decisions. Proponents of EBM commend themselves on their attempt to ‘realign medicine with science’, whilst critics object that EBM emphasises exclusively the science of medicine whilst denying the art of medical practice (Miettinen 2001). Since the broad adoption of EBM principles, there has been increasing pressure on all forms of traditional and complementary medicine (T&CM) to provide the kind of evidence that is deemed of high value from RCTs and reviews of these trials, to enable comparative assessment of efficacy and safety. Without this type of evidence most governments are unwilling to fund provision, research or development of T&CM on anything but a small scale.
Implications for Homeopathy
In the UK, Clinical Commissioning Groups (CGCs), controlling over 60% of the NHS budget, respond to the needs of their local communities when making decisions about the funding of health and social care in their local areas. The CGCs, working with limited budgets, consider many factors when establishing priorities. The judgements of the CGCs are directly informed by the National Institute for Health and Care Excellence (NICE) who are responsible for providing national guidance on the promotion of good health and the prevention and treatment of ill health and currently NICE does not recommend that homeopathy should be used in the treatment of any health condition.
The requirement for evidence has direct implications for the availability of homeopathy in the UK where it has traditionally been part of the national health service (NHS) since its inception in 1948. Over recent years there has been a significant reduction in this service, for example, in Scotland where three large areas have taken the decision to stop referrals to the Glasgow Centre for Integrative Care (formerly the Glasgow Homeopathic Hospital) with the result that many people no longer have access to this service.
Finding the Evidence
The two main resources for evidence-based practice in the UK are Bandolier (www.jr2.ox.ac.uk/Bandolier), written by Oxford scientists, and the Cochrane Library (www.cochrane.co.uk). Both carry easily accessible information about an extensive range of medical conditions and the research results. A brief examination of the Cochrane Library quickly highlights the vast disparity between both the amount and the types of research evidence that are available for allopathic and homeopathic interventions. For example, under ‘rheumatoid arthritis’ there are 58 different Cochrane reviews, with research evidence for each particular kind of intervention reviewed separately: 27 different drugs, different forms of exercise, different forms of electrotherapy and so on. Whereas, across the whole site, homeopathy is only reviewed under 7 areas in total: Irritable bowel syndrome, ADHD/hyperactivity, asthma, dementia, induction of labour, adverse effects of cancer treatment, and the use of Oscillococcinum for prevention and treatment of influenza.
Typically, where homeopathy has been subject to systematic review for a particular condition such as asthma, all kinds of homeopathy are lumped in together. Isopathy, therapeutic prescribing, combination-remedy treatment, individualised prescribing, and so on, are all considered under the label of ‘homeopathy’. Hence it is immediately apparent that homeopathy is not reviewed in an equivalent way to allopathy. We would not find a systematic review of allopathy, including each kind of allopathic intervention for the treatment of asthma, because it would be considered meaningless. Whereas there are so few trials of homeopathy for the treatment of asthma that they are all banded together regardless of whether or not they in any way reflect what happens in the real world.
There have of course been many more research trials than are mentioned in either the Cochrane Library or on Bandolier, but research in homeopathy is so badly under-resourced that there is still only a tiny fraction of the amount that would be necessary to make sensible comparisons with trials of allopathic interventions.
Historically the T&CM research community have responded to the call for evidence and attempts have been made to address this requirement but many researchers have described reasons why this is proving to be a challenging task (Caspi and Bell 2004, Ahn and Kaptchuk 2005, Block and Jonas 2006, Bell, Koithan et al. 2012). These reasons include the following:
Internal / External validity problems
As reliable indicators of causality RCTs are considered to have strong internal validity, but this is often created at the expense of external validity, as they can be a poor model of what happens in healthcare in the ‘real world’. When homeopathy treatment is adapted to fit an RCT model, the intervention being tested commonly bears little relationship to the way the treatment is offered in clinical practice. This is primarily a consequence of attempting to measure a complex and holistic intervention in the reductionist and linear model of the RCT (Mathie, Van Wassenhoven et al. 2014). In other words, efficacy studies for pharmaceutical drugs assume a direct, mechanistic cause-effect relationship between a specific intervention and a specific therapeutic outcome whereas homeopathic practitioners do not necessarily treat a named symptom or disease directly (Bell, Koithan et al. 2012). For example, most RCTs investigating the efficacy of homeopathic remedies for influenza have focused on testing the effects of one homeopathic remedy, such as oscillococcinum, on a limited range of symptoms (Ulbricht, Chao et al. 2011), whereas in the real world a homeopathic practitioner is likely to select from a very wide range of homeopathic remedies for influenza, basing their prescription upon the peculiar symptoms for each individual of a mental, emotional and/or physical nature.
Plausibility bias
Even when trials have produced positive results they are dismissed as flukes, or subject to researcher bias because of the belief that homeopathy simply cannot work. For example, David Taylor Reilly and colleagues conducted a series of trials, over a number of years, in patients with hay fever, asthma and perennial rhinitis that demonstrated a significant difference between the placebo and homeopathic groups. The important thing about these trials was that the results were reproducible and yet they are still not considered firm evidence (Reilly et al, 1986; Reilly et al 1994; Taylor et al, 2000). It is clear that a prior disbelief in homeopathy, rooted in the perceived implausibility of any conceivable mechanism of action, acts as a Kevlar barrier to acceptance of positive results (Rutten, Mathie et al. 2013).
Lack of adequate funding for large scale trials
The greatest challenge for all T&CM research, however, seems to be the scarcity of research funding. Government, charitable and private funding for research into T&CM is low with the result being that research funding for T&CM is vastly out of proportion to the prevalence of T&CM use. This can be seen clearly in the US where funding for National Centre for Complementary and Integrative Health (NCCIH) activity in 2013 amounted to $124 million whilst total funding of health research and development activity in the US amounted to $330,383 million in 2012 (Research!America 2014), indicating that NCCIH funding is equivalent to less than 0.04% of all medical research funding.
Funding for wide scale homeopathy research, of the type that would be needed to develop a robust evidence base does not seem to be available.
What We Mean When We Speak of Evidence
The current notion of EBM is fundamentally based upon the assumption that evidence can provide us with ‘facts’ about the world and that scientific beliefs stand or fall in the light of this evidence. This is however an over-simplistic understanding of the nature of evidence that philosophically would be described as explicitly ‘positivist’.
Positivists believe that scientific questions can be settled in a purely objective manner and that experimental testing allows the scientist to compare his theory directly with the facts. They claim it is possible and essential for the inquirer to adopt a distant, non-interactive posture such that values and other biasing, and confounding factors are thereby automatically excluded from influencing the outcomes. In this way science provides us with a sure route to the truth.
However, over the past 50 years much of the philosophy of science has been preoccupied with challenging the positivist picture of scientific methodology. The challenges fall broadly into two main areas:
• First, observations are necessarily dependent upon the observer
• Secondly, theories are never determined exclusively by data (underdetermination)
Observations and Observers
The positivistic understanding of evidence does not take into consideration culture, contexts and the subjects of knowledge production. What an observer sees that is, the visual experience that an observer has when viewing an object, depends in part on past experience, knowledge and expectations. Individual experiences cannot be logically deduced to be identical or even the same. Observation is dependent upon and coloured by our individual senses and our background beliefs and assumptions, so that what we see is never what really is, even under the most highly controlled experimental settings (Kuhn, 1970, and Feyerabend, 1978).
Our acceptance or rejection of evidence is also value-laden, and our natural tendency is to reject evidence that does not fit with currently held theory; hence, positive results from even the highest-standard RCTs in homeopathy will be rejected by those who do not believe in the possibility that ultra-high dilutions can have an effect.
Underdetermination
Otherwise known as the Duhem-Quine thesis, this states that any given body of evidence may support numerous, even contradictory theories. Scientific theories are inevitably underdetermined by data, such that what counts as good scientific theory depends largely upon other factors: social and political agendas, preferences, biases and whims – not the evidence itself.
We can find clear examples of underdetermination in homeopathy. Many people have carried out meta-analyses of RCTs in homeopathy in an attempt to answer the question of whether or not homeopathy is more effective than placebo. These meta-analyses have been drawing upon the same data sources, published trials of homeopathy, and yet they come up with conflicting and contradictory results. The same data can be used to support either the claim that homeopathy does have an effect over and above placebo and also that it does not have an effect over and above placebo. Both theories are underdetermined by the available data and hence the data can be used to support either one.
In August 2005 The Lancet published a damning meta-analysis of homeopathy trials, which concluded that their results were consistent with the theory that the clinical effects of homeopathy are placebo effects (Shang et al, 2005). An editorial in the same issue called for ‘the end of homeopathy’. This meta-analysis is an extreme example of underdetermination; the final statistical analysis was performed upon only eight trials of homeopathy and six trials of allopathy. The authors used certain criteria for the selection of ‘highest-quality’ trials and these criteria were based upon their own beliefs about what makes a trial high quality, and nothing to do with what makes a trial sound from the perspective of a homeopath. It would be an easy task to use different criteria for the selection of high-quality trials and generate results that support the exact opposite theory.
In more recent years we have seen governmental bodies in different countries examine the same set of research data and come to different conclusions. In the UK (2010) and Australia (2014) evidence checks led to the conclusion that there is no evidence that homeopathy works beyond the placebo effect, whereas the Swiss Report (2012) concluded that homeopathy is effective and should be covered by national healthcare insurance schemes.
Scientific Proof
James Randi, magician and self-proclaimed sceptic, has famously offered $1,000,000 to anyone who can prove that homeopathy is any more than placebo effect; but just what does he mean by proof?
• Randomised, placebo-controlled trials that demonstrate statistically significant results? We have those (Aabel, Laerum et al. 2000, Aabel 2001, Kim, Riedlinger et al. 2005)
• Systematic reviews and meta-analyses that demonstrate effect over and above placebo? We have these (Bellavite, Ortolani et al. 2006, Mathie, Van Wassenhoven et al. 2016)
• Trials involving children, demonstrating positive results? We have those too (Jacobs, Jiménez et al. 2000, Jacobs, Jonas et al. 2003)
• Trials involving the treatment of animals, demonstrating positive results? Yes we also have these (Camerlink, Ellinger et al. 2010, Werner, Sobiraj et al. 2010)
• How about trials that demonstrate the effects of ultra-high dilutions in the test tube? We have many of these (Belon et al, 2004(Belon, Cumps et al. 1999, Lorenz, Schneider et al. 2003))
But none of the above constitute proof…
So just what would serve as ‘scientific proof’ that homeopathy has a positive effect over and above placebo? Well, in fact, no amount of clinical trials could ever prove this because, as any philosopher of science can tell you, proof can never be attained through scientific endeavour.
A large proportion of our beliefs are obtained by a process of projecting from observed (past or present) events to cases that are either unknown, unobserved or in the future. For example: “Every swan I have ever seen has been white. It is likely that the next swan I see will be white.”
Scientific inquiry proceeds in exactly the same way, projecting from (repeated) observation to universal laws. David Hume was the first philosopher to highlight the problems with this form of reasoning, way back in the 18th century. Hume stated that what leads us to believe that the future will resemble the past is a matter of custom or habit and cannot be rationally justified.
Seventeenth century Europeans only had access to white swans, so even a completely unbiased researcher would have had reason to conclude that all swans were white. When Europeans finally travelled to Australia, they discovered that it is not true that all swans are white. If someone had suggested in the 17th century that some swans might be black, this would have been dismissed as entirely fanciful; no one could have predicted that some swans are black.
Karl Popper, a leading influence in 20th-century philosophy of science, is most famous for his belief that theories can never be proven, however many pieces of evidence that we have to support them. Rather, they can only be disproven. It does not matter how many white swans we observe, we could never prove that all swans are white because we can never be sure that we have seen all swans. However, the observation of just one black swan is enough to disprove the theory that all swans are white.
Hence ‘scientific proof’ is a misnomer and the best we can aim for are hypotheses that have not yet been refuted.
We are never going to be able to prove the efficacy of homeopathy through clinical studies. All we can do is add to the number of observations that support the theory that homeopathy has an effect over and above placebo. Even when we have a very large number of these positive trials, they will not constitute proof. Randi’s million dollars are safe.
The High-Dilution Problem
It’s hardly surprising that homeopathy is accused of being no more than placebo. This is a completely logical conclusion from the perspective of a person who thinks and works within the current prevailing scientific paradigm. Positive results from placebo-controlled trials of homeopathy are doubted, as critics claim that remedies are composed of nothing more than water. To accept that homeopathic remedies have an effect would entail revision of much of our basic scientific conception of chemistry and the body (Sehon & Stanley, 2003).
In spite of this apparent irrationality, homeopathy is one of the most widely used forms of T&CM in the world and people continue to use it because they experience clinical benefits.
In 2005 the results from a six-year outcome study at the Bristol Homeopathic Hospital were published (Spence et al 2005). This was an analysis of over 23,000 outpatient consultations from 1997 to 2003. Over 70% of follow-up patients reported clinical improvement following homeopathic treatment, clearly demonstrating the clinical effectiveness of the homeopathic intervention. Whilst this extensive study can tell us that homeopathy was largely effective for this particular group of people, what it cannot demonstrate is that this was not simply placebo effect.
However, if homeopaths can facilitate a placebo-induced healing response in over 70% of people who visit them, many of whom have previously not been helped by various types of allopathic intervention, then surely homeopaths should be highly revered and re-labelled ‘miracle workers’.
Time for a New Paradigm?
The history of science teaches us that there have always been things that could not be explained according to currently held scientific opinion. Science is not static, it evolves as understanding increases and is occasionally subject to a complete revolution, bringing with it an entirely new scientific paradigm (Kuhn, 1970). Kuhn describes how, when we are close to this type of paradigm shift, there will be many things that appear to be incommensurable with the prevailing paradigm. These he terms ‘anomalies’ and they cannot be explained within the framework of current scientific opinion. Homeopathy might be considered an example of an anomaly.
It is normal for people who try to work with these anomalies to be marginalised, as the emergence of a new paradigm is painful for those with vested interests in the existing paradigm. However, the emergence of a new paradigm that embraces holism, vitalism and homeopathy need not involve the rejection of rationalism. Rather we might expect a re-direction of the rationalistic tradition that encompasses better explanation of known anomalies.
Whilst the practice of homeopathy becomes ever more popular, in spite of its incommensurability with the prevailing scientific paradigm, homeopaths should not be at all surprised by resistance and even attack from those whose beliefs are firmly rooted within that paradigm. Until there is an accepted explanation about how ultra-high dilutions can have an effect, then no small amount of ‘scientific evidence’ through clinical trials will convince the sceptics.
Maybe it is time for researchers in homeopathy to focus upon what is of benefit to the development of homeopathic practice instead of spending so much of their time and effort trying to prove that homeopathy has an effect over and above placebo. Just exactly what is it that we do that enables more than 70% of patients to say that they have felt benefit? Clearly homeopaths are doing some things well, but many traditions and beliefs in homeopathy have not been subject to rigorous testing, and research has a lot to offer in this area. If, through research, that 70% figure can be improved, then just imagine the future for homeopathy.
References
Aabel, S. (2001). "Prophylactic and acute treatment with the homeopathic medicine Betula 30c for birch pollen allergy: A double-blind, randomized, placebo-controlled study of consistency of VAS responses." British Homoeopathic Journal 90(2): 73-78.
Aabel, S., E. Laerum, S. Dølvik and P. Djupesland (2000). "Is homeopathic ‘immunotherapy’effective? A double-blind, placebo-controlled trial with the isopathic remedy Betula 30c for patients with birch pollen allergy." British Homoeopathic Journal 89(4): 161-168.
Ahn, A. C. and T. J. Kaptchuk (2005). "Advancing acupuncture research." Alternative Therapies In Health And Medicine 11(3): 40-45.
Bell, I. R., M. Koithan and D. Pincus (2012). "Methodological implications of nonlinear dynamical systems models for whole systems of complementary and alternative medicine." Forschende Komplementärmedizin (2006) 19 Suppl 1: 15-21.
Bellavite, P., R. Ortolani, F. Pontarollo, V. Piasere, G. Benato and A. Conforti (2006). "Immunology and Homeopathy. 4. Clinical Studies—Part 2." Evidence-based Complementary and Alternative Medicine 3(4): 397-409.
Belon, P., J. Cumps, M. Ennis, P. Mannaioni, J. Sainte-Laudy, M. Roberfroid and F. Wiegant (1999). "Inhibition of human basophil degranulation by successive histamine dilutions: results of a European multi-centre trial." Inflammation research 48(13): 17-18.
Belon P, Cumps J, Ennis M, Mannaioni PF, Roberfroid M, Sainte-Laudy J, Wiegant FAC. (2004) “Histamine dilutions modulate basophil activation.” Inflammation research 53: 181-188
Block, K. I. and W. B. Jonas (2006). ""Top of the hierarchy" evidence for integrative medicine: what are the best strategies?" Integrative Cancer Therapies 5(4): 277-281.
Camerlink, I., L. Ellinger, E. J. Bakker and E. A. Lantinga (2010). "Homeopathy as replacement to antibiotics in the case of Escherichia coli diarrhoea in neonatal piglets." Homeopathy 99(1): 57-62.
Caspi, O. and I. R. Bell (2004). "One size does not fit all: aptitude chi treatment interaction (ATI) as a conceptual framework for complementary and alternative medicine outcome research. Part II--research designs and their applications." Journal Of Alternative And Complementary Medicine (New York, N.Y.) 10(4): 698-705.
Colquhoun, D. and B. Isbell (2007). Credible endeavour or pseudoscience? Times Higher Education Supplement: 14-14.
Ernst, E. (2011). Complementary Medicine The evidence so far. Exeter, Peninsula Medical School.
Ernst, E. and M. H. Cohen (2002). "Informed consent in complementary and alternative medicine." American Journal of Ophthalmology 133(2): 300.
Evidence-Based Medicine Working Group (1992) Evidence-based medicine: A new approach to teaching the practice of medicine JAMA 268 (17): 2420-2425
Feyerabend, P. (1978). Science in a free society. London: New Left Books
Guyatt, GH. et al. (2002) Introduction: The Philosophy of Evidence-Based Medicine. Users’ guide to the medical literature: A manual for evidence-based practice Chicago: AMA Press
Hall, H. (2009). "Homeopathy: Still Crazy After All These Years." Skeptic 15(1): 8-9.
Jacobs, J., L. M. Jiménez, S. Malthouse, E. Chapman, D. Crothers, M. Masuk and W. B. Jonas (2000). "Homeopathic treatment of acute childhood diarrhea: results from a clinical trial in Nepal." The journal of alternative and complementary Medicine 6(2): 131-139.
Jacobs, J., W. B. Jonas, M. Jimenez-Perez and D. Crothers (2003). "Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials." The Pediatric infectious disease journal 22(3): 229-234.
Kim, L. S., J. E. Riedlinger, C. M. Baldwin, L. Hilli, S. V. Khalsa, S. A. Messer and R. F. Waters (2005). "Treatment of seasonal allergic rhinitis using homeopathic preparation of common allergens in the southwest region of the US: a randomized, controlled clinical trial." Annals of Pharmacotherapy 39(4): 617-624.
Kuhn TS (1970) The Structure of Scientific Revolutions. [2 ed].Chicago: The University of Chicago Press
Lorenz, I., E. Schneider, P. Stolz, A. Brack and J. Strube (2003). "Sensitive flow cytometric method to test basophil activation influenced by homeopathic histamine dilutions." Forschende Komplementärmedizin/Research in Complementary Medicine 10(6): 316-324.
Mathie, R. T., M. Van Wassenhoven, J. Jacobs, M. Oberbaum, J. Frye, R. K. Manchanda, H. Roniger, F. Dantas, L. A. Legg and J. Clausen (2016). "Systematic review and meta-analysis of randomised, placebo-controlled, trials of individualised homeopathic treatment." Homeopathy 1(105): 18-19.
Mathie, R. T., M. Van Wassenhoven, J. Jacobs, M. Oberbaum, H. Roniger, P. Fisher, J. Frye, R. K. Manchanda, L. Terzan, G. Chaufferin and F. Dantas (2014). "Model validity of randomised placebo-controlled trials of individualised homeopathic treatment." Homeopathy.
Miettinen, O. S. (2001). "The modern scientific physician: 1. Can practice be science?" CMAJ: Canadian Medical Association Journal = Journal De L'association Medicale Canadienne 165(4): 441-442.
Reilly, DT, Taylor MA, McSharry C, Aitchison T (1986) “Is homoeopathy a placebo response? Controlled trial of homoeopathic potency with pollen in hay fever as a model. Lancet ii: 881-886
Reilly, D.T., Taylor, M.A., Campbell, J., Beattie, N., McSharry, C., Aitchison, T., Carter, R., Stevenson, R. (1994) “Is evidence for homoeopathy reproducible?” Lancet 334: 1601-1606
Research!America. (2014). "US Investment in research 2012." Retrieved 30th October, 2014, from http://www.researchamerica.org/research_investment.
Rutten, L., R. T. Mathie, P. Fisher, M. Goossens and M. van Wassenhoven (2013). "Plausibility and evidence: the case of homeopathy." Medicine, Health Care and Philosophy 16(3): 525-532.
Sehon, S.R. & Stanley, D.E. (2003) “A philosophical analysis of the evidence-based medicine debate.” BMC Health Services Research 3:14-24
Shang, A., Huwiler-Müntener, K., Nartey, L., et al. (2005) “Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.” Lancet 366: 726-32
Singh, S. and Ernst, E, (2008). Trick or treatment? : alternative medicine on trial / by Simon Singh, Edzard Ernst, London : Bantam, 2008.
Smith, K. (2008). "Anomalous therapies and public health: a utilitarian bioethical response." Public Health Nursing 25(3): 269-277.
Spence, D.S., Thompson, E.A., Barron, S.J. (2005) “Homeopathic treatment for chronic disease: a 6-year, university-hospital outpatient observational study.” J Altern Complement Med 11: 793-798
Stambolovic, V. (1996). "Medical heresy--the view of a heretic." Social Science & Medicine (1982) 43(5): 601-604.
Taylor, M.A., Reilly, D., Llewellyn-Jones, R.H., McSharry, C., Aitchison, T., Lancaster, T. (2000) “Randomised controlled trial of homoeopathy versus placebo in perennial allergic rhinitis with overview of four trial series.” British Medical Journal 321:471-476
Ulbricht, C., W. Chao, A. Clark, D. Cook, T. Cormier, D. Costa, S. Culwell, F. Gombaut, R. Isaac, E. Rusie, S. Zhou, J. Conquer and E. Ernst (2011). "Oscillococcinum: An Evidence-Based Systematic Review by the Natural Standard Research Collaboration." Alternative & Complementary Therapies 17(1): 41-49.
Weatherley-Jones, E., Thompson, E.A., Thomas, K.J. (2004) “The placebo-controlled trial as a test of complementary and alternative medicine: observations from research experience of individualised homeopathic treatment.” Homeopathy 93:186–189.
Werner, C., A. Sobiraj and A. Sundrum (2010). "Efficacy of homeopathic and antibiotic treatment strategies in cases of mild and moderate bovine clinical mastitis." Journal of dairy research 77(4): 460.
