Veterinary homeopathy: Systematic review of medical conditions studied by randomised trials controlled by other than placebo

Background No systematic review has previously been carried out on randomised controlled trials (RCTs) of veterinary homeopathy in which the control group was an intervention other than placebo (OTP). For eligible peer-reviewed RCTs, the objectives of this study were to assess the risk of bias (RoB) and to quantify the effect size of homeopathic intervention compared with an active comparator or with no treatment. Methods Our systematic review approach complied fully with the PRISMA 2009 Checklist. Cochrane methods were applied to assess RoB and to derive effect size using standard meta-analysis methods. Based on a thorough and systematic literature search, the following key attributes of the published research were distinguished: individualised homeopathy (n = 1 RCT)/non-individualised homeopathy (n = 19); treatment (n = 14)/prophylaxis (n = 6); active controls (n = 18)/untreated controls (n = 2). The trials were highly diverse, representing 12 different medical conditions in 6 different species. Results No trial had sufficiently low RoB to be judged as reliable evidence: 16 of the 20 RCTs had high RoB; the remaining four had uncertain RoB in several domains of assessment. For three trials with uncertain RoB and without overt vested interest, it was inconclusive whether homeopathy combined with conventional intervention was more or was less effective than conventional intervention alone for modulation of immune response in calves, or in the prophylaxis of cattle tick or of diarrhoea in piglets. Conclusion Due to the poor reliability of their data, OTP-controlled trials do not currently provide useful insight into the effectiveness of homeopathy in animals. Electronic supplementary material The online version of this article (doi:10.1186/s12917-015-0542-2) contains supplementary material, which is available to authorized users.


Background
Our group's systematic analysis of the published literature of randomised controlled trials (RCTs) in veterinary homeopathy identified 38 peer-reviewed papers that we regarded as potentially eligible for detailed review [1]. As emphasised in that article, no systematic review of this research evidence had ever previously been carried out.
We have recently reported our review findings for 18 placebo-controlled trials of veterinary homeopathy [2,3], in which there was indecisive evidence whether the use of homeopathy in animals is distinguishable from placebos.
The present paper reports our results from a similarly detailed appraisal and analysis of the remaining papers, each of which reported an RCT where the control group was an intervention other than placebo (OTP).
Our approach here reflects our original literature analysis [1] and so we continue to distinguish peer-reviewed from non-peer-reviewed articles, individualised from non-individualised homeopathy, and treatment from prophylaxis. We thus report our findings from the appraisal of peer-reviewed, OTP-controlled trials of veterinary homeopathy (individualised or non-individualised, treatment or prophylaxis).
Our objective firstly was to assess the study quality (risk of bias) of each eligible RCT [4], together with the direction and statistical significance of treatment or prophylactic effect. For suitable groupings of RCTs (per species-specific medical condition; per type of control group), we aimed then to determine pooled summary statistics by meta-analysis methods and to examine their sensitivity to study quality. In both objectives, our emphasis was focused on trials that satisfied our criteria for reliable evidence [2,3].

Methods
Methods comply with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2009 Checklist (see Additional file 1) [5]. They are compatible with our previously published papers in this series of veterinary reviews [1][2][3], and with our protocol-based systematic reviews and meta-analyses of homeopathy RCTs in human medicine [6,7].
Matters connected with study eligibility, research design categories and the literature search strategy were described in detail in our earlier paper [1]. Only brief descriptions are therefore given here, with additional information that is specific to the methods used for the present paper.

Identifying papers for full data extraction
Each of the following electronic databases was searched up to and including March 2011 [1], with a follow-up search of the same databases up to the end of 2013: AMED, CINAHL, CENTRAL, Embase, HomVetCR, LILACS, PubMed, Science Citation Index, Scopus. There were no language restrictions. During the peer-review process for the current paper, we became aware that relevant additional literature might be contained in CAB Abstracts [8], which we therefore searched up to and including May 2015, and with the intention to update our review and analysis if significant numbers of eligible RCTs were revealed.
In our original literature search [1], 20 records of trials were identified as satisfying the key acceptance criteria for the present systematic review: substantive report of clinical treatment or prophylaxis trial for any medical condition or species in veterinary homeopathic medicine, randomised, controlled by OTP, and published in a peerreviewed journal. One of us screened and categorised each of these potentially relevant papers (plus those identified in our follow-up searchsee Results) to assess their eligibility for full data extraction. The other independently appraised these decisions; any differences of opinion were resolved by consensus discussion.

Exclusion criteria prior to full data extraction
Research using radionically prepared 'homeopathic' medicines. The tested intervention is homeopathy combined with other (complementary or conventional) medicine or therapy.

Data extraction and management
The authors of eligible RCT papers were not approached for clarification on unclear or missing facets of their methods or results [4], though original authors' crossreference to their previously published study methods were taken into account as appropriate [2]. For each of two assessors working independently, relevant data were extracted and then recorded using a standardised data collection format (Microsoft Excel).
No paper reported more than one RCT. For a paper reporting an RCT that involved >2 groups of subjects, we typically pursued data extraction on only one pair of groups: trials that included an OTP as well as a placebo control group had been scrutinised previously [2,3] and were not reappraised for the current study. For studies that comprised >1 homeopathy group, the total sample size reflects the total numbers of subjects in the homeopathy groups combined [9]. This was the approach in all relevant cases, i.e.: where the same homeopathic medicine was used, and with the same timing of administration but in different potency; where the same homeopathic medicine and potency was used, but with different timing of administration; where a different homeopathic medicine was used.

Assessment of risk of bias
Using the standard criteria defined by Cochrane [4], extraction of information from each paper enabled us to answer the question, 'Is the study free from risk of bias?': 'Yes'; 'Unclear'; 'No'. The two assessors' independent judgments were mutually scrutinised and compared, with discrepancies between them resolved by consensus discussion. This approach applied to each of seven assessment domains: I, the method used to generate the random sequence; II, the method of allocation concealment used to implement the random sequence; IIIA, the blinding of trial personnel, including animal owner as appropriate; IIIB, the blinding of outcome assessors; IV, whether all the randomised patients are accounted for in the analysis; V, whether there is evidence of selective outcome reporting; VI, whether there is evidence of other bias, such as extreme data imbalance at baseline.
For domain IV, a trial was automatically regarded as no better than 'unclear' if there was greater than 20 % participant attrition rate, irrespective of whether intention-totreat analysis had been carried out on the data. For domain V, we based our judgment of reported outcomes on a comparison with the details given in the same paper's Methods section (original trial protocols have not been published in veterinary homeopathy). For the purposes of the current paper, we assessed domain V as 'No' (high risk of bias) if the main outcome data were not extractable for metaanalysis. The source of any research funding/sponsorship, or other vested interest such as personnel employment or contract, was not taken into account for risk-of-bias assessment (domain VI), but was reflected in the overarching assessment of risk of bias for each RCT.

Assessment per trial for risk of bias
Using the Cochrane approach [4], each trial was designated overall as follows: low risk of bias for all domains; uncertain risk of bias for one or more domains, and no evident risk of bias in any domain; high risk of bias for one or more domains. A trial with overall low risk of bias comprised reliable evidence. For a trial that did not display high risk of bias in any domain, we regarded its evidence as reliable if the study was assessed as free of bias for each of domains I, IIIA, IIIB and IV [2,3].
Outcome assessment and reporting Identification of 'main outcome measure' per trial For the purposes of risk-of-bias assessment and for assessment of treatment effect, we identified for each trial a single 'main outcome measure' using a refinement of the approaches adopted by Linde et al. [10] and by Shang et al. [11]. The main outcome measure of each trial was based on a hierarchical ranking order (consistent with the World Health Organization's [12] classification system for levels of functioning linked to health condition), and as previously described [2]. This approach ensured that the most clinically important outcome was selected per trial, and also avoided the problem of outcome multiplicity [13].
Unless otherwise indicated, the single end-point (as determined from the start of the intervention) associated with the designated 'main outcome measure' was taken as the last follow-up at which data were reported for that outcome [2].

Analysis of outcomes Summary effect measures for 'main outcome'
For each eligible trial, the effect size was taken as the difference between the homeopathy group and the control group at the designated end-point of the trial, as follows [2]: For a dichotomous measure: odds ratio (OR), with 95 % confidence interval (CI); For a continuous measure: standardised mean difference (SMD), with 95 % CI.
If the original paper did not provide adequate information on the designated main outcome measure to enable data extraction, that trial's outcome was classified as 'not estimable' and a further potentially estimable outcome was not sought.
Under the separate group headings of individualised homeopathy and non-individualised homeopathy, and for the categories below in which there was >1 RCT of a given study design that had extractable data, we aimed to determine pooled summary statistics for: Disease-specific treatment effects per species; Disease-specific prophylactic effects per species.
All calculations and analyses were performed using Review Manager 5.2 (Cochrane). Given our anticipation of heterogeneous data for intervention effects, the random-effects (rather than fixed-effects) model was planned for all meta-analyses [14].

Reflecting study quality overall
Our main analyses used only the data extracted from trials without evident high risk of bias. It was intended that our primary conclusions would be based solely on trials with reliable evidence and without overt vested interest (not funded, directly or indirectly, by a homeopathic pharmacy).

Direction of effect of treatment/prophylaxis per trial
Care was taken faithfully to represent the correct direction of change per trial: e.g. an effect favouring homeopathic intervention was higher rate of recovery but lower faecal egg count. The arithmetic content of the OR, or the sign of the SMD, was adjusted appropriately if so indicated.
We regarded each RCT as a superiority trial [15] unless the original paper stated explicitly in its Methods section that the study was designed as an equivalence or non-inferiority trial [16,17], with corresponding power calculation. For each of three study designs (and regarding each design as a superiority trial), we adopted the following rationale for interpretation of the three possible statistical findings: Statistical finding (i): P ≤ 0.05: Direction of effect toward homeopathy a. Study design: active control. Homeopathy is more effective than a conventional intervention; b.

Demographic details
Additional file 2 illustrates the PRISMA flowchart, which follows on from, and suitably updates [20], that in our earlier paper [1]; our current focus is on OTPcontrolled trials only. Two OTP-controlled trials were rejected from the current review due to their focus on homeopathy combined with another intervention: A29, Dreissman 2010; A30, Lepple 1984. A further two were excluded due to their focus on neither treatment nor prophylaxis: A33, Sharma 1987; A37, Trehan 1994. Four additional eligible records were identified in our follow-up search, and included in analysis. Our subsequent search of CAB Abstracts, conducted during peer review of the current paper, revealed just one potentially eligible trial [21], which we did not consider further. Table 1 details each of the 20 studies eligible for full systematic review: (i) individualised homeopathy/treatment (n = 1); (ii) non-individualised homeopathy/treatment (n = 13); (iii) non-individualised homeopathy/ prophylaxis (n = 6). Data per trial include: nature of the homeopathic intervention; study setting; the RCT's source of funding. Our definitions of 'treatment' and 'prophylaxis' were as previously described [1]. All RCTs were identified as superiority trials, despite the fact that in some cases the original authors applied the terms 'equivalence trial' or 'non-inferiority trial' post-hoc in Results or in Discussion (A15, Faulstich 2006; A39, Braun, 2011).
Extreme diversity characterised the studies as regards species, medical condition, homeopathic medicine, and funding source. In the 20 eligible studies, 6 different species are represented: birds (n = 1); cattle (n = 11); dogs (n = 1); horses (n = 1); pigs (n = 4); sheep (n = 2). Twelve different conditions are represented. None of the trials had a clearly unbiased funding source; we identified overt vested interest in 10 of the 20 trials. Table 2 includes: sample sizes; designated 'main outcome measure'; type of data (whether dichotomous or continuous); study endpoint. Diversity was again apparent, with large variation in sample sizes, main outcome, and timing of the study endpoint. Table 3 shows our risk-of-bias judgments for each eligible trial.

Risk of bias
Many of the papers were written to such a poor standard that risk-of-bias assessments were sometimes challenging. No trial had low risk of bias in all of the Cochrane judgmental domains. Sixteen trials had high risk of bias in one or more domains. Four trials had merely uncertain risk of bias, and the uncertainty was evident in five domains per study (A31, Reis 2006; A32, Reis 2008; A34, Signoretti 2008; A36, Soto 2008). There was therefore no trial that could be designated as reliable evidence.
Only five of the 20 trials had adequate randomisation (domain I), and none of the trials had adequate allocation concealment (domain II) or personnel blinding (domain IIIA). See also Fig. 1 (Risk-of-bias summary graph).

Analysis of outcomes
Fifteen of the 20 trials had extractable data. Due to the diversity of medical conditions, species, types of homeopathic intervention, study designs and outcome measures, as well as the very unclear quality of the evidence, it was not appropriate to carry out meta-analysis on disease-specific intervention.
The direction of effect favoured homeopathy in 11 trials (statistically significantly so in four cases) and favoured control in four trials (statistically significantly so in zero cases)see Table 4, in which the inference from the statistical findings is given by reference to the numbered study designs in Methods: Direction of effect of treatment/prophylaxis per trial.

Trials with uncertain risk of bias
In the absence of any trials with reliable evidence, our main attention has been forced on to the data extracted from three trials with uncertain risk of bias and without overt vested interest (Table 4a)    The number refers to the number of successive serial dilutions to which the starting material has been subjected. The letter refers to the scale on which the dilution has been carried out: the letter D denotes the decimal method of dilution (that is, one part of liquid is added to nine parts of purified water, ethanol, glycerol or lactose); the letter C indicates the centesimal method (one part added to 99 parts of diluent). In homeopathic dilutions above 12C/D24 (10 −24 molar)beyond Avogadro's constant, 6.02 x 10 23 mol −1there are, in theory, no material traces of the original substance; such dilutions are known as 'ultra-molecular'   Figure 1A. N not stated for endpoint, but assumed to be N = 7.
68 d Disparity between data (mean eggs per gram) for conv med group given in text (1,483) and by inspection of Fig 1  (1,300     and so in each case it was statistically inconclusive whether homeopathy combined with conventional intervention was more or was less effective than conventional intervention alonesee Table 4a.

Trials at high risk of bias
Data could not be extracted from the single paper on individualised treatment. Data were extractable from 10 of the 13 relevant RCTs of non-individualised treatment; the direction of effect statistically significantly favoured homeopathy in four of the ten casessee

Discussion
This comprehensive systematic review of the peerreviewed literature has revealed no OTP-controlled RCT of veterinary homeopathy that comprised reliable evidence. Our main conclusions from this study are therefore restricted to the findings of three RCTs, whose quality was unclear and whose methodological diversity contraindicated meta-analysis. Each of these three trials is in the category of non-individualised homeopathic prophylaxis (A32, Reis 2008; A34, Signoretti 2008; A36, Soto 2008), a prescribing approach that some homeopathic practitioners might view as controversial [22]. Each of the three trials is also in the adjunctive category, and manifested statistically inconclusive findings: this result undermines the suggestion that the '[A + B] versus B' study design can 'generate only positive results' [23]. For each of the above three trials, the uncertainty regarding risk of bias was so marked that their findings must be viewed with extreme caution. Given the lack of clarity in so many domains of assessment, this degree of concern would still apply even if we had reflected more leniently the fact that non-blinding of study personnel is a typical feature of an OTP-controlled trial. The admissible evidence in veterinary homeopathy for this study design is therefore unable to provide any compelling information about modulation of immune response in calves (A32, Reis 2008) or adjunctive prophylaxis of cattle tick (A34, Signoretti 2008) or of diarrhoea in piglets (A36, Soto 2008).
In assessing all the studies, we had reason on occasions to question the veracity of the control group as an 'active' or effective intervention (e.g. mineral or protein supplements, or naloxone in pseudopregnancy), and so raised doubts about the 'other than placebo' nature of some of the included trials. We were careful not to attribute 'equivalence' to superiority trials that found no statistically significant difference between homeopathy and active control [16,19].
For OTP-controlled RCTs of non-individualised homeopathic treatment, there is nothing that we can reasonably conclude, given that each one of the trials was judged to be at high risk of bias. Thus, the positive inference we attributed to four trials in this category cannot be regarded as admissible evidence; the spuriously large treatment effect evidenced in two of the trials reinforces those serious concerns. It remains unknown whether non-individualised homeopathy is effective in the prophylaxis of fertility disorders in cattle. Likewise, no conclusions can be made regarding individualised homeopathic treatment, since the single trial in this category was of very low quality and yielded no extractable data for analysis.
Our most recent literature search, conducted during the process of the current paper's peer-review, revealed    only one further OTP-controlled trial that would potentially have been eligible for inclusion [21]. Notwithstanding normal concerns over the possibility that publication bias may have limited the number of 'negative' studies available, the present systematic review seems barely compromised by selection bias due to absence of relevant data. Thus, the sum of the reliable peer-reviewed RCT evidence in veterinary homeopathy (out of 18 placebocontrolled and 20 OTP-controlled RCTs in total) comprises the two placebo-controlled trials that we previously identified [2,3]. Evidence concerning the effective use of homeopathy in animals remains indecisive.