While commenting favorably on the study’s size, almost 3,000 participants, the association said that authors made an overly broad and sweeping interpretation of the results. The authors stated that they found, “no evidence that a multi-strain preparation of lactobacilli and bifidobacteria was effective in prevention of AAD or CDD.” The IPA said this is inappropriate because of the large number of species and strains within the two genuses. In a statement from its Scientific Advisory Board, the IPA said: “The strains are named accurately in the article. However, in the conclusion they should have mentioned that the particular strains did not appear to work in this particular study.”
Formula has been studied before
The association also noted that the article does not mention that the product used in the study is commercially available. A previous article with the study protocol does provide such information (Allen et al. 2012). A previous study had shown that the same product reduced the incidence of CDD (Plummer et al. 2004) and appears to influence the microbiota composition when challenged by antibiotics (Plummer et al. 2005; Madden et al. 2005). IPA questioned why these contraindicating findings of the same product were not discussed in the most recent paper.
The association also cited numerous studies using different single and multi-strain probiotic combinations in a variety of delivery systems (e.g. supplement capsule, powder or yogurt) that also dispute the findings of Allen, et al. (2013) and show efficacy against AAD, as well as CDD (Hempel et al. 2012; Johnston et al. 2012; Rogers et al. 2013; Goldenberg et al. 2013).
IPA also rued the characterization of the study in some media outlets as ‘negative.’ The study is a ‘null’ study and not a ‘negative’ study as the accompanying commentary suggests (Daneman 2013). IPA said null studies are nothing new, citing a meta-analysis (Hempel et al. 2012) on the effect of probiotics on AAD that listed 17 ‘null’ studies out of a total of 63. IPA did admit that this most recent study was the largest of its kind to come up with a null result.
Study design questions
The size of the study – 17,420 patients were assessed for eligibility, and 2,981 were randomized – led to some practicality decisions that may have had an effect on the outcome, IPA said. Among these were:
- Some subjects were recruited despite the fact that they were already on antibiotic therapy for up to a week. As such, it is not clear as to how many subjects were already using antibiotics prior to receiving the study’s probiotic preparation. Most previous probiotic-AAD studies have commenced probiotic supplementation at the start of antibiotic treatment or prior to (e.g.Helicobactereradication therapy).
- Eligible patients could be using either oral or intravenous antibiotics; past studies have focused mainly on oral antibiotics or at least one route of administration. The recruitment of both oral and intravenous antibiotic users may have added to the variability
- The procedure also called for probiotic administration for a fixed time (three weeks.) Supplementation with probiotics in previous studies has usually lasted at least up to the end of the antibiotic treatment, but often longer. In the present study, one third of the subjects had antibiotic therapy for at least 14 days – did they receive probiotics up to the end of the antibiotic therapy?
Question of statistical power
IPA also noted that, while the study was large, its statistical power was not as great as the overall size might imply because of the lower than expected incidence of AAD and CDD was 20% and 4% respectively, in the placebo group. But in reality, it was 10.6% and 0.99% respectively. The result, according to IPA, is that the study was “grossly underpowered.” Any positive effect the probiotics might have had would be lost in statistical analysis, the association said.
In conclusion, IPA said that the results of the study could have been better articulated with a tighter scope on the study’s findings. According to IPA, a better way to think about this study is that it “did not reproduce the same statistically significant, positive effect seen previously for the same microbial preparation for the prevention of AAD and CDD in the elderly.”