Blackmores Institute questions validity of high-profile Cochrane Review of vitamin E and mortality

The institute is the research arm of Australian vitamin manufacturer Blackmores.

Its researcher, Christopher Oliver, along with Southern Cross University’s Stephen Myers, argue that the Cochrane Database Systematic Review of 2012 by Bjelakovic et al​, which concluded that α-tocopherol forms of vitamin E have a statistically significant effect on mortality, had “several methodological ​issues”.

These criticisms have been rejected by the authors of the original review, who called into question the scientific evidence behind the “multi-billion dollar” ​supplement industry.

In the new paper, published in the journal BMC Complementary and Alternative Medicine, ​Oliver and Myers drew particular attention to the significance given in the Cochrane Review to the Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study (ATBC), a factorial design study of Finnish smokers.

Analytical method

They stated this was a pivotal paper in the review, owing to the high mortality rate observed, which resulted in a substantial weighting (42.6%) in the meta-analysis.

“The Cochrane meta-analysis used a three-cell analytical method comparing all vitamin E cells (vitamin E alone plus vitamin E plus beta-carotene) to the placebo-only cell. This had the unfortunate effect of incorrectly inflating the mortality risk attributed to vitamin E by not balancing the contribution to mortality of the beta-carotene intervention,” ​they wrote.

“Re-analysis of the ATBC trial using data derived from the more generally accepted ‘inside the table’ (two-cell — vitamin E versus placebo) or ‘at the margins’ (four-cell all vitamin E versus all non-vitamin E) analytical methods demonstrates a statistically non-significant result.”​

The impartiality of excluding trials with zero mortality was also queried.

In the Cochrane meta-analysis, it was stated that 481 antioxidant trials (involving approximately 42,000 persons) in which there was zero mortality in both the experimental and control groups were found.

However, these trials were excluded from the meta-analysis because “the influence of zero events trials on the final result was not noticeable”.​

Major failing?

Oliver and Myers argue that the “exclusion of low mortality data that implicitly illustrate safety is a major failing that risks substantial bias and undermines the conclusions of the current meta-analysis.​

“Current meta-analysis models are designed to assess efficacy where zero outcome in a clinical trial generally equals a poor response, while in the assessment of safety, a zero outcome is the preferred response,”​ they added.

Blackmores Institue director Dr Lesley Braun concurred, saying: “Studies on safety showing zero deaths are actually positive for safety and are essential to be considered in safety reviews. Safety requires a very different interpretation to efficacy trials, so the research methods should reflect this.”​

The paper concluded that the issues raised around data and methodology were not unique to the Cochrane review.

“This paper raises concerns overall as to the appropriateness of datasets chosen, the validity of methods, and generalisability of results when using meta-analysis as a tool to determine safety,”​ said Oliver and Myers.

“Issues raised in this paper are not unique to the Cochrane study in question. Until we have new tools, there may be a need to rely on conventional narrative systematic literature synthesis in the assessment of safety, or contain our results to specific sub-populations where more conclusive results can be determined.”​

Researchers respond

In response to the paper, Bjelakovic et al.​ issued a staunch defence of their work, also published in BMC​ Complementary and Alternative Medicine.​

“We believe we used the best available approach to analyse beneficial and harmful effects of vitamin E. Furthermore, we assessed whether the different doses of beta-carotene, vitamin A, and vitamin E affect mortality in primary and secondary prevention randomised clinical trials at low risk of bias,”​ they wrote.

“We found that beta-carotene and vitamin E in doses higher than the RDA seem to significantly increase mortality.”​

They added they had addressed the issues around “trials with zero mortality extensively in our review​”, and concluded by criticising the “multi-billion dollar” ​supplement industry.

“We are aware that pharmaceutical companies are influenced by our findings,”​ they wrote.

“Antioxidants, together with other food supplements, have turned into multi-billion dollar markets without much evidence supporting their use.​

“If harmful effects of antioxidant supplements stand to the test of science and time, then we may end up realising that we have also been fooled into believing ‘the more antioxidant supplements, the merrier’.”​

Source: BMC​ Complementary and Alternative Medicine.​

https://doi.org/10.1186/s12906-017-1906-x​

"Validity of a Cochrane Systematic Review and meta-analysis for determining the safety of vitamin E."​

Author: Christopher Oliver, et al.​