The systematic review, titled “Association of Industry Sponsorship With Outcomes of Nutrition Studies, A Systematic Review and Meta-analysis,” was conducted by three Australian researchers and published last week online by JAMA Internal Medicine. The review looked at 12 meta-analyses that in turn delved into hundreds of nutritional studies to determine if company-sponsored research was more likely to come out with positive results on the ingredients or interventions studied. The authors said their analysis suggested the answer is yes, but with a huge caveat.
No statistical significance
“Although industry-sponsored studies were more likely to have conclusions favorable to industry than non–industry-sponsored studies, the difference was not significant. There was also insufficient evidence to assess the quantitative effect of industry sponsorship on the results and quality of nutrition research. These findings suggest but do not establish that industry sponsorship of nutrition studies is associated with conclusions that favor the sponsors,” they wrote.
“It is interesting, but in a sense this review has not that much to do with the dietary supplement industry and has more to do with the state of science in general,” Duffy MacKay ND, senior vice president for scientific and regulatory affairs for the Council for Responsible Nutrition told NutraIngredients-USA.
“It’s one of those situations where you are in a lose/lose position. There is this pressure to do research, to add to the science behind the products, but then people are suspicious of your results,” he said.
The systematic review authors found that in a meta analysis on 70 probiotics studies, 21% of all clinical outcomes in industry sponsored research were favorable (more than one clinical outcome was looked at in each study), whereas 17% of outcomes were favorable in non-industry sponsored research. A paper that looked at calcium studies found no difference in bone health outcomes between the two types of research. Another review of studies on sugar sweetened beverages similarly found no association between industry and non-industry sponsored research, though in that case the studies were looking at evidence of harm as opposed to evidence of benefit as in most other nutritional research.
Expecting the result
MacKay said the authors were clearly expecting to find the result that industry funding induced bias into the picture.
“Industry sponsorship can influence research results in a variety of ways. Methodological quality is only one characteristic that can influence study outcomes. Sponsors can also frame research questions to produce a desirable outcome or to generate research that diverts attention from certain questions,” they wrote.
MacKay said it points to the issue of how often researchers of all stripes may be misled by trying to prove a hypothesis that might be beneficial to them personally, whether by boosting the support for a company’s product, or by raising their personal profile when they publish a provocative result.
“One bias that’s not discussed here is when academic researchers get committed to a hypothesis and they succumb to the pressure to publish a preliminary study that’s shocking and that might get a lot of press. We’ve seen that with some research on dietary ingredients that link them to harm,” MacKay said.
MacKay said that, surprisingly, despite their goal to find bias toward benefit with industry sponsored studies, the authors were fair and forthright enough to include the positive information they uncovered, too. The researchers noted that, “Our review found that industry-sponsored studies were of equal or better quality than those with other funding sources.”
Bias born of data-backed decisions
It stands to reason that a company that is developing an ingredient in a step-by-step fashion will understand its effects and if it is continuing to invest in that development, it has reason to believe in the ingredient’s potential benefits and it expects to get a positive result in the studies it funds. MacKay said the best, and really only defense against charges of potential bias in that research development is to do the best quality research the company can afford.
“Is there junk science out there? Of course there is. But if you have done some in vitro and in vivo work and some preliminary human trials, you are absolutely going to understand that ingredient better and you are going to more effectively design a trial using it than likely would some post doc who’s got an idea and buys some stuff off the shelf for a trial,” MacKay said.
“To avoid charges of bias, you have to do the research at a reputable university. You have to get it peer reviewed and published in a high quality journal. The peer review process is supposed to catch a lot of these problems,” he said.
MacKay said industry-funded research is a fact of life and probably will accelerate in the future. The alternative would a dearth of evidence in the sector, he said.
“The funding from NIH, from the Office of Dietary Supplements, is really limited. Industry sponsored research in the form of public/private partnerships really has to fill in the gaps,” he said.
Source: JAMA Internal Medicine
Published online October 31, 2016. doi:10.1001/jamainternmed.2016.6721
Authors: Nicholas Chatres, Alice Fabbri, Lisa A. Bero