Our Method
We define misinformation as claims which are unlikely to be true compared to competing claims, based on relevant expert knowledge and empirical evidence. Unfortunately, misinformation is crippling the field of nutrition and has the potential to cause serious harm.
How It Works
Our review method is designed to answer one primary question: should the claim be considered as misinformation?
To help answer the primary question, our team identifies and analyses the most relevant and robust studies. When interpreting the research, we’re also mindful of numerous sub-questions that provide critical information when answering the main question, such as:
- Is the claim, or justification for the claim, based on violations of logic or reason?
- What type of evidence would (ideally) offer the strongest support for the claim?
- What is the primary form of evidence that is actually being used to support the claim, and how robust is this type of evidence?
- Is there a sufficient amount of evidence to confidently assess the truthfulness of the claim?
- Does the bulk of strong evidence support the claim?
- Are there any other indications or considerations that the claim is true or false?
- Would a qualification(s) or context improve the precision of the claim?
- Is there ongoing debate among topic experts on the truthfulness of the claim? If so, what scientific nuances are still contentious?
If we conclude that the claim is misleading or unlikely to be true compared to a competing claim, we publish (up to) a 1000-word overview of the claim based on our interpretation of the scientific literature. Additionally, considering that misinformation can be crafted in different ways—ranging from simply the alteration of information to blatant lies—we categorize claims into one of three levels of misinformation: 1 red flag (minor misinformation); 2 red flags (clear misinformation); 3 red flags (gross misinformation).
Levels of Misinformation:
1 Red Flag (Minor Misinformation)
There is an element of truth to the claim but it requires qualification and/or context to align with current evidence. The claimant may be overgeneralising the claim or be unaware of the context in which it applies.
OR
Current evidence supports the claim but is intrinsically unconvincing. There is considerable uncertainty about the truthfulness of the claim and stronger evidence is needed. The claimant may not understand that some forms of evidence only weakly support a claim.
2 Red Flags (Clear Misinformation)
There is little evidence supporting the claim and/or the bulk of strong scientific evidence refutes the claim. The claimant may be unaware of the totality of evidence, or doesn’t understand that different types of evidence and their convergence offer varying degrees of support that a claim is true.
OR
A primary justification for the claim relies strongly on a violation(s) of logic and/or affirms a contradiction which does not render the claim as systematically invalid, but makes it unlikely to be true in the absence of supporting evidence.
OR
No line of evidence informs us of the claims truthfulness, even weakly. Given the absence of evidence, the origins of the claim are unknown. The claimant may have entered the realm of fiction and/or be unfamiliar with hypothesis testing.
3 Red Flags (Gross Misinformation)
The claim has been consistently and convincingly refuted by converging lines of evidence; most experts are confident that the claim is wrong. No qualifications or context can justify the claim. The claimant must be unaware of the relevant literature (or lack thereof) or is, consciously or otherwise, manipulating and fabricating the evidence to support a pre-accepted hypothesis.
OR
The claim relies entirely on a violation of logic and/or affirms a contradiction that renders any justification for the claim as invalid and incoherent. No supporting evidence can justify the claim.
Frequently Asked Questions
Why should our reviews be trusted?
Unlike a lot of people that communicate nutrition information, every member of our team has a comprehensive understanding of fundamental nutrition principles, scientific method, statistical analysis, causal inference, and epistemic logic. We do not guarantee perfect scientific accuracy or the absence of bias; however, we strive to publish reviews that are high-quality, professional, fair and transparent. We encourage people to seek other credible sources of information, especially those we recommend on our resource page. Before publishing misinformation, each write up is peer-reviewed by another member of our team and edits are suggested. If we’re not confident about the truthfulness of a claim, we either delay publication of the review or we reach out to a topic expert for discussion and feedback. We’re also more than open to hear feedback from our readers and we encourage you to contact us with any queries or insights that may improve our work.
What do we consider to be relevant and strong evidence?
Relevant evidence includes peer-reviewed scientific journal articles and regulated publications by government, and neutral non-governmental organisations. Other credible sources are accepted if good reasoning is provided. Unless intrinsic to the claim, personal observations are only considered as evidence (albeit weak) when complimented by other relevant evidence.
Regarding the strength of evidence, we take the position that evidence hierarchies are fluid and determined almost entirely by the claim. There are as many questions that are best answered by one form of evidence than are best answered by another, such that no one study type should sit at the top of a rigid evidence hierarchy. For this reason, we are upfront in our reviews about what type of evidence would (ideally) offer the strongest support for the claim. However, we are also firm believers that for claims of a causal nature to be strongly supported, which are often the claims that attract the most debate, there is a requirement of evidence from prospective studies—specifically, randomised trials and prospective cohort studies with a low probability of bias (including confounding) to explain the effect or lack thereof.