Scientific Review Process

The Governing Rule

One rule governs this entire process: conclusions must be proportional to the quality of the available evidence. A finding supported by replicated randomized trials may be stated with confidence. A finding supported by one small study must be described as preliminary. A plausible mechanism that has never been tested in people must be labeled a hypothesis. When the evidence is insufficient to conclude anything, the published sentence says so. Everything below exists to enforce that rule.

Step 1: Identifying Every Scientific Claim

Before review, a draft is read once for a single purpose: listing every statement that asserts something about the physical world. "Supports joint comfort," "improves recall," "generally well tolerated," "works within weeks": each of these is a claim that requires support, however casually it is phrased. Claims hide easily inside adjectives and transitions, which is why this pass is done deliberately rather than trusted to happen naturally during writing.

Step 2: Separating Facts From Marketing

Each identified claim is sorted into one of four categories, and the finished article must keep them visibly distinct: established evidence (independent, replicated findings), preliminary evidence (early or small studies awaiting confirmation), theoretical mechanisms (biologically plausible but clinically untested ideas), and marketing claims (assertions whose only source is the seller). Marketing frequently blends these categories into one persuasive paragraph; our job is to unblend them.

Step 3: Evaluating Study Quality

Cited studies are read for design before they are read for results. Was the study randomized and controlled, or observational? Was it blinded? Was the population studied similar to the readers who would use the product? Was the tested substance, dose, and duration comparable to what the product actually delivers? A trial of a single ingredient at triple the product's dose is evidence about that ingredient, not about the product, and our articles are required to say so explicitly.

Assessing Clinical Relevance

A study can be well designed and still answer the wrong question. Many trials measure surrogate endpoints, a biomarker, a laboratory score, an inflammation marker, rather than outcomes people actually feel, such as pain, sleep, recall, or daily function. Our review asks whether the measured outcome is one a reader would notice, whether the effect lasted beyond the study period, and whether the benefit applies to the people likely to buy the product rather than the narrower group studied. When a claim rests on surrogate outcomes, the article says so instead of translating a lab value into a life change.

Sample Size Considerations

Small studies are common in the categories we cover, and small samples produce unstable results: effects appear and vanish between trials, and impressive percentages can rest on a handful of participants. We report sample sizes when they are material, treat findings from small trials as preliminary regardless of how positive they are, and give more weight to bodies of evidence than to any single result. A pattern across many small studies means more than one dramatic outlier.

Risk of Bias

Bias is assessed, not assumed away. We note who funded a study, because industry-sponsored trials of a sponsor's own product warrant extra caution even when they are well conducted. We consider publication bias, since null results are published less often than positive ones, which tilts entire literatures toward optimism. And we watch for selection effects in observational research, where healthier people often choose the behavior being studied. None of these automatically disqualifies a finding; each one adjusts how much weight the finding can carry.

Statistical Significance vs. Clinical Significance

A statistically significant result answers a narrow question: the effect probably was not chance. It does not answer the question readers care about: is the effect large enough to notice in daily life? A measurable change on a laboratory scale can be real and still meaningless to the person taking the product. Our reviews distinguish the two, and when a study reports an effect that is statistically real but clinically small, the published language reflects the small part, not just the real part.

Replication of Findings

A single study, however well designed, is a data point rather than an answer. Findings gain standing with us when independent teams reproduce them in different populations. Where a claim rests on one unreplicated trial, our articles identify it as such, and where replication attempts have failed, that failure outweighs the original result. The history of consumer health products is full of first studies that were never confirmed, which is why "one study shows" is a phrase we treat as a caution label.

Scientific Uncertainty

Uncertainty is information, and we publish it rather than smooth it over. Phrases like "results remain inconsistent," "the literature is still limited," and "the available evidence is insufficient to conclude" appear in our articles by design. We consider a reader better served by an honest "unknown" than by false confidence in either direction, positive or negative. Certainty is only expressed where overwhelming, well-replicated consensus supports it.

Reviewing Limitations

Every body of evidence has limitations: durations too short to reveal long-term effects, populations too narrow to generalize, endpoints that measure markers rather than outcomes. The review checks that material limitations appear in the article itself, not just in our notes. A review that reports a study's conclusion while omitting the limitation that qualifies it has misinformed the reader, even though nothing in it was false.

Language Calibration

The final substantive pass revises language so no sentence claims more certainty than its evidence carries. Verbs are matched to evidence strength: established findings "show," preliminary findings "suggest," mechanisms "may" or "could," and marketing "claims." Absolute terms are removed unless the consensus is overwhelming. Promotional vocabulary is removed entirely. The working test for each conclusion is simple: would a careful, skeptical scientific reviewer sign this sentence? If the answer is "maybe not," the sentence is rewritten more conservatively.

Final Editorial Review Before Publication

Before publication, the full article is audited once more against a fixed checklist: overconfidence in either direction, positive or negative bias, commercial framing, unsupported generalizations, extrapolations beyond what the studies tested, and implicit promises. Citations are re-checked to confirm each one supports the exact claim it is attached to, and every link is verified. Only after that audit does an article go live, and the same standard applies again whenever the article is updated. The sources that feed this process are described in our Research & Citation Standards, and the values behind it are summarized in Our Review Principles.