Reading a study

Different designs answer questions with very different levels of confidence. Roughly from weakest to strongest:

• Anecdotes and testimonials — individual reports with no comparison. They can suggest questions but cannot answer them.
• Case reports and case series — doctors describing one or a few patients. Useful for spotting rare side effects.
• Observational studies — researchers watch what happens to people who chose a treatment. Prone to confounding: users may differ from non-users in many ways.
• Open-label and pilot trials — people knowingly receive the treatment, often without a control group.
• Randomised controlled trials — chance decides who gets the treatment, and a control group shows what would have happened anyway.
• Systematic reviews and meta-analyses — pool all relevant trials. Powerful when the underlying trials are good; misleading when they are not.

A large share of peptide research is preclinical: experiments in cells (in vitro) or animals (in vivo). These studies are essential for understanding mechanisms, but they routinely fail to predict results in people. Rodents differ from humans in metabolism, lifespan and healing; lab injuries are standardised in ways real injuries are not; and doses are often far higher than anything tested in humans.

Watch for papers or articles that describe animal findings in human terms. “Accelerates tendon healing” may mean “in rats with surgically cut tendons”.

Small studies are noisy. With 10 or 20 participants, chance alone can produce impressive-looking differences, and rare side effects will simply not show up. Short studies cannot reveal long-term effects. Look for the number of participants in each group, how many dropped out, and how long they were followed.

A “statistically significant” result means the difference is unlikely to be pure chance, not that it is large or meaningful. Ask how big the effect was, in units that matter.

Many conditions improve on their own, and expectation alone changes how people feel. A placebo group captures both. Randomisation makes groups comparable at the start; blinding stops participants and researchers from nudging the results, consciously or not. A trial that is “double-blind, randomised, placebo-controlled” has addressed the most common sources of bias.

Also check the outcome. Lab markers, such as a rise in a hormone level, are easier to move than outcomes people actually care about, such as fewer fractures or better function. A peptide that raises IGF-1 has not thereby been shown to build muscle or improve health.

Industry funding is normal and does not make a study wrong — almost every approved drug was tested in company-funded trials. But funding and authorship matter when weighing small or unreplicated studies. Look at the funding and disclosure sections: Do the authors hold patents on the compound? Did a single lab or company produce nearly all the published work? Has anyone independent reproduced the findings?

A preprint is a paper posted before peer review. Preprints speed up science but have not been checked by independent reviewers, and some change substantially or are never published. Peer review itself is a filter, not a guarantee: journals vary widely in rigour, and some “predatory” journals publish almost anything for a fee. Retracted papers should not be cited as evidence — PubMed marks them clearly.

1. Was it in humans, animals or cells?
2. How many participants, and for how long?
3. Was there a control group — and was it a placebo?
4. Were participants randomised and blinded?
5. Was the main outcome chosen before the study started?
6. Is the outcome something people feel (pain, function) or a lab marker?
7. Who funded it, and do the authors hold patents or sell the product?
8. Has it been peer-reviewed and independently replicated?

The evidence tiers on this site apply a simplified version of these questions to each entry.