Reading Science
In 2015, a group of researchers tried to replicate 100 published psychology studies.
In 2015, a group of researchers tried to replicate 100 published psychology studies. Only 36 of them held up.
That's not a typo. Fewer than 40% of the studies, all of which had been peer-reviewed and published in respected journals, produced the same result when run a second time. The crisis isn't limited to psychology — similar replication failures have been documented in cancer research, economics, and neuroscience.
The takeaway isn't that science is broken. It's that science is messier than the headlines make it look, and the ability to read a paper critically — to tell the strong claims from the shaky ones — is one of the most valuable skills you can develop. This module teaches you how.
---
The Anatomy of a Scientific Paper
Almost every modern scientific paper follows the same structure. Once you know it, you can navigate any paper in any field, even ones outside your expertise.
Abstract — A 200-word summary of the entire paper. Read this first. It tells you what they did, what they found, and whether the paper is worth your time.
Introduction — The background context. What's already known about the topic? What's the unanswered question? What hypothesis is the paper testing? This is where you find the why.
Methods — How exactly the study was done. Sample sizes, equipment, protocols, statistical tests. This is the most boring section and the most important one. If the methods are weak, nothing else in the paper matters.
Results — What the data actually showed. Usually full of graphs, tables, and statistics. No interpretation here — just the findings.
Discussion — What the authors think the results mean. This is where interpretation lives, and where bias most often shows up. Be skeptical here.
References — The papers this work builds on. Useful for tracing where ideas came from.
Reading strategy for beginners: Abstract → Figures → Discussion → Methods (if you trust the discussion). You'll graduate to reading methods first eventually, but start here.
---
What Peer Review Actually Is (and Isn't)
When a paper is peer-reviewed, that means before publication, the journal sent it to 2–4 anonymous experts in the field who read it and gave feedback. They check for errors, evaluate the methods, and recommend whether the journal should publish it, revise it, or reject it.
Peer review is good. It catches major errors and bad science. It's the gold standard for separating real research from cranks.
But peer review is also not a guarantee of truth.
- Reviewers can miss errors, especially in complex statistics or specialized methods
- Reviewers don't usually replicate the experiments — they just read the paper
- Reviewers can have biases, conflicts of interest, or simply be wrong
- Some journals do "peer review" that's barely more than spell-checking — these are called predatory journals and you need to learn to spot them
- Even good peer-reviewed papers can later turn out to be wrong (see Andrew Wakefield, 1998 — peer-reviewed in The Lancet, retracted after fraud was exposed, but not before fueling the modern anti-vaccine movement)
The right framing: peer-reviewed papers are worth taking seriously. They are not worth taking as fact.
---
Spotting Bad Science
Here's a starter checklist for evaluating any study you encounter:
Sample size. How many subjects? A study with 12 people is suggestive at best. A study with 12,000 is much harder to dismiss. Small samples produce noisy data and can easily show effects that aren't real.
Replication. Has anyone else gotten the same result? A single study, no matter how exciting, is preliminary. The phrase "more research is needed" exists for a reason.
P-hacking. If a study measured 20 different variables and reported the one that came out "statistically significant," that's not science — that's fishing. You'll learn more about this in F5.
Funding sources. Who paid for the study? Industry-funded studies aren't automatically wrong, but they require extra scrutiny. A study on sugar's effects funded by the sugar industry deserves more skepticism than one funded by a neutral foundation.
Conflicts of interest. Real papers disclose financial ties at the end. Read them.
Cause vs. correlation. Did the study actually prove that X causes Y? Or just that X and Y tend to happen together? These are not the same thing — this is also covered in detail in F5.
The headline trap. News articles dramatically distort scientific findings. "Coffee linked to cancer" usually means a single study found a tiny correlation in a specific population. Always find the original paper before believing the headline.
---
Where to Find Good Papers
Most scientific papers live behind expensive paywalls. But you have access to more than you think.
PubMed (pubmed.ncbi.nlm.nih.gov) — Free database of biomedical research. Millions of papers indexed. Some are free full-text, some are abstract-only. Start here for anything biology, medicine, or biotech.
Google Scholar (scholar.google.com) — Searches across nearly all academic literature. Sometimes finds free PDF versions hosted on authors' personal sites.
bioRxiv and medRxiv — Preprint servers where researchers post papers before peer review. Useful for seeing the cutting edge, but treat with extra caution since these haven't been vetted yet.
arXiv — Same idea, but for physics, math, and computer science.
Your school library — If you're in a high school with college articulation, you may have free access to journals through your library system. Ask your librarian.
Sci-Hub — Exists, technically illegal in most jurisdictions. Not recommended, but you should know it exists because researchers in poorer institutions rely on it heavily.
A pro tip: if you can't access a paper, email the author. Researchers are usually thrilled when a student writes to ask for their work. Most will send you the PDF directly.
---
Wait, Actually...
Peer review as we know it is surprisingly new. The current system — where journals send manuscripts to outside experts before publishing — only became standard in the 1970s. Before that, most papers were reviewed only by the journal's editor, who made the publication decision personally.
Einstein's 1905 papers, including the one on special relativity, were never peer-reviewed in the modern sense. Neither was Watson and Crick's 1953 paper proposing the double helix. The editor read them, decided they were interesting, and published.
This doesn't mean those papers were unreviewed — scientific work has always been scrutinized by the broader community after publication. But the formal "send it to two anonymous experts first" model is recent enough that your grandparents' generation of scientists mostly didn't operate under it. Science still progressed.
---
Which section of a paper describes exactly how the experiment was conducted?
Which of the following is not a reliable sign of a strong study?
What is a preprint?
Why does funding source matter when evaluating a study?
Try This
Pick a science headline from any major news site published in the last week. Then:
- Find the original paper the article is based on. (The article will usually name the journal or the lead researcher — search PubMed or Google Scholar.)
- Read the abstract.
- Note three differences between what the headline claims and what the paper actually says.
You will find differences. Every time. This exercise, repeated even a few times, permanently changes how you read science news.
---