Why Most Published Research Findings Are False

The mathematical core is valid. If you accept the inputs — how often hypotheses are true, how powerful the studies are, how much bias operates — the formula correctly tells you how often positive findings will be wrong. The six corollaries all follow logically from the model.

The problem is the bridge between the model and the headline. "Most published research findings are false" is a claim about the actual world. To get from the model to the world, you need to know what fraction of research operates in the parameter ranges where findings are mostly false — and the paper provides no empirical evidence for this. The simulation table shows various scenarios, but the values plugged in are the author's own estimates. Modestly different but equally plausible values could reach substantially less alarming conclusions.

The bias parameter is particularly problematic. The paper assigns bias values of 10% to 80% for different research types, but these are stipulated, not measured. Since the model's outputs are highly sensitive to this parameter, the numerical conclusions inherit whatever accuracy or inaccuracy these guesses carry.

The paper also doesn't address its strongest counterargument: researchers don't test hypotheses at random. A geneticist studying a candidate gene identified through decades of functional biology draws from a very different hypothesis pool than a genome-wide scan testing 100,000 variants. If most published findings come from research where prior evidence has already filtered out implausible hypotheses, the effective odds could be substantially better than assumed.

The formula is precise: given its inputs, it outputs an exact probability. But the two inputs doing the most work — pre-study odds and bias — are defined in ways that make them practically unmeasurable.

Pre-study odds (the ratio of true to false hypotheses in a field) requires knowing how many true relationships exist before doing the research — which is what the research is supposed to discover. The paper treats this as a definite quantity while acknowledging it must be guessed. The bias parameter collapses wildly different phenomena — dropping outliers, switching outcomes after seeing results, running twenty subgroup analyses — into a single number, with no empirical basis for the values chosen.

The framework also treats "true relationship" as binary, but most biological relationships exist on a continuum of effect sizes. Whether a genetic variant that raises disease risk by a factor of 1.001 counts as a "true relationship" is a judgment call the model treats as settled.

These definitional issues mean the framework does useful qualitative work — it correctly identifies the structural factors that make findings unreliable — but its apparent quantitative precision (85% for a good trial, 0.1% for discovery-oriented research) masks that these numbers are outputs of guesses.

The core insight is genuinely important: the reliability of a finding depends not just on the study itself but on how likely the hypothesis was to be true before anyone tested it. This is a direct application of Bayes' theorem and is mathematically unimpeachable.

The problem is that the paper presents conditional predictions as established facts. "It can be proven that most claimed research findings are false" is doing rhetorical work the math does not support. What the model proves is that if pre-study odds, power, and bias take certain values, then most findings will be false. Whether those values obtain is an empirical question the paper never answers.

The simulation table illustrates this gap. It shows what the model predicts under chosen parameter values — not measurements of actual false-positive rates. But the paper presents them as descriptions of reality rather than sensitivity analyses.

The paper also doesn't engage with science's self-correcting mechanisms — replication, meta-analysis, theoretical integration. The model evaluates isolated findings at publication, ignoring the cumulative process by which false findings are identified and corrected.

The mathematical framework is fully explicit and independently verifiable. The premises are clearly stated, and each step from assumptions to conclusions can be traced. The paper calls its simulation examples "illustrative" and acknowledges that its parameter estimates are "considerably subjective."

What prevents a higher rating is the gap between this local honesty and the global framing. The title and abstract claim proof where the model offers only conditional prediction. A careful reader can identify where the formal apparatus ends and editorial judgment begins, but only by reading past the assertive framing to hedged language buried in the body text.

Individual citations are accurate — when the paper says a study showed selective outcome reporting (Chan et al. 2004), it did. The problem is one-sidedness. The literature review cites only work supporting the conclusion that research is unreliable. No cited source challenges the model's assumptions, presents evidence of successful replication, or argues that the parameter estimates might be too pessimistic.

Roughly a quarter of the references are to the author's own work or that of close collaborators. Several occupy positions where independent corroboration would have strengthened the argument.

This is an unfunded theoretical essay by an academic epidemiologist with no disclosed conflicts of interest. The thesis runs directly against the institutional interests of the research establishment the author belongs to. No identifiable party benefits structurally from this claim.