Parallel Peer Review: Who's Tried It

The idea of evaluating published research outside the journal system isn't new. Many projects have attempted some version of it — from anonymous comment boards to expert rating services to full alternative review pipelines. Most have struggled with the same problem: who does the reviewing, and why would they keep doing it?

Here's how the major initiatives compare.

Recurring patterns

Many of these initiatives are still running, some very successfully within their niche. But a few patterns keep showing up:

• Volunteer fatigue. Reviewing is work. Without institutional incentives or payment, participation drops. Publons tried to solve this with credit tracking; ResearchHub tries with cryptocurrency. Sustained volunteer effort is the hardest part of the model.
• Drift toward fraud detection. PubPeer set out to be a journal club. It became an image manipulation detector — valuable, but different from what was planned. When anyone can comment on anything, the comments that stick are the ones flagging outright misconduct, not the ones evaluating argument quality.
• Curation is expensive. Faculty of 1000 recruited 8,000 senior scientists to rate papers. Maintaining that network required constant effort and funding, and it changed hands twice.
• Audiences stay academic. Most initiatives target other researchers, not the public. When a lay person encounters a study claiming X, these platforms aren't designed to help them figure out whether to believe it.

The projects with the most staying power — PubPeer, Peer Community In, Review Commons — either found a focused niche or integrated into the existing journal workflow rather than trying to replace it.

Where Litmus fits

Litmus Review uses LLM-driven evaluation with a structured rubric, with every word reviewed by a human editor before publication. The target audience is anyone trying to figure out whether a study's conclusions hold up — not just other academics. The evaluation produces a concrete verdict (solid, credible, dubious, or nonsense) based on a non-compensatory scoring system where a fatal flaw in one dimension can't be offset by strength in another.

This sidesteps the volunteer problem entirely: the evaluation pipeline scales without recruiting reviewers. The editorial gate — a human reads every word before it publishes — is the bottleneck by design. See the LLM policy for more on that tradeoff.