A Peer Reviewer Missed the Point of a Vaccine Study. AI Might Make That Worse.
When a health economist found that school HPV vaccine mandates barely move the needle on cervical cancer rates, his peer reviewer thought he was attacking the vaccine itself. That kind of misread is common. AI-generated reviews could make it routine.
Key points
- Health economist Jason Semprini found that school mandates requiring the HPV vaccine have little measurable effect on population-wide cervical cancer rates.
- A peer reviewer misread Semprini's policy study as a challenge to vaccine safety, a significant factual error that could have killed valid research.
- Peer review, the process by which independent researchers check work before it is published, is already under strain from rising submission volumes.
- AI-generated peer reviews are appearing in academic journals, raising concerns about whether errors like this one will become harder to catch.
Jason Semprini is a health economist at Des Moines University who studies the gap between health policy and health outcomes. His latest project examined school mandates requiring children to receive the HPV vaccine. HPV, human papillomavirus, is a common infection that causes the vast majority of cervical cancer cases.
His finding was counterintuitive. Mandating the vaccine at school does not appear to move the overall rate of cervical cancer much in a given population. That sounds alarming, but it is not entirely surprising. Researchers have long known that firm government mandates can push some families toward active refusal, a backlash effect that can cancel out the gains from broader coverage.
Semprini wrote up the study and submitted it to a journal for peer review. Peer review is the process where independent researchers in the same field read a manuscript, check whether the methods hold up, and recommend whether the journal should publish it. Reviewers are typically volunteers, and the process is usually anonymous.
What went wrong?
The reviewer misunderstood the study's central question. Instead of reading it as an evaluation of a vaccination policy, the reviewer treated it as a claim that the HPV vaccine itself does not prevent cervical cancer. Those are very different statements. One questions a school administrative rule. The other would contradict decades of clinical evidence.
"There's a very big difference there," Semprini told Ars Technica.
The vaccine works. That is settled science. What Semprini was asking is narrower: does making it compulsory in schools actually result in fewer cancer cases across a whole population? The answer, he found, is largely no, at least not through mandates alone.
A reviewer who misses that distinction could block sound, useful research from reaching doctors, policymakers and the public.
Why does this matter beyond one study?
Peer review was already struggling before AI entered the picture. The number of papers submitted to academic journals has grown faster than the pool of qualified reviewers willing to read them for free.
Now, some researchers are using large language models, the technology behind chatbots like ChatGPT, to help write or even generate their peer review reports. A model that skims a paper rather than reading it carefully is likely to produce exactly the kind of surface-level misread that tripped up Semprini's reviewer.
For ordinary readers, the stakes are practical. Published research shapes what doctors recommend, what policies governments adopt, and what health advice reaches patients. Flawed review means flawed research can get through. Accurate research can get blocked.
Common questions
Does this mean the HPV vaccine is not safe or effective?
No. The vaccine's safety and effectiveness are supported by extensive clinical trials. Semprini's study examined whether school mandate policies specifically increase vaccination rates enough to lower cervical cancer across whole populations, not whether the vaccine works on individuals who receive it.
What can patients or parents do with this information?
Talk to your child's doctor about the HPV vaccine regardless of whether your school or state requires it. A mandate's absence or failure is not a reason to skip vaccination; personal and family protection still applies.
How do journals try to catch reviewer errors?
Most journals use editors who read both the manuscript and reviewer comments before making a final decision. A good editor would catch a misread like this one. The concern is that rising submission volumes are stretching editorial capacity thin.
