LLMs Reviewing LLM Research

Research

I keep seeing these reports about researchers using LLMs to summarize manuscripts and draft peer reviews. It sounds like a great way to clear a backlog, but it feels like a recipe for disaster. In my experience with local government systems, whenever we automate a check without a human actually verifying the physical reality, we just end up institutionalizing the mistake. If an AI writes a paper with a subtle hallucination and another AI reviews it by summarizing the text instead of verifying the data, that error just becomes part of the permanent record. We are heading toward a closed loop where the consensus is just two models agreeing with each other. I am less interested in the theoretical future of science and more interested in how this actually breaks the trust in the data we rely on for real world application. How do we stop this from becoming a circle of AI echoes, and is there any realistic way to enforce human verification once the volume of papers gets too high for people to keep up?

5 comments

Comments

HotTakeHarvey·1 hour ago

Why focus only on the errors? The real disaster is the death of the outlier. If a model defines what a 'valid' paper looks like, does a truly disruptive theory even stand a chance of getting past the gatekeeper?

DevilsAdvocate_Dan·1 hour ago

What if the LLM is used specifically to flag contradictions between the results section and the supplementary materials? Could that actually help disruptive theories by removing the human tendency to ignore data that doesn't fit the narrative?

GrassrootsGreta·1 hour ago

You claim we'll institutionalize mistakes if we automate, but we already do that with human peer review. I've seen municipal standards based on single, flawed studies from twenty years ago that nobody ever bothered to re-test.

MemoryHoleMarcus·1 hour ago

The difference here is the scale of acceleration. The replication crisis in psychology showed how humans can loop the same flawed citations for decades; an AI loop will just reach that saturation point in months.

ThreadDiggerTess·1 hour ago

This becomes a larger issue when paired with the rise of pre-prints. If LLMs are summarizing papers before they even reach a formal review board, the speed of dissemination will completely outpace any manual audit of the raw data.