Proof beats prose

Paper

Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories

Authors

Kevin Qinghong Lin, Yuhong Shi and Philip Torr (University of Oxford); Batu El, Pan Lu and James Zou (Stanford University)

Published

June 2026

Hand an AI the same spreadsheet you give a newsroom, and it will write a story that shows exactly where every number comes from. In a new study from Oxford and Stanford, blind readers compared the AI's stories with the human-written originals and preferred the AI's versions roughly three to one. The widest margin across everything readers rated was transparency: whether the story showed its sources.

Read past the "AI beat humans" version of that result and there's a lesson to be learned: What readers rewarded was the sourcing, not the machine. Most data-driven journalism doesn't show its work this way, not because human reporters can't but because it's not a regular part of their workflow. This same discipline is open to any newsroom, and as this paper shows, the tedious part of it is exactly what AI could help with.

What the researchers did

The team built a system called Data2Story that turned a raw dataset into a finished, interactive web article. Instead of one AI model doing everything, they split the work into seven specialized agents arranged like a small newsroom: a Detective that hunts for context, an Analyst that runs the statistics, an Editor that frames the narrative, a Designer that chooses visuals, a Programmer that builds the page and an Auditor that reviews the result. The piece that sets the system apart is the seventh: an Inspector that links every number, quote and chart back to its source.

To test it, the researchers took 18 published, data-driven stories from The Economist, The Pudding and TidyTuesday, and then had Data2Story produce its own version of each from the underlying data. They compared the pairs four ways: an angle-overlap check of how much of each side the other covered, a blind reader rating, an automated re-check of every claim by a separate coding agent and a "computer-use" agent that clicked through the pages as a stand-in for a reader.

The key findings

Provenance is where the system shines. A separate verifier re-executed the claims in each article. For Data2Story's pieces, 93% resolved to a traceable link between the text and its evidence. For the human articles, only 25% did. The reason is mostly mechanical: human stories don't ship with code attached, so the verifier had to guess at how each number was produced. One nuance the researchers were careful about: this measures whether a claim carries a verifiable trail, not whether it's true.

Readers preferred the AI's articles, with a caveat. Across 53 blind reviewers, the agent's articles scored higher on all five rubric dimensions, with an overall mean of 4.21 versus 3.38 on a seven-point scale. Asked to pick a favorite outright, 39 of 53 reviewers chose the AI version, 13 chose the human version and one called it a tie, roughly a three-to-one preference. The biggest margin was on transparency (+1.49); the smallest was on visual design (+0.51).

The win vanishes where craft matters most. Broken out by publication, the agent clearly beat The Economist's short data briefings and TidyTuesday's charts, but tied with The Pudding — the source known for elaborate, hand-built stories. The researchers were blunt about why: Pudding pieces take design teams weeks and carry a single committed thesis the agent doesn't reproduce. The agent won where analytical framing carried the story, and merely matched humans where voice and design mattered.

The AI can't reach the half of the story angle that isn't in the data. When the team checked how much of each side's editorial angle the other covered, the asymmetry was telling: the agent reproduced about half (50.4%) of the human's angle, while the human articles reproduced only 35.1% of the agent's. The half the agent misses is the reporting that lives outside the spreadsheet: expert testimony, accountability framing and the "why" behind the numbers. In one pair, the human story on repair cafés framed broken devices as a problem of manufacturer accountability: companies build phones, cars and tractors so that independent repairers can't get the parts or diagnostic tools to fix them. Working only from the data table, the agent could rank what breaks most often but left that cause to the reader.

What this means for newsrooms

Show your sourcing and readers may reward you. Transparency drew the widest gap of any quality readers scored. Readers respond to seeing where the numbers came from, and the habit costs nothing and needs no AI: Cite specifically, show the methodology and link the data so anyone can follow it.

AI is unusually good at data matching. The genuinely new capability in the paper is machine-checkable sourcing: every claim tied to the code or data that produced it. You don't need this system to borrow the idea, but if the tedious work of tracing each figure is what stops you from showing it, that tracing is exactly the kind of task an agent can take on. It's strongest on number-heavy pieces a small team would otherwise skip.

Don't read this as "AI writes better." The rubric rewarded transparency heavily and the system was built to maximize it. Read this preference as readers liking the clearer, better-sourced version, not as a verdict on machine vs. human writing. On the most design-heavy pieces, The Pudding's "scrollytelling" style of illustrated storytelling, the AI only tied.

Verifiable is not the same as correct. A claim that can be traced can still be wrong. The trail is where checking starts, not a substitute for it.

Important caveats

The reviewers were 53 people recruited through Prolific, made up of general crowd readers rather than journalists, editors or data-visualization specialists, and each one rated only a single article pair. That's a thin and non-expert panel, and it likely favors clean, information-dense pages over subtler editorial craft. The study was also built to surface transparency, so "readers preferred it" is best read as a result inside this test, not a universal law. It's small at 18 article pairs, and the team built, tested and scored its own system, which is standard for this kind of work but worth stating plainly. The paper is a preprint and hasn't been peer-reviewed. The code and a demo are public, which is a point in its favor for anyone who wants to kick the tires.

How we checked this

The paper argues for a provenance standard, so it seemed fair to hold this write-up to it. Before publishing, we pulled the paper itself (arXiv preprint 2606.11176) and traced every number above back to where it appears in the source. Here is that trail:

• 93% vs 25% provenance rate (Section 4.3.4). The 25% figure is not a knock on human journalism: the verifier had the raw dataset for the human articles too, but their prose ships no code to re-run, so most claims can't be machine-checked.
• Overall reader score 4.21 vs 3.38; transparency +1.49, visual design +0.51 (Figure 8a).
• 39 chose the AI, 13 the human, 1 tie (Figure 8c).
• Angle coverage 50.4% vs 35.1% (Figure 6b).
• Beat The Economist and TidyTuesday, tied The Pudding (Figure 8b; the Pudding difference was not statistically significant).
• Seven agents and the repair-cafés example (Sections 3 and 4.3, with the repair-cafés pair in Table 9).

One honest limit, and it's the same one the paper draws: this trail isn't machine-checkable. A reader can't click a button and re-run it the way the paper's verifier re-executes code. The checking here was done by an AI assistant reading the source and matching each number to its figure, with no second person re-deriving them independently. In the paper's own terms, this post behaved like the human articles it describes: the provenance was reconstructed after the fact rather than built in. Closing that gap, so the trail is something anyone can replay, is the work worth doing, whoever or whatever is doing the writing.