A game on 4,578 real A/B tests
Which headline got more clicks?
Upworthy showed thousands of readers one of two headlines for the same story, with the same photo, and counted who clicked. Pick the winner. Mimiq picks too: it made its calls before it ever saw a result.
Where the rounds come from
Each round is one experiment from the Upworthy Research Archive: two headlines for the same story, shown at random to thousands of readers between 2013 and 2015, where one got clearly more clicks (p < 0.01). Only the headline differed.
How Mimiq made its calls
These tests were held out: Mimiq's method was built on other experiments, frozen, and pre-registered before it saw any of these. It reads both headlines in both orders and forecasts which one more people like the readers would tap; simulated readers react to each one on its own.
The full benchmark, misses includedData: the Upworthy Research Archive, J. Nathan Matias, Kevin Munger, Marianne Aubin Le Quere and Charles Ebersole, Scientific Data 2021, licensed CC BY 4.0. Readers shown here are simulated.