Both stores will split your traffic and tell you which gallery wins. Most teams never switch it on, and the ones who do usually stop the test three days early. Here is how to run it so the answer means something.
Reading time: about 7 minutes
Treatments require App Review approval before they start serving.
Free, and available for the default listing or a localised one.
Ranked by how much of your conversion the variable actually controls.
It appears in search results and is cropped on the product page. Nothing else you can test has this much leverage.
Same image, different line. Isolates message from design, which is the cleanest test you can run.
A landscape opener occupies more of the gallery row. It is a big, cheap swing worth testing once.
Bold Contrast against Minimal Light, whole gallery. Coarse, but decisive when you are choosing a direction.
Same assets, reordered. Costs nothing to produce and sometimes moves conversion more than new artwork.
Ratings or press moved from frame five to frame two. Tests whether your audience needs reassurance early.
Change the first screenshot or the first headline, never both. A two-variable test that wins tells you nothing about which change to keep.
Store traffic has a weekly rhythm. A test stopped on day three is measuring the weekend, not your artwork.
Peeking at a running test and stopping the moment it looks positive is how teams ship changes that are noise. Decide the duration before you start and hold it.
A 5–15% lift on conversion is a real, compounding result. Claims of doubling conversion from a screenshot swap almost always come from a listing that started badly broken.
If you get a few hundred page views a week, only a large effect will ever clear the noise. Test coarse changes — a whole art direction, not a word — or fix the obvious problems first and skip testing.
A winner is a winner for the artwork and the audience you had. Both change. Re-run after a major UI change and, if your category is seasonal, at the turn of the season.
Four outcomes that are easy to misread in your own favour.
Yes. Apple's Product Page Optimization lets you run up to three treatments against your live page, with screenshots, app icon or preview video as the variable. Treatments go through App Review before they start serving.
Yes. Store Listing Experiments are built into Play Console at no cost and report installs per variant with a confidence interval. It is the cheapest conversion work available to an Android app, and most teams never switch it on.
One to two weeks at minimum, so the test covers full weekly cycles, and longer if your traffic is low. Decide the duration before you start — stopping early on a favourable day is the most common way these tests mislead.
The first screenshot, then the first headline. Those two carry most of the conversion, because search results show only the first two or three frames and most people decide in seconds.
Generate them from the same upload. Asking for multiple art directions gives you genuinely different compositions of the same screen rather than recolours, which is what a coarse-variable test needs.
Four art directions of the same screen, generated together, is exactly the input a coarse-variable test needs.
See the four directions