Optimize your App Store performance with A/B Testing

Some app stores - that is, Apple and Google - let you test certain aspects of your app store presence against a different version, such as: 

• Screenshots 
• Title 
• Video 
• Descriptions

An A/B test allows you to test different variations against each other (typically with a 50% to 50% split in what the audience sees when they come to your page), and identify whether one variation results in more app downloads than the other.

Before you start an A/B test

Before you run an A/B test, you need to create a hypothesis. Think about what you want to change (e.g. “highlight a positive app review in the description”), the expected results of the change (e.g. “downloads will increase with a positive app review highlighted”), and why you anticipate these results (e.g. “social proof will result in more downloads”). 

Some additional ideas for A/B tests:

• Order of screenshots (the first two screenshots are the most viewed) 
• Different artwork in the screenshots 
• Highlighted numbers, such as number of downloads 
• Emojis to highlight special features 
• Bullet points in the description 
• Longer or shorter descriptions 
• …etc. 

Always start an initiative with some research and analysis - insights from users is a great place to start.
‍

How to set up an A/B Test

Apple 

For Apple, you can test app store screenshots against each other to see which screenshot presence performs better. Google has a few more options (which will be explained later on in this article).

To do this, go to “Product Page Optimization” and click on “Create Test”. 
‍

On the next page, give your test a name and choose how many treatments you want to test against each other (e.g., a different order of your existing screenshots, or upload entirely new screenshots). 
‍

Click on “Create Test” to activate your test. 

For more information: 

In Apple App Store Connect, go to the menu Product Page Optimization in your side bar to create an A/B test, and follow the instructions listed out in this guide.

You can also read a more general overview of Product Page Optimization on this link.

Google 

Go to Grow users > Store presence > Store listing experiments. Name your experiment and choose what you want to test. Pick “Localised experiment” if you want to test descriptions against each other, and “Default graphics experiment” for testing e.g. screenshots.
‍

Set your experiment goals, e.g. the number of variants and the target metric. Click “Next”. 
‍

On the next page, you can choose the metrics you want to test. Click on “Edit Variant” at the bottom of the page to configure the changes you want to test. 

It is recommended only to pick one metric to test against (e.g. description or screenshots). If you pick a number of variables at the same time, you will not know which of these variables were responsible for any performance improvements.
‍

Once you have configured the variant, click “Save”. Don’t forget to send your changes to Google for review. 
‍

‍
‍For more information: ‍

In your Play Console, go to Store Listing Experiments, and follow the instructions in this guide.

You can read more on A/B testing for Android and Apple in this guide.