Last updated: May 11, 2015 12:43
THIS ARTICLE WILL HELP YOU:
- Decide what metrics to track in your app
- Determine how you will create variations
- Learn key design and development considerations for Android and iOS experiments
Mobile is taking over. In fact, according to comScore, mobile devices accounted for 55% of internet usage in the US in Q1 2014. This presents a huge opportunity to optimize across all channels -- desktop web, mobile web, native apps, and beyond.
To succeed in the world of cross-platform and mobile optimization, companies like Facebook, LinkedIn, and Airbnb need to be able to learn quickly, iterate, and respond to their mobile user behavior. In this article, we'll show you how you can do the same for your mobile apps with Optimizely.
TIP:
Can't get enough mobile testing strategy? Check out Optimizely's e-book, A Blueprint for A/B Testing and Optimization in Native Mobile Apps and our article on mobile testing ideas!
With Optimizely, you can experiment and push content without repeated submissions to the Apple App Store, thereby compressing the app development cycle and making your platform more responsive to user behavior.
Optimizely on mobile enables experimentation that will equip you with the data you need to maximize the business value of your app. To do this, you should identify what key metrics you want to optimize through testing. Here are the most common ones that testing can influence:
| **Metric** | **Why It's Important** | **How You Can Test** |
| **Daily Active Users (DAUs)** | DAUs measure your engagement levels and retention, and they're an indicator of your app's stickiness and the lifetime value (LTV) of your users. | Changes to UX, messaging, and features can make your app stickier and increase DAUs. |
|
Downloads/ Registrations | Downloads and registrations serve as the top-of-funnel metric for user engagement, and should correlate with DAUs. | Experiment with your app's icon, name, keywords, and screenshots in the app store to promote downloads. And try simplifying or eliminating your in-app registration flow. |
| **Retention** | DAUs divided by downloads measures your retention. | If you see a high download rate compared to DAUs, you have a clear reason to experiment with retention features. |
| **In-App Purchases (IAPs) or e-commerce** | IAPs (or purchases in an e-commerce app) are a direct source of revenue and a clear conversion metric. | Experimenting with prices and pricing models can help optimize IAPs, especially for freemium apps. E-commerce apps also have a variety of options to drive purchases. |
| **Average Rating & Reviews** | Ratings and reviews quantify what users think of your app and serve as a leading indicator of lifetime value (LTV). | Test different timing and prompts for your ratings and reviews. |
| **Taps and Views** | For apps that contain CTAs, forms, or ads, taps and views can serve as indicators of engagement, or even revenue. | Experiment with changes that make it easier for users to tap CTAs, complete forms, or highlight ad content. |
| **Offline Conversions** | Some apps have offline conversions, such as redemption of app-exclusive offers that occur outside of the app. | Changes to messaging and promotions can affect offline conversion numbers. |
Once you've identified key metrics, consider how you might test experiences that increase user delight and reduce friction. When testing on mobile devices, use these guidelines to shape your hypotheses:
**Design with the physical “sweet spot” in mind. **Only 1/3 of the screen real estate is effortlessly accessible by the thumb. Test to identify the ideal placement for the elements that drive your key conversion metrics.
Know your audience. If you have loyal users on the web, they'll likely intuit your app's functionality but expect more agility. For new or mobile-first users, keep the interface simple, focused on key actions that will drive repeat usage. Explore delivering tailored experiences to each group.
**Focus on NUX (new user experience). **Consider how users will be introduced to your app when they load it for the first time.
- **Social login. **Including social login options (Facebook, Google, etc.) can decrease barriers to entry.
- Make users feel invested early. Mobile users are less likely to type multiple inputs on a small screen. Get them engaged early through discovery and engagement.
- Reduce friction and increase interactivity. With less screen real estate than the web, mobile users appreciate search, filtering, image browsing, and other focusing features early in the funnel.
- Experiment with different flows. For example, your app may include an instructional mode, labelled icons, or full, ungated content on first sign-in. Figure out which experience works best for retention.
****Respect the tap, embrace the swipe, and consider the scroll. ****On the web, clicks are a key way of interacting with content, but in apps, you have the luxury to think through other ways of prompting user interactivity. Touch screens offer the unique ability to swipe and pinch content, which many users find engaging. Touch screens also offer a more natural scrolling experience than desktops do, and users may be more willing to scroll on mobile devices. Discover which flow works better for your users at key interaction points -- more scrolling, tapping to new views, or gestures.
**Less is (still) more. **On mobile devices, you don't have the luxury of designing for the big desktop screen -- so embrace the constraints! Decluttering is already important on the web, but designing for a mobile-first experience means stripping content to its bare minimum. Focus on exposing your main actions and making them simple to perform.
**Balance focus and exploration. **Apps often feature deliberate process flows that aim to guide users through a series of tasks. When designing for the small screen, delight your users with an experience that is focused enough to provide direction, but open enough to drive exploration and repeated use. Focused experiences that constrain user freedom may drive abandonment due to limited functionality, while open and complex flows may confuse users with competing calls to action and a lack of direction.
Most popular apps deliver both task-based functionality and information-seeking content, and users can have different intents based on their source. Mobile users require both because information-seeking is a task that bridges the web and app experiences. This instills confidence in users that they can achieve as much, if not more, through the streamlined app interface, making them more loyal to the app and as a consequence more loyal to the business.
The process flow and milestones you aim to have users achieve should be a central focus of your app testing program. To achieve the optimal flow, you should be constantly testing:
- How and when the app asks for permissions
- The length and steps of your registration flow, and how much content is gated before registration
- How and where ads are featured
- Number, sequence, and format of calls to action throughout your app's flow
By focusing on functionality, not just design, you can drive key insights about how users engage with your app.