AB Testing in Product Development

AB testing is a great way to understand the "voice of customer" by removing guesswork and ultimately making data-driven decisions, quickly learning (when no/ low impact) & moving to the next hypothesis to solve the user’s problem.

Benefits of AB Testing:

• Answer business questions through user’s behaviour - insight to what features matter.
• Data analytics removes the biases or assumptions - learn what works and what is failing.
• Understand the lift (Control A versus Variation B) - impact of causal data.
• Insight to clusters/ segments - group of common needs (type of usage).
• Understand the engagement rate and key pain points.

Through a dedicated focus "on user experience and preference" a shift is made from optimising metrics to a  full understanding of the Online Customer Journey while improving customer experience.

AB Testing can be applied to layouts, design, banners, paid ads/ searches, new products, product enhancements.

AB Testing is when 2 or more versions of a web page are shown to different segments of website visitors at the same time and testing for positive impact on business metrics

A few metrics to monitor when doing AB Testing on websites, keeping in mind that metrics can be grouped into 'primary' and 'secondary':

• page dwell time
• number of demo requests
• cart abandonment rate
• click-through rate (CTR)

Steps to AB Testing:

1. Define the business Q.
2. Brainstorm ways to answer the question.
3. Insight to variations.
4. Define the primary metric.
5. Run the test.
6. Analyse the results, lift and segmentation.
7. Make recommendations or actions.
8. Repeat the process.

Always reiterate. It is a process and continuous feedback will result in an end-to-end journey reflective of user preferences.

For more Follow me on Medium https://medium.com/@aveshnee7