A/B hypothesis testing also known as split testing is a two-sample hypothesis testing technique that is used to compare two choices/ two versions of something to know which choice/version is better. This is done by running a controlled experiment. A/B hypothesis testing can be leveraged by businesses to improve customer experience of their products or services and also improve customer retention This project was done to help the company generate more revenue based on the decision on whether to retain the old design of their website or to use the new design. For this project; Jupyter notebook was used as the IDE as it gives the opportunity to explain the steps carried out in detail for both technical and non-technical users to understand. Pandas library was used for data cleaning, wrangling and overall preparation of the data for the task Scipy library was used for statistical analysis
The data to be used was imported into the working environment using as Pandas DataFrame. Data quality checks was carried out. Checks carried out includes; data type, missing values, number of unique groups (for this project, just two is expected, Variant and control group), test to ensure that a user doesn’t belong to two groups, that is, one user can only belong to one group. After data quality checks, summary statistics was calculated for both groups, then statistical analysis, A/B hypothesis test was carried to help drive business decision.
A total of 97 outliers were observed, but these outliers were not removed as it would distort the property of the data and therefore give unreliable results after analysis. The revenue generated by the control group was observed to be more than that generated by the variant group. The null hypothesis: there is no difference between the income generated the website’s old design and the new design. Alternate hypothesis: There is a difference between the income generated by the website’s old design and the new design. After performing the hypothesis test, it was discovered that the difference was not statistically significant, hence, we fail to reject the null hypothesis. Looking at the summary statistics and the result gotten from the A/B test, the old website design should be retained as it generates more revenue for the company, and there is not enough evidence that the new website design would generate more revenue.