In this step by step guide, we will walk you through the core concept of A/B hypothesis testing and along with theoretical approach, we also take a hands-on glance for build up the strong statistical concept in our mind. Let’s Dive In
Step 1 — Import Required Dependencies
Firstly, we start this project by importing required libraries that help us in our analyses and also make efficient and smooth mindset.
Pandas — For data analysis and handling.
NumPy — For numerical computing and array handling.
Matplotlib/Seaborn — For data visualization.
Step 2 — Loading the Data
It’s time to read our dataset by using pandas built-in method called as (read_csv), through this method we can easily read any CSV data file and store in any two different variables.
Step 3 — Head of Data
If we want to see first 5 records of our dataset than we need to used pandas (head) method.
Step 4 — Information of Data
If we want to take general information about our dataset than we used (info) method for taking in-hand details of our data.
Step 5 — Checking Missing Values
If you want to check missing values that distract your data and your overall analysis, so you can used (isna()) method, through this method you can easily identify how much null values your data actually consist. It represents null values by 0 and 1, where 0 means you don’t have any missing values in your data, while if you get 1 that means simply its opposite.
Step 6 — Impute Missing Values
Now, we knew that, we have some missing values in our dataset, its time to fill the missing gap by imputing virtual values, for doing this we need to use (mean()) method, it means we putting mean values against all null values and after smartly act on this, we finally get balanced dataset.
Now, you have clearly see that all the columns denoted by 0, it means we successfully filled missing values.
Step 7 — Describe the Stats
If we want to get any type of statistical information regarding our dataset than we used (describe()) method and it provide us only numerical column stats.
Step 8 — Check Unique Value
If we want to see the unique value in our dataset, than we used (unique()) or (value_counts()) method, It list-down all the unique values that is present in our dataset.
Step 9 — Calculating Mean
Now, we calculating mean value of purchase column and try to make a sense that how many purchases is in middle.
Step 10 — Campaign Significance
The p-value is a measure of the evidence against a null hypothesis. In A/B testing, the null hypothesis often states that there is no significant difference between the two groups, In our case the Test and Control Campaigns. The p-value tells you how likely it is to observe a difference as extreme as the one you have observed in your data assuming that the null hypothesis is true.
A high p-value (close to 1) suggests that the observed difference between the two groups is likely due to random chance rather than a real significant effect. In your case a p-value of 0.9678 is very close to 1 that is basically indicating a high probability that any observed differences in the number of purchases between the Test and Control Campaigns are not statistically significant.
Conclusion
Based on the p-value and the common significance level (alpha) of 0.05 which is often used in A/B testing, we fail to reject the null hypothesis. In other words, there is no strong statistical evidence to suggest that the Test Campaign is performing significantly better or worse than the Control Campaign in terms of the number of purchases.
Given this result, you might conclude that for the metric of purchases there is no compelling reason to prefer the Test Campaign over the Control Campaign. Both campaigns seem to have a similar impact on the number of purchases at least based on the data you have analyzed.
