Hi, and welcome to a quick introduction on Steps in Experimentation.
When running a business, there are many different moving parts happening all at once.
You might redesign your website, or add a new chat bot, or experiment with your
recommendations you show customers.
But before you jump ahead and launch new products or elements, you're going to want to test
to make sure that the experience is useful to users and that you are achieving the metrics
you are hoping for with the new changes.
This is why it is helpful to carefully draw out your steps in experimentation.
The first step in experimentation is planning.
This is where you ask yourself your pressing business questions, such as: What is the element
you are going to change and what is that change going to look like?
How large is your sample size and how long will you run your experiment?
In the planning phase, you will want to know what business problem you are trying to solve
and what are the metrics and expected outcome?
As mentioned previously, you are going to have to establish your criteria for success
prior to running your test and having a hypothesis helps to understand what you are hoping to
learn from the experiment.
The second step in experimentation is coding and logging.
This is where you set up the test and experiment that you want to run.
For example, you change a call to action button from green to red.
Or updating your algorithm to show your user different recommendations.
The third step in experimentation is performing an A/A test.
As mentioned previously, you would want to run an A/A test to validate the setup of your
experiment by comparing the identical experience on a different random set of users.
For example, your original version, Version A or "control", is a green "Buy Now" button
and your alternate version, your treatment, is also a green "Buy Now" button.
It is good to set up an A/A test to show a random subset of users the same experiment
and another random group of users the same experiment.
You should have different users and different data, and if it is the same experience, but
the metric is different, something is most likely wrong with the setup.
After running your A/A test and you find that the results of the experiment were not statistically
different from one version to the other, you can then run your A/B or multivariate test.
So once you've run your A/B or multivariate test, you can then look at the analytics and
performance of the changes you've made.
What are your metrics showing you and were they as you suspected when you established
your hypothesis?
It's always good to consider various factors that might have affected the performance of
your test, such as seasonality, the segments you ran your experiment on, or a newness effect.
Did just seeing something new trigger excitement in your customers?
And lastly, the final step in experimentation is to make a decision.
Do you ship it or not?
And this might require further discussion among colleagues or stakeholders.
It might also lead to running a new experiment, where you will again begin the process of
running through steps 1-6.
Thanks for watching, give us a like if you found this useful
or you can check out our other videos at tutorials.datasciencedojo.com
For more infomation >> Analyticity|Analytic, Entire Function, Necessary & Sufficient Conditions|Mechanicalstudents.com - Duration: 16:02. 
For more infomation >> 골관절염과 류마티스 관절염 사이에 혼동[다름 이해]|조회수8.212.910 - Duration: 11:21. 
For more infomation >> Super Tom Chạy Lấy Vàng Bắt Tên Trộm Raccoon - Talking Tom Gold Run #34 - Duration: 10:18. 
For more infomation >> 19년 새해 첫날! 고성 옵바위 일출 타임랩스 / 동해가는 휴게소에 이렇게 많은 사람이?! / VLOG 브이로그 여행 - Duration: 5:04. 
Không có nhận xét nào:
Đăng nhận xét