While there were some applications of sequential testing procedure before that, the first significant development in modern sequential testing happened during the Second World War. Early development of sequential statistical testsĮarly development of sequential statistical tests.Online A/B testing remains the focus throughout. The article ends with a brief note on the related concept of tests of power of one. ![]() The article will start with a bit of history to show the evolution of sequential testing which will help put the rest of the article into context. It draws heavily on Chapter 10 (“Sequential testing: continuous monitoring of data”) of my book “Statistical Methods in Online A/B Testing” as well my 2017 white paper “Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method”, but also contains insights not shared before, including commentary on some potentially misleading claims by A/B testing software vendors. This article will present the main available options, namely fully sequential and group sequential tests, their strengths and weaknesses, and attempts to outline what test to use in which scenario. ![]() What is the best design for a statistical test with sequential evaluation of the data at multiple points in time? This is a question anyone who has realized that unaccounted for peeking with intent to stop is the bane of A/B testing eventually comes to ask.
0 Comments
Leave a Reply. |