We apply ideas from post-selection inference to randomization tests for adaptive experiments.
The following is an email exchange with Hyunseung Kang on the nature of randomization inference. This was sparked by a couple of papers by David Freedman and David Lane: A Nonstochastic Interpretation of Reported Significance Levels; Significance Testing in a Nonstochastic Setting.
This post is derived from my talk “Fisher, Statistics, and Randomization” in the Fisher in the 21st Century Conference organized by Fisher’s College, Gonville & Caius. In the first half of that talk, I tried to trace the origin of randomization. Fisher is widely credited as the person who first advocated randomization in a systematic manner. In doing so, he profoundly changed how modern science is being done.
One of Fisher’s most important scientific contributions is the paradigm of randomized experiments. I will try to trace the origin of this idea and highlight how it is rooted in Fisher’s cross-disciplinary interest in mathematics, biology, statistics, …
Project page for randomization and randomization inference.
The meaning of randomization tests has become obscure in statistics education and practice over the last century. This article makes a fresh attempt at rectifying this core concept of statistics. A new term---'quasi-randomization test'---is …