Statistical Theory



Statistical Theory (M16)  R. Samworth

This is a course on parametric statistical theory that goes hand in hand with the Lent term course on nonparametric statistical theory. We begin by reviewing briefly the classical methods and theory of inference based on the likelihood function. Although these methods are usually perfectly adequate for relatively low-dimensional models, they can fail badly in high-dimensions - in particular, when the dimension of the parameter space (usually denoted p) is larger than the number of observations, n. These `large p, small n' problems occur in a very wide range of applications, from microarray experiments in biology to portfolio selection in finance, and are at the forefront of modern Statistics. We will outline some of the most important recent developments in this very active research area.

Classical theory: Review of linear models. Review of likelihood function and related quantities. Distribution theory in no nuisance parameter case. Generalised linear models. Traditional model selection methods (e.g. AIC). Basic results from measure theory and probability, such as modes of convergence, convergence theorems, differentiation under an integral, stochastic order notation. [5]

High dimensional problems: Shrinkage. Ridge regression. Cross-validation. Penalised likelihood. LASSO and associated theory (based on Karuch-Kuhn-Tucker and compatibility conditions) and algorithms. Other penalty functions, e.g. SCAD. Related problems, e.g. Group LASSO, Graphical LASSO, additive models. [7]

Multiple testing and other topics: Bonferroni correction. False discovery rate, Benjamini-Hochberg procedure. Storey's procedure. Other topics, e.g. Covariance matrix estimation, Low rank + sparse estimation. [4]

Pre-requisite Mathematics

Basic familiarity with statistical inference, including point estimation and hypothesis testing, will be assumed. Part IID Principles of Statistics is recommended as background. A small amount of measure theory and convex analysis/optimisation will be used in the course, though we will cover what we need as we go along.

Literature
1. L. Pace and A. Salvan, Principles of Statistical Inference, World Scienti c (1997).

2. T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning, Springer (2009)

3. P. Bühlmann and S. van de Geer, Statistics for High-Dimensional Data, Springer (2011)

 

 

 

 


Back to top