to return to my home page.
Mon, Wed, Fri, at 12.15pm, beginning 6 October 2000
Mill Lane Lecture Room 9
Course material including timetable changes (if any), examples sheets, and solution sheets, will be posted on this page.
Problem sheets:
A Markov process is a probabilistic process for which the future (the next step) depends only on the present state; it has no memory of how the present state was reached. A typical example is a random walk (in two dimensions, the drunkard's walk).
The course starts with a discussion of Markov processes in discrete time, including periodicity and recurrence. For example, a random walk on a lattice of integers returns to the initial position with probability one in one or two dimensions, but in three or more dimensions the probability of recurrence in zero. Some Markov processes settle down to an equilibrium state and these are the next topic in the course. Then Poisson processes are studied, which provide a model for the arrival of calls at a telephone exchange or the rate of detection of radio-active particles by a geiger-counter. The course ends with continuous time Markov processes; a typical application is the theory of queuing.
The material in this course will be useful if you plan to take any of the applicable courses in Part II(B); in particular, it is essential for Applied Probability and helpful for Communication Theory.
The course starts with a discussion of Markov chains in discrete time. The classical example is a random walk (or `drunkard's walk'), which has the celebrated property of being recurrent (i.e. it returns to its starting point with probability one) in one and two dimensions, but not in three or more dimensions.
A central property of `nice' Markov chains is that they settle down into a (stochastic) equilibrium. We shall study such equilibria in some detail.
Moving from discrete time to continuous time, we shall encounter a famous model, called the `Poisson process', for `totally random arrivals' such as the detection of radioactive particles, or the arrival of calls at a telephone exchange.
The Poisson process leads to a study of general continuous-time Markov chains. There are applications to queueing theory and elsewhere.
The course is fundamental to modelling disordered systems which evolve as time passes. Examples of such systems abound in science, and Markov chains are a key ingredient in many mathematical models.
Norris, Markov chains, CUP, 1997.
Grimmett and Stirzaker, Probability and Random Processes, OUP, 1992.
Cinlar, Introduction to stochastic processes, Prentice-Hall, out of print.
Feller, An introduction to probability theory etc, vol 1, Wiley, 1968.
Hoel, Port, Stone, Introduction to stochastic processes, Houghton Mifflin, ?in print.
Karlin, Taylor, A first course in stochastic processes, Academic Press, 1975.
to return to my home page.