Module Details

MA922-7-AU-CO: Stochastic Processes

Note: This module is inactive. Visit the Module Directory to view modules and variants offered during the current academic year.

Year: 2017/18
Department: Mathematical Sciences
Essex credit: 15
ECTS credit: 7.5
Available to Study Abroad / Exchange Students: Yes
Full Year Module Available to Study Abroad / Exchange Students for a Single Term: No
Outside Option: No

Teaching Staff: Dr. David Penman, email:
Contact details: Miss Shauna McNally - Graduate Administrator. email:, Tel 01206 872704

Module is taught during the following terms
Autumn Spring Summer

Module Description

To acquaint students with the use of probability theory to study models of phenomena with a degree of unpredicability about them, such as queues and population growth.


- Examples of stochastic processes.
-Markov chains: definition and examples; random walk, gambler's ruin.
- Chapman-Kolmogorov equation.
- Classification of states: persistent (null or non-null) or transient, periodic or aperidoic.
- Probability generating functions.
- Limiting probabilities.
- Absorption probabilities.
- Branching processes: number of individuals in nth generation. Probability of extinction. Total population until extinction.
- Exponential distribution: definition and properties, memoryless property.
- Continuous-time Markov chains: definition. Birth and death processes.
- Chapman-Kolmogorov and Kolmogorov equations. Limiting probabilities. Balance equations.

On completion of the module, students should be able to:
- understand the Markov property and its application to Markov chains in both discrete and continuous time;
- derive the Chapman-Kolmogorov equation in both discrete and continuous time;
- classify the states of Markov chains;
- calculate mean recurrence times, limiting probability distributions and absorption probabilities for discrete-time Markov chains;
- calculate population-size distributions and extinction probabilities for discrete-time branching processes;
- understand the memoryless property of the exponential distribution;
- derive the Kolmogorov forward and backward equations and use them to find transition probabilities of continuous-time Markov chains.
use balance equations to find limiting probabilities of continuous-time Markov chains.

Learning and Teaching Methods

The module runs at 3 hours per week. There are 5 lectures and one class in every fortnight.

In the Summer term 3 revision lectures are given.


20 per cent Coursework Mark, 80 per cent Exam Mark

Other details

Information about coursework deadlines can be found in the "Coursework Information" section of the Current Students, Useful Information Maths web pages: Coursework and Test Information

Exam Duration and Period

2:00 during Summer Examination period.


  • Recommended reading:
  • S. M. Ross. Introduction to Probability Models, Academic Press
  • Supplementary texts:
  • W. Feller. An Introduction to Probability Theory and its Applications, Vol 1. Wiley.
  • S.M. Ross. Stochastic Processes. Wiley.
  • D.R. Cox & H.D. Miller. The Theory of Stochastic Processes. Chapman & Hall.
  • G.R. Grimmett & D.R. Stirzaker. Probability and Random Processes. Clarendon Press.
  • S. Karlin & H.M. Taylor. A First Couse in Stochastic Processes. Academic Press.
  • J. Kohlas. Stochastic Methods of Operational Research. Cambridge Unversity Press.
  • H.M. Taylor and S. Karlin. An Introduction to Stochastic Modelling. Academic Press.

Further information