Insight: Unlocking the secrets of Victorian and Edwardian life

Essex historians have brought this immense social and economic change vividly to life by using digitised Census data.

The rich resource, which has made available online 35 million household observations and over 200 million individual observations recorded during censuses between 1851 and 1911, is transforming research on social, economic and demographic change in Victorian Britain.

Why Census data is valuable

“Censuses place each individual in the country on census night in particular administrative districts and give standardised information for them. Since the questions asked are largely standard, we can see how the nature of places changes over time,” explained Professor Edward Higgs, co-investigator on the Integrated Census Microdata (I-CeM) Project.

Each census contains vast amounts of information on house and household structures, and for each individual including name, marital status, relationship to the head of the household, age, sex, occupation and some medical disabilities.

“By combining census data with other sources of information such as birth, marriage and death records, we have been able to link information enriching our understanding of the processes of change,” added Professor Higgs.

The challenge

Raw census data, provided by a commercial partner, FindMyPast, gave Professor Higgs a starting point.

Working with his co-investigator, Professor Kevin Schürer of the University of Leicester, two research officers and a team of 30 Essex students, Professor Higgs began the massive task of checking, sorting and standardising the data.

“One of the things we had to do was sort out the data geographically, since a lot of it was jumbled up, and we had to make sure the transcriptions were accurate. Variables, like occupations which could be described in multiple ways, had to be coded too so that the eventual online resource could recognise that things like agricultural labourer, farm labourer, and farmer’s hand were all the same,” said Professor Higgs.

Creating a resource of reliably comparable data took the I-CeM team three years to complete.

What the I-CeM team achieved

Large-scale academic analysis of original manuscript sources had previously been time-consuming, limiting the scope and geographical scale of research.

Today the online I-CeM platform, with its dataset held at the Essex-based UK Data Archive, is transforming research by making available one of the largest historical datasets in the world.

What the I-CeM dataset has made possible

• Populations Past, an online, interactive tool launched by the University of Cambridge in 2018, has revealed stark regional divides in childbirth and child mortality, among other things, in Victorian England and Wales. By combining published records of births, marriages and deaths with the I-CeM dataset, it allows users to select and view maps of a variety of measures including age structure, migration status, marriage, fertility, child mortality and household composition.
• University of Cambridge academic Professor Robert Bennett is using I-CeM to explore patterns of entrepreneurship, evolution of the business population, and how these relate to economic growth. His project aims to create a database of the British business population and entrepreneurs illustrating how entrepreneurship evolved during British development as 'the first industrial nation'.

The I-CeM Project was funded by the Economic and Social Research Council.