British Election Study 2001/02

2001 British Election Study

Content Analysis Coder Reliability



Return to home page EPOP Conference and Advisory Board Conference Planned timetable Subscribe to the BES e-mail list

Questionnaires Data from the BES component surveys Results from the BES campaign survey Press Content Analysis from the BES 2001/02 Publications arising from the BES 2001/02

BES Advisory board Contact information Link to election and public opinion sites

It is necessary to establish that the coding has been conducted reliably – that different coders, or the same coder over time, have produced consistent results. For the content analysis component of the British Election Study 2001 there were two coders. Coder 1 worked on 1054 articles and Coder 2 worked on 386 articles. In order to conduct coder reliability testing a random sample of approximately 5% of the coded articles was drawn using SPSS.

Intra-coder reliability.
Coder 1 recoded 57 articles.
Coder 2 recoded 19 articles.

Inter-coder reliability.
Coder 1 recoded 36 articles coded by Coder 2.

For the purpose of reliability testing the variables have been separated into three categories: Indicator: general facts about the articles which are fixed and not open to coder interpretation; Manifest: information that is evident within the test but must be extrapolated and could, therefore be open to interpretation; Evaluative: the coder must make a subjective decision based on information in the article as a whole.

Type Variable
   
Indicator Date, Id, Newspaper, Day of Week, Newspaper type, Size of Article
  Page Number, Author
   
Manifest Story Type, Setting, Tone, Theme1, Theme 2, Theme 3, Actor 1, Actor 2
  Actor 3, Actor 4
   
Evaluative Reporter Reporter Evaluation of Actor 1, Reporter Evaluation of Actor 2, 1st Evaluator,
  1st Evaluation, 1st Evaluation, 2nd Evaluator, 2nd Evaluation, 2nd Evaluator,
  Tone Towards Actor 1, Tone Towards Actor 2, Density of Policy Information,
  Density of Personality Information

The overall results by category are set out below.
    % Correct by Type of Category  
     
  Indicator Manifest Evaluative
       
Coder 1/Coder 1 99.6 82.5 88.1
       
       
Coder 2/Coder 2 98.7 83.1 88.6
       
       
Coder 1/Coder 2 99.6 83.9 85.7
       

Frequencies for individual variables can be viewed here:

  • Coder 1/Coder 1 Intra-Coder reliability frequencies

  • Coder 2/Coder 2 Intra-Coder reliability frequencies

  • Coder 1/Coder 2 Inter-Coder reliability frequencies


    Copyright University of Essex 2001. All rights reserved