An undergraduate classroom experiment illustrates an effect of observer bias on data collection in animal behaviour

Abstract

Behavioural ecologists frequently collect data that involve the potential for subjective judgement, so it is important that researchers in the field develop awareness of potential issues around bias in data collection. We report the results of an undergraduate classroom experiment in which we estimated the potential for students' a priori expectations to bias their estimates of behaviour. Prior to conducting a set of behavioural observations on a video-recorded flock of foraging pigeons, we randomly primed half of the students to expect the pigeons to be hungry, while the other half were primed to expect the pigeons to be satiated. Students were blind to the treatment and subsequently estimated two variables expected to differ in their potential for subjectivity: the proportion of birds in the flock that were feeding (potentially subjective), and the feeding (peck) rates of two focal individuals (potentially objective). Surprisingly, we found no evidence that observer bias affected the estimate of the percentage of birds foraging. By contrast, we found a large effect of observer bias on feeding rate estimates, with students who expected a hungry state recording inflated feeding rate estimates relative to those that expected a satiated state. We furthermore found that students' expectations of foraging state did not always match their allocated primers: bias was predicted by ‘expected state’ but not by ‘allocated state’. Our experiment illustrates that bias associated with expectation can influence results. Furthermore, a variable we initially expected to be relatively objective proved to have a strong subjective element, inflating the effect of confirmation bias on estimation. We recommend blind data recording even when response variables are thought to be objective, as well as explicit teaching of university students about the potential for bias in data collection, and our experiment suggests a potentially useful way in which this can be undertaken.