Cognitive bias key to understanding REF results

(The REF is an assessment of university research activity which is used to determine funding in coming years for universities across the country in a range of subject areas.)

Although I wanted my Christmas lips to be all about mistletoe and wine they were more occupied answering the question ‘How did you do in the REF?’ How any of us in HE might respond will reflect how we cut the complex ‘cake’ that the REF represents, but what does the science say about all this? Crucially, we are rarely ‘balanced’; we make our decisions using particular slants on the evidence. Given the recency of REF 2014, it is a good time to look at what science shows about these hidden underlying processes, since they also influence how we plan for the future.

The core science here summarises these processes as ‘cognitive biases’. The biases are widespread and detailed and they underpin behavioural economics, to explain why humans can make such duff financial decisions; this work earned Daniel Kahneman his 2002 Nobel Prize. They have also been shown to exert powerful influences in academic psychology, medicine and political science. With more than 100 examples covering a range of domains, there is considerable room for any of us being overly influenced by distinctive sets of biases. That’s why wise decision-makers draw on many perspectives to ‘balance’ a range of overt and covert biases to take stock.

Overall, we can demonstrate an ‘optimistic bias’, which is the preference to see the good in things and to predict positively, regardless of the evidence. The counterparts here are ‘pessimistic bias’ and ‘negativity bias’ where attention inadvertently falls to the downside of any situation. Once outcomes are known – as they are now for REF - winners often demonstrate biases consistent with the outcome, such as ‘superiority bias’, ‘confirmation bias’ or ‘status quo bias’. These types of bias anchor on particular, favourable, features, so the outcomes reflect the winners’ notion of their inherent on-going superiority.

Relying on this explanation can impede development and investment and may weaken personal motivation to do better. It can also be amplified by the ‘egocentric bias’ and/or the ‘false consensus effect’, which allows individuals to over-state their influence in success or to claim that an unfavourable outcome misrepresents the power and significance of their personal contribution.

Under-performance is often explained by referring to circumstances and context.  Regardless of its accuracy, this ‘fundamental attribution error’ is often judged harshly. Importantly, when we make these harsh judgements we can overlook the option of making much-needed system modification.

Sir Alex Ferguson

Equally, where the system is found ‘faulty’, individuals who can do better may not do so. o understand the difference of using the ‘right’ bias according to the outcome, listen to a ‘winner’ explaining their success using the ‘fundamental attribution error’.  Now perhaps you can see why Sir Alex Ferguson was so admired by his players, and – possibly – why you disliked his post-match interviews when Man Utd lost?

Another issue that may be on your lips is to explain the relative success of others by referring to how alike those groups are, compared to your own. This can come into play when attempting to draw the lessons from others’ success Here, ‘outgroup homogeneity bias’ allows us to ‘see’ greater levels of unity among ‘winners’ and unhelpful diffusion among ‘losers’. This bias supports the idea of developing concentrated groups of specialists and is reinforced by repeatedly finding evidence wherever we look (the so called ‘observational selection bias’).

When ‘in-group bias’ is operating, members consolidate around the group, irrespective of external assessments. Finally, there are those groups for whom the REF will have made no difference; they haven’t really bothered looking at the outcomes. Their approach may reflect a ‘just-world bias’; the world is fair and you get what you deserve, which can end by concluding that there’s no need to worry about things.

Irrespective of the issues, it is helpful to keep aware of our own ‘undercurrents’ not least because they are so powerful.  George Kohlrieser recently delivered a provocative TEDx talk using the strapline that just as any of us can be physical hostages, we are just as easily psychological hostages.  As a psychological hostage, instead of us possessing our minds, our minds possess us.