Context is Everything: Performance Measurement and Societal Impact

Assessing wider ‘societal impact’ has become an important feature of modern research systems. Publicly funded research institutions are increasingly expected to prove their worth through detailed reporting of their outcomes beyond the academic sphere. Several nations have introduced research impact evaluations that rely on narrative impact case studies being constructed and submitted for assessment. Using topic modelling and qualitative coding of a dataset of 7,275 case studies submitted for assessment, this paper compares language patterns around research impact across the UK, Australia and Hong Kong. It explores differences between high and low scoring submissions within each country and between national systems. It argues that while similar forms of performance measurement contribute to constrained patterns of reward and recognition in publicly funded research, there are national differences in the performances of societal impact submitted to the assessments. The paper points to the importance of specific national research contexts despite substantial standardisation in the production and assessment of impact case studies. It indicates that national policy settings override the development of a universal understanding of what is highly valued (and less valued) as societal impact.