Deliberative Discussion or Empty Rhetoric? Assessing Argumentative Quality in Televised Election Debates Over Time

On paper, televised election debates are the ultimate place for politicians to communicate their policy stances to the electorate, and the perfect way for the electorate to gather information about the different contenders. In these debates, politicians ideally give reasons for their standpoints in a substantial and composed manner. However, political discourse in general, and election debates in particular, have recently faced critique for displaying a growing use of one-liners, slogans and hollow phrases instead of proper, extensive justifications (Dryzek et al., 2019). Especially in the heat of election times, politicians want to attract attention, frequently at the cost of content and courtesy. The supposed decline in argumentation quality is problematic. Citizens who condemn this type of discourse might tune-out, which leads to less-informed voters. Those who do watch these debates, do not receive proper input to form well-informed, reflective opinions. Rather, they are nudged to make electoral choices based on heuristics, such as who ‘scores’ the most points or uses the catchiest rhetoric. But is argumentation quality really in decline? Do debates truly look different these days, or are we encouraged by the media to simply pay more attention to norm-violating instances? To date, assessment of argumentation quality is still scarce, and the mapping of decline over time is even scarcer. This study contributes to that by quantitatively analyzing the argumentative quality of Belgian election debates over the past 35 years. Besides changes in quality, we assess if quality decline happens at specific points in time and what contextual factors affect these changes. We investigate and expect notable differences to exist between populist and mainstream parties, male and female politicians, and opposition and coalition partners. The content analysis consists of three main parts. First, we use the justification-centered components of the Discourse Quality Index (Steenbergen et al., 2003), looking both at content and sophistication of justifications used in the debates. We supplement this with an automated assessment of cognitive complexity to study the simplicity of discourse, using the LIWC (Zijlstra et al., 2004). Lastly, in order to scrutinize the substantiality of statements, we draw inspiration from fact-checking literature and evaluate the verifiability of politicians’ claims (Hassan et al., 2017). In sum, this paper allows us to gather insights into the evolution of argumentation quality over the years and shed light on its postulated decline. Dryzek, J. S., Bächtiger, A., Chambers, S., Cohen, J., Druckman, J. N., Felicetti, A., … Warren, M. E. (2019). The crisis of democracy and the science of deliberation. Science, 363(6432), 1144–1146. Hassan, N., Zhang, G., Arslan, F., Caraballo, J., Jimenez, D., Gawsane, S., … Tremayne, M. (2017). ClaimBuster: The First-ever End-to-end Fact-checking System. Proceedings of the VLDB Endowment, 10(12), 1945–1948. Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1, 21–48. Zijlstra, H., Meerveld, T. Van, Middendorp, H. Van, Pennebaker, J. ., & Geenen, R. (2004). De Nederlandse versie van de ’Linguistic Inquiry and Word Count’(LIWC). Gedrag, (December 2015).